From patchwork Thu Feb 13 18:01:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973836 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2074.outbound.protection.outlook.com [40.107.223.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ADC5241684; Thu, 13 Feb 2025 18:02:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.74 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469763; cv=fail; b=ai0uMMtSijbLeDbQb55yWskyPcWdDICATToZ+mGuTZ3rXlazhOQoEzDDIaiWSONEZAq7wb/MfimdNzp2P89OVAe51WWxj2hDxdCPLUw4VBG5eKs06S5mX9/BkDqBqJr+GXZq2E4KW44ZTrVTpAOCdH8pcmyrLlI7MEDjRsjtYrI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469763; c=relaxed/simple; bh=BhXKViZjV1QEveffxJ15Y4m0qwb2bPydm+kFF63W6cs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Ch2lcm17p0Xg3usJVrRvcUU18NsU6qC9Lrk1bdX4JFWDUozpF/y9U5i+ll6WJqM3o9kNWZH3S6dfYAPyGGY/esJ88nTRbO6AgJ2o2SWm6MVfCy3bjJACSXsCoobPjMLyKtrFoEilKbwlAzRbsTBmLRDowPwAYUAJdhagQFcHBD0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=dacPDwOj; arc=fail smtp.client-ip=40.107.223.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="dacPDwOj" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FG5eXgpzh2mgQ48IZQbNa/Yn9ajsU51JHeSRVe1oHfRcSNXfEdEoG/WmBgsmCbJvDtTsysg46SkQ5B7CqT0mzuSVXQ2cSjBM3dyUFDGJ5evy9WLyre7K0vFDEed/K0GLo6vGIs1/75LgZWJFEk7DVkWah0sqJvD4CYFSEkC3J08VMUWxOQeFKsvmogMt8okuUCb3vJ1SG0t8yldrTl7o9XKPYoMj0m9dSI6M5SIHFgS9Mk+LgdpPiUTilra/5FwAWxwZudaK8FiiRPMXX2s7FtxNHwi+hiSEYF17TkEviYHG5g7ezs8ar6E4imtAq6R3rcrdfE3Gu5tolJZCxk904w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N86pFw1iIKZgiPehQHP+xkQoVVfDaS8pBROT62bULdU=; b=Od/HKjTL3GfK6I2xKIxj78N567IajWoJCla5nOeAQVoB+NZiUf7CzfU+D7mwGjJ7GZuPKrFfpMNbqIjvtwLhMHuvyVSxoGSHeaVWxdymZP3wN0hKlYYebFHYlWgZycct8Xj1bMHI4fscACk+o8Oxybq2qHHOJ3ZzBFaMH0HJDKvxAIqn1OP9elIjOmeyypnZde6vHFRwjkG56WkdNoWLWzbXjZodr6RZehx75dU9abubs1CrwBrDo7lyoZnk0uvSbLRHX4t2d9Y8Ot01cELI+iKGfqohr0/8ed5MnZGKXIutbQc4jVxnWcOfpqX/izlxQ+ifD1j0RJiRzSeb9wcn0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N86pFw1iIKZgiPehQHP+xkQoVVfDaS8pBROT62bULdU=; b=dacPDwOjMPQ803QrygT4lCOd4JfTiD7Xs6Oviiq1aswLlzPoWBpIXNfIXo2nCSqp0NH+gikup/4/wnHQ9MO6+JpWoTVJvKokviL+VDs1/IJQY1jUkpRkqwEQPVN3hEb9HZ9zuVUK25QfSIUrshPdJ2dtAgf/556BzBkrRRlBKiOD2bENu08dRIk2AxO/Gxkh++7xrjNlMdg9BpvajDi7uA4qmb+0AgB1/rW1K8w0goacNsmcXfhcA2NmGeJQYvOPAoIEabfG0pC3/9cF7LQuYpQkG72IEErUvMwAP9UC1jFlwMD2qPA7fLWCBpuK8oobbpklRIq9+fcwkio2e6Ecuw== Received: from BYAPR01CA0062.prod.exchangelabs.com (2603:10b6:a03:94::39) by PH8PR12MB6699.namprd12.prod.outlook.com (2603:10b6:510:1ce::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.18; Thu, 13 Feb 2025 18:02:38 +0000 Received: from MWH0EPF000A6731.namprd04.prod.outlook.com (2603:10b6:a03:94:cafe::5a) by BYAPR01CA0062.outlook.office365.com (2603:10b6:a03:94::39) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8398.31 via Frontend Transport; Thu, 13 Feb 2025 18:02:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MWH0EPF000A6731.mail.protection.outlook.com (10.167.249.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:02:37 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:12 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:12 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:07 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 01/10] devlink: Remove unused param of devlink_rate_nodes_check Date: Thu, 13 Feb 2025 20:01:25 +0200 Message-ID: <20250213180134.323929-2-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000A6731:EE_|PH8PR12MB6699:EE_ X-MS-Office365-Filtering-Correlation-Id: dd92e055-4519-47d5-4efc-08dd4c58999e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: n9M1C7NpaiSOsaiDJAtXUoGTVCkK9klGUPMtKY325btiwbnb1jettFFRz1ygQM24AfypukThzP/BgDtEO22dlnIzcHbt+ulFnI7euMjvy/cWdT5KA0fqwXtoGzUqb0AU+ZkIOFNR8lsx+gndWPDve1/FZrdSP63TMMDgcoC8WZpclDT/mMn5+a0evaIea6hmWC9N7QOVbf24y+Oy7OtjxuTmBbHG8cczGHD1gkRfGFNLPtehI5tcjUO3X3KMeegZVl3OdJgw/N63ogVyXq8oXUolCIsZZqc5ZLGGhocWslFpWhr/8iFecvm5SUV8i4JxeWwW1oYxMcQmxKg9cFkaHrXodO0l/Nu7pxH3AHtoyjtk1/K7ETKp/5cifx/SwuMmaVsXnZFk+lrnefw9jrowINqc4eq50aMgNufkX4Fu6Kf73cz+8CQh3n6vkOp0QJLANEZ420Xo/fLZ5hAFYKIz+uqGle+zioGb5/qm0QFgCB5X9hA/IxfHjlgS8DG8XGhuafe7V98lo5U/Rw1XqAPJ3W37JmOn5bZb+zMeLRROWNggUkbipTTJ6AcP3mTtGCafPP3CDY/AX2fYSMbCLeQ837QsPjDh7pnQTd+tWO/ZbinYkP7N9WYKoGTTWpRxNvBmVlAG/u/Ine4eJ+GLYCgBN30VWK0tat0YUE4Vk+ri+ISfemCwbg0NoX0s4x1dAsL3seDlgXLlmifGeKTDF3KM5RpvN6cStMdq5dNphplNbTj09kiNNgJdhzG5jqHjSQOw2fF8zfH4h29qOI4KmEeScgPlKtNv7m0iPbiebt5Zr97yrCamh57TNWViYqrserzH8ozLlGiXD63ZwKaf4PyFSKRKdCQPtgKavz3UPlVUZOpHILbOY9YJpo/41W5noR+8NHWT9E3gyrgAVjIsnTSI+avhCVYBqEGuWpHF3f9jxrw/PpZor0PPthlK1SGNUX8JtHAvTjzAC/uFGNG2AooaaZl8xapwm4yvmHYVmlPT7x1BFpWUZ6/elP/0K/upc7yTnaldz5vhm2eHS8Qc0eLs3B+qtx7wl9K/WiV3Af9+dlEp6b3BMjZHs9e1NekWFE5QwSYpn/7OUKaxS4hZMWDeSWdOyVEJdpQUm3YkfJEsBAyhAc2G4xp4BmCXjpZ1tcfpb+bYosAeY2dGnOk43bWojWATPoob5qIk548JkFwQ/zi6Fek9AADYbl3ZMJIiHcI7KGcyuVQoMxa8JRnnfYarv3WxuGtHyp+s1k4cBV4IyGGJXSeBU4sU2IMc6m0wWoK3DmP+O+E7bm6ahpVT7H21nF/zgaZwiQXhUrr7TZr8RP5tR4f7D9U759gcUfn56qebEjn4GZ2qUR6GtemLUfET8N5M2s4aG4sNcyk+pW2wRC0Vh9LF43dxSaSUQI4HgxlcyZpf0pYcHHhrTPLpdmonLdvhKbokE1TN9460QnCgE0kkRcu2N5siLijm3mI56cjdckwixBoWlhSZ95l0sG3kpT3gz5/rxqZ6UrKjaLqZkgg= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:02:37.3037 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dd92e055-4519-47d5-4efc-08dd4c58999e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000A6731.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB6699 From: Cosmin Ratiu The 'mode' param is unused so remove it. Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan Reviewed-by: Kalesh AP --- net/devlink/dev.c | 4 ++-- net/devlink/devl_internal.h | 3 +-- net/devlink/rate.c | 3 +-- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/net/devlink/dev.c b/net/devlink/dev.c index d6e3db300acb..5ff5d9055a8d 100644 --- a/net/devlink/dev.c +++ b/net/devlink/dev.c @@ -713,10 +713,10 @@ int devlink_nl_eswitch_set_doit(struct sk_buff *skb, struct genl_info *info) if (info->attrs[DEVLINK_ATTR_ESWITCH_MODE]) { if (!ops->eswitch_mode_set) return -EOPNOTSUPP; - mode = nla_get_u16(info->attrs[DEVLINK_ATTR_ESWITCH_MODE]); - err = devlink_rate_nodes_check(devlink, mode, info->extack); + err = devlink_rate_nodes_check(devlink, info->extack); if (err) return err; + mode = nla_get_u16(info->attrs[DEVLINK_ATTR_ESWITCH_MODE]); err = ops->eswitch_mode_set(devlink, mode, info->extack); if (err) return err; diff --git a/net/devlink/devl_internal.h b/net/devlink/devl_internal.h index 14eaad9cfe35..9cc7a5f4a19f 100644 --- a/net/devlink/devl_internal.h +++ b/net/devlink/devl_internal.h @@ -297,8 +297,7 @@ int devlink_resources_validate(struct devlink *devlink, struct genl_info *info); /* Rates */ -int devlink_rate_nodes_check(struct devlink *devlink, u16 mode, - struct netlink_ext_ack *extack); +int devlink_rate_nodes_check(struct devlink *devlink, struct netlink_ext_ack *extack); /* Linecards */ unsigned int devlink_linecard_index(struct devlink_linecard *linecard); diff --git a/net/devlink/rate.c b/net/devlink/rate.c index 8828ffaf6cbc..d163c61cdd98 100644 --- a/net/devlink/rate.c +++ b/net/devlink/rate.c @@ -561,8 +561,7 @@ int devlink_nl_rate_del_doit(struct sk_buff *skb, struct genl_info *info) return err; } -int devlink_rate_nodes_check(struct devlink *devlink, u16 mode, - struct netlink_ext_ack *extack) +int devlink_rate_nodes_check(struct devlink *devlink, struct netlink_ext_ack *extack) { struct devlink_rate *devlink_rate; From patchwork Thu Feb 13 18:01:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973838 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2068.outbound.protection.outlook.com [40.107.100.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01F41245AFC; Thu, 13 Feb 2025 18:02:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.68 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469770; cv=fail; b=Y5x//xW8bVIY6lq7YwRoHW25RbiMOpfsXKIPcBkMK8GUVX+/PSWEwf12L+QqllpXWS0Z4v7S9xOIEDzdItCwh7H606Iaybcea47gPerQSVkwfyaAwiuCJzyDiF2qbOAJfS145b8WLpM/v5c+Hd/LOPVOP+8tKoxqIUatVt6ng68= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469770; c=relaxed/simple; bh=ZAEXr7Q02KhvbZHJww5YwcM8wlx7LK8+URSVVKgLi14=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GhCLOtZH2CL0L3VvrGMLRj+s6ViGYd+iQbMFahw5RKveZqSoPY2dd1m5VgH5Fj+bv0NmTkMcvnmMROIQNOm/aqapIYKZFS5VJxFF0PT8Fhlqw6MzAsfQoKS9sMBBog2exubAUCXPA2ueyAkr3gveiAMbcdRZQDHsuYwWaqLGnkk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=SfaDrOq5; arc=fail smtp.client-ip=40.107.100.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="SfaDrOq5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Q7M0+LDsw+A57p19z2suWTCVZj+nOqeAwIU209JPR/UJy3MPHTRf1158DtRK1NmmjDM2Pq+g3mjl5RCcc//eQbJMwyHCIDUXpKaRovnNNsAzzvU71dTto0JziCeBjLpWMMdwUxmsM9lx3skwJUEFW2lh8pUXENLuIUxjjMIoKKjNICaW3T6JHFSQ+wJ7Pni9UHgC94BZ97dMn/GOrQ9T1GmbWPo5huHm/+t5VsnQ7Cqj0eJfzy9bPQ6o7r8XD5eYXcLJQfU8//n80V4IUg857WJYmghljTN4X4fvXxLofx+uPVQMuVO64O9fbnWMFutEG6i8TGa3secuE0z8f7hpyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cEZgWR79MgK/zlaY0b6+/Oeb2ucjqGyLh99aXrGuKks=; b=I/paAw0C3jDjxJMhCfBARgcbGP+zw2bLDqPR5JV3Hgjq+uw1EDYGBZ4vMF6eVgft/pFJZIO3SNgkK2MFnne4gzmd+HchalULK6h5EbrtOVCEJFqiGO5aGYAEUKBJbADjDqVCzawkDRgKp7w0IBtESu79+bpRrFFj5zwREvmAsvcGDb4NueKK3pY68dlIJlFasZz10LSuPTEb0aL4VnjNuHEKJlg0C/M2UtlUUdEtJUoyX48R/NeGXW3lXkxKm+BJUKfWEn1qDGsPDbQJvxFvXjuAo76rGKO97Hlp3T9NCAv/dNE7iAl3lIgvXBwnFLPADjW6NnJmEB/UsFc8IQ2o1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cEZgWR79MgK/zlaY0b6+/Oeb2ucjqGyLh99aXrGuKks=; b=SfaDrOq5KG/nthG5Jl11+80OEjY56wH8C2q1hGuhzjIXqqwipINUfcQGFCI/MLnAByQMXiniapIedDcTDB5Tgi1jG1v6xmkvuMKXE4Nt4+7Ek1MFuDBPaWshACR9+QE1g4poLVTQhqcXQN68VM2lpWAxUlFFgHh7GN1LXA7srZ2VbBL8xhb9zXIy++JeAEw8XTp4iW9TWm87yUbZXvsZP/Cyq2RSLS6MHiCC4HBiNDhEXPEREtYBUl8ET5LLj9bVHkkAfkEkhDuRn49Pl78bejTCKWA8Aop7NjNmvy3DYpMami7yxkSiq4FR+nKNMnMury9SU3uBDc7Ho8zwJZOMBA== Received: from CH0PR03CA0078.namprd03.prod.outlook.com (2603:10b6:610:cc::23) by IA1PR12MB6019.namprd12.prod.outlook.com (2603:10b6:208:3d5::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.16; Thu, 13 Feb 2025 18:02:41 +0000 Received: from CH1PEPF0000AD81.namprd04.prod.outlook.com (2603:10b6:610:cc:cafe::90) by CH0PR03CA0078.outlook.office365.com (2603:10b6:610:cc::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8445.13 via Frontend Transport; Thu, 13 Feb 2025 18:02:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH1PEPF0000AD81.mail.protection.outlook.com (10.167.244.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:02:41 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:17 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:16 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:12 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 02/10] devlink: Store devlink rates in a rate domain Date: Thu, 13 Feb 2025 20:01:26 +0200 Message-ID: <20250213180134.323929-3-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD81:EE_|IA1PR12MB6019:EE_ X-MS-Office365-Filtering-Correlation-Id: 0df51301-e64b-4721-fc7a-08dd4c589c07 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: LgBza8nuZrkCLXuwGkTvxvKHF5KvGcnOe0TRVv9xCYQiJsBp4oT3xs92NOv7YuWfNVxfpIdT2W9Tno3milXGKOB4gugaCJoeTYa4eQNeBrVOt9eev3kP/0JZK5ALcA70IDq0BqAocPpmH8Rv3O3ZbKLCZ6XzPC1e8h9RxLlS/TevMlchX2KB7ebv5z4SeEr/dK14ouO/s20w1Qz88vcOlBUPTzvjGfwrgmBIpmSxt9OnKOsegtckRTeRyJyzHgW8oVkK41ceyEWWty/UFzjFxezCXXqnJy9NsvHNrCW8wBtgKvcek6IdmMhiQDANjbsrm4o4kCsG7YtsVGFguHThFfDTPAPwPToA2yXzpRQSb1eP4hqpjaCOHd4MMfyBSqi/PEFXn0KzkwDIKXOGLggJYnKY5cuTz/JCt7TvU2+JunwJ33eMUnNJR+2/m7FsTnvNOUA0+kAUYBSD7pbL7RdeWAryYgQWR05ZOhSm1P+EnezoAB1bXZkkDlc/1Mj8dXJjIuVc9upjw6IZvLUicCpNe+U/+zAXEwHFxCnqnSbAOYK7YbUnQu1gBTIICcCOV6xpif1GxFbAsHp0FRTviRrIshBKaHjxsnpgPzlgRycibyNZyNnhOg8QTl545E9kiF6GLnMmOoXS+dYXaGBYNj2bicu/6JyZfOhOGVgNYgF/G02QRlVwoyLSgGG/CRxcJXGjiYtCvEF6j+8CPrado1zoH3RQpMAkrE3E7QsRJ/4fglQkGzHUq/CKB6coDdUrQy21ZpBWILaaDOthdGvmD4yBrSARiB5o+8F9Gxxbjnz8uohX7m08qVMOV/A75sDiZQEGaRzpcyp+S6UQTvxoGeBf4Y/fRd3HhUXuyMxL92IQU4pyGhPbFLC2lOwexQMh7fABhvriOpXmigiI6SH7/jjQLzNMOCVWs3B1msnS0YRXFiYu/KLw2dpgjDlBC6E20EvTHSmWYbRjaI95zxsPky8BaJ4i4H3HfSsZ50996/gXowFyZl5alfUI+9341ZCppJu/D/HyjM2s5FqLwFT3AXF5cokMbKN7J6YQQQnQtdCbC9zJNrNCZ4kFK15tuNMrEDT5aWKDT6tVVYbAIJZ/TjGSuLihj0ZZckSjftEeR85GW+Xa8kc19p+sLC9aICh1qk3uRA2cUQVeRq3X7uUHXijstxe9exmeC24hO1SaF4l+dKEylDGr8RaHwSDhi7fjLsByZ3dk5wLmtj85c+XO03OSu9+vmyOyyAoJkRXll599ZJwAs4f86dQDcRxmMqtBY/983KmLfW0pLnaO8+qAVWkrAopQMWUqq0hzyi6SkXnPjvOExsUI6CxwSpp/pGpfeDDDmRCVFW1WAr7eWV/EEqcJ3yvc/OsoCydwRKA39qWSvjbd+8bkZ5HUXDsk9xYMoEnQJnTM/CMvAb5WR26WyKQ8xvV1S4GB3tk1QOr9VgDgKKiW3yioAQRsjr/bhqQ7pHAxOYXP6yv5NGIL5JV7DajF7SS6Ou+QfGOuGYGWz9nsuGg= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:02:41.3518 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0df51301-e64b-4721-fc7a-08dd4c589c07 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD81.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6019 From: Cosmin Ratiu Introduce the concept of a rate domain, a data structure aiming to store devlink rates of one or more devlink objects such that rate nodes within a rate domain can be parents of other rate nodes from the same domain. This commit takes the first steps in that direction by: - introducing a new 'devlink_rate_domain' data structure, allocated on devlink init. - storing devlink rates from the devlink object into the rate domain. - converting all accesses to the new data structure. - adding checks for everything that walks rates in a rate domain so that the correct devlink object is looked at. These checks are now spurious since all rate domains are private to their devlink object, but that is about to change and this is a good moment to add those checks. Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan --- net/devlink/core.c | 11 ++++++++-- net/devlink/dev.c | 2 +- net/devlink/devl_internal.h | 7 ++++++- net/devlink/rate.c | 41 ++++++++++++++++++++++--------------- 4 files changed, 40 insertions(+), 21 deletions(-) diff --git a/net/devlink/core.c b/net/devlink/core.c index f49cd83f1955..06a2e2dce558 100644 --- a/net/devlink/core.c +++ b/net/devlink/core.c @@ -314,6 +314,7 @@ static void devlink_release(struct work_struct *work) mutex_destroy(&devlink->lock); lockdep_unregister_key(&devlink->lock_key); put_device(devlink->dev); + kvfree(devlink->rate_domain); kvfree(devlink); } @@ -424,6 +425,11 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, if (!devlink) return NULL; + devlink->rate_domain = kvzalloc(sizeof(*devlink->rate_domain), GFP_KERNEL); + if (!devlink->rate_domain) + goto err_rate_domain; + INIT_LIST_HEAD(&devlink->rate_domain->rate_list); + ret = xa_alloc_cyclic(&devlinks, &devlink->index, devlink, xa_limit_31b, &last_id, GFP_KERNEL); if (ret < 0) @@ -436,7 +442,6 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, xa_init_flags(&devlink->snapshot_ids, XA_FLAGS_ALLOC); xa_init_flags(&devlink->nested_rels, XA_FLAGS_ALLOC); write_pnet(&devlink->_net, net); - INIT_LIST_HEAD(&devlink->rate_list); INIT_LIST_HEAD(&devlink->linecard_list); INIT_LIST_HEAD(&devlink->sb_list); INIT_LIST_HEAD_RCU(&devlink->dpipe_table_list); @@ -455,6 +460,8 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, return devlink; err_xa_alloc: + kvfree(devlink->rate_domain); +err_rate_domain: kvfree(devlink); return NULL; } @@ -477,7 +484,7 @@ void devlink_free(struct devlink *devlink) WARN_ON(!list_empty(&devlink->resource_list)); WARN_ON(!list_empty(&devlink->dpipe_table_list)); WARN_ON(!list_empty(&devlink->sb_list)); - WARN_ON(!list_empty(&devlink->rate_list)); + WARN_ON(!list_empty(&devlink->rate_domain->rate_list)); WARN_ON(!list_empty(&devlink->linecard_list)); WARN_ON(!xa_empty(&devlink->ports)); diff --git a/net/devlink/dev.c b/net/devlink/dev.c index 5ff5d9055a8d..c926c75cc10d 100644 --- a/net/devlink/dev.c +++ b/net/devlink/dev.c @@ -434,7 +434,7 @@ static void devlink_reload_reinit_sanity_check(struct devlink *devlink) WARN_ON(!list_empty(&devlink->trap_list)); WARN_ON(!list_empty(&devlink->dpipe_table_list)); WARN_ON(!list_empty(&devlink->sb_list)); - WARN_ON(!list_empty(&devlink->rate_list)); + WARN_ON(!list_empty(&devlink->rate_domain->rate_list)); WARN_ON(!list_empty(&devlink->linecard_list)); WARN_ON(!xa_empty(&devlink->ports)); } diff --git a/net/devlink/devl_internal.h b/net/devlink/devl_internal.h index 9cc7a5f4a19f..209b4a4c7070 100644 --- a/net/devlink/devl_internal.h +++ b/net/devlink/devl_internal.h @@ -30,10 +30,14 @@ struct devlink_dev_stats { u32 remote_reload_stats[DEVLINK_RELOAD_STATS_ARRAY_SIZE]; }; +/* Stores devlink rates associated with a rate domain. */ +struct devlink_rate_domain { + struct list_head rate_list; +}; + struct devlink { u32 index; struct xarray ports; - struct list_head rate_list; struct list_head sb_list; struct list_head dpipe_table_list; struct list_head resource_list; @@ -55,6 +59,7 @@ struct devlink { */ struct mutex lock; struct lock_class_key lock_key; + struct devlink_rate_domain *rate_domain; u8 reload_failed:1; refcount_t refcount; struct rcu_work rwork; diff --git a/net/devlink/rate.c b/net/devlink/rate.c index d163c61cdd98..535863bb0c17 100644 --- a/net/devlink/rate.c +++ b/net/devlink/rate.c @@ -36,8 +36,9 @@ devlink_rate_node_get_by_name(struct devlink *devlink, const char *node_name) { static struct devlink_rate *devlink_rate; - list_for_each_entry(devlink_rate, &devlink->rate_list, list) { - if (devlink_rate_is_node(devlink_rate) && + list_for_each_entry(devlink_rate, &devlink->rate_domain->rate_list, list) { + if (devlink_rate->devlink == devlink && + devlink_rate_is_node(devlink_rate) && !strcmp(node_name, devlink_rate->name)) return devlink_rate; } @@ -166,16 +167,18 @@ void devlink_rates_notify_register(struct devlink *devlink) { struct devlink_rate *rate_node; - list_for_each_entry(rate_node, &devlink->rate_list, list) - devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_NEW); + list_for_each_entry(rate_node, &devlink->rate_domain->rate_list, list) + if (rate_node->devlink == devlink) + devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_NEW); } void devlink_rates_notify_unregister(struct devlink *devlink) { struct devlink_rate *rate_node; - list_for_each_entry_reverse(rate_node, &devlink->rate_list, list) - devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_DEL); + list_for_each_entry_reverse(rate_node, &devlink->rate_domain->rate_list, list) + if (rate_node->devlink == devlink) + devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_DEL); } static int @@ -187,7 +190,7 @@ devlink_nl_rate_get_dump_one(struct sk_buff *msg, struct devlink *devlink, int idx = 0; int err = 0; - list_for_each_entry(devlink_rate, &devlink->rate_list, list) { + list_for_each_entry(devlink_rate, &devlink->rate_domain->rate_list, list) { enum devlink_command cmd = DEVLINK_CMD_RATE_NEW; u32 id = NETLINK_CB(cb->skb).portid; @@ -195,6 +198,9 @@ devlink_nl_rate_get_dump_one(struct sk_buff *msg, struct devlink *devlink, idx++; continue; } + if (devlink_rate->devlink != devlink) + continue; + err = devlink_nl_rate_fill(msg, devlink_rate, cmd, id, cb->nlh->nlmsg_seq, flags, NULL); if (err) { @@ -522,7 +528,7 @@ int devlink_nl_rate_new_doit(struct sk_buff *skb, struct genl_info *info) goto err_rate_set; refcount_set(&rate_node->refcnt, 1); - list_add(&rate_node->list, &devlink->rate_list); + list_add(&rate_node->list, &devlink->rate_domain->rate_list); devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_NEW); return 0; @@ -565,8 +571,9 @@ int devlink_rate_nodes_check(struct devlink *devlink, struct netlink_ext_ack *ex { struct devlink_rate *devlink_rate; - list_for_each_entry(devlink_rate, &devlink->rate_list, list) - if (devlink_rate_is_node(devlink_rate)) { + list_for_each_entry(devlink_rate, &devlink->rate_domain->rate_list, list) + if (devlink_rate->devlink == devlink && + devlink_rate_is_node(devlink_rate)) { NL_SET_ERR_MSG(extack, "Rate node(s) exists."); return -EBUSY; } @@ -612,7 +619,7 @@ devl_rate_node_create(struct devlink *devlink, void *priv, char *node_name, } refcount_set(&rate_node->refcnt, 1); - list_add(&rate_node->list, &devlink->rate_list); + list_add(&rate_node->list, &devlink->rate_domain->rate_list); devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_NEW); return rate_node; } @@ -650,7 +657,7 @@ int devl_rate_leaf_create(struct devlink_port *devlink_port, void *priv, devlink_rate->devlink = devlink; devlink_rate->devlink_port = devlink_port; devlink_rate->priv = priv; - list_add_tail(&devlink_rate->list, &devlink->rate_list); + list_add_tail(&devlink_rate->list, &devlink->rate_domain->rate_list); devlink_port->devlink_rate = devlink_rate; devlink_rate_notify(devlink_rate, DEVLINK_CMD_RATE_NEW); @@ -691,13 +698,13 @@ EXPORT_SYMBOL_GPL(devl_rate_leaf_destroy); */ void devl_rate_nodes_destroy(struct devlink *devlink) { - static struct devlink_rate *devlink_rate, *tmp; + struct devlink_rate *devlink_rate, *tmp; const struct devlink_ops *ops = devlink->ops; devl_assert_locked(devlink); - list_for_each_entry(devlink_rate, &devlink->rate_list, list) { - if (!devlink_rate->parent) + list_for_each_entry(devlink_rate, &devlink->rate_domain->rate_list, list) { + if (!devlink_rate->parent || devlink_rate->devlink != devlink) continue; refcount_dec(&devlink_rate->parent->refcnt); @@ -708,8 +715,8 @@ void devl_rate_nodes_destroy(struct devlink *devlink) ops->rate_node_parent_set(devlink_rate, NULL, devlink_rate->priv, NULL, NULL); } - list_for_each_entry_safe(devlink_rate, tmp, &devlink->rate_list, list) { - if (devlink_rate_is_node(devlink_rate)) { + list_for_each_entry_safe(devlink_rate, tmp, &devlink->rate_domain->rate_list, list) { + if (devlink_rate->devlink == devlink && devlink_rate_is_node(devlink_rate)) { ops->rate_node_del(devlink_rate, devlink_rate->priv, NULL); list_del(&devlink_rate->list); kfree(devlink_rate->name); From patchwork Thu Feb 13 18:01:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973837 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2058.outbound.protection.outlook.com [40.107.237.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E5BB242937; Thu, 13 Feb 2025 18:02:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.58 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469765; cv=fail; b=qVfErmx7z1qDL9Vo13JrR9msi2iHEE2SCaTo+h4bc+Rgc6w4APdFu07yB3mIH/3KmRQTJQ/8vfIFerRTwEoKh0rzUJJFlTOBcqu1jnSO3wDFFTiQOtpeGlpRXZw00rclLpTMftvf7xiXgGPD70qMDD67DZtAYTDobMFLIftYetU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469765; c=relaxed/simple; bh=QOVf4+orqAB+OJkeqYOR7se/bD/2GDoKD2fZ0ew9tcc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MpQOj9FFEUfXxge1IqQvR3sKKZNdotveeSkbnoyXpWto35vulrVtQgwos78N8EC9OA7chmIOe4RMu/IwQTDgRv/lEzzqEHGuWF2EytBKUDA2vxpaxm6KTVdR6M4bbaTN3OqGOkzeU3T4h4akFUT08wVzHRPYmTP6gHhkam2Isko= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=QOBbo/pW; arc=fail smtp.client-ip=40.107.237.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="QOBbo/pW" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZLS+3S8zAtgS+ZWiJ6Lzgk1D67yZL01fSuwQ32KGrvBNw+n0uBHmszPuK6Q8KgiIjPuw5qHlGc/wTxXttkrwrX2ybxL58oATz1Rmfw5RlId7RSh/KYfg+qrNIxEBd0Kpb9sJQnJjrX/rMZ9Gwcqn2yUInSWV/28MpZ7CNoP5l2/BKGLpjO5b9Mm+9qsNN1aBhWhANF2rl2RgAhFctMD96FkuGm43APlXNOqHEokSBVpf3bEEbc9vHvPPlr384ANW9DQ+hQJmDPR2ps2jHVmUHFu7xEV1h4UlO3ZIcApki3EIRWFvS2gmtHlOQ8qvu8IkA0NLqlVs2KXaI0nhKmTO+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MSCgfREw3pC9lcEtAaMhTPg7LD7+s4HkRro7cPvahL0=; b=emlE4jPofczBjQZ/HcNmg90EBp7KPywdmomQSeLr7v//i04Ll4gZ8/ZFW3fTHntJSZSzk0Kg/pht8SXWK5azhOaO95IAw49amLToVkcWnoEZqvqrSWGiapw6bawGzOPShUDi2xwBthJH9XUjS11BOToOYMiX39n4HsW+Q5bR/o0aiyYDlr5jnZdtWpldAeqMcL7FO/0a4wWlDazNDbX2RAsNzo2iayAqIF87whiVBXgXGxn6lVNFarWeXoaIlGDx9G8y01k3/rxQs+Sxycc6zCJyo7/mS9RUSXcHkBWl/j8ozT7NRzq3Cl32x2fm+fCo6jb/8YCLEhYvyR9wOX/L8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MSCgfREw3pC9lcEtAaMhTPg7LD7+s4HkRro7cPvahL0=; b=QOBbo/pWNp4IT2sbTPbMJSJnQMTRXCBeJu9aA/8MVFpz3dBtKStz2ur7qhcwpnCBvjs4JbxFQUh+ysvVkSk2CE89S8z1oj4vJusZZ34cclQujBpT8i8iYEVaBummLu87HAoHGGEx5W9SEhd380O4XgPKgP8hbq2zlnEL+jktXKzBG5BHOihYyZkqg0jkuBW1uxAS6arL1eX1KGqEJfJPqjA+qlcJK0c7YzZ9Uqq6+p+sTZJh+Q5pLNpCZXoVQZuPXP8yCVAJDXKucswQRIxHYemPlCKOmdk2CZ9aVm4h/9o5Yoy1vP7fjqGNilxShIQsXwNuclQc2b73KejvmfzzNg== Received: from DS7PR06CA0030.namprd06.prod.outlook.com (2603:10b6:8:54::22) by SJ2PR12MB8738.namprd12.prod.outlook.com (2603:10b6:a03:548::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.13; Thu, 13 Feb 2025 18:02:39 +0000 Received: from DS2PEPF00003443.namprd04.prod.outlook.com (2603:10b6:8:54:cafe::db) by DS7PR06CA0030.outlook.office365.com (2603:10b6:8:54::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8445.16 via Frontend Transport; Thu, 13 Feb 2025 18:02:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS2PEPF00003443.mail.protection.outlook.com (10.167.17.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:02:38 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:22 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:21 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:17 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 03/10] devlink: Serialize access to rate domains Date: Thu, 13 Feb 2025 20:01:27 +0200 Message-ID: <20250213180134.323929-4-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003443:EE_|SJ2PR12MB8738:EE_ X-MS-Office365-Filtering-Correlation-Id: 96fe06d4-d542-4f2f-bd22-08dd4c589a14 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|82310400026|36860700013|376014|1800799024; X-Microsoft-Antispam-Message-Info: 0/d50+X0q4wDWWxNgDxaJaQk5jwMNSfmMGEl9Pqb94Pegw3P7ieMDX7qMxmpBdyaCWE6eZY/mFY3ORlT3ySuZ0C+2s0aS9ujPiwHKZ+AWF/W2zhgD0Llre2sH5kYh3uz2GYCt5FQ/BIkHxGIIKf/EfJp3MEDoZGLm6pqrmSMtOdovkPen2beA5qJ85T+oL5zKfozNTL9STmgUoC+YPf7x18HbiEDVdk4cFBFnNlrtKYeKBGFHlOJ+rOQdqDkPc3eDv9fByQSPdYnO7dz6c1NedbjFXho+zaYdFvLOtFNrfb+bQFKa8w2c2cME8xTHPnPoO4zM5C5nB2PKtWl1V9SS1PIhg62F/Ch14bOJ5ptOiurS6G7pnsvRM4uEeD+kuZNILTmoqZXZSnyl8DM1CJdr5Q1E+uMS1BuB9e4+LJlLQEpuhJj4wIvt3bueO2AXCdv4srWa+hKXDxbTw7piLrRnET0QVqsKF6Dt3UECr/8RW2jsvwykgWHnl3UjYwCdc8VuS/D4yDaUB4T3K9Dax4bffSY0wPjJtp3zBYrs2yfg5Qy6QdR9EMDeevJCSU5skypF3yrr9CjASJIQQ0/IxpdNjOl2qsW0oGmyX1aGRB58lYjgDzZ35e8VVYHJCZaJ3VZNZrjef50HWyn86E1LUzVAHPHVeqpv4wvENlhPPEd/i+2VZnY/c0/ymBdmKTXf/AVM7GZSehCWvR2/c0JruWjg8pES6J3iUx89B9EsyChEdZNdROLvJCEALlRIdKQBdrXZJgDSDKYoHlQ980Mm5HyFq9q1L5gzSs6cIi9jdkkxvGCXZT80m9xZhVSBBZMjFgSUz6NXMDWVvQse91DHlX/Y17Sijebu7a5cvHBmaWwsL9XPEoSb1KeQLjb3NhECXZ59q9X3Hre7yjPvLqeDv09G8z+SKaHnhv9pwM3NBPTnDARXp6IjGJym95pLTwEWk13FC4XvpzxLbgPQI1EFMXxLK7i3v674QIGC3RBvQcJdWCkt7cg4Qwt9o2NwK6jLP9QpKffjLveJvXqdDcuC2yfIBnVuYX0kHYi6FTzhAI2jQfCFbtMTUI+s8vIRIFx36gXLx6/uA4nz1NAdRrORefjHTgc+GAP3r4FOIHVw/wqeYolny1/COTtI5jS1NCjeYjC6x3mRiv+TaPGm2elISJoqTFpqTCxzbezUTfmKTZVvBMxkNRclcrcFD7pHl8YRqEL81flW8aMEQLP0oLv1FCUujqFneg51o1sMrb0c1+6AYoGyQSPmpgI3GVPMxuTcXjxZ4SodvOPK8f4i2f8xpfJLJQQiDFai29RxAN7oIoUTT/nkEaSUBCPf373WWlTZ/8n6UElVH+hnZauCq4/5d3Enx6AmdBpKG3WuFcJ9lmz51uKgJErwvCSrY+KWFj48BVAVw2EOSc5sw9nI+Hq0JOsHL6NZUABk+I6CVy3VFP5k6Fyo/JbksvJiYQrL0SLSipEgYW2ORGn2Y2a9KPP9diS+30HfYSy+Z6gwwjQy0J0hoQ= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(82310400026)(36860700013)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:02:38.1268 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 96fe06d4-d542-4f2f-bd22-08dd4c589a14 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003443.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8738 From: Cosmin Ratiu Access to rates in a rate domain should be serialized. This commit introduces two new functions, devl_rate_domain_lock and devl_rate_domain_unlock, and uses them whenever serial access to a rate domain is needed. For now, they are no-ops since access to a rate domain is protected by the devlink lock. Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan --- net/devlink/devl_internal.h | 4 ++ net/devlink/rate.c | 114 +++++++++++++++++++++++++++--------- 2 files changed, 89 insertions(+), 29 deletions(-) diff --git a/net/devlink/devl_internal.h b/net/devlink/devl_internal.h index 209b4a4c7070..fae81dd6953f 100644 --- a/net/devlink/devl_internal.h +++ b/net/devlink/devl_internal.h @@ -121,6 +121,10 @@ static inline void devl_dev_unlock(struct devlink *devlink, bool dev_lock) device_unlock(devlink->dev); } +static inline void devl_rate_domain_lock(struct devlink *devlink) { } + +static inline void devl_rate_domain_unlock(struct devlink *devlink) { } + typedef void devlink_rel_notify_cb_t(struct devlink *devlink, u32 obj_index); typedef void devlink_rel_cleanup_cb_t(struct devlink *devlink, u32 obj_index, u32 rel_index); diff --git a/net/devlink/rate.c b/net/devlink/rate.c index 535863bb0c17..54e6a9893e3d 100644 --- a/net/devlink/rate.c +++ b/net/devlink/rate.c @@ -167,18 +167,22 @@ void devlink_rates_notify_register(struct devlink *devlink) { struct devlink_rate *rate_node; + devl_rate_domain_lock(devlink); list_for_each_entry(rate_node, &devlink->rate_domain->rate_list, list) if (rate_node->devlink == devlink) devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_NEW); + devl_rate_domain_unlock(devlink); } void devlink_rates_notify_unregister(struct devlink *devlink) { struct devlink_rate *rate_node; + devl_rate_domain_lock(devlink); list_for_each_entry_reverse(rate_node, &devlink->rate_domain->rate_list, list) if (rate_node->devlink == devlink) devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_DEL); + devl_rate_domain_unlock(devlink); } static int @@ -190,6 +194,7 @@ devlink_nl_rate_get_dump_one(struct sk_buff *msg, struct devlink *devlink, int idx = 0; int err = 0; + devl_rate_domain_lock(devlink); list_for_each_entry(devlink_rate, &devlink->rate_domain->rate_list, list) { enum devlink_command cmd = DEVLINK_CMD_RATE_NEW; u32 id = NETLINK_CB(cb->skb).portid; @@ -209,6 +214,7 @@ devlink_nl_rate_get_dump_one(struct sk_buff *msg, struct devlink *devlink, } idx++; } + devl_rate_domain_unlock(devlink); return err; } @@ -225,23 +231,33 @@ int devlink_nl_rate_get_doit(struct sk_buff *skb, struct genl_info *info) struct sk_buff *msg; int err; + devl_rate_domain_lock(devlink); devlink_rate = devlink_rate_get_from_info(devlink, info); - if (IS_ERR(devlink_rate)) - return PTR_ERR(devlink_rate); + if (IS_ERR(devlink_rate)) { + err = PTR_ERR(devlink_rate); + goto unlock; + } msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL); - if (!msg) - return -ENOMEM; + if (!msg) { + err = -ENOMEM; + goto unlock; + } err = devlink_nl_rate_fill(msg, devlink_rate, DEVLINK_CMD_RATE_NEW, info->snd_portid, info->snd_seq, 0, info->extack); - if (err) { - nlmsg_free(msg); - return err; - } + if (err) + goto err_fill; + devl_rate_domain_unlock(devlink); return genlmsg_reply(msg, info); + +err_fill: + nlmsg_free(msg); +unlock: + devl_rate_domain_unlock(devlink); + return err; } static bool @@ -470,18 +486,24 @@ int devlink_nl_rate_set_doit(struct sk_buff *skb, struct genl_info *info) const struct devlink_ops *ops; int err; + devl_rate_domain_lock(devlink); devlink_rate = devlink_rate_get_from_info(devlink, info); - if (IS_ERR(devlink_rate)) - return PTR_ERR(devlink_rate); + if (IS_ERR(devlink_rate)) { + err = PTR_ERR(devlink_rate); + goto unlock; + } ops = devlink->ops; - if (!ops || !devlink_rate_set_ops_supported(ops, info, devlink_rate->type)) - return -EOPNOTSUPP; + if (!ops || !devlink_rate_set_ops_supported(ops, info, devlink_rate->type)) { + err = -EOPNOTSUPP; + goto unlock; + } err = devlink_nl_rate_set(devlink_rate, ops, info); - if (!err) devlink_rate_notify(devlink_rate, DEVLINK_CMD_RATE_NEW); +unlock: + devl_rate_domain_unlock(devlink); return err; } @@ -501,15 +523,21 @@ int devlink_nl_rate_new_doit(struct sk_buff *skb, struct genl_info *info) if (!devlink_rate_set_ops_supported(ops, info, DEVLINK_RATE_TYPE_NODE)) return -EOPNOTSUPP; + devl_rate_domain_lock(devlink); rate_node = devlink_rate_node_get_from_attrs(devlink, info->attrs); - if (!IS_ERR(rate_node)) - return -EEXIST; - else if (rate_node == ERR_PTR(-EINVAL)) - return -EINVAL; + if (!IS_ERR(rate_node)) { + err = -EEXIST; + goto unlock; + } else if (rate_node == ERR_PTR(-EINVAL)) { + err = -EINVAL; + goto unlock; + } rate_node = kzalloc(sizeof(*rate_node), GFP_KERNEL); - if (!rate_node) - return -ENOMEM; + if (!rate_node) { + err = -ENOMEM; + goto unlock; + } rate_node->devlink = devlink; rate_node->type = DEVLINK_RATE_TYPE_NODE; @@ -530,6 +558,7 @@ int devlink_nl_rate_new_doit(struct sk_buff *skb, struct genl_info *info) refcount_set(&rate_node->refcnt, 1); list_add(&rate_node->list, &devlink->rate_domain->rate_list); devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_NEW); + devl_rate_domain_unlock(devlink); return 0; err_rate_set: @@ -538,6 +567,8 @@ int devlink_nl_rate_new_doit(struct sk_buff *skb, struct genl_info *info) kfree(rate_node->name); err_strdup: kfree(rate_node); +unlock: + devl_rate_domain_unlock(devlink); return err; } @@ -547,13 +578,17 @@ int devlink_nl_rate_del_doit(struct sk_buff *skb, struct genl_info *info) struct devlink_rate *rate_node; int err; + devl_rate_domain_lock(devlink); rate_node = devlink_rate_node_get_from_info(devlink, info); - if (IS_ERR(rate_node)) - return PTR_ERR(rate_node); + if (IS_ERR(rate_node)) { + err = PTR_ERR(rate_node); + goto unlock; + } if (refcount_read(&rate_node->refcnt) > 1) { NL_SET_ERR_MSG(info->extack, "Node has children. Cannot delete node."); - return -EBUSY; + err = -EBUSY; + goto unlock; } devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_DEL); @@ -564,20 +599,26 @@ int devlink_nl_rate_del_doit(struct sk_buff *skb, struct genl_info *info) list_del(&rate_node->list); kfree(rate_node->name); kfree(rate_node); +unlock: + devl_rate_domain_unlock(devlink); return err; } int devlink_rate_nodes_check(struct devlink *devlink, struct netlink_ext_ack *extack) { struct devlink_rate *devlink_rate; + int err = 0; + devl_rate_domain_lock(devlink); list_for_each_entry(devlink_rate, &devlink->rate_domain->rate_list, list) if (devlink_rate->devlink == devlink && devlink_rate_is_node(devlink_rate)) { NL_SET_ERR_MSG(extack, "Rate node(s) exists."); - return -EBUSY; + err = -EBUSY; + break; } - return 0; + devl_rate_domain_unlock(devlink); + return err; } /** @@ -595,13 +636,19 @@ devl_rate_node_create(struct devlink *devlink, void *priv, char *node_name, { struct devlink_rate *rate_node; + devl_rate_domain_lock(devlink); + rate_node = devlink_rate_node_get_by_name(devlink, node_name); - if (!IS_ERR(rate_node)) - return ERR_PTR(-EEXIST); + if (!IS_ERR(rate_node)) { + rate_node = ERR_PTR(-EEXIST); + goto unlock; + } rate_node = kzalloc(sizeof(*rate_node), GFP_KERNEL); - if (!rate_node) - return ERR_PTR(-ENOMEM); + if (!rate_node) { + rate_node = ERR_PTR(-ENOMEM); + goto unlock; + } if (parent) { rate_node->parent = parent; @@ -615,12 +662,15 @@ devl_rate_node_create(struct devlink *devlink, void *priv, char *node_name, rate_node->name = kstrdup(node_name, GFP_KERNEL); if (!rate_node->name) { kfree(rate_node); - return ERR_PTR(-ENOMEM); + rate_node = ERR_PTR(-ENOMEM); + goto unlock; } refcount_set(&rate_node->refcnt, 1); list_add(&rate_node->list, &devlink->rate_domain->rate_list); devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_NEW); +unlock: + devl_rate_domain_unlock(devlink); return rate_node; } EXPORT_SYMBOL_GPL(devl_rate_node_create); @@ -648,6 +698,7 @@ int devl_rate_leaf_create(struct devlink_port *devlink_port, void *priv, if (!devlink_rate) return -ENOMEM; + devl_rate_domain_lock(devlink); if (parent) { devlink_rate->parent = parent; refcount_inc(&devlink_rate->parent->refcnt); @@ -660,6 +711,7 @@ int devl_rate_leaf_create(struct devlink_port *devlink_port, void *priv, list_add_tail(&devlink_rate->list, &devlink->rate_domain->rate_list); devlink_port->devlink_rate = devlink_rate; devlink_rate_notify(devlink_rate, DEVLINK_CMD_RATE_NEW); + devl_rate_domain_unlock(devlink); return 0; } @@ -680,11 +732,13 @@ void devl_rate_leaf_destroy(struct devlink_port *devlink_port) if (!devlink_rate) return; + devl_rate_domain_lock(devlink_port->devlink); devlink_rate_notify(devlink_rate, DEVLINK_CMD_RATE_DEL); if (devlink_rate->parent) refcount_dec(&devlink_rate->parent->refcnt); list_del(&devlink_rate->list); devlink_port->devlink_rate = NULL; + devl_rate_domain_unlock(devlink_port->devlink); kfree(devlink_rate); } EXPORT_SYMBOL_GPL(devl_rate_leaf_destroy); @@ -702,6 +756,7 @@ void devl_rate_nodes_destroy(struct devlink *devlink) const struct devlink_ops *ops = devlink->ops; devl_assert_locked(devlink); + devl_rate_domain_lock(devlink); list_for_each_entry(devlink_rate, &devlink->rate_domain->rate_list, list) { if (!devlink_rate->parent || devlink_rate->devlink != devlink) @@ -723,5 +778,6 @@ void devl_rate_nodes_destroy(struct devlink *devlink) kfree(devlink_rate); } } + devl_rate_domain_unlock(devlink); } EXPORT_SYMBOL_GPL(devl_rate_nodes_destroy); From patchwork Thu Feb 13 18:01:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973839 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2064.outbound.protection.outlook.com [40.107.100.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34202245B0A; Thu, 13 Feb 2025 18:02:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469773; cv=fail; b=T50z/GiQnGl5yTQF5k/s+QElu0rjx4ssXMIYYa7ayrdz8OdxeSzvALFrji2zFh5aRF11LF3qF3dBiJxSKFmvgDCj76ShoPaNeTcf8YQZNHHTu5PxtzdYDjFxHY8mlArxchOSVIQ2vypl357JoqFq7PgewtxZPGpKyTR09Lt2dUI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469773; c=relaxed/simple; bh=JDdZxEL2Dk1D+Yd49KjG+qYggkE7GioW8jD4mMIeSkg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pF7nAWTraYYb3gwugDMLPcLIUnGcC1K85ECi6ZsM1DFkjNmW/gWd9ZKy6YPxfo3TxnVmMd2r6mKw2ZPwcPLJBJUt4bwRCewaoUj1fzmWyU3vwGFmE+SYOUBjdIVKK1GqJ7atUps/gYz/rkzrpS9vPTOHNXLFhq2YH9pLJ1O2A7U= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=pUC9+w8a; arc=fail smtp.client-ip=40.107.100.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="pUC9+w8a" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gf2BDT4CmT/k9SATKyABoZmIV/AMaZ7vB3BMlPP7ZSrFlkN7grxDU1K/TBoFpRlzq0DwoBfzCN2pFwhGzDlsMD0LS2vPGI1RRbuDqTDXibBzrd8RMGYY3pOYYq5uWzzYetBt2vBDVE/uGA3cDoagDxls5plEIkikninIJY4cT4gNnD5JvWCB1MEOWoFR8SeZZ4UXHWQtUrgYn9mJTCHZ51l1vNwF2gBionSMRVB0X1JjJbhVHsROBijzD2ruPEkSinGHLmF+8u7PT9enkiuFviEo+MQH/38L3lEDy5H6cH8pgnp9zYPXYaz3F/sijg06Z3poua3dGd/QmziSJqancw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CxIEbbfgk6/pdgzha04CPEYMv2uUqhP5UfIjKKcBYM4=; b=pNabL2rcwVYg/NzqtLKGeyd9N4WBD9lsRtL1oewVeqO2qmf/+xAlYHqAGI9EWJYJfh9O03u7OQRqTa9Zu+kA3SNojUobDJPH84awU4jPfTkj1TiMtL4guybH55Fm79lsmRs666pcTQDeBOS7j6J1byeUMSMF87wT4fRZCZjIhtkSq7/LfSRqNC9GxdV+9ENhferb34SmPK2AtbG860e+2Io+l7HTXuH4iVOWsbHN2qqHEXTOBpzbzgEPdvsJy2rmlxRKxCfOKo3uPlTUFlQdQOf40oUopqQyRxmsbwHmcJJ80w+QvjlvIrG1/pNd0SlbR8C4AeupwCOzdup1D8azgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CxIEbbfgk6/pdgzha04CPEYMv2uUqhP5UfIjKKcBYM4=; b=pUC9+w8aVNLcbzOlZiFcJJEsqiVd86VESJdZd2kZ9LTTyAHX3knPOi2yi59CtRFSS6tk1TLtB307O/T5rph/K8yAnPyLGnX6cqLr4pe2sNYvbiHeq4nG+XZnOK3FEMOLk6ckdPlRjRmgfrYqm19ifzDTp4TXBX46tD9nVSw3Y+/Az9Dd0grjH16wvKGMVVbZErT8Ux+L0uWZ2dg0mABTV4BYilMP0US4MLy2PPm1t+ix3jIiMrVkUdwDC2SJMuQj9KaHrzrT1SrPEipFK3wy1MFs2i+GvGX26CtDC0l+COXUAgqs2pzgkYMFkb/GtsW/Ux2IXnaoMjbhY4nZrPIpeA== Received: from DS7PR03CA0225.namprd03.prod.outlook.com (2603:10b6:5:3ba::20) by DM6PR12MB4268.namprd12.prod.outlook.com (2603:10b6:5:223::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.17; Thu, 13 Feb 2025 18:02:44 +0000 Received: from DS2PEPF00003441.namprd04.prod.outlook.com (2603:10b6:5:3ba:cafe::2d) by DS7PR03CA0225.outlook.office365.com (2603:10b6:5:3ba::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8445.13 via Frontend Transport; Thu, 13 Feb 2025 18:02:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS2PEPF00003441.mail.protection.outlook.com (10.167.17.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:02:43 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:26 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:26 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:22 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 04/10] devlink: Introduce shared rate domains Date: Thu, 13 Feb 2025 20:01:28 +0200 Message-ID: <20250213180134.323929-5-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003441:EE_|DM6PR12MB4268:EE_ X-MS-Office365-Filtering-Correlation-Id: a77181bd-571b-4295-2ec8-08dd4c589d93 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|30052699003|82310400026|1800799024|7416014|376014|36860700013; X-Microsoft-Antispam-Message-Info: y0PZuCPwmN3j6U1ONKAb5ehTBETIdKshNIZ2m6ExMPowbS0O98wGs+pZI0U78vUq/0PpxBIlZ3AAHejtASbTiBbB5AJP8cZHWi7B6unCbIYyZX2W2TmjIgzKeeFLqv+ncucjJf/dtRO7Cb1w20vNJCRqU5ph7+uXjddRv3LHlB1+d4nCTP9Mmt3A5l7hDeZa/V1rjmzNMSIZTxuOWFnSPCShLWcuOL+NLITaM0NjhFzMTFZ0BXrZfekilt7sktBzpV/sIhuX/n9iS/12dNy1LytZ8sNe2nzIEy/0yzL2OusL+bDCcQI4jqH9Oxy7g2goG91KktVGGR4LDhgEwGsMh4qgBgIe7JfxDwr+XWKu9Yt1bS6cwJ7X50XRD2I3OUKydaVx07mdJqlA++Du0RLdC5xOpYCTqCU7/jbXbLoO/r9gd++1WLrsgi9YWxHsu+2fuOmlhrY8U0di14b26DZDU+Yk4uBTTjSjugh86IgM/itKGxQWAOCIO2g08/fLHgUxU2DpAmgSvvgxGEQQuql80aqiZ097GB7400P6tIKR34oUx1ExvJNNM3ySBUQ9kjnAOstbFJBJqT8L5Nk4kBgy3wnE1iqaaduqQ/5mnZ0BA7V9WLOUwvzP0A6bbrp9ux6NINqfyPOAahtQZ3L4ljoUo36OCJO2KvPr+zpCZVavf9IESITa3KU1+h1qTJKmLq9fMLWUQ2aD6Pr9NvcwFOEEDhlOJH8/AiQO0sFGSs0xo1lTkAT0bg8rDbAoveWgwOu5qrDs1aZnXIab5N6yaXb9VqPiPcUcLX97b/hWpDKi0Phd/6/9fFsNMvtAd83VVS6J1/Jy3CzSNjMF2yyefk4f5Xgdc4aZxHT58b8+5YpQkVMV8rmVHlspAwU6vxKbZbZ3iBi9UmyISOAV+MJB9iDtgEG16SbCoAUcIczJosy9QxoMDPuhOd3BFP/OqRYgPdGB6WW28fa8jeYitdc6Ldypymyz/Z/17ujGaLCVbXCCNkIa9lJbyuu5k9e4Rrw1f5U8aObW4KmW2TaUpSPb4bPFE37PaPYWvZKotxnPweEaskQZB6aP9HJC2MawKdEpjiKLHVUPKvWoL/Q3AA0TI9LzGuiM9AyhVijCbe306+goqLIbuI1P777xL+mL7QAHsSsDCWLjiA2KUgt/oyaKb22pwZni0sSW7/eJrOnY+wmvt1ZG4rcx6dYqc3T0+SmwGcikGZcN5jP/+YZ3fCiaUid3RHR/+wZOKHjgKbzITDJVOKP727sRKtkcJH9JMkvFt+Fdi0u5yLWCnN1zBTMdSeG1rnTPj3Teseza2hszAFoX0zYe5B4n8a9DdpPmyDpUi2e84RKqAmaIkQGFocowZioiIEGFpcZMRO4TvdVzCEPBUTJga+NaOucfQzGJjd44O1cmYsUWjOH4mwgYmlhYxtX6q2wgumVudeXlU0CbgIayxuaN6BTS+9vggt1chi/U1dyFbKWJCtdH69Zi27KSGvgbRcpo6Q5akpYTljwVI6BRVsg= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(30052699003)(82310400026)(1800799024)(7416014)(376014)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:02:43.9886 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a77181bd-571b-4295-2ec8-08dd4c589d93 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003441.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4268 From: Cosmin Ratiu The underlying idea is modeling a piece of hardware which: 1. Exposes multiple functions as separate devlink objects. 2. Is capable of instantiating a transmit scheduling tree spanning multiple functions. Modeling this requires devlink rate nodes with parents across other devlink objects. A naive approach that relies on the current one-lock-per-devlink model is impossible, as it would require in some cases acquiring multiple devlink locks in the correct order. Based on the preliminary patches in this series, this commit introduces the concept of a shared rate domain. 1. A shared rate domain stores rate nodes for a piece of hardware that has the properties described at the beginning. 2. A shared rate domain is identified by the devlink operations pointer (a proxy for the device type) and a unique u64 hardware identifier provided by the driver. 3. There is a global registry of reference counted shared rate domains. 4. A devlink object starts out with a private rate domain, and can be switched once to use a shared rate domain with devlink_shared_rate_domain_init. Further calls do nothing. 5. Shared rate domains have an additional mutex serializing access to rate nodes, acquired by the previously introduced functions devl_rate_domain_lock and devl_rate_domain_unlock. These new code paths are unused for now. A caller to devlink_shared_rate_domain_init will be added in a subsequent patch. Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan --- include/net/devlink.h | 8 ++++ net/devlink/core.c | 79 ++++++++++++++++++++++++++++++++++++- net/devlink/dev.c | 2 +- net/devlink/devl_internal.h | 26 ++++++++++-- net/devlink/rate.c | 15 +++++++ 5 files changed, 124 insertions(+), 6 deletions(-) diff --git a/include/net/devlink.h b/include/net/devlink.h index b8783126c1ed..a9675c1810e6 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -1561,6 +1561,14 @@ void devlink_register(struct devlink *devlink); void devlink_unregister(struct devlink *devlink); void devlink_free(struct devlink *devlink); +/* Can be used to tell devlink that shared rate domains are supported. + * The same id needs to be provided for devlink objects that can share + * rate nodes in hw (e.g. contain nodes with parents in other devlink objects). + * This requires holding the devlink lock and can only be called once per object. + * Rate node relationships across different rate domains are not supported. + */ +int devlink_shared_rate_domain_init(struct devlink *devlink, u64 id); + /** * struct devlink_port_ops - Port operations * @port_split: Callback used to split the port into multiple ones. diff --git a/net/devlink/core.c b/net/devlink/core.c index 06a2e2dce558..9d374d84225a 100644 --- a/net/devlink/core.c +++ b/net/devlink/core.c @@ -289,6 +289,80 @@ void devl_unlock(struct devlink *devlink) } EXPORT_SYMBOL_GPL(devl_unlock); +/* A global data struct with all shared rate domains. */ +static struct { + struct mutex lock; /* Acquired after the devlink lock. */ + struct list_head rate_domains; +} devlink_rate_domains = { + .lock = __MUTEX_INITIALIZER(devlink_rate_domains.lock), + .rate_domains = LIST_HEAD_INIT(devlink_rate_domains.rate_domains), +}; + +static bool devlink_rate_domain_eq(struct devlink_rate_domain *rate_domain, + const struct devlink_ops *ops, u64 id) +{ + return rate_domain->ops == ops && rate_domain->id == id; +} + +int devlink_shared_rate_domain_init(struct devlink *devlink, u64 id) +{ + struct devlink_rate_domain *rate_domain; + int err = 0; + + devl_assert_locked(devlink); + + if (devlink->rate_domain->shared) { + if (devlink_rate_domain_eq(devlink->rate_domain, devlink->ops, id)) + return 0; + return -EEXIST; + } + if (!list_empty(&devlink->rate_domain->rate_list)) + return -EINVAL; + + mutex_lock(&devlink_rate_domains.lock); + list_for_each_entry(rate_domain, &devlink_rate_domains.rate_domains, list) { + if (devlink_rate_domain_eq(rate_domain, devlink->ops, id)) { + refcount_inc(&rate_domain->refcount); + goto replace_domain; + } + } + + /* Shared domain not found, create one. */ + rate_domain = kvzalloc(sizeof(*rate_domain), GFP_KERNEL); + if (!rate_domain) { + err = -ENOMEM; + goto unlock; + } + rate_domain->shared = true; + rate_domain->ops = devlink->ops; + rate_domain->id = id; + mutex_init(&rate_domain->lock); + INIT_LIST_HEAD(&rate_domain->rate_list); + refcount_set(&rate_domain->refcount, 1); + list_add_tail(&rate_domain->list, &devlink_rate_domains.rate_domains); +replace_domain: + kvfree(devlink->rate_domain); + devlink->rate_domain = rate_domain; +unlock: + mutex_unlock(&devlink_rate_domains.lock); + return err; +} +EXPORT_SYMBOL_GPL(devlink_shared_rate_domain_init); + +static void devlink_rate_domain_put(struct devlink_rate_domain *rate_domain) +{ + if (rate_domain->shared) { + if (!refcount_dec_and_test(&rate_domain->refcount)) + return; + + WARN_ON(!list_empty(&rate_domain->rate_list)); + mutex_lock(&devlink_rate_domains.lock); + list_del(&rate_domain->list); + mutex_unlock(&devlink_rate_domains.lock); + } + kvfree(rate_domain); +} + /** * devlink_try_get() - try to obtain a reference on a devlink instance * @devlink: instance to reference @@ -314,7 +388,7 @@ static void devlink_release(struct work_struct *work) mutex_destroy(&devlink->lock); lockdep_unregister_key(&devlink->lock_key); put_device(devlink->dev); - kvfree(devlink->rate_domain); + devlink_rate_domain_put(devlink->rate_domain); kvfree(devlink); } @@ -428,6 +502,7 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, devlink->rate_domain = kvzalloc(sizeof(*devlink->rate_domain), GFP_KERNEL); if (!devlink->rate_domain) goto err_rate_domain; + devlink->rate_domain->shared = false; INIT_LIST_HEAD(&devlink->rate_domain->rate_list); ret = xa_alloc_cyclic(&devlinks, &devlink->index, devlink, xa_limit_31b, @@ -484,7 +559,7 @@ void devlink_free(struct devlink *devlink) WARN_ON(!list_empty(&devlink->resource_list)); WARN_ON(!list_empty(&devlink->dpipe_table_list)); WARN_ON(!list_empty(&devlink->sb_list)); - WARN_ON(!list_empty(&devlink->rate_domain->rate_list)); + WARN_ON(devlink_rates_check(devlink)); WARN_ON(!list_empty(&devlink->linecard_list)); WARN_ON(!xa_empty(&devlink->ports)); diff --git a/net/devlink/dev.c b/net/devlink/dev.c index c926c75cc10d..84353a85e8fe 100644 --- a/net/devlink/dev.c +++ b/net/devlink/dev.c @@ -434,7 +434,7 @@ static void devlink_reload_reinit_sanity_check(struct devlink *devlink) WARN_ON(!list_empty(&devlink->trap_list)); WARN_ON(!list_empty(&devlink->dpipe_table_list)); WARN_ON(!list_empty(&devlink->sb_list)); - WARN_ON(!list_empty(&devlink->rate_domain->rate_list)); + WARN_ON(devlink_rates_check(devlink)); WARN_ON(!list_empty(&devlink->linecard_list)); WARN_ON(!xa_empty(&devlink->ports)); } diff --git a/net/devlink/devl_internal.h b/net/devlink/devl_internal.h index fae81dd6953f..7401aab274e5 100644 --- a/net/devlink/devl_internal.h +++ b/net/devlink/devl_internal.h @@ -30,9 +30,20 @@ struct devlink_dev_stats { u32 remote_reload_stats[DEVLINK_RELOAD_STATS_ARRAY_SIZE]; }; -/* Stores devlink rates associated with a rate domain. */ +/* Stores devlink rates associated with a rate domain. + * Multiple devlink objects may share the same domain (when 'shared' is true) + * and rate nodes can have members from multiple devices. + */ struct devlink_rate_domain { + bool shared; + struct list_head list; struct list_head rate_list; + /* Fields below are only used for shared rate domains. */ + const struct devlink_ops *ops; + u64 id; + refcount_t refcount; + /* Serializes access to rates. */ + struct mutex lock; }; struct devlink { @@ -121,9 +132,17 @@ static inline void devl_dev_unlock(struct devlink *devlink, bool dev_lock) device_unlock(devlink->dev); } -static inline void devl_rate_domain_lock(struct devlink *devlink) { } +static inline void devl_rate_domain_lock(struct devlink *devlink) +{ + if (devlink->rate_domain->shared) + mutex_lock(&devlink->rate_domain->lock); +} -static inline void devl_rate_domain_unlock(struct devlink *devlink) { } +static inline void devl_rate_domain_unlock(struct devlink *devlink) +{ + if (devlink->rate_domain->shared) + mutex_unlock(&devlink->rate_domain->lock); +} typedef void devlink_rel_notify_cb_t(struct devlink *devlink, u32 obj_index); typedef void devlink_rel_cleanup_cb_t(struct devlink *devlink, u32 obj_index, @@ -307,6 +326,7 @@ int devlink_resources_validate(struct devlink *devlink, /* Rates */ int devlink_rate_nodes_check(struct devlink *devlink, struct netlink_ext_ack *extack); +int devlink_rates_check(struct devlink *devlink); /* Linecards */ unsigned int devlink_linecard_index(struct devlink_linecard *linecard); diff --git a/net/devlink/rate.c b/net/devlink/rate.c index 54e6a9893e3d..38f18216eb80 100644 --- a/net/devlink/rate.c +++ b/net/devlink/rate.c @@ -621,6 +621,21 @@ int devlink_rate_nodes_check(struct devlink *devlink, struct netlink_ext_ack *ex return err; } +int devlink_rates_check(struct devlink *devlink) +{ + struct devlink_rate *devlink_rate; + int err = 0; + + devl_rate_domain_lock(devlink); + list_for_each_entry(devlink_rate, &devlink->rate_domain->rate_list, list) + if (devlink_rate->devlink == devlink) { + err = -EBUSY; + break; + } + devl_rate_domain_unlock(devlink); + return err; +} + /** * devl_rate_node_create - create devlink rate node * @devlink: devlink instance From patchwork Thu Feb 13 18:01:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973840 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2086.outbound.protection.outlook.com [40.107.223.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 170DF24292A; Thu, 13 Feb 2025 18:03:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.86 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469784; cv=fail; b=GVSNGu4pYsABaINJ10zODFvHwyLycxj4stk8NSfzQ9D3BDTpc1dgTBBlZLBa5mvoVqUK6SXkB11S2ugqaagHKEWkTyHEsvBTofm/LOXnaiIsiDp0L1wvyafqHIdLRxmFjMXgf/1ENyzyk/GQWXUlm6HAqlCIUkB0cSVma1NNyaU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469784; c=relaxed/simple; bh=l+RFeF98hsmYMyhj2o/mtfYeRC4Pf36QPGXocLJCtfk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Zjl+Bh1/SqxoJPWBhVGz4Anw9e2vBoWEsdjB+o+TS0rq+7OdeOLwlL21hlsLNrV6WMXmg08oHv0OlHjCQwMfNrzMN3C8uA95wSQaCjof76vsoULSNWeyA+xeIN65x+zKSBTNT9pccHS5DPVelsNTm051hQ28jyau8LaxK0eQPQo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=uL5O9YoB; arc=fail smtp.client-ip=40.107.223.86 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="uL5O9YoB" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VwCns3oFUmI78VKHkwBHyPMftqtba3luE+kKZ1Ap/VkwMJ0PZeW3wqwSdNhf4LvorD6yFahIrXqP6eZNpnEGentzFzILQQFhg2xhJfepmzc28CrV2+4gZKJkQksxEMc1IsSlF/S2cDWcujSp+pj6xwTZ4vh8jvO82zvOFbxxMrfIZpEVy+Eio+ufuhXaLwpMtXfcfJaT4vLenEI2a9QiOnnc+tecByN7k1Ao5jflDmGeD34sCdWTQ2NNOB3ie5D1vVSUoeIEYokaEPKDxd2ycODRtwt8POwsYec1fyCVptKetesIcKnliLGy6X6j4xgJdIHT18wR0ZGQiIWfiX7leA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LJA9WmaQT0Jbe7DCedFA8gxyY0GJ5FGvLgzw/cKptEE=; b=ut9a1ci/kmCmqHz3UJ4XUJbiTPfre+j2hEkfOrcgctkgus2M+Odt+Z+l6GaLxK42Pv56lzwUqDjreFg3mNdwDsClQAO/hSSLgwUyx2ZQMoL/SwXhdr9ndcnacnn5hMkVTRdncZHv6NlyFcLskKUqREAma60D3UymB+B5MI1vCaemimMLKbakdbK5VpMxDXSZzlvSg3xaJ/4Ahc3Ay7QFHrLOgO71V9pBv+dmrjMeVBRMNCesOfLxPxyi9Tym3Vu8eFypqB23f07ifM0tslDxwjfjComsczbVwHVtnyRCwFx7GgBy9ooo4cThlpIZm5NmrGqlIHiPrPyHyzZNun3H4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LJA9WmaQT0Jbe7DCedFA8gxyY0GJ5FGvLgzw/cKptEE=; b=uL5O9YoBo3Dcnuj8ivUaZldVRssOnAnZqWyR3E8nDv0r7+b6AYg/GaiMSdkApqkdQ//rEE+m0X3D/ee5iQZf9IZJo5J845DIcGOF/dlGBfm+y6tBVXFjk2XLCkwAv5ocX4vO5YMfgRhKGSL7WWy4LTdGEHHx/jI9d5l2XKXTJOSTlmuqBBynY/RfzSWigzPbYwqaA933oKrRHLnRp/jZBqX4ANyBsIHVIUi/abXTIxnHSFhaL1Gy2ChlhuUt1Bnix7DWGXEUOo+GgnWhx17az7dd7x7yp76+fKOlXfT7Pb0sdlcB38hkV7sXt4MCZa135AufiCg7hjVvb47XUYXtMw== Received: from CH0P221CA0011.NAMP221.PROD.OUTLOOK.COM (2603:10b6:610:11c::28) by PH0PR12MB7080.namprd12.prod.outlook.com (2603:10b6:510:21d::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.13; Thu, 13 Feb 2025 18:02:56 +0000 Received: from CH1PEPF0000AD7C.namprd04.prod.outlook.com (2603:10b6:610:11c:cafe::24) by CH0P221CA0011.outlook.office365.com (2603:10b6:610:11c::28) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8445.13 via Frontend Transport; Thu, 13 Feb 2025 18:02:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH1PEPF0000AD7C.mail.protection.outlook.com (10.167.244.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:02:54 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:32 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:31 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:26 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 05/10] devlink: Allow specifying parent device for rate commands Date: Thu, 13 Feb 2025 20:01:29 +0200 Message-ID: <20250213180134.323929-6-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7C:EE_|PH0PR12MB7080:EE_ X-MS-Office365-Filtering-Correlation-Id: f3dee10d-0891-4cf5-e822-08dd4c58a423 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|7416014|376014|82310400026; X-Microsoft-Antispam-Message-Info: 4jU0UaI0o2jH1u3lCC4U9glI9gRpmgf5ol8fBwhsEUCIEtiiHZEFj/dUUhLYb28Lglw27+2uPQ366V6XBp2QbIJAqDftF+ozw0yWl9IlN7DeV/V8iRA6oEg/JqdyPEAqL7NBKwHrNEEIHR/bkL39K2KNceO9ZTnZrzFE3AV3qczXV3WSiFZdhJT08pvEAJKS73xrycnoTbXjvEvORxH1D000D/wUFkfxZYdgx3Iii/Eo/mV97WaZMW9aWhAhXNOujY9oFJRxdwiTLkSfs7KEDocJMVeeyqa0zz3g8tAasDpc98n1+PWuWZmqomhxm3+ysCCOMD5LCbx8XFIx8shRpuoDSk7QIJW77eameM1roTItBP5m4ikjrlQnRSUAxLNcF22H6Uc3C4NN722N95WUOL00p2/tnWbsBlNcYxmw3zSWma3FJKQztpfz6rScWyk5yZrsvl4OEL57xzl6WTEuCIYs4fOQ4lWh/ARtRq1BPfdiXSLNaJ1d52dO7hle9mJti2W0/h9eE3d3tHMXDw++EQF7ojILatD46KV0NCQ9c8k/sCSc59a/YF/TFzjjjymUrA4g9ZcAUzW99191D/D9ORpBEQ+0RrTH/L2wgsy2wOGhOkPm7BeLpQTAgHH9xBzwp86OHNTU4MpmQZdUzn3ep6v7O6HaP3N5hbcxgWPOrP7DOXUljtNqBEEPEDj971/YrUj5qXSJC5irf1bBysE0/2ZDNK686M/jK10a5yENOOAGRipp8Y3J2L6NQAnGNlXL6Zin1x4+qiLqme5iqgRAklb6LaRmP29VnOBwduC/Q3kCotbA277nVMM4Bo2A46kgTzOYANwcR6leSDvdulSfaSfTFEMictwETe30MFtRD0/tX9QxrbPDZlX4PJ8dGPrL2Qfzj6Wjlgsj1OkITc8z/XHXlulw5whL8KZwECyFa+WTyzc37YKsdLdHjbFSHdAT0qLCEPLMuhxqSs9aVsCO4zRBM0NWIpzFJn2osCPMPihrlgaleiEdHNgwe/uJtT9LhVpwWq9VizGm/f+fiW5Ibg2KJ7TiCdFAbkl7ZlX5jsjix6i6ZauJILzYIo+Jf3htFiWD4ZviTs2egi8wCjaiGiG3pWMk+ua/jiqBge/2kpZX0hmGT7FKx+v3pkhisD8qWkuc3lm+i/MBya8FqDNoRZbVMj8SgV08Uxs2xIkhiJmJX0C4rwsASZUlWKNM7c8c/WE+WhNIQYweCKRLPBpDPIfCM88Z//ilvcXCq8/i46FXW6utlvbJ52bjsS1XzGgUoxySPHoQYgYggadSTshHbbNUPWZhbbWZvkqFe4mlMD8g6jy2uFl98AbD2al47drX2ExOUoMkRlXRgzP4SuEb0C3CF+TLSzidRL2Vfdp3la/B1gF2Sd0GNOzA529GzKaHyPHUeiMdMa9dJzNba7avQdsPIuuQsXlvY42TP6LgI9NH8p8qJMKZcZ+W3b2NGbnrTEYhq6XVcW3zlBaLib7x0bXQtpvrRLpNA9tG5xthbQo= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(7416014)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:02:54.9409 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f3dee10d-0891-4cf5-e822-08dd4c58a423 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD7C.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7080 From: Cosmin Ratiu Previously, the parent device was assumed to be the same one as one owning the rate node being manipulated. This patch expands that by allowing rate commands to accept two additional arguments, the parent bus name and the parent dev name. Besides introducing these two attributes in the uapi, this patch: - adds plumbing in the 'rate-new' and 'rate-set' commands to access these attributes. - extends devlink-nl-pre-doit with getting and saving a reference to the parent dev in info->user_ptr[1]. The parent dev needs to exist and be registered and is then unlocked but not put. This ensures that it will stay alive until the end of the request. - changes devlink-nl-post-doit with putting the parent reference, if obtained. Example usage with ynl: Setting parent to a rate node on another device ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/devlink.yaml \ --do rate-set --json '{ \ "bus-name": "pci", \ "dev-name": "0000:08:00.1", \ "port-index": 65537, \ "parent-dev-bus-name": "pci", \ "parent-dev-name": "0000:08:00.0", \ "rate-parent-node-name": "group1" \ }' Getting the rate details: ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/devlink.yaml \ --do rate-get --json '{ \ "bus-name": "pci", \ "dev-name": "0000:08:00.1", \ "port-index": 65537 \ }' Output: {'bus-name': 'pci', 'dev-name': '0000:08:00.1', 'parent-dev-bus-name': 'pci', 'parent-dev-name': '0000:08:00.0', 'port-index': 65537, 'rate-parent-node-name': 'group1', 'rate-tc-bws': 'rate-tx-max': 0, 'rate-tx-priority': 0, 'rate-tx-share': 0, 'rate-tx-weight': 0, 'rate-type': 'leaf'} Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan --- Documentation/netlink/specs/devlink.yaml | 18 ++++-- include/uapi/linux/devlink.h | 3 + net/devlink/netlink.c | 74 +++++++++++++++++++----- net/devlink/netlink_gen.c | 20 ++++--- net/devlink/netlink_gen.h | 7 +++ 5 files changed, 95 insertions(+), 27 deletions(-) diff --git a/Documentation/netlink/specs/devlink.yaml b/Documentation/netlink/specs/devlink.yaml index 09fbb4c03fc8..252db0f080ba 100644 --- a/Documentation/netlink/specs/devlink.yaml +++ b/Documentation/netlink/specs/devlink.yaml @@ -820,6 +820,12 @@ attribute-sets: - name: region-direct type: flag + - + name: parent-dev-bus-name + type: string + - + name: parent-dev-name + type: string - name: dl-dev-stats @@ -2137,8 +2143,8 @@ operations: dont-validate: [ strict ] flags: [ admin-perm ] do: - pre: devlink-nl-pre-doit - post: devlink-nl-post-doit + pre: devlink-nl-pre-doit-parent-dev-optional + post: devlink-nl-post-doit-parent-dev-optional request: attributes: - bus-name @@ -2149,6 +2155,8 @@ operations: - rate-tx-priority - rate-tx-weight - rate-parent-node-name + - parent-dev-bus-name + - parent-dev-name - name: rate-new @@ -2157,8 +2165,8 @@ operations: dont-validate: [ strict ] flags: [ admin-perm ] do: - pre: devlink-nl-pre-doit - post: devlink-nl-post-doit + pre: devlink-nl-pre-doit-parent-dev-optional + post: devlink-nl-post-doit-parent-dev-optional request: attributes: - bus-name @@ -2169,6 +2177,8 @@ operations: - rate-tx-priority - rate-tx-weight - rate-parent-node-name + - parent-dev-bus-name + - parent-dev-name - name: rate-del diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h index 9401aa343673..a7b62e53b7eb 100644 --- a/include/uapi/linux/devlink.h +++ b/include/uapi/linux/devlink.h @@ -614,6 +614,9 @@ enum devlink_attr { DEVLINK_ATTR_REGION_DIRECT, /* flag */ + DEVLINK_ATTR_PARENT_DEV_BUS_NAME, /* string */ + DEVLINK_ATTR_PARENT_DEV_NAME, /* string */ + /* Add new attributes above here, update the spec in * Documentation/netlink/specs/devlink.yaml and re-generate * net/devlink/netlink_gen.c. diff --git a/net/devlink/netlink.c b/net/devlink/netlink.c index 593605c1b1ef..d8e7e23afce2 100644 --- a/net/devlink/netlink.c +++ b/net/devlink/netlink.c @@ -12,6 +12,7 @@ #define DEVLINK_NL_FLAG_NEED_PORT BIT(0) #define DEVLINK_NL_FLAG_NEED_DEVLINK_OR_PORT BIT(1) #define DEVLINK_NL_FLAG_NEED_DEV_LOCK BIT(2) +#define DEVLINK_NL_FLAG_NEED_PARENT_DEV BIT(3) static const struct genl_multicast_group devlink_nl_mcgrps[] = { [DEVLINK_MCGRP_CONFIG] = { .name = DEVLINK_GENL_MCGRP_CONFIG_NAME }, @@ -177,20 +178,15 @@ int devlink_nl_msg_reply_and_new(struct sk_buff **msg, struct genl_info *info) return 0; } -struct devlink * -devlink_get_from_attrs_lock(struct net *net, struct nlattr **attrs, - bool dev_lock) +static struct devlink * +devlink_get_lock(struct net *net, struct nlattr *bus_attr, struct nlattr *dev_attr, bool dev_lock) { + char *busname, *devname; struct devlink *devlink; unsigned long index; - char *busname; - char *devname; - - if (!attrs[DEVLINK_ATTR_BUS_NAME] || !attrs[DEVLINK_ATTR_DEV_NAME]) - return ERR_PTR(-EINVAL); - busname = nla_data(attrs[DEVLINK_ATTR_BUS_NAME]); - devname = nla_data(attrs[DEVLINK_ATTR_DEV_NAME]); + busname = nla_data(bus_attr); + devname = nla_data(dev_attr); devlinks_xa_for_each_registered_get(net, index, devlink) { if (strcmp(devlink->dev->bus->name, busname) == 0 && @@ -206,19 +202,45 @@ devlink_get_from_attrs_lock(struct net *net, struct nlattr **attrs, return ERR_PTR(-ENODEV); } +struct devlink * +devlink_get_from_attrs_lock(struct net *net, struct nlattr **attrs, bool dev_lock) +{ + if (!attrs[DEVLINK_ATTR_BUS_NAME] || !attrs[DEVLINK_ATTR_DEV_NAME]) + return ERR_PTR(-EINVAL); + + return devlink_get_lock(net, attrs[DEVLINK_ATTR_BUS_NAME], + attrs[DEVLINK_ATTR_DEV_NAME], dev_lock); +} + static int __devlink_nl_pre_doit(struct sk_buff *skb, struct genl_info *info, u8 flags) { + bool parent_dev = flags & DEVLINK_NL_FLAG_NEED_PARENT_DEV; bool dev_lock = flags & DEVLINK_NL_FLAG_NEED_DEV_LOCK; + struct devlink *devlink, *parent_devlink = NULL; + struct nlattr **attrs = info->attrs; struct devlink_port *devlink_port; - struct devlink *devlink; int err; - devlink = devlink_get_from_attrs_lock(genl_info_net(info), info->attrs, - dev_lock); - if (IS_ERR(devlink)) - return PTR_ERR(devlink); + if (parent_dev && attrs[DEVLINK_ATTR_PARENT_DEV_BUS_NAME] && + attrs[DEVLINK_ATTR_PARENT_DEV_NAME]) { + parent_devlink = devlink_get_lock(genl_info_net(info), + attrs[DEVLINK_ATTR_PARENT_DEV_BUS_NAME], + attrs[DEVLINK_ATTR_PARENT_DEV_NAME], dev_lock); + if (IS_ERR(parent_devlink)) + return PTR_ERR(parent_devlink); + info->user_ptr[1] = parent_devlink; + /* Drop the parent devlink lock, but don't put it just yet. + * This will keep it alive until the end of the request. + */ + devl_dev_unlock(parent_devlink, dev_lock); + } + devlink = devlink_get_from_attrs_lock(genl_info_net(info), attrs, dev_lock); + if (IS_ERR(devlink)) { + err = PTR_ERR(devlink); + goto parent_put; + } info->user_ptr[0] = devlink; if (flags & DEVLINK_NL_FLAG_NEED_PORT) { devlink_port = devlink_port_get_from_info(devlink, info); @@ -237,6 +259,9 @@ static int __devlink_nl_pre_doit(struct sk_buff *skb, struct genl_info *info, unlock: devl_dev_unlock(devlink, dev_lock); devlink_put(devlink); +parent_put: + if (parent_dev && parent_devlink) + devlink_put(parent_devlink); return err; } @@ -265,6 +290,13 @@ int devlink_nl_pre_doit_port_optional(const struct genl_split_ops *ops, return __devlink_nl_pre_doit(skb, info, DEVLINK_NL_FLAG_NEED_DEVLINK_OR_PORT); } +int devlink_nl_pre_doit_parent_dev_optional(const struct genl_split_ops *ops, + struct sk_buff *skb, + struct genl_info *info) +{ + return __devlink_nl_pre_doit(skb, info, DEVLINK_NL_FLAG_NEED_PARENT_DEV); +} + static void __devlink_nl_post_doit(struct sk_buff *skb, struct genl_info *info, u8 flags) { @@ -274,6 +306,10 @@ static void __devlink_nl_post_doit(struct sk_buff *skb, struct genl_info *info, devlink = info->user_ptr[0]; devl_dev_unlock(devlink, dev_lock); devlink_put(devlink); + if ((flags & DEVLINK_NL_FLAG_NEED_PARENT_DEV) && info->user_ptr[1]) { + devlink = info->user_ptr[1]; + devlink_put(devlink); + } } void devlink_nl_post_doit(const struct genl_split_ops *ops, @@ -289,6 +325,14 @@ devlink_nl_post_doit_dev_lock(const struct genl_split_ops *ops, __devlink_nl_post_doit(skb, info, DEVLINK_NL_FLAG_NEED_DEV_LOCK); } +void +devlink_nl_post_doit_parent_dev_optional(const struct genl_split_ops *ops, + struct sk_buff *skb, + struct genl_info *info) +{ + __devlink_nl_post_doit(skb, info, DEVLINK_NL_FLAG_NEED_PARENT_DEV); +} + static int devlink_nl_inst_single_dumpit(struct sk_buff *msg, struct netlink_callback *cb, int flags, devlink_nl_dump_one_func_t *dump_one, diff --git a/net/devlink/netlink_gen.c b/net/devlink/netlink_gen.c index f9786d51f68f..2dc82cc9a0b1 100644 --- a/net/devlink/netlink_gen.c +++ b/net/devlink/netlink_gen.c @@ -496,7 +496,7 @@ static const struct nla_policy devlink_rate_get_dump_nl_policy[DEVLINK_ATTR_DEV_ }; /* DEVLINK_CMD_RATE_SET - do */ -static const struct nla_policy devlink_rate_set_nl_policy[DEVLINK_ATTR_RATE_TX_WEIGHT + 1] = { +static const struct nla_policy devlink_rate_set_nl_policy[DEVLINK_ATTR_PARENT_DEV_NAME + 1] = { [DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, }, [DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, }, [DEVLINK_ATTR_RATE_NODE_NAME] = { .type = NLA_NUL_STRING, }, @@ -505,10 +505,12 @@ static const struct nla_policy devlink_rate_set_nl_policy[DEVLINK_ATTR_RATE_TX_W [DEVLINK_ATTR_RATE_TX_PRIORITY] = { .type = NLA_U32, }, [DEVLINK_ATTR_RATE_TX_WEIGHT] = { .type = NLA_U32, }, [DEVLINK_ATTR_RATE_PARENT_NODE_NAME] = { .type = NLA_NUL_STRING, }, + [DEVLINK_ATTR_PARENT_DEV_BUS_NAME] = { .type = NLA_NUL_STRING, }, + [DEVLINK_ATTR_PARENT_DEV_NAME] = { .type = NLA_NUL_STRING, }, }; /* DEVLINK_CMD_RATE_NEW - do */ -static const struct nla_policy devlink_rate_new_nl_policy[DEVLINK_ATTR_RATE_TX_WEIGHT + 1] = { +static const struct nla_policy devlink_rate_new_nl_policy[DEVLINK_ATTR_PARENT_DEV_NAME + 1] = { [DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, }, [DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, }, [DEVLINK_ATTR_RATE_NODE_NAME] = { .type = NLA_NUL_STRING, }, @@ -517,6 +519,8 @@ static const struct nla_policy devlink_rate_new_nl_policy[DEVLINK_ATTR_RATE_TX_W [DEVLINK_ATTR_RATE_TX_PRIORITY] = { .type = NLA_U32, }, [DEVLINK_ATTR_RATE_TX_WEIGHT] = { .type = NLA_U32, }, [DEVLINK_ATTR_RATE_PARENT_NODE_NAME] = { .type = NLA_NUL_STRING, }, + [DEVLINK_ATTR_PARENT_DEV_BUS_NAME] = { .type = NLA_NUL_STRING, }, + [DEVLINK_ATTR_PARENT_DEV_NAME] = { .type = NLA_NUL_STRING, }, }; /* DEVLINK_CMD_RATE_DEL - do */ @@ -1160,21 +1164,21 @@ const struct genl_split_ops devlink_nl_ops[74] = { { .cmd = DEVLINK_CMD_RATE_SET, .validate = GENL_DONT_VALIDATE_STRICT, - .pre_doit = devlink_nl_pre_doit, + .pre_doit = devlink_nl_pre_doit_parent_dev_optional, .doit = devlink_nl_rate_set_doit, - .post_doit = devlink_nl_post_doit, + .post_doit = devlink_nl_post_doit_parent_dev_optional, .policy = devlink_rate_set_nl_policy, - .maxattr = DEVLINK_ATTR_RATE_TX_WEIGHT, + .maxattr = DEVLINK_ATTR_PARENT_DEV_NAME, .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO, }, { .cmd = DEVLINK_CMD_RATE_NEW, .validate = GENL_DONT_VALIDATE_STRICT, - .pre_doit = devlink_nl_pre_doit, + .pre_doit = devlink_nl_pre_doit_parent_dev_optional, .doit = devlink_nl_rate_new_doit, - .post_doit = devlink_nl_post_doit, + .post_doit = devlink_nl_post_doit_parent_dev_optional, .policy = devlink_rate_new_nl_policy, - .maxattr = DEVLINK_ATTR_RATE_TX_WEIGHT, + .maxattr = DEVLINK_ATTR_PARENT_DEV_NAME, .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO, }, { diff --git a/net/devlink/netlink_gen.h b/net/devlink/netlink_gen.h index 8f2bd50ddf5e..dd402264a47f 100644 --- a/net/devlink/netlink_gen.h +++ b/net/devlink/netlink_gen.h @@ -27,12 +27,19 @@ int devlink_nl_pre_doit_dev_lock(const struct genl_split_ops *ops, int devlink_nl_pre_doit_port_optional(const struct genl_split_ops *ops, struct sk_buff *skb, struct genl_info *info); +int devlink_nl_pre_doit_parent_dev_optional(const struct genl_split_ops *ops, + struct sk_buff *skb, + struct genl_info *info); void devlink_nl_post_doit(const struct genl_split_ops *ops, struct sk_buff *skb, struct genl_info *info); void devlink_nl_post_doit_dev_lock(const struct genl_split_ops *ops, struct sk_buff *skb, struct genl_info *info); +void +devlink_nl_post_doit_parent_dev_optional(const struct genl_split_ops *ops, + struct sk_buff *skb, + struct genl_info *info); int devlink_nl_get_doit(struct sk_buff *skb, struct genl_info *info); int devlink_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb); From patchwork Thu Feb 13 18:01:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973842 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2079.outbound.protection.outlook.com [40.107.92.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5CDE24A07C; Thu, 13 Feb 2025 18:03:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.79 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469787; cv=fail; b=TB5TwKU2YinWK4+/PCfHNFaBKPUT5Q0hmMrC1iBqg2iMUh5nE8XOhh+62GDf9m7s4JuYwjZheCxS95KUy5ITQffFZTKmT7Qbm0ppd8wBPq1tJ+C/jUaJK0puZDOM+NOzrAv3oAu9VGpzd26aeLioEAF8t5ZL9YyRhjImBu1hBnU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469787; c=relaxed/simple; bh=L4Z1QlNT0mtnEY4RbxNfFe2ED31I2Qao044vvWqDuJg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rQ+x809ZWqwAzpzbGb/gYOlI0a4X2XxDkB4iofaAW5l2DWkqmMgzpGSndvq2hqT7Xd5M0cbppUjJZaZVPRi45S+g9KuodCwiwScj4QXL+oQLnMhYNmMzcTjHu7NMb2O5/LeCPT7o5vMRmAR1+37J9/e8CFOnlBIcWv3dxxSgpVs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=MWPjA7JZ; arc=fail smtp.client-ip=40.107.92.79 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="MWPjA7JZ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=iyHDsL211HC6M33a3mOPhtD92PhZ1b/RGbl4BKLrI8kHRzC7gba7qEv8qOMEvtu7aEr4NkyjxTPs9LbO+GxA4p7x3ZhyJT+PwUeKSVk8L72XWqGt43BQchhaUxOrbO/7HrLN5T6ZiSaZ/DBfVAvWYisWP3VxA3XRt0X0jF5xO7dRJG1lUBNZu8fak95mM0Z34+Vc/SndLhVgQSLhjBX0B+4vUz2/tQVUSZD3hnzP5mlk2ZaybOn3IdNXmSjBQf0dp43CAymfez6Df1plxEj94sH67SyNfkiWCa6wGRQCIAM/N3xNK9WfdUY/CCsJh60dGALB7nr7XVasmQFu58X1Kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XllvOjya/7tVBuqu4C4KLFEiRLbqjpnmsQmoO8VcknI=; b=uTabLbMUkwGQl7DjM5nA3L7J43CENwTKuJl0+0qRbBclTUcqy4Q+1a+IA0h0iL5Jo++AlMJbzqTH5V7UkBAhOTbq/wGmhRnRnmzRZA7xKQoMRVM1iSA6CYtBsMX7IvLyppNv8tWy8lAXnLN1/Q7gn9jAssrLi8dCXYfCHc2YGPkI21fWGOz+tROVYQlJRVK0edU3vwSh+zsKJueIi8wkU7Y2earaGKdDg3xpr08qURZ/sS1XnYhdc4bde2gV6cdmaQ4FoJaHZtlDW3amtyKpsRgh/f4+1/I0JMr7esmKExizhm+micKidGTihkS+npf/VcfuihNVgNq5f8PXDSh6Rg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XllvOjya/7tVBuqu4C4KLFEiRLbqjpnmsQmoO8VcknI=; b=MWPjA7JZv4APDKy3Z1XEe7AaRglLw/sggLnrs1vk6SGy8gfBcS7ymyhdLGIlnI1VZtsuobfd1EC1DmaVwcaoDVRFzXUH2mJXvPFI4i8VnxAd5pdOolXoYS6pZuW/kmA7J4bu3/N3q6k77HjwuoCcerqeLAR9tyOvWn0Z31MN+KtffRD75iXByqyITFiC5fdqEEqpMZ+UwRxISEoXMnKdHUJrS8OBJCH6mMakABa2BVoW2ADwbRURO+KJ6FhiBTMuSyY7rh4+o1PtYsnr6hWZOr+Tl0Ulrz7JLEP/Dp1DgdzDpMVzRH3A90woQrWhFfFmDfyA2F80MO3aJxbicAlO7g== Received: from CH2PR15CA0005.namprd15.prod.outlook.com (2603:10b6:610:51::15) by BY5PR12MB4162.namprd12.prod.outlook.com (2603:10b6:a03:201::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.10; Thu, 13 Feb 2025 18:02:58 +0000 Received: from CH1PEPF0000AD7D.namprd04.prod.outlook.com (2603:10b6:610:51:cafe::61) by CH2PR15CA0005.outlook.office365.com (2603:10b6:610:51::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8445.16 via Frontend Transport; Thu, 13 Feb 2025 18:02:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH1PEPF0000AD7D.mail.protection.outlook.com (10.167.244.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:02:57 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:36 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:36 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:31 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 06/10] devlink: Allow rate node parents from other devlinks Date: Thu, 13 Feb 2025 20:01:30 +0200 Message-ID: <20250213180134.323929-7-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7D:EE_|BY5PR12MB4162:EE_ X-MS-Office365-Filtering-Correlation-Id: 4cf7768e-7691-4657-a796-08dd4c58a5de X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|1800799024|376014|7416014; X-Microsoft-Antispam-Message-Info: SbCReYfU2nrcz8H4HLlHFIO6ar8vEVlgXNTaMBeukChkDf7eE8Wgi9fwihCmrB/6oba49NPo5mLKrNhfq/bLo4kH0Oxbuqt1JXkV8cNvj7JGKD7sFmLmpCoJ6a7YZ3I5CZLn5NcVLl7rnaHlfPLm1Utc5PMg91s9nddDkqKrg0QhJtYYNNLvOnsejSEAB57s+H+JXp4IqpiYfWhj45pKPHKmBvVjFW5K5ck6OY6W7uLC5jjoRgRe2abJyfrkUJkNUvUD/7gad/Sh58aa1nJtxKwaREFfLpIOpExMygKxTPwc0ALwcFvIGWPZjIHJzAo29fge+BvsQ4UwCqf6BHsim6KW3+0z+oM8/3sOni1w3UvvtjIIwEnJWqstr3UBlhzzti/3J2ZoThJFVXKQDmsHmP59+7aRNcetNkbrKGU34Jocr1G8N0vNTGKfykcax6SwaaVtyDQ5+/RWGihpDQW68/yBuNrrEmvbuAy30eIWX/JJbx9T+3X7M/UyrJ4QExz9wKX4mBD9kR6RViefwa793nzM1Hv9/n+hnXs63rLssv/oRU/415UJz4tN0WCKNk4H2BEA1rYglQO/KuRq+pc1HB/th+jJI2vX2Hv9LYpKYuVKVEvQuKgGSLNDml/H9pL6fCBtoj5FhUALI/9zaZ+G+dVYesN+vokW8L6WEFheqsfmmr83RA9qSfKeyJrH5JwDxa8ZzjaeFANvQ+aKoeVnYJMswSaAQNtpE/3V29IPm1JiwrB3jOTOMdJkqh37rc9xCUODBl5OU+fWYeK7UFUZC2exDx80P6PvPGGFjkQyWiuAJhxRM/RkItdHOW2c9VwvmjqDGilp1V2Qs5tFJQz9LvA3WcG3f1jwmdI8WEbtl2Z+L9j14e1T8wsYoiZUVFgLpBV/kzghaxUTAza5x95dypyn0yRVTKlciaQYH9/ILVtebKR88ZFT7YLO+Xr5yd4NKrWwg/NcTdYfXV4tX5ZK6VH4DWaHWTsTlCsQ2yimyiwZQA/RDJPO3zRYSvvPxZxGVUiLsAKc+8Fn1X/Ze2hXQs+ROoc55C6wb4t4jI2yeKqDMvq2NY1F5beYdrwlKgulPM/3V5GHRUDTTGanVjX4fAkqpkt6FSDas5nYI3bUuar89Ar/cX/xmD5gc9lL4MotVOOlXGBmJtm0NgG45JEO/paIddMQrrZnRWrDD6VLadDRwL98v5PWHPwGLlTuV51NVQzF/xKvi/rm5hkaTr+vqaJBPj7UwhwjaCtqQ5YM9Ph+CX1wWAg40FDTUP00+qKtl0iZXSxUXKtMotw5PWTmPOruggcA+jtEErdNnSdVwzdfRrK9RtsHCoUkiiFolSZCnitDZ7XR8hQqnXnEZboILkayrHYebnKCwZVejY/PIrMzNhlyhrdxgCJQRlEUsbid70nGkElRLEPWU1gEXo4QTctZEumIpQ+OrMbHUJXsXrqWBt2uYeT7c0UAxIWUo4VoG7uUn8JuqAdBtRHifPzc8w== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(1800799024)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:02:57.8425 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4cf7768e-7691-4657-a796-08dd4c58a5de X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD7D.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4162 From: Cosmin Ratiu This commit make use of the unlocked parent devlink from info->user_ptr[1] to assign a devlink rate node to the requested parent node. Because it is not locked, none of its mutable fields can be used. But parent setting only requires: 1. Verifying that the same rate domain is used. The rate domain is immutable once set, so this is safe. 2. Comparing devlink_rate->devlink with the requested parent devlink. As the shared devlink rate domain is locked, other entities cannot concurrently make changes to any of its rates. Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan --- net/devlink/rate.c | 46 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 39 insertions(+), 7 deletions(-) diff --git a/net/devlink/rate.c b/net/devlink/rate.c index 38f18216eb80..e6b4f4cb8a01 100644 --- a/net/devlink/rate.c +++ b/net/devlink/rate.c @@ -125,10 +125,26 @@ static int devlink_nl_rate_fill(struct sk_buff *msg, devlink_rate->tx_weight)) goto nla_put_failure; - if (devlink_rate->parent) - if (nla_put_string(msg, DEVLINK_ATTR_RATE_PARENT_NODE_NAME, - devlink_rate->parent->name)) + if (devlink_rate->parent) { + struct devlink_rate *parent = devlink_rate->parent; + + if (nla_put_string(msg, DEVLINK_ATTR_RATE_PARENT_NODE_NAME, parent->name)) goto nla_put_failure; + if (parent->devlink != devlink) { + /* The parent devlink isn't locked, but a reference to + * it is held so it cannot suddenly disappear. + * And since there are rate nodes pointing to it, + * the parent devlink is fully initialized and + * the fields accessed here are valid and immutable. + */ + if (nla_put_string(msg, DEVLINK_ATTR_PARENT_DEV_BUS_NAME, + parent->devlink->dev->bus->name)) + goto nla_put_failure; + if (nla_put_string(msg, DEVLINK_ATTR_PARENT_DEV_NAME, + dev_name(parent->devlink->dev))) + goto nla_put_failure; + } + } genlmsg_end(msg, hdr); return 0; @@ -281,9 +297,18 @@ devlink_nl_rate_parent_node_set(struct devlink_rate *devlink_rate, const char *parent_name = nla_data(nla_parent); const struct devlink_ops *ops = devlink->ops; size_t len = strlen(parent_name); + struct devlink *parent_devlink; struct devlink_rate *parent; int err = -EOPNOTSUPP; + parent_devlink = info->user_ptr[1] ? : devlink; + if (parent_devlink != devlink) { + if (parent_devlink->rate_domain != devlink->rate_domain) { + NL_SET_ERR_MSG(info->extack, + "Cannot set parent to a rate from a different rate domain"); + return -EINVAL; + } + } parent = devlink_rate->parent; if (parent && !len) { @@ -301,7 +326,11 @@ devlink_nl_rate_parent_node_set(struct devlink_rate *devlink_rate, refcount_dec(&parent->refcnt); devlink_rate->parent = NULL; } else if (len) { - parent = devlink_rate_node_get_by_name(devlink, parent_name); + /* parent_devlink (if != devlink) isn't locked, but the rate + * domain, immutable once set, is already locked and the parent + * is only used to determine node owner via pointer comparison. + */ + parent = devlink_rate_node_get_by_name(parent_devlink, parent_name); if (IS_ERR(parent)) return -ENODEV; @@ -762,8 +791,8 @@ EXPORT_SYMBOL_GPL(devl_rate_leaf_destroy); * devl_rate_nodes_destroy - destroy all devlink rate nodes on device * @devlink: devlink instance * - * Unset parent for all rate objects and destroy all rate nodes - * on specified device. + * Unset parent for all rate objects that involve this device and destroy all + * rate nodes on it. */ void devl_rate_nodes_destroy(struct devlink *devlink) { @@ -774,7 +803,9 @@ void devl_rate_nodes_destroy(struct devlink *devlink) devl_rate_domain_lock(devlink); list_for_each_entry(devlink_rate, &devlink->rate_domain->rate_list, list) { - if (!devlink_rate->parent || devlink_rate->devlink != devlink) + if (!devlink_rate->parent || + (devlink_rate->devlink != devlink && + devlink_rate->parent->devlink != devlink)) continue; refcount_dec(&devlink_rate->parent->refcnt); @@ -784,6 +815,7 @@ void devl_rate_nodes_destroy(struct devlink *devlink) else if (devlink_rate_is_node(devlink_rate)) ops->rate_node_parent_set(devlink_rate, NULL, devlink_rate->priv, NULL, NULL); + devlink_rate->parent = NULL; } list_for_each_entry_safe(devlink_rate, tmp, &devlink->rate_domain->rate_list, list) { if (devlink_rate->devlink == devlink && devlink_rate_is_node(devlink_rate)) { From patchwork Thu Feb 13 18:01:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973841 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2042.outbound.protection.outlook.com [40.107.237.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECF3024A07E; Thu, 13 Feb 2025 18:03:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.42 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469787; cv=fail; b=SgFvy05D48DTthWHBGp08aKmwir8XG06BrR4AIrHbMUmf+Tt95m4Kv8NTJZV5SL11iYw/CkW8yJa0F0XAuDupQPmlXAByzyqQ3Jy7+6F2rIAOw5W2RA3H548h9qqNI6gRCPP+omc8IhvcFN34IO1d/EraiumT/1duV2O51mIGHk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469787; c=relaxed/simple; bh=2VlYHJ/HwDRf4sVfg1AAzBqaWZMXo5OGvMOs1kWq5pE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EFsILoTeAwL2NhIG1YohRaIK0YYerekLPnF3dmnWDWLnjoArLCfWHbWMRQyaES0SHUYl2XnDuoR4TkEb5pw1dsscJlVx6yTq8WEZw6Nte2dkFZvb25lxVnCTlUDzOXQM73CEuFCFcZWGdzMlfgOyAjF2Hq3Mci/RLqgcpFS8ptY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=NHJsr0I5; arc=fail smtp.client-ip=40.107.237.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="NHJsr0I5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XhHhRONlBh5VGC99j3SH2xECixfkHjLA+ZrGznDsoGGwd2WQu+g4ua5Vh5Ln+eTVEwn2avK2Lqlt3MJqrDnhFalGtCe0he6W3DenHGIfMsmj8QQAt3CPdn2KnVjIlB+PsS33WPY1gmGF1iwmY6JPeeKZZ44xMZ2wHKaXg9i9Kdaqo9FZVFV3eBS/zZQ82KsVkzkMq+K0+ZRprwqy3oweTr9q+FhDsDuJ0w4nrElCEQzl5gXphqIIZQqC+JP8VVf0vfglZDEd4nX7gGLYh7o44tVNB10t5WAdzRDJmjgKYGOrxxAg2R4gQSvCCChqZkPSh7JnOZM0Sy18pLAuOwQ7Gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aPv3w4MwXGMAvlgMsofVfQEgC4c2caFRPK55N0OTVFo=; b=OcIvKNE4QIH0CpR53v5ryFEVx6JgomwH8HjCDY/qCuoHG1OEBeBRdp3R9gSs+/7uaYN1TvhUCP9XhMCsgRV7Oz/Ho8LHrcFIuRaL3d2//lUuH6+23CM8HKrfKOlssqK5GZ+icKTp3ZHr22P6FeIMv1sGZStU2VvCktoRBHMxZjbaYUBlopcSSMvjOTUWSCjvDx7F9r5B/FI+REVmVJdwq84oZgSYpUxYjSMf1TDNBlXhryN+yO7EHrZDBpgiGB1Sa+JzBj8Vfhwt0S9bMPaUXZIQQg49H1EmRl1RS9rqKURUCYsNogGZYCvHK8oVxwB+7CT7LFcTWI8/cotHVkvQrQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aPv3w4MwXGMAvlgMsofVfQEgC4c2caFRPK55N0OTVFo=; b=NHJsr0I5MLqsBoDZVt0DLjcwfFqdcw0EZJSTHbsi8qUe2vWS7UKP9T8ghuXjezL29eMUzgylnq9gqziNjvgiP/UOHaINe/gtjdijhnLZtZHiQg3LhTlhxYchwsajDSQI5YcU9JhmXt2K/FIfQbtoahcI3R+Y/cAbWOfC1/potXkX0L+lxKuptxgRReHtB2Hh6zSEDMMrs3yRuQEwYIXEEoQiTdd0f4qUJtHnWdCp0+DfytCiFs/R43JSlNYHhh2PJN/ojzAuHzdAk94jETVBYHF85A36qkBT3hNCGUNDMzlL5Y9g5ZvC5XlhH3hPYjjCFWVA5tAhltm1kV0M/J/hnA== Received: from DM5PR08CA0051.namprd08.prod.outlook.com (2603:10b6:4:60::40) by IA1PR12MB6353.namprd12.prod.outlook.com (2603:10b6:208:3e3::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.13; Thu, 13 Feb 2025 18:02:59 +0000 Received: from DS2PEPF00003447.namprd04.prod.outlook.com (2603:10b6:4:60:cafe::f5) by DM5PR08CA0051.outlook.office365.com (2603:10b6:4:60::40) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8398.18 via Frontend Transport; Thu, 13 Feb 2025 18:02:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS2PEPF00003447.mail.protection.outlook.com (10.167.17.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:02:58 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:41 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:40 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:36 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 07/10] net/mlx5: qos: Introduce shared esw qos domains Date: Thu, 13 Feb 2025 20:01:31 +0200 Message-ID: <20250213180134.323929-8-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003447:EE_|IA1PR12MB6353:EE_ X-MS-Office365-Filtering-Correlation-Id: 2a0f9c89-314d-4f0e-7303-08dd4c58a659 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: 5HkIrweEzn25xh9vHKNipcTz6mVjicBgqQ3AKi/Camxn0ACM7zGYx6WQXrU7isMGAzqCFbRg8MaVrtFsKGumuPKO80Xerdp04xFG4r/Os7NJ/rKfqsh908rYQ62Iqw8CPldW1iOSszeCRD8vGu8TPbK1ARqMbxcw/TzsW9v6eJ7jeyysa0G7dmkU4oSa3SXtd0+Cxl3Gpqmyq+W0L69gq8PEJI9eDJrD8WZLL5sGl6xB4EEJ0Sjq6cGwJDrNEspcMk8ZUJeZG+OGUeNA+//dcx1AjAlzPXBpR1FAesyV16O0M9RhwTIz1HWTPbvA2JOy1+zuerhEt9oxjZxq4xhvQtxnjrd1o6qnE0I5KBNJmhOK4Ju/uYTs//1v+lQgx6oFbT57EGfvvjsdvB1tDNbqrGNUL1NTo3nwQ15n6UfVNQiwm2RURvgfD3hQT9xkQgKfUdloTNxAAJj0UF27IZhIsEMScBYNUD5J+mn2NYv0HSqXcIS+Nzi6UJrOoZOVmTGyjeykbqFa/BSsJLvGM+5bNnzVnmwGx4YyGppXFaBBfmEb5qeSDI1XRViKoIt3UyrbrBwYfC71Y0oQAvTNdu22yvgg8BYKaxfiGQ2yawEkG4koL1UXh+lRw+ahleLKYsjocWCS4RFnCdba0GxIdXDnh9BmhVkJU3GoXXS4x9uRcIZ0zcgp1cHeSMvijB59zRGPoGBasCUGoEXMkDBW2ah89yUTDdxlrWhhoAn6/VdaN2Z9cGcY8WMC/2MPH5NySymfT8FYcNQJN+yFOzqJakHebZPkLGE/Xw19MLZQKiD4V4Tksfw9FBf/SPuTAxiw8X0y6fiVEPwP3FyS4NsPS9ydAe02oyb30vKY1aNf5LO14eihciAsI3zrHn3tuzH2JtD6Lbj3+FEFUOR5ZHci7E/GvWvECP0wmq1dNWrF2AnjdwTil/fVHch1io0+I2HNvjp7M9mlwt/MX+/mPHCribdK2vdZdRM2xvaKwp5NG7lyl7HsQrocnj6ykjjFrjDZ2OEPzlcaajNoTitYURaQ0zg/bzC6nZuaY3jj5ELV3yHELdj8RCkWXrS4RymsAlQlXQx/tLmLCV9nHong74BXu/6XSmmf1F34vI8ma9pRWq0cDYAvlxd1ZPhFg1bD76VyCJK67p2e1w+ahmO8yH1DELqPmU02QWl4aC8nroBdkfiJd5H9khmUIGVb4Lwmn5FqBxGrmtW4NCXkJx+sZjJpj73MUttOrKfZR5VIw5WodDcHK5uFEBoseYatX8TM+zo4Ycza+5J5tbtmDin8MvgLMbJWUSTqhlAMrUdhdTRtizEds+cbetU5cuPohoxxzPl6J1Bo3zEB/5RlX6Qs98KhsXg9jYY4blHzCaXJtWRiHZB0hggSqck4QKyyYzFKhjrwemhHaNG1uBwkcYUdbY9HeCRifSTdyD6MmxNlgjAgfui7z4DSWBVk4abFcg1AFEVu6hlAkD8nCEMd8JTON0FvNNL5bnL62U24eyCkMYCcGYdyxJA= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:02:58.6785 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2a0f9c89-314d-4f0e-7303-08dd4c58a659 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003447.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6353 From: Cosmin Ratiu Introduce shared esw qos domains, capable of holding rate groups for multiple E-Switches of the same NIC. Shared qos domains are reference counted and can be discovered via devcom. The devcom comp lock is used in write-mode to prevent init/cleanup races. When initializing a shared qos domain fails due to devcom errors, the code falls back to using a private qos domain while logging a message that cross-esw scheduling cannot be supported. Shared esw qos domains will be used in a future patch to support cross-eswitch scheduling. Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan --- .../net/ethernet/mellanox/mlx5/core/esw/qos.c | 73 +++++++++++++++++-- 1 file changed, 67 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c index 8b7c843446e1..6a469f214187 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c @@ -11,10 +11,17 @@ /* Minimum supported BW share value by the HW is 1 Mbit/sec */ #define MLX5_MIN_BW_SHARE 1 -/* Holds rate nodes associated with an E-Switch. */ +/* Holds rate nodes associated with one or more E-Switches. + * If cross-esw scheduling is supported, this is shared between all + * E-Switches of a NIC. + */ struct mlx5_qos_domain { /* Serializes access to all qos changes in the qos domain. */ struct mutex lock; + /* Whether this domain is shared with other E-Switches. */ + bool shared; + /* The reference count is only used for shared qos domains. */ + refcount_t refcnt; /* List of all mlx5_esw_sched_nodes. */ struct list_head nodes; }; @@ -34,7 +41,7 @@ static void esw_assert_qos_lock_held(struct mlx5_eswitch *esw) lockdep_assert_held(&esw->qos.domain->lock); } -static struct mlx5_qos_domain *esw_qos_domain_alloc(void) +static struct mlx5_qos_domain *esw_qos_domain_alloc(bool shared) { struct mlx5_qos_domain *qos_domain; @@ -44,21 +51,75 @@ static struct mlx5_qos_domain *esw_qos_domain_alloc(void) mutex_init(&qos_domain->lock); INIT_LIST_HEAD(&qos_domain->nodes); + qos_domain->shared = shared; + if (shared) + refcount_set(&qos_domain->refcnt, 1); return qos_domain; } -static int esw_qos_domain_init(struct mlx5_eswitch *esw) +static void esw_qos_devcom_lock(struct mlx5_devcom_comp_dev *devcom, bool shared) { - esw->qos.domain = esw_qos_domain_alloc(); + if (shared) + mlx5_devcom_comp_lock(devcom); +} + +static void esw_qos_devcom_unlock(struct mlx5_devcom_comp_dev *devcom, bool shared) +{ + if (shared) + mlx5_devcom_comp_unlock(devcom); +} + +static int esw_qos_domain_init(struct mlx5_eswitch *esw, bool shared) +{ + struct mlx5_devcom_comp_dev *devcom = esw->dev->priv.hca_devcom_comp; + + if (shared && IS_ERR_OR_NULL(devcom)) { + esw_info(esw->dev, "Cross-esw QoS cannot be initialized because devcom is unavailable."); + shared = false; + } + + esw_qos_devcom_lock(devcom, shared); + if (shared) { + struct mlx5_devcom_comp_dev *pos; + struct mlx5_core_dev *peer_dev; + + mlx5_devcom_for_each_peer_entry(devcom, peer_dev, pos) { + struct mlx5_eswitch *peer_esw = peer_dev->priv.eswitch; + + if (peer_esw->qos.domain && peer_esw->qos.domain->shared) { + esw->qos.domain = peer_esw->qos.domain; + refcount_inc(&esw->qos.domain->refcnt); + goto unlock; + } + } + } + + /* If no shared domain found, this esw will create one. + * Doing it with the devcom comp lock held prevents races with other + * eswitches doing concurrent init. + */ + esw->qos.domain = esw_qos_domain_alloc(shared); +unlock: + esw_qos_devcom_unlock(devcom, shared); return esw->qos.domain ? 0 : -ENOMEM; } static void esw_qos_domain_release(struct mlx5_eswitch *esw) { - kfree(esw->qos.domain); + struct mlx5_devcom_comp_dev *devcom = esw->dev->priv.hca_devcom_comp; + struct mlx5_qos_domain *domain = esw->qos.domain; + bool shared = domain->shared; + + /* Shared domains are released with the devcom comp lock held to + * prevent races with other eswitches doing concurrent init. + */ + esw_qos_devcom_lock(devcom, shared); + if (!shared || refcount_dec_and_test(&domain->refcnt)) + kfree(domain); esw->qos.domain = NULL; + esw_qos_devcom_unlock(devcom, shared); } enum sched_node_type { @@ -829,7 +890,7 @@ int mlx5_esw_qos_init(struct mlx5_eswitch *esw) if (esw->qos.domain) return 0; /* Nothing to change. */ - return esw_qos_domain_init(esw); + return esw_qos_domain_init(esw, false); } void mlx5_esw_qos_cleanup(struct mlx5_eswitch *esw) From patchwork Thu Feb 13 18:01:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973843 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2042.outbound.protection.outlook.com [40.107.243.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6C3024BC13; Thu, 13 Feb 2025 18:03:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.42 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469790; cv=fail; b=AaKa2SHhEWVLYKg/VKeFw3SXZPDbYVM3Mdi1cMA3JBixUAMbtlvut9kgkCtIdMDaVYNKgeJqVAIGAz2Rg0GwcibsiaL3+s+Dx7xOq6MwbghXB3PzKmc/uv2+2jVwE2EXqi1akglDI0Tc/Ygt6O9VnvpCxgU2xV+Qgq/sOzb3SsY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469790; c=relaxed/simple; bh=/RkT7MOaPWRzkHIyutSWxiflTKU0ftTLXHbnW0Lm18c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=us1F+DkHRFQ0AvOpI+sODstNCrnkDvL700uXR3GaSju6H8NwW4sw4SEtJlnqXbVlMzSaETpWBGKxT3cxR5Rn+/1//1H2MPjX98fhP0uhmgTWSjzCsbuzjCBLeMWgMlwhh63Y84TwuP7ibSeLruKmngBzkgJH8wHLSoWUuvlGAik= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=kivBPLua; arc=fail smtp.client-ip=40.107.243.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="kivBPLua" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=EVRlbn9ImM4A1RDA3Lap3RrbvobJffyL/ym/cw8VVeS9aD88q61jTQeRBHoFZhc5iItugA5kVglwCU8hEYAWQxq7rEQ2evHRdjJvAJUUXp2SaALIo6G9mnlN71nSEOwCYifjvfYyd3vPp/yuUh4oCxHghvx7UROlfZI1kA4fnHxN5v5FvhaeCcCzCUfJV6lQt7IfKn8EPfk0Da6xGWFTPBVsOPFeDa6JO6MbomaiCEMRjCI6eDHcIIXv0kVY94fJMC7P/WdJ1ebr8iw1Myv+uIDxZTvA0Ngc/Lg7xVHET/TYAcmdfDbTlA4wUi7awqnDI2pOwvooFmkxioQqhTBxnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=e40PPhyVORJ4EwPSm5oUz+X+G/MUu0k9v9P7J6aSqvE=; b=ZDrXF5WQGIiaJbalnK4w2uQprkb3bzGWfOMnHxqsZ/iVX5SrkaVIv+QRIferA1NnoKm3M8hLXpGYz661ByxEIu3KMbk5NhQ9hgVYNLkePuDz0Viep8JIeba2WMOXaj6Gyj7Ray/mbtrnMDi6hguICVQsyG8tgLBwpyjdOCUYT80nVuMQh4qlmomNf873dzWD6mXqJCkiZvMUS18LrypVjdMQVNtd6rV5yNWBIlEI4AXpygMpnDxEX+BEy1Whp/b3AGEy8vJ2Zlnlwheeitm+l0v3Kjw2Hyw6yfOnyrqolXO9OBsWhQr6pPNKgo9/+r3RTlawaFdQPJqnmXkei7xSfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e40PPhyVORJ4EwPSm5oUz+X+G/MUu0k9v9P7J6aSqvE=; b=kivBPLual0kp6vCW0TfbKUD0+TxfhMWL03OtEZ82AwLT+3ABPZxAqV/KSgRQ++MKr31ZI6hNb3VzahSdFvFlRYlc3BYbPsS3XrT7nZ+FXawUqoQfbk2NIpgsqWxhkyQMn5hxXOdMjSGosYpn4NCV5PN7z3To/vlEvuWZAdJlkj3CHrMci/8VjWqtvSFkZ1N0G7k7eBs+qcf7CSzcSCk0Vo8ZbNirqoSWP1vD7pXOeNaWvidQE0bP3M/K0WTuSXuLocB43EsUWdXWE69Q5TK+4XyNJRKc7i2ZzkcIHKdZTl4CM+F0NlZ5JXB0Dqf3NxsUu5AcP83xGka/cfwcDap3Ug== Received: from CH2PR08CA0016.namprd08.prod.outlook.com (2603:10b6:610:5a::26) by IA0PR12MB8930.namprd12.prod.outlook.com (2603:10b6:208:481::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.11; Thu, 13 Feb 2025 18:03:04 +0000 Received: from CH1PEPF0000AD80.namprd04.prod.outlook.com (2603:10b6:610:5a:cafe::a2) by CH2PR08CA0016.outlook.office365.com (2603:10b6:610:5a::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8445.16 via Frontend Transport; Thu, 13 Feb 2025 18:03:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH1PEPF0000AD80.mail.protection.outlook.com (10.167.244.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:03:04 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:46 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:45 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:41 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 08/10] net/mlx5: qos: Support cross-esw tx scheduling Date: Thu, 13 Feb 2025 20:01:32 +0200 Message-ID: <20250213180134.323929-9-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD80:EE_|IA0PR12MB8930:EE_ X-MS-Office365-Filtering-Correlation-Id: 0d777010-fcda-4593-29f9-08dd4c58a99b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|36860700013|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: YnjT+8X4FOgw5EvdqU5hYctoD7kYdeliGeF6GtwN54TFeXri5/IdjBvy7SeDqIiRnlBNruqMuYAZNB/FdwQ9zcQj8g4ducK97+VAzVLqEWvZe6c9LLGACl6BsURpEl/6UIHa4Ujw1QAOBLlyuG0SfOpCPxCpA7bRHvRTXIcKQlKTbssW8RamKHWKNLOomawHjTti1Do4ztTXGJxhdgSpmhs7BMd1g+KXxn5kA9fNe2TqKSOJuy43EQcSq3G5wCyvhGa+yDOBp00N+df7qucCaUVapoLzZG1xW6GpH7KXEqkUaqHpQ6ZoEZxag6mlTFCf5wCYZkbrM21fViPKNxyGfQEIwAy385n7lFL7HuczfNGtP9p56FE2RBQM0ZpTtIW0l58E8Ui2+dZrA156oT74CPmu3BLVM+mJuA27QcrAOdkKdDlsA/r2TxpKpDYoEX+hbxBnuq7gYQanYCv9D4CSRzW/Vh18+p56ZFsn8TJ37m3S/L5AjhtaJZfKlLju3Ug58r9dYzEeIEzQWnuLb2Sd+WPn+XKGngh+NXWQwU2P079Zcruz1HwPgVPuFftfomKllyk3ndv6V+qHc8fa4K3i6jM8iJv7xTiCirDYBQ+HqML9mwm7D+JUHq2d8TsRqhCFHEFQh0QNinG1297Kwzo5k2t5xwVpPrNMu2piIpnX4l4DNQ+eUe+yHKsCozICyIuvf2gXP7FawFqS3wv+AuKaAiP9SanM8uI9VwVm7hTvIeRkTGGCXSDgt26pkq2LE9KdUSMvorAXjBTrvQ3TAWeVD4Wbtvuxt+mmjVxBrpnxC5S89zSP2hyYLwAVLIdMq+3P8AvrdQyW3J2dokABM3cUWkCd2+V2V7sOMXWHz0wUVY+1ntn9pyw9cQiXpS5PQxZiHZ09Hdygjm9zcd0SFDK63lEI7fI/F41ojupbgIyqsRCPGtjQLqG/ZDwjFP0ibvshKyErdZyTMK+iIazsXDjLzU97FAwoD6B4Hk1OP4/s3f4GRBwDa8PVaUDUDYlQ9vk7pn9Du0XLgaacAk2Q09yyXcc5gFB3ngBnlJ1UsEZ+CKucSvPn7X/k807yxCygiySa2YxxIpoQx2RKz6L8/HpqPvIslDRCtdmwv16T0f8c3zJxSIoIoj6gk/MDpUJ+ZGsELsSP4JC/CPr6XEKSkkB0cx7sr0Ykuuih1dhxQ4BwR5+ALJ/4LQMJami9xqLKW3IWgdiU/2clDMDTj583uBdganS5jsKZElLu5q4IoY4GCjQS8oZy/VYwUFZjm8HdV+ETvgOnUcTJu/o5Yh4ETBqoTNg6ziqn6chvwg/1eSEEuxvkqDu1/UZ2mBKyIBhiefVEwPTFm9QQDUVs8O0apdnp32S5DW5ldR0V2B7DDI0VmYCQ5yUXPQwuvfZz65+Krf+L4HI+ihQ+jVRe56Mlbl5fVVPC4Dj9xbVrXSDUeYIy2PLgOOy2efqTY7Ebx++UlWhYQs+stDkRLLTOXJZ+3Fyv9z0Sspx3TbyQzYn2HSPhb7k= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(36860700013)(82310400026)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:03:04.1267 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0d777010-fcda-4593-29f9-08dd4c58a99b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD80.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8930 From: Cosmin Ratiu Up to now, rate groups could only contain vports from the same E-Switch. This patch relaxes that restriction if the device supports it (HCA_CAP.esw_cross_esw_sched == true) and the right conditions are met: - Link Aggregation (LAG) is enabled. - The E-Switches use the same qos domain. This also enables the use of the previously added shared esw qos domains. Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan --- .../net/ethernet/mellanox/mlx5/core/esw/qos.c | 59 +++++++++++++++---- 1 file changed, 49 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c index 6a469f214187..e6dcfe348a7e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c @@ -147,7 +147,9 @@ struct mlx5_esw_sched_node { enum sched_node_type type; /* The eswitch this node belongs to. */ struct mlx5_eswitch *esw; - /* The children nodes of this node, empty list for leaf nodes. */ + /* The children nodes of this node, empty list for leaf nodes. + * Can be from multiple E-Switches. + */ struct list_head children; /* Valid only if this node is associated with a vport. */ struct mlx5_vport *vport; @@ -398,6 +400,7 @@ static int esw_qos_vport_create_sched_element(struct mlx5_esw_sched_node *vport_ { u32 sched_ctx[MLX5_ST_SZ_DW(scheduling_context)] = {}; struct mlx5_core_dev *dev = vport_node->esw->dev; + struct mlx5_vport *vport = vport_node->vport; void *attr; if (!mlx5_qos_element_type_supported(dev, @@ -408,7 +411,13 @@ static int esw_qos_vport_create_sched_element(struct mlx5_esw_sched_node *vport_ MLX5_SET(scheduling_context, sched_ctx, element_type, SCHEDULING_CONTEXT_ELEMENT_TYPE_VPORT); attr = MLX5_ADDR_OF(scheduling_context, sched_ctx, element_attributes); - MLX5_SET(vport_element, attr, vport_number, vport_node->vport->vport); + MLX5_SET(vport_element, attr, vport_number, vport->vport); + if (vport->dev != dev) { + /* The port is assigned to a node on another eswitch. */ + MLX5_SET(vport_element, attr, eswitch_owner_vhca_id_valid, true); + MLX5_SET(vport_element, attr, eswitch_owner_vhca_id, + MLX5_CAP_GEN(vport->dev, vhca_id)); + } MLX5_SET(scheduling_context, sched_ctx, parent_element_id, vport_node->parent->ix); MLX5_SET(scheduling_context, sched_ctx, max_average_bw, vport_node->max_rate); @@ -887,10 +896,16 @@ static int esw_qos_devlink_rate_to_mbps(struct mlx5_core_dev *mdev, const char * int mlx5_esw_qos_init(struct mlx5_eswitch *esw) { - if (esw->qos.domain) - return 0; /* Nothing to change. */ + bool use_shared_domain = esw->mode == MLX5_ESWITCH_OFFLOADS && + MLX5_CAP_QOS(esw->dev, esw_cross_esw_sched); + + if (esw->qos.domain) { + if (esw->qos.domain->shared == use_shared_domain) + return 0; /* Nothing to change. */ + esw_qos_domain_release(esw); + } - return esw_qos_domain_init(esw, false); + return esw_qos_domain_init(esw, use_shared_domain); } void mlx5_esw_qos_cleanup(struct mlx5_eswitch *esw) @@ -1021,16 +1036,40 @@ int mlx5_esw_devlink_rate_node_del(struct devlink_rate *rate_node, void *priv, return 0; } -int mlx5_esw_qos_vport_update_parent(struct mlx5_vport *vport, struct mlx5_esw_sched_node *parent, - struct netlink_ext_ack *extack) +static int mlx5_esw_validate_cross_esw_scheduling(struct mlx5_eswitch *esw, + struct mlx5_esw_sched_node *parent, + struct netlink_ext_ack *extack) { - struct mlx5_eswitch *esw = vport->dev->priv.eswitch; - int err = 0; + if (!parent || esw == parent->esw) + return 0; - if (parent && parent->esw != esw) { + if (!MLX5_CAP_QOS(esw->dev, esw_cross_esw_sched)) { NL_SET_ERR_MSG_MOD(extack, "Cross E-Switch scheduling is not supported"); return -EOPNOTSUPP; } + if (esw->qos.domain != parent->esw->qos.domain) { + NL_SET_ERR_MSG_MOD(extack, + "Cannot add vport to a parent belonging to a different qos domain"); + return -EOPNOTSUPP; + } + if (!mlx5_lag_is_active(esw->dev)) { + NL_SET_ERR_MSG_MOD(extack, + "Cross E-Switch scheduling requires LAG to be activated"); + return -EOPNOTSUPP; + } + + return 0; +} + +int mlx5_esw_qos_vport_update_parent(struct mlx5_vport *vport, struct mlx5_esw_sched_node *parent, + struct netlink_ext_ack *extack) +{ + struct mlx5_eswitch *esw = vport->dev->priv.eswitch; + int err; + + err = mlx5_esw_validate_cross_esw_scheduling(esw, parent, extack); + if (err) + return err; esw_qos_lock(esw); if (!vport->qos.sched_node && parent) From patchwork Thu Feb 13 18:01:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973844 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2082.outbound.protection.outlook.com [40.107.243.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0493266B51; Thu, 13 Feb 2025 18:03:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.82 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469795; cv=fail; b=SBn4KRN1vdbw/BUG1ip4Yhnp9RmuAiiEUPpPE04TBdQl0tke+Zfptv8Jbp09dcafuUeikFglhNnvI6jGFkuGE8/2E1BSI7r0JY/fw4dU7Cas1y7EEUUYywaITzho6Cy4DZbFjMlzGGPUMYFfp33/IJATzdYYgoU3g8uFpws+IwA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469795; c=relaxed/simple; bh=C83+eZf5KUuNCQScrqnktNXJKq6/WbImoXb5Nr+F0Ec=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=g3ICFCHNkS/8ur2YmmNoqsPDQh70sujjuwP3Hk6BVaa5EMDwU0N/g0xvsPqWFe0YU/xLlHICtG2Z6QOAbhZI+qgw7YYC+OB2+yUuDwA8rZl/aKx8e8hxosEYfnng2qFlnOv5jZ5sq2p7jdIP7feycjzpgh2CS13MLtl8sm0mx1Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=BTr7XhWu; arc=fail smtp.client-ip=40.107.243.82 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="BTr7XhWu" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uEVWh0iYCcvYmn6edSIA4h7c2fmERoG1PADNBq6vNcR+8F+p66iCCWPoAOU6KJI1sl7P2PGkugqFmxUHJz7PlPTMyuqGGWalevSxL26Vc3nI3raKh339nRnd9rOFL9g9qJazLzrqxWbHpFhg4lSyHxavD/PoIMOoiVomo8z7SF0GySRw9Hdz/33beDa+nD3JdoiAyNtKCdYfNnb5Qx5i06mJHuxTrj0KFTXfJ1yY0snPT4xPQcXKemDMY2SvU8Gq/ie4aYDnA53CaS0DQyko0zLRGKchk4VJF/HMIb/+18ZH8tg48yHjE90WZaKuAqQtCt7SdbBBGOO1FAUSlT7vSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+eBV3PeVhl++UF3uYeqBLMNKZ6EeqSrm14B797LVSGU=; b=N6pfdYbYJPsRuBKBqdPE+ZncD/ICDJ/RD/oPM1kPiQPy0L4RINui0mh22ZpS8R2Hm25DtmyVk+c1XXlqyciF3tV4P9n/AwOeY/1LLARVIe7+vfSs8hZDmP+5sJxqYRPO7QdDvZDgfPlA9O7wmtqatRgWdCmFzK5QKlJRjYXnF4iO1f5/bI3N0yMKkYbOXsUGCZXUeKleU5agYaUMdrTfnR+S6/rp+zteSIu1eKBdd1S1XZ4VBzrv0jKA7Q09z/CiTp2Rk7jsQyA6xdVvXjUhX+2PpKtOcYueh5gvAszE3lbfHSWYr74HMZKNsVQ1gXC4ijVhQ/s05+/gRuVM7O/WaA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+eBV3PeVhl++UF3uYeqBLMNKZ6EeqSrm14B797LVSGU=; b=BTr7XhWuebpELzGbMxJigvqVAZDy0znN2EgZT/YvBpE+cApPB5+IXSAJmVKan/jIbZ3Is6Mbm8sVAPvfRiChvBuUgitlX28vr15L4pKHUNykRsWZ4yrEi5JfmJiEwpC1U4cTkbclopEcX3q+J4ht3VsrsGWZjybMskChd3U44SDu/ut22d1/4Hle1sBN+iB9HosUF8TOv9S02jvg5qQ+CfW7lfnwUZ24mwivh527aypdQRMRF5WBVzYPT3VD4mudGr0J8W2Hdypks2tazs/MD56+yqBkiotnY4+kJhxDv9J9/aCCAIYLi4JvY94Z/dtFpg19t/MEnyaPZUftjGhhWA== Received: from CH5P223CA0002.NAMP223.PROD.OUTLOOK.COM (2603:10b6:610:1f3::10) by BL1PR12MB5707.namprd12.prod.outlook.com (2603:10b6:208:386::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.13; Thu, 13 Feb 2025 18:03:06 +0000 Received: from DS2PEPF00003447.namprd04.prod.outlook.com (2603:10b6:610:1f3:cafe::87) by CH5P223CA0002.outlook.office365.com (2603:10b6:610:1f3::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8445.15 via Frontend Transport; Thu, 13 Feb 2025 18:03:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS2PEPF00003447.mail.protection.outlook.com (10.167.17.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:03:05 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:51 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:50 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:46 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 09/10] net/mlx5: qos: Init shared devlink rate domain Date: Thu, 13 Feb 2025 20:01:33 +0200 Message-ID: <20250213180134.323929-10-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003447:EE_|BL1PR12MB5707:EE_ X-MS-Office365-Filtering-Correlation-Id: e3a3ca5c-f924-4cb7-18a9-08dd4c58aaae X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|376014|7416014|82310400026; X-Microsoft-Antispam-Message-Info: RV+DvtWb/txInid3rOzHiR8FhgDJzhS/xqk7ggnJOI1eD1dw/geQ62c1Q3/qrg8rFWwuIcrdJGHLV0xAuqYi1aCx46IBo9Nb+gqUUno/36MfA5ANH4vnnOTf/QKD0xQjp60mtu3pITKbmJjSWIYG4xBgkfUYFU/xRnN9KpJdWSL6GGoe18YrR9L5BrQjXoB8kMYaXX4WEdisNqHITp10mQSOdupsHVV+/O5TFITigP0BIko4vzALfeYF6RqNtZAHmFhIGbOPyfU49/o6o5oDezTvLXkUGRb6S+56128400e5FJxtB/iuJ9FqsheUPq+naSLt32srAT4ZPD3gQBGqacmYpOMWges/RWpCbwsOVmvY0WdVhuoMIDiIG2bfwGqOtINwzFhmEj53u/ZYKuWpZ4KBOT85+/HnybcgamHiQ6BVkIWZ5PnYTf9v4JwXLln5V6ogPNfr7FYiHYJ+m6H05EgvGlT8LiyLwL8LJWaVGbYmRUSR5/HQWCkDFLGE0UlwRpAPfwRYjbGrAv/8kIGYai0BhMyk5yhzy8NFxIksxALwhsVq/RmAFYX9upoDDdRcjVd5nweyZmnmX2G37sDn8mmiaQNrx1PHcIB6nqK7UirB0n4XfTZZdcddQmREQktePmLlKBvsahmSKZl9Bc0f7oA6qA+MpbkObIbVTqGc7tchmxOnyf1W6Rh2he0sQZQpw0sgzYBYMEV3tblBrSxSdEgIzI2P56xLXal7O04ZxTRXQO4c3ueLP/82sVHJULR9C63nR8hKK51ARcL2F7RG5Hhft5SZPeT7Ag9jEsQaSIboRTgwYy4EbFewy1ENXOTEueJ6ApqzDqx5XTgc7lb6aWssdmZMPhFAyI/+ayEv4aCYhEu7IRgY5JAd3+nw4cnT+pdVrzmsaEyUJDK+C+UHDzSIDezUpMUp6he/SWsrXew6kV+1FtqB5I/Bq0ciI2INE2cNoREgnrdlPNPX3y62DfDJW98P5j75aZrbBQ5nAb9SjXv1mPcdcZNd9RWuh9wqYjGS6nHeoTlMlHDxfF1sBq25QWMPAEoKre/+R684JSKFTm+rt2YQCcr5RRiWVnBuGIteTNW+LJBVPzNNfbF58pLBRtXT8hoHHKNJPDakpoxivnV9s9tegNn0kecv0zUBX4xOW61Sw5CDCKTEsEl6e2ah8v0Xo/ze1FzNN0XVIfu0mbdLF5VFImenYLjc+tLZIa1SDKBi/VoknSEzSohqwZ3D74nwhH6e2xRU6mErni1DqnMDGNxFnIEElUy5L/crl52tVs+bWIsptSdbqkdPJu5koCZe+60E2naMpY0e7RsRGMy8akpgBVwHsZqnH6ojaiI6owyyfCLMkbFkXFUZvG5Zjt0Y92mjGF+DjR4wRsnR4MB0THrE6ue8yoLgCxoVxssCUdwx9zyCvT8VaHXT0x2qGGvLs+c82if/BPcHMCaCw2tjn+ZZGtC8QkuGX76DV7E0MHCXW/crYQFzbjM6S9EaCL/JOIMV4VfAhxMnW/g= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(376014)(7416014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:03:05.9441 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e3a3ca5c-f924-4cb7-18a9-08dd4c58aaae X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003447.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5707 From: Cosmin Ratiu If the device can do cross-esw scheduling, switch to using a shared rate domain so that devlink rate nodes can have parents from other functions of the same device. As a fallback mechanism, if switching to a shared rate domain failed, a warning is logged and the code proceeds with trying to use a private qos domain since cross-esw scheduling cannot be supported. Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c index e6dcfe348a7e..9af12ae98691 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c @@ -899,6 +899,20 @@ int mlx5_esw_qos_init(struct mlx5_eswitch *esw) bool use_shared_domain = esw->mode == MLX5_ESWITCH_OFFLOADS && MLX5_CAP_QOS(esw->dev, esw_cross_esw_sched); + if (use_shared_domain) { + u64 guid = mlx5_query_nic_system_image_guid(esw->dev); + int err; + + err = devlink_shared_rate_domain_init(priv_to_devlink(esw->dev), guid); + if (err) { + /* On failure, issue a warning and switch to using a private domain. */ + esw_warn(esw->dev, + "Shared devlink rate domain init failed (err %d), cross-esw QoS not available", + err); + use_shared_domain = false; + } + } + if (esw->qos.domain) { if (esw->qos.domain->shared == use_shared_domain) return 0; /* Nothing to change. */ From patchwork Thu Feb 13 18:01:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13973845 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2080.outbound.protection.outlook.com [40.107.220.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B043266B57; Thu, 13 Feb 2025 18:03:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469796; cv=fail; b=r8xib4SpwzpAeF3AezC6bso8GPqb2VL0fz+0afJ7ICdWpBr7+O2RZEt/8Yn2LsGioVxL/l9ZsqJump1lMcAkdTX+YizmlGM1kQe41I3lNjl1GncRcPPFYLfkYYMIDguClX6nHudWqDcFNUmXY0Fw2MYKL8y7JkntZvViXJb4ZLE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739469796; c=relaxed/simple; bh=67X2KCCvD4r2yIJREe7iNyL0hMK+UcrZy+UUzbMpKco=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jEnDBirm1A8TL6LZpwAs5GxUlVHRbBsXYmKP9DcxMbgNci/lgBT2oXuYNtXlnaGkinAdEElegJgEPzVTCnzl+v2CEtWoRbTAkP+6rtlKqYy0J+enMtAIclZZsPeGtyfnSk33BTdJ6Aa5Txhk2+eFO5thukK5g12LJrl4z9v0UOM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=swKtGxeo; arc=fail smtp.client-ip=40.107.220.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="swKtGxeo" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=r9moTlLyF1aUgBztRw1fFcNIlnydLLV/tjtdO36+4HtSLcGvHMjYYeGUyzgdE6fMpykVJDyZF+WWaSU9EE4js9O5rrTvOdVDq5IlFFzR9svl6ZRnpSKiMsfRKvB75rnw6kCA4RjMGnXKXVzJRKzqLd2/k+CBiZRR3YHGVBiAR7vQAwRKY1MtyzXGCxPODtdG2J/oduE6y12v4rMpagPgpJwPzz6HWKczgsZ7DzTFf4nfb+wnariTyVS224BfvVfY3FsVw681e3iI2P+8OWlWCrgQR2f3dUCR/v0k8J4usWEaUHXKOEtGu26TiD+VJPJ8ccARpt6EptKZmQhgM1Qq2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7qlEhmr/sRLGr0IOve+0Da9ekaqDij9SuQ08xeXPNnk=; b=Hpd4F1v8yMJgjf6sHW9lV18/be9DA2eVXK8XEql78mEBleYO1KdmJ+c9F/FdXYKeoUpxg017VDMNKXgJtXyGh3HLZ5Ngb782SPd3hgHt2V+vuqRerjv2UrU3Bn94cHuEUjAhezvfsUJDxiibhd5dx4DDoDno67DL287ZEHxYfXRbfPyfJ3qMVGSZH5OoZE02mbNk6Qf1UIiWOoXm4cdObACPMV4zGzOm5upZyU8hjD7E2fqfPf1CQhxSP1tj/l6mhFjS0ccH0PEFneJCytRzANrRWP687+XxLk2QJHwd8XJKnGvtw3F9ZGAmf3EQf2SDDWZwe/y972JaHA9gmmTBLg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7qlEhmr/sRLGr0IOve+0Da9ekaqDij9SuQ08xeXPNnk=; b=swKtGxeo+sSArabj/qPhQz3bfQeKjV50pCF0gutlxQRz1OEPHL3IcIgomfhcic3jpnaursFXiDBoUSqnRpl3iD25NU59cLKEbsRqTP6mcC4lMatIhaocf3mqfWl38Rh0HMSMcByt2/yqrGS9vAbwW8icq9+GPsPaN8Q69hYfOx1yvZDp0cqC9SI70LAU1rqsfzByvfOA5zrP7cIiuXQm6fhHQB/VX20VYC2r5E9gB0f4xhRYWAYIHm/TCf99tPlSTfBh2ePvK8m16poDz4XRw4sD131JnZv6ezQH05fWW7Ib4CcSry/7Xqt3scsZ4n5W7tYOV5UnpOpKv++rH2XS1Q== Received: from CH2PR08CA0007.namprd08.prod.outlook.com (2603:10b6:610:5a::17) by MW6PR12MB8707.namprd12.prod.outlook.com (2603:10b6:303:241::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.16; Thu, 13 Feb 2025 18:03:11 +0000 Received: from CH1PEPF0000AD80.namprd04.prod.outlook.com (2603:10b6:610:5a:cafe::ce) by CH2PR08CA0007.outlook.office365.com (2603:10b6:610:5a::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8445.13 via Frontend Transport; Thu, 13 Feb 2025 18:03:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH1PEPF0000AD80.mail.protection.outlook.com (10.167.244.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.10 via Frontend Transport; Thu, 13 Feb 2025 18:03:11 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Feb 2025 10:02:55 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Feb 2025 10:02:55 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Thu, 13 Feb 2025 10:02:51 -0800 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet , "Andrew Lunn" , Jiri Pirko CC: Cosmin Ratiu , Carolina Jubran , Gal Pressman , Mark Bloch , Donald Hunter , Jiri Pirko , Jonathan Corbet , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , , , , Subject: [PATCH net-next 10/10] net/mlx5: Document devlink rates and cross-esw scheduling Date: Thu, 13 Feb 2025 20:01:34 +0200 Message-ID: <20250213180134.323929-11-tariqt@nvidia.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20250213180134.323929-1-tariqt@nvidia.com> References: <20250213180134.323929-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD80:EE_|MW6PR12MB8707:EE_ X-MS-Office365-Filtering-Correlation-Id: 2600d30d-597e-4c54-0dc7-08dd4c58adcc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: Sewzt4lW7Q3JhIfF7JZQyg3tNXeN3E2d6JDyBfxGeI3gZ16xtsUUC/EFM6Ju5VPb1+5uwb8i4Sn7IxHQZeAMqSdAuaMlQRe4Y+nFL/sg+lzKqdHTVfyFQjpcvt2dJHkfqrd5BNUoXta32HF4vtKIvX/WTRqdC0RQYbJvS/XugMXhhui6hPs3XzjlMd9XCTFUJfLTeHG+40fUTV0MnHFEhpyo2fsv6zFK+Dt1AnuJtl+FJUzBdxilTpLokmSotXRdvH7Rvj0K4Cb+Iq5K1CCz8mG7aUA4k76l+7ADL50gZd/s2YDvpiRwQhrGqCvP4901riAqdWJQf7ZUZolNzkhuut3M08jrCpPyVvR8FC8zB6YRJDWnPzvIowexNPwlupPhqSbje/FcfnZ5u4WzUKXuhB2cpvzc2br8zi3W7ikGzf3NIVmOKTSJMXN7qteDK06p1vvShXgJvblLEkfdLCVzwh+RbAF6mbFf6Q89hyxozev36B+CZgZAz7YoetU4JZRKiNVjzoMF+wTiLpRjHTEckL1OkslGb+yBjHXWHTEldGvWbAalMp5KFrmba+LOKx5BJW2GDxr3BdZv0fmmQebBzdefGG/imV0KDflNWhxqBXOMVft1JUBRsTmPUG8nX4nQH9pxZX8gQGw2axQnwPVhxIIEJnfvv7Wwt0QCIcJnmCn8ZiqIqEC9PyA8d7Zbax4hmvjUPKCZkZ3lPb0C7QJzdSrCjXmbkC6ZTJKCl9nHg11+b4wtPjm8KRrwnmNZvPRyu21QroQHXjZOGjDWlDKYs14td1byryn9GkLgpgzXi5qLKJELENZAARM3rKFybcu8L9H8V6gPkQrrZ9MFc2IuFnhBaVttk2M2sU+G3O1L3w12N6N+ApoAVOpNtHvwTJC73yuLHRD2kJ8aDtXTsa7Kbit7W+o6Z8BPUMNV242tsf02HDri5QkuCsfKpc5J6uPlrtZKyaRfkO8/B2gCnsTSpJ1lffFxptSSoXPqDOKoSIuUfn9o2/WzIKmY5HkDNUYY/5adMCgDACHyCNjs8Am6ACpkh0RQXX3C9mUfiBq13czFT1toMs2LR0IIUCfYmIjPGJiV2+eSy8pYN7ZOvfdVCLtvJFujVzvwyLC+rdvFltRVJ6dAq9dvYxBWy8sSKsnOIY6uFxAuoNXvLxyDe7ImQ32agebfqYKQ5Q5biJzM+NVP1xFgAW4DYeKy0a+QmiqTyLy76KAt5lFKuFMHdvNZvY7io9FSKspUZ7v9Xu0dmUPq6zg58aKdClGAbYZgRXFsKEPlDxXARv26YnUs2WZ08ZfEIczZ8gb74JL0hgjZ05XBhNMntQvf31978F9Zv6kaGxyGQ/aD3FTieUGPV6jcTmxi8e+OKyilqoBQ+JXhroqJf/e0jYdb+yr4ZZVI/NH4e494jgI9CSLo4GxOrUjd0LOmyCLc/EkpahRVJ0gfO+Co1fIblvyEG0Yk4ZVOie6LMoysKGvyTVt8xx4QYjB+NP/+2faG1sduXKp4y3HP3lw= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 18:03:11.1892 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2600d30d-597e-4c54-0dc7-08dd4c58adcc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD80.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8707 From: Cosmin Ratiu Extend the devlink-port documentation with a mention that parents can be from different devices. It seems rates were not documented in the mlx5-specific file, so add examples on how to limit VFs and groups and also provide an example of the intended way to achieve cross-esw scheduling. Signed-off-by: Cosmin Ratiu Reviewed-by: Carolina Jubran Signed-off-by: Tariq Toukan --- .../networking/devlink/devlink-port.rst | 2 ++ Documentation/networking/devlink/mlx5.rst | 33 +++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/Documentation/networking/devlink/devlink-port.rst b/Documentation/networking/devlink/devlink-port.rst index 9d22d41a7cd1..1d9e5839eef4 100644 --- a/Documentation/networking/devlink/devlink-port.rst +++ b/Documentation/networking/devlink/devlink-port.rst @@ -417,6 +417,8 @@ API allows to configure following rate object's parameters: Parent node name. Parent node rate limits are considered as additional limits to all node children limits. ``tx_max`` is an upper limit for children. ``tx_share`` is a total bandwidth distributed among children. + If the device supports cross-function scheduling, the parent can be from a + different function of the same underlying device. ``tx_priority`` and ``tx_weight`` can be used simultaneously. In that case nodes with the same priority form a WFQ subgroup in the sibling group diff --git a/Documentation/networking/devlink/mlx5.rst b/Documentation/networking/devlink/mlx5.rst index 7febe0aecd53..61e76da36faf 100644 --- a/Documentation/networking/devlink/mlx5.rst +++ b/Documentation/networking/devlink/mlx5.rst @@ -298,3 +298,36 @@ User commands examples: .. note:: This command can run over all interfaces such as PF/VF and representor ports. + +Rates +===== + +mlx5 devices can limit transmission of individual VFs or a group of them via +the devlink-rate API in switchdev mode. + +User commands examples: + +- Print the existing rates:: + + $ devlink port function rate show + +- Set a max tx limit on traffic from VF0:: + + $ devlink port function rate set pci/0000:82:00.0/1 tx_max 10Gbit + +- Create a rate group with a max tx limit and adding two VFs to it:: + + $ devlink port function rate add pci/0000:82:00.0/group1 tx_max 10Gbit + $ devlink port function rate set pci/0000:82:00.0/1 parent group1 + $ devlink port function rate set pci/0000:82:00.0/2 parent group1 + +- Same scenario, with a min guarantee of 20% of the bandwidth for the first VFs:: + + $ devlink port function rate add pci/0000:82:00.0/group1 tx_max 10Gbit + $ devlink port function rate set pci/0000:82:00.0/1 parent group1 tx_share 2Gbit + $ devlink port function rate set pci/0000:82:00.0/2 parent group1 + +- Cross-device scheduling:: + + $ devlink port function rate add pci/0000:82:00.0/group1 tx_max 10Gbit + $ devlink port function rate set pci/0000:82:00.1/32769 parent group1