From patchwork Tue Sep 20 15:14:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 12982354 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0522ECAAD8 for ; Tue, 20 Sep 2022 15:15:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230310AbiITPPl (ORCPT ); Tue, 20 Sep 2022 11:15:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230096AbiITPPb (ORCPT ); Tue, 20 Sep 2022 11:15:31 -0400 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2125.outbound.protection.outlook.com [40.107.100.125]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26E39E0F3 for ; Tue, 20 Sep 2022 08:15:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NXgsm7UCW48dyOtVugvdf61gAwGvoN6ycWhr0L0yn2NfVvQ2KSGEnQbzXMiM+QmuZ2m7qQEYu9NL2Cx6qxa9pUnl9HVl1itbxFlxcZPNszzSF2uZBmMSgW5DJJV0t+04EZbKcEM9N8TI22dbluSFOonjg3cYdv/nEPCzkzXq7FHgo/5dfaBs9ZCsogxQKtskauAdXMfSM77lg1zhh9IGOlOo81GvYLI3hMLeUMsGTrInCH6FyArKyXGJWSxvgQZmBTKNGFeVPdA5418GrGkf9DOntHK0AwAU0ebzHC+WmiCzQDEs81yfeYr2dt+d5x0WnG6OqnS2jntHg6xvkwUTKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qc4x+yyEmslotEWH6jYRMDHTUS77ETsr2Zdk/4BQSWU=; b=a8uZhWewexkr1yNTCtJ3v+LjgLc197rWthuQUjQZN1g/AUwbQnoXGjjL8UkXsdIY/dpzkLO9DEovMl6SSzSA5Z4ceArqWUvh1xN4N0furF1FSvWsSM1xU0lrB5YRFZBpw5nuKkbWXmmMEQ8JQvUO3jrWXORKVaMd+mIOYdzxteSivaVkR9M1yWTrrGJCLEvxDECWXEhFggVm2Rwx0HvugKUDgsycD6DCufDc/HCrIupEHJLJHJ7ond1g/9uK797eEHOxSleTJac+AGCD605t1uEhHenRX6du6vqyPpA8Ka2KHLQ3gzKs34g+XqYzO3AvyARIhRrkaBum7IsI3CmJBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qc4x+yyEmslotEWH6jYRMDHTUS77ETsr2Zdk/4BQSWU=; b=mxZFsH/I2TDu+bHxmTdMhZmLdC96D8S/aJeQiQP81KQ9rOC760VzKzXu4pblhZy8tdHbjErm7STgu22XyA+7ETOQTDYCenn8k42wwQxI24cF14UY2j6jlvALyoPWBjLdstZOHVV7mLFzssPnEkuR3seEqawup+FYn+GpdUuq1As= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by SJ0PR13MB5301.namprd13.prod.outlook.com (2603:10b6:a03:3d9::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5654.14; Tue, 20 Sep 2022 15:15:26 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::2cbf:e4b1:5bbf:3e54]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::2cbf:e4b1:5bbf:3e54%4]) with mapi id 15.20.5654.014; Tue, 20 Sep 2022 15:15:23 +0000 From: Simon Horman To: David Miller , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, oss-drivers@corigine.com, Diana Wang , Peng Zhang Subject: [PATCH/RFC net-next 1/3] nfp: support VF multi-queues configuration Date: Tue, 20 Sep 2022 16:14:17 +0100 Message-Id: <20220920151419.76050-2-simon.horman@corigine.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220920151419.76050-1-simon.horman@corigine.com> References: <20220920151419.76050-1-simon.horman@corigine.com> X-ClientProxiedBy: AM3PR07CA0064.eurprd07.prod.outlook.com (2603:10a6:207:4::22) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR13MB4842:EE_|SJ0PR13MB5301:EE_ X-MS-Office365-Filtering-Correlation-Id: fe33f3fd-36a3-4c41-01f4-08da9b1af05d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UVrJyOmGXSQ5JvoM1DVJWODNm1oksX/nsgyZZmG48efipxb4YZE/fkeylMCtyKY8TUDbHFq7t/eB7vdsdfrBR7rP1MxMc6T7inDM+V3jn8/fqVGrndImnsOn8LgdFglNN+XKuQBwthsRKrnhNyP+QbZEnxwz8bAYjk135QsvmbOGlZwoiccwiZMBo371vXpIKMj5uvkZ6oYMN/oRiVQWK4X6uxo/g3biUP5HRENrcWDyOLEV+Y6uXwiwGrL6dLkUigNpKmBXeLaZxH7i8iAcH8BsasSF0pljdbbYVQBjy2x04N6iN86IPPIigEbSf1hbD9BoDvHXObiyDU++FdpSk9N6oAbje77T70RPzk2psXffpuCjStCDo8u2YACJsBqPyrStr3ZpXmv9s0hCe7Au8IrkF/hMUYDqpi4qlji5gSOpk2zohNVLDR9CZ+fCrEw7WiKKnbFH2Nj3gwyASCnAYTmTJlGNvjw3HjJkDCUZeeunq7KTTQ8MzbF+OIIXk86ZEKqHx6sQPbXvStPiYT4RlTNWh9axX3LurlhdNVW2kjK+YCRQjYiRW8+0FlCDKOrd7PqFdzfFNqbj8xgM6WrJhEYt/oJJwkmVQIobGpCwTsCvCeJKcs7slH962HB+BP3Ot9eSIY7ol+26ropwP+REPUZBkZ8MlRE/GpLrR3E2NhswlqxPx2+F+a76aho0JWdOp2Y/UjpA9oVC5E/V8VU1fQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR13MB4842.namprd13.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(396003)(366004)(39840400004)(346002)(136003)(376002)(451199015)(6486002)(478600001)(186003)(1076003)(2616005)(6506007)(6512007)(6666004)(107886003)(52116002)(2906002)(44832011)(8936002)(36756003)(5660300002)(110136005)(54906003)(316002)(4326008)(8676002)(41300700001)(86362001)(66476007)(66556008)(66946007)(83380400001)(38100700002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: YmQ5f0ZVdDq5uQKpZPV6Ota2fuPUaX2s0xs8sIX3gyUid1L6DJR5AijeFNgv+H2YY3x07bPiCM28hb+hXqNjj+yjCwPYitvTm5H9mCfDXdb3aBNvKKqmlkCXmot2JCkHHl8mGfusql5xk+V8RIZvG7HN/zeYSWNxnx45aGCCQjeursJidiZpCYJeoNgSvsZEGFBxbzhtVZPfL3MV9QGteqkk/UpSNSpVSHHPuai0AEbhLJT1ontqDGqtO3Udme/PNZ2AVK7xnjXPSXwmY682bC6EMKEIjicbiqqzTKWqLHvj4GXAx++eKM8iiRbscfFkM1bWWyEyyGjnx0bvDbeYFLe8DCzPMo1gYY1w7c51Z+WUVALUX2qtWjjdyE0Z4rBCAbUy/qCHIm+01Y0r2CJldRkb4mvP0QUhK8HyJIwTG8VbJEDlJTGhG1TCBblLU+Ge2+MLkDXIK2U3CfrEOgviqyMAyWYoLvlKF+HLbfumL60ILlFsPqI7B9YJ1Abea0NRKZ0UdP5KeV0X2+ySDg0lfY2BB5eN1FdgMbfWljslZqwmbc88+HHYCYIHf2L5cIeYWMmBBVqFk88TCXpcgXVckRN9tqEE0R47Mgj3y5R+FmKqYAl9ObIvm2i/oZ6CRzHuzokWFx/HcbEX2Rt4L7suJHoLuASxY+QBSVgThzgLZsbiXD6vCThQhNtNLhUiPsoEAYBin0fOofGFesNrU41DJqqA13hMfJZaPoCg3hiNRsVuxveXi6G/RkMRi4SfldyOnyPyPcYiHlyBecfub3nAiZQLWKGcyXRJ5EqYXVjPd729/U4B7dD7+f0marv/zIQjPzZQekJAVzazHVW7Nysc1xDhxGSt+dDKnVXvLwg4Ow3iaedIJTI9RwwnenIL6X4dmqe08b4DYuhIMo5uDOSoh38fOAcKhC+JYv7GHIXUxq5k5SJ+AG0nJNHOTK+3sHqLxUkcYAodNtpY7u3E9Kow5N1iwnuRd3qOo6d44AsTGA6UPOQtQTDlMPKhcPV3JYW5GHLnszUb4OpUwATVV9hJ5vFF+wh7kW0VFq2zErejGef+OmgrphUWLJS8aPbw/m4eWl28UpgZjjLsma+WTIVEWP9uKfx+YTaWek8uUM74/yYWhEOFFK/X9AGii6NzI1MId7w5kPcaR3jwUqDLbtCxIeY+LhUZiw8DSpheBT4bPiN5FvCrn5I0Uj7AegegdBv1KhEjHtOnYMt6R+QuNrpNxd0ZdmQNDDY5RqHAFg9A9PMCWoY1Qtgylv1Ng5ngJDtHWXiCelZLRhrP4jvnepOjlFLGMfU/noe41oDGaPU9BBqoh85uUzkzOjT0IMDNNd35uyOoxltiaE7b4wjTNnSDo/iR5kEPKyhZnEWSAUTrE7gbLalzz2tzcDPnFxri6kvcMtvBo87W3lcLKSSCEUlHaCEG2UOp+xis5KxtzsYA9BdhAQFfjL3HCKPQjD5NEZqFKGNj+hO0Wx9kHmvy3fQga4UW3Sejuuh9hhvNIEzDy73+sTcG/b/pBQ8a1JmbmaK0pas3xv+tK2pfuTliIN896y6Yux8M9ZUsBdhSo9KLDIj4mX/YKqLnmyQJ0qqPO7BA47E1cioODa8jEfOb65FQnQwBowkBSASCVuafWwLdpLAmsTU9fMuCwc3RpZ92rnNY30IxHHMngAgj3WwaaB+MdA== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: fe33f3fd-36a3-4c41-01f4-08da9b1af05d X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2022 15:15:23.1834 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: d1NBF2br34Tnd2dm/MjPsGjQ4cHZXOBapnKsHSE69g0b4UMRKUATR/YXeYHi/fRrqZvKh2Z+/Rcmz66oR1IGYOE2HoJrfTxWzwp1FFS7QS4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR13MB5301 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Diana Wang Add VF setting multi-queue feature. It is to configure the max queue number for each VF, user can still modify the queue number in use by ethtool -l The number set of configuring queues for every vf is {16 8 4 2 1} and total number of configuring queues is not allowed bigger than vf queues resource. If quantity of created VF exceeds expectation, it will check VF number validity based on the queues not used. The condition is that quantity of the rest queues must not smaller than redundant VFs' number. If it meets the condition, it will set one queue per extra VF. If not configured(default mode), the created VFs will divide the total vf-queues equally and it rounds down power of 2. Signed-off-by: Diana Wang Signed-off-by: Simon Horman --- drivers/net/ethernet/netronome/nfp/nfp_main.c | 6 ++ drivers/net/ethernet/netronome/nfp/nfp_main.h | 13 +++ drivers/net/ethernet/netronome/nfp/nfp_net.h | 1 + .../net/ethernet/netronome/nfp/nfp_net_main.c | 3 + .../ethernet/netronome/nfp/nfp_net_sriov.c | 101 ++++++++++++++++++ .../ethernet/netronome/nfp/nfp_net_sriov.h | 3 + 6 files changed, 127 insertions(+) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_main.c b/drivers/net/ethernet/netronome/nfp/nfp_main.c index 873429f7a6da..be1744d5b7ea 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_main.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_main.c @@ -29,6 +29,7 @@ #include "nfp_app.h" #include "nfp_main.h" #include "nfp_net.h" +#include "nfp_net_sriov.h" static const char nfp_driver_name[] = "nfp"; @@ -252,6 +253,10 @@ static int nfp_pcie_sriov_enable(struct pci_dev *pdev, int num_vfs) return -EINVAL; } + err = nfp_vf_queues_config(pf, num_vfs); + if (err) + return err; + err = pci_enable_sriov(pdev, num_vfs); if (err) { dev_warn(&pdev->dev, "Failed to enable PCI SR-IOV: %d\n", err); @@ -782,6 +787,7 @@ static int nfp_pci_probe(struct pci_dev *pdev, if (err) goto err_fw_unload; + pf->default_config_vfs_queue = true; pf->num_vfs = pci_num_vf(pdev); if (pf->num_vfs > pf->limit_vfs) { dev_err(&pdev->dev, diff --git a/drivers/net/ethernet/netronome/nfp/nfp_main.h b/drivers/net/ethernet/netronome/nfp/nfp_main.h index 6805af186f1b..37b2bd6091f0 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_main.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_main.h @@ -17,6 +17,12 @@ #include #include + /* Define how many types of max-q-number is supported to + * configure, currently we support 16, 8, 4, 2, 1. + */ +#define NFP_NET_CFG_QUEUE_TYPE 5 +#define NFP_NET_CFG_MAX_Q(type) (1 << (NFP_NET_CFG_QUEUE_TYPE - (type) - 1)) + struct dentry; struct device; struct pci_dev; @@ -63,6 +69,10 @@ struct nfp_dumpspec { * @irq_entries: Array of MSI-X entries for all vNICs * @limit_vfs: Number of VFs supported by firmware (~0 for PCI limit) * @num_vfs: Number of SR-IOV VFs enabled + * @max_vf_queues: number of queues can be allocated to VFs + * @config_vfs_queue: Array to indicate VF number of each max-queue-num type + * The quantity of distributable queues is {16, 8, 4, 2, 1} + * @default_config_vfs_queue: Is the method of allocating queues to VFS evenly distributed * @fw_loaded: Is the firmware loaded? * @unload_fw_on_remove:Do we need to unload firmware on driver removal? * @sp_indiff: Is the firmware indifferent to physical port speed? @@ -112,6 +122,9 @@ struct nfp_pf { unsigned int limit_vfs; unsigned int num_vfs; + unsigned int max_vf_queues; + u8 config_vfs_queue[NFP_NET_CFG_QUEUE_TYPE]; + bool default_config_vfs_queue; bool fw_loaded; bool unload_fw_on_remove; diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net.h b/drivers/net/ethernet/netronome/nfp/nfp_net.h index a101ff30a1ae..5deeae87b684 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_net.h @@ -78,6 +78,7 @@ /* Queue/Ring definitions */ #define NFP_NET_MAX_TX_RINGS 64 /* Max. # of Tx rings per device */ #define NFP_NET_MAX_RX_RINGS 64 /* Max. # of Rx rings per device */ +#define NFP_NET_CTRL_RINGS 1 /* Max. # of Ctrl rings per device */ #define NFP_NET_MAX_R_VECS (NFP_NET_MAX_TX_RINGS > NFP_NET_MAX_RX_RINGS ? \ NFP_NET_MAX_TX_RINGS : NFP_NET_MAX_RX_RINGS) #define NFP_NET_MAX_IRQS (NFP_NET_NON_Q_VECTORS + NFP_NET_MAX_R_VECS) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_main.c b/drivers/net/ethernet/netronome/nfp/nfp_net_main.c index e2d4c487e8de..e5d5f88e60c7 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_main.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_main.c @@ -296,6 +296,7 @@ static int nfp_net_pf_init_vnics(struct nfp_pf *pf) if (err) goto err_prev_deinit; + pf->max_vf_queues -= nn->max_r_vecs; id++; } @@ -794,6 +795,8 @@ int nfp_net_pci_probe(struct nfp_pf *pf) } } + pf->max_vf_queues = NFP_NET_MAX_R_VECS - NFP_NET_CTRL_RINGS; + err = nfp_net_pf_app_init(pf, qc_bar, stride); if (err) goto err_unmap; diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_sriov.c b/drivers/net/ethernet/netronome/nfp/nfp_net_sriov.c index 6eeeb0fda91f..eca6e65089f4 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_sriov.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_sriov.c @@ -29,6 +29,9 @@ nfp_net_sriov_check(struct nfp_app *app, int vf, u16 cap, const char *msg, bool return -EOPNOTSUPP; } + if (cap == NFP_NET_VF_CFG_MB_CAP_QUEUE_CONFIG) + return 0; + if (vf < 0 || vf >= app->pf->num_vfs) { if (warn) nfp_warn(app->pf->cpp, "invalid VF id %d\n", vf); @@ -309,3 +312,101 @@ int nfp_app_get_vf_config(struct net_device *netdev, int vf, return 0; } + +static int nfp_set_vf_queue_config(struct nfp_pf *pf, int num_vfs) +{ + unsigned char config_content[sizeof(u32)] = {0}; + unsigned int i, j, k, cfg_vf_count, offset; + struct nfp_net *nn; + u32 raw; + int err; + + raw = 0; k = 0; cfg_vf_count = 0; + offset = NFP_NET_VF_CFG_MB_SZ + pf->limit_vfs * NFP_NET_VF_CFG_SZ; + + for (i = 0; i < NFP_NET_CFG_QUEUE_TYPE; i++) { + for (j = 0; j < pf->config_vfs_queue[i]; j++) { + config_content[k++] = NFP_NET_CFG_MAX_Q(i); + cfg_vf_count++; + if (k == sizeof(raw) || cfg_vf_count == num_vfs) { + raw = config_content[0] | + (config_content[1] << BITS_PER_BYTE) | + (config_content[2] << (2 * BITS_PER_BYTE)) | + (config_content[3] << (3 * BITS_PER_BYTE)); + writel(raw, pf->vfcfg_tbl2 + offset); + offset += sizeof(raw); + memset(config_content, 0, sizeof(u32)); + k = 0; + } + } + } + + writew(NFP_NET_VF_CFG_MB_UPD_QUEUE_CONFIG, pf->vfcfg_tbl2 + NFP_NET_VF_CFG_MB_UPD); + + nn = list_first_entry(&pf->vnics, struct nfp_net, vnic_list); + err = nfp_net_reconfig(nn, NFP_NET_CFG_UPDATE_VF); + if (err) { + nfp_warn(pf->cpp, + "FW reconfig VF config queue failed: %d\n", err); + return -EINVAL; + } + + err = readw(pf->vfcfg_tbl2 + NFP_NET_VF_CFG_MB_RET); + if (err) { + nfp_warn(pf->cpp, + "FW refused VF config queue update with errno: %d\n", err); + return -EINVAL; + } + + return 0; +} + +int nfp_vf_queues_config(struct nfp_pf *pf, int num_vfs) +{ + unsigned int i, j, cfg_num_queues = 0, cfg_num_vfs; + + if (nfp_net_sriov_check(pf->app, 0, NFP_NET_VF_CFG_MB_CAP_QUEUE_CONFIG, "max_queue", true)) + return 0; + + /* In default mode, the created VFs divide all the VF queues equally, + * and round down to power of 2 + */ + if (pf->default_config_vfs_queue) { + memset(pf->config_vfs_queue, 0, NFP_NET_CFG_QUEUE_TYPE); + j = pf->max_vf_queues / num_vfs; + for (i = 0; i < NFP_NET_CFG_QUEUE_TYPE; i++) { + if (j >= NFP_NET_CFG_MAX_Q(i)) { + pf->config_vfs_queue[i] = num_vfs; + break; + } + } + return nfp_set_vf_queue_config(pf, num_vfs); + } + + for (i = 0, cfg_num_vfs = 0; i < NFP_NET_CFG_QUEUE_TYPE; i++) { + cfg_num_queues += NFP_NET_CFG_MAX_Q(i) * pf->config_vfs_queue[i]; + cfg_num_vfs += pf->config_vfs_queue[i]; + } + + if (cfg_num_queues > pf->max_vf_queues) { + dev_warn(&pf->pdev->dev, + "Number of queues from configuration is bigger than total queues number.\n"); + return -EINVAL; + } + + cfg_num_queues = pf->max_vf_queues - cfg_num_queues; + + if (num_vfs > cfg_num_vfs) { + cfg_num_vfs = num_vfs - cfg_num_vfs; + if (cfg_num_queues < cfg_num_vfs) { + dev_warn(&pf->pdev->dev, + "Remaining queues are not enough to be allocated.\n"); + return -EINVAL; + } + dev_info(&pf->pdev->dev, + "The extra created VFs are allocated with single queue.\n"); + pf->config_vfs_queue[NFP_NET_CFG_QUEUE_TYPE - 1] += cfg_num_vfs; + } + + return nfp_set_vf_queue_config(pf, num_vfs); +} diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_sriov.h b/drivers/net/ethernet/netronome/nfp/nfp_net_sriov.h index 2d445fa199dc..36df29fdaf0e 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_sriov.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_sriov.h @@ -21,6 +21,7 @@ #define NFP_NET_VF_CFG_MB_CAP_TRUST (0x1 << 4) #define NFP_NET_VF_CFG_MB_CAP_VLAN_PROTO (0x1 << 5) #define NFP_NET_VF_CFG_MB_CAP_RATE (0x1 << 6) +#define NFP_NET_VF_CFG_MB_CAP_QUEUE_CONFIG (0x1 << 7) #define NFP_NET_VF_CFG_MB_RET 0x2 #define NFP_NET_VF_CFG_MB_UPD 0x4 #define NFP_NET_VF_CFG_MB_UPD_MAC (0x1 << 0) @@ -30,6 +31,7 @@ #define NFP_NET_VF_CFG_MB_UPD_TRUST (0x1 << 4) #define NFP_NET_VF_CFG_MB_UPD_VLAN_PROTO (0x1 << 5) #define NFP_NET_VF_CFG_MB_UPD_RATE (0x1 << 6) +#define NFP_NET_VF_CFG_MB_UPD_QUEUE_CONFIG (0x1 << 7) #define NFP_NET_VF_CFG_MB_VF_NUM 0x7 /* VF config entry @@ -67,5 +69,6 @@ int nfp_app_set_vf_link_state(struct net_device *netdev, int vf, int link_state); int nfp_app_get_vf_config(struct net_device *netdev, int vf, struct ifla_vf_info *ivi); +int nfp_vf_queues_config(struct nfp_pf *pf, int num_vfs); #endif /* _NFP_NET_SRIOV_H_ */ From patchwork Tue Sep 20 15:14:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 12982355 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78794C54EE9 for ; Tue, 20 Sep 2022 15:15:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230344AbiITPPn (ORCPT ); Tue, 20 Sep 2022 11:15:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230481AbiITPPf (ORCPT ); Tue, 20 Sep 2022 11:15:35 -0400 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2125.outbound.protection.outlook.com [40.107.100.125]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AC10BCBA for ; Tue, 20 Sep 2022 08:15:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YLTSGQnIJIZdrkmB6WHJtNIBaRQvR3bulqOKf6BAvvxu0977DIvXVJtzNAHvyz9k60pPI5LrpPO0wv9UmVVJoppKSuN9UeNUtNsindZLB/uGEbC0sSKZ7MWv22xmy+2lEpsotbPVwuEe2NmzUOxttU2zVCKAtghtoKUfBQlethRJEi4uTNXw/fSavD3bl0wmG11lxl+i47kXgAPNzV5O1Jcm6Hq8i/14zIrQ/cU/14jQBFZl2h+wXvQml2ZaDh5dAIV5ocro8OwnBd/T4Hbwk5mbsqvUXFIldTyXszn36fo6N7/1O02R6kgq1laNMpg3xiHpG0Mqu4xzWO+MosN5PA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bW2kYODJb27wulPl0MAbAEpTeumiAqhx46AuB1x8QR8=; b=LmR/izOY9OkLrKh7kzaCVMvo2V00zvCO3LhgtbtpHkkBE7Wi19O4TH/Bff46qkUmCViMfVOOWmUJcBnQz0JiPGy0fKTqWDf1Io/ArgC6a+HjyNX8lhEETs81ZFQT27t0IJKgRnrnqYfFFwPXK7vrr2GkrM4v/4Oc8/m7uF5t+EVn62t075hY2tjgZ5rI5poR6EfRLm01xwf86sfpKt+U1bleLNVQmXTp0/5zCrcTPESmjq8BJwuQdNP4CEAdeYem3iiuG5603Qy7rZbE9NJwIGtRpoDPfNoAdrgSyWUwCpfDHnWmHXFhHrB+c4Wnf1hm6YYLHLZywW8qkse9Gcblmw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bW2kYODJb27wulPl0MAbAEpTeumiAqhx46AuB1x8QR8=; b=gcXKL7SFNxls7m5uWGvo1IMWhdVBQNcNAAPlGXtTTIzJoljEnvgV+i5HU0VyfIJltt5etMRiLihmYqDy+oBxUju7o5d5Loze5jAp0sMHekKTCQis9EeL0Yi7I5hSM2D3D1JCzHt3AsxF3bYi+j6+4SHf+7l2JM858nEqkYcEBW0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by SJ0PR13MB5301.namprd13.prod.outlook.com (2603:10b6:a03:3d9::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5654.14; Tue, 20 Sep 2022 15:15:27 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::2cbf:e4b1:5bbf:3e54]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::2cbf:e4b1:5bbf:3e54%4]) with mapi id 15.20.5654.014; Tue, 20 Sep 2022 15:15:27 +0000 From: Simon Horman To: David Miller , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, oss-drivers@corigine.com, Diana Wang , Peng Zhang Subject: [PATCH/RFC net-next 2/3] devlink: Add new "max_vf_queue" generic device param Date: Tue, 20 Sep 2022 16:14:18 +0100 Message-Id: <20220920151419.76050-3-simon.horman@corigine.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220920151419.76050-1-simon.horman@corigine.com> References: <20220920151419.76050-1-simon.horman@corigine.com> X-ClientProxiedBy: AM3PR07CA0064.eurprd07.prod.outlook.com (2603:10a6:207:4::22) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR13MB4842:EE_|SJ0PR13MB5301:EE_ X-MS-Office365-Filtering-Correlation-Id: dcf4b2e3-3521-40b2-c5d1-08da9b1af150 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: A+EeBDfYQKigWQkUnDJ+RuVTnrkEOVhyQH7+a0R6ifTKKZV5SIgbm3LtsOvULA9rn96LBFfzFf66fmaBx3aVWoP+2ALjQ/GYm6OKYY1kLJDYmivZfHd01L8K2vZISMH7f4CsxtQywlbXeqceeDbeELyFA1FRvtsDaTfVX1Pyum5/2AvXS5jU04pd4OYFxtPjs2Eoh/Ucif6avWtuYXrX8UWPQTz5pDghQDQoImT5ou+pHzZhaYkM0fSfgnP46BGJxVll1E1hhC93BS6OlP4trh27od6HsYkfnGsZiQdfAaYrPYU45fyHISbNLWZTWixFF4oBfxskhRmIySy5pgB1zErllZrl/jxNjJm/uApkxFDFxzoxfTDmGCew3kr90w6WwepOXNcP6Y5z5WBA4iKyUA2kpphX5Fumc0igh7ApokH5vw8qDFQBZQPulZbvNN/sSRBDF+OKqYNfRex3ZbfE+jx+Rad91PNi8hiyDUq26iBK5VuuQp406e2vA7nzF2kSzhoqOifr/GsJ1FyNwNj7ZNKw7LR5zUxXnHhkYW+ROSZBV5JNvMWngxPZx/TFLAhEm3YTIEvacMhilteO6FKndnOoveLsQwdUR0pqvCgLlZosOCYWSZ0NDQrGq1x/qiDFa6EQmHk5wLImAIzDH/VIfv/3BYQUNzhMTXFUEznCDR35mcsVI0R5TIC/KyWhvdE2M7BT5HegrJ0kWwecvg1tvzoNqM19+/9zeUFgUar5nTl2YnD01GDEM/llzyMFJMtpDWE0+9w/zXErVfBxF7RJ7w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR13MB4842.namprd13.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(396003)(366004)(39840400004)(346002)(136003)(376002)(451199015)(6486002)(478600001)(186003)(1076003)(2616005)(6506007)(6512007)(6666004)(107886003)(52116002)(2906002)(44832011)(8936002)(36756003)(5660300002)(110136005)(54906003)(316002)(4326008)(8676002)(41300700001)(86362001)(66476007)(66556008)(66946007)(83380400001)(38100700002)(41533002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6iCyXFyw8YLZBGJAVDWsbGrq+2FF+bknv3BJ+jiODYRjLridQbrzqaPQG1fNul+Qw89hHrvvt90Up9+eSZbJW2iJuFgSdgzK26jx9T/R0qe8w2NQhCW6/IgvwB4qCSZntxwnnNpUtStHhPxiPQ+XViidAMyUPMbY4mh7o1+ay+NE/Zu8IDmIGOwLkO+grQ11AOxiPhlypw+XxE4s92iuDaop0p5KI5dRdsYMGoqiYlRVFM4lbUp+Mtf4zxJ7n9wYqiqzOgvftbfeyg89smDpoW59z5bfspmPeG7cnvK4VIncD/dGnQNm6OV4O37cKIFMghlKF0VcKGbQSl7DgRbZEf8pT06RVy8kInPiqVi2jJJVIdUkcLRnyQtCzNyoENz8XF7Tn7s2IbXRZ6LImenuhwz2fUhGi9TQP1U8P8C1WDwpmAx1LgWyx+LO78B1cJb0RTL4MhqyoBowFInlIJ+m59va4R1ArcLTXNRo8j5EmA3W+9X8DYtzselxceQmlS8PruM282eIICMn6p2PczOvfQjp+xnnN8kSlFUyidoPRHNpav2oeDktGxy/7Bmvvu2uV74pLFuFIJGpaHe2bGMNenDym1UtrspLktcLn+6kshKbLA1FJqiOvnA6yUHgdFvXhIUjM6+vJqC2Lr8Q5Aj/SO0ueKSQcabDHl4rw5AZHGMWOhKGCh1oxJqHkyAmsIrCGBJSzXYHM/kdm/UQKuKDxlWSRi/Qnht7OMLCrtQnYliIphd+Vb3lt3i+VXBE+0T0b0uie0LWu7OYx9Mp/txlzi3+nYwkjXyLuhiFyIZ7xQiW2So19RDzHqCzoqAZ0XreqH1XqkGSZb2Z0YIYWSxCaBEeLhABVwrkSSZzNlZIVysiq2smC9gQzQBvqJ01IQ0eC40HLpnwxsWOnM2WFumC2G4Wlf6ZPAyqnzSQJC1hjTF4/dA2EBAnAe0zOc5YRuyLN/u4AQsGyra3t2QL8O1/ojVgtnq4lw2R3QIRX3XDpWMyNA8I0ikx8jCovnN4GUbMWzSvrMKbb72xDxRKZKOhQCOv7Yt3Wn6yfWReyxKXajyF9Da5y177kh3w6KcW/mdyUhRjg6iMEiJCGjnrq3jnnYqWwgFkNUYNCeL/wakfiiTxnYB7W6HbtROnawiBUXBNM/iSjAb2Usa0bFCJKvrQWhX6SHKC7f2RYpcSf7NAaxlw8vyToSDpScC3uUNpIKJYwyUy1h0CJZ1w5DWOOEG4yDvweZrUOc4mXt6lJ3vhhKz1S5YMQkO5qz5E3Qa7STwDeNNb/LEtBnx0IYZFFjLP1R5E9qCuZcPSxmCQRAq0cE0QlaeTV7w4kqfvQcik3LJ9mDEDnkrtYAg+LdjkdXYZHzKUJGdTjQh2dK8ELGXVfUU0+SwBj3Lb2PA1Mx+X7plcVbIBI0WaW2d33+1XydfAw8ORwF0tV+uYKR1wN0ZGGnN40lvVFGQ+nV+l5zuTehzG9lQIsJRKv57U6MkUqXFi7jeuSb8I1k8RHTQPhjYIHU0LO5QD9OpgSGiNzoEb6QmnEnin7/LQMQ6AeAwM4yXAojkCiH5oTKlaoxed5oLSFTmQ0bC93whsp9uH9py+/I2w7mHOOBPjxgaiED3rE+uXQMnS6XuGSaWBiljX6xjoJdDhlmv6XcQsdzUSVVCEASzJRZ8LjOb22TuLCa9IU7oUQA== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: dcf4b2e3-3521-40b2-c5d1-08da9b1af150 X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2022 15:15:24.7497 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: MtyAXv1TrVpVtDJCnwMTsctln+62zfM2LSWtaQURgf5g5k+Oe6dlIZ2fSvdvyc8sT6a9eJzzD8N8eJzdCGQF/JDQ7dOQrL5rq4RM6ihBrUg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR13MB5301 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Peng Zhang VF max-queue-number is the MAX num of queues which the VF has. Add new device generic parameter to configure the max-queue-number of the each VF to be generated dynamically. The string format is decided ad vendor specific. The suggested format is ...-V-W-X-Y-Z, the V represents generating V VFs that have 16 queues, the W represents generating W VFs that have 8 queues, and so on, the Z represents generating Z VFs that have 1 queue. For example, to configure * 1x VF with 128 queues * 1x VF with 64 queues * 0x VF with 32 queues * 0x VF with 16 queues * 12x VF with 8 queues * 2x VF with 4 queues * 2x VF with 2 queues * 0x VF with 1 queue, execute: $ devlink dev param set pci/0000:01:00.0 \ name max_vf_queue value \ "1-1-0-0-12-2-2-0" cmode runtime When created VF number is bigger than that is configured by this parameter, the extra VFs' max-queue-number is decided as vendor specific. If the config doesn't be set, the VFs' max-queue-number is decided as vendor specific. Signed-off-by: Peng Zhang Signed-off-by: Simon Horman --- Documentation/networking/devlink/devlink-params.rst | 5 +++++ include/net/devlink.h | 4 ++++ net/core/devlink.c | 5 +++++ 3 files changed, 14 insertions(+) diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst index 4e01dc32bc08..4b415b1acc9d 100644 --- a/Documentation/networking/devlink/devlink-params.rst +++ b/Documentation/networking/devlink/devlink-params.rst @@ -137,3 +137,8 @@ own name. * - ``event_eq_size`` - u32 - Control the size of asynchronous control events EQ. + * - ``max_vf_queue`` + - String + - Configure the queue of the each VF to be generated dynamically. When + created VF number is bigger than that is configured by this parameter, + the extra VFs' max-queue-number is decided as vendor specific. diff --git a/include/net/devlink.h b/include/net/devlink.h index 264aa98e6da6..b756be29f824 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -496,6 +496,7 @@ enum devlink_param_generic_id { DEVLINK_PARAM_GENERIC_ID_ENABLE_IWARP, DEVLINK_PARAM_GENERIC_ID_IO_EQ_SIZE, DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE, + DEVLINK_PARAM_GENERIC_ID_MAX_VF_QUEUE, /* add new param generic ids above here*/ __DEVLINK_PARAM_GENERIC_ID_MAX, @@ -554,6 +555,9 @@ enum devlink_param_generic_id { #define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME "event_eq_size" #define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE DEVLINK_PARAM_TYPE_U32 +#define DEVLINK_PARAM_GENERIC_MAX_VF_QUEUE_NAME "max_vf_queue" +#define DEVLINK_PARAM_GENERIC_MAX_VF_QUEUE_TYPE DEVLINK_PARAM_TYPE_STRING + #define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \ { \ .id = DEVLINK_PARAM_GENERIC_ID_##_id, \ diff --git a/net/core/devlink.c b/net/core/devlink.c index 7776dc82f88d..f447e8b11a80 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -5145,6 +5145,11 @@ static const struct devlink_param devlink_param_generic[] = { .name = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME, .type = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE, }, + { + .id = DEVLINK_PARAM_GENERIC_ID_MAX_VF_QUEUE, + .name = DEVLINK_PARAM_GENERIC_MAX_VF_QUEUE_NAME, + .type = DEVLINK_PARAM_GENERIC_MAX_VF_QUEUE_TYPE, + }, }; static int devlink_param_generic_verify(const struct devlink_param *param) From patchwork Tue Sep 20 15:14:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 12982356 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BD60C54EE9 for ; Tue, 20 Sep 2022 15:15:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230436AbiITPPo (ORCPT ); Tue, 20 Sep 2022 11:15:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231271AbiITPPh (ORCPT ); Tue, 20 Sep 2022 11:15:37 -0400 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2125.outbound.protection.outlook.com [40.107.100.125]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B8945AC7A for ; Tue, 20 Sep 2022 08:15:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LpN+guizmmHrjEVooYD4EAscV6pe5i1oeOXUwgPEBFmlkQmXluXQpVb0JmsaTu8drqZkceiFHZXUPmykfZK7g6/5ieWlh/3fcD3EqEaiXy36IE9rzl/GVlQbGiPGGSL+F95kRw6cBEMInLaV6TIDQdvhlWHb3LZ3Q+XExGLNLXPce+nZZUTKdRS2wGfURCrSGgb3mOeNZb3pJ9yzN3id3A7mq2C2p8UibJi7Pkd8bdeFurqeZ/bXaIbUMHhkZXO0fJ7VvVFXeDq625YsTvl18CI1syakyS6F9v48qfZl67k43Sa8Jjmy3FgKa3rbsdDAj86ek4YU5Gvlcd3Trn7hgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OPgrCWy1q7HucHuJ2Z6IoxoYadQVFdFA4pZVBWPxGv4=; b=HDY0i1nHP0R1HxzN3SR0wYCJVWm+LiRHj34AsY5KfSsAFW5lctQ5Ydfba9cc/qJbmZjGHTXb9rKAyr4W8lWFIk94oK844m94hClhCmXx9SKBu3K8ga3pCXOPeF9OxjAWsWG++ysWaYScK6Tg88ou5GPtwW60TKazHMRacrg+fenB3hLSYVyP4Pt873rkVrQ+WlTAjmLEOtrzI7rm/Kn3NxSQ/aqiHOu4DStxts96LYVI+VfEIHQFE515IVEPyH/Z5uKNYdKv/WWb8LdqTWY5RcI4qvUPJ6o/q60kSY+DWb9vtGGk6yhowgF5N/C0+z6djpgQAhSIXP22q08gbOkfHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OPgrCWy1q7HucHuJ2Z6IoxoYadQVFdFA4pZVBWPxGv4=; b=HDX7SttU5kIgow6vApTS3T2lhXm1PQkQlKdggacuWzE1Xbp0SIeXVb863zWbLUvQmIvHH47lQVDYq+2AiNZfiRZYCINc9VcF3c9P1qJPs90UL3MHxLlBNnA8/JulV983cZ3KCVZP9GEZQNRXnIk/3pjQ7HSRw1tKPR312Od0quA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by SJ0PR13MB5301.namprd13.prod.outlook.com (2603:10b6:a03:3d9::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5654.14; Tue, 20 Sep 2022 15:15:27 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::2cbf:e4b1:5bbf:3e54]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::2cbf:e4b1:5bbf:3e54%4]) with mapi id 15.20.5654.014; Tue, 20 Sep 2022 15:15:27 +0000 From: Simon Horman To: David Miller , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, oss-drivers@corigine.com, Diana Wang , Peng Zhang Subject: [PATCH/RFC net-next 3/3] nfp: devlink: add the devlink parameter "max_vf_queue" support Date: Tue, 20 Sep 2022 16:14:19 +0100 Message-Id: <20220920151419.76050-4-simon.horman@corigine.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220920151419.76050-1-simon.horman@corigine.com> References: <20220920151419.76050-1-simon.horman@corigine.com> X-ClientProxiedBy: AM3PR07CA0064.eurprd07.prod.outlook.com (2603:10a6:207:4::22) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR13MB4842:EE_|SJ0PR13MB5301:EE_ X-MS-Office365-Filtering-Correlation-Id: 608d7989-0db8-4ecf-5bfd-08da9b1af245 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5+AD0OiSFKvS0GszI76z3Z1QSV9hg/IodrnlZ81XbXYkf+WLnuwgMTtM0Q1WJSngLRp7xGhazqzbH6L/1Y6DI+qQLqRNFQ3bT4jpBV0Bt7b6gHAJPnUA95DbJxrx1yAi87SUXOlDgIMVJBeQCthR/fEsbVfmJ7YM1ye1Jcz3xFISOi4m9oZYD8zU4WTfpAE4PBU040j3y5QruYjP0VS8PsQiwqjjebTQJtVCenGNKNxd9sgwKRfsIieG/oAb2bWhLPXxDhfbTZ15kcqQoPlePDZiYPS5FiLuDstmytpFfKWGzoWjTN6kbTcf7tKsnTRZLVnNs0PC25uwsnHHnBvAUvoDvfLbd7Q2ZSW8wOpsB1ZZ5SJnw6L99RKqK2xs6+C9WkRXl7lSaW5e5Lf7a3RKIradmmx3kal8zh9Du4k+alMZr93SexM+rGrbXrjCs+2gVHaUyj7xtBkHTN2TnZzG6CXhGmEe+V/LzhKw3SVaWO2ue8LF8BKOXECCq2v5eaewPM3eqHTONW10ZPkG0SWor0CCEn76iHTk+zLFI/9fQdHxlkiFXE+PlFS6NGPj5YgvHxS2MStZC7zDiiHt0oEsLjrpdC9/y1f9OLIKkbqNw/8V0kOxJkpwIS3S393rASNUzoOvciY7UPl8kAxBzg3XduoalAmHLsWOx4Fk8nxaNkLc7fVtpS0cz6humH9Ugx8L92B8KrWEuE9YmnKgm0Uo/w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR13MB4842.namprd13.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(396003)(366004)(39840400004)(346002)(136003)(376002)(451199015)(6486002)(478600001)(186003)(1076003)(2616005)(6506007)(6512007)(6666004)(107886003)(52116002)(2906002)(44832011)(8936002)(36756003)(5660300002)(110136005)(54906003)(316002)(4326008)(8676002)(41300700001)(86362001)(66476007)(66556008)(66946007)(83380400001)(38100700002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qo6YwiCQcCfRy+RFSzvVQKoqjS1GJKBHpZapwMZKRhtFsfqeXW7rSWfBkI8BkyArbn2M6blEw0GjNug/r2yYtA8NEwdVfIiOYX9hvRZsmNbkBNVWurIPMjQSaOf1LUNZ1qA1CIrW+tukovXn06X44V0v5RAXHogX821Rg/etb5Q9pSpwAjEsH8YuRZZxmxE30u89umGI+smdk6CXc9E2WvYWaycKj6vC4cu+0UjgWgP8eK6xVrT3lrYoCy+HrO7hkOgjDMwzVyL+/zmvgvJ+YnAf85TgbHlvE75fDzUG2IhuXMje9WfFxvp3kFjRnV49UkQHGaoDfBY9hQyCHPMkOyInZnUoIaTfW6DuqktpmnbZEx8vwtjsemzj1JHwiOzoutqJhwBOw5UNxyxF1Dm0I6KtDjU5JS3SgXf3sZ+VUAehfWyEofaYMXD2A1w3T5a+sh8oFjzWh/pdBcR7mL2NUr0PWM4mDVlVmjaL3z7/9cRLZjNJGMPoxEUkCEpHVeVok9R+VHpU47y4XEMysv3cL4EtcGT2RPULA5hXmE2jQXyDnOgZEVoaUXkur3NDq1ASCCkKF1cRaRhl2OueIWMY/gEYXkSN9MrjY9rLVvKDETZjYrfVPduuXxUI2/phGuRpc7JAc6gVPRB3MoBOUeYRoJpNlgglEFi4CiJpZsgSRbxWA2jl47k6UqVYRpB47MuCiuHERKUSswFtgNIe+XHQjn3XKqLg7pv5Pxf39GdfO/P1NHX6hkymubKmEqV0MX+26Lq+jXzhL7YTSJ9QyzHfS91KJee6/4R5rWKCY55H2P1gTHnJBqPiceYLH+aE0zkA+KixUFXOlibeWY14/EGxH5+fzs5zpJAZOivTHBeZ0O/Sjipc4zEC+IthPV6ZhhXTgZ5lqG4U1+Xr4cuikEOgsvMZ+s499jQfak4ZNztTTIXSUWjUGdysRDsBjSv5zec+XyDnoDrOzjf1NIEiYRDq0RJeebwdk2pvPGaP6lrVBxRv/e4iCW29h0LVsZWnpwmLV3Bz2fVjT242bQIp8tV0ht8hGcs6z7n0dP7J5eZeEFH6JmG0JahmS8nzef2sWp7GZ7I7Vz5necNxixprqN3pO3jZccHwvCo7n8k2aOeOBh38RPoPzhTFOH1CucZ7oxJH2N6e7Qz/tRZrixlqrJi5BPsXcS6xCXTgWPASN5gfyMjpWMvRlQQlSxu/HbYBAkt6ZbepHPfg808oV9vnfuTAKc3eCFWg8/KteD0xETMo2yIdgq/Ycl3G0LD0oGS0Sk7m2SnEhMo3L/MdWjVr4B7zntiAJxCzf0N3lRfXeOwvfNeLpQ0sF28tHfSOJb3qge8dn6oEMRkJ+9sfO+BUU9LcJ7Ece1u8Q1fJp0IthoOGJV3p1KnptZpZ5/Zfo53emwCYmqHCCGsA76TJ3QbfxQ46yhGYNw2s4Wf687OpoeXqEq7j09tRGzeWxywrDuzIWrTrYVBA5BblCU6k4AZN3Fh5Gn6U8Wh7idh81qzzYbQpu7BOGbdjYhGhKrrqABtBmoOZzQqECuHrYv6AQoieXEosVkRgM1k+CONwpADDG/VoTrmVKJBNCMGqJ3Zgk8ZHPSFVGeb406fX3cpGPDU5LvH2og166oLFf28VYJ2dUHHPAY+8DDhEYXmZ/vxS8oLoTHhyPdvvcRmghk33ONlztQLsIg== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 608d7989-0db8-4ecf-5bfd-08da9b1af245 X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2022 15:15:26.3526 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C73tXS9JWp7guvUERf6w3XukiHFUv/jUUzxeGqf5TgzZ1iCtvVVfQo0iBSAHY3YA5t2eO2nqtDekOelgo6LZMm3trdWDGdiohrM9HrVTNKc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR13MB5301 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Peng Zhang VF max-queue-number is the MAX num of queues which the VF has. Currently, different users need that the number of VF max-queue-number is different. Configuring the VF max-queue-number dynamically can better match users. Hence, it is necessary to allow user to configure the number of the VF max-queue-number. The string format is V-W-X-Y-Z, the V represents generating V VFs that have 16 queues, the W represents generating W VFs that have 8 queues, and so on, the z represents generating Z VF that has 1 queues. As far, it supports the VF max-queue-number is 16. For example, to configure * 1x VF with 16 queue * 1x VF with 8 queues * 2x VF with 4 queues * 2x VF with 2 queues * 0x VF with 1 queue, execute: $ devlink dev param set pci/0000:01:00.0 \ name max_vf_queue value "1-1-2-2-0" cmode runtime The nfp also support the variable-length argument, Y-Z, X-Y-Z, W-X-Y-Z and Z, it also is right format and represents that the VFs which aren't configured are all zero. For example, execute: $ devlink dev param set pci/0000:01:00.0 \ name max_vf_queue value "1-1" cmode runtime It represent configure the queue is as follows: * 0x VF with 16 queue * 0x VF with 8 queues * 0x VF with 4 queues * 1x VF with 2 queues * 1x VF with 1 queue When created VF number is bigger than that is configured by this parameter, the extra VFs' max-queue-number is 1. If the config doesn't be set, the nfp vf max-queue-number is 2^n which is round_down the average of the total queue / VF nums. If setting the config is "0-0-0-0-0", it also is as the default way to generate VF. Signed-off-by: Peng Zhang Signed-off-by: Simon Horman --- Documentation/networking/devlink/nfp.rst | 2 + .../ethernet/netronome/nfp/devlink_param.c | 114 ++++++++++++++++++ 2 files changed, 116 insertions(+) diff --git a/Documentation/networking/devlink/nfp.rst b/Documentation/networking/devlink/nfp.rst index a1717db0dfcc..1936cee16dbe 100644 --- a/Documentation/networking/devlink/nfp.rst +++ b/Documentation/networking/devlink/nfp.rst @@ -18,6 +18,8 @@ Parameters - permanent * - ``reset_dev_on_drv_probe`` - permanent + * - ``vf_max_queue`` + - runtime Info versions ============= diff --git a/drivers/net/ethernet/netronome/nfp/devlink_param.c b/drivers/net/ethernet/netronome/nfp/devlink_param.c index db297ee4d7ad..5856b45601f7 100644 --- a/drivers/net/ethernet/netronome/nfp/devlink_param.c +++ b/drivers/net/ethernet/netronome/nfp/devlink_param.c @@ -6,6 +6,7 @@ #include "nfpcore/nfp.h" #include "nfpcore/nfp_nsp.h" #include "nfp_main.h" +#include "nfp_net.h" /** * struct nfp_devlink_param_u8_arg - Devlink u8 parameter get/set arguments @@ -191,7 +192,120 @@ nfp_devlink_param_u8_validate(struct devlink *devlink, u32 id, return 0; } +static int +nfp_devlink_vq_config_get(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct nfp_pf *pf = devlink_priv(devlink); + char config[4 * NFP_NET_CFG_QUEUE_TYPE]; + int i, len; + + if (!pf) + return -ENODEV; + + for (i = 0, len = 0; i < NFP_NET_CFG_QUEUE_TYPE; i++) + len += snprintf(config + len, sizeof(config), "%03d-", pf->config_vfs_queue[i]); + config[len - 1] = '\0'; + + strscpy(ctx->val.vstr, config, sizeof(ctx->val.vstr)); + + return 0; +} + +static int +nfp_devlink_vq_config_set(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + int config_vfs_queue[NFP_NET_CFG_QUEUE_TYPE]; + char vq[__DEVLINK_PARAM_MAX_STRING_VALUE]; + struct nfp_pf *pf = devlink_priv(devlink); + char *num_vf, *value; + u8 config = 0; + int i; + + if (!pf) + return -ENODEV; + + strscpy(vq, ctx->val.vstr, sizeof(vq)); + value = vq; + memset(config_vfs_queue, 0, sizeof(config_vfs_queue)); + + num_vf = strsep(&value, "-"); + while (num_vf) { + if (kstrtouint(num_vf, 10, &config_vfs_queue[config++])) + return -EINVAL; + num_vf = strsep(&value, "-"); + } + + pf->default_config_vfs_queue = true; + memset(pf->config_vfs_queue, 0, sizeof(pf->config_vfs_queue)); + + for (i = NFP_NET_CFG_QUEUE_TYPE - 1; i >= 0; i--) { + if (config >= 1) { + pf->config_vfs_queue[i] = config_vfs_queue[--config]; + if (pf->config_vfs_queue[i] && pf->default_config_vfs_queue) + pf->default_config_vfs_queue = false; + } + } + + return 0; +} + +static int +nfp_devlink_vq_config_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + int config_vfs_queue[NFP_NET_CFG_QUEUE_TYPE]; + char vq[__DEVLINK_PARAM_MAX_STRING_VALUE]; + struct nfp_pf *pf = devlink_priv(devlink); + char *num_vf, *value; + u32 total_q_num = 0; + u32 config = 0; + int i; + + if (!pf) { + NL_SET_ERR_MSG_MOD(extack, "Can't find the device."); + return -ENODEV; + } + + strscpy(vq, val.vstr, sizeof(vq)); + value = vq; + memset(config_vfs_queue, 0, sizeof(config_vfs_queue)); + + num_vf = strsep(&value, "-"); + while (num_vf) { + if (kstrtouint(num_vf, 10, &config_vfs_queue[config++])) { + NL_SET_ERR_MSG_MOD(extack, + "The input format is error, expected format: %d-%d-%d-%d-%d."); + return -EINVAL; + } + + if (config > NFP_NET_CFG_QUEUE_TYPE) { + NL_SET_ERR_MSG_MOD(extack, + "The config queue type is more than the max_cfg_queue_type."); + return -EINVAL; + } + num_vf = strsep(&value, "-"); + } + + for (i = NFP_NET_CFG_QUEUE_TYPE - 1; i >= 0 && config; i--) + total_q_num += config_vfs_queue[--config] * NFP_NET_CFG_MAX_Q(i); + + if (total_q_num > pf->max_vf_queues) { + NL_SET_ERR_MSG_MOD(extack, "The set queue is more than the MAX queue."); + return -EINVAL; + } + + return 0; +} + static const struct devlink_param nfp_devlink_params[] = { + DEVLINK_PARAM_GENERIC(MAX_VF_QUEUE, + BIT(DEVLINK_PARAM_CMODE_RUNTIME), + nfp_devlink_vq_config_get, + nfp_devlink_vq_config_set, + nfp_devlink_vq_config_validate), DEVLINK_PARAM_GENERIC(FW_LOAD_POLICY, BIT(DEVLINK_PARAM_CMODE_PERMANENT), nfp_devlink_param_u8_get,