From patchwork Mon Nov 7 11:57:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 13034276 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6772EC4332F for ; Mon, 7 Nov 2022 11:57:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231667AbiKGL5w (ORCPT ); Mon, 7 Nov 2022 06:57:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230434AbiKGL5u (ORCPT ); Mon, 7 Nov 2022 06:57:50 -0500 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2102.outbound.protection.outlook.com [40.107.102.102]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D6C71A81B for ; Mon, 7 Nov 2022 03:57:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BRTRBMuj9iSZTdxtL879AxYxCwvzYdzLqEZC44lKGbW9pprnX0SwFlakV1wSCV2ynFmXwFN007xFABiT1irQ3Y+SYeOxFLMdH8WE6N5F4VYfjMVhq28nnyOTpPC61DqQrZfVZx6w9J+WEJJRycxAJ6W/gTz0d3fHVF0v6OFR+2JYgehYmwF9OYTOQH24qEFteF1SoJT5ON7fcaKOnqOCXMWfFmMG6ZhrCgoAtLAUfjAw7BBnkgL4JkN3fMXrOwFP03mhOkK2s7kRDHBF+rbCeOBCGTc8ALJDyILXohhJ3d4fJ8GDVZdR/2apSBZAfjsPLJkgQqBwa6vYKpVxb2/G1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Q8swZbsm+fpMIJfx6MoXRVkeQTBrjl98/RXLS7L0Tu0=; b=GqZ26nmbEBZ02tvaMBM8IEv4RVxg0Zz9Ls5J09DMywOIBRh0OtrbAucpfF7KMcQ5Tt4iPhH3nExLNVeXvYFcieks/iYEmkine7YyaxCLM7U9pFwW9UgwFiheBQv0BYguB8gvL3hnWxA1hzLWwIy8VXAV3upbY2VDPSwVDuseFQ8AqG36+umYI+KQ6llpxANmessjeCLMPbYtnkeI15oIlGVg6s8nyrrAwsKyPHn1iwlti92CJYbe5IcTkYm82/+ri09IPuTtp2+BXrU+r8OxeCk7d7QQziY6xMnC1qkaL1IOvUEApBJ/n+/HrqZJ+sbnesD0as/9jeOUzz9ORjCE2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Q8swZbsm+fpMIJfx6MoXRVkeQTBrjl98/RXLS7L0Tu0=; b=FfiRPrSmvu4avTMhqMiynclYex3WicbNXmaqgHgLm78oJJGeK+GMq44gP+g1FIXzHBugkvYL4dFphLTVi7bBIA64YIGnmqwKiCJVd7F2grOupqyVukBTCezVGQzXC1kewDcH/HX+rgC/iQQFGsgSu2E2LXAZppu21s56RiEPqVU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by DM6PR13MB3971.namprd13.prod.outlook.com (2603:10b6:5:28f::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.26; Mon, 7 Nov 2022 11:57:46 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::483b:9e84:fadc:da30]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::483b:9e84:fadc:da30%9]) with mapi id 15.20.5791.025; Mon, 7 Nov 2022 11:57:46 +0000 From: Simon Horman To: David Miller , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, oss-drivers@corigine.com, Walter Heymans , =?utf-8?q?Niklas_S=C3=B6derlu?= =?utf-8?q?nd?= , Louis Peens , Simon Horman Subject: [PATCH net-next] Documentation: nfp: update documentation Date: Mon, 7 Nov 2022 06:57:26 -0500 Message-Id: <20221107115726.22658-1-simon.horman@corigine.com> X-Mailer: git-send-email 2.30.2 X-ClientProxiedBy: BL1PR13CA0198.namprd13.prod.outlook.com (2603:10b6:208:2be::23) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR13MB4842:EE_|DM6PR13MB3971:EE_ X-MS-Office365-Filtering-Correlation-Id: 9df65d10-c6bd-4d95-ec7e-08dac0b748da X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kD51P1kLYoU0RzFxZJsqzC/k5QsFdStAkhrfwmY2KcrpUgIeki+5nJMnzs1BgQrRkS4UIKeqLrpJX6x6cYQKKw55leExzv8AXcdkSM6pHVKQN/TVNzhkt2IdRnkbz5sIchB/evl3nGz7M+mib88l0bFgvrDvAW9jXrGKyjZ+3iBTM6x6zQOsn85Ar/FZxd7uVksF/b+aehPJZZPUWIk5mAdYSZvT/pAxjfBAEI1RZ2jqwNA4zDQqGzyaV/kNSKjry+/RcN5j70W7aTWfej91OiauaO/5rVa0Oh5Zfqy5B08VMUNgE9QdNJWdHyyGpw2itN4oNUyuWw5nEduZ7rw0B1ysfJY01lXgL4jh1fRmJR3Lf++hwnovRu9dfMIf17NjMJ0KnDbTdnmcfH1ynH8WjRnALq0LkltjNQj3WDNKB8WY2cAxtpqocObK9/XkWzEb9KlT3BtP6DnfluDrdKlQp6VZiZkTJEwuKxX4pvR3JviiRR3Tsjyrb71iqeuiCOWrNDiUQifQcsOcLYvYSBbGPSY9YeGpNHamdjdbVx+N5Af1NDRKQmyVEHvdfLJvDQvzvxDNzJa2cP4ehoLGNPNmJ9QjDZgeDP/Fwhc4bQV65xHr+06C9SZe63gC8TVp4Vt6YcMw5y9QeoWK7UZo1BHLpOGZC5Bp2MeSGgttE+nV1t9XhR1ety+2AgJqsRrCEZwx1z4C2gNhY1D24oNYjnTQeiiylIOUW6egDgJoqMHc19NMRqop6bZix+KaGabPzi3MBXFGPUSr0v4O1VsbDFsT0Out6yCa+KFeLijIOZbm8YCbJByIRQvEtwqxNZ1DZ3p+cGqldrn9vgQSDxt8zK4KTA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR13MB4842.namprd13.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(136003)(376002)(39830400003)(346002)(366004)(396003)(451199015)(6506007)(66574015)(6666004)(107886003)(1076003)(2616005)(186003)(52116002)(26005)(6512007)(38350700002)(38100700002)(83380400001)(5660300002)(2906002)(15650500001)(44832011)(110136005)(54906003)(6486002)(316002)(41300700001)(8936002)(478600001)(8676002)(4326008)(66476007)(66556008)(66946007)(36756003)(86362001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?6r9C0zy8Hrl8bFvGn1hqPXyrLSzY?= =?utf-8?q?MQ4d5RkTeNKDHeBBwoIEmMDzdXBCnDLQFRqkVaUiMz8JFY7UmUI8f0n7uzTixJCI9?= =?utf-8?q?/4SQj5Oi+XGNXHT3rMtfTCED/sUPTJI2ntgdSzQRw2r825L43orQ1qV09YEBWK/C8?= =?utf-8?q?PU8KUm9q21kHDf/XnCBve8Nqt7ILGLc50E9PPjYZjao+Edp+DxhEateNhlO8xfsgi?= =?utf-8?q?pI3oxqT4AqeJS+W9RDVT71n86DQWCYWg/zzYabS/js2kqkmLoI+RkMvMdjzUKaAqG?= =?utf-8?q?vvjfJKP4ugMyAaB7BikQCay3S4scivACjBFLD+hrZnqQkfoeJZjuQsOh6crNN45lS?= =?utf-8?q?q9+em8XVkwNIxCXs9/yiv7mELSmd25CxI426KBnG42n+feFtWkoLj5iMJyyh5iaDi?= =?utf-8?q?9s2Ef/IC+e9m0ISZOpPGjh3igczsi6g78srMEmGVAafZDhBSAQoa9hFrJxfpsIfpR?= =?utf-8?q?z0y8wC9hHva+cTNaG6yk2kbjLGx7gplZxNQa3/e4I+D0mW3J+qynShWZKE0uXD9x0?= =?utf-8?q?vr417sLwnRukUBmeBX6tOeqOuIbCsBf32ZEQfeB7F+O6wUpsEMcFoFY7hVUT/jx3J?= =?utf-8?q?S8BjwzyB9SFkRIMcjWz7qCwWFPt4ACVo8it9XcJgD0cOvYgFADxyMfmsWUE+urDme?= =?utf-8?q?M8JP0SZAOYpRhxJaN9jUMIXr5+Erid8EytHMpxHCNxyhBTqhTLIrFSbjFv9Q4eGBa?= =?utf-8?q?fe2imOiBrkAEv06PWXFvh0VjkEs6N3uq7+W0oFI88VR9G1Qee5o8oSgTTZ6D0aqR1?= =?utf-8?q?iXG6MDdUKpIXt8lZcbfx2aFEbM87OqtfDt7PTMydODlYNX0awdhsL2FgAec8JnkkR?= =?utf-8?q?ed9xYVtXemn9yxSrahbeHIaTRrqt4hSrX8ajD0i7c9UogA/wsgIqaG0C1Xr94ol3n?= =?utf-8?q?gjj7oWRH1EF1wf6LeRyw5RWNZKqm2urOzK5e8WVGQiX+65IDK+HJSMF2JurskTOpT?= =?utf-8?q?5XkfrFomfmlY/m0hinuQ2rGHQRRi5nfxQznzi8dZPT6rCahGV7firfc22Z1TZn3Tu?= =?utf-8?q?MZV1eWq467ARjAYiL/ZPGRm+yyR4BNZK6TUSe3LPvrJbjvUcoxJsora66dSN4o7Gj?= =?utf-8?q?EKPBZJdb5L+e1VVsTAWav/QHBNBGB/iEjdlXRThmwB4OkE1O3xoQaVf09Sh9TxLKs?= =?utf-8?q?NJ1WW2A+HauEBzjYm0x0MYqapq7aM95gJunRLYLQyFAPAqP4VDGUpvEQJQapD6vIm?= =?utf-8?q?UpLd/qfKYzcXqMeAL/t9GbLL8beOBPxLahpUPMDN97L9JQycvxTXuWTzwEFJUCIOb?= =?utf-8?q?EgEvDrNNJDvdcKJ2RzTKKh+d9ETcjVS6oZ4Z2spHFYfykE5Oj6JR8Tb8+f5s0Sv/j?= =?utf-8?q?vywST2k+M7L780pdDqrH1Zc357vTq+OktNZNksG/edTm/BIqW2jZYWjYupR9YKwJz?= =?utf-8?q?VCMyL7y68/9eADqU2Ip/tyb3bDsmKBLD6FwkZ7JuMSxYBmN+EjkMj4VG8e5TLDFMy?= =?utf-8?q?QTrZdTgtdwmT79h/TOo0uEl3n05dPqFIZ9POqYKKHUVXBckH9OFqMg30KSliatvHk?= =?utf-8?q?zteo4lNKiGipwBaYlksq94d3NY7K2+W1lA=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9df65d10-c6bd-4d95-ec7e-08dac0b748da X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Nov 2022 11:57:46.1451 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uT4I8AXNLyTFbV/kfmBtCu4cm1s6ew5MRsRvOd0dOFDnQRBrYQab4KSDpPVYlp9WiqXG+BRbdNE+xlIEZoofB1o8qU5ABAhzNCDo/k35Mbs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR13MB3971 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Walter Heymans The NFP documentation is updated to include information about Corigine, and the new NFP3800 chips. The 'Acquiring Firmware' section is updated with new information about where to find firmware. Two new sections are added to expand the coverage of the documentation. The new sections include: - Devlink Info - Configure Device Signed-off-by: Walter Heymans Reviewed-by: Niklas Söderlund Reviewed-by: Louis Peens Signed-off-by: Simon Horman --- .../device_drivers/ethernet/netronome/nfp.rst | 164 +++++++++++++++--- 1 file changed, 144 insertions(+), 20 deletions(-) diff --git a/Documentation/networking/device_drivers/ethernet/netronome/nfp.rst b/Documentation/networking/device_drivers/ethernet/netronome/nfp.rst index ada611fb427c..d50e28dfa95d 100644 --- a/Documentation/networking/device_drivers/ethernet/netronome/nfp.rst +++ b/Documentation/networking/device_drivers/ethernet/netronome/nfp.rst @@ -1,50 +1,56 @@ .. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) -============================================= -Netronome Flow Processor (NFP) Kernel Drivers -============================================= +=========================================== +Network Flow Processor (NFP) Kernel Drivers +=========================================== -Copyright (c) 2019, Netronome Systems, Inc. +:Copyright: |copy| 2019, Netronome Systems, Inc. +:Copyright: |copy| 2022, Corigine, Inc. Contents ======== - `Overview`_ - `Acquiring Firmware`_ +- `Devlink Info`_ +- `Configure Device`_ +- `Statistics`_ Overview ======== -This driver supports Netronome's line of Flow Processor devices, -including the NFP4000, NFP5000, and NFP6000 models, which are also -incorporated in the company's family of Agilio SmartNICs. The SR-IOV -physical and virtual functions for these devices are supported by -the driver. +This driver supports Netronome and Corigine's line of Network Flow Processor +devices, including the NFP3800, NFP4000, NFP5000, and NFP6000 models, which +are also incorporated in the companies' family of Agilio SmartNICs. The SR-IOV +physical and virtual functions for these devices are supported by the driver. Acquiring Firmware ================== -The NFP4000 and NFP6000 devices require application specific firmware -to function. Application firmware can be located either on the host file system +The NFP3800, NFP4000 and NFP6000 devices require application specific firmware +to function. Application firmware can be located either on the host file system or in the device flash (if supported by management firmware). Firmware files on the host filesystem contain card type (`AMDA-*` string), media -config etc. They should be placed in `/lib/firmware/netronome` directory to +config etc. They should be placed in `/lib/firmware/netronome` directory to load firmware from the host file system. Firmware for basic NIC operation is available in the upstream `linux-firmware.git` repository. +A more comprehensive list of firmware can be downloaded from the +`Corigine Support site `_. + Firmware in NVRAM ----------------- Recent versions of management firmware supports loading application -firmware from flash when the host driver gets probed. The firmware loading +firmware from flash when the host driver gets probed. The firmware loading policy configuration may be used to configure this feature appropriately. Devlink or ethtool can be used to update the application firmware on the device flash by providing the appropriate `nic_AMDA*.nffw` file to the respective -command. Users need to take care to write the correct firmware image for the +command. Users need to take care to write the correct firmware image for the card and media configuration to flash. Available storage space in flash depends on the card being used. @@ -79,9 +85,9 @@ You may need to use hard instead of symbolic links on distributions which use old `mkinitrd` command instead of `dracut` (e.g. Ubuntu). After changing firmware files you may need to regenerate the initramfs -image. Initramfs contains drivers and firmware files your system may -need to boot. Refer to the documentation of your distribution to find -out how to update initramfs. Good indication of stale initramfs +image. Initramfs contains drivers and firmware files your system may +need to boot. Refer to the documentation of your distribution to find +out how to update initramfs. Good indication of stale initramfs is system loading wrong driver or firmware on boot, but when driver is later reloaded manually everything works correctly. @@ -89,9 +95,9 @@ Selecting firmware per device ----------------------------- Most commonly all cards on the system use the same type of firmware. -If you want to load specific firmware image for a specific card, you -can use either the PCI bus address or serial number. Driver will print -which files it's looking for when it recognizes a NFP device:: +If you want to load a specific firmware image for a specific card, you +can use either the PCI bus address or serial number. The driver will +print which files it's looking for when it recognizes a NFP device:: nfp: Looking for firmware file in order of priority: nfp: netronome/serial-00-12-34-aa-bb-cc-10-ff.nffw: not found @@ -106,6 +112,15 @@ Note that `serial-*` and `pci-*` files are **not** automatically included in initramfs, you will have to refer to documentation of appropriate tools to find out how to include them. +Running firmware version +------------------------ + +The version of the loaded firmware for a particular interface, +(e.g. enp4s0), or an interface's port (e.g. enp4s0np0) can +be displayed with the ethtool command:: + + $ ethtool -i + Firmware loading policy ----------------------- @@ -132,6 +147,115 @@ abi_drv_load_ifc Defines a list of PF devices allowed to load FW on the device. This variable is not currently user configurable. +Devlink Info +============ + +The devlink info command displays the running and stored firmware versions +on the device, serial number and board information. + +Devlink info command example (replace PCI address):: + + $ devlink dev info pci/0000:03:00.0 + pci/0000:03:00.0: + driver nfp + serial_number CSAAMDA2001-1003000111 + versions: + fixed: + board.id AMDA2001-1003 + board.rev 01 + board.manufacture CSA + board.model mozart + running: + fw.mgmt 22.10.0-rc3 + fw.cpld 0x1000003 + fw.app nic-22.09.0 + chip.init AMDA-2001-1003 1003000111 + stored: + fw.bundle_id bspbundle_1003000111 + fw.mgmt 22.10.0-rc3 + fw.cpld 0x0 + chip.init AMDA-2001-1003 1003000111 + +Configure Device +================ + +This section explains how to use Agilio SmartNICs running basic NIC firmware. + +Configure interface link-speed +------------------------------ +The following steps explains how to change between 10G mode and 25G mode on +Agilio CX 2x25GbE cards. The changing of port speed must be done in order, +port 0 (p0) must be set to 10G before port 1 (p1) may be set to 10G. + +Down the respective interface(s):: + + $ ip link set dev down + $ ip link set dev down + +Set interface link-speed to 10G:: + + $ ethtool -s speed 10000 + $ ethtool -s speed 10000 + +Set interface link-speed to 25G:: + + $ ethtool -s speed 25000 + $ ethtool -s speed 25000 + +Reload driver for changes to take effect:: + + $ rmmod nfp; modprobe nfp + +Configure interface Maximum Transmission Unit (MTU) +--------------------------------------------------- + +The MTU of interfaces can temporarily be set using the iproute2, ip link or +ifconfig tools. Note that this change will not persist. Setting this via +Network Manager, or another appropriate OS configuration tool, is +recommended as changes to the MTU using Network Manager can be made to +persist. + +Set interface MTU to 9000 bytes:: + + $ ip link set dev mtu 9000 + +It is the responsibility of the user or the orchestration layer to set +appropriate MTU values when handling jumbo frames or utilizing tunnels. For +example, if packets sent from a VM are to be encapsulated on the card and +egress a physical port, then the MTU of the VF should be set to lower than +that of the physical port to account for the extra bytes added by the +additional header. If a setup is expected to see fallback traffic between +the SmartNIC and the kernel then the user should also ensure that the PF MTU +is appropriately set to avoid unexpected drops on this path. + +Configure Forward Error Correction (FEC) modes +---------------------------------------------- + +Agilio SmartNICs support FEC mode configuration, e.g. Auto, Firecode Base-R, +ReedSolomon and Off modes. Each physical port's FEC mode can be set +independently using ethtool. The supported FEC modes for an interface can +be viewed using:: + + $ ethtool + +The currently configured FEC mode can be viewed using:: + + $ ethtool --show-fec + +To force the FEC mode for a particular port, auto-negotiation must be disabled +(see the `Auto-negotiation`_ section). An example of how to set the FEC mode +to Reed-Solomon is:: + + $ ethtool --set-fec encoding rs + +Auto-negotiation +---------------- + +To change auto-negotiation settings, the link must first be put down. After the +link is down, auto-negotiation can be enabled or disabled using:: + + ethtool -s autoneg + Statistics ==========