From patchwork Wed Apr 3 20:08:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616628 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6153156B70; Wed, 3 Apr 2024 20:08:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174914; cv=none; b=dpxbJlMB9+uCZV2yDD1e4OULgusW+OhriQgqTQfA0LdneHnaLp2P230QMcDG/h2sEDoCvMW+jJGpFkQzWktoB5TYmf8UJ9VCibvC+nMzcK1hxVAQfsRPg5llDZlj5gEM0tyoEFhw8CyAmsKx0YlCl7xnHLzBMUfiMuLUvMARr5c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174914; c=relaxed/simple; bh=vgqvF/T1JCUMfC6OzKHdjzW5tm0PnI/QN0DFYtq1YN4=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=diyO1TDxohcgY7VSju08jW28wqaS8yY4LdD+dwkWv5PxJ3Hsdw7CcoMe/iuR/bQN7URZ2uwv3duCOUTv1uhxM5XkzDVe+eHiet5gw73C7rQJLkSCL/iTvyePor7gr57lcaJA2aN6uuL8Pr1jkyaAA+A18GM/7IlFaeLiGQN5e8Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Rv9sUr6u; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Rv9sUr6u" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1dff837d674so1770785ad.3; Wed, 03 Apr 2024 13:08:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174912; x=1712779712; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=R6+r4Zsm+3AsCM6pHOirbnqMypJToBlIyMm+FDA+rJY=; b=Rv9sUr6uMGCZZ3GBCvL3HuFKdo2J0pEQo3qrDdYbEQUckh2MAQthhau8ed/u0kiwtJ u6Em1KZAvckjWpeR6LlB6iOUwjd1jJi0VBKo3XGGTmf/Ft/dThx5SnRRAqSpCG3L9tDB BJgNHujtGKlfoVxlmji+Wnqw9i+faSq6clLJx8GlCezJCQpCbLRBI4d3Q4auZR8chFTt WzDkw6KoQfxnQnhwqpC9bXhIWB6lC6J2D+H2FtdcffXynlu1NVIdhcX3z1e4KeNTS4Yd njowmGXCOK5+Me1RTKfOMFguKwWY6LlYNxT6ny302dDOYc6lIudBRg2YbtXqkH+1MfIY cx2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174912; x=1712779712; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=R6+r4Zsm+3AsCM6pHOirbnqMypJToBlIyMm+FDA+rJY=; b=iDPvzQANXxGiL7FziTmhxfvPb3v2gRJVm1J1IJEsQ5m8KxU9H+KnjxiyOd8H4+irVA bPpiQ/yXe4BAaxrGBV1vECTe80Ov/dneNprRCoyeBPyOj1d6XGNrjL7jAq1WJF95JnG4 iGPdLU4ZRv4jDAPPQMKD2QiDGGrkRPKm/gm0mxlrziMOIKC6RailxxPwUO/HP/Fke7m8 ocE1kCgk4YxmxKGrMpq2hQSFWqLtouA7Il2+6Kz22461yTFstaAl7KOqfshBnNzr6LCe jrW/K6H4OdIjP4rvgIWdEzEBPq26DTHZDclu+f58/FKg8gZ9zISfSicP13aTdCvr5ufr fWMg== X-Forwarded-Encrypted: i=1; AJvYcCV7n59gxyNai3Xc3Cf+HGOeSzNK/78hi6HvDsNrirVd/+A44uy4ThdMhcdTODQ+H9kNxAyaEWcOwodEoA8gOchLdUvYqLpgsW1X X-Gm-Message-State: AOJu0YxVL4MvJuSPF3KuHPGVNxA1VsJtKhCP+deEOyY7cOJrZ1gBsFN9 t6LrnmHlzQawrZIPvLqstsNCD14ZYMBnA6eLW6K5lVCKP4JICvF0 X-Google-Smtp-Source: AGHT+IGUcJ5K7PnBeEskPVlOSiGNbIzfNyIJLUwbSjMXWFKAmJpzT1BEDGO3mGOdLom4RDvcxDJyPQ== X-Received: by 2002:a17:902:6b0a:b0:1e0:11a4:30e0 with SMTP id o10-20020a1709026b0a00b001e011a430e0mr328153plk.19.1712174911661; Wed, 03 Apr 2024 13:08:31 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id x15-20020a170902820f00b001dddf29b6e8sm13699110pln.299.2024.04.03.13.08.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:08:30 -0700 (PDT) Subject: [net-next PATCH 01/15] PCI: Add Meta Platforms vendor ID From: Alexander Duyck To: netdev@vger.kernel.org Cc: bhelgaas@google.com, linux-pci@vger.kernel.org, Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:08:28 -0700 Message-ID: <171217490835.1598374.17542029177954737682.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Add Meta as a vendor ID for PCI devices so we can use the macro for future drivers. CC: bhelgaas@google.com CC: linux-pci@vger.kernel.org Signed-off-by: Alexander Duyck Acked-by: Bjorn Helgaas --- include/linux/pci_ids.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h index a0c75e467df3..e5a1d5e9930b 100644 --- a/include/linux/pci_ids.h +++ b/include/linux/pci_ids.h @@ -2598,6 +2598,8 @@ #define PCI_VENDOR_ID_HYGON 0x1d94 +#define PCI_VENDOR_ID_META 0x1d9b + #define PCI_VENDOR_ID_FUNGIBLE 0x1dad #define PCI_VENDOR_ID_HXT 0x1dbf From patchwork Wed Apr 3 20:08:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616629 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5AA3456B70 for ; Wed, 3 Apr 2024 20:08:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174918; cv=none; b=GqC4KIBvKObwcBLYygjXVQL24PMl3jV5WO+1GNsJF+HVM+jAciEDpUgKjRYIUI309nefL/Frc0QYwibaB5yN9nBAMoJhzJ1GPMNYd0oW6opriRxpmLIUQjGgGgoCEDVrF8fx6CLgIlAhc2GkC2+W3UjdAZ4mscDDAAUOdtd1rGs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174918; c=relaxed/simple; bh=vs9rY9utHA3S6f2lTLj+Vd9VjkDK1FD7dTigB+veahA=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jjMIVta9uEW+tsfdQSIUsF/pcsU6VHuu2211x9XVoWqzfVhJvXzRSQJtAd90iddXM6GBLHScJ4WzmDZsQWCBlyqCixgKDqQ0VK2r2tgd/dtwhXv3NmQZNHlxwLxrx9xm0VQNWwoNR0bJ74XsUpMp0M2SOKxq2xsS4cyI4vId574= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MWRBh6x9; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MWRBh6x9" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1e28be94d32so1770045ad.0 for ; Wed, 03 Apr 2024 13:08:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174916; x=1712779716; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=CHzxQHK9Lqcog8W42sHoq7id1vDuaRjkicBhWkJ+UbY=; b=MWRBh6x9ZtakQDWvmHB7SL0Vm4bushKmLHu1LMojw6Dxt+12EqVIqU/V3IK1YCDVjw P0QmUSt3tnK82W4gv/b1z5DXQWOFKZPS+NRihNaaSfDpSWzF+JTOmvyl6xyHAAoa6izR 0q3m4MHxajCMjc91U0nc/R0ALv5wPKTj3SF+xZnWQAlNHQVt3CwCENcR+8eihgV6vQgY /3KOu3eUA8RLs2xJ2ygyFeHVDJkyoQGOD/LO4Aqz0pJhNRmmWSK6gwWCr0jUlaGAKxPf hbyfnnufnIniKJA3W73xanAVYtgUmeYThMkxHaRwgxVPxjvfgxIe4+EH7g3lSfLrdRvV GLAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174916; x=1712779716; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=CHzxQHK9Lqcog8W42sHoq7id1vDuaRjkicBhWkJ+UbY=; b=h3IH59jJVACUgg6xeMEUyIsP0wcBPqnVPu8MERpp+XUkr4Q+HG6aT1eLiQA2NX9IuI PALfUzKEA9rwsLfrjMMdqoss19BWbR/zXBWxG6zTBCSFFq7OhmRCKOQtXbDRLt8CamPi kPVbOgUjC/dcvFfNLCjfEeAjElQHtneLjUxPJiX3OOPQu7YljLAlflKtz8vMJHcz9TLt oI3p1VHlUu2Cpp6YXQ06nW2Br0xFI2xpkIo5hn+F7FxJ2BieEpVv2DShpo31w2qelQyR b2SjMafZmIrEo8QwLUHQy0Gy/JrRG5G9YSu/gUMLHH8SFxEL9pO9ZQBATZL2t6Eb0vna JuLQ== X-Gm-Message-State: AOJu0Yx6wf5Ujw2/IMzjTYnJxSQTbmozx+KhWMkbu/anbBct7cvgs2Ol GyRGYYn7nBGPA8/ZeeZNsStSzhzqHZEM/00uoHjWsKFCKurhwesg X-Google-Smtp-Source: AGHT+IG9FevuM4B+cUZCFlYpok8DJksTB+l+tjEmMUsYzpf99KtVi2k/N5kdBDoEbLX95zu+UXIptw== X-Received: by 2002:a17:902:db10:b0:1e0:a8b8:aba3 with SMTP id m16-20020a170902db1000b001e0a8b8aba3mr404309plx.35.1712174915458; Wed, 03 Apr 2024 13:08:35 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id u1-20020a170902e5c100b001e20574bacesm13765591plf.83.2024.04.03.13.08.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:08:34 -0700 (PDT) Subject: [net-next PATCH 02/15] eth: fbnic: add scaffolding for Meta's NIC driver From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:08:33 -0700 Message-ID: <171217491384.1598374.15535514527169847181.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Create a bare-bones PCI driver for Meta's NIC. Subsequent changes will flesh it out. Signed-off-by: Alexander Duyck --- MAINTAINERS | 7 + drivers/net/ethernet/Kconfig | 1 drivers/net/ethernet/Makefile | 1 drivers/net/ethernet/meta/Kconfig | 29 +++ drivers/net/ethernet/meta/Makefile | 6 + drivers/net/ethernet/meta/fbnic/Makefile | 10 + drivers/net/ethernet/meta/fbnic/fbnic.h | 19 ++ drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 9 + drivers/net/ethernet/meta/fbnic/fbnic_drvinfo.h | 5 + drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 196 +++++++++++++++++++++++ 10 files changed, 283 insertions(+) create mode 100644 drivers/net/ethernet/meta/Kconfig create mode 100644 drivers/net/ethernet/meta/Makefile create mode 100644 drivers/net/ethernet/meta/fbnic/Makefile create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic.h create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_csr.h create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_drvinfo.h create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_pci.c diff --git a/MAINTAINERS b/MAINTAINERS index 6a233e1a3cf2..77efffbd23f9 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14307,6 +14307,13 @@ T: git git://linuxtv.org/media_tree.git F: Documentation/devicetree/bindings/media/amlogic,gx-vdec.yaml F: drivers/staging/media/meson/vdec/ +META ETHERNET DRIVERS +M: Alexander Duyck +M: Jakub Kicinski +R: kernel-team@meta.com +S: Maintained +F: drivers/net/ethernet/meta/ + METHODE UDPU SUPPORT M: Robert Marko S: Maintained diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig index 6a19b5393ed1..0baac25db4f8 100644 --- a/drivers/net/ethernet/Kconfig +++ b/drivers/net/ethernet/Kconfig @@ -122,6 +122,7 @@ source "drivers/net/ethernet/litex/Kconfig" source "drivers/net/ethernet/marvell/Kconfig" source "drivers/net/ethernet/mediatek/Kconfig" source "drivers/net/ethernet/mellanox/Kconfig" +source "drivers/net/ethernet/meta/Kconfig" source "drivers/net/ethernet/micrel/Kconfig" source "drivers/net/ethernet/microchip/Kconfig" source "drivers/net/ethernet/mscc/Kconfig" diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile index 0d872d4efcd1..c03203439c0e 100644 --- a/drivers/net/ethernet/Makefile +++ b/drivers/net/ethernet/Makefile @@ -59,6 +59,7 @@ obj-$(CONFIG_NET_VENDOR_LITEX) += litex/ obj-$(CONFIG_NET_VENDOR_MARVELL) += marvell/ obj-$(CONFIG_NET_VENDOR_MEDIATEK) += mediatek/ obj-$(CONFIG_NET_VENDOR_MELLANOX) += mellanox/ +obj-$(CONFIG_NET_VENDOR_META) += meta/ obj-$(CONFIG_NET_VENDOR_MICREL) += micrel/ obj-$(CONFIG_NET_VENDOR_MICROCHIP) += microchip/ obj-$(CONFIG_NET_VENDOR_MICROSEMI) += mscc/ diff --git a/drivers/net/ethernet/meta/Kconfig b/drivers/net/ethernet/meta/Kconfig new file mode 100644 index 000000000000..8949ab15a02e --- /dev/null +++ b/drivers/net/ethernet/meta/Kconfig @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Meta Platforms network device configuration +# + +config NET_VENDOR_META + bool "Meta Platforms devices" + default y + help + If you have a network (Ethernet) card designed by Meta, say Y. + That's Meta as in the parent company of Facebook. + + Note that the answer to this question doesn't directly affect the + kernel: saying N will just cause the configurator to skip all + the questions about Meta cards. If you say Y, you will be asked for + your specific card in the following questions. + +if NET_VENDOR_META + +config FBNIC + tristate "Meta Platforms Host Network Interface" + depends on PCI_MSI + help + This driver supports Meta Platforms Host Network Interface. + + To compile this driver as a module, choose M here. The module + will be called fbnic. MSI-X interrupt support is required. + +endif # NET_VENDOR_META diff --git a/drivers/net/ethernet/meta/Makefile b/drivers/net/ethernet/meta/Makefile new file mode 100644 index 000000000000..88804f3de963 --- /dev/null +++ b/drivers/net/ethernet/meta/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Makefile for the Meta Platforms network device drivers. +# + +obj-$(CONFIG_FBNIC) += fbnic/ diff --git a/drivers/net/ethernet/meta/fbnic/Makefile b/drivers/net/ethernet/meta/fbnic/Makefile new file mode 100644 index 000000000000..ce277fec875f --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/Makefile @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) Meta Platforms, Inc. and affiliates. + +# +# Makefile for the Meta(R) Host Network Interface +# + +obj-$(CONFIG_FBNIC) += fbnic.o + +fbnic-y := fbnic_pci.o diff --git a/drivers/net/ethernet/meta/fbnic/fbnic.h b/drivers/net/ethernet/meta/fbnic/fbnic.h new file mode 100644 index 000000000000..25702dab8d66 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#ifndef _FBNIC_H_ +#define _FBNIC_H_ + +#include "fbnic_csr.h" + +extern char fbnic_driver_name[]; + +enum fbnic_boards { + fbnic_board_asic +}; + +struct fbnic_info { + unsigned int bar_mask; +}; + +#endif /* _FBNIC_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h new file mode 100644 index 000000000000..72e89c07bf54 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#ifndef _FBNIC_CSR_H_ +#define _FBNIC_CSR_H_ + +#define PCI_DEVICE_ID_META_FBNIC_ASIC 0x0013 + +#endif /* _FBNIC_CSR_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_drvinfo.h b/drivers/net/ethernet/meta/fbnic/fbnic_drvinfo.h new file mode 100644 index 000000000000..809ba6729442 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_drvinfo.h @@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#define DRV_NAME "fbnic" +#define DRV_SUMMARY "Meta(R) Host Network Interface Driver" diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c new file mode 100644 index 000000000000..1cb71cb1de14 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -0,0 +1,196 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#include +#include +#include +#include + +#include "fbnic.h" +#include "fbnic_drvinfo.h" + +char fbnic_driver_name[] = DRV_NAME; + +MODULE_DESCRIPTION(DRV_SUMMARY); +MODULE_LICENSE("GPL"); + +static const struct fbnic_info fbnic_asic_info = { + .bar_mask = BIT(0) | BIT(4) +}; + +static const struct fbnic_info *fbnic_info_tbl[] = { + [fbnic_board_asic] = &fbnic_asic_info, +}; + +static const struct pci_device_id fbnic_pci_tbl[] = { + { PCI_DEVICE_DATA(META, FBNIC_ASIC, fbnic_board_asic) }, + /* required last entry */ + {0, } +}; +MODULE_DEVICE_TABLE(pci, fbnic_pci_tbl); + +/** + * fbnic_probe - Device Initialization Routine + * @pdev: PCI device information struct + * @ent: entry in fbnic_pci_tbl + * + * Returns 0 on success, negative on failure + * + * Initializes a PCI device identified by a pci_dev structure. + * The OS initialization, configuring of the adapter private structure, + * and a hardware reset occur. + **/ +static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) +{ + const struct fbnic_info *info = fbnic_info_tbl[ent->driver_data]; + int err; + + if (pdev->error_state != pci_channel_io_normal) { + dev_err(&pdev->dev, + "PCI device still in an error state. Unable to load...\n"); + return -EIO; + } + + err = pcim_enable_device(pdev); + if (err) { + dev_err(&pdev->dev, "PCI enable device failed: %d\n", err); + return err; + } + + err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(46)); + if (err) + err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); + if (err) { + dev_err(&pdev->dev, "DMA configuration failed: %d\n", err); + return err; + } + + err = pcim_iomap_regions(pdev, info->bar_mask, fbnic_driver_name); + if (err) { + dev_err(&pdev->dev, + "pci_request_selected_regions failed: %d\n", err); + return err; + } + + pci_set_master(pdev); + pci_save_state(pdev); + + return 0; +} + +/** + * fbnic_remove - Device Removal Routine + * @pdev: PCI device information struct + * + * Called by the PCI subsystem to alert the driver that it should release + * a PCI device. The could be caused by a Hot-Plug event, or because the + * driver is going to be removed from memory. + **/ +static void fbnic_remove(struct pci_dev *pdev) +{ +} + +static int fbnic_pm_suspend(struct device *dev) +{ + return 0; +} + +static int __fbnic_pm_resume(struct device *dev) +{ + return 0; +} + +static int __maybe_unused fbnic_pm_resume(struct device *dev) +{ + int err; + + err = __fbnic_pm_resume(dev); + + return err; +} + +static const struct dev_pm_ops fbnic_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(fbnic_pm_suspend, fbnic_pm_resume) +}; + +static void fbnic_shutdown(struct pci_dev *pdev) +{ + fbnic_pm_suspend(&pdev->dev); +} + +static pci_ers_result_t fbnic_err_error_detected(struct pci_dev *pdev, + pci_channel_state_t state) +{ + /* disconnect device if failure is not recoverable via reset */ + if (state == pci_channel_io_perm_failure) + return PCI_ERS_RESULT_DISCONNECT; + + fbnic_pm_suspend(&pdev->dev); + + /* Request a slot reset */ + return PCI_ERS_RESULT_NEED_RESET; +} + +static pci_ers_result_t fbnic_err_slot_reset(struct pci_dev *pdev) +{ + pci_set_power_state(pdev, PCI_D0); + pci_restore_state(pdev); + pci_save_state(pdev); + + if (pci_enable_device_mem(pdev)) { + dev_err(&pdev->dev, + "Cannot re-enable PCI device after reset.\n"); + return PCI_ERS_RESULT_DISCONNECT; + } + + return PCI_ERS_RESULT_RECOVERED; +} + +static void fbnic_err_resume(struct pci_dev *pdev) +{ +} + +static const struct pci_error_handlers fbnic_err_handler = { + .error_detected = fbnic_err_error_detected, + .slot_reset = fbnic_err_slot_reset, + .resume = fbnic_err_resume, +}; + +static struct pci_driver fbnic_driver = { + .name = fbnic_driver_name, + .id_table = fbnic_pci_tbl, + .probe = fbnic_probe, + .remove = fbnic_remove, + .driver.pm = &fbnic_pm_ops, + .shutdown = fbnic_shutdown, + .err_handler = &fbnic_err_handler, +}; + +/** + * fbnic_init_module - Driver Registration Routine + * + * The first routine called when the driver is loaded. All it does is + * register with the PCI subsystem. + **/ +static int __init fbnic_init_module(void) +{ + int err; + + pr_info(DRV_SUMMARY " (%s)", fbnic_driver.name); + + err = pci_register_driver(&fbnic_driver); + + return err; +} +module_init(fbnic_init_module); + +/** + * fbnic_exit_module - Driver Exit Cleanup Routine + * + * Called just before the driver is removed from memory. + **/ +static void __exit fbnic_exit_module(void) +{ + pci_unregister_driver(&fbnic_driver); +} +module_exit(fbnic_exit_module); From patchwork Wed Apr 3 20:08:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616630 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAB3056B70 for ; Wed, 3 Apr 2024 20:08:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174921; cv=none; b=mUmr8JIDjDNtB4c3rxXnm5cOMpiWuW7bwdMSvbXYq8Qhl/TAHomUrltYiRA/HsBd0qgwMN/HtWVmVTEgoicqEA5lzWB+MWPVmtwmxggmQwhUO0jPMB+kVUjr53AZDnWFQDrEtV42bSnd6NUd5Yl3TqxKJ809XHrFtEyRJ32MiAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174921; c=relaxed/simple; bh=m98jrkWA91r6FO/W3KJ8ebcYVj7RUok6ZpNa6yaQgXg=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nE+lgCx0hi06zV8T5Hw3HrewoziyRU6KCI6rOIpFSg39H5d/4ggt1v/PxWLX16IHC5Yvgh0j34KFz6v6VlatR9WsowE9zpfWpcKzabIixnif6RNIe9XS5Pw7gU4mZFsw8O2PQRFTw9HWI5qWJbEvENp55K4e26kopG7E234j0IU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=N3TZkexS; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="N3TZkexS" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-6e704078860so199460b3a.0 for ; Wed, 03 Apr 2024 13:08:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174919; x=1712779719; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=RGx4yOlNRxJMGAJudxuSoGG7AnAEJGrxd45q47xs/Kg=; b=N3TZkexSoK8KvQjW8Ub68mprc8xhhgMDP1W2KFQ3ZoPmJ/zjV+Zxg827zkhnHmf9XU KjTGkEKiCEuwwteyR8jAVE8PX8jvv/+IXHjHlYP6xztUK5zhbmkaW3qzwGTQcOx8KF1K hMI880I1F8mgG9yMpDx7RM/2/W94IGq5bDa75ozsHVK+zlrB0mVX5H6VezwO37okqmln g1J9OtNIyBXzOb9I2ECafeLXkDOpstMIMYmdhFPrGY3MxUrnDHTejwx2OzUEDxtu1a6X WBWX+8swLVqn3EM7QvHedMxCqU+FjOsbr2eiWYAiBOtE6r7H9d5XapyZn03b+abhTP5F xWOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174919; x=1712779719; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=RGx4yOlNRxJMGAJudxuSoGG7AnAEJGrxd45q47xs/Kg=; b=ucyP4E3XD9NEl0Tcimne6+qR591d1xOwTrHdkumIQ6HfEVWrdJ2cVeIfxEC3uvuoIW X1nNAf+0p19h9iQYcxqZ81Cp4djpLXV4U7YXLuCjJPeofDVU7ck56WkiKwH23i8un3lc pfBWDCDQn71cx8UuxJZUMScub+eeu1qD310Dsdyp1d24nnPNAoT29sxNuOgUS8r91nMo FyauORO9Yc31dvbdl3LYuffr31EfpsmquPzZLfLnuqh358zbEgdkut49tMyFxsb4rPbJ txlqyBVaTSaDgEXSv+Xh4g3ZfTmnZ/RxKF2HhI0i1/GqcdfvX16aBoTiIVwXT+zzGwS9 yWkQ== X-Gm-Message-State: AOJu0YxRheOwaRdsjIswVz4/pBsqTblpJRDMNpXeJFuVRcRCwCd/2rX5 O9IglQCT+0Vq/Yvi5zh2MqT/r2HDTwScpr9tlbKcmP1h0Mb11ks/ X-Google-Smtp-Source: AGHT+IHzUQxsbfOoFmL759s4eTRmKJgIC6YBJvJf7q2376uRfVq3ChzpVzBJduieMqishGIQUvBVjw== X-Received: by 2002:a05:6a00:1886:b0:6ea:be87:fd36 with SMTP id x6-20020a056a00188600b006eabe87fd36mr615902pfh.1.1712174919269; Wed, 03 Apr 2024 13:08:39 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id gu5-20020a056a004e4500b006ead4eb1b09sm12165696pfb.124.2024.04.03.13.08.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:08:38 -0700 (PDT) Subject: [net-next PATCH 03/15] eth: fbnic: Allocate core device specific structures and devlink interface From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:08:37 -0700 Message-ID: <171217491765.1598374.8648487319055615080.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck At the core of the fbnic device will be the devlink interface. This interface will eventually provide basic functionality in the event that there are any issues with the network interface. Add support for allocating the MSI-X vectors and setting up the BAR mapping. With this we can start enabling various subsytems and start brining up additional interfaces such the AXI fabric and the firmware mailbox. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/Makefile | 4 + drivers/net/ethernet/meta/fbnic/fbnic.h | 28 ++++++++ drivers/net/ethernet/meta/fbnic/fbnic_devlink.c | 84 +++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_irq.c | 52 ++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 72 +++++++++++++++++++- 5 files changed, 238 insertions(+), 2 deletions(-) create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_devlink.c create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_irq.c diff --git a/drivers/net/ethernet/meta/fbnic/Makefile b/drivers/net/ethernet/meta/fbnic/Makefile index ce277fec875f..c06041e70bc5 100644 --- a/drivers/net/ethernet/meta/fbnic/Makefile +++ b/drivers/net/ethernet/meta/fbnic/Makefile @@ -7,4 +7,6 @@ obj-$(CONFIG_FBNIC) += fbnic.o -fbnic-y := fbnic_pci.o +fbnic-y := fbnic_devlink.o \ + fbnic_irq.o \ + fbnic_pci.o diff --git a/drivers/net/ethernet/meta/fbnic/fbnic.h b/drivers/net/ethernet/meta/fbnic/fbnic.h index 25702dab8d66..f322cea4ce22 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic.h @@ -6,8 +6,36 @@ #include "fbnic_csr.h" +struct fbnic_dev { + struct device *dev; + + u32 __iomem *uc_addr0; + u32 __iomem *uc_addr4; + struct msix_entry *msix_entries; + unsigned short num_irqs; + + u64 dsn; +}; + +/* Reserve entry 0 in the MSI-X "others" array until we have filled all + * 32 of the possible interrupt slots. By doing this we can avoid any + * potential conflicts should we need to enable one of the debug interrupt + * causes later. + */ +enum { + FBNIC_NON_NAPI_VECTORS +}; + extern char fbnic_driver_name[]; +void fbnic_devlink_free(struct fbnic_dev *fbd); +struct fbnic_dev *fbnic_devlink_alloc(struct pci_dev *pdev); +void fbnic_devlink_register(struct fbnic_dev *fbd); +void fbnic_devlink_unregister(struct fbnic_dev *fbd); + +void fbnic_free_irqs(struct fbnic_dev *fbd); +int fbnic_alloc_irqs(struct fbnic_dev *fbd); + enum fbnic_boards { fbnic_board_asic }; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c b/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c new file mode 100644 index 000000000000..91e8135410df --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#include +#include +#include +#include + +#include "fbnic.h" + +#define FBNIC_SN_STR_LEN 24 + +static int fbnic_devlink_info_get(struct devlink *devlink, + struct devlink_info_req *req, + struct netlink_ext_ack *extack) +{ + struct fbnic_dev *fbd = devlink_priv(devlink); + int err; + + if (fbd->dsn) { + unsigned char serial[FBNIC_SN_STR_LEN]; + u8 dsn[8]; + + put_unaligned_be64(fbd->dsn, dsn); + err = snprintf(serial, FBNIC_SN_STR_LEN, "%8phD", dsn); + if (err < 0) + return err; + + err = devlink_info_serial_number_put(req, serial); + if (err) + return err; + } + + return 0; +} + +static const struct devlink_ops fbnic_devlink_ops = { + .info_get = fbnic_devlink_info_get, +}; + +void fbnic_devlink_free(struct fbnic_dev *fbd) +{ + struct devlink *devlink = priv_to_devlink(fbd); + + devlink_free(devlink); +} + +struct fbnic_dev *fbnic_devlink_alloc(struct pci_dev *pdev) +{ + void __iomem * const *iomap_table; + struct devlink *devlink; + struct fbnic_dev *fbd; + + devlink = devlink_alloc(&fbnic_devlink_ops, sizeof(struct fbnic_dev), + &pdev->dev); + if (!devlink) + return NULL; + + fbd = devlink_priv(devlink); + pci_set_drvdata(pdev, fbd); + fbd->dev = &pdev->dev; + + iomap_table = pcim_iomap_table(pdev); + fbd->uc_addr0 = iomap_table[0]; + fbd->uc_addr4 = iomap_table[4]; + + fbd->dsn = pci_get_dsn(pdev); + + return fbd; +} + +void fbnic_devlink_register(struct fbnic_dev *fbd) +{ + struct devlink *devlink = priv_to_devlink(fbd); + + devlink_register(devlink); +} + +void fbnic_devlink_unregister(struct fbnic_dev *fbd) +{ + struct devlink *devlink = priv_to_devlink(fbd); + + devlink_unregister(devlink); +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_irq.c b/drivers/net/ethernet/meta/fbnic/fbnic_irq.c new file mode 100644 index 000000000000..d2fdc51704b9 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_irq.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#include +#include + +#include "fbnic.h" + +void fbnic_free_irqs(struct fbnic_dev *fbd) +{ + struct pci_dev *pdev = to_pci_dev(fbd->dev); + + fbd->num_irqs = 0; + + pci_disable_msix(pdev); + + kfree(fbd->msix_entries); + fbd->msix_entries = NULL; +} + +int fbnic_alloc_irqs(struct fbnic_dev *fbd) +{ + unsigned int wanted_irqs = FBNIC_NON_NAPI_VECTORS; + struct pci_dev *pdev = to_pci_dev(fbd->dev); + struct msix_entry *msix_entries; + int i, num_irqs; + + msix_entries = kcalloc(wanted_irqs, sizeof(*msix_entries), GFP_KERNEL); + if (!msix_entries) + return -ENOMEM; + + for (i = 0; i < wanted_irqs; i++) + msix_entries[i].entry = i; + + num_irqs = pci_enable_msix_range(pdev, msix_entries, + FBNIC_NON_NAPI_VECTORS + 1, + wanted_irqs); + if (num_irqs < 0) { + dev_err(fbd->dev, "Failed to allocate MSI-X entries\n"); + kfree(msix_entries); + return num_irqs; + } + + if (num_irqs < wanted_irqs) + dev_warn(fbd->dev, "Allocated %d IRQs, expected %d\n", + num_irqs, wanted_irqs); + + fbd->msix_entries = msix_entries; + fbd->num_irqs = num_irqs; + + return 0; +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index 1cb71cb1de14..596151396eac 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -43,6 +43,7 @@ MODULE_DEVICE_TABLE(pci, fbnic_pci_tbl); static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) { const struct fbnic_info *info = fbnic_info_tbl[ent->driver_data]; + struct fbnic_dev *fbd; int err; if (pdev->error_state != pci_channel_io_normal) { @@ -72,10 +73,41 @@ static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) return err; } + fbd = fbnic_devlink_alloc(pdev); + if (!fbd) { + dev_err(&pdev->dev, "Devlink allocation failed\n"); + return -ENOMEM; + } + pci_set_master(pdev); pci_save_state(pdev); + fbnic_devlink_register(fbd); + + err = fbnic_alloc_irqs(fbd); + if (err) + goto free_fbd; + + if (!fbd->dsn) { + dev_warn(&pdev->dev, "Reading serial number failed\n"); + goto init_failure_mode; + } + return 0; + +init_failure_mode: + dev_warn(&pdev->dev, "Probe error encountered, entering init failure mode. Normal networking functionality will not be available.\n"); + /* Always return 0 even on error so devlink is registered to allow + * firmware updates for fixes. + */ + return 0; +free_fbd: + pci_disable_device(pdev); + + fbnic_devlink_unregister(fbd); + fbnic_devlink_free(fbd); + + return err; } /** @@ -88,16 +120,49 @@ static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) **/ static void fbnic_remove(struct pci_dev *pdev) { + struct fbnic_dev *fbd = pci_get_drvdata(pdev); + + fbnic_free_irqs(fbd); + + fbnic_devlink_unregister(fbd); + fbnic_devlink_free(fbd); } static int fbnic_pm_suspend(struct device *dev) { + struct fbnic_dev *fbd = dev_get_drvdata(dev); + + /* Free the IRQs so they aren't trying to occupy sleeping CPUs */ + fbnic_free_irqs(fbd); + + /* Hardware is about ot go away, so switch off MMIO access internally */ + WRITE_ONCE(fbd->uc_addr0, NULL); + WRITE_ONCE(fbd->uc_addr4, NULL); + return 0; } static int __fbnic_pm_resume(struct device *dev) { + struct fbnic_dev *fbd = dev_get_drvdata(dev); + void __iomem * const *iomap_table; + int err; + + /* restore MMIO access */ + iomap_table = pcim_iomap_table(to_pci_dev(dev)); + fbd->uc_addr0 = iomap_table[0]; + fbd->uc_addr4 = iomap_table[4]; + + /* rerequest the IRQs */ + err = fbnic_alloc_irqs(fbd); + if (err) + goto err_invalidate_uc_addr; + return 0; +err_invalidate_uc_addr: + WRITE_ONCE(fbd->uc_addr0, NULL); + WRITE_ONCE(fbd->uc_addr4, NULL); + return err; } static int __maybe_unused fbnic_pm_resume(struct device *dev) @@ -133,6 +198,8 @@ static pci_ers_result_t fbnic_err_error_detected(struct pci_dev *pdev, static pci_ers_result_t fbnic_err_slot_reset(struct pci_dev *pdev) { + int err; + pci_set_power_state(pdev, PCI_D0); pci_restore_state(pdev); pci_save_state(pdev); @@ -143,7 +210,10 @@ static pci_ers_result_t fbnic_err_slot_reset(struct pci_dev *pdev) return PCI_ERS_RESULT_DISCONNECT; } - return PCI_ERS_RESULT_RECOVERED; + /* restore device to previous state */ + err = __fbnic_pm_resume(&pdev->dev); + + return err ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED; } static void fbnic_err_resume(struct pci_dev *pdev) From patchwork Wed Apr 3 20:08:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616631 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BFD3155757 for ; Wed, 3 Apr 2024 20:08:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174927; cv=none; b=ZO0VOVpPEfcwd7b40du8V4DY+PKgHl2US8ktgznIxqzf6hm6fnDEgY9b0admfSNzFe4ahUqjOpiuzXAHAqaG54q+GlJi/kE8aV8ODGSvyxzgELztQzDIuGpV1LH5KQpyHv+OdID6oX1jz4sdytuz7LCgIkFX63gGWuBBQxzvka8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174927; c=relaxed/simple; bh=DrE+CEr0QBtmZGRwG0YrGdE2DZuh9jZfurL8RNiRxUc=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rOVWEaJWttm4yYd0eTtxNEWVaFe7T6m3cgngpCyl/e7MhpAurXovLO5h+JByJo8O17DUBtlQ7JxQIhcW4Tax3d6QHN+3yam8Ruv7Ch6WtnLBD7m1SmPS6kQTP2xNofvwZzlE+7poxU8NIQgOZ9GXUUQNv86G41nK6KAx0ztHW5Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PkP2Up4a; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PkP2Up4a" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1e2a2d5f0b7so1650145ad.1 for ; Wed, 03 Apr 2024 13:08:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174924; x=1712779724; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=UcvgWj6/Bx7dXyHTFuLWchMouNvUYunZTDUrZIjhetg=; b=PkP2Up4ah+COtmyh3EGna9jaLLQ2N6XPya4+ToTK9XIKj9XXWfhiT4OFOxKf/y5n9b FQRu122T6bE89kcFFpnaH6AjQ4BvVjGabwtlO6zVFr9Tv3l3dAWVK7T3Wj7kq+JhHY9C Dp+hf0Jm13/dj66gnWn2Cndhm5zMpnROzIBRB7UOjFQdIx0ChmOFHLBGgvrv9SSMyJjo AacWrrKDMk03pf8JwheLCzTXOOLmQDG18RaCnLxB+/SkFJzCtLjbCpGqvx2MIIApwEce zAB1uIqXuWVbsq70T8v89EJNZO7FhHSJolawaDazwVb5WbFlwgWko2PrP8la3Ly83zvV 9xMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174924; x=1712779724; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=UcvgWj6/Bx7dXyHTFuLWchMouNvUYunZTDUrZIjhetg=; b=d3f8gawf9gEiEU9Y5aB5+DzTf4ZNVN3H+5CINdI245P+O7pvVUNaoq8A1A+8SKI6ow z52DduEf2bKoy/Hdjs8ajdMzLJSL6kz5BERiE5XGR+3V+D+/vFsNYKcDIt3BwSEHrunB YWzpw7n0QBjJhUJhVdCrmsTyK4z/RIKbnk108AUALeC4UPqfsvRrdmx6aVGp5bW+r3vo RpTUc6nAxkzHaN0WLCTWsiDHnmTJDbRny87PjlxLK4XeW8vvWrZ2b6qmgo30yf9tayrD xF/Nwjcu9Wxmll8FeDveHnsQDrEbi/Uye0Y+5YPH9Yy4v3qr9MataUHVyy9b4f52e+8R ac/A== X-Gm-Message-State: AOJu0YyzFlcSwHXHvaOjMsWLGR8RnczF0sgIlIN4ojO2dXEfg3X9SlmN D07M8MZs502+FrydEa4lHm6Sbl+50ZdZHsiUXrnhwo1xeO+1KFRb X-Google-Smtp-Source: AGHT+IGz+q37JChCPXMsXyX5kTOlDFY5ijR8gWqxY3YTw7jPdsa0uhSiS+pJlFWn1J7nbBxRBGne7Q== X-Received: by 2002:a17:902:7c04:b0:1e0:b709:139b with SMTP id x4-20020a1709027c0400b001e0b709139bmr244369pll.56.1712174923430; Wed, 03 Apr 2024 13:08:43 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id l11-20020a170903120b00b001dddaa7d046sm13775711plh.29.2024.04.03.13.08.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:08:42 -0700 (PDT) Subject: [net-next PATCH 04/15] eth: fbnic: Add register init to set PCIe/Ethernet device config From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:08:41 -0700 Message-ID: <171217492154.1598374.14769531965073029777.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck As a part of enabling the device the first step is to configure the AXI and Ethernet interfaces to allow for basic traffic. This consists of configuring several registers related to the PCIe and Ethernet MAC as well as configuring the handlers for moving traffic between entities. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/Makefile | 1 drivers/net/ethernet/meta/fbnic/fbnic.h | 36 ++ drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 312 +++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_mac.c | 438 +++++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_mac.h | 25 ++ drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 39 ++ 6 files changed, 851 insertions(+) create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_mac.c create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_mac.h diff --git a/drivers/net/ethernet/meta/fbnic/Makefile b/drivers/net/ethernet/meta/fbnic/Makefile index c06041e70bc5..b8f4511440dc 100644 --- a/drivers/net/ethernet/meta/fbnic/Makefile +++ b/drivers/net/ethernet/meta/fbnic/Makefile @@ -9,4 +9,5 @@ obj-$(CONFIG_FBNIC) += fbnic.o fbnic-y := fbnic_devlink.o \ fbnic_irq.o \ + fbnic_mac.o \ fbnic_pci.o diff --git a/drivers/net/ethernet/meta/fbnic/fbnic.h b/drivers/net/ethernet/meta/fbnic/fbnic.h index f322cea4ce22..63ec82a830cd 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic.h @@ -4,17 +4,23 @@ #ifndef _FBNIC_H_ #define _FBNIC_H_ +#include + #include "fbnic_csr.h" +#include "fbnic_mac.h" struct fbnic_dev { struct device *dev; u32 __iomem *uc_addr0; u32 __iomem *uc_addr4; + const struct fbnic_mac *mac; struct msix_entry *msix_entries; unsigned short num_irqs; u64 dsn; + u32 mps; + u32 readrq; }; /* Reserve entry 0 in the MSI-X "others" array until we have filled all @@ -26,6 +32,36 @@ enum { FBNIC_NON_NAPI_VECTORS }; +static inline bool fbnic_present(struct fbnic_dev *fbd) +{ + return !!READ_ONCE(fbd->uc_addr0); +} + +static inline void fbnic_wr32(struct fbnic_dev *fbd, u32 reg, u32 val) +{ + u32 __iomem *csr = READ_ONCE(fbd->uc_addr0); + + if (csr) + writel(val, csr + reg); +} + +u32 fbnic_rd32(struct fbnic_dev *fbd, u32 reg); + +static inline void +fbnic_rmw32(struct fbnic_dev *fbd, u32 reg, u32 mask, u32 val) +{ + u32 v; + + v = fbnic_rd32(fbd, reg); + v &= ~mask; + v |= val; + fbnic_wr32(fbd, reg, v); +} + +#define wr32(reg, val) fbnic_wr32(fbd, reg, val) +#define rd32(reg) fbnic_rd32(fbd, reg) +#define wrfl() fbnic_rd32(fbd, FBNIC_MASTER_SPARE_0) + extern char fbnic_driver_name[]; void fbnic_devlink_free(struct fbnic_dev *fbd); diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index 72e89c07bf54..eb37e5981e69 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -4,6 +4,318 @@ #ifndef _FBNIC_CSR_H_ #define _FBNIC_CSR_H_ +#include + +#define CSR_BIT(nr) (1u << (nr)) +#define CSR_GENMASK(h, l) GENMASK(h, l) + #define PCI_DEVICE_ID_META_FBNIC_ASIC 0x0013 +#define FBNIC_CLOCK_FREQ (600 * (1000 * 1000)) + +/* Global QM Tx registers */ +#define FBNIC_CSR_START_QM_TX 0x00800 /* CSR section delimiter */ +#define FBNIC_QM_TWQ_DEFAULT_META_L 0x00818 /* 0x02060 */ +#define FBNIC_QM_TWQ_DEFAULT_META_H 0x00819 /* 0x02064 */ + +#define FBNIC_QM_TQS_CTL0 0x0081b /* 0x0206c */ +#define FBNIC_QM_TQS_CTL0_LSO_TS_MASK CSR_BIT(0) +enum { + FBNIC_QM_TQS_CTL0_LSO_TS_FIRST = 0, + FBNIC_QM_TQS_CTL0_LSO_TS_LAST = 1, +}; + +#define FBNIC_QM_TQS_CTL0_PREFETCH_THRESH CSR_GENMASK(7, 1) +enum { + FBNIC_QM_TQS_CTL0_PREFETCH_THRESH_MIN = 16, +}; + +#define FBNIC_QM_TQS_CTL1 0x0081c /* 0x02070 */ +#define FBNIC_QM_TQS_CTL1_MC_MAX_CREDITS CSR_GENMASK(7, 0) +#define FBNIC_QM_TQS_CTL1_BULK_MAX_CREDITS CSR_GENMASK(15, 8) +#define FBNIC_QM_TQS_MTU_CTL0 0x0081d /* 0x02074 */ +#define FBNIC_QM_TQS_MTU_CTL1 0x0081e /* 0x02078 */ +#define FBNIC_QM_TQS_MTU_CTL1_BULK CSR_GENMASK(13, 0) +#define FBNIC_QM_TCQ_CTL0 0x0082d /* 0x020b4 */ +#define FBNIC_QM_TCQ_CTL0_COAL_WAIT CSR_GENMASK(15, 0) +#define FBNIC_QM_TCQ_CTL0_TICK_CYCLES CSR_GENMASK(26, 16) +#define FBNIC_QM_TQS_EDT_TS_RANGE 0x00849 /* 0x2124 */ +#define FBNIC_QM_TNI_TDF_CTL 0x0086c /* 0x021b0 */ +#define FBNIC_QM_TNI_TDF_CTL_MRRS CSR_GENMASK(1, 0) +#define FBNIC_QM_TNI_TDF_CTL_CLS CSR_GENMASK(3, 2) +#define FBNIC_QM_TNI_TDF_CTL_MAX_OT CSR_GENMASK(11, 4) +#define FBNIC_QM_TNI_TDF_CTL_MAX_OB CSR_GENMASK(23, 12) +#define FBNIC_QM_TNI_TDE_CTL 0x0086d /* 0x021b4 */ +#define FBNIC_QM_TNI_TDE_CTL_MRRS CSR_GENMASK(1, 0) +#define FBNIC_QM_TNI_TDE_CTL_CLS CSR_GENMASK(3, 2) +#define FBNIC_QM_TNI_TDE_CTL_MAX_OT CSR_GENMASK(11, 4) +#define FBNIC_QM_TNI_TDE_CTL_MAX_OB CSR_GENMASK(24, 12) +#define FBNIC_QM_TNI_TDE_CTL_MRRS_1K CSR_BIT(25) +#define FBNIC_QM_TNI_TCM_CTL 0x0086e /* 0x021b8 */ +#define FBNIC_QM_TNI_TCM_CTL_MPS CSR_GENMASK(1, 0) +#define FBNIC_QM_TNI_TCM_CTL_CLS CSR_GENMASK(3, 2) +#define FBNIC_QM_TNI_TCM_CTL_MAX_OT CSR_GENMASK(11, 4) +#define FBNIC_QM_TNI_TCM_CTL_MAX_OB CSR_GENMASK(23, 12) +#define FBNIC_CSR_END_QM_TX 0x00873 /* CSR section delimiter */ + +/* Global QM Rx registers */ +#define FBNIC_CSR_START_QM_RX 0x00c00 /* CSR section delimiter */ +#define FBNIC_QM_RCQ_CTL0 0x00c0c /* 0x03030 */ +#define FBNIC_QM_RCQ_CTL0_COAL_WAIT CSR_GENMASK(15, 0) +#define FBNIC_QM_RCQ_CTL0_TICK_CYCLES CSR_GENMASK(26, 16) +#define FBNIC_QM_RNI_RBP_CTL 0x00c2d /* 0x030b4 */ +#define FBNIC_QM_RNI_RBP_CTL_MRRS CSR_GENMASK(1, 0) +#define FBNIC_QM_RNI_RBP_CTL_CLS CSR_GENMASK(3, 2) +#define FBNIC_QM_RNI_RBP_CTL_MAX_OT CSR_GENMASK(11, 4) +#define FBNIC_QM_RNI_RBP_CTL_MAX_OB CSR_GENMASK(23, 12) +#define FBNIC_QM_RNI_RDE_CTL 0x00c2e /* 0x030b8 */ +#define FBNIC_QM_RNI_RDE_CTL_MPS CSR_GENMASK(1, 0) +#define FBNIC_QM_RNI_RDE_CTL_CLS CSR_GENMASK(3, 2) +#define FBNIC_QM_RNI_RDE_CTL_MAX_OT CSR_GENMASK(11, 4) +#define FBNIC_QM_RNI_RDE_CTL_MAX_OB CSR_GENMASK(23, 12) +#define FBNIC_QM_RNI_RCM_CTL 0x00c2f /* 0x030bc */ +#define FBNIC_QM_RNI_RCM_CTL_MPS CSR_GENMASK(1, 0) +#define FBNIC_QM_RNI_RCM_CTL_CLS CSR_GENMASK(3, 2) +#define FBNIC_QM_RNI_RCM_CTL_MAX_OT CSR_GENMASK(11, 4) +#define FBNIC_QM_RNI_RCM_CTL_MAX_OB CSR_GENMASK(23, 12) +#define FBNIC_CSR_END_QM_RX 0x00c34 /* CSR section delimiter */ + +/* TCE registers */ +#define FBNIC_CSR_START_TCE 0x04000 /* CSR section delimiter */ +#define FBNIC_TCE_REG_BASE 0x04000 /* 0x10000 */ + +#define FBNIC_TCE_LSO_CTRL 0x04000 /* 0x10000 */ +#define FBNIC_TCE_LSO_CTRL_TCPF_CLR_1ST CSR_GENMASK(8, 0) +#define FBNIC_TCE_LSO_CTRL_TCPF_CLR_MID CSR_GENMASK(17, 9) +#define FBNIC_TCE_LSO_CTRL_TCPF_CLR_END CSR_GENMASK(26, 18) +#define FBNIC_TCE_LSO_CTRL_IPID_MODE_INC CSR_BIT(27) + +#define FBNIC_TCE_CSO_CTRL 0x04001 /* 0x10004 */ +#define FBNIC_TCE_CSO_CTRL_TCP_ZERO_CSUM CSR_BIT(0) + +#define FBNIC_TCE_TXB_CTRL 0x04002 /* 0x10008 */ +#define FBNIC_TCE_TXB_CTRL_LOAD CSR_BIT(0) +#define FBNIC_TCE_TXB_CTRL_TCAM_ENABLE CSR_BIT(1) +#define FBNIC_TCE_TXB_CTRL_DISABLE CSR_BIT(2) + +#define FBNIC_TCE_TXB_ENQ_WRR_CTRL 0x04003 /* 0x1000c */ +#define FBNIC_TCE_TXB_ENQ_WRR_CTRL_WEIGHT0 CSR_GENMASK(7, 0) +#define FBNIC_TCE_TXB_ENQ_WRR_CTRL_WEIGHT1 CSR_GENMASK(15, 8) +#define FBNIC_TCE_TXB_ENQ_WRR_CTRL_WEIGHT2 CSR_GENMASK(23, 16) + +#define FBNIC_TCE_TXB_TEI_Q0_CTRL 0x04004 /* 0x10010 */ +#define FBNIC_TCE_TXB_TEI_Q1_CTRL 0x04005 /* 0x10014 */ +#define FBNIC_TCE_TXB_MC_Q_CTRL 0x04006 /* 0x10018 */ +#define FBNIC_TCE_TXB_RX_TEI_Q_CTRL 0x04007 /* 0x1001c */ +#define FBNIC_TCE_TXB_RX_BMC_Q_CTRL 0x04008 /* 0x10020 */ +#define FBNIC_TCE_TXB_Q_CTRL_START CSR_GENMASK(10, 0) +#define FBNIC_TCE_TXB_Q_CTRL_SIZE CSR_GENMASK(22, 11) + +#define FBNIC_TCE_TXB_TEI_DWRR_CTRL 0x04009 /* 0x10024 */ +#define FBNIC_TCE_TXB_TEI_DWRR_CTRL_QUANTUM0 CSR_GENMASK(7, 0) +#define FBNIC_TCE_TXB_TEI_DWRR_CTRL_QUANTUM1 CSR_GENMASK(15, 8) +#define FBNIC_TCE_TXB_NTWRK_DWRR_CTRL 0x0400a /* 0x10028 */ +#define FBNIC_TCE_TXB_NTWRK_DWRR_CTRL_QUANTUM0 CSR_GENMASK(7, 0) +#define FBNIC_TCE_TXB_NTWRK_DWRR_CTRL_QUANTUM1 CSR_GENMASK(15, 8) +#define FBNIC_TCE_TXB_NTWRK_DWRR_CTRL_QUANTUM2 CSR_GENMASK(23, 16) + +#define FBNIC_TCE_TXB_CLDR_CFG 0x0400b /* 0x1002c */ +#define FBNIC_TCE_TXB_CLDR_CFG_NUM_SLOT CSR_GENMASK(5, 0) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG(n) (0x0400c + (n)) /* 0x10030 + 4*n */ +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_CNT 16 +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_0_0 CSR_GENMASK(1, 0) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_0_1 CSR_GENMASK(3, 2) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_0_2 CSR_GENMASK(5, 4) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_0_3 CSR_GENMASK(7, 6) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_1_0 CSR_GENMASK(9, 8) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_1_1 CSR_GENMASK(11, 10) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_1_2 CSR_GENMASK(13, 12) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_1_3 CSR_GENMASK(15, 14) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_2_0 CSR_GENMASK(17, 16) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_2_1 CSR_GENMASK(19, 18) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_2_2 CSR_GENMASK(21, 20) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_2_3 CSR_GENMASK(23, 22) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_3_0 CSR_GENMASK(25, 24) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_3_1 CSR_GENMASK(27, 26) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_3_2 CSR_GENMASK(29, 28) +#define FBNIC_TCE_TXB_CLDR_SLOT_CFG_DEST_ID_3_3 CSR_GENMASK(31, 30) + +#define FBNIC_TCE_BMC_MAX_PKTSZ 0x0403a /* 0x100e8 */ +#define FBNIC_TCE_BMC_MAX_PKTSZ_TX CSR_GENMASK(13, 0) +#define FBNIC_TCE_BMC_MAX_PKTSZ_RX CSR_GENMASK(27, 14) +#define FBNIC_TCE_MC_MAX_PKTSZ 0x0403b /* 0x100ec */ +#define FBNIC_TCE_MC_MAX_PKTSZ_TMI CSR_GENMASK(13, 0) + +#define FBNIC_TCE_SOP_PROT_CTRL 0x0403c /* 0x100f0 */ +#define FBNIC_TCE_SOP_PROT_CTRL_TBI CSR_GENMASK(7, 0) +#define FBNIC_TCE_SOP_PROT_CTRL_TTI_FRM CSR_GENMASK(14, 8) +#define FBNIC_TCE_SOP_PROT_CTRL_TTI_CM CSR_GENMASK(18, 15) + +#define FBNIC_TCE_DROP_CTRL 0x0403d /* 0x100f4 */ +#define FBNIC_TCE_DROP_CTRL_TTI_CM_DROP_EN CSR_BIT(0) +#define FBNIC_TCE_DROP_CTRL_TTI_FRM_DROP_EN CSR_BIT(1) +#define FBNIC_TCE_DROP_CTRL_TTI_TBI_DROP_EN CSR_BIT(2) + +#define FBNIC_TCE_TXB_TX_BMC_Q_CTRL 0x0404B /* 0x1012c */ +#define FBNIC_TCE_TXB_BMC_DWRR_CTRL 0x0404C /* 0x10130 */ +#define FBNIC_TCE_TXB_BMC_DWRR_CTRL_QUANTUM0 CSR_GENMASK(7, 0) +#define FBNIC_TCE_TXB_BMC_DWRR_CTRL_QUANTUM1 CSR_GENMASK(15, 8) +#define FBNIC_TCE_TXB_TEI_DWRR_CTRL_EXT 0x0404D /* 0x10134 */ +#define FBNIC_TCE_TXB_NTWRK_DWRR_CTRL_EXT \ + 0x0404E /* 0x10138 */ +#define FBNIC_TCE_TXB_BMC_DWRR_CTRL_EXT 0x0404F /* 0x1013c */ +#define FBNIC_CSR_END_TCE 0x04050 /* CSR section delimiter */ + +/* TMI registers */ +#define FBNIC_CSR_START_TMI 0x04400 /* CSR section delimiter */ +#define FBNIC_TMI_SOP_PROT_CTRL 0x04400 /* 0x11000 */ +#define FBNIC_CSR_END_TMI 0x0443f /* CSR section delimiter */ +/* Rx Buffer Registers */ +#define FBNIC_CSR_START_RXB 0x08000 /* CSR section delimiter */ +enum { + FBNIC_RXB_FIFO_MC = 0, + /* Unused */ + /* Unused */ + FBNIC_RXB_FIFO_NET_TO_BMC = 3, + FBNIC_RXB_FIFO_HOST = 4, + /* Unused */ + FBNIC_RXB_FIFO_BMC_TO_HOST = 6, + /* Unused */ + FBNIC_RXB_FIFO_INDICES = 8 +}; + +#define FBNIC_RXB_CT_SIZE(n) (0x08000 + (n)) /* 0x20000 + 4*n */ +#define FBNIC_RXB_CT_SIZE_CNT 8 +#define FBNIC_RXB_CT_SIZE_HEADER CSR_GENMASK(5, 0) +#define FBNIC_RXB_CT_SIZE_PAYLOAD CSR_GENMASK(11, 6) +#define FBNIC_RXB_CT_SIZE_ENABLE CSR_BIT(12) +#define FBNIC_RXB_PAUSE_DROP_CTRL 0x08008 /* 0x20020 */ +#define FBNIC_RXB_PAUSE_DROP_CTRL_DROP_ENABLE CSR_GENMASK(7, 0) +#define FBNIC_RXB_PAUSE_DROP_CTRL_PAUSE_ENABLE CSR_GENMASK(15, 8) +#define FBNIC_RXB_PAUSE_DROP_CTRL_ECN_ENABLE CSR_GENMASK(23, 16) +#define FBNIC_RXB_PAUSE_DROP_CTRL_PS_ENABLE CSR_GENMASK(27, 24) +#define FBNIC_RXB_PAUSE_THLD(n) (0x08009 + (n)) /* 0x20024 + 4*n */ +#define FBNIC_RXB_PAUSE_THLD_CNT 8 +#define FBNIC_RXB_PAUSE_THLD_ON CSR_GENMASK(12, 0) +#define FBNIC_RXB_PAUSE_THLD_OFF CSR_GENMASK(25, 13) +#define FBNIC_RXB_DROP_THLD(n) (0x08011 + (n)) /* 0x20044 + 4*n */ +#define FBNIC_RXB_DROP_THLD_CNT 8 +#define FBNIC_RXB_DROP_THLD_ON CSR_GENMASK(12, 0) +#define FBNIC_RXB_DROP_THLD_OFF CSR_GENMASK(25, 13) +#define FBNIC_RXB_ECN_THLD(n) (0x0801e + (n)) /* 0x20078 + 4*n */ +#define FBNIC_RXB_ECN_THLD_CNT 8 +#define FBNIC_RXB_ECN_THLD_ON CSR_GENMASK(12, 0) +#define FBNIC_RXB_ECN_THLD_OFF CSR_GENMASK(25, 13) +#define FBNIC_RXB_PBUF_CFG(n) (0x08027 + (n)) /* 0x2009c + 4*n */ +#define FBNIC_RXB_PBUF_CFG_CNT 8 +#define FBNIC_RXB_PBUF_BASE_ADDR CSR_GENMASK(12, 0) +#define FBNIC_RXB_PBUF_SIZE CSR_GENMASK(21, 13) +#define FBNIC_RXB_DWRR_RDE_WEIGHT0 0x0802f /* 0x200bc */ +#define FBNIC_RXB_DWRR_RDE_WEIGHT0_QUANTUM0 CSR_GENMASK(7, 0) +#define FBNIC_RXB_DWRR_RDE_WEIGHT0_QUANTUM1 CSR_GENMASK(15, 8) +#define FBNIC_RXB_DWRR_RDE_WEIGHT0_QUANTUM2 CSR_GENMASK(23, 16) +#define FBNIC_RXB_DWRR_RDE_WEIGHT0_QUANTUM3 CSR_GENMASK(31, 24) +#define FBNIC_RXB_DWRR_RDE_WEIGHT1 0x08030 /* 0x200c0 */ +#define FBNIC_RXB_DWRR_RDE_WEIGHT1_QUANTUM4 CSR_GENMASK(7, 0) +#define FBNIC_RXB_DWRR_BMC_WEIGHT 0x08031 /* 0x200c4 */ +#define FBNIC_RXB_CLDR_PRIO_CFG(n) (0x8034 + (n)) /* 0x200d0 + 4*n */ +#define FBNIC_RXB_CLDR_PRIO_CFG_CNT 16 +#define FBNIC_RXB_ENDIAN_FCS 0x08044 /* 0x20110 */ +enum { + /* Unused */ + /* Unused */ + FBNIC_RXB_DEQUEUE_BMC = 2, + FBNIC_RXB_DEQUEUE_HOST = 3, + FBNIC_RXB_DEQUEUE_INDICES = 4 +}; + +#define FBNIC_RXB_PBUF_CREDIT(n) (0x08047 + (n)) /* 0x2011C + 4*n */ +#define FBNIC_RXB_PBUF_CREDIT_CNT 8 +#define FBNIC_RXB_PBUF_CREDIT_MASK CSR_GENMASK(13, 0) +#define FBNIC_RXB_INTF_CREDIT 0x0804f /* 0x2013C */ +#define FBNIC_RXB_INTF_CREDIT_MASK0 CSR_GENMASK(3, 0) +#define FBNIC_RXB_INTF_CREDIT_MASK1 CSR_GENMASK(7, 4) +#define FBNIC_RXB_INTF_CREDIT_MASK2 CSR_GENMASK(11, 8) +#define FBNIC_RXB_INTF_CREDIT_MASK3 CSR_GENMASK(15, 12) + +#define FBNIC_RXB_PAUSE_EVENT_CNT(n) (0x08053 + (n)) /* 0x2014c + 4*n */ +#define FBNIC_RXB_DROP_FRMS_STS(n) (0x08057 + (n)) /* 0x2015c + 4*n */ +#define FBNIC_RXB_DROP_BYTES_STS_L(n) \ + (0x08080 + 2 * (n)) /* 0x20200 + 8*n */ +#define FBNIC_RXB_DROP_BYTES_STS_H(n) \ + (0x08081 + 2 * (n)) /* 0x20204 + 8*n */ +#define FBNIC_RXB_TRUN_FRMS_STS(n) (0x08091 + (n)) /* 0x20244 + 4*n */ +#define FBNIC_RXB_TRUN_BYTES_STS_L(n) \ + (0x080c0 + 2 * (n)) /* 0x20300 + 8*n */ +#define FBNIC_RXB_TRUN_BYTES_STS_H(n) \ + (0x080c1 + 2 * (n)) /* 0x20304 + 8*n */ +#define FBNIC_RXB_TRANS_PAUSE_STS(n) (0x080d1 + (n)) /* 0x20344 + 4*n */ +#define FBNIC_RXB_TRANS_DROP_STS(n) (0x080d9 + (n)) /* 0x20364 + 4*n */ +#define FBNIC_RXB_TRANS_ECN_STS(n) (0x080e1 + (n)) /* 0x20384 + 4*n */ +enum { + FBNIC_RXB_ENQUEUE_NET = 0, + FBNIC_RXB_ENQUEUE_BMC = 1, + /* Unused */ + /* Unused */ + FBNIC_RXB_ENQUEUE_INDICES = 4 +}; + +#define FBNIC_RXB_DRBO_FRM_CNT_SRC(n) (0x080f9 + (n)) /* 0x203e4 + 4*n */ +#define FBNIC_RXB_DRBO_BYTE_CNT_SRC_L(n) \ + (0x080fd + (n)) /* 0x203f4 + 4*n */ +#define FBNIC_RXB_DRBO_BYTE_CNT_SRC_H(n) \ + (0x08101 + (n)) /* 0x20404 + 4*n */ +#define FBNIC_RXB_INTF_FRM_CNT_DST(n) (0x08105 + (n)) /* 0x20414 + 4*n */ +#define FBNIC_RXB_INTF_BYTE_CNT_DST_L(n) \ + (0x08109 + (n)) /* 0x20424 + 4*n */ +#define FBNIC_RXB_INTF_BYTE_CNT_DST_H(n) \ + (0x0810d + (n)) /* 0x20434 + 4*n */ +#define FBNIC_RXB_PBUF_FRM_CNT_DST(n) (0x08111 + (n)) /* 0x20444 + 4*n */ +#define FBNIC_RXB_PBUF_BYTE_CNT_DST_L(n) \ + (0x08115 + (n)) /* 0x20454 + 4*n */ +#define FBNIC_RXB_PBUF_BYTE_CNT_DST_H(n) \ + (0x08119 + (n)) /* 0x20464 + 4*n */ + +#define FBNIC_RXB_PBUF_FIFO_LEVEL(n) (0x0811d + (n)) /* 0x20474 + 4*n */ + +#define FBNIC_RXB_INTEGRITY_ERR(n) (0x0812f + (n)) /* 0x204bc + 4*n */ +#define FBNIC_RXB_MAC_ERR(n) (0x08133 + (n)) /* 0x204cc + 4*n */ +#define FBNIC_RXB_PARSER_ERR(n) (0x08137 + (n)) /* 0x204dc + 4*n */ +#define FBNIC_RXB_FRM_ERR(n) (0x0813b + (n)) /* 0x204ec + 4*n */ + +#define FBNIC_RXB_DWRR_RDE_WEIGHT0_EXT 0x08143 /* 0x2050c */ +#define FBNIC_RXB_DWRR_RDE_WEIGHT1_EXT 0x08144 /* 0x20510 */ +#define FBNIC_CSR_END_RXB 0x081b1 /* CSR section delimiter */ + +/* Rx Parser and Classifier Registers */ +#define FBNIC_CSR_START_RPC 0x08400 /* CSR section delimiter */ +#define FBNIC_RPC_RMI_CONFIG 0x08400 /* 0x21000 */ +#define FBNIC_RPC_RMI_CONFIG_OH_BYTES CSR_GENMASK(4, 0) +#define FBNIC_RPC_RMI_CONFIG_FCS_PRESENT CSR_BIT(8) +#define FBNIC_RPC_RMI_CONFIG_ENABLE CSR_BIT(12) +#define FBNIC_RPC_RMI_CONFIG_MTU CSR_GENMASK(31, 16) +#define FBNIC_CSR_END_RPC 0x0856b /* CSR section delimiter */ + +/* Fab Registers */ +#define FBNIC_CSR_START_FAB 0x0C000 /* CSR section delimiter */ +#define FBNIC_FAB_AXI4_AR_SPACER_2_CFG 0x0C005 /* 0x30014 */ +#define FBNIC_FAB_AXI4_AR_SPACER_MASK CSR_BIT(16) +#define FBNIC_FAB_AXI4_AR_SPACER_THREADSHOLD CSR_GENMASK(15, 0) +#define FBNIC_CSR_END_FAB 0x0C020 /* CSR section delimiter */ + +/* Master Registers */ +#define FBNIC_CSR_START_MASTER 0x0C400 /* CSR section delimiter */ +#define FBNIC_MASTER_SPARE_0 0x0C41B /* 0x3106c */ +#define FBNIC_CSR_END_MASTER 0x0C41E /* CSR section delimiter */ + +/* PUL User Registers */ +#define FBNIC_CSR_START_PUL_USER 0x31000 /* CSR section delimiter */ +#define FBNIC_PUL_OB_TLP_HDR_AW_CFG 0x3103d /* 0xc40f4 */ +#define FBNIC_PUL_OB_TLP_HDR_AW_CFG_BME CSR_BIT(18) +#define FBNIC_PUL_OB_TLP_HDR_AR_CFG 0x3103e /* 0xc40f8 */ +#define FBNIC_PUL_OB_TLP_HDR_AR_CFG_BME CSR_BIT(18) +#define FBNIC_CSR_END_PUL_USER 0x31080 /* CSR section delimiter */ + +#define FBNIC_MAX_QUEUES 128 + #endif /* _FBNIC_CSR_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c new file mode 100644 index 000000000000..dbbfdc649f37 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c @@ -0,0 +1,438 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#include +#include + +#include "fbnic.h" +#include "fbnic_mac.h" + +static void fbnic_init_readrq(struct fbnic_dev *fbd, unsigned int offset, + unsigned int cls, unsigned int readrq) +{ + u32 val = rd32(offset); + + /* The TDF_CTL masks are a superset of the RNI_RBP ones. So we can + * use them when setting either the TDE_CTF or RNI_RBP registers. + */ + val &= FBNIC_QM_TNI_TDF_CTL_MAX_OT | FBNIC_QM_TNI_TDF_CTL_MAX_OB; + + val |= FIELD_PREP(FBNIC_QM_TNI_TDF_CTL_MRRS, readrq) | + FIELD_PREP(FBNIC_QM_TNI_TDF_CTL_CLS, cls); + + wr32(offset, val); +} + +static void fbnic_init_mps(struct fbnic_dev *fbd, unsigned int offset, + unsigned int cls, unsigned int mps) +{ + u32 val = rd32(offset); + + /* Currently all MPS masks are identical so just use the first one */ + val &= ~(FBNIC_QM_TNI_TCM_CTL_MPS | FBNIC_QM_TNI_TCM_CTL_CLS); + + val |= FIELD_PREP(FBNIC_QM_TNI_TCM_CTL_MPS, mps) | + FIELD_PREP(FBNIC_QM_TNI_TCM_CTL_CLS, cls); + + wr32(offset, val); +} + +static void fbnic_mac_init_axi(struct fbnic_dev *fbd) +{ + bool override_1k = false; + int readrq, mps, cls; + + /* All of the values are based on being a power of 2 starting + * with 64 == 0. Therefore we can either divide by 64 in the + * case of constants, or just subtract 6 from the log2 of the value + * in order to get the value we will be programming into the + * registers. + */ + readrq = ilog2(fbd->readrq) - 6; + if (readrq > 3) + override_1k = true; + readrq = clamp(readrq, 0, 3); + + mps = ilog2(fbd->mps) - 6; + mps = clamp(mps, 0, 3); + + cls = ilog2(L1_CACHE_BYTES) - 6; + cls = clamp(cls, 0, 3); + + /* Configure Tx/Rx AXI Paths w/ Read Request and Max Payload sizes */ + fbnic_init_readrq(fbd, FBNIC_QM_TNI_TDF_CTL, cls, readrq); + fbnic_init_mps(fbd, FBNIC_QM_TNI_TCM_CTL, cls, mps); + + /* Configure QM TNI TDE: + * - Max outstanding AXI beats to 704(768 - 64) - guaranetees 8% of + * buffer capacity to descriptors. + * - Max outstanding transactions to 128 + */ + wr32(FBNIC_QM_TNI_TDE_CTL, + FIELD_PREP(FBNIC_QM_TNI_TDE_CTL_MRRS_1K, override_1k ? 1 : 0) | + FIELD_PREP(FBNIC_QM_TNI_TDE_CTL_MAX_OB, 704) | + FIELD_PREP(FBNIC_QM_TNI_TDE_CTL_MAX_OT, 128) | + FIELD_PREP(FBNIC_QM_TNI_TDE_CTL_MRRS, readrq) | + FIELD_PREP(FBNIC_QM_TNI_TDE_CTL_CLS, cls)); + + fbnic_init_readrq(fbd, FBNIC_QM_RNI_RBP_CTL, cls, readrq); + fbnic_init_mps(fbd, FBNIC_QM_RNI_RDE_CTL, cls, mps); + fbnic_init_mps(fbd, FBNIC_QM_RNI_RCM_CTL, cls, mps); + + /* Enable XALI AR/AW outbound */ + wr32(FBNIC_PUL_OB_TLP_HDR_AW_CFG, + FBNIC_PUL_OB_TLP_HDR_AW_CFG_BME); + wr32(FBNIC_PUL_OB_TLP_HDR_AR_CFG, + FBNIC_PUL_OB_TLP_HDR_AR_CFG_BME); +} + +static void fbnic_mac_init_qm(struct fbnic_dev *fbd) +{ + u32 clock_freq; + + /* Configure TSO behavior */ + fbnic_wr32(fbd, FBNIC_QM_TQS_CTL0, + FIELD_PREP(FBNIC_QM_TQS_CTL0_LSO_TS_MASK, + FBNIC_QM_TQS_CTL0_LSO_TS_LAST) | + FIELD_PREP(FBNIC_QM_TQS_CTL0_PREFETCH_THRESH, + FBNIC_QM_TQS_CTL0_PREFETCH_THRESH_MIN)); + + /* Limit EDT to INT_MAX as this is the limit of the EDT Qdisc */ + fbnic_wr32(fbd, FBNIC_QM_TQS_EDT_TS_RANGE, INT_MAX); + + /* Configure MTU + * Due to known HW issue we cannot set the MTU to within 16 octets + * of a 64 octet aligned boundary. So we will set the TQS_MTU(s) to + * MTU + 1. + */ + fbnic_wr32(fbd, FBNIC_QM_TQS_MTU_CTL0, FBNIC_MAX_JUMBO_FRAME_SIZE + 1); + fbnic_wr32(fbd, FBNIC_QM_TQS_MTU_CTL1, + FIELD_PREP(FBNIC_QM_TQS_MTU_CTL1_BULK, + FBNIC_MAX_JUMBO_FRAME_SIZE + 1)); + + clock_freq = FBNIC_CLOCK_FREQ; + + /* Be aggressive on the timings. We will have the interrupt + * threshold timer tick once every 1 usec and coalese writes for + * up to 80 usecs. + */ + fbnic_wr32(fbd, FBNIC_QM_TCQ_CTL0, + FIELD_PREP(FBNIC_QM_TCQ_CTL0_TICK_CYCLES, + clock_freq / 1000000) | + FIELD_PREP(FBNIC_QM_TCQ_CTL0_COAL_WAIT, + clock_freq / 12500)); + + /* We will have the interrupt threshold timer tick once every + * 1 usec and coalese writes for up to 2 usecs. + */ + fbnic_wr32(fbd, FBNIC_QM_RCQ_CTL0, + FIELD_PREP(FBNIC_QM_RCQ_CTL0_TICK_CYCLES, + clock_freq / 1000000) | + FIELD_PREP(FBNIC_QM_RCQ_CTL0_COAL_WAIT, + clock_freq / 500000)); + + /* Configure spacer control to 64 beats. */ + fbnic_wr32(fbd, FBNIC_FAB_AXI4_AR_SPACER_2_CFG, + FBNIC_FAB_AXI4_AR_SPACER_MASK | + FIELD_PREP(FBNIC_FAB_AXI4_AR_SPACER_THREADSHOLD, 2)); +} + +#define FBNIC_DROP_EN_MASK 0x7d +#define FBNIC_PAUSE_EN_MASK 0x14 +#define FBNIC_ECN_EN_MASK 0x10 + +struct fbnic_fifo_config { + unsigned int addr; + unsigned int size; +}; + +/* Rx FIFO Configuration + * The table consists of 8 entries, of which only 4 are currently used + * The starting addr is in units of 64B and the size is in 2KB units + * Below is the human readable version of the table defined below: + * Function Addr Size + * ---------------------------------- + * network to Host/BMC 384K 64K + * Unused + * Unused + * network to BMC 448K 32K + * network to Host 0 384K + * Unused + * BMC to Host 480K 32K + * Unused + */ +static const struct fbnic_fifo_config fifo_config[] = { + { .addr = 0x1800, .size = 0x20 }, /* network to Host/BMC */ + { }, /* not used */ + { }, /* not used */ + { .addr = 0x1c00, .size = 0x10 }, /* network to BMC */ + { .addr = 0x0000, .size = 0xc0 }, /* network to Host */ + { }, /* not used */ + { .addr = 0x1e00, .size = 0x10 }, /* BMC to Host */ + { } /* not used */ +}; + +static void fbnic_mac_init_rxb(struct fbnic_dev *fbd) +{ + bool rx_enable; + int i; + + rx_enable = !!(fbnic_rd32(fbd, FBNIC_RPC_RMI_CONFIG) & + FBNIC_RPC_RMI_CONFIG_ENABLE); + + for (i = 0; i < 8; i++) { + unsigned int size = fifo_config[i].size; + + /* If we are coming up on a system that already has the + * Rx data path enabled we don't need to reconfigure the + * FIFOs. Instead we can check to verify the values are + * large enough to meet our needs, and use the values to + * populate the flow control, ECN, and drop thresholds. + */ + if (rx_enable) { + size = FIELD_GET(FBNIC_RXB_PBUF_SIZE, + fbnic_rd32(fbd, + FBNIC_RXB_PBUF_CFG(i))); + if (size < fifo_config[i].size) + dev_warn(fbd->dev, + "fifo%d size of %d smaller than expected value of %d\n", + i, size << 11, + fifo_config[i].size << 11); + } else { + /* Program RXB Cuthrough */ + fbnic_wr32(fbd, FBNIC_RXB_CT_SIZE(i), + FIELD_PREP(FBNIC_RXB_CT_SIZE_HEADER, 4) | + FIELD_PREP(FBNIC_RXB_CT_SIZE_PAYLOAD, 2)); + + /* The granularity for the packet buffer size is 2KB + * granularity while the packet buffer base address is + * only 64B granularity + */ + fbnic_wr32(fbd, FBNIC_RXB_PBUF_CFG(i), + FIELD_PREP(FBNIC_RXB_PBUF_BASE_ADDR, + fifo_config[i].addr) | + FIELD_PREP(FBNIC_RXB_PBUF_SIZE, size)); + + /* The granularity for the credits is 64B. This is + * based on RXB_PBUF_SIZE * 32 + 4. + */ + fbnic_wr32(fbd, FBNIC_RXB_PBUF_CREDIT(i), + FIELD_PREP(FBNIC_RXB_PBUF_CREDIT_MASK, + size ? size * 32 + 4 : 0)); + } + + if (!size) + continue; + + /* Pause is size of FIFO with 56KB skid to start/stop */ + fbnic_wr32(fbd, FBNIC_RXB_PAUSE_THLD(i), + !(FBNIC_PAUSE_EN_MASK & (1u << i)) ? 0x1fff : + FIELD_PREP(FBNIC_RXB_PAUSE_THLD_ON, + size * 32 - 0x380) | + FIELD_PREP(FBNIC_RXB_PAUSE_THLD_OFF, 0x380)); + + /* Enable Drop when only one packet is left in the FIFO */ + fbnic_wr32(fbd, FBNIC_RXB_DROP_THLD(i), + !(FBNIC_DROP_EN_MASK & (1u << i)) ? 0x1fff : + FIELD_PREP(FBNIC_RXB_DROP_THLD_ON, + size * 32 - + FBNIC_MAX_JUMBO_FRAME_SIZE / 64) | + FIELD_PREP(FBNIC_RXB_DROP_THLD_OFF, + size * 32 - + FBNIC_MAX_JUMBO_FRAME_SIZE / 64)); + + /* Enable ECN bit when 1/4 of RXB is filled with at least + * 1 room for one full jumbo frame before setting ECN + */ + fbnic_wr32(fbd, FBNIC_RXB_ECN_THLD(i), + !(FBNIC_ECN_EN_MASK & (1u << i)) ? 0x1fff : + FIELD_PREP(FBNIC_RXB_ECN_THLD_ON, + max_t(unsigned int, + size * 32 / 4, + FBNIC_MAX_JUMBO_FRAME_SIZE / 64)) | + FIELD_PREP(FBNIC_RXB_ECN_THLD_OFF, + max_t(unsigned int, + size * 32 / 4, + FBNIC_MAX_JUMBO_FRAME_SIZE / 64))); + } + + /* For now only enable drop and ECN. We need to add driver/kernel + * interfaces for configuring pause. + */ + fbnic_wr32(fbd, FBNIC_RXB_PAUSE_DROP_CTRL, + FIELD_PREP(FBNIC_RXB_PAUSE_DROP_CTRL_DROP_ENABLE, + FBNIC_DROP_EN_MASK) | + FIELD_PREP(FBNIC_RXB_PAUSE_DROP_CTRL_ECN_ENABLE, + FBNIC_ECN_EN_MASK)); + + /* Program INTF credits */ + fbnic_wr32(fbd, FBNIC_RXB_INTF_CREDIT, + FBNIC_RXB_INTF_CREDIT_MASK0 | + FBNIC_RXB_INTF_CREDIT_MASK1 | + FBNIC_RXB_INTF_CREDIT_MASK2 | + FIELD_PREP(FBNIC_RXB_INTF_CREDIT_MASK3, 8)); + + /* Configure calendar slots. + * Rx: 0 - 62 RDE 1st, BMC 2nd + * 63 BMC 1st, RDE 2nd + */ + for (i = 0; i < 16; i++) { + u32 calendar_val = (i == 15) ? 0x1e1b1b1b : 0x1b1b1b1b; + + fbnic_wr32(fbd, FBNIC_RXB_CLDR_PRIO_CFG(i), calendar_val); + } + + /* Split the credits for the DRR up as follows: + * Quantum0: 8000 Network to Host + * Quantum1: 0 Not used + * Quantum2: 80 BMC to Host + * Quantum3: 0 Not used + * Quantum4: 8000 Multicast to Host and BMC + */ + fbnic_wr32(fbd, FBNIC_RXB_DWRR_RDE_WEIGHT0, + FIELD_PREP(FBNIC_RXB_DWRR_RDE_WEIGHT0_QUANTUM0, 0x40) | + FIELD_PREP(FBNIC_RXB_DWRR_RDE_WEIGHT0_QUANTUM2, 0x50)); + fbnic_wr32(fbd, FBNIC_RXB_DWRR_RDE_WEIGHT0_EXT, + FIELD_PREP(FBNIC_RXB_DWRR_RDE_WEIGHT0_QUANTUM0, 0x1f)); + fbnic_wr32(fbd, FBNIC_RXB_DWRR_RDE_WEIGHT1, + FIELD_PREP(FBNIC_RXB_DWRR_RDE_WEIGHT1_QUANTUM4, 0x40)); + fbnic_wr32(fbd, FBNIC_RXB_DWRR_RDE_WEIGHT1_EXT, + FIELD_PREP(FBNIC_RXB_DWRR_RDE_WEIGHT1_QUANTUM4, 0x1f)); + + /* Program RXB FCS Endian register */ + fbnic_wr32(fbd, FBNIC_RXB_ENDIAN_FCS, 0x0aaaaaa0); +} + +static void fbnic_mac_init_txb(struct fbnic_dev *fbd) +{ + int i; + + fbnic_wr32(fbd, FBNIC_TCE_TXB_CTRL, 0); + + /* Configure Tx QM Credits */ + fbnic_wr32(fbd, FBNIC_QM_TQS_CTL1, + FIELD_PREP(FBNIC_QM_TQS_CTL1_MC_MAX_CREDITS, 0x40) | + FIELD_PREP(FBNIC_QM_TQS_CTL1_BULK_MAX_CREDITS, 0x20)); + + /* Initialize internal Tx queues */ + fbnic_wr32(fbd, FBNIC_TCE_TXB_TEI_Q0_CTRL, 0); + fbnic_wr32(fbd, FBNIC_TCE_TXB_TEI_Q1_CTRL, 0); + fbnic_wr32(fbd, FBNIC_TCE_TXB_MC_Q_CTRL, + FIELD_PREP(FBNIC_TCE_TXB_Q_CTRL_SIZE, 0x400) | + FIELD_PREP(FBNIC_TCE_TXB_Q_CTRL_START, 0x000)); + fbnic_wr32(fbd, FBNIC_TCE_TXB_RX_TEI_Q_CTRL, 0); + fbnic_wr32(fbd, FBNIC_TCE_TXB_TX_BMC_Q_CTRL, + FIELD_PREP(FBNIC_TCE_TXB_Q_CTRL_SIZE, 0x200) | + FIELD_PREP(FBNIC_TCE_TXB_Q_CTRL_START, 0x400)); + fbnic_wr32(fbd, FBNIC_TCE_TXB_RX_BMC_Q_CTRL, + FIELD_PREP(FBNIC_TCE_TXB_Q_CTRL_SIZE, 0x200) | + FIELD_PREP(FBNIC_TCE_TXB_Q_CTRL_START, 0x600)); + + fbnic_wr32(fbd, FBNIC_TCE_LSO_CTRL, + FBNIC_TCE_LSO_CTRL_IPID_MODE_INC | + FIELD_PREP(FBNIC_TCE_LSO_CTRL_TCPF_CLR_1ST, TCPHDR_PSH | + TCPHDR_FIN) | + FIELD_PREP(FBNIC_TCE_LSO_CTRL_TCPF_CLR_MID, TCPHDR_PSH | + TCPHDR_CWR | + TCPHDR_FIN) | + FIELD_PREP(FBNIC_TCE_LSO_CTRL_TCPF_CLR_END, TCPHDR_CWR)); + fbnic_wr32(fbd, FBNIC_TCE_CSO_CTRL, 0); + + fbnic_wr32(fbd, FBNIC_TCE_BMC_MAX_PKTSZ, + FIELD_PREP(FBNIC_TCE_BMC_MAX_PKTSZ_TX, + FBNIC_MAX_JUMBO_FRAME_SIZE) | + FIELD_PREP(FBNIC_TCE_BMC_MAX_PKTSZ_RX, + FBNIC_MAX_JUMBO_FRAME_SIZE)); + fbnic_wr32(fbd, FBNIC_TCE_MC_MAX_PKTSZ, + FIELD_PREP(FBNIC_TCE_MC_MAX_PKTSZ_TMI, + FBNIC_MAX_JUMBO_FRAME_SIZE)); + + /* Enable Drops in Tx path, needed for FPGA only */ + fbnic_wr32(fbd, FBNIC_TCE_DROP_CTRL, + FBNIC_TCE_DROP_CTRL_TTI_CM_DROP_EN | + FBNIC_TCE_DROP_CTRL_TTI_FRM_DROP_EN | + FBNIC_TCE_DROP_CTRL_TTI_TBI_DROP_EN); + + /* Configure calendar slots. + * Tx: 0 - 62 TMI 1st, BMC 2nd + * 63 BMC 1st, TMI 2nd + */ + for (i = 0; i < 16; i++) { + u32 calendar_val = (i == 15) ? 0x1e1b1b1b : 0x1b1b1b1b; + + fbnic_wr32(fbd, FBNIC_TCE_TXB_CLDR_SLOT_CFG(i), calendar_val); + } + + /* Configure DWRR */ + fbnic_wr32(fbd, FBNIC_TCE_TXB_ENQ_WRR_CTRL, + FIELD_PREP(FBNIC_TCE_TXB_ENQ_WRR_CTRL_WEIGHT0, 0x64) | + FIELD_PREP(FBNIC_TCE_TXB_ENQ_WRR_CTRL_WEIGHT2, 0x04)); + fbnic_wr32(fbd, FBNIC_TCE_TXB_TEI_DWRR_CTRL, 0); + fbnic_wr32(fbd, FBNIC_TCE_TXB_TEI_DWRR_CTRL_EXT, 0); + fbnic_wr32(fbd, FBNIC_TCE_TXB_BMC_DWRR_CTRL, + FIELD_PREP(FBNIC_TCE_TXB_BMC_DWRR_CTRL_QUANTUM0, 0x50) | + FIELD_PREP(FBNIC_TCE_TXB_BMC_DWRR_CTRL_QUANTUM1, 0x82)); + fbnic_wr32(fbd, FBNIC_TCE_TXB_BMC_DWRR_CTRL_EXT, 0); + fbnic_wr32(fbd, FBNIC_TCE_TXB_NTWRK_DWRR_CTRL, + FIELD_PREP(FBNIC_TCE_TXB_NTWRK_DWRR_CTRL_QUANTUM1, 0x50) | + FIELD_PREP(FBNIC_TCE_TXB_NTWRK_DWRR_CTRL_QUANTUM2, 0x20)); + fbnic_wr32(fbd, FBNIC_TCE_TXB_NTWRK_DWRR_CTRL_EXT, + FIELD_PREP(FBNIC_TCE_TXB_NTWRK_DWRR_CTRL_QUANTUM2, 0x03)); + + /* Configure SOP protocol protection */ + fbnic_wr32(fbd, FBNIC_TCE_SOP_PROT_CTRL, + FIELD_PREP(FBNIC_TCE_SOP_PROT_CTRL_TBI, 0x78) | + FIELD_PREP(FBNIC_TCE_SOP_PROT_CTRL_TTI_FRM, 0x40) | + FIELD_PREP(FBNIC_TCE_SOP_PROT_CTRL_TTI_CM, 0x0c)); + + /* Conservative configuration on MAC interface Start of Packet + * protection FIFO. This sets the minimum depth of the FIFO before + * we start sending packets to the MAC measured in 64B units and + * up to 160 entries deep. + * + * For the ASIC the clock is fast enough that we will likely fill + * the SOP FIFO before the MAC can drain it. So just use a minimum + * value of 8. + * + * For the FPGA we have a clock that is about 3/5 of the MAC clock. + * As such we will need to account for adding more runway before + * transmitting the frames. + * SOP = (9230 / 64) * 2/5 + 8 + * SOP = 66 + */ + fbnic_wr32(fbd, FBNIC_TMI_SOP_PROT_CTRL, 8); + + wrfl(); + fbnic_wr32(fbd, FBNIC_TCE_TXB_CTRL, FBNIC_TCE_TXB_CTRL_TCAM_ENABLE | + FBNIC_TCE_TXB_CTRL_LOAD); +} + +static void fbnic_mac_init_regs(struct fbnic_dev *fbd) +{ + fbnic_mac_init_axi(fbd); + fbnic_mac_init_qm(fbd); + fbnic_mac_init_rxb(fbd); + fbnic_mac_init_txb(fbd); +} + +static const struct fbnic_mac fbnic_mac_asic = { + .init_regs = fbnic_mac_init_regs, +}; + +/** + * fbnic_mac_init - Assign a MAC type and initialize the fbnic device + * @fbd: Device pointer to device to initialize + * + * Returns 0 on success, negative on failure + * + * Initialize the MAC function pointers and initializes the MAC of + * the device. + **/ +int fbnic_mac_init(struct fbnic_dev *fbd) +{ + fbd->mac = &fbnic_mac_asic; + + fbd->mac->init_regs(fbd); + + return 0; +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.h b/drivers/net/ethernet/meta/fbnic/fbnic_mac.h new file mode 100644 index 000000000000..e78a92338a62 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#ifndef _FBNIC_MAC_H_ +#define _FBNIC_MAC_H_ + +#include + +struct fbnic_dev; + +#define FBNIC_MAX_JUMBO_FRAME_SIZE 9742 + +/* This structure defines the interface hooks for the MAC. The MAC hooks + * will be configured as a const struct provided with a set of function + * pointers. + * + * void (*init_regs)(struct fbnic_dev *fbd); + * Initialize MAC registers to enable Tx/Rx paths and FIFOs. + */ +struct fbnic_mac { + void (*init_regs)(struct fbnic_dev *fbd); +}; + +int fbnic_mac_init(struct fbnic_dev *fbd); +#endif /* _FBNIC_MAC_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index 596151396eac..c860516eb23a 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -29,6 +29,35 @@ static const struct pci_device_id fbnic_pci_tbl[] = { }; MODULE_DEVICE_TABLE(pci, fbnic_pci_tbl); +u32 fbnic_rd32(struct fbnic_dev *fbd, u32 reg) +{ + u32 __iomem *csr = READ_ONCE(fbd->uc_addr0); + u32 value; + + if (!csr) + return ~0U; + + value = readl(csr + reg); + + /* If any bits are 0 value should be valid */ + if (~value) + return value; + + /* All 1's may be valid if ZEROs register still works */ + if (reg != FBNIC_MASTER_SPARE_0 && ~readl(csr + FBNIC_MASTER_SPARE_0)) + return value; + + /* Hardware is giving us all 1's reads, assume it is gone */ + WRITE_ONCE(fbd->uc_addr0, NULL); + WRITE_ONCE(fbd->uc_addr4, NULL); + + dev_err(fbd->dev, + "Failed read (idx 0x%x AKA addr 0x%x), disabled CSR access, awaiting reset\n", + reg, reg << 2); + + return ~0U; +} + /** * fbnic_probe - Device Initialization Routine * @pdev: PCI device information struct @@ -88,6 +117,12 @@ static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) if (err) goto free_fbd; + err = fbnic_mac_init(fbd); + if (err) { + dev_err(&pdev->dev, "Failed to initialize MAC: %d\n", err); + goto free_irqs; + } + if (!fbd->dsn) { dev_warn(&pdev->dev, "Reading serial number failed\n"); goto init_failure_mode; @@ -101,6 +136,8 @@ static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) * firmware updates for fixes. */ return 0; +free_irqs: + fbnic_free_irqs(fbd); free_fbd: pci_disable_device(pdev); @@ -158,6 +195,8 @@ static int __fbnic_pm_resume(struct device *dev) if (err) goto err_invalidate_uc_addr; + fbd->mac->init_regs(fbd); + return 0; err_invalidate_uc_addr: WRITE_ONCE(fbd->uc_addr0, NULL); From patchwork Wed Apr 3 20:08:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616632 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B1C9155A59 for ; Wed, 3 Apr 2024 20:08:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174930; cv=none; b=qI+8mLeEkB3eq6K05S2uOSz1jxHS7cjULiqE/hVGJkXQO01A2p2oTF2ClJaecUMD1nA6nC7YT1G3wnOwt/5Dvi/7Zz/pOg35CO0FpdDdfR6sNT/r4Lf1EJPf5GGp0NJmZPGIROoElgJhyefBHIsu0vuOXDUmcGkWM9oPgWqnN5w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174930; c=relaxed/simple; bh=ElqTabw/dNil1VzItfqXvrUOELP1P0PUjwA5mTp90eQ=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Tlj/0ZXQNxU/R/jRZv8hVWPX94Nx9atcAQ8P++NmWghoVfdh1tzt8QEHsgMIgspGkFokBYQxBKQ4C55D4sz4lDSB5Blturr4R5MNlWw7X3H5M1xcHazPswmvukkAiGEdSS4a0qqnCr9+F6u0Ujk8Qn88IRNRptopo6uPyOCn0yk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KRXbzDJ/; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KRXbzDJ/" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-1e28be94d32so1771885ad.0 for ; Wed, 03 Apr 2024 13:08:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174928; x=1712779728; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=fNzJbytlUA4yyW2wGscrOAa5Tn1PXLSTmksBkutofwY=; b=KRXbzDJ/udh35739619kcE1vg8dHnt4CeDigvwM/qARC+XEkMPWkNgZV8XkFHaz1nq HrcX+u6+DJWrS10skv9w9wz3ha6knnN+3tgSnGBmdVOVv+lPwexuVMDMoBFykSuA2/WS Ob6I0bxTroPR9oY4Swh4eZH9W3BgLpueG2112iv94mLsJSGj1nSCWKYKudUe6bJ6zdq5 rRTVkbIiGSdbOdnF9G0DJxZFK9a2mvlEJUlfHG1YwCqlrGrD3a7wK8BhK2X9rrUZHrrS Da8w67g+wNpX3IvPOPzBXYPQo65iBjjadEVpzR4TxZzn+xf424CWLiMV6EUO8DcrRR/e R50Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174928; x=1712779728; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=fNzJbytlUA4yyW2wGscrOAa5Tn1PXLSTmksBkutofwY=; b=WWpdY/vCFmLeQvaZqFEyV/YB1isnOQCPGtcU0MTIxE4l1J2rVgv705DHd5XFfmWUFN TqLaK2BKzfZBxsujAGMnd/YLxyHBPA1zXi7Scd0sXJ8z4u/SDLrvx5dLV5yNYZ0KzON5 MFbieUzL5SGlgOttXS8mrddWhASGa0kyfQzGeSjH79wab04wC1Z0nNKNdl0VIZvCBW8T jMoQERYOnRlm4xoNQHhMwBHh5+/DsKTdb5I34XkfpB8xblFZhUR35cW7eAOn44X6MKhb +SlGofSYcI1yk9krk7tp++8sog61ESHhiZAJk4qrt3aBoqexaNpkjOBiocdfBt9Cq8ZV C9DQ== X-Gm-Message-State: AOJu0Yzkpmcl6mEej1XQnWvNRC5Vt75zLC9CFcN995Sk8LxElCVDJRBN A+fyLkpjjXf9KNth8WpsHgraVUUfqEFRxdVK4w+3leMtF4PrqqOl X-Google-Smtp-Source: AGHT+IEAfufLgeqUFRPuYhZVt8uodVN7DC3ybDvvQRS4JhB1HxYr7Iax3jMwSt9UNY8LCokO/fD/cQ== X-Received: by 2002:a17:902:f34d:b0:1e2:a31e:2062 with SMTP id q13-20020a170902f34d00b001e2a31e2062mr289441ple.53.1712174927540; Wed, 03 Apr 2024 13:08:47 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id i15-20020a170902c94f00b001dd02f4c2casm13774687pla.164.2024.04.03.13.08.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:08:46 -0700 (PDT) Subject: [net-next PATCH 05/15] eth: fbnic: add message parsing for FW messages From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:08:45 -0700 Message-ID: <171217492560.1598374.8486151971412546101.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Add FW message formatting and parsing. The TLV format should look very familiar to those familiar with netlink. Since we don't have to deal with backward compatibility we tweaked the format a little to make it easier to deal with, and more appropriate for tightly coupled interfaces like driver<>FW communication. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/Makefile | 3 drivers/net/ethernet/meta/fbnic/fbnic_tlv.c | 529 +++++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_tlv.h | 175 +++++++++ 3 files changed, 706 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_tlv.c create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_tlv.h diff --git a/drivers/net/ethernet/meta/fbnic/Makefile b/drivers/net/ethernet/meta/fbnic/Makefile index b8f4511440dc..0434ee0b3069 100644 --- a/drivers/net/ethernet/meta/fbnic/Makefile +++ b/drivers/net/ethernet/meta/fbnic/Makefile @@ -10,4 +10,5 @@ obj-$(CONFIG_FBNIC) += fbnic.o fbnic-y := fbnic_devlink.o \ fbnic_irq.o \ fbnic_mac.o \ - fbnic_pci.o + fbnic_pci.o \ + fbnic_tlv.o diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_tlv.c b/drivers/net/ethernet/meta/fbnic/fbnic_tlv.c new file mode 100644 index 000000000000..88d7b15fa798 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_tlv.c @@ -0,0 +1,529 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#include +#include +#include +#include +#include +#include + +#include "fbnic_tlv.h" + +/** + * fbnic_tlv_msg_alloc - Allocate page and initialize FW message header + * @msg_id: Identifier for new message we are starting + * + * Returns pointer to start of message, or NULL on failure. + * + * Allocates a page and initializes message header at start of page. + * Initial message size is 1 DWORD which is just the header. + **/ +struct fbnic_tlv_msg *fbnic_tlv_msg_alloc(u16 msg_id) +{ + struct fbnic_tlv_hdr hdr = { 0 }; + struct fbnic_tlv_msg *msg; + + msg = (struct fbnic_tlv_msg *)__get_free_page(GFP_KERNEL); + if (!msg) + return NULL; + + /* Start with zero filled header and then back fill with data */ + hdr.type = msg_id; + hdr.is_msg = 1; + hdr.len = cpu_to_le16(1); + + /* Copy header into start of message */ + msg->hdr = hdr; + + return msg; +} + +/** + * fbnic_tlv_attr_put_flag - Add flag value to message + * @msg: Message header we are adding flag attribute to + * @attr_id: ID of flag attribute we are adding to message + * + * Returns -ENOSPC if there is no room for the attribute. Otherwise 0. + * + * Adds a 1 DWORD flag attribute to the message. The presence of this + * attribute can be used as a boolean value indicating true, otherwise the + * value is considered false. + **/ +int fbnic_tlv_attr_put_flag(struct fbnic_tlv_msg *msg, const u16 attr_id) +{ + int attr_max_len = PAGE_SIZE - offset_in_page(msg) - sizeof(*msg); + struct fbnic_tlv_hdr hdr = { 0 }; + struct fbnic_tlv_msg *attr; + + attr_max_len -= le16_to_cpu(msg->hdr.len) * sizeof(u32); + if (attr_max_len < sizeof(*attr)) + return -ENOSPC; + + /* get header pointer and bump attr to start of data */ + attr = &msg[le16_to_cpu(msg->hdr.len)]; + + /* Record attribute type and size */ + hdr.type = attr_id; + hdr.len = cpu_to_le16(sizeof(hdr)); + + attr->hdr = hdr; + le16_add_cpu(&msg->hdr.len, + FBNIC_TLV_MSG_SIZE(le16_to_cpu(hdr.len))); + + return 0; +} + +/** + * fbnic_tlv_attr_put_value - Add data to message + * @msg: Message header we are adding flag attribute to + * @attr_id: ID of flag attribute we are adding to message + * @value: Pointer to data to be stored + * @len: Size of data to be stored. + * + * Returns -ENOSPC if there is no room for the attribute. Otherwise 0. + * + * Adds header and copies data pointed to by value into the message. The + * result is rounded up to the nearest DWORD for sizing so that the + * headers remain aligned. + * + * The assumption is that the value field is in a format where byte + * ordering can be guaranteed such as a byte array or a little endian + * format. + **/ +int fbnic_tlv_attr_put_value(struct fbnic_tlv_msg *msg, const u16 attr_id, + const void *value, const int len) +{ + int attr_max_len = PAGE_SIZE - offset_in_page(msg) - sizeof(*msg); + struct fbnic_tlv_hdr hdr = { 0 }; + struct fbnic_tlv_msg *attr; + + attr_max_len -= le16_to_cpu(msg->hdr.len) * sizeof(u32); + if (attr_max_len < sizeof(*attr) + len) + return -ENOSPC; + + /* get header pointer and bump attr to start of data */ + attr = &msg[le16_to_cpu(msg->hdr.len)]; + + /* Record attribute type and size */ + hdr.type = attr_id; + hdr.len = cpu_to_le16(sizeof(hdr) + len); + + /* Zero pad end of region to be written if we aren't aligned */ + if (len % sizeof(hdr)) + attr->value[len / sizeof(hdr)] = 0; + + /* Copy data over */ + memcpy(attr->value, value, len); + + attr->hdr = hdr; + le16_add_cpu(&msg->hdr.len, + FBNIC_TLV_MSG_SIZE(le16_to_cpu(hdr.len))); + + return 0; +} + +/** + * __fbnic_tlv_attr_put_int - Add integer to message + * @msg: Message header we are adding flag attribute to + * @attr_id: ID of flag attribute we are adding to message + * @value: Data to be stored + * @len: Size of data to be stored, either 4 or 8 bytes. + * + * Returns -ENOSPC if there is no room for the attribute. Otherwise 0. + * + * Adds header and copies data pointed to by value into the message. Will + * format the data as little endian. + **/ +int __fbnic_tlv_attr_put_int(struct fbnic_tlv_msg *msg, const u16 attr_id, + s64 value, const int len) +{ + __le64 le64_value = cpu_to_le64(value); + + return fbnic_tlv_attr_put_value(msg, attr_id, &le64_value, len); +} + +/** + * fbnic_tlv_attr_put_mac_addr - Add mac_addr to message + * @msg: Message header we are adding flag attribute to + * @attr_id: ID of flag attribute we are adding to message + * @mac_addr: Byte pointer to MAC address to be stored + * + * Returns -ENOSPC if there is no room for the attribute. Otherwise 0. + * + * Adds header and copies data pointed to by mac_addr into the message. Will + * copy the address raw so it will be in big endian with start of MAC + * address at start of attribute. + **/ +int fbnic_tlv_attr_put_mac_addr(struct fbnic_tlv_msg *msg, const u16 attr_id, + const u8 *mac_addr) +{ + return fbnic_tlv_attr_put_value(msg, attr_id, mac_addr, ETH_ALEN); +} + +/** + * fbnic_tlv_attr_put_string - Add string to message + * @msg: Message header we are adding flag attribute to + * @attr_id: ID of flag attribute we are adding to message + * @string: Byte pointer to null terminated string to be stored + * + * Returns -ENOSPC if there is no room for the attribute. Otherwise 0. + * + * Adds header and copies data pointed to by string into the message. Will + * copy the address raw so it will be in byte order. + **/ +int fbnic_tlv_attr_put_string(struct fbnic_tlv_msg *msg, u16 attr_id, + const char *string) +{ + int attr_max_len = PAGE_SIZE - sizeof(*msg); + int str_len = 1; + + /* The max length will be message minus existing message and new + * attribute header. Since the message is measured in DWORDs we have + * to multiply the size by 4. + * + * The string length doesn't include the \0 so we have to add one to + * the final value, so start with that as our initial value. + * + * We will verify if the string will fit in fbnic_tlv_attr_put_value() + */ + attr_max_len -= le16_to_cpu(msg->hdr.len) * sizeof(u32); + str_len += strnlen(string, attr_max_len); + + return fbnic_tlv_attr_put_value(msg, attr_id, string, str_len); +} + +/** + * fbnic_tlv_attr_get_unsigned - Retrieve unsigned value from result + * @attr: Attribute to retrieve data from + * + * Returns unsigned 64b value containing integer value + **/ +u64 fbnic_tlv_attr_get_unsigned(struct fbnic_tlv_msg *attr) +{ + __le64 le64_value = 0; + + memcpy(&le64_value, &attr->value[0], + le16_to_cpu(attr->hdr.len) - sizeof(*attr)); + + return le64_to_cpu(le64_value); +} + +/** + * fbnic_tlv_attr_get_signed - Retrieve signed value from result + * @attr: Attribute to retrieve data from + * + * Returns signed 64b value containing integer value + **/ +s64 fbnic_tlv_attr_get_signed(struct fbnic_tlv_msg *attr) +{ + int shift = (8 + sizeof(*attr) - le16_to_cpu(attr->hdr.len)) * 8; + __le64 le64_value = 0; + s64 value; + + /* Copy the value and adjust for byte ordering */ + memcpy(&le64_value, &attr->value[0], + le16_to_cpu(attr->hdr.len) - sizeof(*attr)); + value = le64_to_cpu(le64_value); + + /* Sign extend the return value by using a pair of shifts */ + return (value << shift) >> shift; +} + +/** + * fbnic_tlv_attr_get_string - Retrieve string value from result + * @attr: Attribute to retrieve data from + * @str: Pointer to an allocated string to store the data + * @max_size: The maximum size which can be in str + * + * Returns the size of the string read from firmware + **/ +size_t fbnic_tlv_attr_get_string(struct fbnic_tlv_msg *attr, char *str, + size_t max_size) +{ + max_size = min_t(size_t, max_size, + (le16_to_cpu(attr->hdr.len) * 4) - sizeof(*attr)); + memcpy(str, &attr->value, max_size); + + return max_size; +} + +/** + * fbnic_tlv_attr_nest_start - Add nested attribute header to message + * @msg: Message header we are adding flag attribute to + * @attr_id: ID of flag attribute we are adding to message + * + * Returns NULL if there is no room for the attribute. Otherwise a pointer + * to the new attribute header. + * + * New header length is stored initially in DWORDs. + **/ +struct fbnic_tlv_msg *fbnic_tlv_attr_nest_start(struct fbnic_tlv_msg *msg, + u16 attr_id) +{ + int attr_max_len = PAGE_SIZE - offset_in_page(msg) - sizeof(*msg); + struct fbnic_tlv_msg *attr = &msg[le16_to_cpu(msg->hdr.len)]; + struct fbnic_tlv_hdr hdr = { 0 }; + + /* Make sure we have space for at least the nest header plus one more */ + attr_max_len -= le16_to_cpu(msg->hdr.len) * sizeof(u32); + if (attr_max_len < sizeof(*attr) * 2) + return NULL; + + /* Record attribute type and size */ + hdr.type = attr_id; + + /* Add current message length to account for consumption within the + * page and leave it as a multiple of DWORDs, we will shift to + * bytes when we close it out. + */ + hdr.len = cpu_to_le16(1); + + attr->hdr = hdr; + + return attr; +} + +/** + * fbnic_tlv_attr_nest_stop - Close out nested attribute and add it to message + * @msg: Message header we are adding flag attribute to + * + * Closes out nested attribute, adds length to message, and then bumps + * length from DWORDs to bytes to match other attributes. + **/ +void fbnic_tlv_attr_nest_stop(struct fbnic_tlv_msg *msg) +{ + struct fbnic_tlv_msg *attr = &msg[le16_to_cpu(msg->hdr.len)]; + u16 len = le16_to_cpu(attr->hdr.len); + + /* Add attribute to message if there is more than just a header */ + if (len <= 1) + return; + + le16_add_cpu(&msg->hdr.len, len); + + /* convert from DWORDs to bytes */ + attr->hdr.len = cpu_to_le16(len * sizeof(u32)); +} + +static int +fbnic_tlv_attr_validate(struct fbnic_tlv_msg *attr, + const struct fbnic_tlv_index *tlv_index) +{ + u16 len = le16_to_cpu(attr->hdr.len) - sizeof(*attr); + u16 attr_id = attr->hdr.type; + __le32 *value = &attr->value[0]; + + if (attr->hdr.is_msg) + return -EINVAL; + + if (attr_id >= FBNIC_TLV_RESULTS_MAX) + return -EINVAL; + + while (tlv_index->id != attr_id) { + if (tlv_index->id == FBNIC_TLV_ATTR_ID_UNKNOWN) { + if (attr->hdr.cannot_ignore) + return -ENOENT; + return le16_to_cpu(attr->hdr.len); + } + + tlv_index++; + } + + if (offset_in_page(attr) + len > PAGE_SIZE - sizeof(*attr)) + return -E2BIG; + + switch (tlv_index->type) { + case FBNIC_TLV_STRING: + if (!len || len > tlv_index->len) + return -EINVAL; + if (((char *)value)[len - 1]) + return -EINVAL; + break; + case FBNIC_TLV_FLAG: + if (len) + return -EINVAL; + break; + case FBNIC_TLV_UNSIGNED: + case FBNIC_TLV_SIGNED: + if (tlv_index->len > sizeof(__le64)) + return -EINVAL; + fallthrough; + case FBNIC_TLV_BINARY: + if (!len || len > tlv_index->len) + return -EINVAL; + break; + case FBNIC_TLV_NESTED: + case FBNIC_TLV_ARRAY: + if (len % 4) + return -EINVAL; + break; + default: + return -EINVAL; + } + + return 0; +} + +/** + * fbnic_tlv_attr_parse_array - Parse array of attributes into results array + * @attr: Start of attributes in the message + * @len: Length of attributes in the message + * @results: Array of pointers to store the results of parsing + * @tlv_index: List of TLV attributes to be parsed from message + * @tlv_attr_id: Specific ID that is repeated in array + * @array_len: Number of results to store in results array + * + * Returns 0 on success, or negative value on error. + * + * Will take a list of attributes and a parser definition and will capture + * the results in the results array to have the data extracted later. + **/ +int fbnic_tlv_attr_parse_array(struct fbnic_tlv_msg *attr, int len, + struct fbnic_tlv_msg **results, + const struct fbnic_tlv_index *tlv_index, + u16 tlv_attr_id, size_t array_len) +{ + int i = 0; + + /* Initialize results table to NULL. */ + memset(results, 0, array_len * sizeof(results[0])); + + /* Nothing to parse if header was only thing there */ + if (!len) + return 0; + + /* Work through list of attributes, parsing them as necessary */ + while (len > 0) { + u16 attr_id = attr->hdr.type; + u16 attr_len; + int err; + + if (tlv_attr_id != attr_id) + return -EINVAL; + + /* stop parsing on full error */ + err = fbnic_tlv_attr_validate(attr, tlv_index); + if (err < 0) + return err; + + if (i >= array_len) + return -ENOSPC; + + results[i++] = attr; + + attr_len = FBNIC_TLV_MSG_SIZE(le16_to_cpu(attr->hdr.len)); + len -= attr_len; + attr += attr_len; + } + + return len == 0 ? 0 : -EINVAL; +} + +/** + * fbnic_tlv_attr_parse - Parse attributes into a list of attribute results + * @attr: Start of attributes in the message + * @len: Length of attributes in the message + * @results: Array of pointers to store the results of parsing + * @tlv_index: List of TLV attributes to be parsed from message + * + * Returns 0 on success, or negative value on error. + * + * Will take a list of attributes and a parser definition and will capture + * the results in the results array to have the data extracted later. + **/ +int fbnic_tlv_attr_parse(struct fbnic_tlv_msg *attr, int len, + struct fbnic_tlv_msg **results, + const struct fbnic_tlv_index *tlv_index) +{ + /* Initialize results table to NULL. */ + memset(results, 0, sizeof(results[0]) * FBNIC_TLV_RESULTS_MAX); + + /* Nothing to parse if header was only thing there */ + if (!len) + return 0; + + /* Work through list of attributes, parsing them as necessary */ + while (len > 0) { + int err = fbnic_tlv_attr_validate(attr, tlv_index); + u16 attr_id = attr->hdr.type; + u16 attr_len; + + /* stop parsing on full error */ + if (err < 0) + return err; + + /* Ignore results for unsupported values */ + if (!err) { + /* Do not overwrite existing entries */ + if (results[attr_id]) + return -EADDRINUSE; + + results[attr_id] = attr; + } + + attr_len = FBNIC_TLV_MSG_SIZE(le16_to_cpu(attr->hdr.len)); + len -= attr_len; + attr += attr_len; + } + + return len == 0 ? 0 : -EINVAL; +} + +/** + * fbnic_tlv_msg_parse - Parse message and process via predetermined functions + * @opaque: Value passed to parser function to enable driver access + * @msg: Message to be parsed. + * @parser: TLV message parser definition. + * + * Returns 0 on success, or negative value on error. + * + * Will take a message a number of message types via the attribute parsing + * definitions and function provided for the parser array. + **/ +int fbnic_tlv_msg_parse(void *opaque, struct fbnic_tlv_msg *msg, + const struct fbnic_tlv_parser *parser) +{ + struct fbnic_tlv_msg *results[FBNIC_TLV_RESULTS_MAX]; + u16 msg_id = msg->hdr.type; + int err; + + if (!msg->hdr.is_msg) + return -EINVAL; + + if (le16_to_cpu(msg->hdr.len) > PAGE_SIZE / sizeof(u32)) + return -E2BIG; + + while (parser->id != msg_id) { + if (parser->id == FBNIC_TLV_MSG_ID_UNKNOWN) + return -ENOENT; + parser++; + } + + err = fbnic_tlv_attr_parse(&msg[1], le16_to_cpu(msg->hdr.len) - 1, + results, parser->attr); + if (err) + return err; + + return parser->func(opaque, results); +} + +/** + * fbnic_tlv_parser_error - called if message doesn't match known type + * @opaque: (unused) + * @results: (unused) + * + * Returns -EBADMSG to indicate the message is an unsupported type + **/ +int fbnic_tlv_parser_error(void *opaque, struct fbnic_tlv_msg **results) +{ + return -EBADMSG; +} + +void fbnic_tlv_attr_addr_copy(u8 *dest, struct fbnic_tlv_msg *src) +{ + u8 *mac_addr; + + mac_addr = fbnic_tlv_attr_get_value_ptr(src); + memcpy(dest, mac_addr, ETH_ALEN); +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_tlv.h b/drivers/net/ethernet/meta/fbnic/fbnic_tlv.h new file mode 100644 index 000000000000..67300ab44353 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_tlv.h @@ -0,0 +1,175 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#ifndef _FBNIC_TLV_H_ +#define _FBNIC_TLV_H_ + +#include +#include +#include +#include + +#define FBNIC_TLV_MSG_ALIGN(len) ALIGN(len, sizeof(u32)) +#define FBNIC_TLV_MSG_SIZE(len) \ + (FBNIC_TLV_MSG_ALIGN(len) / sizeof(u32)) + +/* TLV Header Format + * 3 2 1 + * 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 + * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + * | Length |M|I|RSV| Type / ID | + * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + * + * The TLV header format described above will be used for transferring + * messages between the host and the firmware. To ensure byte ordering + * we have defined all fields as being little endian. + * Type/ID: Identifier for message and/or attribute + * RSV: Reserved field for future use, likely as additional flags + * I: cannot_ignore flag, identifies if unrecognized attribute can be ignored + * M: is_msg, indicates that this is the start of a new message + * Length: Total length of message in dwords including header + * or + * Total length of attribute in bytes including header + */ +struct fbnic_tlv_hdr { +#if defined(__LITTLE_ENDIAN_BITFIELD) + u16 type : 12; /* 0 .. 11 Type / ID */ + u16 rsvd : 2; /* 12 .. 13 Reserved for future use */ + u16 cannot_ignore : 1; /* 14 Attribute can be ignored */ + u16 is_msg : 1; /* 15 Header belongs to message */ +#elif defined(__BIG_ENDIAN_BITFIELD) + u16 is_msg : 1; /* 15 Header belongs to message */ + u16 cannot_ignore : 1; /* 14 Attribute can be ignored */ + u16 rsvd : 2; /* 13 .. 12 Reserved for future use */ + u16 type : 12; /* 11 .. 0 Type / ID */ +#else +#error "Missing defines from byteorder.h" +#endif + __le16 len; /* 16 .. 32 length including TLV header */ +}; + +#define FBNIC_TLV_RESULTS_MAX 32 + +struct fbnic_tlv_msg { + struct fbnic_tlv_hdr hdr; + __le32 value[]; +}; + +#define FBNIC_TLV_MSG_ID_UNKNOWN USHRT_MAX + +enum fbnic_tlv_type { + FBNIC_TLV_STRING, + FBNIC_TLV_FLAG, + FBNIC_TLV_UNSIGNED, + FBNIC_TLV_SIGNED, + FBNIC_TLV_BINARY, + FBNIC_TLV_NESTED, + FBNIC_TLV_ARRAY, + __FBNIC_TLV_MAX_TYPE +}; + +/* TLV Index + * Defines the relationship between the attribute IDs and their types. + * For each entry in the index there will be a size and type associated + * with it so that we can use this to parse the data and verify it matches + * the expected layout. + */ +struct fbnic_tlv_index { + u16 id; + u16 len; + enum fbnic_tlv_type type; +}; + +#define TLV_MAX_DATA (PAGE_SIZE - 512) +#define FBNIC_TLV_ATTR_ID_UNKNOWN USHRT_MAX +#define FBNIC_TLV_ATTR_STRING(id, len) { id, len, FBNIC_TLV_STRING } +#define FBNIC_TLV_ATTR_FLAG(id) { id, 0, FBNIC_TLV_FLAG } +#define FBNIC_TLV_ATTR_U32(id) { id, sizeof(u32), FBNIC_TLV_UNSIGNED } +#define FBNIC_TLV_ATTR_U64(id) { id, sizeof(u64), FBNIC_TLV_UNSIGNED } +#define FBNIC_TLV_ATTR_S32(id) { id, sizeof(s32), FBNIC_TLV_SIGNED } +#define FBNIC_TLV_ATTR_S64(id) { id, sizeof(s64), FBNIC_TLV_SIGNED } +#define FBNIC_TLV_ATTR_MAC_ADDR(id) { id, ETH_ALEN, FBNIC_TLV_BINARY } +#define FBNIC_TLV_ATTR_NESTED(id) { id, 0, FBNIC_TLV_NESTED } +#define FBNIC_TLV_ATTR_ARRAY(id) { id, 0, FBNIC_TLV_ARRAY } +#define FBNIC_TLV_ATTR_RAW_DATA(id) { id, TLV_MAX_DATA, FBNIC_TLV_BINARY } +#define FBNIC_TLV_ATTR_LAST { FBNIC_TLV_ATTR_ID_UNKNOWN, 0, 0 } + +struct fbnic_tlv_parser { + u16 id; + const struct fbnic_tlv_index *attr; + int (*func)(void *opaque, + struct fbnic_tlv_msg **results); +}; + +#define FBNIC_TLV_PARSER(id, attr, func) { FBNIC_TLV_MSG_ID_##id, attr, func } + +static inline void * +fbnic_tlv_attr_get_value_ptr(struct fbnic_tlv_msg *attr) +{ + return (void *)&attr->value[0]; +} + +static inline bool fbnic_tlv_attr_get_bool(struct fbnic_tlv_msg *attr) +{ + return !!attr; +} + +u64 fbnic_tlv_attr_get_unsigned(struct fbnic_tlv_msg *attr); +s64 fbnic_tlv_attr_get_signed(struct fbnic_tlv_msg *attr); +size_t fbnic_tlv_attr_get_string(struct fbnic_tlv_msg *attr, char *str, + size_t max_size); + +#define get_unsigned_result(id, location) \ +do { \ + struct fbnic_tlv_msg *result = results[id]; \ + if (result) \ + location = fbnic_tlv_attr_get_unsigned(result); \ +} while (0) + +#define get_signed_result(id, location) \ +do { \ + struct fbnic_tlv_msg *result = results[id]; \ + if (result) \ + location = fbnic_tlv_attr_get_signed(result); \ +} while (0) + +#define get_string_result(id, size, str, max_size) \ +do { \ + struct fbnic_tlv_msg *result = results[id]; \ + if (result) \ + size = fbnic_tlv_attr_get_string(result, str, max_size); \ +} while (0) + +#define get_bool(id) (!!(results[id])) + +struct fbnic_tlv_msg *fbnic_tlv_msg_alloc(u16 msg_id); +int fbnic_tlv_attr_put_flag(struct fbnic_tlv_msg *msg, const u16 attr_id); +int fbnic_tlv_attr_put_value(struct fbnic_tlv_msg *msg, const u16 attr_id, + const void *value, const int len); +int __fbnic_tlv_attr_put_int(struct fbnic_tlv_msg *msg, const u16 attr_id, + s64 value, const int len); +#define fbnic_tlv_attr_put_int(msg, attr_id, value) \ + __fbnic_tlv_attr_put_int(msg, attr_id, value, \ + FBNIC_TLV_MSG_ALIGN(sizeof(value))) +int fbnic_tlv_attr_put_mac_addr(struct fbnic_tlv_msg *msg, const u16 attr_id, + const u8 *mac_addr); +int fbnic_tlv_attr_put_string(struct fbnic_tlv_msg *msg, u16 attr_id, + const char *string); +struct fbnic_tlv_msg *fbnic_tlv_attr_nest_start(struct fbnic_tlv_msg *msg, + u16 attr_id); +void fbnic_tlv_attr_nest_stop(struct fbnic_tlv_msg *msg); +void fbnic_tlv_attr_addr_copy(u8 *dest, struct fbnic_tlv_msg *src); +int fbnic_tlv_attr_parse_array(struct fbnic_tlv_msg *attr, int len, + struct fbnic_tlv_msg **results, + const struct fbnic_tlv_index *tlv_index, + u16 tlv_attr_id, size_t array_len); +int fbnic_tlv_attr_parse(struct fbnic_tlv_msg *attr, int len, + struct fbnic_tlv_msg **results, + const struct fbnic_tlv_index *tlv_index); +int fbnic_tlv_msg_parse(void *opaque, struct fbnic_tlv_msg *msg, + const struct fbnic_tlv_parser *parser); +int fbnic_tlv_parser_error(void *opaque, struct fbnic_tlv_msg **results); + +#define FBNIC_TLV_MSG_ERROR \ + FBNIC_TLV_PARSER(UNKNOWN, NULL, fbnic_tlv_parser_error) +#endif /* _FBNIC_TLV_H_ */ From patchwork Wed Apr 3 20:08:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616633 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C26915573A for ; Wed, 3 Apr 2024 20:08:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174935; cv=none; b=ifWa792bdXLvV73M4uCbeDcb2woWD990do2bFJqLrY8+CQWj2IbNX0z2+W2j1gBzfpaCcs2BkfBUhlp3bnpGM3yivFpaEKxJvc5VlCtXAGb2MSPJYcTD5q6zdZYCNVHvAI/yeFkU22oiVaam8BCXbDhD8+dHV8Lc/fw76erT3QE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174935; c=relaxed/simple; bh=aCWgr28p0oQ0J8rn2vBYSlF0igU6IP4SMmQevucdS8s=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=f+MSSaic/5RuQ9FMRoePP+IJZfp9Mv+Zt9qXbP47l9spA6g27do8WYZS2FYTiYg5XXGtETo51VlsHax+9DwZ/8Slo1uTjrnSN9CbumElKj7EzhkuhISL2XLFEptAQAPAv0LkiLJsRjrQh2CIwDO7WQ0+XAtvVQIgXpa1lqwYNBY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IGsG9dfC; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IGsG9dfC" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1e27e174ccbso1342115ad.2 for ; Wed, 03 Apr 2024 13:08:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174932; x=1712779732; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=w8tib3aBwNU3wRDDA+SLtoeIxkeVZkZcFVThXsH6sdk=; b=IGsG9dfCoBGrVsOoy9fwY8qFFzvFN2VRXAArld71f/pQYWb8XmGHO9mOMWAC78nyWV l7+oCr2HF/swEVtnJcBf0ygnPnNH7qCYy2uwP0ROCySakiJGF8dPFGQzwJARSGteYuGp JTuXIHLkZgtrAKQOSiCBh9tBDzpHAjexXCpQsqne0+xylC/mtNnuW3F56CU8JIFHNjp4 arcYgtIM/EKvY4bu9MGgRGOsksQyvuUMYFAiKPZuI3k1JfaANuvrFnNpDIruPGEzjdQJ DMxcoQqpCTck+bVHx4Ja2tRlF01mD7LryJ64RqZp+crbXEZ+GQ3v3hbi4/IQn2OQSjjB 7wVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174932; x=1712779732; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=w8tib3aBwNU3wRDDA+SLtoeIxkeVZkZcFVThXsH6sdk=; b=UVfwSGqdZPJw3EoJQjVQ125iMwLSgtxDyR4oZ71ez4n99Zn4ukq5rtMvjhOWPuiDlz 7Tt+Ao7qU775r8xfReCAtILqEo5E9O4YTMhegv9dLrPOMKtvKgRie4KqCOW8L0XMhO2t 2422sImQd9+iKMD9HrCWWSdkfjO2neCfl4X/CADE+hB1U50AMR+VD4IXme59g6sRFmth ME1TDJdBzDXlQRLsPIHnpoci/H0Wyeh2TecbaHganIblzXfpUGJZdAdWZm89qRb7D5Ia QL3Awjl/grqGX89Sq/wMXi78TdtLikZNQQbQNWmhxci2207R67ecUsFwUu1Q5mnhzAKp DHbQ== X-Gm-Message-State: AOJu0YwIrhaPLAv1qyKSyMmZ7P8+Bhjbwoqm694NDDHfJ+9MZH8Sk7WB KeDwYKSh90cnyq/B8TVwekKdZKaLR+RAD67jEHZrwEIESxBjKICe X-Google-Smtp-Source: AGHT+IECdx7GkMOAuT35TvYKSxFYZBxJTFgWrdmtC63O3w/mUwkWobIazR6ISSuDDcdjBMzlCczRwA== X-Received: by 2002:a17:902:d4c5:b0:1e2:1234:dde7 with SMTP id o5-20020a170902d4c500b001e21234dde7mr308441plg.61.1712174932313; Wed, 03 Apr 2024 13:08:52 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id c4-20020a170902c2c400b001e0bcccc800sm2694513pla.35.2024.04.03.13.08.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:08:51 -0700 (PDT) Subject: [net-next PATCH 06/15] eth: fbnic: add FW communication mechanism From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:08:49 -0700 Message-ID: <171217492974.1598374.10545415888363821969.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Add a mechanism for sending messages to and receiving messages from the FW. The FW has fairly limited functionality, so the mechanism doesn't have to support high message rate. Use device mailbox registers to form two rings, one "to" and one "from" the device. The rings are just a convention between driver and FW, not a HW construct. We don't expect messages larger than 4k so use page-sized buffers. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/Makefile | 1 drivers/net/ethernet/meta/fbnic/fbnic.h | 18 + drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 79 ++++++ drivers/net/ethernet/meta/fbnic/fbnic_fw.c | 354 +++++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_fw.h | 26 ++ drivers/net/ethernet/meta/fbnic/fbnic_irq.c | 79 ++++++ drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 59 +++++ 7 files changed, 616 insertions(+) create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_fw.c create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_fw.h diff --git a/drivers/net/ethernet/meta/fbnic/Makefile b/drivers/net/ethernet/meta/fbnic/Makefile index 0434ee0b3069..7b63cd5b09d4 100644 --- a/drivers/net/ethernet/meta/fbnic/Makefile +++ b/drivers/net/ethernet/meta/fbnic/Makefile @@ -8,6 +8,7 @@ obj-$(CONFIG_FBNIC) += fbnic.o fbnic-y := fbnic_devlink.o \ + fbnic_fw.o \ fbnic_irq.o \ fbnic_mac.o \ fbnic_pci.o \ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic.h b/drivers/net/ethernet/meta/fbnic/fbnic.h index 63ec82a830cd..28d7f8a880da 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic.h @@ -7,6 +7,7 @@ #include #include "fbnic_csr.h" +#include "fbnic_fw.h" #include "fbnic_mac.h" struct fbnic_dev { @@ -16,8 +17,13 @@ struct fbnic_dev { u32 __iomem *uc_addr4; const struct fbnic_mac *mac; struct msix_entry *msix_entries; + unsigned int fw_msix_vector; unsigned short num_irqs; + struct fbnic_fw_mbx mbx[FBNIC_IPC_MBX_INDICES]; + /* Lock protecting Tx Mailbox queue to prevent possible races */ + spinlock_t fw_tx_lock; + u64 dsn; u32 mps; u32 readrq; @@ -29,6 +35,7 @@ struct fbnic_dev { * causes later. */ enum { + FBNIC_FW_MSIX_ENTRY, FBNIC_NON_NAPI_VECTORS }; @@ -62,6 +69,14 @@ fbnic_rmw32(struct fbnic_dev *fbd, u32 reg, u32 mask, u32 val) #define rd32(reg) fbnic_rd32(fbd, reg) #define wrfl() fbnic_rd32(fbd, FBNIC_MASTER_SPARE_0) +bool fbnic_fw_present(struct fbnic_dev *fbd); +u32 fbnic_fw_rd32(struct fbnic_dev *fbd, u32 reg); +void fbnic_fw_wr32(struct fbnic_dev *fbd, u32 reg, u32 val); + +#define fw_rd32(reg) fbnic_fw_rd32(fbd, reg) +#define fw_wr32(reg, val) fbnic_fw_wr32(fbd, reg, val) +#define fw_wrfl() fbnic_fw_rd32(fbd, FBNIC_FW_ZERO_REG) + extern char fbnic_driver_name[]; void fbnic_devlink_free(struct fbnic_dev *fbd); @@ -69,6 +84,9 @@ struct fbnic_dev *fbnic_devlink_alloc(struct pci_dev *pdev); void fbnic_devlink_register(struct fbnic_dev *fbd); void fbnic_devlink_unregister(struct fbnic_dev *fbd); +int fbnic_fw_enable_mbx(struct fbnic_dev *fbd); +void fbnic_fw_disable_mbx(struct fbnic_dev *fbd); + void fbnic_free_irqs(struct fbnic_dev *fbd); int fbnic_alloc_irqs(struct fbnic_dev *fbd); diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index eb37e5981e69..6755af1c948f 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -9,10 +9,60 @@ #define CSR_BIT(nr) (1u << (nr)) #define CSR_GENMASK(h, l) GENMASK(h, l) +#define DESC_BIT(nr) BIT_ULL(nr) +#define DESC_GENMASK(h, l) GENMASK_ULL(h, l) + #define PCI_DEVICE_ID_META_FBNIC_ASIC 0x0013 #define FBNIC_CLOCK_FREQ (600 * (1000 * 1000)) +/* Register Definitions + * + * The registers are laid as indexes into an le32 array. As such the actual + * address is 4 times the index value. Below each register is defined as 3 + * fields, name, index, and Address. + * + * Name Index Address + *************************************************************************/ +/* Interrupt Registers */ +#define FBNIC_CSR_START_INTR 0x00000 /* CSR section delimiter */ +#define FBNIC_INTR_STATUS(n) (0x00000 + (n)) /* 0x00000 + 4*n */ +#define FBNIC_INTR_STATUS_CNT 8 +#define FBNIC_INTR_MASK(n) (0x00008 + (n)) /* 0x00020 + 4*n */ +#define FBNIC_INTR_MASK_CNT 8 +#define FBNIC_INTR_SET(n) (0x00010 + (n)) /* 0x00040 + 4*n */ +#define FBNIC_INTR_SET_CNT 8 +#define FBNIC_INTR_CLEAR(n) (0x00018 + (n)) /* 0x00060 + 4*n */ +#define FBNIC_INTR_CLEAR_CNT 8 +#define FBNIC_INTR_SW_STATUS(n) (0x00020 + (n)) /* 0x00080 + 4*n */ +#define FBNIC_INTR_SW_STATUS_CNT 8 +#define FBNIC_INTR_SW_AC_MODE(n) (0x00028 + (n)) /* 0x000a0 + 4*n */ +#define FBNIC_INTR_SW_AC_MODE_CNT 8 +#define FBNIC_INTR_MASK_SET(n) (0x00030 + (n)) /* 0x000c0 + 4*n */ +#define FBNIC_INTR_MASK_SET_CNT 8 +#define FBNIC_INTR_MASK_CLEAR(n) (0x00038 + (n)) /* 0x000e0 + 4*n */ +#define FBNIC_INTR_MASK_CLEAR_CNT 8 +#define FBNIC_MAX_MSIX_VECS 256U +#define FBNIC_INTR_MSIX_CTRL(n) (0x00040 + (n)) /* 0x00100 + 4*n */ +#define FBNIC_INTR_MSIX_CTRL_VECTOR_MASK CSR_GENMASK(7, 0) +#define FBNIC_INTR_MSIX_CTRL_ENABLE CSR_BIT(31) + +#define FBNIC_CSR_END_INTR 0x0005f /* CSR section delimiter */ + +/* Interrupt MSIX Registers */ +#define FBNIC_CSR_START_INTR_CQ 0x00400 /* CSR section delimiter */ +#define FBNIC_INTR_CQ_REARM(n) \ + (0x00400 + ((n) * 4)) /* 0x01000 + 0x10*n */ +#define FBNIC_INTR_CQ_REARM_CNT 256 +#define FBNIC_INTR_CQ_REARM_RCQ_TIMEOUT CSR_GENMASK(13, 0) +#define FBNIC_INTR_CQ_REARM_RCQ_TIMEOUT_UPD_EN CSR_BIT(14) +#define FBNIC_INTR_CQ_REARM_TCQ_TIMEOUT CSR_GENMASK(28, 15) +#define FBNIC_INTR_CQ_REARM_TCQ_TIMEOUT_UPD_EN CSR_BIT(29) +#define FBNIC_INTR_CQ_REARM_INTR_RELOAD CSR_BIT(30) +#define FBNIC_INTR_CQ_REARM_INTR_UNMASK CSR_BIT(31) + +#define FBNIC_CSR_END_INTR_CQ 0x007fe /* CSR section delimiter */ + /* Global QM Tx registers */ #define FBNIC_CSR_START_QM_TX 0x00800 /* CSR section delimiter */ #define FBNIC_QM_TWQ_DEFAULT_META_L 0x00818 /* 0x02060 */ @@ -318,4 +368,33 @@ enum { #define FBNIC_MAX_QUEUES 128 +/* BAR 4 CSRs */ + +/* The IPC mailbox consists of 32 mailboxes, with each mailbox consisting + * of 32 4 byte registers. We will use 2 registers per descriptor so the + * length of the mailbox is reduced to 16. + * + * Currently we use an offset of 0x6000 on BAR4 for the mailbox so we just + * have to do the math and determine the offset based on the mailbox + * direction and index inside that mailbox. + */ +#define FBNIC_IPC_MBX_DESC_LEN 16 +#define FBNIC_IPC_MBX(mbx_idx, desc_idx) \ + ((((mbx_idx) * FBNIC_IPC_MBX_DESC_LEN + (desc_idx)) * 2) + 0x6000) + +/* Use first register in mailbox to flush writes */ +#define FBNIC_FW_ZERO_REG FBNIC_IPC_MBX(0, 0) + +enum { + FBNIC_IPC_MBX_RX_IDX, + FBNIC_IPC_MBX_TX_IDX, + FBNIC_IPC_MBX_INDICES, +}; + +#define FBNIC_IPC_MBX_DESC_LEN_MASK DESC_GENMASK(63, 48) +#define FBNIC_IPC_MBX_DESC_EOM DESC_BIT(46) +#define FBNIC_IPC_MBX_DESC_ADDR_MASK DESC_GENMASK(45, 3) +#define FBNIC_IPC_MBX_DESC_FW_CMPL DESC_BIT(1) +#define FBNIC_IPC_MBX_DESC_HOST_CMPL DESC_BIT(0) + #endif /* _FBNIC_CSR_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c new file mode 100644 index 000000000000..71647044aa23 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c @@ -0,0 +1,354 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#include +#include +#include +#include +#include +#include + +#include "fbnic.h" +#include "fbnic_tlv.h" + +static void __fbnic_mbx_wr_desc(struct fbnic_dev *fbd, int mbx_idx, + int desc_idx, u64 desc) +{ + fw_wr32(FBNIC_IPC_MBX(mbx_idx, desc_idx) + 1, upper_32_bits(desc)); + fw_wrfl(); + fw_wr32(FBNIC_IPC_MBX(mbx_idx, desc_idx), lower_32_bits(desc)); +} + +static u64 __fbnic_mbx_rd_desc(struct fbnic_dev *fbd, int mbx_idx, int desc_idx) +{ + u64 ret_val; + + ret_val = fw_rd32(FBNIC_IPC_MBX(mbx_idx, desc_idx)); + ret_val += (u64)fw_rd32(FBNIC_IPC_MBX(mbx_idx, desc_idx) + 1) << 32; + + return ret_val; +} + +static void fbnic_mbx_init_desc_ring(struct fbnic_dev *fbd, int mbx_idx) +{ + int desc_idx; + + /* Initialize first descriptor to all 0s. Doing this gives us a + * solid stop for the firmware to hit when it is done looping + * through the ring. + */ + __fbnic_mbx_wr_desc(fbd, mbx_idx, 0, 0); + + fw_wrfl(); + + /* We then fill the rest of the ring starting at the end and moving + * back toward descriptor 0 with skip descriptors that have no + * length nor address, and tell the firmware that they can skip + * them and just move past them to the one we initialized to 0. + */ + for (desc_idx = FBNIC_IPC_MBX_DESC_LEN; --desc_idx;) { + __fbnic_mbx_wr_desc(fbd, mbx_idx, desc_idx, + FBNIC_IPC_MBX_DESC_FW_CMPL | + FBNIC_IPC_MBX_DESC_HOST_CMPL); + fw_wrfl(); + } +} + +void fbnic_mbx_init(struct fbnic_dev *fbd) +{ + int i; + + /* Initialize lock to protect Tx ring */ + spin_lock_init(&fbd->fw_tx_lock); + + /* reinitialize mailbox memory */ + for (i = 0; i < FBNIC_IPC_MBX_INDICES; i++) + memset(&fbd->mbx[i], 0, sizeof(struct fbnic_fw_mbx)); + + /* Clear any stale causes in vector 0 as that is used for doorbell */ + wr32(FBNIC_INTR_CLEAR(0), 1u << FBNIC_FW_MSIX_ENTRY); + + for (i = 0; i < FBNIC_IPC_MBX_INDICES; i++) + fbnic_mbx_init_desc_ring(fbd, i); +} + +static int fbnic_mbx_map_msg(struct fbnic_dev *fbd, int mbx_idx, + struct fbnic_tlv_msg *msg, u16 length, u8 eom) +{ + struct fbnic_fw_mbx *mbx = &fbd->mbx[mbx_idx]; + u8 tail = mbx->tail; + dma_addr_t addr; + int direction; + + if (!mbx->ready || !fbnic_fw_present(fbd)) + return -ENODEV; + + direction = (mbx_idx == FBNIC_IPC_MBX_RX_IDX) ? DMA_FROM_DEVICE : + DMA_TO_DEVICE; + + if (mbx->head == ((tail + 1) % FBNIC_IPC_MBX_DESC_LEN)) + return -EBUSY; + + addr = dma_map_single(fbd->dev, msg, PAGE_SIZE, direction); + if (dma_mapping_error(fbd->dev, addr)) { + free_page((unsigned long)msg); + + return -ENOSPC; + } + + mbx->buf_info[tail].msg = msg; + mbx->buf_info[tail].addr = addr; + + mbx->tail = (tail + 1) % FBNIC_IPC_MBX_DESC_LEN; + + fw_wr32(FBNIC_IPC_MBX(mbx_idx, mbx->tail), 0); + + __fbnic_mbx_wr_desc(fbd, mbx_idx, tail, + FIELD_PREP(FBNIC_IPC_MBX_DESC_LEN_MASK, length) | + (addr & FBNIC_IPC_MBX_DESC_ADDR_MASK) | + (eom ? FBNIC_IPC_MBX_DESC_EOM : 0) | + FBNIC_IPC_MBX_DESC_HOST_CMPL); + + return 0; +} + +static void fbnic_mbx_unmap_and_free_msg(struct fbnic_dev *fbd, int mbx_idx, + int desc_idx) +{ + struct fbnic_fw_mbx *mbx = &fbd->mbx[mbx_idx]; + int direction; + + if (!mbx->buf_info[desc_idx].msg) + return; + + direction = (mbx_idx == FBNIC_IPC_MBX_RX_IDX) ? DMA_FROM_DEVICE : + DMA_TO_DEVICE; + dma_unmap_single(fbd->dev, mbx->buf_info[desc_idx].addr, + PAGE_SIZE, direction); + + free_page((unsigned long)mbx->buf_info[desc_idx].msg); + mbx->buf_info[desc_idx].msg = NULL; +} + +static void fbnic_mbx_clean_desc_ring(struct fbnic_dev *fbd, int mbx_idx) +{ + int i; + + fbnic_mbx_init_desc_ring(fbd, mbx_idx); + + for (i = FBNIC_IPC_MBX_DESC_LEN; i--;) + fbnic_mbx_unmap_and_free_msg(fbd, mbx_idx, i); +} + +void fbnic_mbx_clean(struct fbnic_dev *fbd) +{ + int i; + + for (i = 0; i < FBNIC_IPC_MBX_INDICES; i++) + fbnic_mbx_clean_desc_ring(fbd, i); +} + +#define FBNIC_MBX_MAX_PAGE_SIZE FIELD_MAX(FBNIC_IPC_MBX_DESC_LEN_MASK) +#define FBNIC_RX_PAGE_SIZE min_t(int, PAGE_SIZE, FBNIC_MBX_MAX_PAGE_SIZE) + +static int fbnic_mbx_alloc_rx_msgs(struct fbnic_dev *fbd) +{ + struct fbnic_fw_mbx *rx_mbx = &fbd->mbx[FBNIC_IPC_MBX_RX_IDX]; + u8 tail = rx_mbx->tail, head = rx_mbx->head, count; + int err = 0; + + /* Do nothing if mailbox is not ready, or we already have pages on + * the ring that can be used by the firmware + */ + if (!rx_mbx->ready) + return -ENODEV; + + /* Fill all but 1 unused descriptors in the Rx queue. */ + count = (head - tail - 1) % FBNIC_IPC_MBX_DESC_LEN; + while (!err && count--) { + struct fbnic_tlv_msg *msg; + + msg = (struct fbnic_tlv_msg *)__get_free_page(GFP_ATOMIC | + __GFP_NOWARN); + if (!msg) { + err = -ENOMEM; + break; + } + + err = fbnic_mbx_map_msg(fbd, FBNIC_IPC_MBX_RX_IDX, msg, + FBNIC_RX_PAGE_SIZE, 0); + if (err) + free_page((unsigned long)msg); + } + + return err; +} + +static void fbnic_mbx_process_tx_msgs(struct fbnic_dev *fbd) +{ + struct fbnic_fw_mbx *tx_mbx = &fbd->mbx[FBNIC_IPC_MBX_TX_IDX]; + u8 head = tx_mbx->head; + u64 desc; + + while (head != tx_mbx->tail) { + desc = __fbnic_mbx_rd_desc(fbd, FBNIC_IPC_MBX_TX_IDX, head); + if (!(desc & FBNIC_IPC_MBX_DESC_FW_CMPL)) + break; + + fbnic_mbx_unmap_and_free_msg(fbd, FBNIC_IPC_MBX_TX_IDX, head); + + head++; + head %= FBNIC_IPC_MBX_DESC_LEN; + } + + /* record head for next interrupt */ + tx_mbx->head = head; +} + +static void fbnic_mbx_postinit_desc_ring(struct fbnic_dev *fbd, int mbx_idx) +{ + struct fbnic_fw_mbx *mbx = &fbd->mbx[mbx_idx]; + + /* This is a one time init, so just exit if it is completed */ + if (mbx->ready) + return; + + mbx->ready = true; + + switch (mbx_idx) { + case FBNIC_IPC_MBX_RX_IDX: + /* Make sure we have a page for the FW to write to */ + fbnic_mbx_alloc_rx_msgs(fbd); + break; + } +} + +static void fbnic_mbx_postinit(struct fbnic_dev *fbd) +{ + int i; + + /* We only need to do this on the first interrupt following init. + * this primes the mailbox so that we will have cleared all the + * skip descriptors. + */ + if (!(rd32(FBNIC_INTR_STATUS(0)) & (1u << FBNIC_FW_MSIX_ENTRY))) + return; + + wr32(FBNIC_INTR_CLEAR(0), 1u << FBNIC_FW_MSIX_ENTRY); + + for (i = 0; i < FBNIC_IPC_MBX_INDICES; i++) + fbnic_mbx_postinit_desc_ring(fbd, i); +} + +static const struct fbnic_tlv_parser fbnic_fw_tlv_parser[] = { + FBNIC_TLV_MSG_ERROR +}; + +static void fbnic_mbx_process_rx_msgs(struct fbnic_dev *fbd) +{ + struct fbnic_fw_mbx *rx_mbx = &fbd->mbx[FBNIC_IPC_MBX_RX_IDX]; + u8 head = rx_mbx->head; + u64 desc, length; + + while (head != rx_mbx->tail) { + struct fbnic_tlv_msg *msg; + int err; + + desc = __fbnic_mbx_rd_desc(fbd, FBNIC_IPC_MBX_RX_IDX, head); + if (!(desc & FBNIC_IPC_MBX_DESC_FW_CMPL)) + break; + + dma_unmap_single(fbd->dev, rx_mbx->buf_info[head].addr, + PAGE_SIZE, DMA_FROM_DEVICE); + + msg = rx_mbx->buf_info[head].msg; + + length = FIELD_GET(FBNIC_IPC_MBX_DESC_LEN_MASK, desc); + + if (le16_to_cpu(msg->hdr.len) * sizeof(u32) > length) + dev_warn(fbd->dev, "Mailbox message length mismatch\n"); + + /* If parsing fails dump conents of message to dmesg */ + err = fbnic_tlv_msg_parse(fbd, msg, fbnic_fw_tlv_parser); + if (err) { + dev_warn(fbd->dev, "Unable to process message: %d\n", + err); + print_hex_dump(KERN_WARNING, "fbnic:", + DUMP_PREFIX_OFFSET, 16, 2, + msg, length, true); + } + + dev_dbg(fbd->dev, "Parsed msg type %d\n", msg->hdr.type); + + free_page((unsigned long)rx_mbx->buf_info[head].msg); + rx_mbx->buf_info[head].msg = NULL; + + head++; + head %= FBNIC_IPC_MBX_DESC_LEN; + } + + /* record head for next interrupt */ + rx_mbx->head = head; + + /* Make sure we have at least one page for the FW to write to */ + fbnic_mbx_alloc_rx_msgs(fbd); +} + +void fbnic_mbx_poll(struct fbnic_dev *fbd) +{ + fbnic_mbx_postinit(fbd); + + fbnic_mbx_process_tx_msgs(fbd); + fbnic_mbx_process_rx_msgs(fbd); +} + +int fbnic_mbx_poll_tx_ready(struct fbnic_dev *fbd) +{ + struct fbnic_fw_mbx *tx_mbx; + int attempts = 50; + + /* Immediate fail if BAR4 isn't there */ + if (!fbnic_fw_present(fbd)) + return -ENODEV; + + tx_mbx = &fbd->mbx[FBNIC_IPC_MBX_TX_IDX]; + while (!tx_mbx->ready && --attempts) { + msleep(200); + fbnic_mbx_poll(fbd); + } + + return attempts ? 0 : -ETIMEDOUT; +} + +void fbnic_mbx_flush_tx(struct fbnic_dev *fbd) +{ + struct fbnic_fw_mbx *tx_mbx; + int attempts = 50; + u8 count = 0; + + /* Nothing to do if there is no mailbox */ + if (!fbnic_fw_present(fbd)) + return; + + /* Record current Rx stats */ + tx_mbx = &fbd->mbx[FBNIC_IPC_MBX_TX_IDX]; + + /* Nothing to do if mailbox never got to ready */ + if (!tx_mbx->ready) + return; + + /* give firmware time to process packet, + * we will wait up to 10 seconds which is 50 waits of 200ms. + */ + do { + u8 head = tx_mbx->head; + + if (head == tx_mbx->tail) + break; + + msleep(200); + fbnic_mbx_process_tx_msgs(fbd); + + count += (tx_mbx->head - head) % FBNIC_IPC_MBX_DESC_LEN; + } while (count < FBNIC_IPC_MBX_DESC_LEN && --attempts); +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h new file mode 100644 index 000000000000..c143079f881c --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#ifndef _FBNIC_FW_H_ +#define _FBNIC_FW_H_ + +#include + +struct fbnic_dev; +struct fbnic_tlv_msg; + +struct fbnic_fw_mbx { + u8 ready, head, tail; + struct { + struct fbnic_tlv_msg *msg; + dma_addr_t addr; + } buf_info[FBNIC_IPC_MBX_DESC_LEN]; +}; + +void fbnic_mbx_init(struct fbnic_dev *fbd); +void fbnic_mbx_clean(struct fbnic_dev *fbd); +void fbnic_mbx_poll(struct fbnic_dev *fbd); +int fbnic_mbx_poll_tx_ready(struct fbnic_dev *fbd); +void fbnic_mbx_flush_tx(struct fbnic_dev *fbd); + +#endif /* _FBNIC_FW_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_irq.c b/drivers/net/ethernet/meta/fbnic/fbnic_irq.c index d2fdc51704b9..d5b08910577c 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_irq.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_irq.c @@ -6,10 +6,88 @@ #include "fbnic.h" +static irqreturn_t fbnic_fw_msix_intr(int __always_unused irq, void *data) +{ + struct fbnic_dev *fbd = (struct fbnic_dev *)data; + + fbnic_mbx_poll(fbd); + + wr32(FBNIC_INTR_MASK_CLEAR(0), 1u << FBNIC_FW_MSIX_ENTRY); + + return IRQ_HANDLED; +} + +/** + * fbnic_fw_enable_mbx - Configure and initialize Firmware Mailbox + * @fbd: Pointer to device to initialize + * + * This function will initialize the firmware mailbox rings, enable the IRQ + * and initialize the communication between the Firmware and the host. The + * firmware is expected to respond to the initialization by sending an + * interrupt essentially notifying the host that it has seen the + * initialization and is now synced up. + **/ +int fbnic_fw_enable_mbx(struct fbnic_dev *fbd) +{ + u32 vector; + int err; + + vector = fbd->fw_msix_vector; + + /* Request the IRQ for MAC link vector. + * Map MAC cause to it, and unmask it + */ + err = request_threaded_irq(vector, NULL, &fbnic_fw_msix_intr, 0, + dev_name(fbd->dev), fbd); + if (err) + return err; + + /* Initialize mailbox and attempt to poll it into ready state */ + fbnic_mbx_init(fbd); + err = fbnic_mbx_poll_tx_ready(fbd); + if (err) + dev_warn(fbd->dev, "FW mailbox did not enter ready state\n"); + + /* Enable interrupts */ + wr32(FBNIC_INTR_SW_AC_MODE(0), ~(1u << FBNIC_FW_MSIX_ENTRY)); + wr32(FBNIC_INTR_MASK_CLEAR(0), 1u << FBNIC_FW_MSIX_ENTRY); + + return 0; +} + +/** + * fbnic_fw_disable_mbx - Disable mailbox and place it in standby state + * @fbd: Pointer to device to disable + * + * This function will disable the mailbox interrupt, free any messages still + * in the mailbox and place it into a standby state. The firmware is + * expected to see the update and assume that the host is in the reset state. + **/ +void fbnic_fw_disable_mbx(struct fbnic_dev *fbd) +{ + /* Disable interrupt and free vector */ + wr32(FBNIC_INTR_MASK_SET(0), 1u << FBNIC_FW_MSIX_ENTRY); + + /* Re-enable auto-clear for the mailbox register */ + wr32(FBNIC_INTR_SW_AC_MODE(0), ~0); + + /* Free the vector */ + free_irq(fbd->fw_msix_vector, fbd); + + /* Make sure disabling logs message is sent, must be done here to + * avoid risk of completing without a running interrupt. + */ + fbnic_mbx_flush_tx(fbd); + + /* Flush any remaining entries */ + fbnic_mbx_clean(fbd); +} + void fbnic_free_irqs(struct fbnic_dev *fbd) { struct pci_dev *pdev = to_pci_dev(fbd->dev); + fbd->fw_msix_vector = 0; fbd->num_irqs = 0; pci_disable_msix(pdev); @@ -48,5 +126,6 @@ int fbnic_alloc_irqs(struct fbnic_dev *fbd) fbd->msix_entries = msix_entries; fbd->num_irqs = num_irqs; + fbd->fw_msix_vector = msix_entries[FBNIC_FW_MSIX_ENTRY].vector; return 0; } diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index c860516eb23a..1873fba9c6c2 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -58,6 +58,48 @@ u32 fbnic_rd32(struct fbnic_dev *fbd, u32 reg) return ~0U; } +bool fbnic_fw_present(struct fbnic_dev *fbd) +{ + return !!READ_ONCE(fbd->uc_addr4); +} + +void fbnic_fw_wr32(struct fbnic_dev *fbd, u32 reg, u32 val) +{ + u32 __iomem *csr = READ_ONCE(fbd->uc_addr4); + + if (csr) + writel(val, csr + reg); +} + +u32 fbnic_fw_rd32(struct fbnic_dev *fbd, u32 reg) +{ + u32 __iomem *csr = READ_ONCE(fbd->uc_addr4); + u32 value; + + if (!csr) + return ~0U; + + value = readl(csr + reg); + + /* If any bits are 0 value should be valid */ + if (~value) + return value; + + /* All 1's may be valid if ZEROs register still works */ + if (reg != FBNIC_FW_ZERO_REG && ~readl(csr + FBNIC_FW_ZERO_REG)) + return value; + + /* Hardware is giving us all 1's reads, assume it is gone */ + WRITE_ONCE(fbd->uc_addr0, NULL); + WRITE_ONCE(fbd->uc_addr4, NULL); + + dev_err(fbd->dev, + "Failed read (idx 0x%x AKA addr 0x%x), disabled CSR access, awaiting reset\n", + reg, reg << 2); + + return ~0U; +} + /** * fbnic_probe - Device Initialization Routine * @pdev: PCI device information struct @@ -123,6 +165,13 @@ static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto free_irqs; } + err = fbnic_fw_enable_mbx(fbd); + if (err) { + dev_err(&pdev->dev, + "Firmware mailbox initialization failure\n"); + goto free_irqs; + } + if (!fbd->dsn) { dev_warn(&pdev->dev, "Reading serial number failed\n"); goto init_failure_mode; @@ -159,6 +208,7 @@ static void fbnic_remove(struct pci_dev *pdev) { struct fbnic_dev *fbd = pci_get_drvdata(pdev); + fbnic_fw_disable_mbx(fbd); fbnic_free_irqs(fbd); fbnic_devlink_unregister(fbd); @@ -169,6 +219,8 @@ static int fbnic_pm_suspend(struct device *dev) { struct fbnic_dev *fbd = dev_get_drvdata(dev); + fbnic_fw_disable_mbx(fbd); + /* Free the IRQs so they aren't trying to occupy sleeping CPUs */ fbnic_free_irqs(fbd); @@ -197,7 +249,14 @@ static int __fbnic_pm_resume(struct device *dev) fbd->mac->init_regs(fbd); + /* Reenable mailbox */ + err = fbnic_fw_enable_mbx(fbd); + if (err) + goto err_free_irqs; + return 0; +err_free_irqs: + fbnic_free_irqs(fbd); err_invalidate_uc_addr: WRITE_ONCE(fbd->uc_addr0, NULL); WRITE_ONCE(fbd->uc_addr4, NULL); From patchwork Wed Apr 3 20:08:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616634 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-oa1-f52.google.com (mail-oa1-f52.google.com [209.85.160.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6864A15573A for ; Wed, 3 Apr 2024 20:08:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174939; cv=none; b=KtzP8bzH1zbUSMW5v9ntwcJsQUl8/mXF4EwgvPkhQj+MQKTtdRgwNUzAK1x+Qo7b7d4vo258n3mFj9jqv6Spf8cQ8lkmNj6zpQv8pDZt6q1oAo9iKdfjbyNMgRgAV8z5/m2EDO+45kndXjUaA/oOpCCjQjpt4v+jQwY9GncEpH8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174939; c=relaxed/simple; bh=+MiZ/conZwuq9mIbKp9aVgasZUzHiuNm1Cv3jKeM5kU=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=T2EAigtv1ZEJJF9r2mgsDC2hF2TGQU9h1l4KWsL0jn4DZ+K0x/anOAJROY4X9hvAxJMEg9qv0CqjbizTl/PT63wDtR3WBXu2k0RAV/vhmN/WMiuxpE6+ocxWsrm2S/MOLas+p83Hxxs+B6c591z2/rbz/un+wTbjtpX2F6pGLeg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CUpZtA+3; arc=none smtp.client-ip=209.85.160.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CUpZtA+3" Received: by mail-oa1-f52.google.com with SMTP id 586e51a60fabf-22a1e72f683so144740fac.0 for ; Wed, 03 Apr 2024 13:08:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174936; x=1712779736; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=Jd4oLTcIMj85oQuy4D5v9aVoLssERA4xqr6Nv9/G37Q=; b=CUpZtA+3VwGGQmuhiM044DT/+VFg4C5iYBEnzl28HPJ7BBZfgN1kf8p8sR7GR9ni+8 RNNjpPKZqjvFUTiF2Y7L04xYp+EH8s15wD61u7GkHmFK4CJlT3NSxGqJ9Lojof0GCxLd U1nVikoagq5rZzdjAPL9XBCuqqyVOYHnJqCLfrWVNLlvk3aHhfLwfgsh9T2IAIXtqEFH hR8Ux/Ra8VElHLS6MqNU/8/5ThZQBric2l+gJgwVIF+3z+dsV55E7aD8uEApNfWEAbgG lxpprL4wjHEJI0jGHs2fLx2uhGnil9ZFYXcI1xCV3lyV/BJlh703bUvWaMBuNvtLIZuc ucNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174936; x=1712779736; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Jd4oLTcIMj85oQuy4D5v9aVoLssERA4xqr6Nv9/G37Q=; b=jcixbYRx0a5FEKbntKBcctUxqQBDQ5O4KRqCUp0aNNsIF+uitbhhjHeRBBILIBfzAU sn+K8UYd72o+CGjYJbCvwlVUwImvDSsP92swEp97Yic8/yxMIKtcksr2z36hqSSJNZt9 aMcPpzKKUegWQJ1cOjKbFUr/ChJF6XgDizyWeasvzhhr812Y9qmfm2Idgv8Eo7ImL+Mj Pn1MPxMV9GD8pCt2qO9+4BrFfg9eWrnJ7qxG2g/Wnk3hNbOC8Iu34s9O5ytxsE5ymw6B MsxYYDJpD4tuRu6+8wv5geRWGJEo+BtG44TYif2ameJspUv8oxpNKITcChacPN0ZwxXm K/1w== X-Gm-Message-State: AOJu0Yw0xrqyZAjKWP4O7MqP2eFip0ROLiCg9P9PXEZmimDzpsJYSBDW gOvCN/vO5DQUtPYtS0nMoLmIkG2oxQo73uMGmwAwJHcRVjr1NvxT X-Google-Smtp-Source: AGHT+IHyqLrnJm+5VUsm7ScDJcLs7SrJfAL/B60uAKUwFidygFvcnNKJ4QPPXpk0/hSpWUHlEPMAfA== X-Received: by 2002:a05:6870:1590:b0:221:3ba2:fff0 with SMTP id j16-20020a056870159000b002213ba2fff0mr312659oab.45.1712174936291; Wed, 03 Apr 2024 13:08:56 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id x9-20020aa79a49000000b006e647716b6esm12604184pfj.149.2024.04.03.13.08.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:08:55 -0700 (PDT) Subject: [net-next PATCH 07/15] eth: fbnic: allocate a netdevice and napi vectors with queues From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:08:54 -0700 Message-ID: <171217493453.1598374.2269514228508217276.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Allocate a netdev and figure out basics like how many queues we need, MAC address, MTU bounds. Kick off a service task to do various periodic things like health checking. The service task only runs when device is open. We have four levels of objects here: - ring - a HW ring with head / tail pointers, - triad - two submission and one completion ring, - NAPI - NAPI, with one IRQ and any number of Rx and Tx triads, - Netdev - The ultimate container of the rings and napi vectors. The "triad" is the only less-than-usual construct. On Rx we have two "free buffer" submission rings, one for packet headers and one for packet data. On Tx we have separate rings for XDP Tx and normal Tx. So we ended up with ring triplets in both directions. We keep NAPIs on a local list, even though core already maintains a list. Later on having a separate list will matter for live reconfig. We introduce the list already, the churn would not be worth it. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/Makefile | 4 drivers/net/ethernet/meta/fbnic/fbnic.h | 13 + drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 26 ++ drivers/net/ethernet/meta/fbnic/fbnic_irq.c | 2 drivers/net/ethernet/meta/fbnic/fbnic_netdev.c | 180 ++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_netdev.h | 37 +++ drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 141 +++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 268 ++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_txrx.h | 55 +++++ 9 files changed, 724 insertions(+), 2 deletions(-) create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_netdev.c create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_netdev.h create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_txrx.h diff --git a/drivers/net/ethernet/meta/fbnic/Makefile b/drivers/net/ethernet/meta/fbnic/Makefile index 7b63cd5b09d4..f2ea90e0c14f 100644 --- a/drivers/net/ethernet/meta/fbnic/Makefile +++ b/drivers/net/ethernet/meta/fbnic/Makefile @@ -11,5 +11,7 @@ fbnic-y := fbnic_devlink.o \ fbnic_fw.o \ fbnic_irq.o \ fbnic_mac.o \ + fbnic_netdev.o \ fbnic_pci.o \ - fbnic_tlv.o + fbnic_tlv.o \ + fbnic_txrx.o diff --git a/drivers/net/ethernet/meta/fbnic/fbnic.h b/drivers/net/ethernet/meta/fbnic/fbnic.h index 28d7f8a880da..92a36959547c 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic.h @@ -5,6 +5,7 @@ #define _FBNIC_H_ #include +#include #include "fbnic_csr.h" #include "fbnic_fw.h" @@ -12,6 +13,7 @@ struct fbnic_dev { struct device *dev; + struct net_device *netdev; u32 __iomem *uc_addr0; u32 __iomem *uc_addr4; @@ -20,6 +22,8 @@ struct fbnic_dev { unsigned int fw_msix_vector; unsigned short num_irqs; + struct delayed_work service_task; + struct fbnic_fw_mbx mbx[FBNIC_IPC_MBX_INDICES]; /* Lock protecting Tx Mailbox queue to prevent possible races */ spinlock_t fw_tx_lock; @@ -27,6 +31,9 @@ struct fbnic_dev { u64 dsn; u32 mps; u32 readrq; + + /* Number of TCQs/RCQs available on hardware */ + u16 max_num_queues; }; /* Reserve entry 0 in the MSI-X "others" array until we have filled all @@ -77,6 +84,11 @@ void fbnic_fw_wr32(struct fbnic_dev *fbd, u32 reg, u32 val); #define fw_wr32(reg, val) fbnic_fw_wr32(fbd, reg, val) #define fw_wrfl() fbnic_fw_rd32(fbd, FBNIC_FW_ZERO_REG) +static inline bool fbnic_init_failure(struct fbnic_dev *fbd) +{ + return !fbd->netdev; +} + extern char fbnic_driver_name[]; void fbnic_devlink_free(struct fbnic_dev *fbd); @@ -95,6 +107,7 @@ enum fbnic_boards { }; struct fbnic_info { + unsigned int max_num_queues; unsigned int bar_mask; }; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index 6755af1c948f..39980974e21f 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -366,7 +366,33 @@ enum { #define FBNIC_PUL_OB_TLP_HDR_AR_CFG_BME CSR_BIT(18) #define FBNIC_CSR_END_PUL_USER 0x31080 /* CSR section delimiter */ +/* Queue Registers + * + * The queue register offsets are specific for a given queue grouping. So to + * find the actual register offset it is necessary to combine FBNIC_QUEUE(n) + * with the register to get the actual register offset like so: + * FBNIC_QUEUE_TWQ0_CTL(n) == FBNIC_QUEUE(n) + FBNIC_QUEUE_TWQ0_CTL + */ +#define FBNIC_CSR_START_QUEUE 0x40000 /* CSR section delimiter */ +#define FBNIC_QUEUE_STRIDE 0x400 /* 0x1000 */ +#define FBNIC_QUEUE(n)\ + (0x40000 + FBNIC_QUEUE_STRIDE * (n)) /* 0x100000 + 0x1000*n */ + +#define FBNIC_QUEUE_TWQ0_TAIL 0x002 /* 0x008 */ +#define FBNIC_QUEUE_TWQ1_TAIL 0x003 /* 0x00c */ + +/* Tx Completion Queue Registers */ +#define FBNIC_QUEUE_TCQ_HEAD 0x081 /* 0x204 */ + +/* Rx Completion Queue Registers */ +#define FBNIC_QUEUE_RCQ_HEAD 0x201 /* 0x804 */ + +/* Rx Buffer Descriptor Queue Registers */ +#define FBNIC_QUEUE_BDQ_HPQ_TAIL 0x241 /* 0x904 */ +#define FBNIC_QUEUE_BDQ_PPQ_TAIL 0x242 /* 0x908 */ + #define FBNIC_MAX_QUEUES 128 +#define FBNIC_CSR_END_QUEUE (0x40000 + 0x400 * FBNIC_MAX_QUEUES - 1) /* BAR 4 CSRs */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_irq.c b/drivers/net/ethernet/meta/fbnic/fbnic_irq.c index d5b08910577c..a20070683f48 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_irq.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_irq.c @@ -5,6 +5,7 @@ #include #include "fbnic.h" +#include "fbnic_txrx.h" static irqreturn_t fbnic_fw_msix_intr(int __always_unused irq, void *data) { @@ -103,6 +104,7 @@ int fbnic_alloc_irqs(struct fbnic_dev *fbd) struct msix_entry *msix_entries; int i, num_irqs; + wanted_irqs += min_t(unsigned int, num_online_cpus(), FBNIC_MAX_RXQS); msix_entries = kcalloc(wanted_irqs, sizeof(*msix_entries), GFP_KERNEL); if (!msix_entries) return -ENOMEM; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c new file mode 100644 index 000000000000..82c21bcb9c3f --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c @@ -0,0 +1,180 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#include +#include +#include + +#include "fbnic.h" +#include "fbnic_netdev.h" +#include "fbnic_txrx.h" + +int __fbnic_open(struct fbnic_net *fbn) +{ + int err; + + err = fbnic_alloc_napi_vectors(fbn); + if (err) + return err; + + err = netif_set_real_num_tx_queues(fbn->netdev, + fbn->num_tx_queues); + if (err) + goto free_resources; + + err = netif_set_real_num_rx_queues(fbn->netdev, + fbn->num_rx_queues); + if (err) + goto free_resources; + + return 0; +free_resources: + fbnic_free_napi_vectors(fbn); + return err; +} + +static int fbnic_open(struct net_device *netdev) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + int err; + + err = __fbnic_open(fbn); + if (!err) + fbnic_up(fbn); + + return err; +} + +static int fbnic_stop(struct net_device *netdev) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + + fbnic_down(fbn); + + fbnic_free_napi_vectors(fbn); + + return 0; +} + +static const struct net_device_ops fbnic_netdev_ops = { + .ndo_open = fbnic_open, + .ndo_stop = fbnic_stop, + .ndo_validate_addr = eth_validate_addr, + .ndo_start_xmit = fbnic_xmit_frame, +}; + +void fbnic_reset_queues(struct fbnic_net *fbn, + unsigned int tx, unsigned int rx) +{ + struct fbnic_dev *fbd = fbn->fbd; + unsigned int max_napis; + + max_napis = fbd->num_irqs - FBNIC_NON_NAPI_VECTORS; + + tx = min(tx, max_napis); + fbn->num_tx_queues = tx; + + rx = min(rx, max_napis); + fbn->num_rx_queues = rx; + + fbn->num_napi = max(tx, rx); +} + +/** + * fbnic_netdev_alloc - Allocate a netdev and associate with fbnic + * @fbd: Driver specific structure to associate netdev with + * + * Allocate and initialize the netdev and netdev private structure. Bind + * together the hardware, netdev, and pci data structures. + **/ +struct net_device *fbnic_netdev_alloc(struct fbnic_dev *fbd) +{ + struct net_device *netdev; + struct fbnic_net *fbn; + int default_queues; + + netdev = alloc_etherdev_mq(sizeof(*fbn), FBNIC_MAX_RXQS); + if (!netdev) + return NULL; + + SET_NETDEV_DEV(netdev, fbd->dev); + fbd->netdev = netdev; + + netdev->netdev_ops = &fbnic_netdev_ops; + + fbn = netdev_priv(netdev); + fbn->netdev = netdev; + fbn->fbd = fbd; + INIT_LIST_HEAD(&fbn->napis); + + default_queues = netif_get_num_default_rss_queues(); + if (default_queues > fbd->max_num_queues) + default_queues = fbd->max_num_queues; + + fbnic_reset_queues(fbn, default_queues, default_queues); + + netdev->min_mtu = IPV6_MIN_MTU; + netdev->max_mtu = FBNIC_MAX_JUMBO_FRAME_SIZE - ETH_HLEN; + + netif_carrier_off(netdev); + + netif_tx_stop_all_queues(netdev); + + return netdev; +} + +/** + * fbnic_netdev_free - Free the netdev associate with fbnic + * @fbd: Driver specific structure to free netdev from + * + * Allocate and initialize the netdev and netdev private structure. Bind + * together the hardware, netdev, and pci data structures. + **/ +void fbnic_netdev_free(struct fbnic_dev *fbd) +{ + free_netdev(fbd->netdev); + fbd->netdev = NULL; +} + +static int fbnic_dsn_to_mac_addr(u64 dsn, char *addr) +{ + addr[0] = (dsn >> 56) & 0xFF; + addr[1] = (dsn >> 48) & 0xFF; + addr[2] = (dsn >> 40) & 0xFF; + addr[3] = (dsn >> 16) & 0xFF; + addr[4] = (dsn >> 8) & 0xFF; + addr[5] = dsn & 0xFF; + + return is_valid_ether_addr(addr) ? 0 : -EINVAL; +} + +/** + * fbnic_netdev_register - Initialize general software structures + * @netdev: Netdev containing structure to initialize and register + * + * Initialize the MAC address for the netdev and register it. + **/ +int fbnic_netdev_register(struct net_device *netdev) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + struct fbnic_dev *fbd = fbn->fbd; + u64 dsn = fbd->dsn; + u8 addr[ETH_ALEN]; + int err; + + err = fbnic_dsn_to_mac_addr(dsn, addr); + if (!err) { + ether_addr_copy(netdev->perm_addr, addr); + eth_hw_addr_set(netdev, addr); + } else { + dev_err(fbd->dev, "MAC addr %pM invalid\n", addr); + return err; + } + + return register_netdev(netdev); +} + +void fbnic_netdev_unregister(struct net_device *netdev) +{ + unregister_netdev(netdev); +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h new file mode 100644 index 000000000000..8d12abe5fb57 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#ifndef _FBNIC_NETDEV_H_ +#define _FBNIC_NETDEV_H_ + +#include + +#include "fbnic_txrx.h" + +struct fbnic_net { + struct fbnic_ring *tx[FBNIC_MAX_TXQS]; + struct fbnic_ring *rx[FBNIC_MAX_RXQS]; + + struct net_device *netdev; + struct fbnic_dev *fbd; + + u16 num_napi; + + u16 num_tx_queues; + u16 num_rx_queues; + + struct list_head napis; +}; + +int __fbnic_open(struct fbnic_net *fbn); +void fbnic_up(struct fbnic_net *fbn); +void fbnic_down(struct fbnic_net *fbn); + +struct net_device *fbnic_netdev_alloc(struct fbnic_dev *fbd); +void fbnic_netdev_free(struct fbnic_dev *fbd); +int fbnic_netdev_register(struct net_device *netdev); +void fbnic_netdev_unregister(struct net_device *netdev); +void fbnic_reset_queues(struct fbnic_net *fbn, + unsigned int tx, unsigned int rx); + +#endif /* _FBNIC_NETDEV_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index 1873fba9c6c2..67f6112ec5f0 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -4,10 +4,12 @@ #include #include #include +#include #include #include "fbnic.h" #include "fbnic_drvinfo.h" +#include "fbnic_netdev.h" char fbnic_driver_name[] = DRV_NAME; @@ -15,6 +17,7 @@ MODULE_DESCRIPTION(DRV_SUMMARY); MODULE_LICENSE("GPL"); static const struct fbnic_info fbnic_asic_info = { + .max_num_queues = FBNIC_MAX_QUEUES, .bar_mask = BIT(0) | BIT(4) }; @@ -55,6 +58,10 @@ u32 fbnic_rd32(struct fbnic_dev *fbd, u32 reg) "Failed read (idx 0x%x AKA addr 0x%x), disabled CSR access, awaiting reset\n", reg, reg << 2); + /* Notify stack that device has lost (PCIe) link */ + if (!fbnic_init_failure(fbd)) + netif_device_detach(fbd->netdev); + return ~0U; } @@ -97,9 +104,56 @@ u32 fbnic_fw_rd32(struct fbnic_dev *fbd, u32 reg) "Failed read (idx 0x%x AKA addr 0x%x), disabled CSR access, awaiting reset\n", reg, reg << 2); + /* Notify stack that device has lost (PCIe) link */ + if (!fbnic_init_failure(fbd)) + netif_device_detach(fbd->netdev); + return ~0U; } +static void fbnic_service_task_start(struct fbnic_net *fbn) +{ + struct fbnic_dev *fbd = fbn->fbd; + + schedule_delayed_work(&fbd->service_task, HZ); +} + +static void fbnic_service_task_stop(struct fbnic_net *fbn) +{ + struct net_device *netdev = fbn->netdev; + struct fbnic_dev *fbd = fbn->fbd; + + cancel_delayed_work(&fbd->service_task); + netif_carrier_off(netdev); +} + +void fbnic_up(struct fbnic_net *fbn) +{ + netif_tx_start_all_queues(fbn->netdev); + + fbnic_service_task_start(fbn); +} + +void fbnic_down(struct fbnic_net *fbn) +{ + fbnic_service_task_stop(fbn); + + netif_tx_disable(fbn->netdev); +} + +static void fbnic_service_task(struct work_struct *work) +{ + struct fbnic_dev *fbd = container_of(to_delayed_work(work), + struct fbnic_dev, service_task); + + rtnl_lock(); + + if (netif_running(fbd->netdev)) + schedule_delayed_work(&fbd->service_task, HZ); + + rtnl_unlock(); +} + /** * fbnic_probe - Device Initialization Routine * @pdev: PCI device information struct @@ -114,6 +168,7 @@ u32 fbnic_fw_rd32(struct fbnic_dev *fbd, u32 reg) static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) { const struct fbnic_info *info = fbnic_info_tbl[ent->driver_data]; + struct net_device *netdev; struct fbnic_dev *fbd; int err; @@ -150,11 +205,16 @@ static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) return -ENOMEM; } + /* populate driver with hardware-specific info and handlers */ + fbd->max_num_queues = info->max_num_queues; + pci_set_master(pdev); pci_save_state(pdev); fbnic_devlink_register(fbd); + INIT_DELAYED_WORK(&fbd->service_task, fbnic_service_task); + err = fbnic_alloc_irqs(fbd); if (err) goto free_fbd; @@ -177,8 +237,22 @@ static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto init_failure_mode; } + netdev = fbnic_netdev_alloc(fbd); + if (!netdev) { + dev_err(&pdev->dev, "Netdev allocation failed\n"); + goto init_failure_mode; + } + + err = fbnic_netdev_register(netdev); + if (err) { + dev_err(&pdev->dev, "Netdev registration failed: %d\n", err); + goto ifm_free_netdev; + } + return 0; +ifm_free_netdev: + fbnic_netdev_free(fbd); init_failure_mode: dev_warn(&pdev->dev, "Probe error encountered, entering init failure mode. Normal networking functionality will not be available.\n"); /* Always return 0 even on error so devlink is registered to allow @@ -192,7 +266,6 @@ static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) fbnic_devlink_unregister(fbd); fbnic_devlink_free(fbd); - return err; } @@ -208,6 +281,14 @@ static void fbnic_remove(struct pci_dev *pdev) { struct fbnic_dev *fbd = pci_get_drvdata(pdev); + if (!fbnic_init_failure(fbd)) { + struct net_device *netdev = fbd->netdev; + + fbnic_netdev_unregister(netdev); + cancel_delayed_work_sync(&fbd->service_task); + fbnic_netdev_free(fbd); + } + fbnic_fw_disable_mbx(fbd); fbnic_free_irqs(fbd); @@ -218,7 +299,21 @@ static void fbnic_remove(struct pci_dev *pdev) static int fbnic_pm_suspend(struct device *dev) { struct fbnic_dev *fbd = dev_get_drvdata(dev); + struct net_device *netdev = fbd->netdev; + if (fbnic_init_failure(fbd)) + goto null_uc_addr; + + rtnl_lock(); + + netif_device_detach(netdev); + + if (netif_running(netdev)) + netdev->netdev_ops->ndo_stop(netdev); + + rtnl_unlock(); + +null_uc_addr: fbnic_fw_disable_mbx(fbd); /* Free the IRQs so they aren't trying to occupy sleeping CPUs */ @@ -234,7 +329,9 @@ static int fbnic_pm_suspend(struct device *dev) static int __fbnic_pm_resume(struct device *dev) { struct fbnic_dev *fbd = dev_get_drvdata(dev); + struct net_device *netdev = fbd->netdev; void __iomem * const *iomap_table; + struct fbnic_net *fbn; int err; /* restore MMIO access */ @@ -254,7 +351,29 @@ static int __fbnic_pm_resume(struct device *dev) if (err) goto err_free_irqs; + /* no netdev means there isn't a network interface to bring up */ + if (fbnic_init_failure(fbd)) + return 0; + + fbn = netdev_priv(netdev); + + /* reset the queues if needed */ + fbnic_reset_queues(fbn, fbn->num_tx_queues, fbn->num_rx_queues); + + rtnl_lock(); + + if (netif_running(netdev)) { + err = __fbnic_open(fbn); + if (err) + goto err_disable_mbx; + } + + rtnl_unlock(); + return 0; +err_disable_mbx: + rtnl_unlock(); + fbnic_fw_disable_mbx(fbd); err_free_irqs: fbnic_free_irqs(fbd); err_invalidate_uc_addr: @@ -263,11 +382,30 @@ static int __fbnic_pm_resume(struct device *dev) return err; } +static void __fbnic_pm_attach(struct device *dev) +{ + struct fbnic_dev *fbd = dev_get_drvdata(dev); + struct net_device *netdev = fbd->netdev; + struct fbnic_net *fbn; + + if (fbnic_init_failure(fbd)) + return; + + fbn = netdev_priv(netdev); + + if (netif_running(netdev)) + fbnic_up(fbn); + + netif_device_attach(netdev); +} + static int __maybe_unused fbnic_pm_resume(struct device *dev) { int err; err = __fbnic_pm_resume(dev); + if (!err) + __fbnic_pm_attach(dev); return err; } @@ -316,6 +454,7 @@ static pci_ers_result_t fbnic_err_slot_reset(struct pci_dev *pdev) static void fbnic_err_resume(struct pci_dev *pdev) { + __fbnic_pm_attach(&pdev->dev); } static const struct pci_error_handlers fbnic_err_handler = { diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c new file mode 100644 index 000000000000..366386a9721a --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c @@ -0,0 +1,268 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#include + +#include "fbnic_netdev.h" +#include "fbnic_txrx.h" +#include "fbnic.h" + +netdev_tx_t fbnic_xmit_frame(struct sk_buff *skb, struct net_device *dev) +{ + dev_kfree_skb_any(skb); + return NETDEV_TX_OK; +} + +static int fbnic_poll(struct napi_struct *napi, int budget) +{ + return 0; +} + +static irqreturn_t fbnic_msix_clean_rings(int __always_unused irq, void *data) +{ + struct fbnic_napi_vector *nv = data; + + napi_schedule_irqoff(&nv->napi); + + return IRQ_HANDLED; +} + +static void fbnic_remove_tx_ring(struct fbnic_net *fbn, + struct fbnic_ring *txr) +{ + if (!(txr->flags & FBNIC_RING_F_STATS)) + return; + + /* Remove pointer to the Tx ring */ + WARN_ON(fbn->tx[txr->q_idx] && fbn->tx[txr->q_idx] != txr); + fbn->tx[txr->q_idx] = NULL; +} + +static void fbnic_remove_rx_ring(struct fbnic_net *fbn, + struct fbnic_ring *rxr) +{ + if (!(rxr->flags & FBNIC_RING_F_STATS)) + return; + + /* Remove pointer to the Rx ring */ + WARN_ON(fbn->rx[rxr->q_idx] && fbn->rx[rxr->q_idx] != rxr); + fbn->rx[rxr->q_idx] = NULL; +} + +static void fbnic_free_napi_vector(struct fbnic_net *fbn, + struct fbnic_napi_vector *nv) +{ + struct fbnic_dev *fbd = nv->fbd; + u32 v_idx = nv->v_idx; + int i, j; + + for (i = 0; i < nv->txt_count; i++) { + fbnic_remove_tx_ring(fbn, &nv->qt[i].sub0); + fbnic_remove_tx_ring(fbn, &nv->qt[i].cmpl); + } + + for (j = 0; j < nv->rxt_count; j++, i++) { + fbnic_remove_rx_ring(fbn, &nv->qt[i].sub0); + fbnic_remove_rx_ring(fbn, &nv->qt[i].sub1); + fbnic_remove_rx_ring(fbn, &nv->qt[i].cmpl); + } + + free_irq(fbd->msix_entries[v_idx].vector, nv); + netif_napi_del(&nv->napi); + list_del(&nv->napis); + kfree(nv); +} + +void fbnic_free_napi_vectors(struct fbnic_net *fbn) +{ + struct fbnic_napi_vector *nv, *temp; + + list_for_each_entry_safe(nv, temp, &fbn->napis, napis) + fbnic_free_napi_vector(fbn, nv); +} + +static void fbnic_name_napi_vector(struct fbnic_napi_vector *nv) +{ + unsigned char *dev_name = nv->napi.dev->name; + + if (!nv->rxt_count) + snprintf(nv->name, sizeof(nv->name), "%s-Tx-%u", dev_name, + nv->v_idx - FBNIC_NON_NAPI_VECTORS); + else + snprintf(nv->name, sizeof(nv->name), "%s-TxRx-%u", dev_name, + nv->v_idx - FBNIC_NON_NAPI_VECTORS); +} + +static void fbnic_ring_init(struct fbnic_ring *ring, u32 __iomem *doorbell, + int q_idx, u8 flags) +{ + ring->doorbell = doorbell; + ring->q_idx = q_idx; + ring->flags = flags; +} + +static int fbnic_alloc_napi_vector(struct fbnic_dev *fbd, struct fbnic_net *fbn, + unsigned int v_count, unsigned int v_idx, + unsigned int txq_count, unsigned int txq_idx, + unsigned int rxq_count, unsigned int rxq_idx) +{ + int txt_count = txq_count, rxt_count = rxq_count; + u32 __iomem *uc_addr = fbd->uc_addr0; + struct fbnic_napi_vector *nv; + struct fbnic_q_triad *qt; + int qt_count, err; + u32 __iomem *db; + u32 vector; + + qt_count = txt_count + rxq_count; + if (!qt_count) + return -EINVAL; + + /* If MMIO has already failed there are no rings to initialize */ + if (!uc_addr) + return -EIO; + + /* allocate NAPI vector and queue triads */ + nv = kzalloc(struct_size(nv, qt, qt_count), GFP_KERNEL); + if (!nv) + return -ENOMEM; + + /* record queue triad counts */ + nv->txt_count = txt_count; + nv->rxt_count = rxt_count; + + /* provide pointer back to fbnic and MSI-X vectors */ + nv->fbd = fbd; + nv->v_idx = v_idx; + + /* tie napi to netdev */ + list_add(&nv->napis, &fbn->napis); + netif_napi_add(fbn->netdev, &nv->napi, fbnic_poll); + + /* tie nv back to PCIe dev */ + nv->dev = fbd->dev; + + /* initialize vector name */ + fbnic_name_napi_vector(nv); + + /* request the IRQ for napi vector */ + vector = fbd->msix_entries[v_idx].vector; + err = request_irq(vector, &fbnic_msix_clean_rings, IRQF_SHARED, + nv->name, nv); + if (err) + goto napi_del; + + /* Initialize queue triads */ + qt = nv->qt; + + while (txt_count) { + /* Configure Tx queue */ + db = &uc_addr[FBNIC_QUEUE(txq_idx) + FBNIC_QUEUE_TWQ0_TAIL]; + + /* Assign Tx queue to netdev if applicable */ + if (txq_count > 0) { + u8 flags = FBNIC_RING_F_CTX | FBNIC_RING_F_STATS; + + fbnic_ring_init(&qt->sub0, db, txq_idx, flags); + fbn->tx[txq_idx] = &qt->sub0; + txq_count--; + } else { + fbnic_ring_init(&qt->sub0, db, 0, + FBNIC_RING_F_DISABLED); + } + + /* Configure Tx completion queue */ + db = &uc_addr[FBNIC_QUEUE(txq_idx) + FBNIC_QUEUE_TCQ_HEAD]; + fbnic_ring_init(&qt->cmpl, db, 0, 0); + + /* Update Tx queue index */ + txt_count--; + txq_idx += v_count; + + /* move to next queue triad */ + qt++; + } + + while (rxt_count) { + /* Configure header queue */ + db = &uc_addr[FBNIC_QUEUE(rxq_idx) + FBNIC_QUEUE_BDQ_HPQ_TAIL]; + fbnic_ring_init(&qt->sub0, db, 0, FBNIC_RING_F_CTX); + + /* Configure payload queue */ + db = &uc_addr[FBNIC_QUEUE(rxq_idx) + FBNIC_QUEUE_BDQ_PPQ_TAIL]; + fbnic_ring_init(&qt->sub1, db, 0, FBNIC_RING_F_CTX); + + /* Configure Rx completion queue */ + db = &uc_addr[FBNIC_QUEUE(rxq_idx) + FBNIC_QUEUE_RCQ_HEAD]; + fbnic_ring_init(&qt->cmpl, db, rxq_idx, FBNIC_RING_F_STATS); + fbn->rx[rxq_idx] = &qt->cmpl; + + /* Update Rx queue index */ + rxt_count--; + rxq_idx += v_count; + + /* move to next queue triad */ + qt++; + } + + return 0; + +napi_del: + netif_napi_del(&nv->napi); + list_del(&nv->napis); + kfree(nv); + return err; +} + +int fbnic_alloc_napi_vectors(struct fbnic_net *fbn) +{ + unsigned int txq_idx = 0, rxq_idx = 0, v_idx = FBNIC_NON_NAPI_VECTORS; + unsigned int num_tx = fbn->num_tx_queues; + unsigned int num_rx = fbn->num_rx_queues; + unsigned int num_napi = fbn->num_napi; + struct fbnic_dev *fbd = fbn->fbd; + int err; + + /* Allocate 1 Tx queue per napi vector */ + if (num_napi < FBNIC_MAX_TXQS && num_napi == num_tx + num_rx) { + while (num_tx) { + err = fbnic_alloc_napi_vector(fbd, fbn, + num_napi, v_idx, + 1, txq_idx, 0, 0); + if (err) + goto free_vectors; + + /* update counts and index */ + num_tx--; + txq_idx++; + + v_idx++; + } + } + + /* Allocate Tx/Rx queue pairs per vector, or allocate remaining Rx */ + while (num_rx | num_tx) { + int tqpv = DIV_ROUND_UP(num_tx, num_napi - txq_idx); + int rqpv = DIV_ROUND_UP(num_rx, num_napi - rxq_idx); + + err = fbnic_alloc_napi_vector(fbd, fbn, num_napi, v_idx, + tqpv, txq_idx, rqpv, rxq_idx); + if (err) + goto free_vectors; + + /* update counts and index */ + num_tx -= tqpv; + txq_idx++; + + num_rx -= rqpv; + rxq_idx++; + + v_idx++; + } + + return 0; + +free_vectors: + fbnic_free_napi_vectors(fbn); + return -ENOMEM; +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h new file mode 100644 index 000000000000..e7f1208a3543 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#ifndef _FBNIC_TXRX_H_ +#define _FBNIC_TXRX_H_ + +#include +#include + +struct fbnic_net; + +#define FBNIC_MAX_TXQS 128u +#define FBNIC_MAX_RXQS 128u + +#define FBNIC_RING_F_DISABLED BIT(0) +#define FBNIC_RING_F_CTX BIT(1) +#define FBNIC_RING_F_STATS BIT(2) /* ring's stats may be used */ + +struct fbnic_ring { + u32 __iomem *doorbell; /* pointer to CSR space for ring */ + u16 size_mask; /* size of ring in descriptors - 1 */ + u8 q_idx; /* logical netdev ring index */ + u8 flags; /* ring flags (FBNIC_RING_F_*) */ + + u32 head, tail; /* head/tail of ring */ +}; + +struct fbnic_q_triad { + struct fbnic_ring sub0, sub1, cmpl; +}; + +struct fbnic_napi_vector { + struct napi_struct napi; + struct device *dev; /* Device for DMA unmapping */ + struct fbnic_dev *fbd; + char name[IFNAMSIZ + 9]; + + u16 v_idx; + u8 txt_count; + u8 rxt_count; + + struct list_head napis; + + struct fbnic_q_triad qt[]; +}; + +#define FBNIC_MAX_TXQS 128u +#define FBNIC_MAX_RXQS 128u + +netdev_tx_t fbnic_xmit_frame(struct sk_buff *skb, struct net_device *dev); + +int fbnic_alloc_napi_vectors(struct fbnic_net *fbn); +void fbnic_free_napi_vectors(struct fbnic_net *fbn); + +#endif /* _FBNIC_TXRX_H_ */ From patchwork Wed Apr 3 20:08:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616635 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-oa1-f42.google.com (mail-oa1-f42.google.com [209.85.160.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0922155A4F for ; Wed, 3 Apr 2024 20:09:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174943; cv=none; b=UrvBekXT/2Qaz021OtdaXGwZgLPozlh4gqOL6NfLFHaKGuRaoT2fmlW7mkmHPXqjQCUdiMlg1g+YrmRp1TLVLfCOj6w+Q2fimJBDp1RooXDybxVww9sHV57wg2Xz5Q1NQu9oU/EGHPKfMvJiWEbrn6xGK6b6E9UHrQ8+gpTMKhE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174943; c=relaxed/simple; bh=5H4n2T5Rlx9ROrSnHotUx4XeZEhaw+6mj49KpYW67dQ=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Mk2TBEajYeyfYJa7dNIGjgmOtPdxU1fibS5U8W7zQ+ZHKlNVBVsDCHU9YjcxjwDIvYAUngEYUq5/iXEmzgr+RdvW09zpB7XfKRDqYyUncF/MLZcrQejWHoUkoBogLASdLxM+BFJncPkfFd4zWb7yjQ9KYFznKNHymdiKHpGQuD0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=k3zH5djB; arc=none smtp.client-ip=209.85.160.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="k3zH5djB" Received: by mail-oa1-f42.google.com with SMTP id 586e51a60fabf-22200c78d4fso150399fac.1 for ; Wed, 03 Apr 2024 13:09:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174941; x=1712779741; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=RKu2dEhc7pt4Mm1LCzVtJcvP0ADmc43z1LQ9F1nD92c=; b=k3zH5djB35Tz/4pe0Q4BvdP0mR7Kp0E6rwOgrCrO3UsggTQDPMHW+Ja8Hs/ZTeRvl5 3D68mP51BZBXnbJKT/jBH47n2S2uiLQ9frAxx3WR6jgeIZ9R3CV+KbhKh6Huppzw2F0t V2TQ6NaoU2ImfPyjMnZibFy6NgNkQg/dA9Fkm7fF7dzi5SFtpEXSTBCgE6wKXzuEnK7X zIL7zgMbVKcOBqz8W8/49irbIEULxWGtDcs2aqQkSwYshYiZp5qgMDxchwhfCSxUKFv/ U84SBW8YMnM+Z96YUfO9ST4XwejLDO3+3lf3/mLf3hbwq8yprIy3hYc/Nqv7Y+Afg8M+ bgzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174941; x=1712779741; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=RKu2dEhc7pt4Mm1LCzVtJcvP0ADmc43z1LQ9F1nD92c=; b=RgBEUxSqkvkXb1BwKpI8AWMO7SPpmDDjdwsVpSa3bCA2KqNsEnREb6hYo6vPHWryYd rFbjkq+jNLX3RhiGneREauD7gcyIauiBGgBjtHdlrIq9SIQwPG5wk+Nxei4VP+ERtlDc Dh7I695FRLAjiKUIaLn37ThOMfA9vcVzMYw97x3X0HBnyMJsVZfnsbtkrCqKNdQafCzU /cIL2No6yeaMnsIy+NzNPDDqyDEEqWx6IePeI7l0i9SQ6Xc0qssRUJYGCbB0wy7LYx4E Xqe2Skg2zFR9EzNK3jP4Fp+lLL6uycb+JzkbosUD1kFBwBJ2MqaMlePEcQR36bCQfEOA IFiw== X-Gm-Message-State: AOJu0Yw9rWR6twUOFbsSTwI2JhWCAUp8Cbbmz1ApVl8yL4nWsqAKQ0vJ oqvWQCMMM1bkKGRzyq8d7Zf+T3yqtrScHF03We5cT2dnxVfQX53g X-Google-Smtp-Source: AGHT+IH5fmtl0tX2WXjFWtqxluzaHxZhMrt6z5SO7TymIOgKPUa1lLpDku+q+AfUg1v+jnZFSIujtQ== X-Received: by 2002:a05:6870:9a1f:b0:220:a138:f59f with SMTP id fo31-20020a0568709a1f00b00220a138f59fmr293088oab.31.1712174940586; Wed, 03 Apr 2024 13:09:00 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id d15-20020aa7814f000000b006e47e57d976sm12264617pfn.166.2024.04.03.13.08.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:08:59 -0700 (PDT) Subject: [net-next PATCH 08/15] eth: fbnic: implement Tx queue alloc/start/stop/free From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:08:58 -0700 Message-ID: <171217493846.1598374.6900488541450015791.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Implement basic management operations for Tx queues. Allocate memory for submission and completion rings. Learn how to start the queues, stop them, and wait for HW to be idle. We call HW rings "descriptor rings" (stored in ring->desc), and SW context rings "buffer rings" (stored in ring->*_buf union). This is the first patch which actually touches CSRs so add CSR helpers. No actual datapath / packet handling here, yet. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 66 ++++ drivers/net/ethernet/meta/fbnic/fbnic_netdev.c | 9 + drivers/net/ethernet/meta/fbnic/fbnic_netdev.h | 2 drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 19 + drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 425 ++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_txrx.h | 22 + 6 files changed, 542 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index 39980974e21f..e50c2827590b 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -61,10 +61,18 @@ #define FBNIC_INTR_CQ_REARM_INTR_RELOAD CSR_BIT(30) #define FBNIC_INTR_CQ_REARM_INTR_UNMASK CSR_BIT(31) +#define FBNIC_INTR_RCQ_TIMEOUT(n) \ + (0x00401 + ((n) * 4)) /* 0x01004 + 0x10*n */ +#define FBNIC_INTR_RCQ_TIMEOUT_CNT 256 +#define FBNIC_INTR_TCQ_TIMEOUT(n) \ + (0x00402 + ((n) * 4)) /* 0x01008 + 0x10*n */ +#define FBNIC_INTR_TCQ_TIMEOUT_CNT 256 #define FBNIC_CSR_END_INTR_CQ 0x007fe /* CSR section delimiter */ /* Global QM Tx registers */ #define FBNIC_CSR_START_QM_TX 0x00800 /* CSR section delimiter */ +#define FBNIC_QM_TWQ_IDLE(n) (0x00800 + (n)) /* 0x02000 + 0x4*n */ +#define FBNIC_QM_TWQ_IDLE_CNT 8 #define FBNIC_QM_TWQ_DEFAULT_META_L 0x00818 /* 0x02060 */ #define FBNIC_QM_TWQ_DEFAULT_META_H 0x00819 /* 0x02064 */ @@ -86,10 +94,16 @@ enum { #define FBNIC_QM_TQS_MTU_CTL0 0x0081d /* 0x02074 */ #define FBNIC_QM_TQS_MTU_CTL1 0x0081e /* 0x02078 */ #define FBNIC_QM_TQS_MTU_CTL1_BULK CSR_GENMASK(13, 0) +#define FBNIC_QM_TCQ_IDLE(n) (0x00821 + (n)) /* 0x02084 + 0x4*n */ +#define FBNIC_QM_TCQ_IDLE_CNT 4 #define FBNIC_QM_TCQ_CTL0 0x0082d /* 0x020b4 */ #define FBNIC_QM_TCQ_CTL0_COAL_WAIT CSR_GENMASK(15, 0) #define FBNIC_QM_TCQ_CTL0_TICK_CYCLES CSR_GENMASK(26, 16) +#define FBNIC_QM_TQS_IDLE(n) (0x00830 + (n)) /* 0x020c0 + 0x4*n */ +#define FBNIC_QM_TQS_IDLE_CNT 4 #define FBNIC_QM_TQS_EDT_TS_RANGE 0x00849 /* 0x2124 */ +#define FBNIC_QM_TDE_IDLE(n) (0x00853 + (n)) /* 0x0214c + 0x4*n */ +#define FBNIC_QM_TDE_IDLE_CNT 8 #define FBNIC_QM_TNI_TDF_CTL 0x0086c /* 0x021b0 */ #define FBNIC_QM_TNI_TDF_CTL_MRRS CSR_GENMASK(1, 0) #define FBNIC_QM_TNI_TDF_CTL_CLS CSR_GENMASK(3, 2) @@ -110,9 +124,15 @@ enum { /* Global QM Rx registers */ #define FBNIC_CSR_START_QM_RX 0x00c00 /* CSR section delimiter */ +#define FBNIC_QM_RCQ_IDLE(n) (0x00c00 + (n)) /* 0x03000 + 0x4*n */ +#define FBNIC_QM_RCQ_IDLE_CNT 4 #define FBNIC_QM_RCQ_CTL0 0x00c0c /* 0x03030 */ #define FBNIC_QM_RCQ_CTL0_COAL_WAIT CSR_GENMASK(15, 0) #define FBNIC_QM_RCQ_CTL0_TICK_CYCLES CSR_GENMASK(26, 16) +#define FBNIC_QM_HPQ_IDLE(n) (0x00c0f + (n)) /* 0x0303c + 0x4*n */ +#define FBNIC_QM_HPQ_IDLE_CNT 4 +#define FBNIC_QM_PPQ_IDLE(n) (0x00c13 + (n)) /* 0x0304c + 0x4*n */ +#define FBNIC_QM_PPQ_IDLE_CNT 4 #define FBNIC_QM_RNI_RBP_CTL 0x00c2d /* 0x030b4 */ #define FBNIC_QM_RNI_RBP_CTL_MRRS CSR_GENMASK(1, 0) #define FBNIC_QM_RNI_RBP_CTL_CLS CSR_GENMASK(3, 2) @@ -219,6 +239,8 @@ enum { /* TMI registers */ #define FBNIC_CSR_START_TMI 0x04400 /* CSR section delimiter */ #define FBNIC_TMI_SOP_PROT_CTRL 0x04400 /* 0x11000 */ +#define FBNIC_TMI_DROP_CTRL 0x04401 /* 0x11004 */ +#define FBNIC_TMI_DROP_CTRL_EN CSR_BIT(0) #define FBNIC_CSR_END_TMI 0x0443f /* CSR section delimiter */ /* Rx Buffer Registers */ #define FBNIC_CSR_START_RXB 0x08000 /* CSR section delimiter */ @@ -378,12 +400,56 @@ enum { #define FBNIC_QUEUE(n)\ (0x40000 + FBNIC_QUEUE_STRIDE * (n)) /* 0x100000 + 0x1000*n */ +#define FBNIC_QUEUE_TWQ0_CTL 0x000 /* 0x000 */ +#define FBNIC_QUEUE_TWQ1_CTL 0x001 /* 0x004 */ +#define FBNIC_QUEUE_TWQ_CTL_RESET CSR_BIT(0) +#define FBNIC_QUEUE_TWQ_CTL_ENABLE CSR_BIT(1) #define FBNIC_QUEUE_TWQ0_TAIL 0x002 /* 0x008 */ #define FBNIC_QUEUE_TWQ1_TAIL 0x003 /* 0x00c */ +#define FBNIC_QUEUE_TWQ0_SIZE 0x00a /* 0x028 */ +#define FBNIC_QUEUE_TWQ1_SIZE 0x00b /* 0x02c */ +#define FBNIC_QUEUE_TWQ_SIZE_MASK CSR_GENMASK(3, 0) + +#define FBNIC_QUEUE_TWQ0_BAL 0x020 /* 0x080 */ +#define FBNIC_QUEUE_BAL_MASK CSR_GENMASK(31, 7) +#define FBNIC_QUEUE_TWQ0_BAH 0x021 /* 0x084 */ +#define FBNIC_QUEUE_TWQ1_BAL 0x022 /* 0x088 */ +#define FBNIC_QUEUE_TWQ1_BAH 0x023 /* 0x08c */ + /* Tx Completion Queue Registers */ +#define FBNIC_QUEUE_TCQ_CTL 0x080 /* 0x200 */ +#define FBNIC_QUEUE_TCQ_CTL_RESET CSR_BIT(0) +#define FBNIC_QUEUE_TCQ_CTL_ENABLE CSR_BIT(1) + #define FBNIC_QUEUE_TCQ_HEAD 0x081 /* 0x204 */ +#define FBNIC_QUEUE_TCQ_SIZE 0x084 /* 0x210 */ +#define FBNIC_QUEUE_TCQ_SIZE_MASK CSR_GENMASK(3, 0) + +#define FBNIC_QUEUE_TCQ_BAL 0x0a0 /* 0x280 */ +#define FBNIC_QUEUE_TCQ_BAH 0x0a1 /* 0x284 */ + +/* Tx Interrupt Manager Registers */ +#define FBNIC_QUEUE_TIM_CTL 0x0c0 /* 0x300 */ +#define FBNIC_QUEUE_TIM_CTL_MSIX_MASK CSR_GENMASK(7, 0) + +#define FBNIC_QUEUE_TIM_THRESHOLD 0x0c1 /* 0x304 */ +#define FBNIC_QUEUE_TIM_THRESHOLD_TWD_MASK CSR_GENMASK(14, 0) + +#define FBNIC_QUEUE_TIM_CLEAR 0x0c2 /* 0x308 */ +#define FBNIC_QUEUE_TIM_CLEAR_MASK CSR_BIT(0) +#define FBNIC_QUEUE_TIM_SET 0x0c3 /* 0x30c */ +#define FBNIC_QUEUE_TIM_SET_MASK CSR_BIT(0) +#define FBNIC_QUEUE_TIM_MASK 0x0c4 /* 0x310 */ +#define FBNIC_QUEUE_TIM_MASK_MASK CSR_BIT(0) + +#define FBNIC_QUEUE_TIM_TIMER 0x0c5 /* 0x314 */ + +#define FBNIC_QUEUE_TIM_COUNTS 0x0c6 /* 0x318 */ +#define FBNIC_QUEUE_TIM_COUNTS_CNT1_MASK CSR_GENMASK(30, 16) +#define FBNIC_QUEUE_TIM_COUNTS_CNT0_MASK CSR_GENMASK(14, 0) + /* Rx Completion Queue Registers */ #define FBNIC_QUEUE_RCQ_HEAD 0x201 /* 0x804 */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c index 82c21bcb9c3f..dce3827d4398 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c @@ -17,6 +17,10 @@ int __fbnic_open(struct fbnic_net *fbn) if (err) return err; + err = fbnic_alloc_resources(fbn); + if (err) + goto free_napi_vectors; + err = netif_set_real_num_tx_queues(fbn->netdev, fbn->num_tx_queues); if (err) @@ -29,6 +33,8 @@ int __fbnic_open(struct fbnic_net *fbn) return 0; free_resources: + fbnic_free_resources(fbn); +free_napi_vectors: fbnic_free_napi_vectors(fbn); return err; } @@ -51,6 +57,7 @@ static int fbnic_stop(struct net_device *netdev) fbnic_down(fbn); + fbnic_free_resources(fbn); fbnic_free_napi_vectors(fbn); return 0; @@ -107,6 +114,8 @@ struct net_device *fbnic_netdev_alloc(struct fbnic_dev *fbd) fbn->fbd = fbd; INIT_LIST_HEAD(&fbn->napis); + fbn->txq_size = FBNIC_TXQ_SIZE_DEFAULT; + default_queues = netif_get_num_default_rss_queues(); if (default_queues > fbd->max_num_queues) default_queues = fbd->max_num_queues; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h index 8d12abe5fb57..b3c39c10c3f7 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h @@ -15,6 +15,8 @@ struct fbnic_net { struct net_device *netdev; struct fbnic_dev *fbd; + u32 txq_size; + u16 num_napi; u16 num_tx_queues; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index 67f6112ec5f0..12d7fbf22d27 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -129,16 +129,33 @@ static void fbnic_service_task_stop(struct fbnic_net *fbn) void fbnic_up(struct fbnic_net *fbn) { + fbnic_enable(fbn); + + /* Enable Tx/Rx processing */ + fbnic_napi_enable(fbn); netif_tx_start_all_queues(fbn->netdev); fbnic_service_task_start(fbn); } -void fbnic_down(struct fbnic_net *fbn) +static void fbnic_down_noidle(struct fbnic_net *fbn) { fbnic_service_task_stop(fbn); + /* Disable Tx/Rx Processing */ + fbnic_napi_disable(fbn); netif_tx_disable(fbn->netdev); + + fbnic_disable(fbn); +} + +void fbnic_down(struct fbnic_net *fbn) +{ + fbnic_down_noidle(fbn); + + fbnic_wait_all_queues_idle(fbn->fbd, false); + + fbnic_flush(fbn); } static void fbnic_service_task(struct work_struct *work) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c index 366386a9721a..dd05ed96d8fc 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c @@ -1,18 +1,50 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) Meta Platforms, Inc. and affiliates. */ +#include #include #include "fbnic_netdev.h" #include "fbnic_txrx.h" #include "fbnic.h" +static u32 __iomem *fbnic_ring_csr_base(const struct fbnic_ring *ring) +{ + unsigned long csr_base = (unsigned long)ring->doorbell; + + csr_base &= ~(FBNIC_QUEUE_STRIDE * sizeof(u32) - 1); + + return (u32 __iomem *)csr_base; +} + +static u32 fbnic_ring_rd32(struct fbnic_ring *ring, unsigned int csr) +{ + u32 __iomem *csr_base = fbnic_ring_csr_base(ring); + + return readl(csr_base + csr); +} + +static void fbnic_ring_wr32(struct fbnic_ring *ring, unsigned int csr, u32 val) +{ + u32 __iomem *csr_base = fbnic_ring_csr_base(ring); + + writel(val, csr_base + csr); +} + netdev_tx_t fbnic_xmit_frame(struct sk_buff *skb, struct net_device *dev) { dev_kfree_skb_any(skb); return NETDEV_TX_OK; } +static void fbnic_nv_irq_disable(struct fbnic_napi_vector *nv) +{ + struct fbnic_dev *fbd = nv->fbd; + u32 v_idx = nv->v_idx; + + wr32(FBNIC_INTR_MASK_SET(v_idx / 32), 1 << (v_idx % 32)); +} + static int fbnic_poll(struct napi_struct *napi, int budget) { return 0; @@ -266,3 +298,396 @@ int fbnic_alloc_napi_vectors(struct fbnic_net *fbn) fbnic_free_napi_vectors(fbn); return -ENOMEM; } + +static void fbnic_free_ring_resources(struct device *dev, + struct fbnic_ring *ring) +{ + kvfree(ring->buffer); + ring->buffer = NULL; + + /* If size is not set there are no descriptors present */ + if (!ring->size) + return; + + dma_free_coherent(dev, ring->size, ring->desc, ring->dma); + ring->size_mask = 0; + ring->size = 0; +} + +static int fbnic_alloc_tx_ring_desc(struct fbnic_net *fbn, + struct fbnic_ring *txr) +{ + struct device *dev = fbn->netdev->dev.parent; + size_t size; + + /* round size up to nearest 4K */ + size = ALIGN(array_size(sizeof(*txr->desc), fbn->txq_size), 4096); + + txr->desc = dma_alloc_coherent(dev, size, &txr->dma, + GFP_KERNEL | __GFP_NOWARN); + if (!txr->desc) + return -ENOMEM; + + /* txq_size should be a power of 2, so mask is just that -1 */ + txr->size_mask = fbn->txq_size - 1; + txr->size = size; + + return 0; +} + +static int fbnic_alloc_tx_ring_buffer(struct fbnic_ring *txr) +{ + size_t size = array_size(sizeof(*txr->tx_buf), txr->size_mask + 1); + + txr->tx_buf = kvzalloc(size, GFP_KERNEL | __GFP_NOWARN); + + return txr->tx_buf ? 0 : -ENOMEM; +} + +static int fbnic_alloc_tx_ring_resources(struct fbnic_net *fbn, + struct fbnic_ring *txr) +{ + struct device *dev = fbn->netdev->dev.parent; + int err; + + if (txr->flags & FBNIC_RING_F_DISABLED) + return 0; + + err = fbnic_alloc_tx_ring_desc(fbn, txr); + if (err) + return err; + + if (!(txr->flags & FBNIC_RING_F_CTX)) + return 0; + + err = fbnic_alloc_tx_ring_buffer(txr); + if (err) + goto free_desc; + + return 0; + +free_desc: + fbnic_free_ring_resources(dev, txr); + return err; +} + +static void fbnic_free_qt_resources(struct fbnic_net *fbn, + struct fbnic_q_triad *qt) +{ + struct device *dev = fbn->netdev->dev.parent; + + fbnic_free_ring_resources(dev, &qt->cmpl); + fbnic_free_ring_resources(dev, &qt->sub1); + fbnic_free_ring_resources(dev, &qt->sub0); +} + +static int fbnic_alloc_tx_qt_resources(struct fbnic_net *fbn, + struct fbnic_q_triad *qt) +{ + struct device *dev = fbn->netdev->dev.parent; + int err; + + err = fbnic_alloc_tx_ring_resources(fbn, &qt->sub0); + if (err) + return err; + + err = fbnic_alloc_tx_ring_resources(fbn, &qt->cmpl); + if (err) + goto free_sub0; + + return 0; + +free_sub0: + fbnic_free_ring_resources(dev, &qt->sub0); + return err; +} + +static void fbnic_free_nv_resources(struct fbnic_net *fbn, + struct fbnic_napi_vector *nv) +{ + int i; + + /* Free Tx Resources */ + for (i = 0; i < nv->txt_count; i++) + fbnic_free_qt_resources(fbn, &nv->qt[i]); +} + +static int fbnic_alloc_nv_resources(struct fbnic_net *fbn, + struct fbnic_napi_vector *nv) +{ + int i, err; + + /* Allocate Tx Resources */ + for (i = 0; i < nv->txt_count; i++) { + err = fbnic_alloc_tx_qt_resources(fbn, &nv->qt[i]); + if (err) + goto free_resources; + } + + return 0; + +free_resources: + while (i--) + fbnic_free_qt_resources(fbn, &nv->qt[i]); + return err; +} + +void fbnic_free_resources(struct fbnic_net *fbn) +{ + struct fbnic_napi_vector *nv; + + list_for_each_entry(nv, &fbn->napis, napis) + fbnic_free_nv_resources(fbn, nv); +} + +int fbnic_alloc_resources(struct fbnic_net *fbn) +{ + struct fbnic_napi_vector *nv; + int err = -ENODEV; + + list_for_each_entry(nv, &fbn->napis, napis) { + err = fbnic_alloc_nv_resources(fbn, nv); + if (err) + goto free_resources; + } + + return 0; + +free_resources: + list_for_each_entry_continue_reverse(nv, &fbn->napis, napis) + fbnic_free_nv_resources(fbn, nv); + + return err; +} + +static void fbnic_disable_twq0(struct fbnic_ring *txr) +{ + u32 twq_ctl = fbnic_ring_rd32(txr, FBNIC_QUEUE_TWQ0_CTL); + + twq_ctl &= ~FBNIC_QUEUE_TWQ_CTL_ENABLE; + + fbnic_ring_wr32(txr, FBNIC_QUEUE_TWQ0_CTL, twq_ctl); +} + +static void fbnic_disable_tcq(struct fbnic_ring *txr) +{ + fbnic_ring_wr32(txr, FBNIC_QUEUE_TCQ_CTL, 0); + fbnic_ring_wr32(txr, FBNIC_QUEUE_TIM_MASK, FBNIC_QUEUE_TIM_MASK_MASK); +} + +void fbnic_napi_disable(struct fbnic_net *fbn) +{ + struct fbnic_napi_vector *nv; + + list_for_each_entry(nv, &fbn->napis, napis) { + napi_disable(&nv->napi); + + fbnic_nv_irq_disable(nv); + } +} + +void fbnic_disable(struct fbnic_net *fbn) +{ + struct fbnic_dev *fbd = fbn->fbd; + struct fbnic_napi_vector *nv; + int i; + + list_for_each_entry(nv, &fbn->napis, napis) { + /* disable Tx Queue Triads */ + for (i = 0; i < nv->txt_count; i++) { + struct fbnic_q_triad *qt = &nv->qt[i]; + + fbnic_disable_twq0(&qt->sub0); + fbnic_disable_tcq(&qt->cmpl); + } + } + + wrfl(); +} + +static void fbnic_tx_flush(struct fbnic_dev *fbd) +{ + netdev_warn(fbd->netdev, "tiggerring Tx flush\n"); + + fbnic_rmw32(fbd, FBNIC_TMI_DROP_CTRL, FBNIC_TMI_DROP_CTRL_EN, + FBNIC_TMI_DROP_CTRL_EN); +} + +static void fbnic_tx_flush_off(struct fbnic_dev *fbd) +{ + fbnic_rmw32(fbd, FBNIC_TMI_DROP_CTRL, FBNIC_TMI_DROP_CTRL_EN, 0); +} + +struct fbnic_idle_regs { + u32 reg_base; + u8 reg_cnt; +}; + +static bool fbnic_all_idle(struct fbnic_dev *fbd, + const struct fbnic_idle_regs *regs, + unsigned int nregs) +{ + unsigned int i, j; + + for (i = 0; i < nregs; i++) { + for (j = 0; j < regs[i].reg_cnt; j++) { + if (fbnic_rd32(fbd, regs[i].reg_base + j) != ~0U) + return false; + } + } + return true; +} + +static void fbnic_idle_dump(struct fbnic_dev *fbd, + const struct fbnic_idle_regs *regs, + unsigned int nregs, const char *dir, int err) +{ + unsigned int i, j; + + netdev_err(fbd->netdev, "error waiting for %s idle %d\n", dir, err); + for (i = 0; i < nregs; i++) + for (j = 0; j < regs[i].reg_cnt; j++) + netdev_err(fbd->netdev, "0x%04x: %08x\n", + regs[i].reg_base + j, + fbnic_rd32(fbd, regs[i].reg_base + j)); +} + +int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail) +{ + static const struct fbnic_idle_regs tx[] = { + { FBNIC_QM_TWQ_IDLE(0), FBNIC_QM_TWQ_IDLE_CNT, }, + { FBNIC_QM_TQS_IDLE(0), FBNIC_QM_TQS_IDLE_CNT, }, + { FBNIC_QM_TDE_IDLE(0), FBNIC_QM_TDE_IDLE_CNT, }, + { FBNIC_QM_TCQ_IDLE(0), FBNIC_QM_TCQ_IDLE_CNT, }, + }; + bool idle; + int err; + + err = read_poll_timeout_atomic(fbnic_all_idle, idle, idle, 2, 500000, + false, fbd, tx, ARRAY_SIZE(tx)); + if (err == -ETIMEDOUT) { + fbnic_tx_flush(fbd); + err = read_poll_timeout_atomic(fbnic_all_idle, idle, idle, + 2, 500000, false, + fbd, tx, ARRAY_SIZE(tx)); + fbnic_tx_flush_off(fbd); + } + if (err) { + fbnic_idle_dump(fbd, tx, ARRAY_SIZE(tx), "Tx", err); + if (may_fail) + return err; + } + + return err; +} + +void fbnic_flush(struct fbnic_net *fbn) +{ + struct fbnic_napi_vector *nv; + + list_for_each_entry(nv, &fbn->napis, napis) { + int i; + + /* Flush any processed Tx Queue Triads and drop the rest */ + for (i = 0; i < nv->txt_count; i++) { + struct fbnic_q_triad *qt = &nv->qt[i]; + struct netdev_queue *tx_queue; + + /* Reset completion queue descriptor ring */ + memset(qt->cmpl.desc, 0, qt->cmpl.size); + + /* Reset BQL associated with Tx queue */ + tx_queue = netdev_get_tx_queue(nv->napi.dev, + qt->sub0.q_idx); + netdev_tx_reset_queue(tx_queue); + } + } +} + +static void fbnic_enable_twq0(struct fbnic_ring *twq) +{ + u32 log_size = fls(twq->size_mask); + + if (!twq->size_mask) + return; + + /* reset head/tail */ + fbnic_ring_wr32(twq, FBNIC_QUEUE_TWQ0_CTL, FBNIC_QUEUE_TWQ_CTL_RESET); + twq->tail = 0; + twq->head = 0; + + /* Store descriptor ring address and size */ + fbnic_ring_wr32(twq, FBNIC_QUEUE_TWQ0_BAL, lower_32_bits(twq->dma)); + fbnic_ring_wr32(twq, FBNIC_QUEUE_TWQ0_BAH, upper_32_bits(twq->dma)); + + /* write lower 4 bits of log size as 64K ring size is 0 */ + fbnic_ring_wr32(twq, FBNIC_QUEUE_TWQ0_SIZE, log_size & 0xf); + + fbnic_ring_wr32(twq, FBNIC_QUEUE_TWQ0_CTL, FBNIC_QUEUE_TWQ_CTL_ENABLE); +} + +static void fbnic_enable_tcq(struct fbnic_napi_vector *nv, + struct fbnic_ring *tcq) +{ + u32 log_size = fls(tcq->size_mask); + + if (!tcq->size_mask) + return; + + /* reset head/tail */ + fbnic_ring_wr32(tcq, FBNIC_QUEUE_TCQ_CTL, FBNIC_QUEUE_TCQ_CTL_RESET); + tcq->tail = 0; + tcq->head = 0; + + /* Store descriptor ring address and size */ + fbnic_ring_wr32(tcq, FBNIC_QUEUE_TCQ_BAL, lower_32_bits(tcq->dma)); + fbnic_ring_wr32(tcq, FBNIC_QUEUE_TCQ_BAH, upper_32_bits(tcq->dma)); + + /* write lower 4 bits of log size as 64K ring size is 0 */ + fbnic_ring_wr32(tcq, FBNIC_QUEUE_TCQ_SIZE, log_size & 0xf); + + /* Store interrupt information for the completion queue */ + fbnic_ring_wr32(tcq, FBNIC_QUEUE_TIM_CTL, nv->v_idx); + fbnic_ring_wr32(tcq, FBNIC_QUEUE_TIM_THRESHOLD, tcq->size_mask / 2); + fbnic_ring_wr32(tcq, FBNIC_QUEUE_TIM_MASK, 0); + + /* Enable queue */ + fbnic_ring_wr32(tcq, FBNIC_QUEUE_TCQ_CTL, FBNIC_QUEUE_TCQ_CTL_ENABLE); +} + +void fbnic_enable(struct fbnic_net *fbn) +{ + struct fbnic_dev *fbd = fbn->fbd; + struct fbnic_napi_vector *nv; + int i; + + list_for_each_entry(nv, &fbn->napis, napis) { + /* Setup Tx Queue Triads */ + for (i = 0; i < nv->txt_count; i++) { + struct fbnic_q_triad *qt = &nv->qt[i]; + + fbnic_enable_twq0(&qt->sub0); + fbnic_enable_tcq(nv, &qt->cmpl); + } + } + + wrfl(); +} + +static void fbnic_nv_irq_enable(struct fbnic_napi_vector *nv) +{ + struct fbnic_dev *fbd = nv->fbd; + + wr32(FBNIC_INTR_CQ_REARM(nv->v_idx), FBNIC_INTR_CQ_REARM_INTR_UNMASK); +} + +void fbnic_napi_enable(struct fbnic_net *fbn) +{ + struct fbnic_napi_vector *nv; + + list_for_each_entry(nv, &fbn->napis, napis) { + napi_enable(&nv->napi); + + fbnic_nv_irq_enable(nv); + } +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h index e7f1208a3543..2898e0dccf7a 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h @@ -12,17 +12,30 @@ struct fbnic_net; #define FBNIC_MAX_TXQS 128u #define FBNIC_MAX_RXQS 128u +#define FBNIC_TXQ_SIZE_DEFAULT 1024 + #define FBNIC_RING_F_DISABLED BIT(0) #define FBNIC_RING_F_CTX BIT(1) #define FBNIC_RING_F_STATS BIT(2) /* ring's stats may be used */ struct fbnic_ring { + /* Pointer to buffer specific info */ + union { + void *buffer; /* Generic pointer */ + void **tx_buf; /* TWQ */ + }; + u32 __iomem *doorbell; /* pointer to CSR space for ring */ + __le64 *desc; /* descriptor ring memory */ u16 size_mask; /* size of ring in descriptors - 1 */ u8 q_idx; /* logical netdev ring index */ u8 flags; /* ring flags (FBNIC_RING_F_*) */ u32 head, tail; /* head/tail of ring */ + + /* slow path fields follow */ + dma_addr_t dma; /* phys addr of descriptor memory */ + size_t size; /* size of descriptor ring in memory */ }; struct fbnic_q_triad { @@ -51,5 +64,14 @@ netdev_tx_t fbnic_xmit_frame(struct sk_buff *skb, struct net_device *dev); int fbnic_alloc_napi_vectors(struct fbnic_net *fbn); void fbnic_free_napi_vectors(struct fbnic_net *fbn); +int fbnic_alloc_resources(struct fbnic_net *fbn); +void fbnic_free_resources(struct fbnic_net *fbn); +void fbnic_napi_enable(struct fbnic_net *fbn); +void fbnic_napi_disable(struct fbnic_net *fbn); +void fbnic_enable(struct fbnic_net *fbn); +void fbnic_disable(struct fbnic_net *fbn); +void fbnic_flush(struct fbnic_net *fbn); + +int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail); #endif /* _FBNIC_TXRX_H_ */ From patchwork Wed Apr 3 20:09:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616636 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93279155758 for ; Wed, 3 Apr 2024 20:09:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174947; cv=none; b=swvpnq1hgt3wZB//sZoLGmySCUyCtREt924xPYrulhZ23HolVdqlhXcEJgGVJvL6FbjAx3IhkxxHShMraRDd1NqEcMdd+UWlEQb5YOhJHfnDXX8Ji8O5IzJ1g0+aORVWUxI+MWQt5ZbiS8bhbiK45VmonPRtVI61F4Qi7mRniP8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174947; c=relaxed/simple; bh=KYPadNLROppIXX2jYwputd1Q5RIgnWhSnfm+NMt/Lck=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=PfdJQNHG5g0H+Nrjy7V6aL78uHp3aQodETZuI3KIaJPf2tiWIUYcqGf3n7uRlPn7XXmw0aj4Lgc0sMQUs1N5cpJufrbthb+EIe2V4UeonzKL7S1vDRsTmeqB2+jCuUihcaIUdcjeVWQmhCPQJ+2V1ZwSnGqf8hjM7OVpe964yAs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TeoRFek4; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TeoRFek4" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1e28efd8335so1706875ad.0 for ; Wed, 03 Apr 2024 13:09:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174945; x=1712779745; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=9k1XmAzI3Hgm+OnwrkZSij5lZ6182abKvcydSr8PAQM=; b=TeoRFek4kZ8ffQsHK7I21wTe7MhpoXhmYNQKeZ0SXRoIXQ/LiAJddLlnyur66wXAKv PxsLXvB35YIKuiL5Nfsmuy8QRfX2rjZdEBBmC3vnFLvetiGWnC+A8aR9kAle3I5hU6Kf MLcBzy3zxWjLQW7DXBcYAK6p7Lh80p7VtrTsK8kx87tNA67uQYNEh+J1jCN3PBlG/H3w 5GxyiU3ziSuKNz6vJt5UkPgmBjN562v6P5knWiIJ+iC+5GqgIoh5i+AclUn2PPWm7EDS Hi0GqE8lxSoYR6I/u5fBpOBRjAdInKgYYcB10NjeUky2cD404+Oty6pXGLgnCPqXsS3N IoLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174945; x=1712779745; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=9k1XmAzI3Hgm+OnwrkZSij5lZ6182abKvcydSr8PAQM=; b=LaV99OWJQ0jDW3zGPHGetHSnnAE0XCjN9CPcm/Y8FM+pEHRKnQG86MpF3l6lgBJt+3 Vm56Ve9lp4pmMfL1kKL3ZsyFBns3WJmr6WNgkV61qnBs4lNjQgPLCjDd5aIZZz7Mi2qk d2JGVPP4mq2VzEBQDgOXJbo6ucYO8eQomLUqAyjl+Gg1Xji0gjlfe6ZwHuUOnKgTRV2V A8HMfh0rNgSA81HozA8TivHjTji0a2Ek/PFpDWnHpcSdYABvEpfgGoPv5Tl2mDS6a46k qibk1BYJ4x0l8Zc28fOOS4XLiVhQAxsKMOZF+jLo4VyE4ldYAgPw+zeJn5oPdPLP087v xm6w== X-Gm-Message-State: AOJu0YzygNEgJDhQCQQSgsi9s5sbOqZy8up0BKg/IzBIg2jYq3nw63SY f0abEM8+R+UuiZcD9qy6OPdh5R2k/cNR2xCYusYmCDcNDWSXZjQ2 X-Google-Smtp-Source: AGHT+IGpX1FBeKPY5fBQ7Fa5RptdXFYhYvMLs2OBqtWv4ar4K39xgOoUW6BQYSTsosDVstGj++QbTA== X-Received: by 2002:a17:903:2656:b0:1df:f681:3cd8 with SMTP id je22-20020a170903265600b001dff6813cd8mr369337plb.12.1712174944878; Wed, 03 Apr 2024 13:09:04 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id d3-20020a170902cec300b001e294f2f30dsm1885360plg.93.2024.04.03.13.09.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:09:04 -0700 (PDT) Subject: [net-next PATCH 09/15] eth: fbnic: implement Rx queue alloc/start/stop/free From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:09:02 -0700 Message-ID: <171217494276.1598374.468010123854919775.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Implement control path parts of Rx queue handling. The NIC consumes memory in pages. It takes a full page and places packets into it in a configurable manner (with the ability to define headroom / tailroom as well as head alignment requirements). As mentioned in prior patches there are two page submissions queues one for packet headers and second (optional) for packet payloads. For now feed both queues from a single page pool. Use the page pool "fragment" API, as we can't predict upfront how the page will be sliced. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 70 ++++ drivers/net/ethernet/meta/fbnic/fbnic_netdev.c | 3 drivers/net/ethernet/meta/fbnic/fbnic_netdev.h | 3 drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 2 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 466 ++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_txrx.h | 30 ++ 6 files changed, 568 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index e50c2827590b..33832a4f78ea 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -16,6 +16,10 @@ #define FBNIC_CLOCK_FREQ (600 * (1000 * 1000)) +/* Rx Buffer Descriptor Format */ +#define FBNIC_BD_PAGE_ADDR_MASK DESC_GENMASK(45, 12) +#define FBNIC_BD_PAGE_ID_MASK DESC_GENMASK(63, 48) + /* Register Definitions * * The registers are laid as indexes into an le32 array. As such the actual @@ -451,12 +455,78 @@ enum { #define FBNIC_QUEUE_TIM_COUNTS_CNT0_MASK CSR_GENMASK(14, 0) /* Rx Completion Queue Registers */ +#define FBNIC_QUEUE_RCQ_CTL 0x200 /* 0x800 */ +#define FBNIC_QUEUE_RCQ_CTL_RESET CSR_BIT(0) +#define FBNIC_QUEUE_RCQ_CTL_ENABLE CSR_BIT(1) + #define FBNIC_QUEUE_RCQ_HEAD 0x201 /* 0x804 */ +#define FBNIC_QUEUE_RCQ_SIZE 0x204 /* 0x810 */ +#define FBNIC_QUEUE_RCQ_SIZE_MASK CSR_GENMASK(3, 0) + +#define FBNIC_QUEUE_RCQ_BAL 0x220 /* 0x880 */ +#define FBNIC_QUEUE_RCQ_BAH 0x221 /* 0x884 */ + /* Rx Buffer Descriptor Queue Registers */ +#define FBNIC_QUEUE_BDQ_CTL 0x240 /* 0x900 */ +#define FBNIC_QUEUE_BDQ_CTL_RESET CSR_BIT(0) +#define FBNIC_QUEUE_BDQ_CTL_ENABLE CSR_BIT(1) +#define FBNIC_QUEUE_BDQ_CTL_PPQ_ENABLE CSR_BIT(30) + #define FBNIC_QUEUE_BDQ_HPQ_TAIL 0x241 /* 0x904 */ #define FBNIC_QUEUE_BDQ_PPQ_TAIL 0x242 /* 0x908 */ +#define FBNIC_QUEUE_BDQ_HPQ_SIZE 0x247 /* 0x91c */ +#define FBNIC_QUEUE_BDQ_PPQ_SIZE 0x248 /* 0x920 */ +#define FBNIC_QUEUE_BDQ_SIZE_MASK CSR_GENMASK(3, 0) + +#define FBNIC_QUEUE_BDQ_HPQ_BAL 0x260 /* 0x980 */ +#define FBNIC_QUEUE_BDQ_HPQ_BAH 0x261 /* 0x984 */ +#define FBNIC_QUEUE_BDQ_PPQ_BAL 0x262 /* 0x988 */ +#define FBNIC_QUEUE_BDQ_PPQ_BAH 0x263 /* 0x98c */ + +/* Rx DMA Engine Configuration */ +#define FBNIC_QUEUE_RDE_CTL0 0x2a0 /* 0xa80 */ +#define FBNIC_QUEUE_RDE_CTL0_EN_HDR_SPLIT CSR_BIT(31) +#define FBNIC_QUEUE_RDE_CTL0_DROP_MODE_MASK CSR_GENMASK(30, 29) +enum { + FBNIC_QUEUE_RDE_CTL0_DROP_IMMEDIATE = 0, + FBNIC_QUEUE_RDE_CTL0_DROP_WAIT = 1, + FBNIC_QUEUE_RDE_CTL0_DROP_NEVER = 2, +}; + +#define FBNIC_QUEUE_RDE_CTL0_MIN_HROOM_MASK CSR_GENMASK(28, 20) +#define FBNIC_QUEUE_RDE_CTL0_MIN_TROOM_MASK CSR_GENMASK(19, 11) + +#define FBNIC_QUEUE_RDE_CTL1 0x2a1 /* 0xa84 */ +#define FBNIC_QUEUE_RDE_CTL1_MAX_HDR_MASK CSR_GENMASK(24, 12) +#define FBNIC_QUEUE_RDE_CTL1_PAYLD_OFF_MASK CSR_GENMASK(11, 9) +#define FBNIC_QUEUE_RDE_CTL1_PAYLD_PG_CL_MASK CSR_GENMASK(8, 6) +#define FBNIC_QUEUE_RDE_CTL1_PADLEN_MASK CSR_GENMASK(5, 2) +#define FBNIC_QUEUE_RDE_CTL1_PAYLD_PACK_MASK CSR_GENMASK(1, 0) +enum { + FBNIC_QUEUE_RDE_CTL1_PAYLD_PACK_NONE = 0, + FBNIC_QUEUE_RDE_CTL1_PAYLD_PACK_ALL = 1, + FBNIC_QUEUE_RDE_CTL1_PAYLD_PACK_RSS = 2, +}; + +/* Rx Interrupt Manager Registers */ +#define FBNIC_QUEUE_RIM_CTL 0x2c0 /* 0xb00 */ +#define FBNIC_QUEUE_RIM_CTL_MSIX_MASK CSR_GENMASK(7, 0) + +#define FBNIC_QUEUE_RIM_THRESHOLD 0x2c1 /* 0xb04 */ +#define FBNIC_QUEUE_RIM_THRESHOLD_RCD_MASK CSR_GENMASK(14, 0) + +#define FBNIC_QUEUE_RIM_CLEAR 0x2c2 /* 0xb08 */ +#define FBNIC_QUEUE_RIM_CLEAR_MASK CSR_BIT(0) +#define FBNIC_QUEUE_RIM_SET 0x2c3 /* 0xb0c */ +#define FBNIC_QUEUE_RIM_SET_MASK CSR_BIT(0) +#define FBNIC_QUEUE_RIM_MASK 0x2c4 /* 0xb10 */ +#define FBNIC_QUEUE_RIM_MASK_MASK CSR_BIT(0) + +#define FBNIC_QUEUE_RIM_COAL_STATUS 0x2c5 /* 0xb14 */ +#define FBNIC_QUEUE_RIM_RCD_COUNT_MASK CSR_GENMASK(30, 16) +#define FBNIC_QUEUE_RIM_TIMER_MASK CSR_GENMASK(13, 0) #define FBNIC_MAX_QUEUES 128 #define FBNIC_CSR_END_QUEUE (0x40000 + 0x400 * FBNIC_MAX_QUEUES - 1) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c index dce3827d4398..171b159cc006 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c @@ -115,6 +115,9 @@ struct net_device *fbnic_netdev_alloc(struct fbnic_dev *fbd) INIT_LIST_HEAD(&fbn->napis); fbn->txq_size = FBNIC_TXQ_SIZE_DEFAULT; + fbn->hpq_size = FBNIC_HPQ_SIZE_DEFAULT; + fbn->ppq_size = FBNIC_PPQ_SIZE_DEFAULT; + fbn->rcq_size = FBNIC_RCQ_SIZE_DEFAULT; default_queues = netif_get_num_default_rss_queues(); if (default_queues > fbd->max_num_queues) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h index b3c39c10c3f7..18f93e9431cc 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h @@ -16,6 +16,9 @@ struct fbnic_net { struct fbnic_dev *fbd; u32 txq_size; + u32 hpq_size; + u32 ppq_size; + u32 rcq_size; u16 num_napi; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index 12d7fbf22d27..d6598c81a5f9 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -131,6 +131,8 @@ void fbnic_up(struct fbnic_net *fbn) { fbnic_enable(fbn); + fbnic_fill(fbn); + /* Enable Tx/Rx processing */ fbnic_napi_enable(fbn); netif_tx_start_all_queues(fbn->netdev); diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c index dd05ed96d8fc..484cab7342da 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c @@ -3,6 +3,8 @@ #include #include +#include +#include #include "fbnic_netdev.h" #include "fbnic_txrx.h" @@ -31,12 +33,128 @@ static void fbnic_ring_wr32(struct fbnic_ring *ring, unsigned int csr, u32 val) writel(val, csr_base + csr); } +static inline unsigned int fbnic_desc_unused(struct fbnic_ring *ring) +{ + return (ring->head - ring->tail - 1) & ring->size_mask; +} + netdev_tx_t fbnic_xmit_frame(struct sk_buff *skb, struct net_device *dev) { dev_kfree_skb_any(skb); return NETDEV_TX_OK; } +static void fbnic_page_pool_init(struct fbnic_ring *ring, unsigned int idx, + struct page *page) +{ + struct fbnic_rx_buf *rx_buf = &ring->rx_buf[idx]; + + page_pool_fragment_page(page, PAGECNT_BIAS_MAX); + rx_buf->pagecnt_bias = PAGECNT_BIAS_MAX; + rx_buf->page = page; +} + +static void fbnic_page_pool_drain(struct fbnic_ring *ring, unsigned int idx, + struct fbnic_napi_vector *nv, int budget) +{ + struct fbnic_rx_buf *rx_buf = &ring->rx_buf[idx]; + struct page *page = rx_buf->page; + + if (!page_pool_unref_page(page, rx_buf->pagecnt_bias)) + page_pool_put_unrefed_page(nv->page_pool, page, -1, !!budget); + + rx_buf->page = NULL; +} + +static void fbnic_clean_bdq(struct fbnic_napi_vector *nv, int napi_budget, + struct fbnic_ring *ring, unsigned int hw_head) +{ + unsigned int head = ring->head; + + if (head == hw_head) + return; + + do { + fbnic_page_pool_drain(ring, head, nv, napi_budget); + + head++; + head &= ring->size_mask; + } while (head != hw_head); + + ring->head = head; +} + +static __le64 fbnic_bd_prep(struct page *page, u16 id) +{ + dma_addr_t dma = page_pool_get_dma_addr(page); + u64 bd; + + bd = (FBNIC_BD_PAGE_ADDR_MASK & dma) | + FIELD_PREP(FBNIC_BD_PAGE_ID_MASK, id); + + return cpu_to_le64(bd); +} + +static void fbnic_fill_bdq(struct fbnic_napi_vector *nv, struct fbnic_ring *bdq) +{ + unsigned int count = fbnic_desc_unused(bdq); + unsigned int i = bdq->tail; + + if (!count) + return; + + do { + struct page *page; + __le64 *bd; + + page = page_pool_dev_alloc_pages(nv->page_pool); + if (!page) + break; + + fbnic_page_pool_init(bdq, i, page); + + bd = &bdq->desc[i]; + *bd = fbnic_bd_prep(page, i); + + i++; + i &= bdq->size_mask; + + count--; + } while (count); + + if (bdq->tail != i) { + bdq->tail = i; + + /* Force DMA writes to flush before writing to tail */ + dma_wmb(); + + writel(i, bdq->doorbell); + } +} + +static void fbnic_put_pkt_buff(struct fbnic_napi_vector *nv, + struct fbnic_pkt_buff *pkt, int budget) +{ + struct skb_shared_info *shinfo; + struct page *page; + int nr_frags; + + if (!pkt->buff.data_hard_start) + return; + + shinfo = xdp_get_shared_info_from_buff(&pkt->buff); + nr_frags = pkt->nr_frags; + + while (nr_frags--) { + page = skb_frag_page(&shinfo->frags[nr_frags]); + page_pool_put_full_page(nv->page_pool, page, !!budget); + } + + page = virt_to_page(pkt->buff.data_hard_start); + page_pool_put_full_page(nv->page_pool, page, !!budget); + pkt->buff.data_hard_start = NULL; +} + static void fbnic_nv_irq_disable(struct fbnic_napi_vector *nv) { struct fbnic_dev *fbd = nv->fbd; @@ -100,6 +218,7 @@ static void fbnic_free_napi_vector(struct fbnic_net *fbn, } free_irq(fbd->msix_entries[v_idx].vector, nv); + page_pool_destroy(nv->page_pool); netif_napi_del(&nv->napi); list_del(&nv->napis); kfree(nv); @@ -125,6 +244,42 @@ static void fbnic_name_napi_vector(struct fbnic_napi_vector *nv) nv->v_idx - FBNIC_NON_NAPI_VECTORS); } +static int fbnic_alloc_nv_page_pool(struct fbnic_net *fbn, + struct fbnic_napi_vector *nv) +{ + struct page_pool_params pp_params = { + .order = 0, + .flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV, + .pool_size = (fbn->hpq_size + fbn->ppq_size) * nv->rxt_count, + .nid = NUMA_NO_NODE, + .dev = nv->dev, + .dma_dir = DMA_BIDIRECTIONAL, + .offset = 0, + .max_len = PAGE_SIZE + }; + struct page_pool *pp; + + /* Page pool cannot exceed a size of 32768. This doesn't limit the + * pages on the ring but the number we can have cached waiting on + * the next use. + * + * TBD: Can this be reduced further? Would a multiple of + * NAPI_POLL_WEIGHT possibly make more sense? The question is how + * may pages do we need to hold in reserve to get the best return + * without hogging too much system memory. + */ + if (pp_params.pool_size > 32768) + pp_params.pool_size = 32768; + + pp = page_pool_create(&pp_params); + if (IS_ERR(pp)) + return PTR_ERR(pp); + + nv->page_pool = pp; + + return 0; +} + static void fbnic_ring_init(struct fbnic_ring *ring, u32 __iomem *doorbell, int q_idx, u8 flags) { @@ -174,6 +329,13 @@ static int fbnic_alloc_napi_vector(struct fbnic_dev *fbd, struct fbnic_net *fbn, /* tie nv back to PCIe dev */ nv->dev = fbd->dev; + /* allocate page pool */ + if (rxq_count) { + err = fbnic_alloc_nv_page_pool(fbn, nv); + if (err) + goto napi_del; + } + /* initialize vector name */ fbnic_name_napi_vector(nv); @@ -182,7 +344,7 @@ static int fbnic_alloc_napi_vector(struct fbnic_dev *fbd, struct fbnic_net *fbn, err = request_irq(vector, &fbnic_msix_clean_rings, IRQF_SHARED, nv->name, nv); if (err) - goto napi_del; + goto pp_destroy; /* Initialize queue triads */ qt = nv->qt; @@ -239,6 +401,8 @@ static int fbnic_alloc_napi_vector(struct fbnic_dev *fbd, struct fbnic_net *fbn, return 0; +pp_destroy: + page_pool_destroy(nv->page_pool); napi_del: netif_napi_del(&nv->napi); list_del(&nv->napis); @@ -371,6 +535,77 @@ static int fbnic_alloc_tx_ring_resources(struct fbnic_net *fbn, return err; } +static int fbnic_alloc_rx_ring_desc(struct fbnic_net *fbn, + struct fbnic_ring *rxr) +{ + struct device *dev = fbn->netdev->dev.parent; + u32 rxq_size; + size_t size; + + switch (rxr->doorbell - fbnic_ring_csr_base(rxr)) { + case FBNIC_QUEUE_BDQ_HPQ_TAIL: + rxq_size = fbn->hpq_size; + break; + case FBNIC_QUEUE_BDQ_PPQ_TAIL: + rxq_size = fbn->ppq_size; + break; + case FBNIC_QUEUE_RCQ_HEAD: + rxq_size = fbn->rcq_size; + break; + default: + return -EINVAL; + } + + /* round size up to nearest 4K */ + size = ALIGN(array_size(sizeof(*rxr->desc), rxq_size), 4096); + + rxr->desc = dma_alloc_coherent(dev, size, &rxr->dma, + GFP_KERNEL | __GFP_NOWARN); + if (!rxr->desc) + return -ENOMEM; + + /* rxq_size should be a power of 2, so mask is just that -1 */ + rxr->size_mask = rxq_size - 1; + rxr->size = size; + + return 0; +} + +static int fbnic_alloc_rx_ring_buffer(struct fbnic_ring *rxr) +{ + size_t size = array_size(sizeof(*rxr->rx_buf), rxr->size_mask + 1); + + if (rxr->flags & FBNIC_RING_F_CTX) + size = sizeof(*rxr->rx_buf) * (rxr->size_mask + 1); + else + size = sizeof(*rxr->pkt); + + rxr->rx_buf = kvzalloc(size, GFP_KERNEL | __GFP_NOWARN); + + return rxr->rx_buf ? 0 : -ENOMEM; +} + +static int fbnic_alloc_rx_ring_resources(struct fbnic_net *fbn, + struct fbnic_ring *rxr) +{ + struct device *dev = fbn->netdev->dev.parent; + int err; + + err = fbnic_alloc_rx_ring_desc(fbn, rxr); + if (err) + return err; + + err = fbnic_alloc_rx_ring_buffer(rxr); + if (err) + goto free_desc; + + return 0; + +free_desc: + fbnic_free_ring_resources(dev, rxr); + return err; +} + static void fbnic_free_qt_resources(struct fbnic_net *fbn, struct fbnic_q_triad *qt) { @@ -402,20 +637,50 @@ static int fbnic_alloc_tx_qt_resources(struct fbnic_net *fbn, return err; } +static int fbnic_alloc_rx_qt_resources(struct fbnic_net *fbn, + struct fbnic_q_triad *qt) +{ + struct device *dev = fbn->netdev->dev.parent; + int err; + + err = fbnic_alloc_rx_ring_resources(fbn, &qt->sub0); + if (err) + return err; + + err = fbnic_alloc_rx_ring_resources(fbn, &qt->sub1); + if (err) + goto free_sub0; + + err = fbnic_alloc_rx_ring_resources(fbn, &qt->cmpl); + if (err) + goto free_sub1; + + return 0; + +free_sub1: + fbnic_free_ring_resources(dev, &qt->sub1); +free_sub0: + fbnic_free_ring_resources(dev, &qt->sub0); + return err; +} + static void fbnic_free_nv_resources(struct fbnic_net *fbn, struct fbnic_napi_vector *nv) { - int i; + int i, j; /* Free Tx Resources */ for (i = 0; i < nv->txt_count; i++) fbnic_free_qt_resources(fbn, &nv->qt[i]); + + for (j = 0; j < nv->rxt_count; j++, i++) + fbnic_free_qt_resources(fbn, &nv->qt[i]); } static int fbnic_alloc_nv_resources(struct fbnic_net *fbn, struct fbnic_napi_vector *nv) { - int i, err; + int i, j, err; /* Allocate Tx Resources */ for (i = 0; i < nv->txt_count; i++) { @@ -424,6 +689,13 @@ static int fbnic_alloc_nv_resources(struct fbnic_net *fbn, goto free_resources; } + /* Allocate Rx Resources */ + for (j = 0; j < nv->rxt_count; j++, i++) { + err = fbnic_alloc_rx_qt_resources(fbn, &nv->qt[i]); + if (err) + goto free_resources; + } + return 0; free_resources: @@ -475,6 +747,21 @@ static void fbnic_disable_tcq(struct fbnic_ring *txr) fbnic_ring_wr32(txr, FBNIC_QUEUE_TIM_MASK, FBNIC_QUEUE_TIM_MASK_MASK); } +static void fbnic_disable_bdq(struct fbnic_ring *hpq, struct fbnic_ring *ppq) +{ + u32 bdq_ctl = fbnic_ring_rd32(hpq, FBNIC_QUEUE_BDQ_CTL); + + bdq_ctl &= ~FBNIC_QUEUE_BDQ_CTL_ENABLE; + + fbnic_ring_wr32(hpq, FBNIC_QUEUE_BDQ_CTL, bdq_ctl); +} + +static void fbnic_disable_rcq(struct fbnic_ring *rxr) +{ + fbnic_ring_wr32(rxr, FBNIC_QUEUE_RCQ_CTL, 0); + fbnic_ring_wr32(rxr, FBNIC_QUEUE_RIM_MASK, FBNIC_QUEUE_RIM_MASK_MASK); +} + void fbnic_napi_disable(struct fbnic_net *fbn) { struct fbnic_napi_vector *nv; @@ -490,7 +777,7 @@ void fbnic_disable(struct fbnic_net *fbn) { struct fbnic_dev *fbd = fbn->fbd; struct fbnic_napi_vector *nv; - int i; + int i, j; list_for_each_entry(nv, &fbn->napis, napis) { /* disable Tx Queue Triads */ @@ -500,6 +787,14 @@ void fbnic_disable(struct fbnic_net *fbn) fbnic_disable_twq0(&qt->sub0); fbnic_disable_tcq(&qt->cmpl); } + + /* disable Rx Queue Triads */ + for (j = 0; j < nv->rxt_count; j++, i++) { + struct fbnic_q_triad *qt = &nv->qt[i]; + + fbnic_disable_bdq(&qt->sub0, &qt->sub1); + fbnic_disable_rcq(&qt->cmpl); + } } wrfl(); @@ -559,6 +854,10 @@ int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail) { FBNIC_QM_TQS_IDLE(0), FBNIC_QM_TQS_IDLE_CNT, }, { FBNIC_QM_TDE_IDLE(0), FBNIC_QM_TDE_IDLE_CNT, }, { FBNIC_QM_TCQ_IDLE(0), FBNIC_QM_TCQ_IDLE_CNT, }, + }, rx[] = { + { FBNIC_QM_HPQ_IDLE(0), FBNIC_QM_HPQ_IDLE_CNT, }, + { FBNIC_QM_PPQ_IDLE(0), FBNIC_QM_PPQ_IDLE_CNT, }, + { FBNIC_QM_RCQ_IDLE(0), FBNIC_QM_RCQ_IDLE_CNT, }, }; bool idle; int err; @@ -578,6 +877,10 @@ int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail) return err; } + err = read_poll_timeout_atomic(fbnic_all_idle, idle, idle, 2, 500000, + false, fbd, rx, ARRAY_SIZE(rx)); + if (err) + fbnic_idle_dump(fbd, rx, ARRAY_SIZE(rx), "Rx", err); return err; } @@ -586,7 +889,7 @@ void fbnic_flush(struct fbnic_net *fbn) struct fbnic_napi_vector *nv; list_for_each_entry(nv, &fbn->napis, napis) { - int i; + int i, j; /* Flush any processed Tx Queue Triads and drop the rest */ for (i = 0; i < nv->txt_count; i++) { @@ -601,6 +904,38 @@ void fbnic_flush(struct fbnic_net *fbn) qt->sub0.q_idx); netdev_tx_reset_queue(tx_queue); } + + /* Flush any processed Rx Queue Triads and drop the rest */ + for (j = 0; j < nv->rxt_count; j++, i++) { + struct fbnic_q_triad *qt = &nv->qt[i]; + + /* Clean the work queues of unprocessed work */ + fbnic_clean_bdq(nv, 0, &qt->sub0, qt->sub0.tail); + fbnic_clean_bdq(nv, 0, &qt->sub1, qt->sub1.tail); + + /* Reset completion queue descriptor ring */ + memset(qt->cmpl.desc, 0, qt->cmpl.size); + + fbnic_put_pkt_buff(nv, qt->cmpl.pkt, 0); + } + } +} + +void fbnic_fill(struct fbnic_net *fbn) +{ + struct fbnic_napi_vector *nv; + + list_for_each_entry(nv, &fbn->napis, napis) { + int i, j; + + /* Populate pages in the BDQ rings to use for Rx */ + for (j = 0, i = nv->txt_count; j < nv->rxt_count; j++, i++) { + struct fbnic_q_triad *qt = &nv->qt[i]; + + /* populate the header and payload BDQs */ + fbnic_fill_bdq(nv, &qt->sub0); + fbnic_fill_bdq(nv, &qt->sub1); + } } } @@ -655,11 +990,102 @@ static void fbnic_enable_tcq(struct fbnic_napi_vector *nv, fbnic_ring_wr32(tcq, FBNIC_QUEUE_TCQ_CTL, FBNIC_QUEUE_TCQ_CTL_ENABLE); } +static void fbnic_enable_bdq(struct fbnic_ring *hpq, struct fbnic_ring *ppq) +{ + u32 bdq_ctl = FBNIC_QUEUE_BDQ_CTL_ENABLE; + u32 log_size; + + /* reset head/tail */ + fbnic_ring_wr32(hpq, FBNIC_QUEUE_BDQ_CTL, FBNIC_QUEUE_BDQ_CTL_RESET); + ppq->tail = 0; + ppq->head = 0; + hpq->tail = 0; + hpq->head = 0; + + log_size = fls(hpq->size_mask); + + /* Store descriptor ring address and size */ + fbnic_ring_wr32(hpq, FBNIC_QUEUE_BDQ_HPQ_BAL, lower_32_bits(hpq->dma)); + fbnic_ring_wr32(hpq, FBNIC_QUEUE_BDQ_HPQ_BAH, upper_32_bits(hpq->dma)); + + /* write lower 4 bits of log size as 64K ring size is 0 */ + fbnic_ring_wr32(hpq, FBNIC_QUEUE_BDQ_HPQ_SIZE, log_size & 0xf); + + if (!ppq->size_mask) + goto write_ctl; + + log_size = fls(ppq->size_mask); + + /* Add enabling of PPQ to BDQ control */ + bdq_ctl |= FBNIC_QUEUE_BDQ_CTL_PPQ_ENABLE; + + /* Store descriptor ring address and size */ + fbnic_ring_wr32(ppq, FBNIC_QUEUE_BDQ_PPQ_BAL, lower_32_bits(ppq->dma)); + fbnic_ring_wr32(ppq, FBNIC_QUEUE_BDQ_PPQ_BAH, upper_32_bits(ppq->dma)); + fbnic_ring_wr32(ppq, FBNIC_QUEUE_BDQ_PPQ_SIZE, log_size & 0xf); + +write_ctl: + fbnic_ring_wr32(hpq, FBNIC_QUEUE_BDQ_CTL, bdq_ctl); +} + +static void fbnic_config_drop_mode_rcq(struct fbnic_napi_vector *nv, + struct fbnic_ring *rcq) +{ + u32 drop_mode, rcq_ctl; + + drop_mode = FBNIC_QUEUE_RDE_CTL0_DROP_IMMEDIATE; + + /* Specify packet layout */ + rcq_ctl = FIELD_PREP(FBNIC_QUEUE_RDE_CTL0_DROP_MODE_MASK, drop_mode) | + FIELD_PREP(FBNIC_QUEUE_RDE_CTL0_MIN_HROOM_MASK, FBNIC_RX_HROOM) | + FIELD_PREP(FBNIC_QUEUE_RDE_CTL0_MIN_TROOM_MASK, FBNIC_RX_TROOM); + + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RDE_CTL0, rcq_ctl); +} + +static void fbnic_enable_rcq(struct fbnic_napi_vector *nv, + struct fbnic_ring *rcq) +{ + u32 log_size = fls(rcq->size_mask); + u32 rcq_ctl; + + fbnic_config_drop_mode_rcq(nv, rcq); + + rcq_ctl = FIELD_PREP(FBNIC_QUEUE_RDE_CTL1_PADLEN_MASK, FBNIC_RX_PAD) | + FIELD_PREP(FBNIC_QUEUE_RDE_CTL1_MAX_HDR_MASK, + FBNIC_RX_MAX_HDR) | + FIELD_PREP(FBNIC_QUEUE_RDE_CTL1_PAYLD_OFF_MASK, + FBNIC_RX_PAYLD_OFFSET) | + FIELD_PREP(FBNIC_QUEUE_RDE_CTL1_PAYLD_PG_CL_MASK, + FBNIC_RX_PAYLD_PG_CL); + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RDE_CTL1, rcq_ctl); + + /* reset head/tail */ + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RCQ_CTL, FBNIC_QUEUE_RCQ_CTL_RESET); + rcq->head = 0; + rcq->tail = 0; + + /* Store descriptor ring address and size */ + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RCQ_BAL, lower_32_bits(rcq->dma)); + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RCQ_BAH, upper_32_bits(rcq->dma)); + + /* write lower 4 bits of log size as 64K ring size is 0 */ + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RCQ_SIZE, log_size & 0xf); + + /* Store interrupt information for the completion queue */ + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RIM_CTL, nv->v_idx); + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RIM_THRESHOLD, rcq->size_mask / 2); + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RIM_MASK, 0); + + /* Enable queue */ + fbnic_ring_wr32(rcq, FBNIC_QUEUE_RCQ_CTL, FBNIC_QUEUE_RCQ_CTL_ENABLE); +} + void fbnic_enable(struct fbnic_net *fbn) { struct fbnic_dev *fbd = fbn->fbd; struct fbnic_napi_vector *nv; - int i; + int i, j; list_for_each_entry(nv, &fbn->napis, napis) { /* Setup Tx Queue Triads */ @@ -669,6 +1095,15 @@ void fbnic_enable(struct fbnic_net *fbn) fbnic_enable_twq0(&qt->sub0); fbnic_enable_tcq(nv, &qt->cmpl); } + + /* Setup Rx Queue Triads */ + for (j = 0; j < nv->rxt_count; j++, i++) { + struct fbnic_q_triad *qt = &nv->qt[i]; + + fbnic_enable_bdq(&qt->sub0, &qt->sub1); + fbnic_config_drop_mode_rcq(nv, &qt->cmpl); + fbnic_enable_rcq(nv, &qt->cmpl); + } } wrfl(); @@ -683,11 +1118,30 @@ static void fbnic_nv_irq_enable(struct fbnic_napi_vector *nv) void fbnic_napi_enable(struct fbnic_net *fbn) { + u32 irqs[FBNIC_MAX_MSIX_VECS / 32] = {}; + struct fbnic_dev *fbd = fbn->fbd; struct fbnic_napi_vector *nv; + int i; list_for_each_entry(nv, &fbn->napis, napis) { napi_enable(&nv->napi); fbnic_nv_irq_enable(nv); + + /* Record bit used for NAPI IRQs so we can + * set the mask appropriately + */ + irqs[nv->v_idx / 32] |= BIT(nv->v_idx % 32); } + + /* Force the first interrupt on the device to guarantee + * that any packets that may have been enqueued during the + * bringup are processed. + */ + for (i = 0; i < ARRAY_SIZE(irqs); i++) { + if (!irqs[i]) + continue; + wr32(FBNIC_INTR_SET(i), irqs[i]); + } + wrfl(); } diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h index 2898e0dccf7a..200f3b893d02 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h @@ -6,6 +6,7 @@ #include #include +#include struct fbnic_net; @@ -13,16 +14,43 @@ struct fbnic_net; #define FBNIC_MAX_RXQS 128u #define FBNIC_TXQ_SIZE_DEFAULT 1024 +#define FBNIC_HPQ_SIZE_DEFAULT 256 +#define FBNIC_PPQ_SIZE_DEFAULT 256 +#define FBNIC_RCQ_SIZE_DEFAULT 1024 + +#define FBNIC_RX_TROOM \ + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) +#define FBNIC_RX_HROOM \ + (ALIGN(FBNIC_RX_TROOM + NET_SKB_PAD, 128) - FBNIC_RX_TROOM) +#define FBNIC_RX_PAD 0 +#define FBNIC_RX_MAX_HDR (1536 - FBNIC_RX_PAD) +#define FBNIC_RX_PAYLD_OFFSET 0 +#define FBNIC_RX_PAYLD_PG_CL 0 #define FBNIC_RING_F_DISABLED BIT(0) #define FBNIC_RING_F_CTX BIT(1) #define FBNIC_RING_F_STATS BIT(2) /* ring's stats may be used */ +struct fbnic_pkt_buff { + struct xdp_buff buff; + u32 data_truesize; + u16 data_len; + u16 nr_frags; +}; + +#define PAGECNT_BIAS_MAX USHRT_MAX +struct fbnic_rx_buf { + struct page *page; + unsigned int pagecnt_bias; +}; + struct fbnic_ring { /* Pointer to buffer specific info */ union { void *buffer; /* Generic pointer */ void **tx_buf; /* TWQ */ + struct fbnic_pkt_buff *pkt; /* RCQ */ + struct fbnic_rx_buf *rx_buf; /* BDQ */ }; u32 __iomem *doorbell; /* pointer to CSR space for ring */ @@ -45,6 +73,7 @@ struct fbnic_q_triad { struct fbnic_napi_vector { struct napi_struct napi; struct device *dev; /* Device for DMA unmapping */ + struct page_pool *page_pool; struct fbnic_dev *fbd; char name[IFNAMSIZ + 9]; @@ -71,6 +100,7 @@ void fbnic_napi_disable(struct fbnic_net *fbn); void fbnic_enable(struct fbnic_net *fbn); void fbnic_disable(struct fbnic_net *fbn); void fbnic_flush(struct fbnic_net *fbn); +void fbnic_fill(struct fbnic_net *fbn); int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail); From patchwork Wed Apr 3 20:09:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616637 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A013715574B for ; Wed, 3 Apr 2024 20:09:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174951; cv=none; b=CoEj8UPol3VP2L/6A3kU4M04LaHR8IzL7tayZiCcACZXBNfwI+y5a4/mDz59xDcZgxDeN46DBvFnkRgouIXwzvctivJ+Q2ysysYBQMPthqUS0Q78AQE/RwGfolU2KUwSMM4sK44stcQssLiKL6kEY0QFMvAW1Je+LThOZu6cdxo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174951; c=relaxed/simple; bh=23KmmaYiqQkr0tj+OAHLJ0KuHfoP+Ff3x59fVhsQQIs=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=kTIseRsvyRI04ppAxvX7CWUozENafF0m3Kfb8qGmhjKP7apJbrKdGFFJr5BjrAo3i4X7uuYozvnrvUyMYAniZZUo+vEuYZfB8uGMKnvx483CSM6kJEMtD/iyYx5len3uY6XpJCUCR97i1opjvdbpx6IrO/H8v/Rbkgi131wXnzA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UUK05O+c; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UUK05O+c" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1e0411c0a52so1964075ad.0 for ; Wed, 03 Apr 2024 13:09:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174949; x=1712779749; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=iu0hQ67fs24G1ZXplfD2ETIFWbhf1FY58TfffN4VD/Q=; b=UUK05O+cgS16iCfBXJnlSi7aNp7Se/3UycVUBMQ1iopnsyB9wg1oK5eMtjR648Yjv6 achuI/SoyZYff2RPF2zHtbeR6B7rS397SVVvAkueBgS1fSB+VQnK8YkrviVBVyxiD8I0 9A8gihfe4cuTRHUlyGrYb72Q+drbu1L9S65ZrDuGVB2v1BmgsGCnyAuDRff9KgOiFmEg wvfaTqU7XPaNKhV23mIvPXBwX4fx8iqnXOk0jDQ0jhBfPVjGv2JgSVBVWzURXuKivEpp oicaAoGW0rX3AZ8/OpKAVMswXKsSz1ID16/yqdmbpbzZ8oIY8l4hpIg3u969Mnd9dzFB ZKhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174949; x=1712779749; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=iu0hQ67fs24G1ZXplfD2ETIFWbhf1FY58TfffN4VD/Q=; b=fsL/HgAT69WHLqKjLETg7gcmyee65n8YwZWSmjPwYgGdwu+NV3+tFCEJPjelklGEnN 1SVw90m3HqZvWt1oLlkUt4HRSyarkvc0J/PLKyTNnz9Yrvh2d1D4Zw/0AM1bIdv44tX2 7bACYv5IOv6bt/I4xpKXq2ibIVG6cP2c2OJnJx1cmT7zrVnig98BqnBb/TGJ9Bmwj1YD qSXP8KysTLKKdCOjF0Q3gEKwM4I9LIZc6UhsiUWECW6UnL9zp7HGNekK+J+ThjDV/Heo 7NKM1U7SaVHghxNC/XK8dkBBLR1QavvMGjsG3hBJ6NnPzqmok6TARWbgoyGgpBwtFpeO gAsA== X-Gm-Message-State: AOJu0YxO6PrwmBS/WOKELrUNpLOxqJc2XDYJPRlh5bqit3tvdV1lCWGM zt9fbMUu+BrjI2wBIlNEaijVJJGCT4TZpFM7nWh/SKdNTFgjBPeI X-Google-Smtp-Source: AGHT+IGo7DW7QnLAj3TZaL4niwfxNVQ2D+ywyUBY5AqD1sdn3czZdWjqQlA4HAmoTtBu3ldMgOyChg== X-Received: by 2002:a17:902:d54e:b0:1dd:de1a:bd02 with SMTP id z14-20020a170902d54e00b001ddde1abd02mr314220plf.41.1712174948794; Wed, 03 Apr 2024 13:09:08 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id bi8-20020a170902bf0800b001e0b2851db7sm13745899plb.105.2024.04.03.13.09.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:09:08 -0700 (PDT) Subject: [net-next PATCH 10/15] eth: fbnic: Add initial messaging to notify FW of our presence From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:09:07 -0700 Message-ID: <171217494704.1598374.3204376739007363054.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck After the driver loads we need to get some intitial capabilities from the firmware to determine what the device is capable of and what functionality needs to be enabled. Specifically we receive information about the current state of the link and if a BMC is present. After that when we bring the interface up we will need the ability to take ownership from the FW. To do that we will need to notify it that we are taking control before we start configuring the traffic classifier and MAC. Once we have ownership we need to notify the firmware that we are still present and active. To do that we will send a regular heartbeat to the FW. If the FW doesn't receive the heartbeat in a timely fashion it will retake control of the RPC and MAC and assume that the host has gone offline. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/fbnic.h | 5 drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 8 drivers/net/ethernet/meta/fbnic/fbnic_fw.c | 409 ++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_fw.h | 85 +++++ drivers/net/ethernet/meta/fbnic/fbnic_netdev.c | 18 + drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 28 ++ 6 files changed, 553 insertions(+) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic.h b/drivers/net/ethernet/meta/fbnic/fbnic.h index 92a36959547c..4f18d703dae8 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic.h @@ -25,9 +25,14 @@ struct fbnic_dev { struct delayed_work service_task; struct fbnic_fw_mbx mbx[FBNIC_IPC_MBX_INDICES]; + struct fbnic_fw_cap fw_cap; /* Lock protecting Tx Mailbox queue to prevent possible races */ spinlock_t fw_tx_lock; + unsigned long last_heartbeat_request; + unsigned long last_heartbeat_response; + u8 fw_heartbeat_enabled; + u64 dsn; u32 mps; u32 readrq; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index 33832a4f78ea..8b035c4e068e 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -12,6 +12,14 @@ #define DESC_BIT(nr) BIT_ULL(nr) #define DESC_GENMASK(h, l) GENMASK_ULL(h, l) +/* Defines the minimum firmware version required by the driver */ +#define MIN_FW_MAJOR_VERSION 0 +#define MIN_FW_MINOR_VERSION 10 +#define MIN_FW_BUILD_VERSION 6 +#define MIN_FW_VERSION_CODE (MIN_FW_MAJOR_VERSION * (1u << 24) + \ + MIN_FW_MINOR_VERSION * (1u << 16) + \ + MIN_FW_BUILD_VERSION) + #define PCI_DEVICE_ID_META_FBNIC_ASIC 0x0013 #define FBNIC_CLOCK_FREQ (600 * (1000 * 1000)) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c index 71647044aa23..4c3098364fed 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c @@ -2,6 +2,7 @@ /* Copyright (c) Meta Platforms, Inc. and affiliates. */ #include +#include #include #include #include @@ -184,6 +185,22 @@ static int fbnic_mbx_alloc_rx_msgs(struct fbnic_dev *fbd) return err; } +static int fbnic_mbx_map_tlv_msg(struct fbnic_dev *fbd, + struct fbnic_tlv_msg *msg) +{ + unsigned long flags; + int err; + + spin_lock_irqsave(&fbd->fw_tx_lock, flags); + + err = fbnic_mbx_map_msg(fbd, FBNIC_IPC_MBX_TX_IDX, msg, + le16_to_cpu(msg->hdr.len) * sizeof(u32), 1); + + spin_unlock_irqrestore(&fbd->fw_tx_lock, flags); + + return err; +} + static void fbnic_mbx_process_tx_msgs(struct fbnic_dev *fbd) { struct fbnic_fw_mbx *tx_mbx = &fbd->mbx[FBNIC_IPC_MBX_TX_IDX]; @@ -205,6 +222,60 @@ static void fbnic_mbx_process_tx_msgs(struct fbnic_dev *fbd) tx_mbx->head = head; } +/** + * fbnic_fw_xmit_simple_msg - Transmit a simple single TLV message w/o data + * @fbd: FBNIC device structure + * @msg_type: ENUM value indicating message type to send + * + * Returns the following values: + * -EOPNOTSUPP: Is not ASIC so mailbox is not supported + * -ENODEV: Device I/O error + * -ENOMEM: Failed to allocate message + * -EBUSY: No space in mailbox + * -ENOSPC: DMA mapping failed + * + * This function sends a single TLV header indicating the host wants to take + * some action. However there are no other side effects which means that any + * response will need to be caught via a completion if this action is + * expected to kick off a resultant action. + */ +static int fbnic_fw_xmit_simple_msg(struct fbnic_dev *fbd, u32 msg_type) +{ + struct fbnic_tlv_msg *msg; + int err = 0; + + if (!fbnic_fw_present(fbd)) + return -ENODEV; + + msg = fbnic_tlv_msg_alloc(msg_type); + if (!msg) + return -ENOMEM; + + err = fbnic_mbx_map_tlv_msg(fbd, msg); + if (err) + free_page((unsigned long)msg); + + return err; +} + +/** + * fbnic_fw_xmit_cap_msg - Allocate and populate a FW capabilities message + * @fbd: FBNIC device structure + * + * Returns NULL on failure to allocate, error pointer on error, or pointer + * to new TLV test message. + * + * Sends a single TLV header indicating the host wants the firmware to + * confirm the capabilities and version. + **/ +static int fbnic_fw_xmit_cap_msg(struct fbnic_dev *fbd) +{ + int err = fbnic_fw_xmit_simple_msg(fbd, FBNIC_TLV_MSG_ID_HOST_CAP_REQ); + + /* return 0 if we are not calling this on ASIC */ + return (err == -EOPNOTSUPP) ? 0 : err; +} + static void fbnic_mbx_postinit_desc_ring(struct fbnic_dev *fbd, int mbx_idx) { struct fbnic_fw_mbx *mbx = &fbd->mbx[mbx_idx]; @@ -220,6 +291,16 @@ static void fbnic_mbx_postinit_desc_ring(struct fbnic_dev *fbd, int mbx_idx) /* Make sure we have a page for the FW to write to */ fbnic_mbx_alloc_rx_msgs(fbd); break; + case FBNIC_IPC_MBX_TX_IDX: + /* Force version to 1 if we successfully requested an update + * from the firmware. This should be overwritten once we get + * the actual version from the firmware in the capabilities + * request message. + */ + if (!fbnic_fw_xmit_cap_msg(fbd) && + !fbd->fw_cap.running.mgmt.version) + fbd->fw_cap.running.mgmt.version = 1; + break; } } @@ -240,7 +321,335 @@ static void fbnic_mbx_postinit(struct fbnic_dev *fbd) fbnic_mbx_postinit_desc_ring(fbd, i); } +/** + * fbnic_fw_xmit_ownership_msg - Create and transmit a host ownership message + * to FW mailbox + * + * @fbd: FBNIC device structure + * @take_ownership: take/release the ownership + * + * Returns 0 on success, negative value on failure + * + * Notifies the firmware that the driver either takes ownership of the NIC + * (when @take_ownership is true) or releases it. + */ +int fbnic_fw_xmit_ownership_msg(struct fbnic_dev *fbd, bool take_ownership) +{ + unsigned long req_time = jiffies; + struct fbnic_tlv_msg *msg; + int err = 0; + + if (!fbnic_fw_present(fbd)) + return -ENODEV; + + msg = fbnic_tlv_msg_alloc(FBNIC_TLV_MSG_ID_OWNERSHIP_REQ); + if (!msg) + return -ENOMEM; + + if (take_ownership) { + err = fbnic_tlv_attr_put_flag(msg, FBNIC_FW_OWNERSHIP_FLAG); + if (err) + goto free_message; + } + + err = fbnic_mbx_map_tlv_msg(fbd, msg); + if (err) + goto free_message; + + /* Initialize heartbeat, set last response to 1 second in the past + * so that we will trigger a timeout if the firmware doesn't respond + */ + fbd->last_heartbeat_response = req_time - HZ; + + fbd->last_heartbeat_request = req_time; + + /* set heartbeat detection based on if we are taking ownership */ + fbd->fw_heartbeat_enabled = take_ownership; + + return err; + +free_message: + free_page((unsigned long)msg); + return err; +} + +static const struct fbnic_tlv_index fbnic_fw_cap_resp_index[] = { + FBNIC_TLV_ATTR_U32(FBNIC_FW_CAP_RESP_VERSION), + FBNIC_TLV_ATTR_FLAG(FBNIC_FW_CAP_RESP_BMC_PRESENT), + FBNIC_TLV_ATTR_MAC_ADDR(FBNIC_FW_CAP_RESP_BMC_MAC_ADDR), + FBNIC_TLV_ATTR_ARRAY(FBNIC_FW_CAP_RESP_BMC_MAC_ARRAY), + FBNIC_TLV_ATTR_U32(FBNIC_FW_CAP_RESP_STORED_VERSION), + FBNIC_TLV_ATTR_U32(FBNIC_FW_CAP_RESP_ACTIVE_FW_SLOT), + FBNIC_TLV_ATTR_STRING(FBNIC_FW_CAP_RESP_VERSION_COMMIT_STR, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE), + FBNIC_TLV_ATTR_U32(FBNIC_FW_CAP_RESP_BMC_ALL_MULTI), + FBNIC_TLV_ATTR_U32(FBNIC_FW_CAP_RESP_FW_LINK_SPEED), + FBNIC_TLV_ATTR_U32(FBNIC_FW_CAP_RESP_FW_LINK_FEC), + FBNIC_TLV_ATTR_STRING(FBNIC_FW_CAP_RESP_STORED_COMMIT_STR, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE), + FBNIC_TLV_ATTR_U32(FBNIC_FW_CAP_RESP_CMRT_VERSION), + FBNIC_TLV_ATTR_U32(FBNIC_FW_CAP_RESP_STORED_CMRT_VERSION), + FBNIC_TLV_ATTR_STRING(FBNIC_FW_CAP_RESP_CMRT_COMMIT_STR, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE), + FBNIC_TLV_ATTR_STRING(FBNIC_FW_CAP_RESP_STORED_CMRT_COMMIT_STR, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE), + FBNIC_TLV_ATTR_U32(FBNIC_FW_CAP_RESP_UEFI_VERSION), + FBNIC_TLV_ATTR_STRING(FBNIC_FW_CAP_RESP_UEFI_COMMIT_STR, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE), + FBNIC_TLV_ATTR_LAST +}; + +static int fbnic_fw_parse_bmc_addrs(u8 bmc_mac_addr[][ETH_ALEN], + struct fbnic_tlv_msg *attr, int len) +{ + int attr_len = le16_to_cpu(attr->hdr.len) / sizeof(u32) - 1; + struct fbnic_tlv_msg *mac_results[8]; + int err, i = 0; + + /* make sure we have enough room to process all the MAC addresses */ + if (len > 8) + return -ENOSPC; + + /* Parse the array */ + err = fbnic_tlv_attr_parse_array(&attr[1], attr_len, mac_results, + fbnic_fw_cap_resp_index, + FBNIC_FW_CAP_RESP_BMC_MAC_ADDR, len); + if (err) + return err; + + /* Copy results into MAC addr array */ + for (i = 0; i < len && mac_results[i]; i++) + fbnic_tlv_attr_addr_copy(bmc_mac_addr[i], mac_results[i]); + + /* Zero remaining unused addresses */ + while (i < len) + eth_zero_addr(bmc_mac_addr[i++]); + + return 0; +} + +static int fbnic_fw_parse_cap_resp(void *opaque, struct fbnic_tlv_msg **results) +{ + u32 active_slot = 0, all_multi = 0; + struct fbnic_dev *fbd = opaque; + u32 speed = 0, fec = 0; + size_t commit_size = 0; + bool bmc_present; + int err; + + get_unsigned_result(FBNIC_FW_CAP_RESP_VERSION, + fbd->fw_cap.running.mgmt.version); + + if (!fbd->fw_cap.running.mgmt.version) + return -EINVAL; + + if (fbd->fw_cap.running.mgmt.version < MIN_FW_VERSION_CODE) { + char running_ver[FBNIC_FW_VER_MAX_SIZE]; + + fbnic_mk_fw_ver_str(fbd->fw_cap.running.mgmt.version, + running_ver); + dev_err(fbd->dev, "Device firmware version(%s) is older than minimum required version(%02d.%02d.%02d)\n", + running_ver, + MIN_FW_MAJOR_VERSION, + MIN_FW_MINOR_VERSION, + MIN_FW_BUILD_VERSION); + /* Disable TX mailbox to prevent card use until firmware is + * updated. + */ + fbd->mbx[FBNIC_IPC_MBX_TX_IDX].ready = false; + return -EINVAL; + } + + get_string_result(FBNIC_FW_CAP_RESP_VERSION_COMMIT_STR, commit_size, + fbd->fw_cap.running.mgmt.commit, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE); + if (!commit_size) + dev_warn(fbd->dev, "Firmware did not send mgmt commit!\n"); + + get_unsigned_result(FBNIC_FW_CAP_RESP_STORED_VERSION, + fbd->fw_cap.stored.mgmt.version); + get_string_result(FBNIC_FW_CAP_RESP_STORED_COMMIT_STR, commit_size, + fbd->fw_cap.stored.mgmt.commit, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE); + + get_unsigned_result(FBNIC_FW_CAP_RESP_CMRT_VERSION, + fbd->fw_cap.running.bootloader.version); + get_string_result(FBNIC_FW_CAP_RESP_CMRT_COMMIT_STR, commit_size, + fbd->fw_cap.running.bootloader.commit, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE); + + get_unsigned_result(FBNIC_FW_CAP_RESP_STORED_CMRT_VERSION, + fbd->fw_cap.stored.bootloader.version); + get_string_result(FBNIC_FW_CAP_RESP_STORED_CMRT_COMMIT_STR, commit_size, + fbd->fw_cap.stored.bootloader.commit, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE); + + get_unsigned_result(FBNIC_FW_CAP_RESP_UEFI_VERSION, + fbd->fw_cap.stored.undi.version); + get_string_result(FBNIC_FW_CAP_RESP_UEFI_COMMIT_STR, commit_size, + fbd->fw_cap.stored.undi.commit, + FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE); + + get_unsigned_result(FBNIC_FW_CAP_RESP_ACTIVE_FW_SLOT, active_slot); + fbd->fw_cap.active_slot = active_slot; + + get_unsigned_result(FBNIC_FW_CAP_RESP_FW_LINK_SPEED, speed); + get_unsigned_result(FBNIC_FW_CAP_RESP_FW_LINK_FEC, fec); + fbd->fw_cap.link_speed = speed; + fbd->fw_cap.link_fec = fec; + + bmc_present = !!results[FBNIC_FW_CAP_RESP_BMC_PRESENT]; + if (bmc_present) { + struct fbnic_tlv_msg *attr; + + attr = results[FBNIC_FW_CAP_RESP_BMC_MAC_ARRAY]; + if (!attr) + return -EINVAL; + + err = fbnic_fw_parse_bmc_addrs(fbd->fw_cap.bmc_mac_addr, + attr, 4); + if (err) + return err; + + get_unsigned_result(FBNIC_FW_CAP_RESP_BMC_ALL_MULTI, all_multi); + } else { + memset(fbd->fw_cap.bmc_mac_addr, 0, + sizeof(fbd->fw_cap.bmc_mac_addr)); + } + + fbd->fw_cap.bmc_present = bmc_present; + + if (results[FBNIC_FW_CAP_RESP_BMC_ALL_MULTI] || !bmc_present) + fbd->fw_cap.all_multi = all_multi; + + return 0; +} + +static const struct fbnic_tlv_index fbnic_ownership_resp_index[] = { + FBNIC_TLV_ATTR_LAST +}; + +static int fbnic_fw_parse_ownership_resp(void *opaque, + struct fbnic_tlv_msg **results) +{ + struct fbnic_dev *fbd = (struct fbnic_dev *)opaque; + + /* Count the ownership response as a heartbeat reply */ + fbd->last_heartbeat_response = jiffies; + + return 0; +} + +static const struct fbnic_tlv_index fbnic_heartbeat_resp_index[] = { + FBNIC_TLV_ATTR_LAST +}; + +static int fbnic_fw_parse_heartbeat_resp(void *opaque, + struct fbnic_tlv_msg **results) +{ + struct fbnic_dev *fbd = (struct fbnic_dev *)opaque; + + fbd->last_heartbeat_response = jiffies; + + return 0; +} + +static int fbnic_fw_xmit_heartbeat_message(struct fbnic_dev *fbd) +{ + unsigned long req_time = jiffies; + struct fbnic_tlv_msg *msg; + int err = 0; + + if (!fbnic_fw_present(fbd)) + return -ENODEV; + + msg = fbnic_tlv_msg_alloc(FBNIC_TLV_MSG_ID_HEARTBEAT_REQ); + if (!msg) + return -ENOMEM; + + err = fbnic_mbx_map_tlv_msg(fbd, msg); + if (err) + goto free_message; + + fbd->last_heartbeat_request = req_time; + + return err; + +free_message: + free_page((unsigned long)msg); + return err; +} + +static bool fbnic_fw_heartbeat_current(struct fbnic_dev *fbd) +{ + unsigned long last_response = fbd->last_heartbeat_response; + unsigned long last_request = fbd->last_heartbeat_request; + + return !time_before(last_response, last_request); +} + +int fbnic_fw_init_heartbeat(struct fbnic_dev *fbd, bool poll) +{ + int err = -ETIMEDOUT; + int attempts = 50; + + if (!fbnic_fw_present(fbd)) + return -ENODEV; + + while (attempts--) { + msleep(200); + if (poll) + fbnic_mbx_poll(fbd); + + if (!fbnic_fw_heartbeat_current(fbd)) + continue; + + /* Place new message on mailbox to elicit a response */ + err = fbnic_fw_xmit_heartbeat_message(fbd); + if (err) + dev_warn(fbd->dev, + "Failed to send heartbeat message\n"); + break; + } + + return err; +} + +void fbnic_fw_check_heartbeat(struct fbnic_dev *fbd) +{ + unsigned long last_request = fbd->last_heartbeat_request; + int err; + + /* Do not check heartbeat or send another request until current + * period has expired. Otherwise we might start spamming requests. + */ + if (time_is_after_jiffies(last_request + FW_HEARTBEAT_PERIOD)) + return; + + /* We already reported no mailbox. Wait for it to come back */ + if (!fbd->fw_heartbeat_enabled) + return; + + /* Was the last heartbeat response long time ago? */ + if (!fbnic_fw_heartbeat_current(fbd)) { + dev_warn(fbd->dev, + "Firmware did not respond to heartbeat message\n"); + fbd->fw_heartbeat_enabled = false; + } + + /* Place new message on mailbox to elicit a response */ + err = fbnic_fw_xmit_heartbeat_message(fbd); + if (err) + dev_warn(fbd->dev, "Failed to send heartbeat message\n"); +} + static const struct fbnic_tlv_parser fbnic_fw_tlv_parser[] = { + FBNIC_TLV_PARSER(FW_CAP_RESP, fbnic_fw_cap_resp_index, + fbnic_fw_parse_cap_resp), + FBNIC_TLV_PARSER(OWNERSHIP_RESP, fbnic_ownership_resp_index, + fbnic_fw_parse_ownership_resp), + FBNIC_TLV_PARSER(HEARTBEAT_RESP, fbnic_heartbeat_resp_index, + fbnic_fw_parse_heartbeat_resp), FBNIC_TLV_MSG_ERROR }; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h index c143079f881c..40d314f963ea 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h @@ -4,6 +4,7 @@ #ifndef _FBNIC_FW_H_ #define _FBNIC_FW_H_ +#include #include struct fbnic_dev; @@ -17,10 +18,94 @@ struct fbnic_fw_mbx { } buf_info[FBNIC_IPC_MBX_DESC_LEN]; }; +// FW_VER_MAX_SIZE must match ETHTOOL_FWVERS_LEN +#define FBNIC_FW_VER_MAX_SIZE 32 +// Formatted version is in the format XX.YY.ZZ_RRR_COMMIT +#define FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE (FBNIC_FW_VER_MAX_SIZE - 13) +#define FBNIC_FW_LOG_MAX_SIZE 256 + +struct fbnic_fw_ver { + u32 version; + char commit[FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE]; +}; + +struct fbnic_fw_cap { + struct { + struct fbnic_fw_ver mgmt, bootloader; + } running; + struct { + struct fbnic_fw_ver mgmt, bootloader, undi; + } stored; + u8 active_slot; + u8 bmc_mac_addr[4][ETH_ALEN]; + u8 bmc_present : 1; + u8 all_multi : 1; + u8 link_speed; + u8 link_fec; +}; + void fbnic_mbx_init(struct fbnic_dev *fbd); void fbnic_mbx_clean(struct fbnic_dev *fbd); void fbnic_mbx_poll(struct fbnic_dev *fbd); int fbnic_mbx_poll_tx_ready(struct fbnic_dev *fbd); void fbnic_mbx_flush_tx(struct fbnic_dev *fbd); +int fbnic_fw_xmit_ownership_msg(struct fbnic_dev *fbd, bool take_ownership); +int fbnic_fw_init_heartbeat(struct fbnic_dev *fbd, bool poll); +void fbnic_fw_check_heartbeat(struct fbnic_dev *fbd); + +#define fbnic_mk_full_fw_ver_str(_rev_id, _delim, _commit, _str) \ +do { \ + const u32 __rev_id = _rev_id; \ + snprintf(_str, sizeof(_str), "%02lu.%02lu.%02lu-%03lu%s%s", \ + FIELD_GET(FBNIC_FW_CAP_RESP_VERSION_MAJOR, __rev_id), \ + FIELD_GET(FBNIC_FW_CAP_RESP_VERSION_MINOR, __rev_id), \ + FIELD_GET(FBNIC_FW_CAP_RESP_VERSION_PATCH, __rev_id), \ + FIELD_GET(FBNIC_FW_CAP_RESP_VERSION_BUILD, __rev_id), \ + _delim, _commit); \ +} while (0) +#define fbnic_mk_fw_ver_str(_rev_id, _str) \ + fbnic_mk_full_fw_ver_str(_rev_id, "", "", _str) + +#define FW_HEARTBEAT_PERIOD (10 * HZ) + +enum { + FBNIC_TLV_MSG_ID_HOST_CAP_REQ = 0x10, + FBNIC_TLV_MSG_ID_FW_CAP_RESP = 0x11, + FBNIC_TLV_MSG_ID_OWNERSHIP_REQ = 0x12, + FBNIC_TLV_MSG_ID_OWNERSHIP_RESP = 0x13, + FBNIC_TLV_MSG_ID_HEARTBEAT_REQ = 0x14, + FBNIC_TLV_MSG_ID_HEARTBEAT_RESP = 0x15, +}; + +#define FBNIC_FW_CAP_RESP_VERSION_MAJOR CSR_GENMASK(31, 24) +#define FBNIC_FW_CAP_RESP_VERSION_MINOR CSR_GENMASK(23, 16) +#define FBNIC_FW_CAP_RESP_VERSION_PATCH CSR_GENMASK(15, 8) +#define FBNIC_FW_CAP_RESP_VERSION_BUILD CSR_GENMASK(7, 0) +enum { + FBNIC_FW_CAP_RESP_VERSION = 0x0, + FBNIC_FW_CAP_RESP_BMC_PRESENT = 0x1, + FBNIC_FW_CAP_RESP_BMC_MAC_ADDR = 0x2, + FBNIC_FW_CAP_RESP_BMC_MAC_ARRAY = 0x3, + FBNIC_FW_CAP_RESP_STORED_VERSION = 0x4, + FBNIC_FW_CAP_RESP_ACTIVE_FW_SLOT = 0x5, + FBNIC_FW_CAP_RESP_VERSION_COMMIT_STR = 0x6, + FBNIC_FW_CAP_RESP_BMC_ALL_MULTI = 0x8, + FBNIC_FW_CAP_RESP_FW_STATE = 0x9, + FBNIC_FW_CAP_RESP_FW_LINK_SPEED = 0xa, + FBNIC_FW_CAP_RESP_FW_LINK_FEC = 0xb, + FBNIC_FW_CAP_RESP_STORED_COMMIT_STR = 0xc, + FBNIC_FW_CAP_RESP_CMRT_VERSION = 0xd, + FBNIC_FW_CAP_RESP_STORED_CMRT_VERSION = 0xe, + FBNIC_FW_CAP_RESP_CMRT_COMMIT_STR = 0xf, + FBNIC_FW_CAP_RESP_STORED_CMRT_COMMIT_STR = 0x10, + FBNIC_FW_CAP_RESP_UEFI_VERSION = 0x11, + FBNIC_FW_CAP_RESP_UEFI_COMMIT_STR = 0x12, + FBNIC_FW_CAP_RESP_MSG_MAX +}; + +enum { + FBNIC_FW_OWNERSHIP_FLAG = 0x0, + FBNIC_FW_OWNERSHIP_MSG_MAX +}; #endif /* _FBNIC_FW_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c index 171b159cc006..bbc2f21060dc 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c @@ -11,6 +11,7 @@ int __fbnic_open(struct fbnic_net *fbn) { + struct fbnic_dev *fbd = fbn->fbd; int err; err = fbnic_alloc_napi_vectors(fbn); @@ -31,7 +32,22 @@ int __fbnic_open(struct fbnic_net *fbn) if (err) goto free_resources; + /* Send ownership message and flush to verify FW has seen it */ + err = fbnic_fw_xmit_ownership_msg(fbd, true); + if (err) { + dev_warn(fbd->dev, + "Error %d sending host ownership message to the firmware\n", + err); + goto free_resources; + } + + err = fbnic_fw_init_heartbeat(fbd, false); + if (err) + goto release_ownership; + return 0; +release_ownership: + fbnic_fw_xmit_ownership_msg(fbn->fbd, false); free_resources: fbnic_free_resources(fbn); free_napi_vectors: @@ -57,6 +73,8 @@ static int fbnic_stop(struct net_device *netdev) fbnic_down(fbn); + fbnic_fw_xmit_ownership_msg(fbn->fbd, false); + fbnic_free_resources(fbn); fbnic_free_napi_vectors(fbn); diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index d6598c81a5f9..8408f0d5f54a 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -160,6 +160,30 @@ void fbnic_down(struct fbnic_net *fbn) fbnic_flush(fbn); } +static void fbnic_health_check(struct fbnic_dev *fbd) +{ + struct fbnic_fw_mbx *tx_mbx = &fbd->mbx[FBNIC_IPC_MBX_TX_IDX]; + + /* As long as the heart is beating the FW is healty */ + if (fbd->fw_heartbeat_enabled) + return; + + /* If the Tx mailbox still has messages sitting in it then there likely + * isn't anything we can do. We will wait until the mailbox is empty to + * report the fault so we can collect the crashlog. + */ + if (tx_mbx->head != tx_mbx->tail) + return; + + /* TBD: Need to add a more thorough recovery here. + * Specifically I need to verify what all the firmware will have + * changed since we had setup and it rebooted. May just need to + * perform a down/up. For now we will just reclaim ownership so + * the heartbeat can catch the next fault. + */ + fbnic_fw_xmit_ownership_msg(fbd, true); +} + static void fbnic_service_task(struct work_struct *work) { struct fbnic_dev *fbd = container_of(to_delayed_work(work), @@ -167,6 +191,10 @@ static void fbnic_service_task(struct work_struct *work) rtnl_lock(); + fbnic_fw_check_heartbeat(fbd); + + fbnic_health_check(fbd); + if (netif_running(fbd->netdev)) schedule_delayed_work(&fbd->service_task, HZ); From patchwork Wed Apr 3 20:09:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616638 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F484155736 for ; Wed, 3 Apr 2024 20:09:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174957; cv=none; b=aFAz3VojDPk+C9tJi7mXUSULPRNCPFDtqRNCJPhugOAgbBFqLzxSLYc8214w+/W3MKwc/EHLPXrDgmTkXlYXBp55nA3TU6WXy4IGH72iMxByeVgtFV9kFXSPz1unWGpEz8diTs1Gc9Ca64J8yRhMECH7gNYfu+ThKrzPN857RxE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174957; c=relaxed/simple; bh=2yEbbdRjAfyLv6AbyoYgtGGTERBEMWLq6ZGKcDDajD0=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=LpyRV/uFPZX7u9aAMoog93VyVj6OVd1wI3lIcdaacN9sNKrnF+yXFRWbKRFuOe0fFGfyg+8h1CmEAdZtXPUm/khloNr5VUyebAG1t9mer2IyzaQ3xECBBsk4rfTOT5Cpt7mCZJn/b0F0xxyWap4QbBv2f6cptCYgLVVBEM73GI0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Es9+sJvh; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Es9+sJvh" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6e6bee809b8so193696b3a.1 for ; Wed, 03 Apr 2024 13:09:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174954; x=1712779754; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=elSOqNAygeLugyGUtm84X7lmma0y5sNgZID7faN/js0=; b=Es9+sJvhjuypnu+3z7QCfoqP0+RH4s/d04aHracrch2o7GTwFUczmOYGQvUQ1OD8NN Ez/HRp5A/xtqNdFGcNl0V4SAmiDCanpHbqLJ2ifM2rfgY1X6mCXYVNVPlTFyewFxl4Pk xR2UZtDmdQryxICjbI2EC2WgmTRIbi1bR1sqkh0x5OVBXcXeJjD3euWmyhi3YoTxd5N6 54rr+XZGT0dBLr/dmKaTRBZCZiEY6A6spi5wIxZC5bPDWiueowtGRbLtVT7NmUJUtIL5 bIRBTVLzOAnQbQJddxKWZzS882Nu184b8mNCl/1WeRZ+hoe//Ljrc7lC2Diy2zmpYawJ viUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174954; x=1712779754; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=elSOqNAygeLugyGUtm84X7lmma0y5sNgZID7faN/js0=; b=OL86+rovQxJxTaJ31OfMX5mi35zokNpiR9KLGWHTNh51xCKXgeJE6K31JOKSdj6d+O rIVY9n4NIFz+1gJ/zZXGT6NtJ10w96RgniVxnXJAVQvmKlw4cOwTY4wnIC84JVnYfTLt mMSQEfnQ1yydilRw1AaguBDZYBxUQERrIPZ95ZbvJzxDf9iQH73TF3qJ6nmGjK1jp+L6 OiWCYCaPUigGSIfRXY0PKcrHr54S9h881oZMzhl11+2EgQJWrrX4RhRMoQMrOCRqZHwu GA6nT6YmBtpuUBU5qxu1Sn+9kpt2Cpl+8+USeNkiTbrDFBXTJ4U6oR72eVx+GZioQxNa jnjA== X-Gm-Message-State: AOJu0YyPaXOBsOfcxLMkyWiG3niAT/d0r5F/6jUcr5UrMNnXPcooEL1W Rv8FEAMVM8BOVNJyRDjb/WmFt02QTWncivTMytnxnLGz8Hl7/Ssi X-Google-Smtp-Source: AGHT+IHHT7jtoIo7mjvRfMdSfmXaj0upPN39lyrpOm5zpz8/jlMbcffkeKCt886N3+dI6wcDdwDy6A== X-Received: by 2002:a05:6a21:2b14:b0:1a3:e0b5:418 with SMTP id ss20-20020a056a212b1400b001a3e0b50418mr616381pzb.36.1712174953369; Wed, 03 Apr 2024 13:09:13 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id y44-20020a056a0003ac00b006eadc87233dsm11864302pfs.165.2024.04.03.13.09.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:09:12 -0700 (PDT) Subject: [net-next PATCH 11/15] eth: fbnic: Enable Ethernet link setup From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:09:11 -0700 Message-ID: <171217495098.1598374.12824051034972793514.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Add the logic needed to enable the Ethernet link to be configured and the link to be detected. We have to partially rely on the FW for this as there are parts of the MAC configuration that are shared between multiple ports so we ask the firmware to complete those pieces on our behalf. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/fbnic.h | 18 + drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 143 ++++++ drivers/net/ethernet/meta/fbnic/fbnic_fw.c | 60 ++ drivers/net/ethernet/meta/fbnic/fbnic_fw.h | 22 + drivers/net/ethernet/meta/fbnic/fbnic_irq.c | 118 +++++ drivers/net/ethernet/meta/fbnic/fbnic_mac.c | 587 ++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_mac.h | 58 ++ drivers/net/ethernet/meta/fbnic/fbnic_netdev.c | 12 drivers/net/ethernet/meta/fbnic/fbnic_netdev.h | 7 drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 73 +++ drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 21 + drivers/net/ethernet/meta/fbnic/fbnic_txrx.h | 1 12 files changed, 1119 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic.h b/drivers/net/ethernet/meta/fbnic/fbnic.h index 4f18d703dae8..202f005e1cfd 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic.h @@ -20,6 +20,7 @@ struct fbnic_dev { const struct fbnic_mac *mac; struct msix_entry *msix_entries; unsigned int fw_msix_vector; + unsigned int mac_msix_vector; unsigned short num_irqs; struct delayed_work service_task; @@ -37,6 +38,13 @@ struct fbnic_dev { u32 mps; u32 readrq; + /* Tri-state value indicating state of link. + * 0 - Up + * 1 - Down + * 2 - Event - Requires checking as link state may have changed + */ + s8 link_state; + /* Number of TCQs/RCQs available on hardware */ u16 max_num_queues; }; @@ -48,6 +56,7 @@ struct fbnic_dev { */ enum { FBNIC_FW_MSIX_ENTRY, + FBNIC_MAC_MSIX_ENTRY, FBNIC_NON_NAPI_VECTORS }; @@ -89,6 +98,11 @@ void fbnic_fw_wr32(struct fbnic_dev *fbd, u32 reg, u32 val); #define fw_wr32(reg, val) fbnic_fw_wr32(fbd, reg, val) #define fw_wrfl() fbnic_fw_rd32(fbd, FBNIC_FW_ZERO_REG) +static inline bool fbnic_bmc_present(struct fbnic_dev *fbd) +{ + return fbd->fw_cap.bmc_present; +} + static inline bool fbnic_init_failure(struct fbnic_dev *fbd) { return !fbd->netdev; @@ -104,6 +118,10 @@ void fbnic_devlink_unregister(struct fbnic_dev *fbd); int fbnic_fw_enable_mbx(struct fbnic_dev *fbd); void fbnic_fw_disable_mbx(struct fbnic_dev *fbd); +int fbnic_mac_get_link(struct fbnic_dev *fbd, bool *link); +int fbnic_mac_enable(struct fbnic_dev *fbd); +void fbnic_mac_disable(struct fbnic_dev *fbd); + void fbnic_free_irqs(struct fbnic_dev *fbd); int fbnic_alloc_irqs(struct fbnic_dev *fbd); diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index 8b035c4e068e..39c98d2dce12 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -58,6 +58,10 @@ #define FBNIC_INTR_MSIX_CTRL(n) (0x00040 + (n)) /* 0x00100 + 4*n */ #define FBNIC_INTR_MSIX_CTRL_VECTOR_MASK CSR_GENMASK(7, 0) #define FBNIC_INTR_MSIX_CTRL_ENABLE CSR_BIT(31) +enum { + FBNIC_INTR_MSIX_CTRL_MAC_IDX = 6, + FBNIC_INTR_MSIX_CTRL_PCS_IDX = 34, +}; #define FBNIC_CSR_END_INTR 0x0005f /* CSR section delimiter */ @@ -392,6 +396,145 @@ enum { #define FBNIC_MASTER_SPARE_0 0x0C41B /* 0x3106c */ #define FBNIC_CSR_END_MASTER 0x0C41E /* CSR section delimiter */ +/* MAC PCS registers */ +#define FBNIC_CSR_START_PCS 0x10000 /* CSR section delimiter */ +#define FBNIC_PCS_CONTROL1_0 0x10000 /* 0x40000 */ +#define FBNIC_PCS_CONTROL1_RESET CSR_BIT(15) +#define FBNIC_PCS_CONTROL1_LOOPBACK CSR_BIT(14) +#define FBNIC_PCS_CONTROL1_SPEED_SELECT_ALWAYS CSR_BIT(13) +#define FBNIC_PCS_CONTROL1_SPEED_ALWAYS CSR_BIT(6) +#define FBNIC_PCS_VENDOR_VL_INTVL_0 0x10202 /* 0x40808 */ +#define FBNIC_PCS_VL0_0_CHAN_0 0x10208 /* 0x40820 */ +#define FBNIC_PCS_VL0_1_CHAN_0 0x10209 /* 0x40824 */ +#define FBNIC_PCS_VL1_0_CHAN_0 0x1020a /* 0x40828 */ +#define FBNIC_PCS_VL1_1_CHAN_0 0x1020b /* 0x4082c */ +#define FBNIC_PCS_VL2_0_CHAN_0 0x1020c /* 0x40830 */ +#define FBNIC_PCS_VL2_1_CHAN_0 0x1020d /* 0x40834 */ +#define FBNIC_PCS_VL3_0_CHAN_0 0x1020e /* 0x40838 */ +#define FBNIC_PCS_VL3_1_CHAN_0 0x1020f /* 0x4083c */ +#define FBNIC_PCS_MODE_VL_CHAN_0 0x10210 /* 0x40840 */ +#define FBNIC_PCS_MODE_HI_BER25 CSR_BIT(2) +#define FBNIC_PCS_MODE_DISABLE_MLD CSR_BIT(1) +#define FBNIC_PCS_MODE_ENA_CLAUSE49 CSR_BIT(0) +#define FBNIC_PCS_CONTROL1_1 0x10400 /* 0x41000 */ +#define FBNIC_PCS_VENDOR_VL_INTVL_1 0x10602 /* 0x41808 */ +#define FBNIC_PCS_VL0_0_CHAN_1 0x10608 /* 0x41820 */ +#define FBNIC_PCS_VL0_1_CHAN_1 0x10609 /* 0x41824 */ +#define FBNIC_PCS_VL1_0_CHAN_1 0x1060a /* 0x41828 */ +#define FBNIC_PCS_VL1_1_CHAN_1 0x1060b /* 0x4182c */ +#define FBNIC_PCS_VL2_0_CHAN_1 0x1060c /* 0x41830 */ +#define FBNIC_PCS_VL2_1_CHAN_1 0x1060d /* 0x41834 */ +#define FBNIC_PCS_VL3_0_CHAN_1 0x1060e /* 0x41838 */ +#define FBNIC_PCS_VL3_1_CHAN_1 0x1060f /* 0x4183c */ +#define FBNIC_PCS_MODE_VL_CHAN_1 0x10610 /* 0x41840 */ +#define FBNIC_CSR_END_PCS 0x10668 /* CSR section delimiter */ + +#define FBNIC_CSR_START_RSFEC 0x10800 /* CSR section delimiter */ +#define FBNIC_RSFEC_CONTROL(n)\ + (0x10800 + 8 * (n)) /* 0x42000 + 32*n */ +#define FBNIC_RSFEC_CONTROL_AM16_COPY_DIS CSR_BIT(3) +#define FBNIC_RSFEC_CONTROL_KP_ENABLE CSR_BIT(8) +#define FBNIC_RSFEC_CONTROL_TC_PAD_ALTER CSR_BIT(10) +#define FBNIC_RSFEC_MAX_LANES 4 +#define FBNIC_RSFEC_CCW_LO(n) \ + (0x10802 + 8 * (n)) /* 0x42008 + 32*n */ +#define FBNIC_RSFEC_CCW_HI(n) \ + (0x10803 + 8 * (n)) /* 0x4200c + 32*n */ +#define FBNIC_RSFEC_NCCW_LO(n) \ + (0x10804 + 8 * (n)) /* 0x42010 + 32*n */ +#define FBNIC_RSFEC_NCCW_HI(n) \ + (0x10805 + 8 * (n)) /* 0x42014 + 32*n */ +#define FBNIC_RSFEC_SYMBLERR_LO(n) \ + (0x10880 + 8 * (n)) /* 0x42200 + 32*n */ +#define FBNIC_RSFEC_SYMBLERR_HI(n) \ + (0x10881 + 8 * (n)) /* 0x42204 + 32*n */ +#define FBNIC_CSR_END_RSFEC 0x108c8 /* CSR section delimiter */ + +/* MAC MAC registers */ +#define FBNIC_CSR_START_MAC_MAC 0x11000 /* CSR section delimiter */ +#define FBNIC_MAC_COMMAND_CONFIG 0x11002 /* 0x44008 */ +#define FBNIC_MAC_COMMAND_CONFIG_RX_PAUSE_DIS CSR_BIT(29) +#define FBNIC_MAC_COMMAND_CONFIG_TX_PAUSE_DIS CSR_BIT(28) +#define FBNIC_MAC_COMMAND_CONFIG_FLT_HDL_DIS CSR_BIT(27) +#define FBNIC_MAC_COMMAND_CONFIG_TX_PAD_EN CSR_BIT(11) +#define FBNIC_MAC_COMMAND_CONFIG_LOOPBACK_EN CSR_BIT(10) +#define FBNIC_MAC_COMMAND_CONFIG_PROMISC_EN CSR_BIT(4) +#define FBNIC_MAC_COMMAND_CONFIG_RX_ENA CSR_BIT(1) +#define FBNIC_MAC_COMMAND_CONFIG_TX_ENA CSR_BIT(0) +#define FBNIC_MAC_FRM_LENGTH 0x11005 /* 0x44014 */ +#define FBNIC_MAC_TX_IPG_LENGTH 0x11011 /* 0x44044 */ +#define FBNIC_MAC_TX_IPG_LENGTH_COMP CSR_GENMASK(31, 16) +#define FBNIC_MAC_TX_IPG_LENGTH_TXIPG CSR_GENMASK(5, 3) +#define FBNIC_MAC_CL01_PAUSE_QUANTA 0x11015 /* 0x44054 */ +#define FBNIC_MAC_CL01_QUANTA_THRESH 0x11019 /* 0x44064 */ +#define FBNIC_MAC_XIF_MODE 0x11020 /* 0x44080 */ +#define FBNIC_MAC_XIF_MODE_TX_MAC_RS_ERR CSR_BIT(8) +#define FBNIC_MAC_XIF_MODE_XGMII CSR_BIT(0) +#define FBNIC_CSR_END_MAC_MAC 0x11028 /* CSR section delimiter */ + +/* MAC CSR registers */ +#define FBNIC_CSR_START_MAC_CSR 0x11800 /* CSR section delimiter */ +#define FBNIC_MAC_CTRL 0x11800 /* 0x46000 */ +#define FBNIC_MAC_CTRL_RESET_FF_TX_CLK CSR_BIT(14) +#define FBNIC_MAC_CTRL_RESET_FF_RX_CLK CSR_BIT(13) +#define FBNIC_MAC_CTRL_RESET_TX_CLK CSR_BIT(12) +#define FBNIC_MAC_CTRL_RESET_RX_CLK CSR_BIT(11) +#define FBNIC_MAC_CTRL_TX_CRC CSR_BIT(8) +#define FBNIC_MAC_CTRL_CFG_MODE128 CSR_BIT(10) +#define FBNIC_MAC_SERDES_CTRL 0x11807 /* 0x4601c */ +#define FBNIC_MAC_SERDES_CTRL_RESET_PCS_REF_CLK CSR_BIT(26) +#define FBNIC_MAC_SERDES_CTRL_RESET_F91_REF_CLK CSR_BIT(25) +#define FBNIC_MAC_SERDES_CTRL_RESET_SD_TX_CLK CSR_GENMASK(24, 23) +#define FBNIC_MAC_SERDES_CTRL_RESET_SD_RX_CLK CSR_GENMASK(22, 21) +#define FBNIC_MAC_SERDES_CTRL_SD_8X CSR_GENMASK(18, 17) +#define FBNIC_MAC_SERDES_CTRL_F91_1LANE_IN0 CSR_BIT(9) +#define FBNIC_MAC_SERDES_CTRL_RXLAUI_ENA_IN0 CSR_BIT(7) +#define FBNIC_MAC_SERDES_CTRL_PCS100_ENA_IN0 CSR_BIT(6) +#define FBNIC_MAC_SERDES_CTRL_PACER_10G_MASK CSR_GENMASK(1, 0) +#define FBNIC_MAC_PCS_STS0 0x11808 /* 0x46020 */ +#define FBNIC_MAC_PCS_STS0_LINK CSR_BIT(27) +#define FBNIC_MAC_PCS_STS0_BLOCK_LOCK CSR_GENMASK(24, 5) +#define FBNIC_MAC_PCS_STS0_AMPS_LOCK CSR_GENMASK(4, 1) +#define FBNIC_MAC_PCS_STS1 0x11809 /* 0x46024 */ +#define FBNIC_MAC_PCS_STS1_FCFEC_LOCK CSR_GENMASK(11, 8) +#define FBNIC_MAC_PCS_INTR_STS 0x11814 /* 0x46050 */ +#define FBNIC_MAC_PCS_INTR_LINK_DOWN CSR_BIT(1) +#define FBNIC_MAC_PCS_INTR_LINK_UP CSR_BIT(0) +#define FBNIC_MAC_PCS_INTR_MASK 0x11816 /* 0x46058 */ +#define FBNIC_MAC_ENET_LED 0x11820 /* 0x46080 */ +#define FBNIC_MAC_ENET_LED_OVERRIDE_EN CSR_GENMASK(2, 0) +#define FBNIC_MAC_ENET_LED_OVERRIDE_VAL CSR_GENMASK(6, 4) +enum { + FBNIC_MAC_ENET_LED_OVERRIDE_ACTIVITY = 0x1, + FBNIC_MAC_ENET_LED_OVERRIDE_AMBER = 0x2, + FBNIC_MAC_ENET_LED_OVERRIDE_BLUE = 0x4, +}; + +#define FBNIC_MAC_ENET_LED_BLINK_RATE_MASK CSR_GENMASK(11, 8) +enum { + FBNIC_MAC_ENET_LED_BLINK_RATE_5HZ = 0xf, +}; + +#define FBNIC_MAC_ENET_LED_BLUE_MASK CSR_GENMASK(18, 16) +enum { + FBNIC_MAC_ENET_LED_BLUE_50G = 0x2, + FBNIC_MAC_ENET_LED_BLUE_100G = 0x4, +}; + +#define FBNIC_MAC_ENET_LED_AMBER_MASK CSR_GENMASK(21, 20) +enum { + FBNIC_MAC_ENET_LED_AMBER_25G = 0x1, + FBNIC_MAC_ENET_LED_AMBER_50G = 0x2, +}; + +#define FBNIC_MAC_ENET_SIG_DETECT 0x11824 /* 0x46090 */ +#define FBNIC_MAC_ENET_SIG_DETECT_PCS_MASK CSR_GENMASK(1, 0) +#define FBNIC_MAC_ENET_FEC_CTRL 0x11825 /* 0x46094 */ +#define FBNIC_MAC_ENET_FEC_CTRL_FEC_ENA CSR_GENMASK(27, 24) +#define FBNIC_MAC_ENET_FEC_CTRL_KP_MODE_ENA CSR_GENMASK(11, 8) +#define FBNIC_MAC_ENET_FEC_CTRL_F91_ENA CSR_GENMASK(3, 0) +#define FBNIC_CSR_END_MAC_CSR 0x1184e /* CSR section delimiter */ + /* PUL User Registers */ #define FBNIC_CSR_START_PUL_USER 0x31000 /* CSR section delimiter */ #define FBNIC_PUL_OB_TLP_HDR_AW_CFG 0x3103d /* 0xc40f4 */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c index 4c3098364fed..af38d5934bbf 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c @@ -643,6 +643,63 @@ void fbnic_fw_check_heartbeat(struct fbnic_dev *fbd) dev_warn(fbd->dev, "Failed to send heartbeat message\n"); } +/** + * fbnic_fw_xmit_comphy_set_msg - Create and transmit a comphy set request + * + * @fbd: FBNIC device structure + * @speed: Indicates link speed, composed of modulation and number of lanes + * + * Returns 0 on success, negative value on failure + * + * Asks the firmware to reconfigure the comphy for this slice to the target + * speed. + */ +int fbnic_fw_xmit_comphy_set_msg(struct fbnic_dev *fbd, u32 speed) +{ + struct fbnic_tlv_msg *msg; + int err = 0; + + if (!fbnic_fw_present(fbd)) + return -ENODEV; + + msg = fbnic_tlv_msg_alloc(FBNIC_TLV_MSG_ID_COMPHY_SET_REQ); + if (!msg) + return -ENOMEM; + + err = fbnic_tlv_attr_put_int(msg, FBNIC_COMPHY_SET_PAM4, + !!(speed & FBNIC_LINK_MODE_PAM4)); + if (err) + goto free_message; + + err = fbnic_mbx_map_tlv_msg(fbd, msg); + if (err) + goto free_message; + + return 0; + +free_message: + free_page((unsigned long)msg); + return err; +} + +static const struct fbnic_tlv_index fbnic_comphy_set_resp_index[] = { + FBNIC_TLV_ATTR_S32(FBNIC_COMPHY_SET_ERROR), + FBNIC_TLV_ATTR_LAST +}; + +static int fbnic_fw_parse_comphy_set_resp(void *opaque, + struct fbnic_tlv_msg **results) +{ + struct fbnic_dev *fbd = (struct fbnic_dev *)opaque; + int err_resp = 0; + + get_signed_result(FBNIC_COMPHY_SET_ERROR, err_resp); + if (err_resp) + dev_err(fbd->dev, "COMPHY_SET returned %d\n", err_resp); + + return 0; +} + static const struct fbnic_tlv_parser fbnic_fw_tlv_parser[] = { FBNIC_TLV_PARSER(FW_CAP_RESP, fbnic_fw_cap_resp_index, fbnic_fw_parse_cap_resp), @@ -650,6 +707,9 @@ static const struct fbnic_tlv_parser fbnic_fw_tlv_parser[] = { fbnic_fw_parse_ownership_resp), FBNIC_TLV_PARSER(HEARTBEAT_RESP, fbnic_heartbeat_resp_index, fbnic_fw_parse_heartbeat_resp), + FBNIC_TLV_PARSER(COMPHY_SET_RESP, + fbnic_comphy_set_resp_index, + fbnic_fw_parse_comphy_set_resp), FBNIC_TLV_MSG_ERROR }; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h index 40d314f963ea..ea4802537d31 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h @@ -52,6 +52,7 @@ void fbnic_mbx_flush_tx(struct fbnic_dev *fbd); int fbnic_fw_xmit_ownership_msg(struct fbnic_dev *fbd, bool take_ownership); int fbnic_fw_init_heartbeat(struct fbnic_dev *fbd, bool poll); void fbnic_fw_check_heartbeat(struct fbnic_dev *fbd); +int fbnic_fw_xmit_comphy_set_msg(struct fbnic_dev *fbd, u32 speed); #define fbnic_mk_full_fw_ver_str(_rev_id, _delim, _commit, _str) \ do { \ @@ -76,6 +77,8 @@ enum { FBNIC_TLV_MSG_ID_OWNERSHIP_RESP = 0x13, FBNIC_TLV_MSG_ID_HEARTBEAT_REQ = 0x14, FBNIC_TLV_MSG_ID_HEARTBEAT_RESP = 0x15, + FBNIC_TLV_MSG_ID_COMPHY_SET_REQ = 0x3E, + FBNIC_TLV_MSG_ID_COMPHY_SET_RESP = 0x3F, }; #define FBNIC_FW_CAP_RESP_VERSION_MAJOR CSR_GENMASK(31, 24) @@ -104,8 +107,27 @@ enum { FBNIC_FW_CAP_RESP_MSG_MAX }; +enum { + FBNIC_FW_LINK_SPEED_25R1 = 1, + FBNIC_FW_LINK_SPEED_50R2 = 2, + FBNIC_FW_LINK_SPEED_50R1 = 3, + FBNIC_FW_LINK_SPEED_100R2 = 4, +}; + +enum { + FBNIC_FW_LINK_FEC_NONE = 1, + FBNIC_FW_LINK_FEC_RS = 2, + FBNIC_FW_LINK_FEC_BASER = 3, +}; + enum { FBNIC_FW_OWNERSHIP_FLAG = 0x0, FBNIC_FW_OWNERSHIP_MSG_MAX }; + +enum { + FBNIC_COMPHY_SET_PAM4 = 0x0, + FBNIC_COMPHY_SET_ERROR = 0x1, + FBNIC_COMPHY_SET_MSG_MAX +}; #endif /* _FBNIC_FW_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_irq.c b/drivers/net/ethernet/meta/fbnic/fbnic_irq.c index a20070683f48..33b5f15e2c40 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_irq.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_irq.c @@ -84,11 +84,127 @@ void fbnic_fw_disable_mbx(struct fbnic_dev *fbd) fbnic_mbx_clean(fbd); } +static irqreturn_t fbnic_mac_msix_intr(int __always_unused irq, void *data) +{ + struct fbnic_dev *fbd = data; + + if (fbd->mac->get_link_event(fbd)) + fbd->link_state = FBNIC_LINK_EVENT; + else + wr32(FBNIC_INTR_MASK_CLEAR(0), 1u << FBNIC_MAC_MSIX_ENTRY); + + return IRQ_HANDLED; +} + +/** + * fbnic_mac_get_link - Retrieve the current link state of the MAC + * @fbd: Device to retrieve the link state of + * @link: pointer to boolean value that will store link state + * + * This function will query the hardware to determine the state of the + * hardware to determine the link status of the device. If it is unable to + * communicate with the device it will return ENODEV and return false + * indicating the link is down. + **/ +int fbnic_mac_get_link(struct fbnic_dev *fbd, bool *link) +{ + const struct fbnic_mac *mac = fbd->mac; + + *link = true; + + /* In an interrupt driven setup we can just skip the check if + * the link is up as the interrupt should toggle it to the EVENT + * state if the link has changed state at any time since the last + * check. + */ + if (fbd->link_state == FBNIC_LINK_UP) + goto skip_check; + + *link = mac->get_link(fbd); + + wr32(FBNIC_INTR_MASK_CLEAR(0), 1u << FBNIC_MAC_MSIX_ENTRY); +skip_check: + if (!fbnic_present(fbd)) { + *link = false; + return -ENODEV; + } + + return 0; +} + +/** + * fbnic_mac_enable - Configure the MAC to enable it to advertise link + * @fbd: Pointer to device to initialize + * + * This function provides basic bringup for the CMAC and sets the link + * state to FBNIC_LINK_EVENT which tells the link state check that the + * current state is unknown and that interrupts must be enabled after the + * check is completed. + **/ +int fbnic_mac_enable(struct fbnic_dev *fbd) +{ + const struct fbnic_mac *mac = fbd->mac; + u32 vector = fbd->mac_msix_vector; + int err; + + /* Request the IRQ for MAC link vector. + * Map MAC cause to it, and unmask it + */ + err = request_irq(vector, &fbnic_mac_msix_intr, 0, + fbd->netdev->name, fbd); + if (err) + return err; + + wr32(FBNIC_INTR_MSIX_CTRL(FBNIC_INTR_MSIX_CTRL_PCS_IDX), + FBNIC_MAC_MSIX_ENTRY | FBNIC_INTR_MSIX_CTRL_ENABLE); + + err = mac->enable(fbd); + if (err) { + /* Disable interrupt */ + wr32(FBNIC_INTR_MSIX_CTRL(FBNIC_INTR_MSIX_CTRL_PCS_IDX), + FBNIC_MAC_MSIX_ENTRY); + wr32(FBNIC_INTR_MASK_SET(0), 1u << FBNIC_MAC_MSIX_ENTRY); + + /* Free the vector */ + free_irq(fbd->mac_msix_vector, fbd); + } + + return err; +} + +/** + * fbnic_mac_disable - Teardown the MAC to prepare for stopping + * @fbd: Pointer to device that is stopping + * + * This function undoes the work done in fbnic_mac_enable and prepares the + * device to no longer receive traffic on the host interface. + **/ +void fbnic_mac_disable(struct fbnic_dev *fbd) +{ + const struct fbnic_mac *mac = fbd->mac; + + /* Nothing to do if link is already disabled */ + if (fbd->link_state == FBNIC_LINK_DISABLED) + return; + + mac->disable(fbd); + + /* Disable interrupt */ + wr32(FBNIC_INTR_MSIX_CTRL(FBNIC_INTR_MSIX_CTRL_PCS_IDX), + FBNIC_MAC_MSIX_ENTRY); + wr32(FBNIC_INTR_MASK_SET(0), 1u << FBNIC_MAC_MSIX_ENTRY); + + /* Free the vector */ + free_irq(fbd->mac_msix_vector, fbd); +} + void fbnic_free_irqs(struct fbnic_dev *fbd) { struct pci_dev *pdev = to_pci_dev(fbd->dev); + fbd->mac_msix_vector = 0; fbd->fw_msix_vector = 0; + fbd->num_irqs = 0; pci_disable_msix(pdev); @@ -128,6 +244,8 @@ int fbnic_alloc_irqs(struct fbnic_dev *fbd) fbd->msix_entries = msix_entries; fbd->num_irqs = num_irqs; + fbd->mac_msix_vector = msix_entries[FBNIC_MAC_MSIX_ENTRY].vector; fbd->fw_msix_vector = msix_entries[FBNIC_FW_MSIX_ENTRY].vector; + return 0; } diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c index dbbfdc649f37..64c4dde30b9d 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c @@ -2,10 +2,12 @@ /* Copyright (c) Meta Platforms, Inc. and affiliates. */ #include +#include #include #include "fbnic.h" #include "fbnic_mac.h" +#include "fbnic_netdev.h" static void fbnic_init_readrq(struct fbnic_dev *fbd, unsigned int offset, unsigned int cls, unsigned int readrq) @@ -415,8 +417,593 @@ static void fbnic_mac_init_regs(struct fbnic_dev *fbd) fbnic_mac_init_txb(fbd); } +static int fbnic_mac_get_link_event_asic(struct fbnic_dev *fbd) +{ + u32 pcs_intr_mask = rd32(FBNIC_MAC_PCS_INTR_STS); + + if (pcs_intr_mask & FBNIC_MAC_PCS_INTR_LINK_DOWN) + return -1; + + return (pcs_intr_mask & FBNIC_MAC_PCS_INTR_LINK_UP) ? 1 : 0; +} + +static u32 __fbnic_mac_config_asic(struct fbnic_dev *fbd) +{ + /* Enable MAC Promiscuous mode and Tx padding */ + u32 command_config = FBNIC_MAC_COMMAND_CONFIG_TX_PAD_EN | + FBNIC_MAC_COMMAND_CONFIG_PROMISC_EN; + struct fbnic_net *fbn = netdev_priv(fbd->netdev); + u32 rxb_pause_ctrl; + + /* Set class 0 Quanta and refresh */ + wr32(FBNIC_MAC_CL01_PAUSE_QUANTA, 0xffff); + wr32(FBNIC_MAC_CL01_QUANTA_THRESH, 0x7fff); + + /* Enable generation of pause frames if enabled */ + rxb_pause_ctrl = rd32(FBNIC_RXB_PAUSE_DROP_CTRL); + rxb_pause_ctrl &= ~FBNIC_RXB_PAUSE_DROP_CTRL_PAUSE_ENABLE; + if (!fbn->tx_pause) + command_config |= FBNIC_MAC_COMMAND_CONFIG_TX_PAUSE_DIS; + else + rxb_pause_ctrl |= + FIELD_PREP(FBNIC_RXB_PAUSE_DROP_CTRL_PAUSE_ENABLE, + FBNIC_PAUSE_EN_MASK); + wr32(FBNIC_RXB_PAUSE_DROP_CTRL, rxb_pause_ctrl); + + if (!fbn->rx_pause) + command_config |= FBNIC_MAC_COMMAND_CONFIG_RX_PAUSE_DIS; + + /* Disable fault handling if no FEC is requested */ + if ((fbn->fec & FBNIC_FEC_MODE_MASK) == FBNIC_FEC_OFF) + command_config |= FBNIC_MAC_COMMAND_CONFIG_FLT_HDL_DIS; + + return command_config; +} + +static bool fbnic_mac_get_pcs_link_status(struct fbnic_dev *fbd) +{ + struct fbnic_net *fbn = netdev_priv(fbd->netdev); + u32 pcs_status, lane_mask = ~0; + + pcs_status = rd32(FBNIC_MAC_PCS_STS0); + if (!(pcs_status & FBNIC_MAC_PCS_STS0_LINK)) + return false; + + /* Define the expected lane mask for the status bits we need to check */ + switch (fbn->link_mode & FBNIC_LINK_MODE_MASK) { + case FBNIC_LINK_100R2: + lane_mask = 0xf; + break; + case FBNIC_LINK_50R1: + lane_mask = 3; + break; + case FBNIC_LINK_50R2: + switch (fbn->fec & FBNIC_FEC_MODE_MASK) { + case FBNIC_FEC_OFF: + lane_mask = 0x63; + break; + case FBNIC_FEC_RS: + lane_mask = 5; + break; + case FBNIC_FEC_BASER: + lane_mask = 0xf; + break; + } + break; + case FBNIC_LINK_25R1: + lane_mask = 1; + break; + } + + /* Use an XOR to remove the bits we expect to see set */ + switch (fbn->fec & FBNIC_FEC_MODE_MASK) { + case FBNIC_FEC_OFF: + lane_mask ^= FIELD_GET(FBNIC_MAC_PCS_STS0_BLOCK_LOCK, + pcs_status); + break; + case FBNIC_FEC_RS: + lane_mask ^= FIELD_GET(FBNIC_MAC_PCS_STS0_AMPS_LOCK, + pcs_status); + break; + case FBNIC_FEC_BASER: + lane_mask ^= FIELD_GET(FBNIC_MAC_PCS_STS1_FCFEC_LOCK, + rd32(FBNIC_MAC_PCS_STS1)); + break; + } + + /* If all lanes cancelled then we have a lock on all lanes */ + return !lane_mask; +} + +#define FBNIC_MAC_ENET_LED_DEFAULT \ + (FIELD_PREP(FBNIC_MAC_ENET_LED_AMBER_MASK, \ + FBNIC_MAC_ENET_LED_AMBER_50G | \ + FBNIC_MAC_ENET_LED_AMBER_25G) | \ + FIELD_PREP(FBNIC_MAC_ENET_LED_BLUE_MASK, \ + FBNIC_MAC_ENET_LED_BLUE_100G | \ + FBNIC_MAC_ENET_LED_BLUE_50G)) +#define FBNIC_MAC_ENET_LED_ACTIVITY_DEFAULT \ + FIELD_PREP(FBNIC_MAC_ENET_LED_BLINK_RATE_MASK, \ + FBNIC_MAC_ENET_LED_BLINK_RATE_5HZ) +#define FBNIC_MAC_ENET_LED_ACTIVITY_ON \ + FIELD_PREP(FBNIC_MAC_ENET_LED_OVERRIDE_EN, \ + FBNIC_MAC_ENET_LED_OVERRIDE_ACTIVITY) +#define FBNIC_MAC_ENET_LED_AMBER \ + (FIELD_PREP(FBNIC_MAC_ENET_LED_OVERRIDE_EN, \ + FBNIC_MAC_ENET_LED_OVERRIDE_BLUE | \ + FBNIC_MAC_ENET_LED_OVERRIDE_AMBER) | \ + FIELD_PREP(FBNIC_MAC_ENET_LED_OVERRIDE_VAL, \ + FBNIC_MAC_ENET_LED_OVERRIDE_AMBER)) +#define FBNIC_MAC_ENET_LED_BLUE \ + (FIELD_PREP(FBNIC_MAC_ENET_LED_OVERRIDE_EN, \ + FBNIC_MAC_ENET_LED_OVERRIDE_BLUE | \ + FBNIC_MAC_ENET_LED_OVERRIDE_AMBER) | \ + FIELD_PREP(FBNIC_MAC_ENET_LED_OVERRIDE_VAL, \ + FBNIC_MAC_ENET_LED_OVERRIDE_BLUE)) + +static void fbnic_set_led_state_asic(struct fbnic_dev *fbd, int state) +{ + struct fbnic_net *fbn = netdev_priv(fbd->netdev); + u32 led_csr = FBNIC_MAC_ENET_LED_DEFAULT; + + switch (state) { + case FBNIC_LED_OFF: + led_csr |= FBNIC_MAC_ENET_LED_AMBER | + FBNIC_MAC_ENET_LED_ACTIVITY_ON; + break; + case FBNIC_LED_ON: + led_csr |= FBNIC_MAC_ENET_LED_BLUE | + FBNIC_MAC_ENET_LED_ACTIVITY_ON; + break; + case FBNIC_LED_RESTORE: + led_csr |= FBNIC_MAC_ENET_LED_ACTIVITY_DEFAULT; + + /* Don't set LEDs on if link isn't up */ + if (fbd->link_state != FBNIC_LINK_UP) + break; + /* Don't set LEDs for supported autoneg modes */ + if ((fbn->link_mode & FBNIC_LINK_AUTO) && + (fbn->link_mode & FBNIC_LINK_MODE_MASK) != FBNIC_LINK_50R2) + break; + + /* Set LEDs based on link speed + * 100G Blue, + * 50G Blue & Amber + * 25G Amber + */ + switch (fbn->link_mode & FBNIC_LINK_MODE_MASK) { + case FBNIC_LINK_100R2: + led_csr |= FBNIC_MAC_ENET_LED_BLUE; + break; + case FBNIC_LINK_50R1: + case FBNIC_LINK_50R2: + led_csr |= FBNIC_MAC_ENET_LED_BLUE; + fallthrough; + case FBNIC_LINK_25R1: + led_csr |= FBNIC_MAC_ENET_LED_AMBER; + break; + } + break; + default: + return; + } + + wr32(FBNIC_MAC_ENET_LED, led_csr); +} + +static bool fbnic_mac_get_link_asic(struct fbnic_dev *fbd) +{ + u32 cmd_cfg, mac_ctrl; + int link_direction; + bool link; + + /* If disabled do not update link_state nor change settings */ + if (fbd->link_state == FBNIC_LINK_DISABLED) + return false; + + link_direction = fbnic_mac_get_link_event_asic(fbd); + + /* Clear interrupt state due to recent changes. */ + wr32(FBNIC_MAC_PCS_INTR_STS, + FBNIC_MAC_PCS_INTR_LINK_DOWN | FBNIC_MAC_PCS_INTR_LINK_UP); + + /* If link bounced down clear the PCS_STS bit related to link */ + if (link_direction < 0) { + wr32(FBNIC_MAC_PCS_STS0, FBNIC_MAC_PCS_STS0_LINK | + FBNIC_MAC_PCS_STS0_BLOCK_LOCK | + FBNIC_MAC_PCS_STS0_AMPS_LOCK); + wr32(FBNIC_MAC_PCS_STS1, FBNIC_MAC_PCS_STS1_FCFEC_LOCK); + } + + link = fbnic_mac_get_pcs_link_status(fbd); + cmd_cfg = __fbnic_mac_config_asic(fbd); + mac_ctrl = rd32(FBNIC_MAC_CTRL); + + /* Depending on the event we will unmask the cause that will force a + * transition, and update the Tx to reflect our status to the remote + * link partner. + */ + if (link) { + mac_ctrl &= ~(FBNIC_MAC_CTRL_RESET_FF_TX_CLK | + FBNIC_MAC_CTRL_RESET_TX_CLK | + FBNIC_MAC_CTRL_RESET_FF_RX_CLK | + FBNIC_MAC_CTRL_RESET_RX_CLK); + cmd_cfg |= FBNIC_MAC_COMMAND_CONFIG_RX_ENA | + FBNIC_MAC_COMMAND_CONFIG_TX_ENA; + fbd->link_state = FBNIC_LINK_UP; + } else { + mac_ctrl |= FBNIC_MAC_CTRL_RESET_FF_TX_CLK | + FBNIC_MAC_CTRL_RESET_TX_CLK | + FBNIC_MAC_CTRL_RESET_FF_RX_CLK | + FBNIC_MAC_CTRL_RESET_RX_CLK; + fbd->link_state = FBNIC_LINK_DOWN; + } + + wr32(FBNIC_MAC_CTRL, mac_ctrl); + wr32(FBNIC_MAC_COMMAND_CONFIG, cmd_cfg); + + /* Toggle LED settings to enable LEDs manually if necessary */ + fbnic_set_led_state_asic(fbd, FBNIC_LED_RESTORE); + + if (link_direction) + wr32(FBNIC_MAC_PCS_INTR_MASK, + link ? ~FBNIC_MAC_PCS_INTR_LINK_DOWN : + ~FBNIC_MAC_PCS_INTR_LINK_UP); + + return link; +} + +static void fbnic_mac_pre_config(struct fbnic_dev *fbd) +{ + u32 serdes_ctrl, mac_ctrl, xif_mode, enet_fec_ctrl = 0; + struct fbnic_net *fbn = netdev_priv(fbd->netdev); + + /* set reset bits and enable appending of Tx CRC */ + mac_ctrl = FBNIC_MAC_CTRL_RESET_FF_TX_CLK | + FBNIC_MAC_CTRL_RESET_FF_RX_CLK | + FBNIC_MAC_CTRL_RESET_TX_CLK | + FBNIC_MAC_CTRL_RESET_RX_CLK | + FBNIC_MAC_CTRL_TX_CRC; + serdes_ctrl = FBNIC_MAC_SERDES_CTRL_RESET_PCS_REF_CLK | + FBNIC_MAC_SERDES_CTRL_RESET_F91_REF_CLK | + FBNIC_MAC_SERDES_CTRL_RESET_SD_TX_CLK | + FBNIC_MAC_SERDES_CTRL_RESET_SD_RX_CLK; + xif_mode = FBNIC_MAC_XIF_MODE_TX_MAC_RS_ERR; + + switch (fbn->link_mode & FBNIC_LINK_MODE_MASK) { + case FBNIC_LINK_25R1: + /* Enable XGMII to run w/ 10G pacer */ + xif_mode |= FBNIC_MAC_XIF_MODE_XGMII; + serdes_ctrl |= FBNIC_MAC_SERDES_CTRL_PACER_10G_MASK; + if (fbn->fec & FBNIC_FEC_RS) + serdes_ctrl |= FBNIC_MAC_SERDES_CTRL_F91_1LANE_IN0; + break; + case FBNIC_LINK_50R2: + if (!(fbn->fec & FBNIC_FEC_RS)) + serdes_ctrl |= FBNIC_MAC_SERDES_CTRL_RXLAUI_ENA_IN0; + break; + case FBNIC_LINK_100R2: + mac_ctrl |= FBNIC_MAC_CTRL_CFG_MODE128; + serdes_ctrl |= FBNIC_MAC_SERDES_CTRL_PCS100_ENA_IN0; + enet_fec_ctrl |= FBNIC_MAC_ENET_FEC_CTRL_KP_MODE_ENA; + fallthrough; + case FBNIC_LINK_50R1: + serdes_ctrl |= FBNIC_MAC_SERDES_CTRL_SD_8X; + if (fbn->fec & FBNIC_FEC_AUTO) + fbn->fec = FBNIC_FEC_AUTO | FBNIC_FEC_RS; + break; + } + + switch (fbn->fec & FBNIC_FEC_MODE_MASK) { + case FBNIC_FEC_RS: + enet_fec_ctrl |= FBNIC_MAC_ENET_FEC_CTRL_F91_ENA; + break; + case FBNIC_FEC_BASER: + enet_fec_ctrl |= FBNIC_MAC_ENET_FEC_CTRL_FEC_ENA; + break; + case FBNIC_FEC_OFF: + break; + default: + dev_err(fbd->dev, "Unsupported FEC mode detected"); + } + + /* Store updated config to MAC */ + wr32(FBNIC_MAC_CTRL, mac_ctrl); + wr32(FBNIC_MAC_SERDES_CTRL, serdes_ctrl); + wr32(FBNIC_MAC_XIF_MODE, xif_mode); + wr32(FBNIC_MAC_ENET_FEC_CTRL, enet_fec_ctrl); + + /* flush writes to allow time for MAC to go into resets */ + wrfl(); + + /* Set signal detect for all lanes */ + wr32(FBNIC_MAC_ENET_SIG_DETECT, FBNIC_MAC_ENET_SIG_DETECT_PCS_MASK); +} + +static void fbnic_mac_pcs_config(struct fbnic_dev *fbd) +{ + u32 pcs_mode = 0, rsfec_ctrl = 0, vl_intvl = 0; + struct fbnic_net *fbn = netdev_priv(fbd->netdev); + int i; + + /* Set link mode specific lane and FEC values */ + switch (fbn->link_mode & FBNIC_LINK_MODE_MASK) { + case FBNIC_LINK_25R1: + if (fbn->fec & FBNIC_FEC_RS) + vl_intvl = 20479; + else + pcs_mode |= FBNIC_PCS_MODE_DISABLE_MLD; + pcs_mode |= FBNIC_PCS_MODE_HI_BER25 | + FBNIC_PCS_MODE_ENA_CLAUSE49; + break; + case FBNIC_LINK_50R1: + rsfec_ctrl |= FBNIC_RSFEC_CONTROL_KP_ENABLE; + fallthrough; + case FBNIC_LINK_50R2: + rsfec_ctrl |= FBNIC_RSFEC_CONTROL_TC_PAD_ALTER; + vl_intvl = 20479; + break; + case FBNIC_LINK_100R2: + rsfec_ctrl |= FBNIC_RSFEC_CONTROL_AM16_COPY_DIS | + FBNIC_RSFEC_CONTROL_KP_ENABLE; + pcs_mode |= FBNIC_PCS_MODE_DISABLE_MLD; + vl_intvl = 16383; + break; + } + + for (i = 0; i < 4; i++) + wr32(FBNIC_RSFEC_CONTROL(i), rsfec_ctrl); + + wr32(FBNIC_PCS_MODE_VL_CHAN_0, pcs_mode); + wr32(FBNIC_PCS_MODE_VL_CHAN_1, pcs_mode); + + wr32(FBNIC_PCS_VENDOR_VL_INTVL_0, vl_intvl); + wr32(FBNIC_PCS_VENDOR_VL_INTVL_1, vl_intvl); + + /* Update IPG to account for vl_intvl */ + wr32(FBNIC_MAC_TX_IPG_LENGTH, + FIELD_PREP(FBNIC_MAC_TX_IPG_LENGTH_COMP, vl_intvl) | 0xc); + + /* Program lane markers indicating which lanes are in use + * and what speeds we are transmitting at. + */ + switch (fbn->link_mode & FBNIC_LINK_MODE_MASK) { + case FBNIC_LINK_100R2: + wr32(FBNIC_PCS_VL0_0_CHAN_0, 0x68c1); + wr32(FBNIC_PCS_VL0_1_CHAN_0, 0x21); + wr32(FBNIC_PCS_VL1_0_CHAN_0, 0x719d); + wr32(FBNIC_PCS_VL1_1_CHAN_0, 0x8e); + wr32(FBNIC_PCS_VL2_0_CHAN_0, 0x4b59); + wr32(FBNIC_PCS_VL2_1_CHAN_0, 0xe8); + wr32(FBNIC_PCS_VL3_0_CHAN_0, 0x954d); + wr32(FBNIC_PCS_VL3_1_CHAN_0, 0x7b); + wr32(FBNIC_PCS_VL0_0_CHAN_1, 0x68c1); + wr32(FBNIC_PCS_VL0_1_CHAN_1, 0x21); + wr32(FBNIC_PCS_VL1_0_CHAN_1, 0x719d); + wr32(FBNIC_PCS_VL1_1_CHAN_1, 0x8e); + wr32(FBNIC_PCS_VL2_0_CHAN_1, 0x4b59); + wr32(FBNIC_PCS_VL2_1_CHAN_1, 0xe8); + wr32(FBNIC_PCS_VL3_0_CHAN_1, 0x954d); + wr32(FBNIC_PCS_VL3_1_CHAN_1, 0x7b); + break; + case FBNIC_LINK_50R2: + wr32(FBNIC_PCS_VL0_0_CHAN_1, 0x7690); + wr32(FBNIC_PCS_VL0_1_CHAN_1, 0x47); + wr32(FBNIC_PCS_VL1_0_CHAN_1, 0xc4f0); + wr32(FBNIC_PCS_VL1_1_CHAN_1, 0xe6); + wr32(FBNIC_PCS_VL2_0_CHAN_1, 0x65c5); + wr32(FBNIC_PCS_VL2_1_CHAN_1, 0x9b); + wr32(FBNIC_PCS_VL3_0_CHAN_1, 0x79a2); + wr32(FBNIC_PCS_VL3_1_CHAN_1, 0x3d); + fallthrough; + case FBNIC_LINK_50R1: + wr32(FBNIC_PCS_VL0_0_CHAN_0, 0x7690); + wr32(FBNIC_PCS_VL0_1_CHAN_0, 0x47); + wr32(FBNIC_PCS_VL1_0_CHAN_0, 0xc4f0); + wr32(FBNIC_PCS_VL1_1_CHAN_0, 0xe6); + wr32(FBNIC_PCS_VL2_0_CHAN_0, 0x65c5); + wr32(FBNIC_PCS_VL2_1_CHAN_0, 0x9b); + wr32(FBNIC_PCS_VL3_0_CHAN_0, 0x79a2); + wr32(FBNIC_PCS_VL3_1_CHAN_0, 0x3d); + break; + case FBNIC_LINK_25R1: + wr32(FBNIC_PCS_VL0_0_CHAN_0, 0x68c1); + wr32(FBNIC_PCS_VL0_1_CHAN_0, 0x21); + wr32(FBNIC_PCS_VL1_0_CHAN_0, 0xc4f0); + wr32(FBNIC_PCS_VL1_1_CHAN_0, 0xe6); + wr32(FBNIC_PCS_VL2_0_CHAN_0, 0x65c5); + wr32(FBNIC_PCS_VL2_1_CHAN_0, 0x9b); + wr32(FBNIC_PCS_VL3_0_CHAN_0, 0x79a2); + wr32(FBNIC_PCS_VL3_1_CHAN_0, 0x3d); + break; + } +} + +static bool fbnic_mac_pcs_reset_complete(struct fbnic_dev *fbd) +{ + return !(rd32(FBNIC_PCS_CONTROL1_0) & FBNIC_PCS_CONTROL1_RESET) && + !(rd32(FBNIC_PCS_CONTROL1_1) & FBNIC_PCS_CONTROL1_RESET); +} + +static int fbnic_mac_post_config(struct fbnic_dev *fbd) +{ + struct fbnic_net *fbn = netdev_priv(fbd->netdev); + u32 serdes_ctrl, reset_complete, lane_mask; + int err; + + /* Clear resets for XPCS and F91 reference clocks */ + serdes_ctrl = rd32(FBNIC_MAC_SERDES_CTRL); + serdes_ctrl &= ~FBNIC_MAC_SERDES_CTRL_RESET_PCS_REF_CLK; + if (fbn->fec & FBNIC_FEC_RS) + serdes_ctrl &= ~FBNIC_MAC_SERDES_CTRL_RESET_F91_REF_CLK; + wr32(FBNIC_MAC_SERDES_CTRL, serdes_ctrl); + + /* Reset PCS and flush reset value */ + wr32(FBNIC_PCS_CONTROL1_0, + FBNIC_PCS_CONTROL1_RESET | + FBNIC_PCS_CONTROL1_SPEED_SELECT_ALWAYS | + FBNIC_PCS_CONTROL1_SPEED_ALWAYS); + wr32(FBNIC_PCS_CONTROL1_1, + FBNIC_PCS_CONTROL1_RESET | + FBNIC_PCS_CONTROL1_SPEED_SELECT_ALWAYS | + FBNIC_PCS_CONTROL1_SPEED_ALWAYS); + + /* poll for completion of reset */ + err = readx_poll_timeout(fbnic_mac_pcs_reset_complete, fbd, + reset_complete, reset_complete, + 1000, 150000); + if (err) + return err; + + /* Flush any stale link status info */ + wr32(FBNIC_MAC_PCS_STS0, FBNIC_MAC_PCS_STS0_LINK | + FBNIC_MAC_PCS_STS0_BLOCK_LOCK | + FBNIC_MAC_PCS_STS0_AMPS_LOCK); + + /* Report starting state as "Link Event" to force detection of link */ + fbd->link_state = FBNIC_LINK_EVENT; + + /* Force link down to allow for link detection */ + netif_carrier_off(fbn->netdev); + + /* create simple bitmask for 2 or 1 lane setups */ + lane_mask = (fbn->link_mode & FBNIC_LINK_MODE_R2) ? 3 : 1; + + /* release the brakes and allow Tx/Rx to come out of reset */ + serdes_ctrl &= + ~(FIELD_PREP(FBNIC_MAC_SERDES_CTRL_RESET_SD_TX_CLK, lane_mask) | + FIELD_PREP(FBNIC_MAC_SERDES_CTRL_RESET_SD_RX_CLK, lane_mask)); + wr32(FBNIC_MAC_SERDES_CTRL, serdes_ctrl); + + fbn->link_mode &= ~FBNIC_LINK_AUTO; + + /* Ask firmware to configure the PHY for the correct encoding mode */ + return fbnic_fw_xmit_comphy_set_msg(fbd, + fbn->link_mode & + FBNIC_LINK_MODE_MASK); +} + +static void fbnic_mac_get_fw_settings(struct fbnic_dev *fbd) +{ + struct fbnic_net *fbn = netdev_priv(fbd->netdev); + u8 fec = fbn->fec; + u8 link_mode; + + /* Update FEC first to reflect FW current mode */ + if (fbn->fec & FBNIC_FEC_AUTO) { + switch (fbd->fw_cap.link_fec) { + case FBNIC_FW_LINK_FEC_NONE: + fec = FBNIC_FEC_OFF; + break; + case FBNIC_FW_LINK_FEC_RS: + fec = FBNIC_FEC_RS; + break; + case FBNIC_FW_LINK_FEC_BASER: + fec = FBNIC_FEC_BASER; + break; + default: + return; + } + } + + /* Do nothing if AUTO mode is not engaged */ + if (fbn->link_mode & FBNIC_LINK_AUTO) { + switch (fbd->fw_cap.link_speed) { + case FBNIC_FW_LINK_SPEED_25R1: + link_mode = FBNIC_LINK_25R1; + break; + case FBNIC_FW_LINK_SPEED_50R2: + link_mode = FBNIC_LINK_50R2; + break; + case FBNIC_FW_LINK_SPEED_50R1: + link_mode = FBNIC_LINK_50R1; + fec = FBNIC_FEC_RS; + break; + case FBNIC_FW_LINK_SPEED_100R2: + link_mode = FBNIC_LINK_100R2; + fec = FBNIC_FEC_RS; + break; + default: + return; + } + + fbn->link_mode = link_mode; + fbn->fec = fec; + } +} + +static int fbnic_mac_enable_asic(struct fbnic_dev *fbd) +{ + /* Mask and clear the PCS interrupt, will be enabled by link handler */ + wr32(FBNIC_MAC_PCS_INTR_MASK, ~0); + wr32(FBNIC_MAC_PCS_INTR_STS, ~0); + + /* Pull in settings from FW */ + fbnic_mac_get_fw_settings(fbd); + + /* Configure MAC registers */ + fbnic_mac_pre_config(fbd); + + /* Configure PCS block */ + fbnic_mac_pcs_config(fbd); + + /* Configure flow control and error correction */ + wr32(FBNIC_MAC_COMMAND_CONFIG, __fbnic_mac_config_asic(fbd)); + + /* Configure maximum frame size */ + wr32(FBNIC_MAC_FRM_LENGTH, FBNIC_MAX_JUMBO_FRAME_SIZE); + + /* Configure LED defaults */ + fbnic_set_led_state_asic(fbd, FBNIC_LED_RESTORE); + + return fbnic_mac_post_config(fbd); +} + +static void fbnic_mac_disable_asic(struct fbnic_dev *fbd) +{ + u32 mask = FBNIC_MAC_COMMAND_CONFIG_LOOPBACK_EN; + u32 cmd_cfg = rd32(FBNIC_MAC_COMMAND_CONFIG); + u32 mac_ctrl = rd32(FBNIC_MAC_CTRL); + + /* Clear link state to disable any further transitions */ + fbd->link_state = FBNIC_LINK_DISABLED; + + /* Clear Tx and Rx enable bits to disable MAC, ignore other values */ + if (!fbnic_bmc_present(fbd)) { + mask |= FBNIC_MAC_COMMAND_CONFIG_RX_ENA | + FBNIC_MAC_COMMAND_CONFIG_TX_ENA; + mac_ctrl |= FBNIC_MAC_CTRL_RESET_FF_TX_CLK | + FBNIC_MAC_CTRL_RESET_TX_CLK | + FBNIC_MAC_CTRL_RESET_FF_RX_CLK | + FBNIC_MAC_CTRL_RESET_RX_CLK; + + /* Restore LED defaults */ + fbnic_set_led_state_asic(fbd, FBNIC_LED_RESTORE); + } + + /* Check mask for enabled bits, if any set clear and write back */ + if (mask & cmd_cfg) { + wr32(FBNIC_MAC_COMMAND_CONFIG, cmd_cfg & ~mask); + wr32(FBNIC_MAC_CTRL, mac_ctrl); + } + + /* Disable loopback, and flush write */ + wr32(FBNIC_PCS_CONTROL1_0, + FBNIC_PCS_CONTROL1_RESET | + FBNIC_PCS_CONTROL1_SPEED_SELECT_ALWAYS | + FBNIC_PCS_CONTROL1_SPEED_ALWAYS); + wr32(FBNIC_PCS_CONTROL1_1, + FBNIC_PCS_CONTROL1_RESET | + FBNIC_PCS_CONTROL1_SPEED_SELECT_ALWAYS | + FBNIC_PCS_CONTROL1_SPEED_ALWAYS); +} + static const struct fbnic_mac fbnic_mac_asic = { + .enable = fbnic_mac_enable_asic, + .disable = fbnic_mac_disable_asic, .init_regs = fbnic_mac_init_regs, + .get_link = fbnic_mac_get_link_asic, + .get_link_event = fbnic_mac_get_link_event_asic, }; /** diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.h b/drivers/net/ethernet/meta/fbnic/fbnic_mac.h index e78a92338a62..5aa089093206 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_mac.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.h @@ -10,14 +10,72 @@ struct fbnic_dev; #define FBNIC_MAX_JUMBO_FRAME_SIZE 9742 +enum { + FBNIC_LINK_DISABLED = 0, + FBNIC_LINK_DOWN = 1, + FBNIC_LINK_UP = 2, + FBNIC_LINK_EVENT = 3, +}; + +enum { + FBNIC_LED_STROBE_INIT, + FBNIC_LED_ON, + FBNIC_LED_OFF, + FBNIC_LED_RESTORE, +}; + +/* Treat the FEC bits as a bitmask laid out as follows: + * Bit 0: RS Enabled + * Bit 1: BASER(Firecode) Enabled + * Bit 2: Autoneg FEC + */ +enum { + FBNIC_FEC_OFF = 0, + FBNIC_FEC_RS = 1, + FBNIC_FEC_BASER = 2, + FBNIC_FEC_AUTO = 4, +}; + +#define FBNIC_FEC_MODE_MASK (FBNIC_FEC_AUTO - 1) + +/* Treat the link modes as a set of moldulation/lanes bitmask: + * Bit 0: Lane Count, 0 = R1, 1 = R2 + * Bit 1: Modulation, 0 = NRZ, 1 = PAM4 + * Bit 2: Autoneg Modulation/Lane Configuration + */ +enum { + FBNIC_LINK_25R1 = 0, + FBNIC_LINK_50R2 = 1, + FBNIC_LINK_50R1 = 2, + FBNIC_LINK_100R2 = 3, + FBNIC_LINK_AUTO = 4, +}; + +#define FBNIC_LINK_MODE_R2 (FBNIC_LINK_50R2) +#define FBNIC_LINK_MODE_PAM4 (FBNIC_LINK_50R1) +#define FBNIC_LINK_MODE_MASK (FBNIC_LINK_AUTO - 1) + /* This structure defines the interface hooks for the MAC. The MAC hooks * will be configured as a const struct provided with a set of function * pointers. * + * bool (*get_link)(struct fbnic_dev *fbd); + * Get the current link state for the MAC. + * int (*get_link_event)(struct fbnic_dev *fbd) + * Get the current link event status, reports true if link has + * changed to either up (1) or down (-1). + * void (*enable)(struct fbnic_dev *fbd); + * Configure and enable MAC to enable link if not already enabled + * void (*disable)(struct fbnic_dev *fbd); + * Shutdown the link if we are the only consumer of it. * void (*init_regs)(struct fbnic_dev *fbd); * Initialize MAC registers to enable Tx/Rx paths and FIFOs. */ struct fbnic_mac { + bool (*get_link)(struct fbnic_dev *fbd); + int (*get_link_event)(struct fbnic_dev *fbd); + int (*enable)(struct fbnic_dev *fbd); + void (*disable)(struct fbnic_dev *fbd); void (*init_regs)(struct fbnic_dev *fbd); }; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c index bbc2f21060dc..c49ace7f2156 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c @@ -45,6 +45,10 @@ int __fbnic_open(struct fbnic_net *fbn) if (err) goto release_ownership; + err = fbnic_mac_enable(fbd); + if (err) + goto release_ownership; + return 0; release_ownership: fbnic_fw_xmit_ownership_msg(fbn->fbd, false); @@ -72,6 +76,7 @@ static int fbnic_stop(struct net_device *netdev) struct fbnic_net *fbn = netdev_priv(netdev); fbnic_down(fbn); + fbnic_mac_disable(fbn->fbd); fbnic_fw_xmit_ownership_msg(fbn->fbd, false); @@ -146,6 +151,13 @@ struct net_device *fbnic_netdev_alloc(struct fbnic_dev *fbd) netdev->min_mtu = IPV6_MIN_MTU; netdev->max_mtu = FBNIC_MAX_JUMBO_FRAME_SIZE - ETH_HLEN; + /* Default to accept pause frames w/ attempt to autoneg the value */ + fbn->autoneg_pause = true; + fbn->rx_pause = true; + fbn->tx_pause = false; + + fbn->fec = FBNIC_FEC_AUTO | FBNIC_FEC_RS; + fbn->link_mode = FBNIC_LINK_AUTO | FBNIC_LINK_50R2; netif_carrier_off(netdev); netif_tx_stop_all_queues(netdev); diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h index 18f93e9431cc..3976fb1a0eac 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h @@ -22,9 +22,16 @@ struct fbnic_net { u16 num_napi; + u8 autoneg_pause; + u8 tx_pause; + u8 rx_pause; + u8 fec; + u8 link_mode; + u16 num_tx_queues; u16 num_rx_queues; + u64 link_down_events; struct list_head napis; }; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index 8408f0d5f54a..f243950c68bb 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -160,6 +160,78 @@ void fbnic_down(struct fbnic_net *fbn) fbnic_flush(fbn); } +static char *fbnic_report_fec(struct fbnic_dev *fbd) +{ + struct fbnic_net *fbn = netdev_priv(fbd->netdev); + + if (fbn->link_mode & FBNIC_LINK_MODE_PAM4) + return "Clause 91 RS(544,514)"; + + switch (fbn->fec & FBNIC_FEC_MODE_MASK) { + case FBNIC_FEC_OFF: + return "Off"; + case FBNIC_FEC_BASER: + return "Clause 74 BaseR"; + case FBNIC_FEC_RS: + return "Clause 91 RS(528,514)"; + } + + return "Unknown"; +} + +static void fbnic_link_check(struct fbnic_dev *fbd) +{ + struct net_device *netdev = fbd->netdev; + bool link_found = false; + int err; + + err = fbnic_mac_get_link(fbd, &link_found); + if (err) { + /* TBD: For now we do nothing. In the future we should have + * the link_check function request a reset. + * + * We would do this here as the reset will likely involve + * us having to tear down the interface which will require + * us taking the RTNL lock in order to coordinate the + * teardown and bringup before and after the reset. + */ + return; + } + + if (!link_found) { + if (netif_carrier_ok(netdev)) { + struct fbnic_net *fbn = netdev_priv(netdev); + + netdev_err(netdev, "NIC Link is Down\n"); + fbn->link_down_events++; + } + netif_carrier_off(netdev); + return; + } + + if (!netif_carrier_ok(netdev)) { + struct fbnic_net *fbn = netdev_priv(netdev); + + netdev_info(netdev, + "NIC Link is Up, %d Mbps (%s), Flow control: %s\n", + ((fbn->link_mode & FBNIC_LINK_MODE_PAM4) ? + 50000 : 25000) * + ((fbn->link_mode & FBNIC_LINK_MODE_R2) ? + 2 : 1), + (fbn->link_mode & FBNIC_LINK_MODE_PAM4) ? + "PAM4" : "NRZ", + (fbn->rx_pause ? + (fbn->tx_pause ? "ON - Tx/Rx" : "ON - Rx") : + (fbn->tx_pause ? "ON - Tx" : "OFF"))); + netdev_info(netdev, "FEC autoselect %s encoding: %s\n", + (fbn->fec & FBNIC_FEC_AUTO) ? + "enabled" : "disabled", + fbnic_report_fec(fbd)); + fbnic_config_drop_mode(fbn); + } + netif_carrier_on(netdev); +} + static void fbnic_health_check(struct fbnic_dev *fbd) { struct fbnic_fw_mbx *tx_mbx = &fbd->mbx[FBNIC_IPC_MBX_TX_IDX]; @@ -192,6 +264,7 @@ static void fbnic_service_task(struct work_struct *work) rtnl_lock(); fbnic_fw_check_heartbeat(fbd); + fbnic_link_check(fbd); fbnic_health_check(fbd); diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c index 484cab7342da..2967ff53305a 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c @@ -1031,9 +1031,14 @@ static void fbnic_enable_bdq(struct fbnic_ring *hpq, struct fbnic_ring *ppq) static void fbnic_config_drop_mode_rcq(struct fbnic_napi_vector *nv, struct fbnic_ring *rcq) { + struct fbnic_net *fbn = netdev_priv(nv->napi.dev); u32 drop_mode, rcq_ctl; - drop_mode = FBNIC_QUEUE_RDE_CTL0_DROP_IMMEDIATE; + /* Drop mode is only supported on when flow control is disabled */ + if (!fbn->tx_pause) + drop_mode = FBNIC_QUEUE_RDE_CTL0_DROP_IMMEDIATE; + else + drop_mode = FBNIC_QUEUE_RDE_CTL0_DROP_NEVER; /* Specify packet layout */ rcq_ctl = FIELD_PREP(FBNIC_QUEUE_RDE_CTL0_DROP_MODE_MASK, drop_mode) | @@ -1043,6 +1048,20 @@ static void fbnic_config_drop_mode_rcq(struct fbnic_napi_vector *nv, fbnic_ring_wr32(rcq, FBNIC_QUEUE_RDE_CTL0, rcq_ctl); } +void fbnic_config_drop_mode(struct fbnic_net *fbn) +{ + struct fbnic_napi_vector *nv; + int i; + + list_for_each_entry(nv, &fbn->napis, napis) { + for (i = 0; i < nv->rxt_count; i++) { + struct fbnic_q_triad *qt = &nv->qt[nv->txt_count + i]; + + fbnic_config_drop_mode_rcq(nv, &qt->cmpl); + } + } +} + static void fbnic_enable_rcq(struct fbnic_napi_vector *nv, struct fbnic_ring *rcq) { diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h index 200f3b893d02..812e4bb245db 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h @@ -99,6 +99,7 @@ void fbnic_napi_enable(struct fbnic_net *fbn); void fbnic_napi_disable(struct fbnic_net *fbn); void fbnic_enable(struct fbnic_net *fbn); void fbnic_disable(struct fbnic_net *fbn); +void fbnic_config_drop_mode(struct fbnic_net *fbn); void fbnic_flush(struct fbnic_net *fbn); void fbnic_fill(struct fbnic_net *fbn); From patchwork Wed Apr 3 20:09:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616639 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B935E156240 for ; Wed, 3 Apr 2024 20:09:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174960; cv=none; b=XVyvmquLk6nabgcw3WvyqdMsbCKnPyokWV49V8uGGMhaymQcq92ect1pnGSXs4HsxdsLiKpAGyEMCFulWKrYTTSn3W+waTdq425McknD9FOsPo60KwBn+/lrKrMy/PmosgxU7K9qRypKKBILIV4X3pRqc2yjGDDvs8Hkyg79lnY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174960; c=relaxed/simple; bh=hXHLOrUbaPK4t8JTLIRRCdriWfmTxhWllRIOfi6fA6Y=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dlYlOaVVS0XhM1ZEkMBhRM0wJQvKrRwCfiN0qedOncsFSEEnIuxGS9seTyQce8QMzITQTXFeZDEN/ylNtosjk0W1Pc2rzqpuDJCgYet6Mg8xZSGTC5mbGCMbS72TU3bLW/H9G+fDm3Ud/uf5Nd6plmKRBVc19w5ppzVD7uozOIU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YMUzLPiR; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YMUzLPiR" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6e6b6f86975so160815b3a.1 for ; Wed, 03 Apr 2024 13:09:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174958; x=1712779758; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=db2Zi8gUKhvG2OYZvelvNuSUVR4ymhA84T8mia8RnMY=; b=YMUzLPiRQgITv+w1Oa/86Qa02gvg64CUi/63cyk+EgkVav1loY2knHKaQ1FxsLvRf8 g1NYN1BM6r2X8yThexH+1+BsuklBsnzrQE1S6jVY/9R7nmicjSHrai2Yqa512KbzrHpp emlO6O0fWo33KCNT5U4xh0DeskHixnbIxbvuUwyAjreEtSg696s9MUhvH/kEu6VtStZ9 n+bXO8tXfJ7iNxS6uwygnvjVun2BWlMKTUiAYBa2qcI9w4NRIhnYzQ4PQaFsfUq3zooi gu8bm/ad1ryRodrzYkNnpoddrv9JcEWwlihJlSJfp4paBT4UU/FW37STy+8GWG0cJ2w4 OvCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174958; x=1712779758; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=db2Zi8gUKhvG2OYZvelvNuSUVR4ymhA84T8mia8RnMY=; b=BhignIw1PxR4VXhxwzCvH+kyfbQ9BIKwfeXDj4wWXCQWb7IBaJH/1bdbZAVtEF9MjW PxBpD9HeaEuHQkK0yvd9jfsUFZsr/GW2H2WANtJRNOo8WyPfvsDaBM4ViLeEQhT3moVp h7StWhg967b1fzZuZfe0Rh/HPAxAkwc1aKP9E1QdL0lYFzf2mbvu2+q38WA2jq+3a5cm i8QIddhZ7+I+hib+Sm2WNqM4IC3nAAPKXPPyqoN6FkGqVkGGEpRtomHSM/KocrVzMVjb ZRfviWWaHTaUJvUPwbM6SQMgbZ+JXe2QFyB84RBVhwYsYtdTPHa1tTcGIGDUoOVtN459 sq4Q== X-Gm-Message-State: AOJu0YyzB4CwfXJ8hVDP+3QCOJ6ZdlUqAUHuY6T4RjC5XMwn9LOuRfB3 8fLf6WNA3btXSk5t2UtMnsgGFY/hJd16EN6cZbBhLLMRzzPre7ga X-Google-Smtp-Source: AGHT+IETIRKbtHiaARtw3B8Q/8zXV9jvs2TD+tqwE8unGxLHef47mACFffe62Cj9IjTr4dcu5qBVUQ== X-Received: by 2002:a05:6a00:140e:b0:6ec:da37:d09 with SMTP id l14-20020a056a00140e00b006ecda370d09mr642459pfu.16.1712174957923; Wed, 03 Apr 2024 13:09:17 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id j18-20020a632312000000b005d6a0b2efb3sm12033422pgj.21.2024.04.03.13.09.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:09:17 -0700 (PDT) Subject: [net-next PATCH 12/15] eth: fbnic: add basic Tx handling From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:09:15 -0700 Message-ID: <171217495560.1598374.4872808628328829588.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Handle Tx of simple packets. Support checksum offload and gather. Use .ndo_features_check to make sure packet geometry will be supported by the HW, i.e. we can fit the header lengths into the descriptor fields. The device writes to the completion rings the position of the tail (consumer) pointer. Read all those writebacks, obviously the last one will be the most recent, complete skbs up to that point. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 66 ++++ drivers/net/ethernet/meta/fbnic/fbnic_netdev.c | 9 + drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 395 ++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_txrx.h | 15 + 4 files changed, 483 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index 39c98d2dce12..0819ddc1dcc8 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -24,6 +24,72 @@ #define FBNIC_CLOCK_FREQ (600 * (1000 * 1000)) +/* Transmit Work Descriptor Format */ +/* Length, Type, Offset Masks and Shifts */ +#define FBNIC_TWD_L2_HLEN_MASK DESC_GENMASK(5, 0) + +#define FBNIC_TWD_L3_TYPE_MASK DESC_GENMASK(7, 6) +enum { + FBNIC_TWD_L3_TYPE_OTHER = 0, + FBNIC_TWD_L3_TYPE_IPV4 = 1, + FBNIC_TWD_L3_TYPE_IPV6 = 2, + FBNIC_TWD_L3_TYPE_V6V6 = 3, +}; + +#define FBNIC_TWD_L3_OHLEN_MASK DESC_GENMASK(15, 8) +#define FBNIC_TWD_L3_IHLEN_MASK DESC_GENMASK(23, 16) + +enum { + FBNIC_TWD_L4_TYPE_OTHER = 0, + FBNIC_TWD_L4_TYPE_TCP = 1, + FBNIC_TWD_L4_TYPE_UDP = 2, +}; + +#define FBNIC_TWD_CSUM_OFFSET_MASK DESC_GENMASK(27, 24) +#define FBNIC_TWD_L4_HLEN_MASK DESC_GENMASK(31, 28) + +/* Flags and Type */ +#define FBNIC_TWD_L4_TYPE_MASK DESC_GENMASK(33, 32) +#define FBNIC_TWD_FLAG_REQ_TS DESC_BIT(34) +#define FBNIC_TWD_FLAG_REQ_LSO DESC_BIT(35) +#define FBNIC_TWD_FLAG_REQ_CSO DESC_BIT(36) +#define FBNIC_TWD_FLAG_REQ_COMPLETION DESC_BIT(37) +#define FBNIC_TWD_FLAG_DEST_MAC DESC_BIT(43) +#define FBNIC_TWD_FLAG_DEST_BMC DESC_BIT(44) +#define FBNIC_TWD_FLAG_DEST_FW DESC_BIT(45) +#define FBNIC_TWD_TYPE_MASK DESC_GENMASK(47, 46) +enum { + FBNIC_TWD_TYPE_META = 0, + FBNIC_TWD_TYPE_OPT_META = 1, + FBNIC_TWD_TYPE_AL = 2, + FBNIC_TWD_TYPE_LAST_AL = 3, +}; + +/* MSS and Completion Req */ +#define FBNIC_TWD_MSS_MASK DESC_GENMASK(61, 48) + +#define FBNIC_TWD_TS_MASK DESC_GENMASK(39, 0) +#define FBNIC_TWD_ADDR_MASK DESC_GENMASK(45, 0) +#define FBNIC_TWD_LEN_MASK DESC_GENMASK(63, 48) + +/* Tx Completion Descriptor Format */ +#define FBNIC_TCD_TYPE0_HEAD0_MASK DESC_GENMASK(15, 0) +#define FBNIC_TCD_TYPE0_HEAD1_MASK DESC_GENMASK(31, 16) + +#define FBNIC_TCD_TYPE1_TS_MASK DESC_GENMASK(39, 0) + +#define FBNIC_TCD_STATUS_MASK DESC_GENMASK(59, 48) +#define FBNIC_TCD_STATUS_TS_INVALID DESC_BIT(48) +#define FBNIC_TCD_STATUS_ILLEGAL_TS_REQ DESC_BIT(49) +#define FBNIC_TCD_TWQ1 DESC_BIT(60) +#define FBNIC_TCD_TYPE_MASK DESC_GENMASK(62, 61) +enum { + FBNIC_TCD_TYPE_0 = 0, + FBNIC_TCD_TYPE_1 = 1, +}; + +#define FBNIC_TCD_DONE DESC_BIT(63) + /* Rx Buffer Descriptor Format */ #define FBNIC_BD_PAGE_ADDR_MASK DESC_GENMASK(45, 12) #define FBNIC_BD_PAGE_ID_MASK DESC_GENMASK(63, 48) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c index c49ace7f2156..91d4ea2bfb29 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c @@ -91,6 +91,7 @@ static const struct net_device_ops fbnic_netdev_ops = { .ndo_stop = fbnic_stop, .ndo_validate_addr = eth_validate_addr, .ndo_start_xmit = fbnic_xmit_frame, + .ndo_features_check = fbnic_features_check, }; void fbnic_reset_queues(struct fbnic_net *fbn, @@ -148,6 +149,14 @@ struct net_device *fbnic_netdev_alloc(struct fbnic_dev *fbd) fbnic_reset_queues(fbn, default_queues, default_queues); + netdev->features |= + NETIF_F_SG | + NETIF_F_HW_CSUM; + + netdev->hw_features |= netdev->features; + netdev->vlan_features |= netdev->features; + netdev->hw_enc_features |= netdev->features; + netdev->min_mtu = IPV6_MIN_MTU; netdev->max_mtu = FBNIC_MAX_JUMBO_FRAME_SIZE - ETH_HLEN; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c index 2967ff53305a..ad4cb059c959 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) Meta Platforms, Inc. and affiliates. */ +#include #include #include #include @@ -10,6 +11,14 @@ #include "fbnic_txrx.h" #include "fbnic.h" +struct fbnic_xmit_cb { + u32 bytecount; + u8 desc_count; + int hw_head; +}; + +#define FBNIC_XMIT_CB(_skb) ((struct fbnic_xmit_cb *)(skb->cb)) + static u32 __iomem *fbnic_ring_csr_base(const struct fbnic_ring *ring) { unsigned long csr_base = (unsigned long)ring->doorbell; @@ -38,12 +47,313 @@ static inline unsigned int fbnic_desc_unused(struct fbnic_ring *ring) return (ring->head - ring->tail - 1) & ring->size_mask; } -netdev_tx_t fbnic_xmit_frame(struct sk_buff *skb, struct net_device *dev) +static inline unsigned int fbnic_desc_used(struct fbnic_ring *ring) +{ + return (ring->tail - ring->head) & ring->size_mask; +} + +static inline struct netdev_queue *txring_txq(const struct net_device *dev, + const struct fbnic_ring *ring) +{ + return netdev_get_tx_queue(dev, ring->q_idx); +} + +static inline int fbnic_maybe_stop_tx(const struct net_device *dev, + struct fbnic_ring *ring, + const unsigned int size) +{ + struct netdev_queue *txq = txring_txq(dev, ring); + int res; + + res = netif_txq_maybe_stop(txq, fbnic_desc_unused(ring), size, + FBNIC_TX_DESC_WAKEUP); + + return !res; +} + +static inline bool fbnic_tx_sent_queue(struct sk_buff *skb, + struct fbnic_ring *ring) +{ + struct netdev_queue *dev_queue = txring_txq(skb->dev, ring); + unsigned int bytecount = FBNIC_XMIT_CB(skb)->bytecount; + bool xmit_more = netdev_xmit_more(); + + /* TBD: Request completion more often if xmit_more becomes large */ + + return __netdev_tx_sent_queue(dev_queue, bytecount, xmit_more); +} + +static void fbnic_unmap_single_twd(struct device *dev, __le64 *twd) +{ + u64 raw_twd = le64_to_cpu(*twd); + unsigned int len; + dma_addr_t dma; + + dma = FIELD_GET(FBNIC_TWD_ADDR_MASK, raw_twd); + len = FIELD_GET(FBNIC_TWD_LEN_MASK, raw_twd); + + dma_unmap_single(dev, dma, len, DMA_TO_DEVICE); +} + +static void fbnic_unmap_page_twd(struct device *dev, __le64 *twd) +{ + u64 raw_twd = le64_to_cpu(*twd); + unsigned int len; + dma_addr_t dma; + + dma = FIELD_GET(FBNIC_TWD_ADDR_MASK, raw_twd); + len = FIELD_GET(FBNIC_TWD_LEN_MASK, raw_twd); + + dma_unmap_page(dev, dma, len, DMA_TO_DEVICE); +} + +#define FBNIC_TWD_TYPE(_type) \ + cpu_to_le64(FIELD_PREP(FBNIC_TWD_TYPE_MASK, FBNIC_TWD_TYPE_##_type)) + +static bool +fbnic_tx_offloads(struct fbnic_ring *ring, struct sk_buff *skb, __le64 *meta) +{ + unsigned int l2len, i3len; + + if (unlikely(skb->ip_summed != CHECKSUM_PARTIAL)) + return false; + + l2len = skb_mac_header_len(skb); + i3len = skb_checksum_start(skb) - skb_network_header(skb); + + *meta |= cpu_to_le64(FIELD_PREP(FBNIC_TWD_CSUM_OFFSET_MASK, + skb->csum_offset / 2)); + + *meta |= cpu_to_le64(FBNIC_TWD_FLAG_REQ_CSO); + + *meta |= cpu_to_le64(FIELD_PREP(FBNIC_TWD_L2_HLEN_MASK, l2len / 2) | + FIELD_PREP(FBNIC_TWD_L3_IHLEN_MASK, i3len / 2)); + return false; +} + +static bool +fbnic_tx_map(struct fbnic_ring *ring, struct sk_buff *skb, __le64 *meta) { + struct device *dev = skb->dev->dev.parent; + unsigned int tail = ring->tail, first; + unsigned int size, data_len; + skb_frag_t *frag; + dma_addr_t dma; + __le64 *twd; + + ring->tx_buf[tail] = skb; + + tail++; + tail &= ring->size_mask; + first = tail; + + size = skb_headlen(skb); + data_len = skb->data_len; + + if (size > FIELD_MAX(FBNIC_TWD_LEN_MASK)) + goto dma_error; + + dma = dma_map_single(dev, skb->data, size, DMA_TO_DEVICE); + + for (frag = &skb_shinfo(skb)->frags[0];; frag++) { + twd = &ring->desc[tail]; + + if (dma_mapping_error(dev, dma)) + goto dma_error; + + *twd = cpu_to_le64(FIELD_PREP(FBNIC_TWD_ADDR_MASK, dma) | + FIELD_PREP(FBNIC_TWD_LEN_MASK, size) | + FIELD_PREP(FBNIC_TWD_TYPE_MASK, + FBNIC_TWD_TYPE_AL)); + + tail++; + tail &= ring->size_mask; + + if (!data_len) + break; + + size = skb_frag_size(frag); + data_len -= size; + + if (size > FIELD_MAX(FBNIC_TWD_LEN_MASK)) + goto dma_error; + + dma = skb_frag_dma_map(dev, frag, 0, size, DMA_TO_DEVICE); + } + + *twd |= FBNIC_TWD_TYPE(LAST_AL); + + FBNIC_XMIT_CB(skb)->desc_count = ((twd - meta) + 1) & ring->size_mask; + + ring->tail = tail; + + /* Verify there is room for another packet */ + fbnic_maybe_stop_tx(skb->dev, ring, FBNIC_MAX_SKB_DESC); + + if (fbnic_tx_sent_queue(skb, ring)) { + *meta |= cpu_to_le64(FBNIC_TWD_FLAG_REQ_COMPLETION); + + /* Force DMA writes to flush before writing to tail */ + dma_wmb(); + + writel(tail, ring->doorbell); + } + + return false; +dma_error: + if (net_ratelimit()) + netdev_err(skb->dev, "TX DMA map failed\n"); + + while (tail != first) { + tail--; + tail &= ring->size_mask; + twd = &ring->desc[tail]; + if (tail == first) + fbnic_unmap_single_twd(dev, twd); + else + fbnic_unmap_page_twd(dev, twd); + } + + return true; +} + +#define FBNIC_MIN_FRAME_LEN 60 + +static netdev_tx_t +fbnic_xmit_frame_ring(struct sk_buff *skb, struct fbnic_ring *ring) +{ + __le64 *meta = &ring->desc[ring->tail]; + u16 desc_needed; + + if (skb_put_padto(skb, FBNIC_MIN_FRAME_LEN)) + goto err_count; + + /* need: 1 descriptor per page, + * + 1 desc for skb_head, + * + 2 desc for metadata and timestamp metadata + * + 7 desc gap to keep tail from touching head + * otherwise try next time + */ + desc_needed = skb_shinfo(skb)->nr_frags + 10; + if (fbnic_maybe_stop_tx(skb->dev, ring, desc_needed)) + return NETDEV_TX_BUSY; + + *meta = cpu_to_le64(FBNIC_TWD_FLAG_DEST_MAC); + + /* Write all members within DWORD to condense this into 2 4B writes */ + FBNIC_XMIT_CB(skb)->bytecount = skb->len; + FBNIC_XMIT_CB(skb)->desc_count = 0; + + if (fbnic_tx_offloads(ring, skb, meta)) + goto err_free; + + if (fbnic_tx_map(ring, skb, meta)) + goto err_free; + + return NETDEV_TX_OK; + +err_free: dev_kfree_skb_any(skb); +err_count: return NETDEV_TX_OK; } +netdev_tx_t fbnic_xmit_frame(struct sk_buff *skb, struct net_device *dev) +{ + struct fbnic_net *fbn = netdev_priv(dev); + unsigned int q_map = skb->queue_mapping; + + return fbnic_xmit_frame_ring(skb, fbn->tx[q_map]); +} + +netdev_features_t +fbnic_features_check(struct sk_buff *skb, struct net_device *dev, + netdev_features_t features) +{ + unsigned int l2len, l3len; + + if (unlikely(skb->ip_summed != CHECKSUM_PARTIAL)) + return features; + + l2len = skb_mac_header_len(skb); + l3len = skb_checksum_start(skb) - skb_network_header(skb); + + /* Check header lengths are multiple of 2. + * In case of 6in6 we support longer headers (IHLEN + OHLEN) + * but keep things simple for now, 512B is plenty. + */ + if ((l2len | l3len | skb->csum_offset) % 2 || + !FIELD_FIT(FBNIC_TWD_L2_HLEN_MASK, l2len / 2) || + !FIELD_FIT(FBNIC_TWD_L3_IHLEN_MASK, l3len / 2) || + !FIELD_FIT(FBNIC_TWD_CSUM_OFFSET_MASK, skb->csum_offset / 2)) + return features & ~NETIF_F_CSUM_MASK; + + return features; +} + +static void fbnic_clean_twq0(struct fbnic_napi_vector *nv, int napi_budget, + struct fbnic_ring *ring, bool discard, + unsigned int hw_head) +{ + u64 total_bytes = 0, total_packets = 0; + unsigned int head = ring->head; + struct netdev_queue *txq; + unsigned int clean_desc; + + clean_desc = (hw_head - head) & ring->size_mask; + + while (clean_desc) { + struct sk_buff *skb = ring->tx_buf[head]; + unsigned int desc_cnt; + + desc_cnt = FBNIC_XMIT_CB(skb)->desc_count; + if (desc_cnt > clean_desc) + break; + + ring->tx_buf[head] = NULL; + + clean_desc -= desc_cnt; + + while (!(ring->desc[head] & FBNIC_TWD_TYPE(AL))) { + head++; + head &= ring->size_mask; + desc_cnt--; + } + + fbnic_unmap_single_twd(nv->dev, &ring->desc[head]); + head++; + head &= ring->size_mask; + desc_cnt--; + + while (desc_cnt--) { + fbnic_unmap_page_twd(nv->dev, &ring->desc[head]); + head++; + head &= ring->size_mask; + } + + total_bytes += FBNIC_XMIT_CB(skb)->bytecount; + total_packets += 1; + + napi_consume_skb(skb, napi_budget); + } + + if (!total_bytes) + return; + + ring->head = head; + + txq = txring_txq(nv->napi.dev, ring); + + if (unlikely(discard)) { + netdev_tx_completed_queue(txq, total_packets, total_bytes); + return; + } + + netif_txq_completed_wake(txq, total_packets, total_bytes, + fbnic_desc_unused(ring), + FBNIC_TX_DESC_WAKEUP); +} + static void fbnic_page_pool_init(struct fbnic_ring *ring, unsigned int idx, struct page *page) { @@ -66,6 +376,65 @@ static void fbnic_page_pool_drain(struct fbnic_ring *ring, unsigned int idx, rx_buf->page = NULL; } +static void fbnic_clean_twq(struct fbnic_napi_vector *nv, int napi_budget, + struct fbnic_q_triad *qt, s32 head0) +{ + if (head0 >= 0) + fbnic_clean_twq0(nv, napi_budget, &qt->sub0, false, head0); +} + +static void +fbnic_clean_tcq(struct fbnic_napi_vector *nv, struct fbnic_q_triad *qt, + int napi_budget) +{ + struct fbnic_ring *cmpl = &qt->cmpl; + __le64 *raw_tcd, done; + u32 head = cmpl->head; + s32 head0 = -1; + + done = (head & (cmpl->size_mask + 1)) ? 0 : cpu_to_le64(FBNIC_TCD_DONE); + raw_tcd = &cmpl->desc[head & cmpl->size_mask]; + + /* Walk the completion queue collecting the heads reported by NIC */ + while ((*raw_tcd & cpu_to_le64(FBNIC_TCD_DONE)) == done) { + u64 tcd; + + dma_rmb(); + + tcd = le64_to_cpu(*raw_tcd); + + switch (FIELD_GET(FBNIC_TCD_TYPE_MASK, tcd)) { + case FBNIC_TCD_TYPE_0: + if (!(tcd & FBNIC_TCD_TWQ1)) + head0 = FIELD_GET(FBNIC_TCD_TYPE0_HEAD0_MASK, + tcd); + /* Currently all err status bits are related to + * timestamps and as those have yet to be added + * they are skipped for now. + */ + break; + default: + break; + } + + raw_tcd++; + head++; + if (!(head & cmpl->size_mask)) { + done ^= cpu_to_le64(FBNIC_TCD_DONE); + raw_tcd = &cmpl->desc[0]; + } + } + + /* Record the current head/tail of the queue */ + if (cmpl->head != head) { + cmpl->head = head; + writel(head & cmpl->size_mask, cmpl->doorbell); + } + + /* Unmap and free processed buffers */ + fbnic_clean_twq(nv, napi_budget, qt, head0); +} + static void fbnic_clean_bdq(struct fbnic_napi_vector *nv, int napi_budget, struct fbnic_ring *ring, unsigned int hw_head) { @@ -153,7 +522,7 @@ static void fbnic_put_pkt_buff(struct fbnic_napi_vector *nv, page = virt_to_page(pkt->buff.data_hard_start); page_pool_put_full_page(nv->page_pool, page, !!budget); pkt->buff.data_hard_start = NULL; -} +}; static void fbnic_nv_irq_disable(struct fbnic_napi_vector *nv) { @@ -163,8 +532,27 @@ static void fbnic_nv_irq_disable(struct fbnic_napi_vector *nv) wr32(FBNIC_INTR_MASK_SET(v_idx / 32), 1 << (v_idx % 32)); } +static void fbnic_nv_irq_rearm(struct fbnic_napi_vector *nv) +{ + struct fbnic_dev *fbd = nv->fbd; + u32 v_idx = nv->v_idx; + + wr32(FBNIC_INTR_CQ_REARM(v_idx), FBNIC_INTR_CQ_REARM_INTR_UNMASK); +} + static int fbnic_poll(struct napi_struct *napi, int budget) { + struct fbnic_napi_vector *nv = container_of(napi, + struct fbnic_napi_vector, + napi); + int i; + + for (i = 0; i < nv->txt_count; i++) + fbnic_clean_tcq(nv, &nv->qt[i], budget); + + if (likely(napi_complete_done(napi, 0))) + fbnic_nv_irq_rearm(nv); + return 0; } @@ -896,6 +1284,9 @@ void fbnic_flush(struct fbnic_net *fbn) struct fbnic_q_triad *qt = &nv->qt[i]; struct netdev_queue *tx_queue; + /* Clean the work queues of unprocessed work */ + fbnic_clean_twq0(nv, 0, &qt->sub0, true, qt->sub0.tail); + /* Reset completion queue descriptor ring */ memset(qt->cmpl.desc, 0, qt->cmpl.size); diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h index 812e4bb245db..0c424c49866d 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h @@ -10,6 +10,18 @@ struct fbnic_net; +/* Guarantee we have space needed for storing the buffer + * To store the buffer we need: + * 1 descriptor per page + * + 1 descriptor for skb head + * + 2 descriptors for metadata and optional metadata + * + 7 descriptors to keep tail out of the same cachline as head + * If we cannot guarantee that then we should return TX_BUSY + */ +#define FBNIC_MAX_SKB_DESC (MAX_SKB_FRAGS + 10) +#define FBNIC_TX_DESC_WAKEUP (FBNIC_MAX_SKB_DESC * 2) +#define FBNIC_TX_DESC_MIN roundup_pow_of_two(FBNIC_TX_DESC_WAKEUP) + #define FBNIC_MAX_TXQS 128u #define FBNIC_MAX_RXQS 128u @@ -90,6 +102,9 @@ struct fbnic_napi_vector { #define FBNIC_MAX_RXQS 128u netdev_tx_t fbnic_xmit_frame(struct sk_buff *skb, struct net_device *dev); +netdev_features_t +fbnic_features_check(struct sk_buff *skb, struct net_device *dev, + netdev_features_t features); int fbnic_alloc_napi_vectors(struct fbnic_net *fbn); void fbnic_free_napi_vectors(struct fbnic_net *fbn); From patchwork Wed Apr 3 20:09:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616640 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9224C155753 for ; Wed, 3 Apr 2024 20:09:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174964; cv=none; b=X9W0AOaclIDTiN2Ergd7ybvVB3EKh1zje4ugg4QbL0UWhIqRB9gfcUWOYuhQwk4Z0gNyIH5ttQep9HzogRf6Rce3o2iMofLrPNJYeF/3ZxdLmhbQsNo7RLknxLl2+V9x2zXtlcogqNjiQngRsxRpY3Kzge0OeIAkNV2oKKIMxuk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174964; c=relaxed/simple; bh=JMwIk43Prwg5QIyglmqg1zVb3NAzQ2l7MJdOq8e+R4E=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=l3XHTKHzIWxqtNEuq667rJ7J6stHJDXj+y9W6cGLaCou64CgZXNwIJwH/KKEq2nfzuqXrqEgd6bNlNOIPWcM2TYW/ZaG7Fb+SB0S0JKYjUh+jzZPS+v86tjAnkgsShzX5tCVJpgznLJjh1fw7pPKCa/n7LNeYVnrWUJUX/n1mbM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QvuP4c8R; arc=none smtp.client-ip=209.85.216.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QvuP4c8R" Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-29f93c4946cso134240a91.1 for ; Wed, 03 Apr 2024 13:09:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174962; x=1712779762; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=K7v/8hzcg/KMvKwGHaAddaJQKNMrQ7jwRIzNwuM6XpE=; b=QvuP4c8RvjctVGwax2JNv86E1p5GbqOb2eqehFlAGZkZ3YRP+PYgwryS2MM6bBoiGW gMPtEY/dz9HMLmyUvE4Bekk5UtiE+TnndyV+7CzomKbbnRJ64NnDK3n66tzFLnAKA3wA 6dgCJiqix0oOzDL0K8QseqIUkqlm2mwtEaZU2WzTuTNnePPha4RS4/+5oZtvNctGB/0l TjrUgzZyM093wOb3MvjIxvg4ZRMrBKi4/5UC+p0aT0bF5AUEmGu0zcrs0AkM0oI2mTOT f48BqBw0/gkXZFNgypLm96ORd25iMNkDXxMaG6zTbjyJaVMlOz93ERcke6RvAZTe72nK Fd3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174962; x=1712779762; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=K7v/8hzcg/KMvKwGHaAddaJQKNMrQ7jwRIzNwuM6XpE=; b=o3FGxG+iSTLnxPW3nHmibAu9wFRoMLrzuMSaryr4aOEUhheNGyiGk0MtV1iVauaNhE nJJJL6k+Ndkw8aDtTPDFsxmmjSzL0loIORrlqbf61pQ59wMBvSthqZE5bRaFUNZRcKu/ qGbjE26gfJzDuk1N383XVeqDvkigIUg8muml4yMFF2LnoYIGfAy5m6OscsREt190ZheY Mulr7BPRFXKifrEyyegv0bya/O8Y0ijtLNGORYpwaudm6f1d8d+Wip827/1IPiOp26Q0 zLS8HpFs5v5mlvOPgNc5O0LgXgmMU2+mQuy2rpZu1pj5tXzo4gM4sXWomvf0HD18V5kb 6QFg== X-Gm-Message-State: AOJu0Yxwz9xpqoICjirH3vPCguOQVMRi4sej9+u4HR+4JjQbUtVm+4vN xWvHulGtHdbjdt9nhz7XEizwh3xlYrJ56V4+UF8ZymAu+zZTdhsR X-Google-Smtp-Source: AGHT+IEsb90+wnaVQW15wBcjoAsOblK/DFNGPqvbc9TXAndOE1SbNiI9GKq1XXgLFwuzqRtIxpqlMQ== X-Received: by 2002:a17:90a:c696:b0:2a2:a5c7:8ac4 with SMTP id n22-20020a17090ac69600b002a2a5c78ac4mr572779pjt.37.1712174961949; Wed, 03 Apr 2024 13:09:21 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([98.97.103.43]) by smtp.gmail.com with ESMTPSA id m6-20020a17090a34c600b0029be7922b32sm104923pjf.26.2024.04.03.13.09.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:09:21 -0700 (PDT) Subject: [net-next PATCH 13/15] eth: fbnic: add basic Rx handling From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:09:20 -0700 Message-ID: <171217496013.1598374.10126180029382922588.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Handle Rx packets with basic csum and Rx hash offloads. NIC writes back to the completion ring a head buffer descriptor (data buffer allocated from header pages), variable number of payload descriptors (data buffers in payload pages), an optional metadata descriptor (type 3) and finally the primary metadata descriptor (type 2). This format makes scatter support fairly easy - start gathering the pages when we see head page, gather until we see the primary metadata descriptor, do the processing. Use XDP infra to collect the packet fragments as we traverse the descriptors. XDP itself is not supported yet, but it will be soon. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 60 ++++ drivers/net/ethernet/meta/fbnic/fbnic_netdev.c | 4 drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 3 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 322 ++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_txrx.h | 2 5 files changed, 387 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index 0819ddc1dcc8..f61b401fdd5c 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -90,6 +90,66 @@ enum { #define FBNIC_TCD_DONE DESC_BIT(63) +/* Rx Completion Queue Descriptors */ +#define FBNIC_RCD_TYPE_MASK DESC_GENMASK(62, 61) +enum { + FBNIC_RCD_TYPE_HDR_AL = 0, + FBNIC_RCD_TYPE_PAY_AL = 1, + FBNIC_RCD_TYPE_OPT_META = 2, + FBNIC_RCD_TYPE_META = 3, +}; + +#define FBNIC_RCD_DONE DESC_BIT(63) + +/* Address/Length Completion Descriptors */ +#define FBNIC_RCD_AL_BUFF_ID_MASK DESC_GENMASK(15, 0) +#define FBNIC_RCD_AL_BUFF_LEN_MASK DESC_GENMASK(28, 16) +#define FBNIC_RCD_AL_BUFF_OFF_MASK DESC_GENMASK(43, 32) +#define FBNIC_RCD_AL_PAGE_FIN DESC_BIT(60) + +/* Header AL specific values */ +#define FBNIC_RCD_HDR_AL_OVERFLOW DESC_BIT(53) +#define FBNIC_RCD_HDR_AL_DMA_HINT_MASK DESC_GENMASK(59, 54) +enum { + FBNIC_RCD_HDR_AL_DMA_HINT_NONE = 0, + FBNIC_RCD_HDR_AL_DMA_HINT_L2 = 1, + FBNIC_RCD_HDR_AL_DMA_HINT_L3 = 2, + FBNIC_RCD_HDR_AL_DMA_HINT_L4 = 4, +}; + +/* Optional Metadata Completion Descriptors */ +#define FBNIC_RCD_OPT_META_TS_MASK DESC_GENMASK(39, 0) +#define FBNIC_RCD_OPT_META_ACTION_MASK DESC_GENMASK(45, 40) +#define FBNIC_RCD_OPT_META_ACTION DESC_BIT(57) +#define FBNIC_RCD_OPT_META_TS DESC_BIT(58) +#define FBNIC_RCD_OPT_META_TYPE_MASK DESC_GENMASK(60, 59) + +/* Metadata Completion Descriptors */ +#define FBNIC_RCD_META_RSS_HASH_MASK DESC_GENMASK(31, 0) +#define FBNIC_RCD_META_L2_CSUM_MASK DESC_GENMASK(47, 32) +#define FBNIC_RCD_META_L3_TYPE_MASK DESC_GENMASK(49, 48) +enum { + FBNIC_RCD_META_L3_TYPE_OTHER = 0, + FBNIC_RCD_META_L3_TYPE_IPV4 = 1, + FBNIC_RCD_META_L3_TYPE_IPV6 = 2, + FBNIC_RCD_META_L3_TYPE_V6V6 = 3, +}; + +#define FBNIC_RCD_META_L4_TYPE_MASK DESC_GENMASK(51, 50) +enum { + FBNIC_RCD_META_L4_TYPE_OTHER = 0, + FBNIC_RCD_META_L4_TYPE_TCP = 1, + FBNIC_RCD_META_L4_TYPE_UDP = 2, +}; + +#define FBNIC_RCD_META_L4_CSUM_UNNECESSARY DESC_BIT(52) +#define FBNIC_RCD_META_ERR_MAC_EOP DESC_BIT(53) +#define FBNIC_RCD_META_ERR_TRUNCATED_FRAME DESC_BIT(54) +#define FBNIC_RCD_META_ERR_PARSER DESC_BIT(55) +#define FBNIC_RCD_META_UNCORRECTABLE_ERR_MASK \ + (FBNIC_RCD_META_ERR_MAC_EOP | FBNIC_RCD_META_ERR_TRUNCATED_FRAME) +#define FBNIC_RCD_META_ECN DESC_BIT(60) + /* Rx Buffer Descriptor Format */ #define FBNIC_BD_PAGE_ADDR_MASK DESC_GENMASK(45, 12) #define FBNIC_BD_PAGE_ID_MASK DESC_GENMASK(63, 48) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c index 91d4ea2bfb29..792bdfa7429d 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c @@ -150,8 +150,10 @@ struct net_device *fbnic_netdev_alloc(struct fbnic_dev *fbd) fbnic_reset_queues(fbn, default_queues, default_queues); netdev->features |= + NETIF_F_RXHASH | NETIF_F_SG | - NETIF_F_HW_CSUM; + NETIF_F_HW_CSUM | + NETIF_F_RXCSUM; netdev->hw_features |= netdev->features; netdev->vlan_features |= netdev->features; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index f243950c68bb..d897b0d65abf 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -268,6 +268,9 @@ static void fbnic_service_task(struct work_struct *work) fbnic_health_check(fbd); + if (netif_carrier_ok(fbd->netdev)) + fbnic_napi_depletion_check(fbd->netdev); + if (netif_running(fbd->netdev)) schedule_delayed_work(&fbd->service_task, HZ); diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c index ad4cb059c959..fe35112b9075 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c @@ -131,6 +131,24 @@ fbnic_tx_offloads(struct fbnic_ring *ring, struct sk_buff *skb, __le64 *meta) return false; } +static void +fbnic_rx_csum(u64 rcd, struct sk_buff *skb, struct fbnic_ring *rcq) +{ + skb_checksum_none_assert(skb); + + if (unlikely(!(skb->dev->features & NETIF_F_RXCSUM))) + return; + + if (FIELD_GET(FBNIC_RCD_META_L4_CSUM_UNNECESSARY, rcd)) { + skb->ip_summed = CHECKSUM_UNNECESSARY; + } else { + u16 csum = FIELD_GET(FBNIC_RCD_META_L2_CSUM_MASK, rcd); + + skb->ip_summed = CHECKSUM_COMPLETE; + skb->csum = (__force __wsum)csum; + } +} + static bool fbnic_tx_map(struct fbnic_ring *ring, struct sk_buff *skb, __le64 *meta) { @@ -364,6 +382,16 @@ static void fbnic_page_pool_init(struct fbnic_ring *ring, unsigned int idx, rx_buf->page = page; } +static struct page *fbnic_page_pool_get(struct fbnic_ring *ring, + unsigned int idx) +{ + struct fbnic_rx_buf *rx_buf = &ring->rx_buf[idx]; + + rx_buf->pagecnt_bias--; + + return rx_buf->page; +} + static void fbnic_page_pool_drain(struct fbnic_ring *ring, unsigned int idx, struct fbnic_napi_vector *nv, int budget) { @@ -501,6 +529,98 @@ static void fbnic_fill_bdq(struct fbnic_napi_vector *nv, struct fbnic_ring *bdq) } } +static unsigned int fbnic_hdr_pg_start(unsigned int pg_off) +{ + /* The headroom of the first header may be larger than FBNIC_RX_HROOM + * due to alignment. So account for that by just making the page + * offset 0 if we are starting at the first header. + */ + if (ALIGN(FBNIC_RX_HROOM, 128) > FBNIC_RX_HROOM && + pg_off == ALIGN(FBNIC_RX_HROOM, 128)) + return 0; + + return pg_off - FBNIC_RX_HROOM; +} + +static unsigned int fbnic_hdr_pg_end(unsigned int pg_off, unsigned int len) +{ + /* Determine the end of the buffer by finding the start of the next + * and then subtracting the headroom from that frame. + */ + pg_off += len + FBNIC_RX_TROOM + FBNIC_RX_HROOM; + + return ALIGN(pg_off, 128) - FBNIC_RX_HROOM; +} + +static void fbnic_pkt_prepare(struct fbnic_napi_vector *nv, u64 rcd, + struct fbnic_pkt_buff *pkt, + struct fbnic_q_triad *qt) +{ + unsigned int hdr_pg_off = FIELD_GET(FBNIC_RCD_AL_BUFF_OFF_MASK, rcd); + unsigned int hdr_pg_idx = FIELD_GET(FBNIC_RCD_AL_BUFF_ID_MASK, rcd); + struct page *page = fbnic_page_pool_get(&qt->sub0, hdr_pg_idx); + unsigned int len = FIELD_GET(FBNIC_RCD_AL_BUFF_LEN_MASK, rcd); + unsigned int frame_sz, hdr_pg_start, hdr_pg_end, headroom; + unsigned char *hdr_start; + + /* data_hard_start should always be NULL when this is called */ + WARN_ON_ONCE(pkt->buff.data_hard_start); + + /* Short-cut the end caclulation if we know page is fully consumed */ + hdr_pg_end = FIELD_GET(FBNIC_RCD_AL_PAGE_FIN, rcd) ? + PAGE_SIZE : fbnic_hdr_pg_end(hdr_pg_off, len); + hdr_pg_start = fbnic_hdr_pg_start(hdr_pg_off); + + frame_sz = hdr_pg_end - hdr_pg_start; + xdp_init_buff(&pkt->buff, frame_sz, NULL); + + /* Sync DMA buffer */ + dma_sync_single_range_for_cpu(nv->dev, page_pool_get_dma_addr(page), + hdr_pg_start, frame_sz, + DMA_BIDIRECTIONAL); + + /* Build frame around buffer */ + hdr_start = page_address(page) + hdr_pg_start; + headroom = hdr_pg_off - hdr_pg_start + FBNIC_RX_PAD; + + xdp_prepare_buff(&pkt->buff, hdr_start, headroom, + len - FBNIC_RX_PAD, true); + + pkt->data_truesize = 0; + pkt->data_len = 0; + pkt->nr_frags = 0; +} + +static void fbnic_add_rx_frag(struct fbnic_napi_vector *nv, u64 rcd, + struct fbnic_pkt_buff *pkt, + struct fbnic_q_triad *qt) +{ + unsigned int pg_off = FIELD_GET(FBNIC_RCD_AL_BUFF_OFF_MASK, rcd); + unsigned int pg_idx = FIELD_GET(FBNIC_RCD_AL_BUFF_ID_MASK, rcd); + unsigned int len = FIELD_GET(FBNIC_RCD_AL_BUFF_LEN_MASK, rcd); + struct page *page = fbnic_page_pool_get(&qt->sub1, pg_idx); + struct skb_shared_info *shinfo; + unsigned int truesize; + + truesize = FIELD_GET(FBNIC_RCD_AL_PAGE_FIN, rcd) ? PAGE_SIZE - pg_off : + ALIGN(len, 128); + + /* Sync DMA buffer */ + dma_sync_single_range_for_cpu(nv->dev, page_pool_get_dma_addr(page), + pg_off, truesize, DMA_BIDIRECTIONAL); + + /* Add page to xdp shared info */ + shinfo = xdp_get_shared_info_from_buff(&pkt->buff); + + /* We use gso_segs to store truesize */ + pkt->data_truesize += truesize; + + __skb_fill_page_desc_noacc(shinfo, pkt->nr_frags++, page, pg_off, len); + + /* Store data_len in gso_size */ + pkt->data_len += len; +} + static void fbnic_put_pkt_buff(struct fbnic_napi_vector *nv, struct fbnic_pkt_buff *pkt, int budget) { @@ -522,7 +642,167 @@ static void fbnic_put_pkt_buff(struct fbnic_napi_vector *nv, page = virt_to_page(pkt->buff.data_hard_start); page_pool_put_full_page(nv->page_pool, page, !!budget); pkt->buff.data_hard_start = NULL; -}; +} + +static struct sk_buff *fbnic_build_skb(struct fbnic_napi_vector *nv, + struct fbnic_pkt_buff *pkt) +{ + unsigned int nr_frags = pkt->nr_frags; + struct skb_shared_info *shinfo; + unsigned int truesize; + struct sk_buff *skb; + + truesize = xdp_data_hard_end(&pkt->buff) + FBNIC_RX_TROOM - + pkt->buff.data_hard_start; + + /* Build frame around buffer */ + skb = napi_build_skb(pkt->buff.data_hard_start, truesize); + if (unlikely(!skb)) + return NULL; + + /* Push data pointer to start of data, put tail to end of data */ + skb_reserve(skb, pkt->buff.data - pkt->buff.data_hard_start); + __skb_put(skb, pkt->buff.data_end - pkt->buff.data); + + /* Add tracking for metadata at the start of the frame */ + skb_metadata_set(skb, pkt->buff.data - pkt->buff.data_meta); + + /* Add Rx frags */ + if (nr_frags) { + /* Verify that shared info didn't move */ + shinfo = xdp_get_shared_info_from_buff(&pkt->buff); + WARN_ON(skb_shinfo(skb) != shinfo); + + skb->truesize += pkt->data_truesize; + skb->data_len += pkt->data_len; + shinfo->nr_frags = nr_frags; + skb->len += pkt->data_len; + } + + skb_mark_for_recycle(skb); + + /* Set MAC header specific fields */ + skb->protocol = eth_type_trans(skb, nv->napi.dev); + + return skb; +} + +static enum pkt_hash_types fbnic_skb_hash_type(u64 rcd) +{ + return (FBNIC_RCD_META_L4_TYPE_MASK & rcd) ? PKT_HASH_TYPE_L4 : + (FBNIC_RCD_META_L3_TYPE_MASK & rcd) ? PKT_HASH_TYPE_L3 : + PKT_HASH_TYPE_L2; +} + +static void fbnic_populate_skb_fields(struct fbnic_napi_vector *nv, + u64 rcd, struct sk_buff *skb, + struct fbnic_q_triad *qt) +{ + struct net_device *netdev = nv->napi.dev; + struct fbnic_ring *rcq = &qt->cmpl; + + fbnic_rx_csum(rcd, skb, rcq); + + if (netdev->features & NETIF_F_RXHASH) + skb_set_hash(skb, + FIELD_GET(FBNIC_RCD_META_RSS_HASH_MASK, rcd), + fbnic_skb_hash_type(rcd)); + + skb_record_rx_queue(skb, rcq->q_idx); +} + +static bool fbnic_rcd_metadata_err(u64 rcd) +{ + return !!(FBNIC_RCD_META_UNCORRECTABLE_ERR_MASK & rcd); +} + +static int fbnic_clean_rcq(struct fbnic_napi_vector *nv, + struct fbnic_q_triad *qt, int budget) +{ + struct fbnic_ring *rcq = &qt->cmpl; + struct fbnic_pkt_buff *pkt; + s32 head0 = -1, head1 = -1; + __le64 *raw_rcd, done; + u32 head = rcq->head; + u64 packets = 0; + + done = (head & (rcq->size_mask + 1)) ? cpu_to_le64(FBNIC_RCD_DONE) : 0; + raw_rcd = &rcq->desc[head & rcq->size_mask]; + pkt = rcq->pkt; + + /* Walk the completion queue collecting the heads reported by NIC */ + while (likely(packets < budget)) { + struct sk_buff *skb = ERR_PTR(-EINVAL); + u64 rcd; + + if ((*raw_rcd & cpu_to_le64(FBNIC_RCD_DONE)) == done) + break; + + dma_rmb(); + + rcd = le64_to_cpu(*raw_rcd); + + switch (FIELD_GET(FBNIC_RCD_TYPE_MASK, rcd)) { + case FBNIC_RCD_TYPE_HDR_AL: + head0 = FIELD_GET(FBNIC_RCD_AL_BUFF_ID_MASK, rcd); + fbnic_pkt_prepare(nv, rcd, pkt, qt); + + break; + case FBNIC_RCD_TYPE_PAY_AL: + head1 = FIELD_GET(FBNIC_RCD_AL_BUFF_ID_MASK, rcd); + fbnic_add_rx_frag(nv, rcd, pkt, qt); + + break; + case FBNIC_RCD_TYPE_OPT_META: + /* Only type 0 is currently supported */ + if (FIELD_GET(FBNIC_RCD_OPT_META_TYPE_MASK, rcd)) + break; + + /* We currently ignore the action table index */ + break; + case FBNIC_RCD_TYPE_META: + if (likely(!fbnic_rcd_metadata_err(rcd))) + skb = fbnic_build_skb(nv, pkt); + + /* populate skb and invalidate XDP */ + if (!IS_ERR_OR_NULL(skb)) { + fbnic_populate_skb_fields(nv, rcd, skb, qt); + + packets++; + + napi_gro_receive(&nv->napi, skb); + } + + pkt->buff.data_hard_start = NULL; + + break; + } + + raw_rcd++; + head++; + if (!(head & rcq->size_mask)) { + done ^= cpu_to_le64(FBNIC_RCD_DONE); + raw_rcd = &rcq->desc[0]; + } + } + + /* Unmap and free processed buffers */ + if (head0 >= 0) + fbnic_clean_bdq(nv, budget, &qt->sub0, head0); + fbnic_fill_bdq(nv, &qt->sub0); + + if (head1 >= 0) + fbnic_clean_bdq(nv, budget, &qt->sub1, head1); + fbnic_fill_bdq(nv, &qt->sub1); + + /* Record the current head/tail of the queue */ + if (rcq->head != head) { + rcq->head = head; + writel(head & rcq->size_mask, rcq->doorbell); + } + + return packets; +} static void fbnic_nv_irq_disable(struct fbnic_napi_vector *nv) { @@ -545,12 +825,19 @@ static int fbnic_poll(struct napi_struct *napi, int budget) struct fbnic_napi_vector *nv = container_of(napi, struct fbnic_napi_vector, napi); - int i; + int i, j, work_done = 0; for (i = 0; i < nv->txt_count; i++) fbnic_clean_tcq(nv, &nv->qt[i], budget); - if (likely(napi_complete_done(napi, 0))) + if (budget) + for (j = 0; j < nv->rxt_count; j++, i++) + work_done += fbnic_clean_rcq(nv, &nv->qt[i], budget); + + if (work_done >= budget) + return budget; + + if (likely(napi_complete_done(napi, work_done))) fbnic_nv_irq_rearm(nv); return 0; @@ -1555,3 +1842,32 @@ void fbnic_napi_enable(struct fbnic_net *fbn) } wrfl(); } + +void fbnic_napi_depletion_check(struct net_device *netdev) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + u32 irqs[FBNIC_MAX_MSIX_VECS / 32] = {}; + struct fbnic_dev *fbd = fbn->fbd; + struct fbnic_napi_vector *nv; + int i, j; + + list_for_each_entry(nv, &fbn->napis, napis) { + /* Find RQs which are completely out of pages */ + for (i = nv->txt_count, j = 0; j < nv->rxt_count; j++, i++) { + /* Assume 4 pages is always enough to fit a packet + * and therefore generate a completion and an IRQ. + */ + if (fbnic_desc_used(&nv->qt[i].sub0) < 4 || + fbnic_desc_used(&nv->qt[i].sub1) < 4) + irqs[nv->v_idx / 32] |= BIT(nv->v_idx % 32); + } + } + + for (i = 0; i < ARRAY_SIZE(irqs); i++) { + if (!irqs[i]) + continue; + wr32(FBNIC_INTR_MASK_CLEAR(i), irqs[i]); + wr32(FBNIC_INTR_SET(i), irqs[i]); + } + wrfl(); +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h index 0c424c49866d..4e43d41c781a 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.h @@ -5,6 +5,7 @@ #define _FBNIC_TXRX_H_ #include +#include #include #include @@ -118,6 +119,7 @@ void fbnic_config_drop_mode(struct fbnic_net *fbn); void fbnic_flush(struct fbnic_net *fbn); void fbnic_fill(struct fbnic_net *fbn); +void fbnic_napi_depletion_check(struct net_device *netdev); int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail); #endif /* _FBNIC_TXRX_H_ */ From patchwork Wed Apr 3 20:09:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616641 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4A711553B5 for ; Wed, 3 Apr 2024 20:09:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174970; cv=none; b=aTnMP/Ua3YFeZWb87W9x+hp+v98J+cCjj2O+YmpIb9Jd1eyvVyfRw5iSv8WrhyKDkmmaezMeuOb3UaPK0mNij+Hu8Iy6rp0my9SbVUX8WvtlMoF1B2M+q0Rzl7jug8kKKm4NLtv9XTJanIB8OswFk6h04PzI7KJLLcDDsB6nAEY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174970; c=relaxed/simple; bh=RFnDliZKIiHfgZ9M/whv8+Lh4lkKrpzESfLjeNVTeeY=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UpY6JEU4MRX3HRRqm/pLqN8t/XvlREk8dJjp0ZDfKdAxBZHA26Cc2weC1TFkCvj4i9dYmEDvTsZC+GT4hr3QNVhl+HfmbZS9jU7yPhjmxO+muQ2qV6P5CWRlw+GfwYkahqUMWTTSGhURW3DHxeWEIimrMP2Cr5Jl2YFwxYea/P8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=F/L/eLYO; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="F/L/eLYO" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1e2987e9d67so1327745ad.1 for ; Wed, 03 Apr 2024 13:09:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174967; x=1712779767; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=qKT0uQ13D1PsUgXpXjL00Tzi0pSn3GNggNZzWWZe9rE=; b=F/L/eLYODQEFuGMMyrJG95E0ZxHTutL0N1x6sQezOMd7luXMDTUcwfX/X/F4qiQ32Q FTSwmNeQRICxIuE7oc51KH+t+qk/EEn2393XXBtKbNMlEtfQGibl8gwEmKIk2zc/tqWQ QkHTJsOzvwL4R8n7cqBVmFFChcuBTr8YAwFw55AcX6/UXcLz2lHHJuBKFE/YtZ3wyenA sBzWfht2G/XRRkWrwl+YGIdiIK8rI0ceyPOwor4R/3Od4g7YLVKOngO12/K4dCErT0Zd 5lrWxyr9AVfBs1QfkjGPEQpFSudlO/e7KZMFEZyCsZ62pjHMVDLNH7srAWwvy11OQuzW l/oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174967; x=1712779767; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=qKT0uQ13D1PsUgXpXjL00Tzi0pSn3GNggNZzWWZe9rE=; b=BNJZUznqfQE99689KM4NkCmBoZFi9rK9RjIHSvpsAY6lY9TB5/SWaJYct4y/jFdcXN fgqEO3mFTOcz+P6nRTDlDsgDUNGgkanKcqL4CnhMwD5Bb9FFKZRRuzVilYabo8ge5G3/ HUYyf0yo6HJDOLb3/a8XRyrF/UOT/OmAms4D/YaETVmHea3AgKq5qu1YXI5XmGDVd6Pk TjVoqiJ/4bP02+8Kxec1Vs8575ApmH382cfVTQ2dxGBp9Hm+HxnzyS3AWXhYha6vAxU2 zxsjgMzpxg8r+dtOU4FIQ3F4rfQWlBvCO3s7fIh8b2bMzoNkaKfiLNIlNLWcew64YkAO vYEw== X-Gm-Message-State: AOJu0YxJspSZgIZkzTlnFQqohOKDyPMQiC21Mwgm6iTEbnE0XhkzIPrF IDjovO+Jlcm1hhovKqZu4m+gWq03kcNNDSNCHjcagYmuyy8mtba9 X-Google-Smtp-Source: AGHT+IHfbp5aBYQhq3mVN+aAxQ6QWAsui5DDXyeY6t056/n6seorB0kAnnl0mFqyFhSyZByesG3sTA== X-Received: by 2002:a17:902:d512:b0:1dd:1c6f:af51 with SMTP id b18-20020a170902d51200b001dd1c6faf51mr994079plg.16.1712174966849; Wed, 03 Apr 2024 13:09:26 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([2605:59c8:43f:400:9e5c:8eff:fe4f:f2d0]) by smtp.gmail.com with ESMTPSA id l9-20020a170903120900b001e02875930asm13700387plh.25.2024.04.03.13.09.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:09:25 -0700 (PDT) Subject: [net-next PATCH 14/15] eth: fbnic: add L2 address programming From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:09:24 -0700 Message-ID: <171217496413.1598374.9426860801410281670.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck Program the Rx TCAM to control L2 forwarding. Since we are in full control of the NIC we need to make sure we include BMC forwarding in the rules. When host is not present BMC will program the TCAM to get onto the network but once we take ownership it's up to Linux driver to make sure BMC L2 addresses are handled correctly. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/Makefile | 1 drivers/net/ethernet/meta/fbnic/fbnic.h | 10 + drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 14 + drivers/net/ethernet/meta/fbnic/fbnic_devlink.c | 2 drivers/net/ethernet/meta/fbnic/fbnic_netdev.c | 231 ++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_netdev.h | 3 drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 3 drivers/net/ethernet/meta/fbnic/fbnic_rpc.c | 338 +++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_rpc.h | 139 +++++++++ 9 files changed, 741 insertions(+) create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_rpc.c create mode 100644 drivers/net/ethernet/meta/fbnic/fbnic_rpc.h diff --git a/drivers/net/ethernet/meta/fbnic/Makefile b/drivers/net/ethernet/meta/fbnic/Makefile index f2ea90e0c14f..5210844ebe63 100644 --- a/drivers/net/ethernet/meta/fbnic/Makefile +++ b/drivers/net/ethernet/meta/fbnic/Makefile @@ -13,5 +13,6 @@ fbnic-y := fbnic_devlink.o \ fbnic_mac.o \ fbnic_netdev.o \ fbnic_pci.o \ + fbnic_rpc.o \ fbnic_tlv.o \ fbnic_txrx.o diff --git a/drivers/net/ethernet/meta/fbnic/fbnic.h b/drivers/net/ethernet/meta/fbnic/fbnic.h index 202f005e1cfd..0a62dc129d7e 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic.h @@ -10,6 +10,7 @@ #include "fbnic_csr.h" #include "fbnic_fw.h" #include "fbnic_mac.h" +#include "fbnic_rpc.h" struct fbnic_dev { struct device *dev; @@ -38,6 +39,10 @@ struct fbnic_dev { u32 mps; u32 readrq; + /* Local copy of the devices TCAM */ + struct fbnic_mac_addr mac_addr[FBNIC_RPC_TCAM_MACDA_NUM_ENTRIES]; + u8 mac_addr_boundary; + /* Tri-state value indicating state of link. * 0 - Up * 1 - Down @@ -103,6 +108,11 @@ static inline bool fbnic_bmc_present(struct fbnic_dev *fbd) return fbd->fw_cap.bmc_present; } +static inline void fbnic_bmc_set_present(struct fbnic_dev *fbd, bool present) +{ + fbd->fw_cap.bmc_present = present; +} + static inline bool fbnic_init_failure(struct fbnic_dev *fbd) { return !fbd->netdev; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index f61b401fdd5c..613b50bf829c 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -508,8 +508,22 @@ enum { #define FBNIC_RPC_RMI_CONFIG_FCS_PRESENT CSR_BIT(8) #define FBNIC_RPC_RMI_CONFIG_ENABLE CSR_BIT(12) #define FBNIC_RPC_RMI_CONFIG_MTU CSR_GENMASK(31, 16) +#define FBNIC_RPC_TCAM_MACDA_VALIDATE 0x0852d /* 0x214b4 */ #define FBNIC_CSR_END_RPC 0x0856b /* CSR section delimiter */ +/* RPC RAM Registers */ + +#define FBNIC_CSR_START_RPC_RAM 0x08800 /* CSR section delimiter */ +#define FBNIC_RPC_ACT_TBL_NUM_ENTRIES 64 + +/* TCAM Tables */ +#define FBNIC_RPC_TCAM_VALIDATE CSR_BIT(31) +#define FBNIC_RPC_TCAM_MACDA(m, n) \ + (0x08b80 + ((n) * 0x20) + (m)) /* 0x022e00 + 0x80*n + 4*m */ +#define FBNIC_RPC_TCAM_MACDA_VALUE CSR_GENMASK(15, 0) +#define FBNIC_RPC_TCAM_MACDA_MASK CSR_GENMASK(31, 16) +#define FBNIC_CSR_END_RPC_RAM 0x08f1f /* CSR section delimiter */ + /* Fab Registers */ #define FBNIC_CSR_START_FAB 0x0C000 /* CSR section delimiter */ #define FBNIC_FAB_AXI4_AR_SPACER_2_CFG 0x0C005 /* 0x30014 */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c b/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c index 91e8135410df..b007e7bddf81 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c @@ -66,6 +66,8 @@ struct fbnic_dev *fbnic_devlink_alloc(struct pci_dev *pdev) fbd->dsn = pci_get_dsn(pdev); + fbd->mac_addr_boundary = FBNIC_RPC_TCAM_MACDA_DEFAULT_BOUNDARY; + return fbd; } diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c index 792bdfa7429d..349560821435 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c @@ -7,6 +7,7 @@ #include "fbnic.h" #include "fbnic_netdev.h" +#include "fbnic_rpc.h" #include "fbnic_txrx.h" int __fbnic_open(struct fbnic_net *fbn) @@ -48,6 +49,8 @@ int __fbnic_open(struct fbnic_net *fbn) err = fbnic_mac_enable(fbd); if (err) goto release_ownership; + /* Pull the BMC config and initialize the RPC */ + fbnic_bmc_rpc_init(fbd); return 0; release_ownership: @@ -86,12 +89,240 @@ static int fbnic_stop(struct net_device *netdev) return 0; } +static int fbnic_uc_sync(struct net_device *netdev, const unsigned char *addr) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + struct fbnic_mac_addr *avail_addr; + + if (WARN_ON(!is_valid_ether_addr(addr))) + return -EADDRNOTAVAIL; + + avail_addr = __fbnic_uc_sync(fbn->fbd, addr); + if (!avail_addr) + return -ENOSPC; + + /* Add type flag indicating this address is in use by the host */ + set_bit(FBNIC_MAC_ADDR_T_UNICAST, avail_addr->act_tcam); + + return 0; +} + +static int fbnic_uc_unsync(struct net_device *netdev, const unsigned char *addr) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + struct fbnic_dev *fbd = fbn->fbd; + int i, ret; + + /* Scan from middle of list to bottom, filling bottom up. + * Skip the first entry which is reserved for dev_addr and + * leave the last entry to use for promiscuous filtering. + */ + for (i = fbd->mac_addr_boundary, ret = -ENOENT; + i < FBNIC_RPC_TCAM_MACDA_HOST_ADDR_IDX && ret; i++) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[i]; + + if (!ether_addr_equal(mac_addr->value.addr8, addr)) + continue; + + ret = __fbnic_uc_unsync(mac_addr); + } + + return ret; +} + +static int fbnic_mc_sync(struct net_device *netdev, const unsigned char *addr) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + struct fbnic_mac_addr *avail_addr; + + if (WARN_ON(!is_multicast_ether_addr(addr))) + return -EADDRNOTAVAIL; + + avail_addr = __fbnic_mc_sync(fbn->fbd, addr); + if (!avail_addr) + return -ENOSPC; + + /* Add type flag indicating this address is in use by the host */ + set_bit(FBNIC_MAC_ADDR_T_MULTICAST, avail_addr->act_tcam); + + return 0; +} + +static int fbnic_mc_unsync(struct net_device *netdev, const unsigned char *addr) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + struct fbnic_dev *fbd = fbn->fbd; + int i, ret; + + /* Scan from middle of list to top, filling top down. + * Skip over the address reserved for the BMC MAC and + * exclude index 0 as that belongs to the broadcast address + */ + for (i = fbd->mac_addr_boundary, ret = -ENOENT; + --i > FBNIC_RPC_TCAM_MACDA_BROADCAST_IDX && ret;) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[i]; + + if (!ether_addr_equal(mac_addr->value.addr8, addr)) + continue; + + ret = __fbnic_mc_unsync(mac_addr); + } + + return ret; +} + +void __fbnic_set_rx_mode(struct net_device *netdev) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + bool uc_promisc = false, mc_promisc = false; + struct fbnic_dev *fbd = fbn->fbd; + struct fbnic_mac_addr *mac_addr; + int err; + + /* Populate host address from dev_addr */ + mac_addr = &fbd->mac_addr[FBNIC_RPC_TCAM_MACDA_HOST_ADDR_IDX]; + if (!ether_addr_equal(mac_addr->value.addr8, netdev->dev_addr) || + mac_addr->state != FBNIC_TCAM_S_VALID) { + ether_addr_copy(mac_addr->value.addr8, netdev->dev_addr); + mac_addr->state = FBNIC_TCAM_S_UPDATE; + set_bit(FBNIC_MAC_ADDR_T_UNICAST, mac_addr->act_tcam); + } + + /* Populate broadcast address if broadcast is enabled */ + mac_addr = &fbd->mac_addr[FBNIC_RPC_TCAM_MACDA_BROADCAST_IDX]; + if (netdev->flags & IFF_BROADCAST) { + if (!is_broadcast_ether_addr(mac_addr->value.addr8) || + mac_addr->state != FBNIC_TCAM_S_VALID) { + eth_broadcast_addr(mac_addr->value.addr8); + mac_addr->state = FBNIC_TCAM_S_ADD; + } + set_bit(FBNIC_MAC_ADDR_T_BROADCAST, mac_addr->act_tcam); + } else if (mac_addr->state == FBNIC_TCAM_S_VALID) { + __fbnic_xc_unsync(mac_addr, FBNIC_MAC_ADDR_T_BROADCAST); + } + + /* synchronize unicast and multicast address lists */ + err = __dev_uc_sync(netdev, fbnic_uc_sync, fbnic_uc_unsync); + if (err == -ENOSPC) + uc_promisc = true; + err = __dev_mc_sync(netdev, fbnic_mc_sync, fbnic_mc_unsync); + if (err == -ENOSPC) + mc_promisc = true; + + uc_promisc |= !!(netdev->flags & IFF_PROMISC); + mc_promisc |= !!(netdev->flags & IFF_ALLMULTI) || uc_promisc; + + /* Populate last TCAM entry with promiscuous entry and 0/1 bit mask */ + mac_addr = &fbd->mac_addr[FBNIC_RPC_TCAM_MACDA_PROMISC_IDX]; + if (uc_promisc) { + if (!is_zero_ether_addr(mac_addr->value.addr8) || + mac_addr->state != FBNIC_TCAM_S_VALID) { + eth_zero_addr(mac_addr->value.addr8); + eth_broadcast_addr(mac_addr->mask.addr8); + clear_bit(FBNIC_MAC_ADDR_T_ALLMULTI, + mac_addr->act_tcam); + set_bit(FBNIC_MAC_ADDR_T_PROMISC, + mac_addr->act_tcam); + mac_addr->state = FBNIC_TCAM_S_ADD; + } + } else if (mc_promisc) { + /* We have to add a special handler for multicast as the + * BMC may have an all-multi rule already in place. As such + * adding a rule ourselves won't do any good so we will have + * to modify the rules for the ALL MULTI below if the BMC + * already has the rule in place. + */ + if (!fbd->fw_cap.all_multi && + (!is_multicast_ether_addr(mac_addr->value.addr8) || + mac_addr->state != FBNIC_TCAM_S_VALID)) { + eth_zero_addr(mac_addr->value.addr8); + eth_broadcast_addr(mac_addr->mask.addr8); + mac_addr->value.addr8[0] ^= 1; + mac_addr->mask.addr8[0] ^= 1; + set_bit(FBNIC_MAC_ADDR_T_ALLMULTI, + mac_addr->act_tcam); + clear_bit(FBNIC_MAC_ADDR_T_PROMISC, + mac_addr->act_tcam); + mac_addr->state = FBNIC_TCAM_S_ADD; + } + } else if (mac_addr->state == FBNIC_TCAM_S_VALID) { + if (test_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam)) { + clear_bit(FBNIC_MAC_ADDR_T_ALLMULTI, + mac_addr->act_tcam); + clear_bit(FBNIC_MAC_ADDR_T_PROMISC, + mac_addr->act_tcam); + } else { + mac_addr->state = FBNIC_TCAM_S_DELETE; + } + } + + /* Add rules for BMC all multicast if it is enabled */ + fbnic_bmc_rpc_all_multi_config(fbd, mc_promisc); + + /* sift out any unshared BMC rules and place them in BMC only section */ + fbnic_sift_macda(fbd); + + /* Write updates to hardware */ + fbnic_write_macda(fbd); +} + +static void fbnic_set_rx_mode(struct net_device *netdev) +{ + /* no need to update the hardware if we are not running */ + if (netif_running(netdev)) + __fbnic_set_rx_mode(netdev); +} + +static int fbnic_set_mac(struct net_device *netdev, void *p) +{ + struct sockaddr *addr = p; + + if (!is_valid_ether_addr(addr->sa_data)) + return -EADDRNOTAVAIL; + + eth_hw_addr_set(netdev, addr->sa_data); + + fbnic_set_rx_mode(netdev); + + return 0; +} + +void fbnic_clear_rx_mode(struct net_device *netdev) +{ + struct fbnic_net *fbn = netdev_priv(netdev); + struct fbnic_dev *fbd = fbn->fbd; + int idx; + + for (idx = ARRAY_SIZE(fbd->mac_addr); idx--;) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[idx]; + + if (mac_addr->state != FBNIC_TCAM_S_VALID) + continue; + + bitmap_clear(mac_addr->act_tcam, + FBNIC_MAC_ADDR_T_HOST_START, + FBNIC_MAC_ADDR_T_HOST_LEN); + + if (bitmap_empty(mac_addr->act_tcam, + FBNIC_RPC_TCAM_ACT_NUM_ENTRIES)) + mac_addr->state = FBNIC_TCAM_S_DELETE; + } + + /* Write updates to hardware */ + fbnic_write_macda(fbd); + + __dev_uc_unsync(netdev, NULL); + __dev_mc_unsync(netdev, NULL); +} + static const struct net_device_ops fbnic_netdev_ops = { .ndo_open = fbnic_open, .ndo_stop = fbnic_stop, .ndo_validate_addr = eth_validate_addr, .ndo_start_xmit = fbnic_xmit_frame, .ndo_features_check = fbnic_features_check, + .ndo_set_mac_address = fbnic_set_mac, + .ndo_set_rx_mode = fbnic_set_rx_mode, }; void fbnic_reset_queues(struct fbnic_net *fbn, diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h index 3976fb1a0eac..40e155cf1865 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h @@ -46,4 +46,7 @@ void fbnic_netdev_unregister(struct net_device *netdev); void fbnic_reset_queues(struct fbnic_net *fbn, unsigned int tx, unsigned int rx); +void __fbnic_set_rx_mode(struct net_device *netdev); +void fbnic_clear_rx_mode(struct net_device *netdev); + #endif /* _FBNIC_NETDEV_H_ */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index d897b0d65abf..fbd2c15c9a99 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -133,6 +133,8 @@ void fbnic_up(struct fbnic_net *fbn) fbnic_fill(fbn); + __fbnic_set_rx_mode(fbn->netdev); + /* Enable Tx/Rx processing */ fbnic_napi_enable(fbn); netif_tx_start_all_queues(fbn->netdev); @@ -148,6 +150,7 @@ static void fbnic_down_noidle(struct fbnic_net *fbn) fbnic_napi_disable(fbn); netif_tx_disable(fbn->netdev); + fbnic_clear_rx_mode(fbn->netdev); fbnic_disable(fbn); } diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c new file mode 100644 index 000000000000..ac8814778919 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c @@ -0,0 +1,338 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#include + +#include "fbnic.h" +#include "fbnic_netdev.h" +#include "fbnic_rpc.h" + +static int fbnic_read_macda_entry(struct fbnic_dev *fbd, unsigned int idx, + struct fbnic_mac_addr *mac_addr) +{ + __be16 *mask, *value; + int i; + + mask = &mac_addr->mask.addr16[FBNIC_RPC_TCAM_MACDA_WORD_LEN - 1]; + value = &mac_addr->value.addr16[FBNIC_RPC_TCAM_MACDA_WORD_LEN - 1]; + + for (i = 0; i < FBNIC_RPC_TCAM_MACDA_WORD_LEN; i++) { + u32 macda = rd32(FBNIC_RPC_TCAM_MACDA(idx, i)); + + *mask-- = htons(FIELD_GET(FBNIC_RPC_TCAM_MACDA_MASK, macda)); + *value-- = htons(FIELD_GET(FBNIC_RPC_TCAM_MACDA_VALUE, macda)); + } + + return (rd32(FBNIC_RPC_TCAM_MACDA(idx, i)) & + FBNIC_RPC_TCAM_VALIDATE) ? 0 : -EINVAL; +} + +void fbnic_bmc_rpc_all_multi_config(struct fbnic_dev *fbd, + bool enable_host) +{ + struct fbnic_mac_addr *mac_addr; + + /* We need to add the all multicast filter at the end of the + * multicast address list. This way if there are any that are + * shared between the host and the BMC they can be directed to + * both. Otherwise the remainder just get sent directly to the + * BMC. + */ + mac_addr = &fbd->mac_addr[fbd->mac_addr_boundary - 1]; + if (fbnic_bmc_present(fbd) && fbd->fw_cap.all_multi) { + if (mac_addr->state != FBNIC_TCAM_S_VALID) { + eth_zero_addr(mac_addr->value.addr8); + eth_broadcast_addr(mac_addr->mask.addr8); + mac_addr->value.addr8[0] ^= 1; + mac_addr->mask.addr8[0] ^= 1; + set_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam); + mac_addr->state = FBNIC_TCAM_S_ADD; + } + if (enable_host) + set_bit(FBNIC_MAC_ADDR_T_ALLMULTI, + mac_addr->act_tcam); + else + clear_bit(FBNIC_MAC_ADDR_T_ALLMULTI, + mac_addr->act_tcam); + } else if (!test_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam) && + !is_zero_ether_addr(mac_addr->mask.addr8) && + mac_addr->state == FBNIC_TCAM_S_VALID) { + clear_bit(FBNIC_MAC_ADDR_T_ALLMULTI, mac_addr->act_tcam); + clear_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam); + mac_addr->state = FBNIC_TCAM_S_DELETE; + } +} + +void fbnic_bmc_rpc_init(struct fbnic_dev *fbd) +{ + int i = FBNIC_RPC_TCAM_MACDA_BMC_ADDR_IDX; + struct fbnic_mac_addr *mac_addr; + u32 macda_validate; + + /* Verify that RPC is already enabled, if not abort */ + macda_validate = rd32(FBNIC_RPC_TCAM_MACDA_VALIDATE); + if (!(macda_validate & (1u << i))) + return; + + /* Read BMC MACDA entry and validate it */ + mac_addr = &fbd->mac_addr[i]; + if (fbnic_read_macda_entry(fbd, i, mac_addr)) + return; + + /* If BMC MAC addr is valid then record it and flag it as valid */ + if (!is_valid_ether_addr(mac_addr->value.addr8)) + return; + + set_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam); + mac_addr->state = FBNIC_TCAM_S_VALID; + + /* Record the BMC Multicast addresses */ + for (i++; i < FBNIC_RPC_TCAM_MACDA_BROADCAST_IDX; i++) { + if (!(macda_validate & (1u << i))) + continue; + + mac_addr = &fbd->mac_addr[i]; + if (fbnic_read_macda_entry(fbd, i, mac_addr)) + continue; + + if (is_broadcast_ether_addr(mac_addr->value.addr8)) { + mac_addr->state = FBNIC_TCAM_S_DELETE; + continue; + } + + if (!is_multicast_ether_addr(mac_addr->value.addr8)) + continue; + + set_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam); + mac_addr->state = FBNIC_TCAM_S_VALID; + } + + /* Validate Broadcast is also present, record it and tag it */ + if (macda_validate & (1u << i)) { + mac_addr = &fbd->mac_addr[i]; + eth_broadcast_addr(mac_addr->value.addr8); + set_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam); + mac_addr->state = FBNIC_TCAM_S_ADD; + } + + /* Record the shared BMC Multicast addresses */ + for (i++; i <= FBNIC_RPC_TCAM_MACDA_PROMISC_IDX; i++) { + if (!(macda_validate & (1u << i))) + continue; + + mac_addr = &fbd->mac_addr[i]; + if (fbnic_read_macda_entry(fbd, i, mac_addr)) + continue; + + if (!is_multicast_ether_addr(mac_addr->value.addr8)) + continue; + + /* it isn't an exact match filter it must be an all-multi. */ + if (!is_zero_ether_addr(mac_addr->mask.addr8)) { + fbd->fw_cap.all_multi = 1; + + /* If it isn't in the correct spot don't record it */ + if (i != fbd->mac_addr_boundary - 1) + continue; + } + + set_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam); + mac_addr->state = FBNIC_TCAM_S_VALID; + } + + fbnic_bmc_rpc_all_multi_config(fbd, false); + + fbnic_bmc_set_present(fbd, true); +} + +struct fbnic_mac_addr *__fbnic_uc_sync(struct fbnic_dev *fbd, + const unsigned char *addr) +{ + struct fbnic_mac_addr *avail_addr = NULL; + unsigned int i; + + /* Scan from middle of list to bottom, filling bottom up. + * Skip the first entry which is reserved for dev_addr and + * leave the last entry to use for promiscuous filtering. + */ + for (i = fbd->mac_addr_boundary - 1; + i < FBNIC_RPC_TCAM_MACDA_HOST_ADDR_IDX; i++) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[i]; + + if (mac_addr->state == FBNIC_TCAM_S_DISABLED) { + avail_addr = mac_addr; + } else if (ether_addr_equal(mac_addr->value.addr8, addr)) { + avail_addr = mac_addr; + break; + } + } + + if (avail_addr && avail_addr->state == FBNIC_TCAM_S_DISABLED) { + ether_addr_copy(avail_addr->value.addr8, addr); + eth_zero_addr(avail_addr->mask.addr8); + avail_addr->state = FBNIC_TCAM_S_ADD; + } + + return avail_addr; +} + +struct fbnic_mac_addr *__fbnic_mc_sync(struct fbnic_dev *fbd, + const unsigned char *addr) +{ + struct fbnic_mac_addr *avail_addr = NULL; + unsigned int i; + + /* Scan from middle of list to top, filling top down. + * Skip over the address reserved for the BMC MAC and + * exclude index 0 as that belongs to the broadcast address + */ + for (i = fbd->mac_addr_boundary; + --i > FBNIC_RPC_TCAM_MACDA_BROADCAST_IDX;) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[i]; + + if (mac_addr->state == FBNIC_TCAM_S_DISABLED) { + avail_addr = mac_addr; + } else if (ether_addr_equal(mac_addr->value.addr8, addr)) { + avail_addr = mac_addr; + break; + } + } + + /* Scan the BMC addresses to see if it may have already + * reserved the address. + */ + while (--i) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[i]; + + if (!is_zero_ether_addr(mac_addr->mask.addr8)) + continue; + + /* Only move on if we find a match */ + if (!ether_addr_equal(mac_addr->value.addr8, addr)) + continue; + + /* We need to pull this address to the shared area */ + if (avail_addr) { + memcpy(avail_addr, mac_addr, sizeof(*mac_addr)); + mac_addr->state = FBNIC_TCAM_S_DELETE; + avail_addr->state = FBNIC_TCAM_S_ADD; + } + + break; + } + + if (avail_addr && avail_addr->state == FBNIC_TCAM_S_DISABLED) { + ether_addr_copy(avail_addr->value.addr8, addr); + eth_zero_addr(avail_addr->mask.addr8); + avail_addr->state = FBNIC_TCAM_S_ADD; + } + + return avail_addr; +} + +int __fbnic_xc_unsync(struct fbnic_mac_addr *mac_addr, unsigned int tcam_idx) +{ + if (!test_and_clear_bit(tcam_idx, mac_addr->act_tcam)) + return -ENOENT; + + if (bitmap_empty(mac_addr->act_tcam, FBNIC_RPC_TCAM_ACT_NUM_ENTRIES)) + mac_addr->state = FBNIC_TCAM_S_DELETE; + + return 0; +} + +void fbnic_sift_macda(struct fbnic_dev *fbd) +{ + int dest, src; + + /* move BMC only addresses back into BMC region */ + for (dest = FBNIC_RPC_TCAM_MACDA_BMC_ADDR_IDX, + src = FBNIC_RPC_TCAM_MACDA_MULTICAST_IDX; + ++dest < FBNIC_RPC_TCAM_MACDA_BROADCAST_IDX && + src < fbd->mac_addr_boundary;) { + struct fbnic_mac_addr *dest_addr = &fbd->mac_addr[dest]; + + if (dest_addr->state != FBNIC_TCAM_S_DISABLED) + continue; + + while (src < fbd->mac_addr_boundary) { + struct fbnic_mac_addr *src_addr = &fbd->mac_addr[src++]; + + /* Verify BMC bit is set */ + if (!test_bit(FBNIC_MAC_ADDR_T_BMC, src_addr->act_tcam)) + continue; + + /* Verify filter isn't already disabled */ + if (src_addr->state == FBNIC_TCAM_S_DISABLED || + src_addr->state == FBNIC_TCAM_S_DELETE) + continue; + + /* Verify only BMC bit is set */ + if (bitmap_weight(src_addr->act_tcam, + FBNIC_RPC_TCAM_ACT_NUM_ENTRIES) != 1) + continue; + + /* Verify we are not moving wildcard address */ + if (!is_zero_ether_addr(src_addr->mask.addr8)) + continue; + + memcpy(dest_addr, src_addr, sizeof(*src_addr)); + src_addr->state = FBNIC_TCAM_S_DELETE; + dest_addr->state = FBNIC_TCAM_S_ADD; + } + } +} + +static void fbnic_clear_macda_entry(struct fbnic_dev *fbd, unsigned int idx) +{ + int i; + + /* invalidate entry and clear addr state info */ + for (i = 0; i <= FBNIC_RPC_TCAM_MACDA_WORD_LEN; i++) + wr32(FBNIC_RPC_TCAM_MACDA(idx, i), 0); +} + +static void fbnic_write_macda_entry(struct fbnic_dev *fbd, unsigned int idx, + struct fbnic_mac_addr *mac_addr) +{ + __be16 *mask, *value; + int i; + + mask = &mac_addr->mask.addr16[FBNIC_RPC_TCAM_MACDA_WORD_LEN - 1]; + value = &mac_addr->value.addr16[FBNIC_RPC_TCAM_MACDA_WORD_LEN - 1]; + + for (i = 0; i < FBNIC_RPC_TCAM_MACDA_WORD_LEN; i++) + wr32(FBNIC_RPC_TCAM_MACDA(idx, i), + FIELD_PREP(FBNIC_RPC_TCAM_MACDA_MASK, ntohs(*mask--)) | + FIELD_PREP(FBNIC_RPC_TCAM_MACDA_VALUE, ntohs(*value--))); + + wrfl(); + + wr32(FBNIC_RPC_TCAM_MACDA(idx, i), FBNIC_RPC_TCAM_VALIDATE); +} + +void fbnic_write_macda(struct fbnic_dev *fbd) +{ + int idx; + + for (idx = ARRAY_SIZE(fbd->mac_addr); idx--;) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[idx]; + + /* Check if update flag is set else exit. */ + if (!(mac_addr->state & FBNIC_TCAM_S_UPDATE)) + continue; + + /* Clear by writing 0s. */ + if (mac_addr->state == FBNIC_TCAM_S_DELETE) { + /* invalidate entry and clear addr state info */ + fbnic_clear_macda_entry(fbd, idx); + memset(mac_addr, 0, sizeof(*mac_addr)); + + continue; + } + + fbnic_write_macda_entry(fbd, idx, mac_addr); + + mac_addr->state = FBNIC_TCAM_S_VALID; + } +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_rpc.h b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.h new file mode 100644 index 000000000000..1b59b10ba677 --- /dev/null +++ b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.h @@ -0,0 +1,139 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) Meta Platforms, Inc. and affiliates. */ + +#ifndef _FBNIC_RPC_H_ +#define _FBNIC_RPC_H_ + +#include +#include + +/* The TCAM state definitions follow an expected ordering. + * They start out disabled, then move through the following states: + * Disabled 0 -> Add 2 + * Add 2 -> Valid 1 + * + * Valid 1 -> Add/Update 2 + * Add 2 -> Valid 1 + * + * Valid 1 -> Delete 3 + * Delete 3 -> Disabled 0 + */ +enum { + FBNIC_TCAM_S_DISABLED = 0, + FBNIC_TCAM_S_VALID = 1, + FBNIC_TCAM_S_ADD = 2, + FBNIC_TCAM_S_UPDATE = FBNIC_TCAM_S_ADD, + FBNIC_TCAM_S_DELETE = 3, +}; + +/* 32 MAC Destination Address TCAM Entries + * 4 registers DA[1:0], DA[3:2], DA[5:4], Validate + */ +#define FBNIC_RPC_TCAM_MACDA_WORD_LEN 3 +#define FBNIC_RPC_TCAM_MACDA_NUM_ENTRIES 32 + +#define FBNIC_RPC_TCAM_ACT_WORD_LEN 11 +#define FBNIC_RPC_TCAM_ACT_NUM_ENTRIES 64 + +struct fbnic_mac_addr { + union { + unsigned char addr8[ETH_ALEN]; + __be16 addr16[FBNIC_RPC_TCAM_MACDA_WORD_LEN]; + } mask, value; + unsigned char state; + DECLARE_BITMAP(act_tcam, FBNIC_RPC_TCAM_ACT_NUM_ENTRIES); +}; + +struct fbnic_act_tcam { + struct { + u16 tcam[FBNIC_RPC_TCAM_ACT_WORD_LEN]; + } mask, value; + unsigned char state; + u16 rss_en_mask; + u32 dest; +}; + +enum { + FBNIC_RSS_EN_HOST_ETHER, + FBNIC_RSS_EN_XCAST_ETHER, +#define FBNIC_RSS_EN_NUM_UNICAST FBNIC_RSS_EN_XCAST_ETHER + FBNIC_RSS_EN_NUM_ENTRIES +}; + +/* Reserve the first 2 entries for the use by the BMC so that we can + * avoid allowing rules to get in the way of BMC unicast traffic. + */ +#define FBNIC_RPC_ACT_TBL_BMC_OFFSET 0 +#define FBNIC_RPC_ACT_TBL_BMC_ALL_MULTI_OFFSET 1 + +/* We reserve the last 14 entries for RSS rules on the host. The BMC + * unicast rule will need to be populated above these and is expected to + * use MACDA TCAM entry 23 to store the BMC MAC address. + */ +#define FBNIC_RPC_ACT_TBL_RSS_OFFSET \ + (FBNIC_RPC_ACT_TBL_NUM_ENTRIES - FBNIC_RSS_EN_NUM_ENTRIES) + +/* Flags used to identify the owner for this MAC filter. Note that any + * flags set for Broadcast thru Promisc indicate that the rule belongs + * to the RSS filters for the host. + */ +enum { + FBNIC_MAC_ADDR_T_BMC = 0, + FBNIC_MAC_ADDR_T_BROADCAST = FBNIC_RPC_ACT_TBL_RSS_OFFSET, +#define FBNIC_MAC_ADDR_T_HOST_START FBNIC_MAC_ADDR_T_BROADCAST + FBNIC_MAC_ADDR_T_MULTICAST, + FBNIC_MAC_ADDR_T_UNICAST, + FBNIC_MAC_ADDR_T_ALLMULTI, /* BROADCAST ... MULTICAST*/ + FBNIC_MAC_ADDR_T_PROMISC, /* BROADCAST ... UNICAST */ + FBNIC_MAC_ADDR_T_HOST_LAST +}; + +#define FBNIC_MAC_ADDR_T_HOST_LEN \ + (FBNIC_MAC_ADDR_T_HOST_LAST - FBNIC_MAC_ADDR_T_HOST_START) + +#define FBNIC_RPC_TCAM_ACT1_L2_MACDA_IDX CSR_GENMASK(9, 5) +#define FBNIC_RPC_TCAM_ACT1_L2_MACDA_VALID CSR_BIT(10) + +/* TCAM 0 - 3 reserved for BMC MAC addresses */ +#define FBNIC_RPC_TCAM_MACDA_BMC_ADDR_IDX 0 +/* TCAM 4 reserved for broadcast MAC address */ +#define FBNIC_RPC_TCAM_MACDA_BROADCAST_IDX 4 +/* TCAMs 5 - 30 will be used for multicast and unicast addresses. The + * boundary between the two can be variable it is currently set to 24 + * on which the unicast addresses start. The general idea is that we will + * always go top-down with unicast, and bottom-up with multicast so that + * there should be free-space in the middle between the two. + * + * The entry at MADCA_DEFAULT_BOUNDARY is a special case as it can be used + * for the ALL MULTI address if the list is full, or the BMC has requested + * it. + */ +#define FBNIC_RPC_TCAM_MACDA_MULTICAST_IDX 5 +#define FBNIC_RPC_TCAM_MACDA_DEFAULT_BOUNDARY 24 +#define FBNIC_RPC_TCAM_MACDA_HOST_ADDR_IDX 30 +/* Reserved for use to record Multicast promisc, or Promiscuous */ +#define FBNIC_RPC_TCAM_MACDA_PROMISC_IDX 31 + +struct fbnic_dev; + +void fbnic_bmc_rpc_init(struct fbnic_dev *fbd); +void fbnic_bmc_rpc_all_multi_config(struct fbnic_dev *fbd, bool enable_host); + +int __fbnic_xc_unsync(struct fbnic_mac_addr *mac_addr, unsigned int tcam_idx); +struct fbnic_mac_addr *__fbnic_uc_sync(struct fbnic_dev *fbd, + const unsigned char *addr); +struct fbnic_mac_addr *__fbnic_mc_sync(struct fbnic_dev *fbd, + const unsigned char *addr); +void fbnic_sift_macda(struct fbnic_dev *fbd); +void fbnic_write_macda(struct fbnic_dev *fbd); + +static inline int __fbnic_uc_unsync(struct fbnic_mac_addr *mac_addr) +{ + return __fbnic_xc_unsync(mac_addr, FBNIC_MAC_ADDR_T_UNICAST); +} + +static inline int __fbnic_mc_unsync(struct fbnic_mac_addr *mac_addr) +{ + return __fbnic_xc_unsync(mac_addr, FBNIC_MAC_ADDR_T_MULTICAST); +} +#endif /* _FBNIC_RPC_H_ */ From patchwork Wed Apr 3 20:09:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 13616642 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14D0B155729 for ; Wed, 3 Apr 2024 20:09:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174974; cv=none; b=c/RgaKL22Fw2nZxO1dlxM/UcWwH8b+z22rpehpsW3mB/gFHD49HQqZLDwi4/LJpE45UStC5jV6kKlXGvv5mO9JgFrpS7shw/BXGzatm+ROxNzdPOHacWCx5LvHDCSzGFPpQCGndbwTUeniCAB7jkGL/vGqDad678B7Q1NZOVA+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712174974; c=relaxed/simple; bh=fyLitH6FshR5Kt9JiJSpsQDjwK5ITq7vqX3gkp8j4XE=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=CLF6TjUI1cBtlzwDysF7jRNR+JOwUrBnoCzB3YdtdR7lLHiGoBuxchElRH8OQoVSmtv9VepKWogJIZTMQ1HMVo3pNTrKTKFFk839Y67rKt1HJFuMl6cfQ3u3tMCLbvVmx/rOkk+NX1zD7uY65UyBp3kHdl4lUehD0In53y9Xy68= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RbCli3/D; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RbCli3/D" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1e28856ed7aso1809535ad.0 for ; Wed, 03 Apr 2024 13:09:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712174971; x=1712779771; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:from:to:cc:subject :date:message-id:reply-to; bh=dNo+Gd0AOEUD63EbBoveGFVgpZLA2mxSSIcOuEuqhKE=; b=RbCli3/D4wfUqYiW0DBr+3NgtjycmvUu7XxC/NvmkE0wwf7bTWDUbxPePSIVC/sh6s 0WIUNY+78wsKJyAw9V/5YDjRrkS+XDqWxEYqcZwvTjPyrafxx2W+7d6ONnkXJCWkU9wz P1e/l4GArB7Nxneh++ssgdx+UPjH3qW7ubNGBLJAfMH/Z4yklqrce+1FT0ZL7wtp8VTZ vS3Uhak3elknyPDNtlzx1nntvTZEP2e/B8vpY4kGHJGk5gGdbVTdZDNkko1kVILY95IT DYKnhM0820F5HHWvzv2dAUMeNtCLjUS92XRT0lQzFT9eKQCt4H6LDMrAA30/6tyVZrQW On1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712174971; x=1712779771; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=dNo+Gd0AOEUD63EbBoveGFVgpZLA2mxSSIcOuEuqhKE=; b=EhjD5vV41sk6ielA/IDSSAr3ymM4XiDihVkYVAdV/PZ78hVSAFDrY7yDxozXyQdLS2 m570YgTyo+47MLLrIOlHDa21EGXNa9Wr+nfuFzzHXDN45zS+mYBioAoKhHhttzvsm+Vl 2ncIbhNrFNF7JiEQwFJHo2TlYTLuQ3TXRmRsunyEEYDcTJi7lrWFTx3qkJDCe7x1FO4K DxBU0whBqWkVkv8hi3vNiyOeGy1iEVoMFM4S6vFkMVraIwuGkvOggxjufLhh9gMSS2sM JOyaEDZmK8XNwPBpt9tX7e/6LgvRctPwqxug7nhmtkf15NbpFRNRCUV8crtyK1g+A2uw 04HQ== X-Gm-Message-State: AOJu0YwoteDqYUzdcGjqzRNqQgkjCzMBbokeEGuQcMfjXqe8E0w2PPIh lo12WWGi0dDXLw7uLHdPJwJ4LQwALNBvCCyxtQANbopElIHuiDEl X-Google-Smtp-Source: AGHT+IF5w4U8StnGtYneys7WONsz+QRJGsz7+pXDTosNYMbalRzwqJfdoB0xSx6aoEMTgc+yWoR8JA== X-Received: by 2002:a17:903:41c9:b0:1e2:a2f3:2130 with SMTP id u9-20020a17090341c900b001e2a2f32130mr423966ple.12.1712174971477; Wed, 03 Apr 2024 13:09:31 -0700 (PDT) Received: from ahduyck-xeon-server.home.arpa ([2605:59c8:43f:400:9e5c:8eff:fe4f:f2d0]) by smtp.gmail.com with ESMTPSA id u14-20020a170903124e00b001e2086fddecsm13705401plh.139.2024.04.03.13.09.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 13:09:30 -0700 (PDT) Subject: [net-next PATCH 15/15] eth: fbnic: write the TCAM tables used for RSS control and Rx to host From: Alexander Duyck To: netdev@vger.kernel.org Cc: Alexander Duyck , kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com Date: Wed, 03 Apr 2024 13:09:29 -0700 Message-ID: <171217496908.1598374.735014163887838492.stgit@ahduyck-xeon-server.home.arpa> In-Reply-To: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> References: <171217454226.1598374.8971335637623132496.stgit@ahduyck-xeon-server.home.arpa> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Alexander Duyck RSS is controlled by the Rx filter tables. Program rules matching on appropriate traffic types and set hashing fields using actions. We need a separate set of rules for broadcast and multicast because the action there needs to include forwarding to BMC. This patch only initializes the default settings, the control of the configuration using ethtool will come soon. With this the necessary rules are put in place to enable Rx of packets by the host. Signed-off-by: Alexander Duyck --- drivers/net/ethernet/meta/fbnic/fbnic.h | 1 drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 59 ++++ drivers/net/ethernet/meta/fbnic/fbnic_netdev.c | 6 drivers/net/ethernet/meta/fbnic/fbnic_netdev.h | 7 drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 4 drivers/net/ethernet/meta/fbnic/fbnic_rpc.c | 371 ++++++++++++++++++++++++ drivers/net/ethernet/meta/fbnic/fbnic_rpc.h | 52 +++ 7 files changed, 499 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/meta/fbnic/fbnic.h b/drivers/net/ethernet/meta/fbnic/fbnic.h index 0a62dc129d7e..5186d097cb8b 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic.h @@ -40,6 +40,7 @@ struct fbnic_dev { u32 readrq; /* Local copy of the devices TCAM */ + struct fbnic_act_tcam act_tcam[FBNIC_RPC_TCAM_ACT_NUM_ENTRIES]; struct fbnic_mac_addr mac_addr[FBNIC_RPC_TCAM_MACDA_NUM_ENTRIES]; u8 mac_addr_boundary; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h index 613b50bf829c..4fb572421202 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h @@ -508,20 +508,79 @@ enum { #define FBNIC_RPC_RMI_CONFIG_FCS_PRESENT CSR_BIT(8) #define FBNIC_RPC_RMI_CONFIG_ENABLE CSR_BIT(12) #define FBNIC_RPC_RMI_CONFIG_MTU CSR_GENMASK(31, 16) + +#define FBNIC_RPC_ACT_TBL0_DEFAULT 0x0840a /* 0x21028 */ +#define FBNIC_RPC_ACT_TBL0_DROP CSR_BIT(0) +#define FBNIC_RPC_ACT_TBL0_DEST_MASK CSR_GENMASK(3, 1) +enum { + FBNIC_RPC_ACT_TBL0_DEST_HOST = 1, + FBNIC_RPC_ACT_TBL0_DEST_BMC = 2, + FBNIC_RPC_ACT_TBL0_DEST_EI = 4, +}; + +#define FBNIC_RPC_ACT_TBL0_DMA_HINT CSR_GENMASK(24, 16) +#define FBNIC_RPC_ACT_TBL0_RSS_CTXT_ID CSR_BIT(30) + +#define FBNIC_RPC_ACT_TBL1_DEFAULT 0x0840b /* 0x2102c */ +#define FBNIC_RPC_ACT_TBL1_RSS_ENA_MASK CSR_GENMASK(15, 0) +enum { + FBNIC_RPC_ACT_TBL1_RSS_ENA_IP_SRC = 1, + FBNIC_RPC_ACT_TBL1_RSS_ENA_IP_DST = 2, + FBNIC_RPC_ACT_TBL1_RSS_ENA_L4_SRC = 4, + FBNIC_RPC_ACT_TBL1_RSS_ENA_L4_DST = 8, + FBNIC_RPC_ACT_TBL1_RSS_ENA_L2_DA = 16, + FBNIC_RPC_ACT_TBL1_RSS_ENA_L4_RSS_BYTE = 32, + FBNIC_RPC_ACT_TBL1_RSS_ENA_IV6_FL_LBL = 64, + FBNIC_RPC_ACT_TBL1_RSS_ENA_OV6_FL_LBL = 128, + FBNIC_RPC_ACT_TBL1_RSS_ENA_DSCP = 256, + FBNIC_RPC_ACT_TBL1_RSS_ENA_L3_PROT = 512, + FBNIC_RPC_ACT_TBL1_RSS_ENA_L4_PROT = 1024, +}; + +#define FBNIC_RPC_RSS_KEY(n) (0x0840c + (n)) /* 0x21030 + 4*n */ +#define FBNIC_RPC_RSS_KEY_BIT_LEN 425 +#define FBNIC_RPC_RSS_KEY_BYTE_LEN \ + DIV_ROUND_UP(FBNIC_RPC_RSS_KEY_BIT_LEN, 8) +#define FBNIC_RPC_RSS_KEY_DWORD_LEN \ + DIV_ROUND_UP(FBNIC_RPC_RSS_KEY_BIT_LEN, 32) +#define FBNIC_RPC_RSS_KEY_LAST_IDX \ + (FBNIC_RPC_RSS_KEY_DWORD_LEN - 1) +#define FBNIC_RPC_RSS_KEY_LAST_MASK \ + CSR_GENMASK(31, \ + FBNIC_RPC_RSS_KEY_DWORD_LEN * 32 - \ + FBNIC_RPC_RSS_KEY_BIT_LEN) + #define FBNIC_RPC_TCAM_MACDA_VALIDATE 0x0852d /* 0x214b4 */ #define FBNIC_CSR_END_RPC 0x0856b /* CSR section delimiter */ /* RPC RAM Registers */ #define FBNIC_CSR_START_RPC_RAM 0x08800 /* CSR section delimiter */ +#define FBNIC_RPC_ACT_TBL0(n) (0x08800 + (n)) /* 0x22000 + 4*n */ +#define FBNIC_RPC_ACT_TBL1(n) (0x08840 + (n)) /* 0x22100 + 4*n */ #define FBNIC_RPC_ACT_TBL_NUM_ENTRIES 64 /* TCAM Tables */ #define FBNIC_RPC_TCAM_VALIDATE CSR_BIT(31) + +/* 64 Action TCAM Entries, 12 registers + * 3 mixed, src port, dst port, 6 L4 words, and Validate + */ +#define FBNIC_RPC_TCAM_ACT(m, n) \ + (0x08880 + ((n) * 0x40) + (m)) /* 0x22200 + 0x100*n + 4*m*/ + +#define FBNIC_RPC_TCAM_ACT_VALUE CSR_GENMASK(15, 0) +#define FBNIC_RPC_TCAM_ACT_MASK CSR_GENMASK(31, 16) + #define FBNIC_RPC_TCAM_MACDA(m, n) \ (0x08b80 + ((n) * 0x20) + (m)) /* 0x022e00 + 0x80*n + 4*m */ #define FBNIC_RPC_TCAM_MACDA_VALUE CSR_GENMASK(15, 0) #define FBNIC_RPC_TCAM_MACDA_MASK CSR_GENMASK(31, 16) + +#define FBNIC_RPC_RSS_TBL(n, m) \ + (0x08d20 + ((n) * 0x100) + (m)) /* 0x023480 + 0x400*n + 4*m */ +#define FBNIC_RPC_RSS_TBL_COUNT 2 +#define FBNIC_RPC_RSS_TBL_SIZE 256 #define FBNIC_CSR_END_RPC_RAM 0x08f1f /* CSR section delimiter */ /* Fab Registers */ diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c index 349560821435..2844f0bfb9c4 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.c @@ -51,6 +51,7 @@ int __fbnic_open(struct fbnic_net *fbn) goto release_ownership; /* Pull the BMC config and initialize the RPC */ fbnic_bmc_rpc_init(fbd); + fbnic_rss_reinit(fbd, fbn); return 0; release_ownership: @@ -263,6 +264,7 @@ void __fbnic_set_rx_mode(struct net_device *netdev) fbnic_sift_macda(fbd); /* Write updates to hardware */ + fbnic_write_rules(fbd); fbnic_write_macda(fbd); } @@ -380,6 +382,10 @@ struct net_device *fbnic_netdev_alloc(struct fbnic_dev *fbd) fbnic_reset_queues(fbn, default_queues, default_queues); + fbnic_reset_indir_tbl(fbn); + fbnic_rss_key_fill(fbn->rss_key); + fbnic_rss_init_en_mask(fbn); + netdev->features |= NETIF_F_RXHASH | NETIF_F_SG | diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h index 40e155cf1865..6cfe820e4bba 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_netdev.h @@ -6,6 +6,8 @@ #include +#include "fbnic_csr.h" +#include "fbnic_rpc.h" #include "fbnic_txrx.h" struct fbnic_net { @@ -31,7 +33,12 @@ struct fbnic_net { u16 num_tx_queues; u16 num_rx_queues; + u8 indir_tbl[FBNIC_RPC_RSS_TBL_COUNT][FBNIC_RPC_RSS_TBL_SIZE]; + u32 rss_key[FBNIC_RPC_RSS_KEY_DWORD_LEN]; + u32 rss_flow_hash[FBNIC_NUM_HASH_OPT]; + u64 link_down_events; + struct list_head napis; }; diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c index fbd2c15c9a99..82f1afd7ae4b 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c @@ -133,6 +133,8 @@ void fbnic_up(struct fbnic_net *fbn) fbnic_fill(fbn); + fbnic_rss_reinit_hw(fbn->fbd, fbn); + __fbnic_set_rx_mode(fbn->netdev); /* Enable Tx/Rx processing */ @@ -151,6 +153,8 @@ static void fbnic_down_noidle(struct fbnic_net *fbn) netif_tx_disable(fbn->netdev); fbnic_clear_rx_mode(fbn->netdev); + fbnic_clear_rules(fbn->fbd); + fbnic_rss_disable_hw(fbn->fbd); fbnic_disable(fbn); } diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c index ac8814778919..09b06c4a8153 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c +++ b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c @@ -2,11 +2,102 @@ /* Copyright (c) Meta Platforms, Inc. and affiliates. */ #include +#include #include "fbnic.h" #include "fbnic_netdev.h" #include "fbnic_rpc.h" +void fbnic_reset_indir_tbl(struct fbnic_net *fbn) +{ + unsigned int num_rx = fbn->num_rx_queues; + unsigned int i; + + for (i = 0; i < FBNIC_RPC_RSS_TBL_SIZE; i++) { + fbn->indir_tbl[0][i] = ethtool_rxfh_indir_default(i, num_rx); + fbn->indir_tbl[1][i] = ethtool_rxfh_indir_default(i, num_rx); + } +} + +void fbnic_rss_key_fill(u32 *buffer) +{ + static u32 rss_key[FBNIC_RPC_RSS_KEY_DWORD_LEN]; + + net_get_random_once(rss_key, sizeof(rss_key)); + rss_key[FBNIC_RPC_RSS_KEY_LAST_IDX] &= FBNIC_RPC_RSS_KEY_LAST_MASK; + + memcpy(buffer, rss_key, sizeof(rss_key)); +} + +#define RX_HASH_OPT_L4 \ + (RXH_IP_SRC | RXH_IP_DST | RXH_L4_B_0_1 | RXH_L4_B_2_3) +#define RX_HASH_OPT_L3 \ + (RXH_IP_SRC | RXH_IP_DST) +#define RX_HASH_OPT_L2 RXH_L2DA + +void fbnic_rss_init_en_mask(struct fbnic_net *fbn) +{ + fbn->rss_flow_hash[FBNIC_TCP4_HASH_OPT] = RX_HASH_OPT_L4; + fbn->rss_flow_hash[FBNIC_TCP6_HASH_OPT] = RX_HASH_OPT_L4; + + fbn->rss_flow_hash[FBNIC_UDP4_HASH_OPT] = RX_HASH_OPT_L3; + fbn->rss_flow_hash[FBNIC_UDP6_HASH_OPT] = RX_HASH_OPT_L3; + fbn->rss_flow_hash[FBNIC_IPV4_HASH_OPT] = RX_HASH_OPT_L3; + fbn->rss_flow_hash[FBNIC_IPV6_HASH_OPT] = RX_HASH_OPT_L3; + + fbn->rss_flow_hash[FBNIC_ETHER_HASH_OPT] = RX_HASH_OPT_L2; +} + +void fbnic_rss_disable_hw(struct fbnic_dev *fbd) +{ + /* Disable RPC by clearing enable bit and configuration */ + if (!fbnic_bmc_present(fbd)) + wr32(FBNIC_RPC_RMI_CONFIG, + FIELD_PREP(FBNIC_RPC_RMI_CONFIG_OH_BYTES, 20)); +} + +#define FBNIC_FH_2_RSSEM_BIT(_fh, _rssem, _val) \ + FIELD_PREP(FBNIC_RPC_ACT_TBL1_RSS_ENA_##_rssem, \ + FIELD_GET(RXH_##_fh, _val)) +static u16 fbnic_flow_hash_2_rss_en_mask(struct fbnic_net *fbn, int flow_type) +{ + u32 flow_hash = fbn->rss_flow_hash[flow_type]; + u32 rss_en_mask = 0; + + rss_en_mask |= FBNIC_FH_2_RSSEM_BIT(L2DA, L2_DA, flow_hash); + rss_en_mask |= FBNIC_FH_2_RSSEM_BIT(IP_SRC, IP_SRC, flow_hash); + rss_en_mask |= FBNIC_FH_2_RSSEM_BIT(IP_DST, IP_DST, flow_hash); + rss_en_mask |= FBNIC_FH_2_RSSEM_BIT(L4_B_0_1, L4_SRC, flow_hash); + rss_en_mask |= FBNIC_FH_2_RSSEM_BIT(L4_B_2_3, L4_DST, flow_hash); + + return rss_en_mask; +} + +void fbnic_rss_reinit_hw(struct fbnic_dev *fbd, struct fbnic_net *fbn) +{ + unsigned int i; + + for (i = 0; i < FBNIC_RPC_RSS_TBL_SIZE; i++) { + fbnic_wr32(fbd, FBNIC_RPC_RSS_TBL(0, i), fbn->indir_tbl[0][i]); + fbnic_wr32(fbd, FBNIC_RPC_RSS_TBL(1, i), fbn->indir_tbl[1][i]); + } + + for (i = 0; i < FBNIC_RPC_RSS_KEY_DWORD_LEN; i++) + wr32(FBNIC_RPC_RSS_KEY(i), fbn->rss_key[i]); + + /* Default action for this to drop w/ no destination */ + wr32(FBNIC_RPC_ACT_TBL0_DEFAULT, FBNIC_RPC_ACT_TBL0_DROP); + wrfl(); + + wr32(FBNIC_RPC_ACT_TBL1_DEFAULT, 0); + + /* If it isn't already enabled set the RMI Config value to enable RPC */ + wr32(FBNIC_RPC_RMI_CONFIG, + FIELD_PREP(FBNIC_RPC_RMI_CONFIG_MTU, FBNIC_MAX_JUMBO_FRAME_SIZE) | + FIELD_PREP(FBNIC_RPC_RMI_CONFIG_OH_BYTES, 20) | + FBNIC_RPC_RMI_CONFIG_ENABLE); +} + static int fbnic_read_macda_entry(struct fbnic_dev *fbd, unsigned int idx, struct fbnic_mac_addr *mac_addr) { @@ -30,7 +121,9 @@ static int fbnic_read_macda_entry(struct fbnic_dev *fbd, unsigned int idx, void fbnic_bmc_rpc_all_multi_config(struct fbnic_dev *fbd, bool enable_host) { + struct fbnic_act_tcam *act_tcam; struct fbnic_mac_addr *mac_addr; + int j; /* We need to add the all multicast filter at the end of the * multicast address list. This way if there are any that are @@ -61,13 +154,51 @@ void fbnic_bmc_rpc_all_multi_config(struct fbnic_dev *fbd, clear_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam); mac_addr->state = FBNIC_TCAM_S_DELETE; } + + /* We have to add a special handler for multicast as the + * BMC may have an all-multi rule already in place. As such + * adding a rule ourselves won't do any good so we will have + * to modify the rules for the ALL MULTI below if the BMC + * already has the rule in place. + */ + act_tcam = &fbd->act_tcam[FBNIC_RPC_ACT_TBL_BMC_ALL_MULTI_OFFSET]; + + /* If we are not enabling the rule just delete it. We will fall + * back to the RSS rules that support the multicast addresses. + */ + if (!fbnic_bmc_present(fbd) || !fbd->fw_cap.all_multi || enable_host) { + if (act_tcam->state == FBNIC_TCAM_S_VALID) + act_tcam->state = FBNIC_TCAM_S_DELETE; + return; + } + + /* Rewrite TCAM rule 23 to handle BMC all-multi traffic */ + act_tcam->dest = FIELD_PREP(FBNIC_RPC_ACT_TBL0_DEST_MASK, + FBNIC_RPC_ACT_TBL0_DEST_BMC); + act_tcam->mask.tcam[0] = 0xffff; + + /* MACDA 0 - 3 is reserved for the BMC MAC address */ + act_tcam->value.tcam[1] = + FIELD_PREP(FBNIC_RPC_TCAM_ACT1_L2_MACDA_IDX, + fbd->mac_addr_boundary - 1) | + FBNIC_RPC_TCAM_ACT1_L2_MACDA_VALID; + act_tcam->mask.tcam[1] = 0xffff & + ~FBNIC_RPC_TCAM_ACT1_L2_MACDA_IDX & + ~FBNIC_RPC_TCAM_ACT1_L2_MACDA_VALID; + + for (j = 2; j < FBNIC_RPC_TCAM_ACT_WORD_LEN; j++) + act_tcam->mask.tcam[j] = 0xffff; + + act_tcam->state = FBNIC_TCAM_S_UPDATE; } void fbnic_bmc_rpc_init(struct fbnic_dev *fbd) { int i = FBNIC_RPC_TCAM_MACDA_BMC_ADDR_IDX; + struct fbnic_act_tcam *act_tcam; struct fbnic_mac_addr *mac_addr; u32 macda_validate; + int j; /* Verify that RPC is already enabled, if not abort */ macda_validate = rd32(FBNIC_RPC_TCAM_MACDA_VALIDATE); @@ -140,11 +271,116 @@ void fbnic_bmc_rpc_init(struct fbnic_dev *fbd) mac_addr->state = FBNIC_TCAM_S_VALID; } + /* Rewrite TCAM rule 0 if it isn't present to relocate BMC rules */ + act_tcam = &fbd->act_tcam[FBNIC_RPC_ACT_TBL_BMC_OFFSET]; + act_tcam->dest = FIELD_PREP(FBNIC_RPC_ACT_TBL0_DEST_MASK, + FBNIC_RPC_ACT_TBL0_DEST_BMC); + act_tcam->mask.tcam[0] = 0xffff; + + /* MACDA 0 - 3 is reserved for the BMC MAC address + * to account for that we have to mask out the lower 2 bits + * of the macda by performing an &= with 0x1c. + */ + act_tcam->value.tcam[1] = FBNIC_RPC_TCAM_ACT1_L2_MACDA_VALID; + act_tcam->mask.tcam[1] = 0xffff & + ~FIELD_PREP(FBNIC_RPC_TCAM_ACT1_L2_MACDA_IDX, 0x1c) & + ~FBNIC_RPC_TCAM_ACT1_L2_MACDA_VALID; + + for (j = 2; j < FBNIC_RPC_TCAM_ACT_WORD_LEN; j++) + act_tcam->mask.tcam[j] = 0xffff; + + act_tcam->state = FBNIC_TCAM_S_UPDATE; + fbnic_bmc_rpc_all_multi_config(fbd, false); fbnic_bmc_set_present(fbd, true); } +#define FBNIC_ACT1_INIT(_l4, _udp, _ip, _v6) \ + (((_l4) ? FBNIC_RPC_TCAM_ACT1_L4_VALID : 0) | \ + ((_udp) ? FBNIC_RPC_TCAM_ACT1_L4_IS_UDP : 0) | \ + ((_ip) ? FBNIC_RPC_TCAM_ACT1_IP_VALID : 0) | \ + ((_v6) ? FBNIC_RPC_TCAM_ACT1_IP_IS_V6 : 0)) + +void fbnic_rss_reinit(struct fbnic_dev *fbd, struct fbnic_net *fbn) +{ + static const u32 act1_value[FBNIC_NUM_HASH_OPT] = { + FBNIC_ACT1_INIT(1, 1, 1, 1), /* UDP6 */ + FBNIC_ACT1_INIT(1, 1, 1, 0), /* UDP4 */ + FBNIC_ACT1_INIT(1, 0, 1, 1), /* TCP6 */ + FBNIC_ACT1_INIT(1, 0, 1, 0), /* TCP4 */ + FBNIC_ACT1_INIT(0, 0, 1, 1), /* IP6 */ + FBNIC_ACT1_INIT(0, 0, 1, 0), /* IP4 */ + 0 /* Ether */ + }; + unsigned int i; + + /* To support scenarios where a BMC is present we must write the + * rules twice, once for the unicast cases, and once again for + * the broadcast/multicast cases as we have to support 2 destinations. + */ + BUILD_BUG_ON(FBNIC_RSS_EN_NUM_UNICAST * 2 != FBNIC_RSS_EN_NUM_ENTRIES); + BUILD_BUG_ON(ARRAY_SIZE(act1_value) != FBNIC_NUM_HASH_OPT); + + /* Program RSS hash enable mask for host in action TCAM/table. */ + for (i = fbnic_bmc_present(fbd) ? 0 : FBNIC_RSS_EN_NUM_UNICAST; + i < FBNIC_RSS_EN_NUM_ENTRIES; i++) { + unsigned int idx = i + FBNIC_RPC_ACT_TBL_RSS_OFFSET; + struct fbnic_act_tcam *act_tcam = &fbd->act_tcam[idx]; + u32 flow_hash, dest, rss_en_mask; + int flow_type, j; + u16 value = 0; + + flow_type = i % FBNIC_RSS_EN_NUM_UNICAST; + flow_hash = fbn->rss_flow_hash[flow_type]; + + /* set DEST_HOST based on absence of RXH_DISCARD */ + dest = FIELD_PREP(FBNIC_RPC_ACT_TBL0_DEST_MASK, + !(RXH_DISCARD & flow_hash) ? + FBNIC_RPC_ACT_TBL0_DEST_HOST : 0); + + if (i >= FBNIC_RSS_EN_NUM_UNICAST && fbnic_bmc_present(fbd)) + dest |= FIELD_PREP(FBNIC_RPC_ACT_TBL0_DEST_MASK, + FBNIC_RPC_ACT_TBL0_DEST_BMC); + + if (!dest) + dest = FBNIC_RPC_ACT_TBL0_DROP; + + if (act1_value[flow_type] & FBNIC_RPC_TCAM_ACT1_L4_VALID) + dest |= FIELD_PREP(FBNIC_RPC_ACT_TBL0_DMA_HINT, + FBNIC_RCD_HDR_AL_DMA_HINT_L4); + + rss_en_mask = fbnic_flow_hash_2_rss_en_mask(fbn, flow_type); + + act_tcam->dest = dest; + act_tcam->rss_en_mask = rss_en_mask; + act_tcam->state = FBNIC_TCAM_S_UPDATE; + + act_tcam->mask.tcam[0] = 0xffff; + + /* We reserve the upper 8 MACDA TCAM entries for host + * unicast. So we set the value to 24, and the mask the + * lower bits so that the lower entries can be used as + * multicast or BMC addresses. + */ + if (i < FBNIC_RSS_EN_NUM_UNICAST) + value = FIELD_PREP(FBNIC_RPC_TCAM_ACT1_L2_MACDA_IDX, + fbd->mac_addr_boundary); + value |= FBNIC_RPC_TCAM_ACT1_L2_MACDA_VALID; + + flow_type = i % FBNIC_RSS_EN_NUM_UNICAST; + value |= act1_value[flow_type]; + + act_tcam->value.tcam[1] = value; + act_tcam->mask.tcam[1] = ~value; + + for (j = 2; j < FBNIC_RPC_TCAM_ACT_WORD_LEN; j++) + act_tcam->mask.tcam[j] = 0xffff; + + act_tcam->state = FBNIC_TCAM_S_UPDATE; + } +} + struct fbnic_mac_addr *__fbnic_uc_sync(struct fbnic_dev *fbd, const unsigned char *addr) { @@ -292,6 +528,38 @@ static void fbnic_clear_macda_entry(struct fbnic_dev *fbd, unsigned int idx) wr32(FBNIC_RPC_TCAM_MACDA(idx, i), 0); } +static void fbnic_clear_macda(struct fbnic_dev *fbd) +{ + int idx; + + for (idx = ARRAY_SIZE(fbd->mac_addr); idx--;) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[idx]; + + if (mac_addr->state == FBNIC_TCAM_S_DISABLED) + continue; + + if (test_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam)) { + if (fbnic_bmc_present(fbd)) + continue; + dev_warn_once(fbd->dev, + "Found BMC MAC address w/ BMC not present\n"); + } + + fbnic_clear_macda_entry(fbd, idx); + + /* If rule was already destined for deletion just wipe it now */ + if (mac_addr->state == FBNIC_TCAM_S_DELETE) { + memset(mac_addr, 0, sizeof(*mac_addr)); + continue; + } + + /* Change state to update so that we will rewrite + * this tcam the next time fbnic_write_macda is called. + */ + mac_addr->state = FBNIC_TCAM_S_UPDATE; + } +} + static void fbnic_write_macda_entry(struct fbnic_dev *fbd, unsigned int idx, struct fbnic_mac_addr *mac_addr) { @@ -336,3 +604,106 @@ void fbnic_write_macda(struct fbnic_dev *fbd) mac_addr->state = FBNIC_TCAM_S_VALID; } } + +static void fbnic_clear_act_tcam(struct fbnic_dev *fbd, unsigned int idx) +{ + int i; + + /* invalidate entry and clear addr state info */ + for (i = 0; i <= FBNIC_RPC_TCAM_ACT_WORD_LEN; i++) + wr32(FBNIC_RPC_TCAM_ACT(idx, i), 0); +} + +void fbnic_clear_rules(struct fbnic_dev *fbd) +{ + u32 dest = FIELD_PREP(FBNIC_RPC_ACT_TBL0_DEST_MASK, + FBNIC_RPC_ACT_TBL0_DEST_BMC); + int i = FBNIC_RPC_TCAM_ACT_NUM_ENTRIES - 1; + struct fbnic_act_tcam *act_tcam; + + /* Clear MAC rules */ + fbnic_clear_macda(fbd); + + /* If BMC is present we need to preserve the last rule which + * will be used to route traffic to the BMC if it is received. + * + * At this point it should be the only MAC address in the MACDA + * so any unicast or multicast traffic received should be routed + * to it. So leave the last rule in place. + * + * It will be rewritten to add the host again when we bring + * the interface back up. + */ + if (fbnic_bmc_present(fbd)) { + act_tcam = &fbd->act_tcam[i]; + + if (act_tcam->state == FBNIC_TCAM_S_VALID && + (act_tcam->dest & dest)) { + wr32(FBNIC_RPC_ACT_TBL0(i), dest); + wr32(FBNIC_RPC_ACT_TBL1(i), 0); + + act_tcam->state = FBNIC_TCAM_S_UPDATE; + + i--; + } + } + + /* Work from the bottom up deleting all other rules from hardware */ + do { + act_tcam = &fbd->act_tcam[i]; + + if (act_tcam->state != FBNIC_TCAM_S_VALID) + continue; + + fbnic_clear_act_tcam(fbd, i); + act_tcam->state = FBNIC_TCAM_S_UPDATE; + } while (i--); +} + +static void fbnic_delete_act_tcam(struct fbnic_dev *fbd, unsigned int idx) +{ + fbnic_clear_act_tcam(fbd, idx); + memset(&fbd->act_tcam[idx], 0, sizeof(struct fbnic_act_tcam)); +} + +static void fbnic_update_act_tcam(struct fbnic_dev *fbd, unsigned int idx) +{ + struct fbnic_act_tcam *act_tcam = &fbd->act_tcam[idx]; + int i; + + /* Update entry by writing the destination and RSS mask */ + wr32(FBNIC_RPC_ACT_TBL0(idx), act_tcam->dest); + wr32(FBNIC_RPC_ACT_TBL1(idx), act_tcam->rss_en_mask); + + /* Write new TCAM rule to hardware */ + for (i = 0; i < FBNIC_RPC_TCAM_ACT_WORD_LEN; i++) + wr32(FBNIC_RPC_TCAM_ACT(idx, i), + FIELD_PREP(FBNIC_RPC_TCAM_ACT_MASK, + act_tcam->mask.tcam[i]) | + FIELD_PREP(FBNIC_RPC_TCAM_ACT_VALUE, + act_tcam->value.tcam[i])); + + wrfl(); + + wr32(FBNIC_RPC_TCAM_ACT(idx, i), FBNIC_RPC_TCAM_VALIDATE); + act_tcam->state = FBNIC_TCAM_S_VALID; +} + +void fbnic_write_rules(struct fbnic_dev *fbd) +{ + int i; + + /* Flush any pending action table rules */ + for (i = 0; i < FBNIC_RPC_ACT_TBL_NUM_ENTRIES; i++) { + struct fbnic_act_tcam *act_tcam = &fbd->act_tcam[i]; + + /* Check if update flag is set else exit. */ + if (!(act_tcam->state & FBNIC_TCAM_S_UPDATE)) + continue; + + if (act_tcam->state == FBNIC_TCAM_S_DELETE) + fbnic_delete_act_tcam(fbd, i); + else + fbnic_update_act_tcam(fbd, i); + } +} diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_rpc.h b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.h index 1b59b10ba677..d62935f722a2 100644 --- a/drivers/net/ethernet/meta/fbnic/fbnic_rpc.h +++ b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.h @@ -54,9 +54,21 @@ struct fbnic_act_tcam { }; enum { + FBNIC_RSS_EN_HOST_UDP6, + FBNIC_RSS_EN_HOST_UDP4, + FBNIC_RSS_EN_HOST_TCP6, + FBNIC_RSS_EN_HOST_TCP4, + FBNIC_RSS_EN_HOST_IP6, + FBNIC_RSS_EN_HOST_IP4, FBNIC_RSS_EN_HOST_ETHER, + FBNIC_RSS_EN_XCAST_UDP6, +#define FBNIC_RSS_EN_NUM_UNICAST FBNIC_RSS_EN_XCAST_UDP6 + FBNIC_RSS_EN_XCAST_UDP4, + FBNIC_RSS_EN_XCAST_TCP6, + FBNIC_RSS_EN_XCAST_TCP4, + FBNIC_RSS_EN_XCAST_IP6, + FBNIC_RSS_EN_XCAST_IP4, FBNIC_RSS_EN_XCAST_ETHER, -#define FBNIC_RSS_EN_NUM_UNICAST FBNIC_RSS_EN_XCAST_ETHER FBNIC_RSS_EN_NUM_ENTRIES }; @@ -91,8 +103,22 @@ enum { #define FBNIC_MAC_ADDR_T_HOST_LEN \ (FBNIC_MAC_ADDR_T_HOST_LAST - FBNIC_MAC_ADDR_T_HOST_START) +#define FBNIC_RPC_TCAM_ACT0_IPSRC_IDX CSR_GENMASK(2, 0) +#define FBNIC_RPC_TCAM_ACT0_IPSRC_VALID CSR_BIT(3) +#define FBNIC_RPC_TCAM_ACT0_IPDST_IDX CSR_GENMASK(6, 4) +#define FBNIC_RPC_TCAM_ACT0_IPDST_VALID CSR_BIT(7) +#define FBNIC_RPC_TCAM_ACT0_OUTER_IPSRC_IDX CSR_GENMASK(10, 8) +#define FBNIC_RPC_TCAM_ACT0_OUTER_IPSRC_VALID CSR_BIT(11) +#define FBNIC_RPC_TCAM_ACT0_OUTER_IPDST_IDX CSR_GENMASK(14, 12) +#define FBNIC_RPC_TCAM_ACT0_OUTER_IPDST_VALID CSR_BIT(15) + #define FBNIC_RPC_TCAM_ACT1_L2_MACDA_IDX CSR_GENMASK(9, 5) #define FBNIC_RPC_TCAM_ACT1_L2_MACDA_VALID CSR_BIT(10) +#define FBNIC_RPC_TCAM_ACT1_IP_IS_V6 CSR_BIT(11) +#define FBNIC_RPC_TCAM_ACT1_IP_VALID CSR_BIT(12) +#define FBNIC_RPC_TCAM_ACT1_OUTER_IP_VALID CSR_BIT(13) +#define FBNIC_RPC_TCAM_ACT1_L4_IS_UDP CSR_BIT(14) +#define FBNIC_RPC_TCAM_ACT1_L4_VALID CSR_BIT(15) /* TCAM 0 - 3 reserved for BMC MAC addresses */ #define FBNIC_RPC_TCAM_MACDA_BMC_ADDR_IDX 0 @@ -114,11 +140,32 @@ enum { /* Reserved for use to record Multicast promisc, or Promiscuous */ #define FBNIC_RPC_TCAM_MACDA_PROMISC_IDX 31 +enum { + FBNIC_UDP6_HASH_OPT, + FBNIC_UDP4_HASH_OPT, + FBNIC_TCP6_HASH_OPT, + FBNIC_TCP4_HASH_OPT, +#define FBNIC_L4_HASH_OPT FBNIC_TCP4_HASH_OPT + FBNIC_IPV6_HASH_OPT, + FBNIC_IPV4_HASH_OPT, +#define FBNIC_IP_HASH_OPT FBNIC_IPV4_HASH_OPT + FBNIC_ETHER_HASH_OPT, + FBNIC_NUM_HASH_OPT, +}; + struct fbnic_dev; +struct fbnic_net; void fbnic_bmc_rpc_init(struct fbnic_dev *fbd); void fbnic_bmc_rpc_all_multi_config(struct fbnic_dev *fbd, bool enable_host); +void fbnic_reset_indir_tbl(struct fbnic_net *fbn); +void fbnic_rss_key_fill(u32 *buffer); +void fbnic_rss_init_en_mask(struct fbnic_net *fbn); +void fbnic_rss_disable_hw(struct fbnic_dev *fbd); +void fbnic_rss_reinit_hw(struct fbnic_dev *fbd, struct fbnic_net *fbn); +void fbnic_rss_reinit(struct fbnic_dev *fbd, struct fbnic_net *fbn); + int __fbnic_xc_unsync(struct fbnic_mac_addr *mac_addr, unsigned int tcam_idx); struct fbnic_mac_addr *__fbnic_uc_sync(struct fbnic_dev *fbd, const unsigned char *addr); @@ -136,4 +183,7 @@ static inline int __fbnic_mc_unsync(struct fbnic_mac_addr *mac_addr) { return __fbnic_xc_unsync(mac_addr, FBNIC_MAC_ADDR_T_MULTICAST); } + +void fbnic_clear_rules(struct fbnic_dev *fbd); +void fbnic_write_rules(struct fbnic_dev *fbd); #endif /* _FBNIC_RPC_H_ */