From patchwork Fri May 10 03:04:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660833 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B2BE2148844 for ; Fri, 10 May 2024 03:04:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310285; cv=none; b=o4ZuMpebKAWxf5hBSMclHQpQEYM+tSE6aBwCedqX2PiY5aaDbzIUAxnffqm9wfGkcpW0GrLyhu6+UleRdq41iL5m4lWQ55i60lVg9+IswArS/o1oNgDChOZf0h8fEAW23Mnj7XwYdTbIW48/iiEF9TRx4qp64GfWhhd3kJ0oT+w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310285; c=relaxed/simple; bh=LjPHzwCFyMwVfjqom5DoydYZbbWV7DUpKtZlFgS7J7E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Pa/AxtPK9HPgNVG6G+Wx2tMbHg/Rxp5y6OykaceEz+Sx3umcQo327/amMmBGqXKw67yO3HqUjFr4NJl2MR/6zUHHD0ezFIfutfwBwH2OSfevvR0Jjkzd0wE/pncXAazMiAfVpaILVGWJdB9ixuk2ufyrU76LuvXR+xSF04XfNVU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FNF/NkKC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FNF/NkKC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8793C32781; Fri, 10 May 2024 03:04:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310285; bh=LjPHzwCFyMwVfjqom5DoydYZbbWV7DUpKtZlFgS7J7E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FNF/NkKCNF9n51neduwFsiHN83TNGSfTRrPR9k2eCSjSK2TXifi3Y8ESaD9kNkWa0 FyiyldKyLnRcqOOxEId7S0QMJ2EbeRtHYfdJdSe1cls0+4SYUFERTFm+w0RQSKIQTd tVyvkE+1LNp63dEj0APLThf+9m3StZyLHpSUa9Z7f6Mwbp0hqN8xhWcYrwwKCfPyCu LcDmMbXUebD8XLxnPyqucT/r/RfWq83TB1CAMqrgjXfgeluPYLnSfOi1FicNMyJmIY YoYLf2nTm5Fy+zkoNNdBFRjfGaIhiuVOdloiSdZvFMAQrebJt8zmGqJLNKkCiz1cnO oTdsggprhfACQ== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Jakub Kicinski Subject: [RFC net-next 01/15] psp: add documentation Date: Thu, 9 May 2024 20:04:21 -0700 Message-ID: <20240510030435.120935-2-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Add documentation of things which belong in the docs rather than commit messages. Signed-off-by: Jakub Kicinski --- Documentation/networking/index.rst | 1 + Documentation/networking/psp.rst | 138 +++++++++++++++++++++++++++++ 2 files changed, 139 insertions(+) create mode 100644 Documentation/networking/psp.rst diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst index 7664c0bfe461..0376029ecbdf 100644 --- a/Documentation/networking/index.rst +++ b/Documentation/networking/index.rst @@ -94,6 +94,7 @@ Refer to :ref:`netdev-FAQ` for a guide on netdev development process specifics. ppp_generic proc_net_tcp pse-pd/index + psp radiotap-headers rds regulatory diff --git a/Documentation/networking/psp.rst b/Documentation/networking/psp.rst new file mode 100644 index 000000000000..a39b464813ab --- /dev/null +++ b/Documentation/networking/psp.rst @@ -0,0 +1,138 @@ +.. SPDX-License-Identifier: GPL-2.0-only + +===================== +PSP Security Protocol +===================== + +Protocol +======== + +PSP Security Protocol (PSP) was defined at Google and published in: + +https://raw.githubusercontent.com/google/psp/main/doc/PSP_Arch_Spec.pdf + +This section briefly covers protocol aspects crucial for understanding +the kernel API. Refer to the protocol specification for further details. + +Note that the kernel implementation and documentation uses the term +"secret state" in place of "master key", it is both less confusing +to an average developer and is less likely to run afoul any naming +guidelines. + +Derived Rx keys +--------------- + +PSP borrows some terms and mechanisms from IPsec. PSP was designed +with HW offloads in mind. The key feature of PSP is that Rx keys for every +connection do not have to be stored by the receiver but can be derived +from secret state and information present in packet headers. +This makes it possible to implement receivers which require a constant +amount of memory regardless of the number of connections (``O(1)`` scaling). + +Tx keys have to be stored like with any other protocol, but Tx is much +less latency sensitive than Rx, and delays in fetching keys from slow +memory is less likely to cause packet drops. + +Key rotation +------------ + +The secret state known only to the receiver is fundamental to the design. +Per specification this state cannot be directly accessible (it must be +impossible to read it out of the hardware of the receiver NIC). +Moreover, it has to be "rotated" periodically (usually daily). Rotation +means that new secret state gets generated (by a random number generator +of the device), and used for all new connections. To avoid disrupting +old connections the old secret state remains in the NIC. A phase bit +carried in the packet headers indicates which generation of secret state +the packet has been encrypted with. + +User facing API +=============== + +PSP is designed primarily for hardware offloads. There is currently +no software fallback for systems which do not have PSP capable NICs. +There is also no standard (or otherwise defined) way of establishing +a PSP-secured connection or exchanging the symmetric keys. + +The expectation is that higher layer protocols will take care of +protocol and key negotiation. For example one may use TLS key exchange, +announce the PSP capability, and switch to PSP if both endpoints +are PSP-capable. + +All configuration of PSP is performed via the PSP netlink family. + +Device discovery +---------------- + +The PSP netlink family defines operations to retrieve information +about the PSP devices available on the system, configure them and +access PSP related statistics. + +Securing a connection +--------------------- + +PSP encryption is currently only supported for TCP connections. +Rx and Tx keys are allocated separately. First the ``rx-assoc`` +Netlink command needs to be issued, specifying a target TCP socket. +Kernel will allocate a new PSP Rx key from the NIC and associate it +with given socket. At this stage socket will accept both PSP-secured +and plain text TCP packets. + +Tx keys are installed using the ``tx-assoc`` Netlink command. +Once the Tx keys are installed all data read from the socket will +be PSP-secured. In other words act of installing Tx keys has the secondary +effect on the Rx direction, requring all received packets to be encrypted. +Since packet reception is asynchronous, to make it possible for the +application to trust that any data read from the socket after the ``tx-assoc`` +call returns success has been encrypted, the kernel will scan the receive +queue of the socket at ``tx-assoc`` time. If any enqueued packet was received +in clear text the Tx association will fail, and application should retry +installing the Tx key after draining the socket (this should not be necessary +if both endpoints are well behaved). + +Rotation notifications +---------------------- + +The rotations of secret state happen asynchornously and are usually +performed by management daemons, not under application control. +The PSP netlink family will generate a notification whenever keys +are rotated. The applications are expected to re-establish connections +before keys are rotated again. + +Kernel implementation +===================== + +Driver notes +------------ + +Drivers are expected to start with no PSP enabled (``psp-versions-ena`` +in ``dev-get`` set to ``0``) whenever possible. The user space should +not depend on this behavior, as future extension may necessitate creation +of devices with PSP already enabled, nonetheless drivers should not enable +PSP by default. Enabling PSP should be the responsibility of the system +component which also takes care of key rotation. + +Note that ``psp-versions-ena`` is expected to be used only for enabling +receive processing. The device is not expected to reject transmit requests +after ``psp-versions-ena`` has been disabled. User may also disable +``psp-versions-ena`` while there are active associations, which will +break all PSP Rx processing. + +Drivers are expected to ensure that secret state is usable upon init +(working keys can be allocated), and that no duplicate keys may be generated +(reuse of SPI without key rotation). Drivers may achieve this by rotating +keys twice before registering the PSP device. + +Drivers must use ``psp_skb_get_assoc_rcu()`` to check if PSP Tx offload +was requested for given skb. On Rx drivers should allocate and populate +the ``SKB_EXT_PSP`` skb extension, and set the skb->decrypted bit to 1. + +Kernel implementation notes +--------------------------- + +PSP implementation follows the TLS offload more closely than the IPsec +offload, with per-socket state, and the use of skb->decrypted to prevent +clear text leaks. + +PSP device is separate from netdev, to make it possible to "delegate" +PSP offload capabilities to software devices (e.g. ``veth``). From patchwork Fri May 10 03:04:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660834 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 747527E772 for ; Fri, 10 May 2024 03:04:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310286; cv=none; b=l46tcMSJoPwjpRi2drAq3ZJljYtg2zn8tSlKvsoCsfZ93kG0YrUS+sfFVxXYB3I3X6P65M1RXvgn1Y3r1AObcxlR/VycrzeYOArJYSxEZulVSDr9uvyM++ZT0Yatc9Qi8QnBfCufawQyxEdMql3vfeSM6rGfSveSHty3AXqADOw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310286; c=relaxed/simple; bh=/m4b1i5EbOGbaRgCrVPJOa1iREd97WxHT73MrEuA4JQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I2iaatUF37O+QksbxYsvXj3JLvB/+ARWhnnVxjB7hDflqYL663XLo1GqjVS3g60RJCGMD1a8GaHcwIL7S4H0hn2kVoiAmE9jjWQHa+0Fd/+zxT7fK9+EEzPqF5gE2iWfdM6eS3XPR7CLgqr9U29Y5QPfiz9aHmNcdsCKpuZLwEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=L/Ce5y/e; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="L/Ce5y/e" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7C499C3277B; Fri, 10 May 2024 03:04:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310286; bh=/m4b1i5EbOGbaRgCrVPJOa1iREd97WxHT73MrEuA4JQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L/Ce5y/ejzBqWZVk3noJdRpvRJy6qsEAkQ0KV2iKZxk/uHGgQLf3VPFBAlMUevw+1 0O1x+fDXnp2vIlu79sHln/IZ5BnM32IxnIkpPi92A10f5ZWnAmZ7CiftoDXBX0qn4D EqKxjo3TKWL0JVojxygFk4XgoJMCw2LZoCAhTLidp3EMro/Z3lM+CTKnavwspz210h 6NV2LCRw9AeUeZx+nQkFCray4jNyk3kJD2tR3Oq2M8sn1gV+e+8s5vPQj/jdToozsP KP0MefU5GBnGJGqZALy+yeEXWQXqibsDtcTeBety9MtZKMOi3FCx+BisSlYICE7ynu WjxOLw1xxQijA== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Jakub Kicinski Subject: [RFC net-next 02/15] psp: base PSP device support Date: Thu, 9 May 2024 20:04:22 -0700 Message-ID: <20240510030435.120935-3-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Add a netlink family for PSP and allow drivers to register support. The "PSP device" is its own object. This allows us to perform more flexible reference counting / lifetime control than if PSP information was part of net_device. In the future we should also be able to "delegate" PSP access to software devices, such as *vlan, veth or netkit more easily. Signed-off-by: Jakub Kicinski --- Documentation/netlink/specs/psp.yaml | 94 +++++++++++ include/linux/netdevice.h | 4 + include/net/psp.h | 12 ++ include/net/psp/functions.h | 14 ++ include/net/psp/types.h | 102 ++++++++++++ include/uapi/linux/psp.h | 42 +++++ net/Kconfig | 1 + net/Makefile | 1 + net/psp/Kconfig | 13 ++ net/psp/Makefile | 5 + net/psp/psp-nl-gen.c | 65 ++++++++ net/psp/psp-nl-gen.h | 30 ++++ net/psp/psp.h | 31 ++++ net/psp/psp_main.c | 130 ++++++++++++++++ net/psp/psp_nl.c | 223 +++++++++++++++++++++++++++ tools/net/ynl/Makefile.deps | 1 + 16 files changed, 768 insertions(+) create mode 100644 Documentation/netlink/specs/psp.yaml create mode 100644 include/net/psp.h create mode 100644 include/net/psp/functions.h create mode 100644 include/net/psp/types.h create mode 100644 include/uapi/linux/psp.h create mode 100644 net/psp/Kconfig create mode 100644 net/psp/Makefile create mode 100644 net/psp/psp-nl-gen.c create mode 100644 net/psp/psp-nl-gen.h create mode 100644 net/psp/psp.h create mode 100644 net/psp/psp_main.c create mode 100644 net/psp/psp_nl.c diff --git a/Documentation/netlink/specs/psp.yaml b/Documentation/netlink/specs/psp.yaml new file mode 100644 index 000000000000..dbb5ef148045 --- /dev/null +++ b/Documentation/netlink/specs/psp.yaml @@ -0,0 +1,94 @@ +# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) + +name: psp + +doc: + PSP Security Protocol Generic Netlink family. + +definitions: + - + type: enum + name: version + entries: [ hdr0-aes-gcm-128, hdr0-aes-gcm-256, + hdr0-aes-gmac-128, hdr0-aes-gmac-256, ] + +attribute-sets: + - + name: dev + attributes: + - + name: id + doc: PSP device ID. + type: u32 + checks: + min: 1 + - + name: ifindex + doc: ifindex of the main netdevice linked to the PSP device. + type: u32 + - + name: psp-versions-cap + doc: Bitmask of PSP versions supported by the device. + type: u32 + enum: version + enum-as-flags: true + - + name: psp-versions-ena + doc: Bitmask of currently enabled (accepted on Rx) PSP versions. + type: u32 + enum: version + enum-as-flags: true + +operations: + list: + - + name: dev-get + doc: Get / dump information about PSP capable devices on the system. + attribute-set: dev + do: + request: + attributes: + - id + reply: &dev-all + attributes: + - id + - ifindex + - psp-versions-cap + - psp-versions-ena + pre: psp-device-get-locked + post: psp-device-unlock + dump: + reply: *dev-all + - + name: dev-add-ntf + doc: Notification about device appearing. + notify: dev-get + mcgrp: mgmt + - + name: dev-del-ntf + doc: Notification about device disappearing. + notify: dev-get + mcgrp: mgmt + - + name: dev-set + doc: Set the configuration of a PSP device. + attribute-set: dev + do: + request: + attributes: + - id + - psp-versions-ena + reply: + attributes: [] + pre: psp-device-get-locked + post: psp-device-unlock + - + name: dev-change-ntf + doc: Notification about device configuration being changed. + notify: dev-get + mcgrp: mgmt + +mcast-groups: + list: + - + name: mgmt diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index cf261fb89d73..7327ed157bc2 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -1874,6 +1874,7 @@ enum netdev_reg_state { * device struct * @mpls_ptr: mpls_dev struct pointer * @mctp_ptr: MCTP specific data + * @psp_dev: PSP crypto device registered for this netdev * * @dev_addr: Hw address (before bcast, * because most packets are unicast) @@ -2251,6 +2252,9 @@ struct net_device { #if IS_ENABLED(CONFIG_MCTP) struct mctp_dev __rcu *mctp_ptr; #endif +#if IS_ENABLED(CONFIG_INET_PSP) + struct psp_dev __rcu *psp_dev; +#endif /* * Cache lines mostly used on receive path (including eth_type_trans()) diff --git a/include/net/psp.h b/include/net/psp.h new file mode 100644 index 000000000000..33bb4d1dc46e --- /dev/null +++ b/include/net/psp.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __NET_PSP_ALL_H +#define __NET_PSP_ALL_H + +#include +#include +#include + +/* Do not add any code here. Put it in the sub-headers instead. */ + +#endif /* __NET_PSP_ALL_H */ diff --git a/include/net/psp/functions.h b/include/net/psp/functions.h new file mode 100644 index 000000000000..074f9df9afc3 --- /dev/null +++ b/include/net/psp/functions.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __NET_PSP_HELPERS_H +#define __NET_PSP_HELPERS_H + +#include + +/* Driver-facing API */ +struct psp_dev * +psp_dev_create(struct net_device *netdev, struct psp_dev_ops *psd_ops, + struct psp_dev_caps *psd_caps, void *priv_ptr); +void psp_dev_unregister(struct psp_dev *psd); + +#endif /* __NET_PSP_HELPERS_H */ diff --git a/include/net/psp/types.h b/include/net/psp/types.h new file mode 100644 index 000000000000..dbc5423a53df --- /dev/null +++ b/include/net/psp/types.h @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __NET_PSP_H +#define __NET_PSP_H + +#include +#include + +struct netlink_ext_ack; + +#define PSP_DEFAULT_UDP_PORT 1000 + +struct psphdr { + u8 nexthdr; + u8 hdrlen; + u8 crypt_offset; + u8 verfl; + __be32 spi; + __be64 iv; + __be64 vc[]; /* optional */ +}; + +#define PSP_SPI_KEY_ID GENMASK(30, 0) +#define PSP_SPI_KEY_PHASE BIT(31) + +#define PSPHDR_CRYPT_OFFSET GENMASK(5, 0) + +#define PSPHDR_VERFL_SAMPLE BIT(7) +#define PSPHDR_VERFL_DROP BIT(6) +#define PSPHDR_VERFL_VERSION GENMASK(5, 2) +#define PSPHDR_VERFL_VIRT BIT(1) +#define PSPHDR_VERFL_ONE BIT(0) + +#define PSP_HDRLEN_NOOPT ((sizeof(struct psphdr) - 8) / 8) + +/** + * struct psp_dev_config - PSP device configuration + * @versions: PSP versions enabled on the device + */ +struct psp_dev_config { + u32 versions; +}; + +/** + * struct psp_dev - PSP device struct + * @main_netdev: original netdevice of this PSP device + * @ops: driver callbacks + * @caps: device capabilities + * @drv_priv: driver priv pointer + * @lock: instance lock, protects all fields + * @refcnt: reference count for the instance + * @id: instance id + * @config: current device configuration + * + * @rcu: RCU head for freeing the structure + */ +struct psp_dev { + struct net_device *main_netdev; + + struct psp_dev_ops *ops; + struct psp_dev_caps *caps; + void *drv_priv; + + struct mutex lock; + refcount_t refcnt; + + u32 id; + + struct psp_dev_config config; + + struct rcu_head rcu; +}; + +/** + * struct psp_dev_caps - PSP device capabilities + */ +struct psp_dev_caps { + /** + * @versions: mask of supported PSP versions + * Set this field to 0 to indicate PSP is not supported at all. + */ + u32 versions; +}; + +#define PSP_V0_KEY 16 +#define PSP_V1_KEY 32 +#define PSP_MAX_KEY 32 + +/** + * struct psp_dev_ops - netdev driver facing PSP callbacks + */ +struct psp_dev_ops { + /** + * @set_config: set configuration of a PSP device + * Driver can inspect @psd->config for the previous configuration. + * Core will update @psd->config with @config on success. + */ + int (*set_config)(struct psp_dev *psd, struct psp_dev_config *conf, + struct netlink_ext_ack *extack); +}; + +#endif /* __NET_PSP_H */ diff --git a/include/uapi/linux/psp.h b/include/uapi/linux/psp.h new file mode 100644 index 000000000000..4a404f085190 --- /dev/null +++ b/include/uapi/linux/psp.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ +/* Do not edit directly, auto-generated from: */ +/* Documentation/netlink/specs/psp.yaml */ +/* YNL-GEN uapi header */ + +#ifndef _UAPI_LINUX_PSP_H +#define _UAPI_LINUX_PSP_H + +#define PSP_FAMILY_NAME "psp" +#define PSP_FAMILY_VERSION 1 + +enum psp_version { + PSP_VERSION_HDR0_AES_GCM_128, + PSP_VERSION_HDR0_AES_GCM_256, + PSP_VERSION_HDR0_AES_GMAC_128, + PSP_VERSION_HDR0_AES_GMAC_256, +}; + +enum { + PSP_A_DEV_ID = 1, + PSP_A_DEV_IFINDEX, + PSP_A_DEV_PSP_VERSIONS_CAP, + PSP_A_DEV_PSP_VERSIONS_ENA, + + __PSP_A_DEV_MAX, + PSP_A_DEV_MAX = (__PSP_A_DEV_MAX - 1) +}; + +enum { + PSP_CMD_DEV_GET = 1, + PSP_CMD_DEV_ADD_NTF, + PSP_CMD_DEV_DEL_NTF, + PSP_CMD_DEV_SET, + PSP_CMD_DEV_CHANGE_NTF, + + __PSP_CMD_MAX, + PSP_CMD_MAX = (__PSP_CMD_MAX - 1) +}; + +#define PSP_MCGRP_MGMT "mgmt" + +#endif /* _UAPI_LINUX_PSP_H */ diff --git a/net/Kconfig b/net/Kconfig index f0a8692496ff..3079ed6711c0 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -69,6 +69,7 @@ config SKB_EXTENSIONS menu "Networking options" source "net/packet/Kconfig" +source "net/psp/Kconfig" source "net/unix/Kconfig" source "net/tls/Kconfig" source "net/xfrm/Kconfig" diff --git a/net/Makefile b/net/Makefile index 65bb8c72a35e..c47b1ae4540d 100644 --- a/net/Makefile +++ b/net/Makefile @@ -18,6 +18,7 @@ obj-$(CONFIG_INET) += ipv4/ obj-$(CONFIG_TLS) += tls/ obj-$(CONFIG_XFRM) += xfrm/ obj-$(CONFIG_UNIX) += unix/ +obj-$(CONFIG_INET_PSP) += psp/ obj-y += ipv6/ obj-$(CONFIG_PACKET) += packet/ obj-$(CONFIG_NET_KEY) += key/ diff --git a/net/psp/Kconfig b/net/psp/Kconfig new file mode 100644 index 000000000000..55f9dd87446b --- /dev/null +++ b/net/psp/Kconfig @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# PSP configuration +# +config INET_PSP + bool "PSP Security Protocol support" + depends on INET + help + Enable kernel support for the PSP protocol. + For more information see: + https://raw.githubusercontent.com/google/psp/main/doc/PSP_Arch_Spec.pdf + + If unsure, say N. diff --git a/net/psp/Makefile b/net/psp/Makefile new file mode 100644 index 000000000000..41b51d06e560 --- /dev/null +++ b/net/psp/Makefile @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0-only + +obj-$(CONFIG_INET_PSP) += psp.o + +psp-y := psp_main.o psp_nl.o psp-nl-gen.o diff --git a/net/psp/psp-nl-gen.c b/net/psp/psp-nl-gen.c new file mode 100644 index 000000000000..859712e7c2c1 --- /dev/null +++ b/net/psp/psp-nl-gen.c @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) +/* Do not edit directly, auto-generated from: */ +/* Documentation/netlink/specs/psp.yaml */ +/* YNL-GEN kernel source */ + +#include +#include + +#include "psp-nl-gen.h" + +#include + +/* PSP_CMD_DEV_GET - do */ +static const struct nla_policy psp_dev_get_nl_policy[PSP_A_DEV_ID + 1] = { + [PSP_A_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1), +}; + +/* PSP_CMD_DEV_SET - do */ +static const struct nla_policy psp_dev_set_nl_policy[PSP_A_DEV_PSP_VERSIONS_ENA + 1] = { + [PSP_A_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1), + [PSP_A_DEV_PSP_VERSIONS_ENA] = NLA_POLICY_MASK(NLA_U32, 0xf), +}; + +/* Ops table for psp */ +static const struct genl_split_ops psp_nl_ops[] = { + { + .cmd = PSP_CMD_DEV_GET, + .pre_doit = psp_device_get_locked, + .doit = psp_nl_dev_get_doit, + .post_doit = psp_device_unlock, + .policy = psp_dev_get_nl_policy, + .maxattr = PSP_A_DEV_ID, + .flags = GENL_CMD_CAP_DO, + }, + { + .cmd = PSP_CMD_DEV_GET, + .dumpit = psp_nl_dev_get_dumpit, + .flags = GENL_CMD_CAP_DUMP, + }, + { + .cmd = PSP_CMD_DEV_SET, + .pre_doit = psp_device_get_locked, + .doit = psp_nl_dev_set_doit, + .post_doit = psp_device_unlock, + .policy = psp_dev_set_nl_policy, + .maxattr = PSP_A_DEV_PSP_VERSIONS_ENA, + .flags = GENL_CMD_CAP_DO, + }, +}; + +static const struct genl_multicast_group psp_nl_mcgrps[] = { + [PSP_NLGRP_MGMT] = { "mgmt", }, +}; + +struct genl_family psp_nl_family __ro_after_init = { + .name = PSP_FAMILY_NAME, + .version = PSP_FAMILY_VERSION, + .netnsok = true, + .parallel_ops = true, + .module = THIS_MODULE, + .split_ops = psp_nl_ops, + .n_split_ops = ARRAY_SIZE(psp_nl_ops), + .mcgrps = psp_nl_mcgrps, + .n_mcgrps = ARRAY_SIZE(psp_nl_mcgrps), +}; diff --git a/net/psp/psp-nl-gen.h b/net/psp/psp-nl-gen.h new file mode 100644 index 000000000000..a099686cab5d --- /dev/null +++ b/net/psp/psp-nl-gen.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ +/* Do not edit directly, auto-generated from: */ +/* Documentation/netlink/specs/psp.yaml */ +/* YNL-GEN kernel header */ + +#ifndef _LINUX_PSP_GEN_H +#define _LINUX_PSP_GEN_H + +#include +#include + +#include + +int psp_device_get_locked(const struct genl_split_ops *ops, + struct sk_buff *skb, struct genl_info *info); +void +psp_device_unlock(const struct genl_split_ops *ops, struct sk_buff *skb, + struct genl_info *info); + +int psp_nl_dev_get_doit(struct sk_buff *skb, struct genl_info *info); +int psp_nl_dev_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb); +int psp_nl_dev_set_doit(struct sk_buff *skb, struct genl_info *info); + +enum { + PSP_NLGRP_MGMT, +}; + +extern struct genl_family psp_nl_family; + +#endif /* _LINUX_PSP_GEN_H */ diff --git a/net/psp/psp.h b/net/psp/psp.h new file mode 100644 index 000000000000..94d0cc31a61f --- /dev/null +++ b/net/psp/psp.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __PSP_PSP_H +#define __PSP_PSP_H + +#include +#include +#include +#include +#include + +extern struct xarray psp_devs; +extern struct mutex psp_devs_lock; + +void psp_dev_destroy(struct psp_dev *psd); +int psp_dev_check_access(struct psp_dev *psd, struct net *net); + +void psp_nl_notify_dev(struct psp_dev *psd, u32 cmd); + +static inline void psp_dev_get(struct psp_dev *psd) +{ + refcount_inc(&psd->refcnt); +} + +static inline void psp_dev_put(struct psp_dev *psd) +{ + if (refcount_dec_and_test(&psd->refcnt)) + psp_dev_destroy(psd); +} + +#endif /* __PSP_PSP_H */ diff --git a/net/psp/psp_main.c b/net/psp/psp_main.c new file mode 100644 index 000000000000..cf463f757892 --- /dev/null +++ b/net/psp/psp_main.c @@ -0,0 +1,130 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include +#include +#include +#include + +#include "psp.h" +#include "psp-nl-gen.h" + +DEFINE_XARRAY_ALLOC1(psp_devs); +struct mutex psp_devs_lock; + +/** + * DOC: PSP locking + * + * psp_devs_lock protects the psp_devs xarray. + * Ordering is take the psp_devs_lock and then the instance lock. + * Each instance is protected by RCU, and has a refcount. + * When driver unregisters the instance gets flushed, but struct sticks around. + */ + +/** + * psp_dev_check_access() - check if user in a given net ns can access PSP dev + * @psd: PSP device structure user is trying to access + * @net: net namespace user is in + * + * Return: 0 if PSP device should be visible in @net, errno otherwise. + */ +int psp_dev_check_access(struct psp_dev *psd, struct net *net) +{ + if (dev_net(psd->main_netdev) == net) + return 0; + return -ENOENT; +} + +/** + * psp_dev_create() - create and register PSP device + * @netdev: main netdevice + * @psd_ops: driver callbacks + * @psd_caps: device capabilities + * @priv_ptr: back-pointer to driver private data + * + * Return: pointer to allocated PSP device, or ERR_PTR. + */ +struct psp_dev * +psp_dev_create(struct net_device *netdev, + struct psp_dev_ops *psd_ops, struct psp_dev_caps *psd_caps, + void *priv_ptr) +{ + struct psp_dev *psd; + static u32 last_id; + int err; + + if (WARN_ON(!psd_caps->versions || + !psd_ops->set_config)) + return ERR_PTR(-EINVAL); + + psd = kzalloc(sizeof(*psd), GFP_KERNEL); + if (!psd) + return ERR_PTR(-ENOMEM); + + psd->main_netdev = netdev; + psd->ops = psd_ops; + psd->caps = psd_caps; + psd->drv_priv = priv_ptr; + + mutex_init(&psd->lock); + refcount_set(&psd->refcnt, 1); + + mutex_lock(&psp_devs_lock); + err = xa_alloc_cyclic(&psp_devs, &psd->id, psd, xa_limit_31b, + &last_id, GFP_KERNEL); + if (err) { + mutex_unlock(&psp_devs_lock); + kfree(psd); + return ERR_PTR(err); + } + mutex_lock(&psd->lock); + mutex_unlock(&psp_devs_lock); + + psp_nl_notify_dev(psd, PSP_CMD_DEV_ADD_NTF); + + rcu_assign_pointer(netdev->psp_dev, psd); + + mutex_unlock(&psd->lock); + + return psd; +} +EXPORT_SYMBOL(psp_dev_create); + +void psp_dev_destroy(struct psp_dev *psd) +{ + mutex_destroy(&psd->lock); + kfree_rcu(psd, rcu); +} + +/** + * psp_dev_unregister() - unregister PSP device + * @psd: PSP device structure + */ +void psp_dev_unregister(struct psp_dev *psd) +{ + mutex_lock(&psp_devs_lock); + mutex_lock(&psd->lock); + + psp_nl_notify_dev(psd, PSP_CMD_DEV_DEL_NTF); + xa_erase(&psp_devs, psd->id); + mutex_unlock(&psp_devs_lock); + + rcu_assign_pointer(psd->main_netdev->psp_dev, NULL); + + psd->ops = NULL; + psd->drv_priv = NULL; + + mutex_unlock(&psd->lock); + + psp_dev_put(psd); +} +EXPORT_SYMBOL(psp_dev_unregister); + +static int __init psp_init(void) +{ + mutex_init(&psp_devs_lock); + + return genl_register_family(&psp_nl_family); +} + +subsys_initcall(psp_init); diff --git a/net/psp/psp_nl.c b/net/psp/psp_nl.c new file mode 100644 index 000000000000..fda5ce800f82 --- /dev/null +++ b/net/psp/psp_nl.c @@ -0,0 +1,223 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include +#include +#include +#include + +#include "psp-nl-gen.h" +#include "psp.h" + +/* Netlink helpers */ + +static struct sk_buff *psp_nl_reply_new(struct genl_info *info) +{ + struct sk_buff *rsp; + void *hdr; + + rsp = genlmsg_new(GENLMSG_DEFAULT_SIZE, GFP_KERNEL); + if (!rsp) + return NULL; + + hdr = genlmsg_iput(rsp, info); + if (!hdr) { + nlmsg_free(rsp); + return NULL; + } + + return rsp; +} + +static int psp_nl_reply_send(struct sk_buff *rsp, struct genl_info *info) +{ + /* Note that this *only* works with a single message per skb! */ + nlmsg_end(rsp, (struct nlmsghdr *)rsp->data); + + return genlmsg_reply(rsp, info); +} + +/* Device stuff */ + +static struct psp_dev * +psp_device_get_and_lock(struct net *net, struct nlattr *dev_id) +{ + struct psp_dev *psd; + int err; + + mutex_lock(&psp_devs_lock); + psd = xa_load(&psp_devs, nla_get_u32(dev_id)); + if (!psd) { + mutex_unlock(&psp_devs_lock); + return ERR_PTR(-ENODEV); + } + + mutex_lock(&psd->lock); + mutex_unlock(&psp_devs_lock); + + err = psp_dev_check_access(psd, net); + if (err) { + mutex_unlock(&psd->lock); + return ERR_PTR(err); + } + + return psd; +} + +int psp_device_get_locked(const struct genl_split_ops *ops, + struct sk_buff *skb, struct genl_info *info) +{ + if (GENL_REQ_ATTR_CHECK(info, PSP_A_DEV_ID)) + return -EINVAL; + + info->user_ptr[0] = psp_device_get_and_lock(genl_info_net(info), + info->attrs[PSP_A_DEV_ID]); + return PTR_ERR_OR_ZERO(info->user_ptr[0]); +} + +void +psp_device_unlock(const struct genl_split_ops *ops, struct sk_buff *skb, + struct genl_info *info) +{ + struct psp_dev *psd = info->user_ptr[0]; + + mutex_unlock(&psd->lock); +} + +static int +psp_nl_dev_fill(struct psp_dev *psd, struct sk_buff *rsp, + const struct genl_info *info) +{ + void *hdr; + + hdr = genlmsg_iput(rsp, info); + if (!hdr) + return -EMSGSIZE; + + if (nla_put_u32(rsp, PSP_A_DEV_ID, psd->id) || + nla_put_u32(rsp, PSP_A_DEV_IFINDEX, psd->main_netdev->ifindex) || + nla_put_u32(rsp, PSP_A_DEV_PSP_VERSIONS_CAP, psd->caps->versions) || + nla_put_u32(rsp, PSP_A_DEV_PSP_VERSIONS_ENA, psd->config.versions)) + goto err_cancel_msg; + + genlmsg_end(rsp, hdr); + return 0; + +err_cancel_msg: + genlmsg_cancel(rsp, hdr); + return -EMSGSIZE; +} + +void psp_nl_notify_dev(struct psp_dev *psd, u32 cmd) +{ + struct genl_info info; + struct sk_buff *ntf; + + if (!genl_has_listeners(&psp_nl_family, dev_net(psd->main_netdev), + PSP_NLGRP_MGMT)) + return; + + ntf = genlmsg_new(GENLMSG_DEFAULT_SIZE, GFP_KERNEL); + if (!ntf) + return; + + genl_info_init_ntf(&info, &psp_nl_family, cmd); + if (psp_nl_dev_fill(psd, ntf, &info)) { + nlmsg_free(ntf); + return; + } + + genlmsg_multicast_netns(&psp_nl_family, dev_net(psd->main_netdev), ntf, + 0, PSP_NLGRP_MGMT, GFP_KERNEL); +} + +int psp_nl_dev_get_doit(struct sk_buff *req, struct genl_info *info) +{ + struct psp_dev *psd = info->user_ptr[0]; + struct sk_buff *rsp; + int err; + + rsp = genlmsg_new(GENLMSG_DEFAULT_SIZE, GFP_KERNEL); + if (!rsp) + return -ENOMEM; + + err = psp_nl_dev_fill(psd, rsp, info); + if (err) + goto err_free_msg; + + return genlmsg_reply(rsp, info); + +err_free_msg: + nlmsg_free(rsp); + return err; +} + +static int +psp_nl_dev_get_dumpit_one(struct sk_buff *rsp, struct netlink_callback *cb, + struct psp_dev *psd) +{ + if (psp_dev_check_access(psd, sock_net(rsp->sk))) + return 0; + + return psp_nl_dev_fill(psd, rsp, genl_info_dump(cb)); +} + +int psp_nl_dev_get_dumpit(struct sk_buff *rsp, struct netlink_callback *cb) +{ + struct psp_dev *psd; + int err = 0; + + mutex_lock(&psp_devs_lock); + xa_for_each_start(&psp_devs, cb->args[0], psd, cb->args[0]) { + mutex_lock(&psd->lock); + err = psp_nl_dev_get_dumpit_one(rsp, cb, psd); + mutex_unlock(&psd->lock); + if (err) + break; + } + mutex_unlock(&psp_devs_lock); + + return err; +} + +int psp_nl_dev_set_doit(struct sk_buff *skb, struct genl_info *info) +{ + struct psp_dev *psd = info->user_ptr[0]; + struct psp_dev_config new_config; + struct sk_buff *rsp; + int err; + + memcpy(&new_config, &psd->config, sizeof(new_config)); + + if (info->attrs[PSP_A_DEV_PSP_VERSIONS_ENA]) { + new_config.versions = + nla_get_u32(info->attrs[PSP_A_DEV_PSP_VERSIONS_ENA]); + if (new_config.versions & ~psd->caps->versions) { + NL_SET_ERR_MSG(info->extack, "Requested PSP versions not supported by the device"); + return -EINVAL; + } + } else { + NL_SET_ERR_MSG(info->extack, "No settings present"); + return -EINVAL; + } + + rsp = psp_nl_reply_new(info); + if (!rsp) + return -ENOMEM; + + if (memcmp(&new_config, &psd->config, sizeof(new_config))) { + err = psd->ops->set_config(psd, &new_config, info->extack); + if (err) + goto err_free_rsp; + + memcpy(&psd->config, &new_config, sizeof(new_config)); + } + + psp_nl_notify_dev(psd, PSP_CMD_DEV_CHANGE_NTF); + + return psp_nl_reply_send(rsp, info); + +err_free_rsp: + nlmsg_free(rsp); + return err; +} diff --git a/tools/net/ynl/Makefile.deps b/tools/net/ynl/Makefile.deps index f4e8eb79c1b8..e191ea3cefc0 100644 --- a/tools/net/ynl/Makefile.deps +++ b/tools/net/ynl/Makefile.deps @@ -25,3 +25,4 @@ CFLAGS_nfsd:=$(call get_hdr_inc,_LINUX_NFSD_NETLINK_H,nfsd_netlink.h) CFLAGS_ovs_datapath:=$(call get_hdr_inc,__LINUX_OPENVSWITCH_H,openvswitch.h) CFLAGS_ovs_flow:=$(call get_hdr_inc,__LINUX_OPENVSWITCH_H,openvswitch.h) CFLAGS_ovs_vport:=$(call get_hdr_inc,__LINUX_OPENVSWITCH_H,openvswitch.h) +CFLAGS_psp:=$(call get_hdr_inc,_LINUX_PSP_H,psp.h) From patchwork Fri May 10 03:04:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660835 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8F56148FED for ; Fri, 10 May 2024 03:04:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310286; cv=none; b=iK0ec//M0Z/ArBc19IGkab6YtwC0aqXx5y45zSXwYJtGn88gqaFQonJLlXpOoohp9VrCxwQ9WYKli7/aj5rNVzRQ6sbMrla082VWB1EZN8hee69jgfxgpQWnhl4YMSQMCFdNDfIOnyFRDD7rvL9k0MdfgA6LUsEFwBHwDQcBqzg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310286; c=relaxed/simple; bh=DvCxHVlyFlhacfr5ZL0+wOC0lydStxwkNZBk+8RHov8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YNtWVNEDK4dLdeHPShBI32NJul3e5ML69dn87wRPA/Xs/dIx1XGB9hCBJGj66zIAf9DNSaNDvMpXd8yFKUIx21cHxrgGvp7scAGaZ4pZmRKtGjk4f0b5B/aNLTw7zXfXk9aAAf6oU87PVvno7jrm12kxw4u33TExp2cjfbSUhzk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XSyFojcd; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XSyFojcd" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 34421C2BBFC; Fri, 10 May 2024 03:04:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310286; bh=DvCxHVlyFlhacfr5ZL0+wOC0lydStxwkNZBk+8RHov8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XSyFojcdBNkO/C0reChayv+xVLhFOG5t4FyLSy1HNCT4hC3OLHDsG94niMDhkdu3N tQCCA7gELyHHK8Lrj4pR0Zwpwg6ZdM1M/OwzU6DUdEGCdz4z6hQMtvTj6r9PmOfzCa rNhiFCkBPoiH/4jZiiWXqY1yJkyUTmr2dlIllun/qKfGrSB8FWs4x5BtQgnAkY2YiP qJqHjSBgc/AWfrrh46m52JihHNL6or2etUkeOv04xyzQlMtcz7xQ5aHFL2ELtQknJM mSgvQGmKGmi4S+qygRVslK7+Yot7tXfeFI2rT/tzpA2v2NntBD5DQnpFlbeBX6nwqt XJ4zUacJmeJkg== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Jakub Kicinski Subject: [RFC net-next 03/15] net: modify core data structures for PSP datapath support Date: Thu, 9 May 2024 20:04:23 -0700 Message-ID: <20240510030435.120935-4-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Add pointers to psp data structures to core networking structs, and an SKB extension to carry the PSP information from the drivers to the socket layer. Signed-off-by: Jakub Kicinski --- Split out to a separate patch for ease of review, I will squash if that's not helpful. --- include/linux/skbuff.h | 3 +++ include/linux/tcp.h | 3 +++ include/net/psp/functions.h | 5 +++++ include/net/psp/types.h | 7 +++++++ include/net/sock.h | 4 ++++ net/core/skbuff.c | 4 ++++ net/core/sock.c | 2 ++ net/ipv4/inet_connection_sock.c | 2 ++ net/ipv4/tcp_minisocks.c | 6 ++++-- net/mptcp/protocol.c | 2 ++ 10 files changed, 36 insertions(+), 2 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index c0b97c93a6de..4689255c66d2 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -4659,6 +4659,9 @@ enum skb_ext_id { #endif #if IS_ENABLED(CONFIG_MCTP_FLOWS) SKB_EXT_MCTP, +#endif +#if IS_ENABLED(CONFIG_INET_PSP) + SKB_EXT_PSP, #endif SKB_EXT_NUM, /* must be last */ }; diff --git a/include/linux/tcp.h b/include/linux/tcp.h index 6a5e08b937b3..368ea3a2b338 100644 --- a/include/linux/tcp.h +++ b/include/linux/tcp.h @@ -551,6 +551,9 @@ struct tcp_timewait_sock { #ifdef CONFIG_TCP_AO struct tcp_ao_info __rcu *ao_info; #endif +#if IS_ENABLED(CONFIG_INET_PSP) + struct psp_assoc __rcu *psp_assoc; +#endif }; static inline struct tcp_timewait_sock *tcp_twsk(const struct sock *sk) diff --git a/include/net/psp/functions.h b/include/net/psp/functions.h index 074f9df9afc3..9ff0f2b5744f 100644 --- a/include/net/psp/functions.h +++ b/include/net/psp/functions.h @@ -5,10 +5,15 @@ #include +struct tcp_timewait_sock; + /* Driver-facing API */ struct psp_dev * psp_dev_create(struct net_device *netdev, struct psp_dev_ops *psd_ops, struct psp_dev_caps *psd_caps, void *priv_ptr); void psp_dev_unregister(struct psp_dev *psd); +static inline void psp_sk_assoc_free(struct sock *sk) { } +static inline void psp_twsk_assoc_free(struct tcp_timewait_sock *tw) { } + #endif /* __NET_PSP_HELPERS_H */ diff --git a/include/net/psp/types.h b/include/net/psp/types.h index dbc5423a53df..a23d9bd9ce96 100644 --- a/include/net/psp/types.h +++ b/include/net/psp/types.h @@ -86,6 +86,13 @@ struct psp_dev_caps { #define PSP_V1_KEY 32 #define PSP_MAX_KEY 32 +struct psp_skb_ext { + __be32 spi; + /* generation and version are 8b but we don't want holes */ + u16 generation; + u16 version; +}; + /** * struct psp_dev_ops - netdev driver facing PSP callbacks */ diff --git a/include/net/sock.h b/include/net/sock.h index 0450494a1766..dc4c46ac0984 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -249,6 +249,7 @@ struct sk_filter; * @sk_dst_cache: destination cache * @sk_dst_pending_confirm: need to confirm neighbour * @sk_policy: flow policy + * @psp_assoc: PSP association, if socket is PSP-secured * @sk_receive_queue: incoming packets * @sk_wmem_alloc: transmit queue bytes committed * @sk_tsq_flags: TCP Small Queues flags @@ -436,6 +437,9 @@ struct sock { struct mem_cgroup *sk_memcg; #ifdef CONFIG_XFRM struct xfrm_policy __rcu *sk_policy[2]; +#endif +#if IS_ENABLED(CONFIG_INET_PSP) + struct psp_assoc __rcu *psp_assoc; #endif __cacheline_group_end(sock_read_rxtx); diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 466999a7515e..1b6821d8dede 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -77,6 +77,7 @@ #include #include #include +#include #include #include @@ -4957,6 +4958,9 @@ static const u8 skb_ext_type_len[] = { #if IS_ENABLED(CONFIG_MCTP_FLOWS) [SKB_EXT_MCTP] = SKB_EXT_CHUNKSIZEOF(struct mctp_flow), #endif +#if IS_ENABLED(CONFIG_INET_PSP) + [SKB_EXT_PSP] = SKB_EXT_CHUNKSIZEOF(struct psp_skb_ext), +#endif }; static __always_inline unsigned int skb_ext_total_length(void) diff --git a/net/core/sock.c b/net/core/sock.c index 8d6e638b5426..24e9113e0417 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -142,6 +142,7 @@ #include #include +#include #include #include @@ -3757,6 +3758,7 @@ void sk_common_release(struct sock *sk) sock_orphan(sk); xfrm_sk_free_policy(sk); + psp_sk_assoc_free(sk); sock_put(sk); } diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index 3b38610958ee..10d4be66046a 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #if IS_ENABLED(CONFIG_IPV6) @@ -1226,6 +1227,7 @@ void inet_csk_destroy_sock(struct sock *sk) sk_stream_kill_queues(sk); xfrm_sk_free_policy(sk); + psp_sk_assoc_free(sk); this_cpu_dec(*sk->sk_prot->orphan_count); diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c index 7d543569a180..660e890f3c74 100644 --- a/net/ipv4/tcp_minisocks.c +++ b/net/ipv4/tcp_minisocks.c @@ -23,6 +23,7 @@ #include #include #include +#include static bool tcp_in_window(u32 seq, u32 end_seq, u32 s_win, u32 e_win) { @@ -377,15 +378,16 @@ static void tcp_md5_twsk_free_rcu(struct rcu_head *head) void tcp_twsk_destructor(struct sock *sk) { + struct tcp_timewait_sock *twsk = tcp_twsk(sk); + #ifdef CONFIG_TCP_MD5SIG if (static_branch_unlikely(&tcp_md5_needed.key)) { - struct tcp_timewait_sock *twsk = tcp_twsk(sk); - if (twsk->tw_md5_key) call_rcu(&twsk->tw_md5_key->rcu, tcp_md5_twsk_free_rcu); } #endif tcp_ao_destroy_sock(sk, true); + psp_twsk_assoc_free(twsk); } EXPORT_SYMBOL_GPL(tcp_twsk_destructor); diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index bb8f96f2b86f..cd79bcecebc2 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -23,6 +23,7 @@ #include #include #include +#include #include "protocol.h" #include "mib.h" @@ -3010,6 +3011,7 @@ static void __mptcp_destroy_sock(struct sock *sk) WARN_ON_ONCE(msk->rmem_released); sk_stream_kill_queues(sk); xfrm_sk_free_policy(sk); + psp_sk_assoc_free(sk); sock_put(sk); } From patchwork Fri May 10 03:04:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660836 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A15181494DB for ; Fri, 10 May 2024 03:04:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310287; cv=none; b=LWHFqUx9xAXqSQBWCQWv08n5DwsTItZ9yo7CnnFXHCQNNdi5bcgLyD1nYcckUWO2R8p1a9ChMP//RmDDAdk19WmGxyeRw9d5pOTa8wupYCSOWw4Im5rcvDzrZyJembV6AD7rQT+w4vmEHN+MPYtXW98znDPAoM4Ds0D0kUxDO2M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310287; c=relaxed/simple; bh=Ym6ILrcUtQ1GojIxH94PPU61ka8pqvHFVJYm017G8y4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AdImzhOtRdLBvghs240nJhJh4WWU1w70nybV/+/jq1BFbu7JXQ6ojl5ogF6B1Rl+EUF2/3FMElkSnG22x8/UsiR+TjAkzHhBPhlfvRKGPjnJwZCkDgffkju/hIvj+4HdpvGPcDna6PKcF2nmp3EkuNl9QE/xc+tplOfV93UbieA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=blDHSvuY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="blDHSvuY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3282C32786; Fri, 10 May 2024 03:04:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310287; bh=Ym6ILrcUtQ1GojIxH94PPU61ka8pqvHFVJYm017G8y4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=blDHSvuYncQZCjHUZJFS+wT6wpBSksL38qUKzZGEO5P3oNizSUfMUIm2lIQTpWmQd WltPvEX1y0rum+dVk5dmkMb/thz19gKKHbGy4rnC5WZaVhJvGjeGxTs109qeEMIJHZ R1kgPk3Lqz8e/spNQQOYmh8pm8K8wH4N6Aq/u7dUypq7LKHlPqk899GYSTJE9H4zGX pcM/vg361aB4nkrQ4WD3NmP7YiZyAqo02kfHuMEQXoSpcJxe0Ztc8aweHtrc+7mfZS TOf6qt3F5wxSh/EDMonbLzcX2VWrk53vZZ6HMgCSJ9yAjizZnwnJizksQgKEi2TZeg NljDb0EBjY3uQ== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Jakub Kicinski Subject: [RFC net-next 04/15] tcp: add datapath logic for PSP with inline key exchange Date: Thu, 9 May 2024 20:04:24 -0700 Message-ID: <20240510030435.120935-5-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Add validation points and state propagation to support PSP key exchange inline, on TCP connections. The expectation is that application will use some well established mechanism like TLS handshake to establish a secure channel over the connection and if both endpoints are PSP-capable - exchange and install PSP keys. Because the connection can existing in PSP-unsecured and PSP-secured state we need to make sure that there are no race conditions or retransmission leaks. On Tx - mark packets with the skb->decrypted bit when PSP key is at the enqueue time. Drivers should only encrypt packets with this bit set. This prevents retransmissions getting encrypted when original transmission was not. Similarly to TLS, we'll use sk->sk_validate_xmit_skb to make sure PSP skbs can't "escape" via a PSP-unaware device without being encrypted. On Rx - validation is done under socket lock. This moves the validation point later than xfrm, for example. Please see the documentation patch for more details on the flow of securing a connection, but for the purpose of this patch what's important is that we want to enforce the invariant that once connection is secured any skb in the receive queue has been encrypted with PSP. Add trivialities like GRO and coalescing checks. This change only adds the validation points, for ease of review. Subsequent change will add the ability to install keys, and flesh the enforcement logic out Signed-off-by: Jakub Kicinski --- include/net/dropreason-core.h | 6 +++ include/net/psp/functions.h | 74 +++++++++++++++++++++++++++++++++++ net/core/gro.c | 2 + net/ipv4/tcp.c | 2 + net/ipv4/tcp_ipv4.c | 9 ++++- net/ipv4/tcp_minisocks.c | 15 +++++++ net/ipv4/tcp_output.c | 16 +++++--- net/ipv6/tcp_ipv6.c | 10 +++++ net/psp/Kconfig | 1 + 9 files changed, 128 insertions(+), 7 deletions(-) diff --git a/include/net/dropreason-core.h b/include/net/dropreason-core.h index 9707ab54fdd5..29f1de9f5148 100644 --- a/include/net/dropreason-core.h +++ b/include/net/dropreason-core.h @@ -92,6 +92,8 @@ FN(PACKET_SOCK_ERROR) \ FN(TC_CHAIN_NOTFOUND) \ FN(TC_RECLASSIFY_LOOP) \ + FN(PSP_INPUT) \ + FN(PSP_OUTPUT) \ FNe(MAX) /** @@ -418,6 +420,10 @@ enum skb_drop_reason { * iterations. */ SKB_DROP_REASON_TC_RECLASSIFY_LOOP, + /** @SKB_DROP_REASON_PSP_INPUT: PSP input checks failed */ + SKB_DROP_REASON_PSP_INPUT, + /** @SKB_DROP_REASON_PSP_OUTPUT: PSP output checks failed */ + SKB_DROP_REASON_PSP_OUTPUT, /** * @SKB_DROP_REASON_MAX: the maximum of core drop reasons, which * shouldn't be used as a real 'reason' - only for tracing code gen diff --git a/include/net/psp/functions.h b/include/net/psp/functions.h index 9ff0f2b5744f..7a3b897ecc69 100644 --- a/include/net/psp/functions.h +++ b/include/net/psp/functions.h @@ -3,6 +3,8 @@ #ifndef __NET_PSP_HELPERS_H #define __NET_PSP_HELPERS_H +#include +#include #include struct tcp_timewait_sock; @@ -13,7 +15,79 @@ psp_dev_create(struct net_device *netdev, struct psp_dev_ops *psd_ops, struct psp_dev_caps *psd_caps, void *priv_ptr); void psp_dev_unregister(struct psp_dev *psd); +/* Kernel-facing API */ +#if IS_ENABLED(CONFIG_INET_PSP) static inline void psp_sk_assoc_free(struct sock *sk) { } +static inline void +psp_twsk_init(struct tcp_timewait_sock *tw, struct sock *sk) { } static inline void psp_twsk_assoc_free(struct tcp_timewait_sock *tw) { } +static inline void +psp_enqueue_set_decrypted(struct sock *sk, struct sk_buff *skb) +{ +} + +static inline unsigned long +__psp_skb_coalesce_diff(const struct sk_buff *one, const struct sk_buff *two, + unsigned long diffs) +{ + return diffs; +} + +static inline enum skb_drop_reason +psp_sk_rx_policy_check(struct sock *sk, struct sk_buff *skb) +{ + return 0; +} + +static inline enum skb_drop_reason +psp_twsk_rx_policy_check(struct tcp_timewait_sock *tw, struct sk_buff *skb) +{ + return 0; +} + +static inline struct psp_assoc *psp_skb_get_assoc_rcu(struct sk_buff *skb) +{ + return NULL; +} +#else +static inline void psp_sk_assoc_free(struct sock *sk) { } +static inline void +psp_twsk_init(struct tcp_timewait_sock *tw, struct sock *sk) { } +static inline void psp_twsk_assoc_free(struct tcp_timewait_sock *tw) { } + +static inline void +psp_enqueue_set_decrypted(struct sock *sk, struct sk_buff *skb) { } + +static inline unsigned long +__psp_skb_coalesce_diff(const struct sk_buff *one, const struct sk_buff *two, + unsigned long diffs) +{ + return diffs; +} + +static inline enum skb_drop_reason +psp_sk_rx_policy_check(struct sock *sk, struct sk_buff *skb) +{ + return 0; +} + +static inline enum skb_drop_reason +psp_twsk_rx_policy_check(struct tcp_timewait_sock *tw, struct sk_buff *skb) +{ + return 0; +} + +static inline struct psp_assoc *psp_skb_get_assoc_rcu(struct sk_buff *skb) +{ + return NULL; +} +#endif + +static inline unsigned long +psp_skb_coalesce_diff(const struct sk_buff *one, const struct sk_buff *two) +{ + return __psp_skb_coalesce_diff(one, two, 0); +} + #endif /* __NET_PSP_HELPERS_H */ diff --git a/net/core/gro.c b/net/core/gro.c index e2f84947fb74..56c6c0e43a11 100644 --- a/net/core/gro.c +++ b/net/core/gro.c @@ -1,4 +1,5 @@ // SPDX-License-Identifier: GPL-2.0-or-later +#include #include #include #include @@ -387,6 +388,7 @@ static void gro_list_prepare(const struct list_head *head, diffs |= skb_get_nfct(p) ^ skb_get_nfct(skb); diffs |= gro_list_prepare_tc_ext(skb, p, diffs); + diffs |= __psp_skb_coalesce_diff(skb, p, diffs); } NAPI_GRO_CB(p)->same_flow = !diffs; diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 06aab937d60a..b81447bcb884 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -275,6 +275,7 @@ #include #include #include +#include #include #include @@ -674,6 +675,7 @@ void tcp_skb_entail(struct sock *sk, struct sk_buff *skb) tcb->seq = tcb->end_seq = tp->write_seq; tcb->tcp_flags = TCPHDR_ACK; __skb_header_release(skb); + psp_enqueue_set_decrypted(sk, skb); tcp_add_write_queue_tail(sk, skb); sk_wmem_queued_add(sk, skb->truesize); sk_mem_charge(sk, skb->truesize); diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 95e3d28b83b8..9539c4a7b55d 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -71,6 +71,7 @@ #include #include #include +#include #include #include @@ -1895,6 +1896,10 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb) enum skb_drop_reason reason; struct sock *rsk; + reason = psp_sk_rx_policy_check(sk, skb); + if (reason) + goto err_discard; + if (sk->sk_state == TCP_ESTABLISHED) { /* Fast path */ struct dst_entry *dst; @@ -1956,6 +1961,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb) reason = SKB_DROP_REASON_TCP_CSUM; trace_tcp_bad_csum(skb); TCP_INC_STATS(sock_net(sk), TCP_MIB_CSUMERRORS); +err_discard: TCP_INC_STATS(sock_net(sk), TCP_MIB_INERRS); goto discard; } @@ -2057,7 +2063,8 @@ bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb, !mptcp_skb_can_collapse(tail, skb) || skb_cmp_decrypted(tail, skb) || thtail->doff != th->doff || - memcmp(thtail + 1, th + 1, hdrlen - sizeof(*th))) + memcmp(thtail + 1, th + 1, hdrlen - sizeof(*th)) || + psp_skb_coalesce_diff(tail, skb)) goto no_coalesce; __skb_pull(skb, hdrlen); diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c index 660e890f3c74..501b8bc8573e 100644 --- a/net/ipv4/tcp_minisocks.c +++ b/net/ipv4/tcp_minisocks.c @@ -101,8 +101,11 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb, { struct tcp_options_received tmp_opt; struct tcp_timewait_sock *tcptw = tcp_twsk((struct sock *)tw); + enum skb_drop_reason psp_drop; bool paws_reject = false; + psp_drop = psp_twsk_rx_policy_check(tcptw, skb); + tmp_opt.saw_tstamp = 0; if (th->doff > (sizeof(*th) >> 2) && tcptw->tw_ts_recent_stamp) { tcp_parse_options(twsk_net(tw), skb, &tmp_opt, 0, NULL); @@ -119,6 +122,9 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb, if (tw->tw_substate == TCP_FIN_WAIT2) { /* Just repeat all the checks of tcp_rcv_state_process() */ + if (psp_drop) + goto out_put; + /* Out of window, send ACK */ if (paws_reject || !tcp_in_window(TCP_SKB_CB(skb)->seq, TCP_SKB_CB(skb)->end_seq, @@ -183,6 +189,9 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb, (TCP_SKB_CB(skb)->seq == TCP_SKB_CB(skb)->end_seq || th->rst))) { /* In window segment, it may be only reset or bare ack. */ + if (psp_drop) + goto out_put; + if (th->rst) { /* This is TIME_WAIT assassination, in two flavors. * Oh well... nobody has a sufficient solution to this @@ -234,6 +243,9 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb, return TCP_TW_SYN; } + if (psp_drop) + goto out_put; + if (paws_reject) __NET_INC_STATS(twsk_net(tw), LINUX_MIB_PAWSESTABREJECTED); @@ -250,6 +262,8 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb, return tcp_timewait_check_oow_rate_limit( tw, skb, LINUX_MIB_TCPACKSKIPPEDTIMEWAIT); } + +out_put: inet_twsk_put(tw); return TCP_TW_SUCCESS; } @@ -332,6 +346,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo) tcp_time_wait_init(sk, tcptw); tcp_ao_time_wait(tcptw, tp); + psp_twsk_init(tcptw, sk); /* Get the TIME_WAIT timeout firing. */ if (timeo < rto) diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 95caf8aaa8be..90d0a1d759ae 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include @@ -400,13 +401,15 @@ static void tcp_ecn_send(struct sock *sk, struct sk_buff *skb, /* Constructs common control bits of non-data skb. If SYN/FIN is present, * auto increment end seqno. */ -static void tcp_init_nondata_skb(struct sk_buff *skb, u32 seq, u8 flags) +static void tcp_init_nondata_skb(struct sk_buff *skb, struct sock *sk, + u32 seq, u8 flags) { skb->ip_summed = CHECKSUM_PARTIAL; TCP_SKB_CB(skb)->tcp_flags = flags; tcp_skb_pcount_set(skb, 1); + psp_enqueue_set_decrypted(sk, skb); TCP_SKB_CB(skb)->seq = seq; if (flags & (TCPHDR_SYN | TCPHDR_FIN)) @@ -1497,6 +1500,7 @@ static void tcp_queue_skb(struct sock *sk, struct sk_buff *skb) /* Advance write_seq and place onto the write_queue. */ WRITE_ONCE(tp->write_seq, TCP_SKB_CB(skb)->end_seq); __skb_header_release(skb); + psp_enqueue_set_decrypted(sk, skb); tcp_add_write_queue_tail(sk, skb); sk_wmem_queued_add(sk, skb->truesize); sk_mem_charge(sk, skb->truesize); @@ -3611,7 +3615,7 @@ void tcp_send_fin(struct sock *sk) skb_reserve(skb, MAX_TCP_HEADER); sk_forced_mem_schedule(sk, skb->truesize); /* FIN eats a sequence byte, write_seq advanced by tcp_queue_skb(). */ - tcp_init_nondata_skb(skb, tp->write_seq, + tcp_init_nondata_skb(skb, sk, tp->write_seq, TCPHDR_ACK | TCPHDR_FIN); tcp_queue_skb(sk, skb); } @@ -3639,7 +3643,7 @@ void tcp_send_active_reset(struct sock *sk, gfp_t priority, /* Reserve space for headers and prepare control bits. */ skb_reserve(skb, MAX_TCP_HEADER); - tcp_init_nondata_skb(skb, tcp_acceptable_seq(sk), + tcp_init_nondata_skb(skb, sk, tcp_acceptable_seq(sk), TCPHDR_ACK | TCPHDR_RST); tcp_mstamp_refresh(tcp_sk(sk)); /* Send it off. */ @@ -4129,7 +4133,7 @@ int tcp_connect(struct sock *sk) if (unlikely(!buff)) return -ENOBUFS; - tcp_init_nondata_skb(buff, tp->write_seq++, TCPHDR_SYN); + tcp_init_nondata_skb(buff, sk, tp->write_seq++, TCPHDR_SYN); tcp_mstamp_refresh(tp); tp->retrans_stamp = tcp_time_stamp_ts(tp); tcp_connect_queue_skb(sk, buff); @@ -4261,7 +4265,7 @@ void __tcp_send_ack(struct sock *sk, u32 rcv_nxt) /* Reserve space for headers and prepare control bits. */ skb_reserve(buff, MAX_TCP_HEADER); - tcp_init_nondata_skb(buff, tcp_acceptable_seq(sk), TCPHDR_ACK); + tcp_init_nondata_skb(buff, sk, tcp_acceptable_seq(sk), TCPHDR_ACK); /* We do not want pure acks influencing TCP Small Queues or fq/pacing * too much. @@ -4307,7 +4311,7 @@ static int tcp_xmit_probe_skb(struct sock *sk, int urgent, int mib) * end to send an ack. Don't queue or clone SKB, just * send it. */ - tcp_init_nondata_skb(skb, tp->snd_una - !urgent, TCPHDR_ACK); + tcp_init_nondata_skb(skb, sk, tp->snd_una - !urgent, TCPHDR_ACK); NET_INC_STATS(sock_net(sk), mib); return tcp_transmit_skb(sk, skb, 0, (__force gfp_t)0); } diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c index 37201c4fb393..6991464511c3 100644 --- a/net/ipv6/tcp_ipv6.c +++ b/net/ipv6/tcp_ipv6.c @@ -61,6 +61,7 @@ #include #include #include +#include #include #include @@ -1607,6 +1608,10 @@ int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb) if (skb->protocol == htons(ETH_P_IP)) return tcp_v4_do_rcv(sk, skb); + reason = psp_sk_rx_policy_check(sk, skb); + if (reason) + goto err_discard; + /* * socket locking is here for SMP purposes as backlog rcv * is currently called with bh processing disabled. @@ -1688,6 +1693,7 @@ int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb) reason = SKB_DROP_REASON_TCP_CSUM; trace_tcp_bad_csum(skb); TCP_INC_STATS(sock_net(sk), TCP_MIB_CSUMERRORS); +err_discard: TCP_INC_STATS(sock_net(sk), TCP_MIB_INERRS); goto discard; @@ -1992,6 +1998,10 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb) __this_cpu_write(tcp_tw_isn, isn); goto process; } + + drop_reason = psp_twsk_rx_policy_check(tcp_twsk(sk), skb); + if (drop_reason) + break; } /* to ACK */ fallthrough; diff --git a/net/psp/Kconfig b/net/psp/Kconfig index 55f9dd87446b..5e3908a40945 100644 --- a/net/psp/Kconfig +++ b/net/psp/Kconfig @@ -5,6 +5,7 @@ config INET_PSP bool "PSP Security Protocol support" depends on INET + select SKB_DECRYPTED help Enable kernel support for the PSP protocol. For more information see: From patchwork Fri May 10 03:04:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660837 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D464149C69 for ; Fri, 10 May 2024 03:04:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310288; cv=none; b=fXokJwl4uGAPEUABMTRrJfLS8c14/fbaB8jxYzVZYGGXp6Cwie4Po0IZZJAVo5IUvO9KTXIzBtv5bkMx1JqbyyywDPedea5cu5meuy8PKy4ZqGpRLYjioUS2e2+EK9XUfJdcYTOZnnrjTgDjkWy1V5VgV0X6sgfnXPdG21Ea+NY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310288; c=relaxed/simple; bh=e0JsmvJt93FAvmG6GMhINWvfi9/aEVwM5EgwCvA417E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bf4Mdg5HzTy7mjIbaqUNprl6OP4SgmjF7A3iL4og8bdanjurCw4+hRQAV7/jYIhkmuQKKijkNO6l/0x6IHDBLDzwWLqRSpbBztuDgmE6QzEUZztEKIGKhISvQ/d2IY7RZgcNUwGSw5ai/EQ79PstXUK4vcQ1rpgh0AkIDoBJh4M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JSMbivSK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JSMbivSK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E2CFC2BBFC; Fri, 10 May 2024 03:04:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310288; bh=e0JsmvJt93FAvmG6GMhINWvfi9/aEVwM5EgwCvA417E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JSMbivSKw0OaGMwXRac2gLVt4Ii9ou2NVCftS//vMmgz9g8jbfz4nShj1mBtfd5by sOvcM186osbiXK7Gan5QMyv+L95uKf/LbUTZ/dLYLeO/MASHKAdCx1zHMKp+NmJ9FE vSnjld9nUcEqTsygSqdI/9cYq0io0gp2fPqYgmyMexDbxzubu4x8ACqN7mxATZGoGR MsSwIxAkGeu0WO35BQ4+358Fuq3mcpyVz2SdL3yn3e2qcA+G2Wrvh6emdpHplTgug+ WGnWInRbV5vSGoi8Nc3lWi6ZP0ZZlgA64T5K6qE3egoxkPM3B7FFvtgBYvbRsUHb3F 8qFiLXvqDURhA== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Jakub Kicinski Subject: [RFC net-next 05/15] psp: add op for rotation of secret state Date: Thu, 9 May 2024 20:04:25 -0700 Message-ID: <20240510030435.120935-6-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Rotating the secret state is a key part of the PSP protocol design. Some external daemon needs to do it once a day, or so. Add a netlink op to perform this operation. Add a notification group for informing users that key has been rotated and they should rekey (next rotation will cut them off). Signed-off-by: Jakub Kicinski --- Documentation/netlink/specs/psp.yaml | 21 +++++++++++++++ include/net/psp/types.h | 5 ++++ include/uapi/linux/psp.h | 3 +++ net/psp/psp-nl-gen.c | 15 +++++++++++ net/psp/psp-nl-gen.h | 2 ++ net/psp/psp_main.c | 3 ++- net/psp/psp_nl.c | 40 ++++++++++++++++++++++++++++ 7 files changed, 88 insertions(+), 1 deletion(-) diff --git a/Documentation/netlink/specs/psp.yaml b/Documentation/netlink/specs/psp.yaml index dbb5ef148045..6ae598a60b3f 100644 --- a/Documentation/netlink/specs/psp.yaml +++ b/Documentation/netlink/specs/psp.yaml @@ -88,7 +88,28 @@ name: psp notify: dev-get mcgrp: mgmt + - + name: key-rotate + doc: Rotate the secret state. + attribute-set: dev + do: + request: + attributes: + - id + reply: + attributes: + - id + pre: psp-device-get-locked + post: psp-device-unlock + - + name: key-rotate-ntf + doc: Notification about secret state getting rotated. + notify: key-rotate + mcgrp: use + mcast-groups: list: - name: mgmt + - + name: use diff --git a/include/net/psp/types.h b/include/net/psp/types.h index a23d9bd9ce96..43950caf70f1 100644 --- a/include/net/psp/types.h +++ b/include/net/psp/types.h @@ -104,6 +104,11 @@ struct psp_dev_ops { */ int (*set_config)(struct psp_dev *psd, struct psp_dev_config *conf, struct netlink_ext_ack *extack); + + /** + * @key_rotate: rotate the secret state + */ + int (*key_rotate)(struct psp_dev *psd, struct netlink_ext_ack *extack); }; #endif /* __NET_PSP_H */ diff --git a/include/uapi/linux/psp.h b/include/uapi/linux/psp.h index 4a404f085190..cbfbf3f0f364 100644 --- a/include/uapi/linux/psp.h +++ b/include/uapi/linux/psp.h @@ -32,11 +32,14 @@ enum { PSP_CMD_DEV_DEL_NTF, PSP_CMD_DEV_SET, PSP_CMD_DEV_CHANGE_NTF, + PSP_CMD_KEY_ROTATE, + PSP_CMD_KEY_ROTATE_NTF, __PSP_CMD_MAX, PSP_CMD_MAX = (__PSP_CMD_MAX - 1) }; #define PSP_MCGRP_MGMT "mgmt" +#define PSP_MCGRP_USE "use" #endif /* _UAPI_LINUX_PSP_H */ diff --git a/net/psp/psp-nl-gen.c b/net/psp/psp-nl-gen.c index 859712e7c2c1..7f49577ac72f 100644 --- a/net/psp/psp-nl-gen.c +++ b/net/psp/psp-nl-gen.c @@ -21,6 +21,11 @@ static const struct nla_policy psp_dev_set_nl_policy[PSP_A_DEV_PSP_VERSIONS_ENA [PSP_A_DEV_PSP_VERSIONS_ENA] = NLA_POLICY_MASK(NLA_U32, 0xf), }; +/* PSP_CMD_KEY_ROTATE - do */ +static const struct nla_policy psp_key_rotate_nl_policy[PSP_A_DEV_ID + 1] = { + [PSP_A_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1), +}; + /* Ops table for psp */ static const struct genl_split_ops psp_nl_ops[] = { { @@ -46,10 +51,20 @@ static const struct genl_split_ops psp_nl_ops[] = { .maxattr = PSP_A_DEV_PSP_VERSIONS_ENA, .flags = GENL_CMD_CAP_DO, }, + { + .cmd = PSP_CMD_KEY_ROTATE, + .pre_doit = psp_device_get_locked, + .doit = psp_nl_key_rotate_doit, + .post_doit = psp_device_unlock, + .policy = psp_key_rotate_nl_policy, + .maxattr = PSP_A_DEV_ID, + .flags = GENL_CMD_CAP_DO, + }, }; static const struct genl_multicast_group psp_nl_mcgrps[] = { [PSP_NLGRP_MGMT] = { "mgmt", }, + [PSP_NLGRP_USE] = { "use", }, }; struct genl_family psp_nl_family __ro_after_init = { diff --git a/net/psp/psp-nl-gen.h b/net/psp/psp-nl-gen.h index a099686cab5d..00a2d4ec59e4 100644 --- a/net/psp/psp-nl-gen.h +++ b/net/psp/psp-nl-gen.h @@ -20,9 +20,11 @@ psp_device_unlock(const struct genl_split_ops *ops, struct sk_buff *skb, int psp_nl_dev_get_doit(struct sk_buff *skb, struct genl_info *info); int psp_nl_dev_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb); int psp_nl_dev_set_doit(struct sk_buff *skb, struct genl_info *info); +int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info); enum { PSP_NLGRP_MGMT, + PSP_NLGRP_USE, }; extern struct genl_family psp_nl_family; diff --git a/net/psp/psp_main.c b/net/psp/psp_main.c index cf463f757892..536f50d82fb2 100644 --- a/net/psp/psp_main.c +++ b/net/psp/psp_main.c @@ -54,7 +54,8 @@ psp_dev_create(struct net_device *netdev, int err; if (WARN_ON(!psd_caps->versions || - !psd_ops->set_config)) + !psd_ops->set_config || + !psd_ops->key_rotate)) return ERR_PTR(-EINVAL); psd = kzalloc(sizeof(*psd), GFP_KERNEL); diff --git a/net/psp/psp_nl.c b/net/psp/psp_nl.c index fda5ce800f82..b7006e50dc87 100644 --- a/net/psp/psp_nl.c +++ b/net/psp/psp_nl.c @@ -221,3 +221,43 @@ int psp_nl_dev_set_doit(struct sk_buff *skb, struct genl_info *info) nlmsg_free(rsp); return err; } + +int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info) +{ + struct psp_dev *psd = info->user_ptr[0]; + struct genl_info ntf_info; + struct sk_buff *ntf, *rsp; + int err; + + rsp = psp_nl_reply_new(info); + if (!rsp) + return -ENOMEM; + + genl_info_init_ntf(&ntf_info, &psp_nl_family, PSP_CMD_KEY_ROTATE); + ntf = psp_nl_reply_new(&ntf_info); + if (!ntf) { + err = -ENOMEM; + goto err_free_rsp; + } + + if (nla_put_u32(rsp, PSP_A_DEV_ID, psd->id) || + nla_put_u32(ntf, PSP_A_DEV_ID, psd->id)) { + err = -EMSGSIZE; + goto err_free_ntf; + } + + err = psd->ops->key_rotate(psd, info->extack); + if (err) + goto err_free_ntf; + + nlmsg_end(ntf, (struct nlmsghdr *)ntf->data); + genlmsg_multicast_netns(&psp_nl_family, dev_net(psd->main_netdev), ntf, + 0, PSP_NLGRP_USE, GFP_KERNEL); + return psp_nl_reply_send(rsp, info); + +err_free_ntf: + nlmsg_free(ntf); +err_free_rsp: + nlmsg_free(rsp); + return err; +} From patchwork Fri May 10 03:04:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660838 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05D6D149DE6 for ; Fri, 10 May 2024 03:04:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310289; cv=none; b=Htp8gN4SG0/XfNB2i3pfRrs7hg7eYy5NBRANwhaUk97inc0mVLN7uxMSyuO3Fw7GRX86AjqTb/0zMf1G1MZNVT0IQeQSwDcDacDRjGtPHOeM5rxZf8/JNYeHv8ywRlTcaolPcdH1cXrUqfGLouorAQlh+WXcEXOrdYxxyves1NQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310289; c=relaxed/simple; bh=RuKLxb3Y6oarj0HN/kjKDoGKGa0lwH6zpcHkkrbcooI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QgTrmjP6fZGkYy3bOUgmgKB0hiOg6/mbsNoClLcFf1CBbnbRtZPzX5k75SkE68kBt+cF+EAo55qyS9e54OXJbBQElZOZOjXnTTjVSsdZYpI084R95kJk2cPAR5PDcnafMh9UgE3gS1cHeuPFV0N4GCSkPotmIvAOsD41xZcNjyo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LJ4BVLl2; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LJ4BVLl2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2E109C32781; Fri, 10 May 2024 03:04:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310288; bh=RuKLxb3Y6oarj0HN/kjKDoGKGa0lwH6zpcHkkrbcooI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LJ4BVLl2oOUD7HHhJIeipY7VjSf9JrWSSt/A3KRllcQzjd1pUQjm/1UPgO908nYnl tJ4dCUSMQQ9x2Xf7bHxLiy4Z++Ox08TsPWr7sAhFE/kChquvLqRj37zbTiam6Xswo0 y6XvACszjiVcRR/ivPRrfNQbdw0wtc973SqxX0QKA0yOpZhsviDKImRKmttUVUeQIw boKHNyMJHJltnPAJt79D5iEw4l549T7YSWsai3w/K3IltXc022PMQTpneteCZVQgAH O5z8FNct36amQ6g2AU8RkgWFrl1pf4rX0qDZS3hu3T5ortdCBpubzUpeV0EBTA+nTx 2EQZh3aMITPOw== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Jakub Kicinski Subject: [RFC net-next 06/15] net: psp: add socket security association code Date: Thu, 9 May 2024 20:04:26 -0700 Message-ID: <20240510030435.120935-7-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Add the ability to install PSP Rx and Tx crypto keys on TCP connections. Netlink ops are provided for both operations. Rx side combines allocating a new Rx key and installing it on the socket. Theoretically these are separate actions, but in practice they will always be used one after the other. We can add distinct "alloc" and "install" ops later. Signed-off-by: Jakub Kicinski --- Documentation/netlink/specs/psp.yaml | 71 ++++++++ include/net/psp/functions.h | 69 ++++++-- include/net/psp/types.h | 55 ++++++ include/uapi/linux/psp.h | 21 +++ net/psp/Kconfig | 1 + net/psp/Makefile | 2 +- net/psp/psp-nl-gen.c | 39 ++++ net/psp/psp-nl-gen.h | 7 + net/psp/psp.h | 22 +++ net/psp/psp_main.c | 11 +- net/psp/psp_nl.c | 244 +++++++++++++++++++++++++ net/psp/psp_sock.c | 255 +++++++++++++++++++++++++++ 12 files changed, 783 insertions(+), 14 deletions(-) create mode 100644 net/psp/psp_sock.c diff --git a/Documentation/netlink/specs/psp.yaml b/Documentation/netlink/specs/psp.yaml index 6ae598a60b3f..a81f19e7b20d 100644 --- a/Documentation/netlink/specs/psp.yaml +++ b/Documentation/netlink/specs/psp.yaml @@ -38,6 +38,44 @@ name: psp type: u32 enum: version enum-as-flags: true + - + name: assoc + attributes: + - + name: dev-id + doc: PSP device ID. + type: u32 + checks: + min: 1 + - + name: version + doc: | + PSP versions (AEAD and protocol version) used by this association, + dictates the size of the key. + type: u32 + enum: version + - + name: rx-key + type: nest + nested-attributes: keys + - + name: tx-key + type: nest + nested-attributes: keys + - + name: sock-fd + doc: Sockets which should be bound to the association immediately. + type: u32 + - + name: keys + attributes: + - + name: key + type: binary + - + name: spi + doc: Security Parameters Index (SPI) of the association. + type: u32 operations: list: @@ -107,6 +145,39 @@ name: psp notify: key-rotate mcgrp: use + - + name: rx-assoc + doc: Allocate a new Rx key + SPI pair, associate it with a socket. + attribute-set: assoc + do: + request: + attributes: + - dev-id + - version + - sock-fd + reply: + attributes: + - dev-id + - version + - rx-key + pre: psp-assoc-device-get-locked + post: psp-device-unlock + - + name: tx-assoc + doc: Add a PSP Tx association. + attribute-set: assoc + do: + request: + attributes: + - dev-id + - version + - tx-key + - sock-fd + reply: + attributes: [] + pre: psp-assoc-device-get-locked + post: psp-device-unlock + mcast-groups: list: - diff --git a/include/net/psp/functions.h b/include/net/psp/functions.h index 7a3b897ecc69..ad81322dd4ed 100644 --- a/include/net/psp/functions.h +++ b/include/net/psp/functions.h @@ -4,6 +4,7 @@ #define __NET_PSP_HELPERS_H #include +#include #include #include @@ -16,39 +17,78 @@ psp_dev_create(struct net_device *netdev, struct psp_dev_ops *psd_ops, void psp_dev_unregister(struct psp_dev *psd); /* Kernel-facing API */ +void psp_assoc_put(struct psp_assoc *pas); + +static inline void *psp_assoc_drv_data(struct psp_assoc *pas) +{ + return pas->drv_data; +} + #if IS_ENABLED(CONFIG_INET_PSP) -static inline void psp_sk_assoc_free(struct sock *sk) { } -static inline void -psp_twsk_init(struct tcp_timewait_sock *tw, struct sock *sk) { } -static inline void psp_twsk_assoc_free(struct tcp_timewait_sock *tw) { } +void psp_sk_assoc_free(struct sock *sk); +void psp_twsk_init(struct tcp_timewait_sock *tw, struct sock *sk); +void psp_twsk_assoc_free(struct tcp_timewait_sock *tw); +enum skb_drop_reason +psp_twsk_rx_policy_check(struct tcp_timewait_sock *tw, struct sk_buff *skb); + +static inline struct psp_assoc *psp_sk_assoc(struct sock *sk) +{ + return rcu_dereference_check(sk->psp_assoc, lockdep_sock_is_held(sk)); +} static inline void psp_enqueue_set_decrypted(struct sock *sk, struct sk_buff *skb) { + struct psp_assoc *pas; + + pas = psp_sk_assoc(sk); + if (pas && pas->tx.spi) + skb->decrypted = 1; } static inline unsigned long __psp_skb_coalesce_diff(const struct sk_buff *one, const struct sk_buff *two, unsigned long diffs) { + struct psp_skb_ext *a, *b; + + a = skb_ext_find(one, SKB_EXT_PSP); + b = skb_ext_find(two, SKB_EXT_PSP); + + diffs |= (!!a) ^ (!!b); + if (!diffs && unlikely(a)) + diffs |= memcmp(a, b, sizeof(*a)); return diffs; } +static inline enum skb_drop_reason +__psp_sk_rx_policy_check(struct psp_skb_ext *pse, struct psp_assoc *pas) +{ + if (!pse) { + if (pas && READ_ONCE(pas->rx_required)) + return SKB_DROP_REASON_PSP_INPUT; + return 0; + } + + if (pas && pas->rx.spi == pse->spi && + pas->generation == pse->generation && + pas->version == pse->version) + return 0; + return SKB_DROP_REASON_PSP_INPUT; +} + static inline enum skb_drop_reason psp_sk_rx_policy_check(struct sock *sk, struct sk_buff *skb) { - return 0; -} - -static inline enum skb_drop_reason -psp_twsk_rx_policy_check(struct tcp_timewait_sock *tw, struct sk_buff *skb) -{ - return 0; + return __psp_sk_rx_policy_check(skb_ext_find(skb, SKB_EXT_PSP), + psp_sk_assoc(sk)); } static inline struct psp_assoc *psp_skb_get_assoc_rcu(struct sk_buff *skb) { - return NULL; + if (!skb->decrypted || !skb->sk || !sk_fullsock(skb->sk)) + return NULL; + return rcu_dereference(skb->sk->psp_assoc); } #else static inline void psp_sk_assoc_free(struct sock *sk) { } @@ -56,6 +96,11 @@ static inline void psp_twsk_init(struct tcp_timewait_sock *tw, struct sock *sk) { } static inline void psp_twsk_assoc_free(struct tcp_timewait_sock *tw) { } +static inline struct psp_assoc *psp_sk_assoc(struct sock *sk) +{ + return NULL; +} + static inline void psp_enqueue_set_decrypted(struct sock *sk, struct sk_buff *skb) { } diff --git a/include/net/psp/types.h b/include/net/psp/types.h index 43950caf70f1..e39abf10c03c 100644 --- a/include/net/psp/types.h +++ b/include/net/psp/types.h @@ -51,6 +51,7 @@ struct psp_dev_config { * @refcnt: reference count for the instance * @id: instance id * @config: current device configuration + * @active_assocs: list of registered associations * * @rcu: RCU head for freeing the structure */ @@ -68,6 +69,8 @@ struct psp_dev { struct psp_dev_config config; + struct list_head active_assocs; + struct rcu_head rcu; }; @@ -80,6 +83,12 @@ struct psp_dev_caps { * Set this field to 0 to indicate PSP is not supported at all. */ u32 versions; + + /** + * @assoc_drv_spc: size of driver-specific state in Tx assoc + * Determines the size of struct psp_assoc::drv_spc + */ + u32 assoc_drv_spc; }; #define PSP_V0_KEY 16 @@ -93,6 +102,30 @@ struct psp_skb_ext { u16 version; }; +struct psp_key_parsed { + __be32 spi; + u8 key[PSP_MAX_KEY]; +}; + +struct psp_assoc { + struct psp_dev *psd; + + u8 generation; + u8 version; + u8 key_sz; + u8 rx_required; + + struct psp_key_parsed tx; + struct psp_key_parsed rx; + + refcount_t refcnt; + struct rcu_head rcu; + struct work_struct work; + struct list_head assocs_list; + + u8 drv_data[] __aligned(8); +}; + /** * struct psp_dev_ops - netdev driver facing PSP callbacks */ @@ -109,6 +142,28 @@ struct psp_dev_ops { * @key_rotate: rotate the secret state */ int (*key_rotate)(struct psp_dev *psd, struct netlink_ext_ack *extack); + + /** + * @rx_spi_alloc: allocate an Rx SPI+key pair + * Allocate an Rx SPI and resulting derived key. + * This key should remain valid until key rotation. + */ + int (*rx_spi_alloc)(struct psp_dev *psd, u32 version, + struct psp_key_parsed *assoc, + struct netlink_ext_ack *extack); + + /** + * @tx_key_add: add a Tx key to the device + * Install an association in the device. Core will allocate space + * for the driver to use at drv_data. + */ + int (*tx_key_add)(struct psp_dev *psd, struct psp_assoc *pas, + struct netlink_ext_ack *extack); + /** + * @tx_key_del: remove a Tx key from the device + * Remove an association from the device. + */ + void (*tx_key_del)(struct psp_dev *psd, struct psp_assoc *pas); }; #endif /* __NET_PSP_H */ diff --git a/include/uapi/linux/psp.h b/include/uapi/linux/psp.h index cbfbf3f0f364..607c42c39ba5 100644 --- a/include/uapi/linux/psp.h +++ b/include/uapi/linux/psp.h @@ -26,6 +26,25 @@ enum { PSP_A_DEV_MAX = (__PSP_A_DEV_MAX - 1) }; +enum { + PSP_A_ASSOC_DEV_ID = 1, + PSP_A_ASSOC_VERSION, + PSP_A_ASSOC_RX_KEY, + PSP_A_ASSOC_TX_KEY, + PSP_A_ASSOC_SOCK_FD, + + __PSP_A_ASSOC_MAX, + PSP_A_ASSOC_MAX = (__PSP_A_ASSOC_MAX - 1) +}; + +enum { + PSP_A_KEYS_KEY = 1, + PSP_A_KEYS_SPI, + + __PSP_A_KEYS_MAX, + PSP_A_KEYS_MAX = (__PSP_A_KEYS_MAX - 1) +}; + enum { PSP_CMD_DEV_GET = 1, PSP_CMD_DEV_ADD_NTF, @@ -34,6 +53,8 @@ enum { PSP_CMD_DEV_CHANGE_NTF, PSP_CMD_KEY_ROTATE, PSP_CMD_KEY_ROTATE_NTF, + PSP_CMD_RX_ASSOC, + PSP_CMD_TX_ASSOC, __PSP_CMD_MAX, PSP_CMD_MAX = (__PSP_CMD_MAX - 1) diff --git a/net/psp/Kconfig b/net/psp/Kconfig index 5e3908a40945..a7d24691a7e1 100644 --- a/net/psp/Kconfig +++ b/net/psp/Kconfig @@ -6,6 +6,7 @@ config INET_PSP bool "PSP Security Protocol support" depends on INET select SKB_DECRYPTED + select SOCK_VALIDATE_XMIT help Enable kernel support for the PSP protocol. For more information see: diff --git a/net/psp/Makefile b/net/psp/Makefile index 41b51d06e560..eb5ff3c5bfb2 100644 --- a/net/psp/Makefile +++ b/net/psp/Makefile @@ -2,4 +2,4 @@ obj-$(CONFIG_INET_PSP) += psp.o -psp-y := psp_main.o psp_nl.o psp-nl-gen.o +psp-y := psp_main.o psp_nl.o psp_sock.o psp-nl-gen.o diff --git a/net/psp/psp-nl-gen.c b/net/psp/psp-nl-gen.c index 7f49577ac72f..9fdd6f831803 100644 --- a/net/psp/psp-nl-gen.c +++ b/net/psp/psp-nl-gen.c @@ -10,6 +10,12 @@ #include +/* Common nested types */ +const struct nla_policy psp_keys_nl_policy[PSP_A_KEYS_SPI + 1] = { + [PSP_A_KEYS_KEY] = { .type = NLA_BINARY, }, + [PSP_A_KEYS_SPI] = { .type = NLA_U32, }, +}; + /* PSP_CMD_DEV_GET - do */ static const struct nla_policy psp_dev_get_nl_policy[PSP_A_DEV_ID + 1] = { [PSP_A_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1), @@ -26,6 +32,21 @@ static const struct nla_policy psp_key_rotate_nl_policy[PSP_A_DEV_ID + 1] = { [PSP_A_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1), }; +/* PSP_CMD_RX_ASSOC - do */ +static const struct nla_policy psp_rx_assoc_nl_policy[PSP_A_ASSOC_SOCK_FD + 1] = { + [PSP_A_ASSOC_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1), + [PSP_A_ASSOC_VERSION] = NLA_POLICY_MAX(NLA_U32, 3), + [PSP_A_ASSOC_SOCK_FD] = { .type = NLA_U32, }, +}; + +/* PSP_CMD_TX_ASSOC - do */ +static const struct nla_policy psp_tx_assoc_nl_policy[PSP_A_ASSOC_SOCK_FD + 1] = { + [PSP_A_ASSOC_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1), + [PSP_A_ASSOC_VERSION] = NLA_POLICY_MAX(NLA_U32, 3), + [PSP_A_ASSOC_TX_KEY] = NLA_POLICY_NESTED(psp_keys_nl_policy), + [PSP_A_ASSOC_SOCK_FD] = { .type = NLA_U32, }, +}; + /* Ops table for psp */ static const struct genl_split_ops psp_nl_ops[] = { { @@ -60,6 +81,24 @@ static const struct genl_split_ops psp_nl_ops[] = { .maxattr = PSP_A_DEV_ID, .flags = GENL_CMD_CAP_DO, }, + { + .cmd = PSP_CMD_RX_ASSOC, + .pre_doit = psp_assoc_device_get_locked, + .doit = psp_nl_rx_assoc_doit, + .post_doit = psp_device_unlock, + .policy = psp_rx_assoc_nl_policy, + .maxattr = PSP_A_ASSOC_SOCK_FD, + .flags = GENL_CMD_CAP_DO, + }, + { + .cmd = PSP_CMD_TX_ASSOC, + .pre_doit = psp_assoc_device_get_locked, + .doit = psp_nl_tx_assoc_doit, + .post_doit = psp_device_unlock, + .policy = psp_tx_assoc_nl_policy, + .maxattr = PSP_A_ASSOC_SOCK_FD, + .flags = GENL_CMD_CAP_DO, + }, }; static const struct genl_multicast_group psp_nl_mcgrps[] = { diff --git a/net/psp/psp-nl-gen.h b/net/psp/psp-nl-gen.h index 00a2d4ec59e4..25268ed11fb5 100644 --- a/net/psp/psp-nl-gen.h +++ b/net/psp/psp-nl-gen.h @@ -11,8 +11,13 @@ #include +/* Common nested types */ +extern const struct nla_policy psp_keys_nl_policy[PSP_A_KEYS_SPI + 1]; + int psp_device_get_locked(const struct genl_split_ops *ops, struct sk_buff *skb, struct genl_info *info); +int psp_assoc_device_get_locked(const struct genl_split_ops *ops, + struct sk_buff *skb, struct genl_info *info); void psp_device_unlock(const struct genl_split_ops *ops, struct sk_buff *skb, struct genl_info *info); @@ -21,6 +26,8 @@ int psp_nl_dev_get_doit(struct sk_buff *skb, struct genl_info *info); int psp_nl_dev_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb); int psp_nl_dev_set_doit(struct sk_buff *skb, struct genl_info *info); int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info); +int psp_nl_rx_assoc_doit(struct sk_buff *skb, struct genl_info *info); +int psp_nl_tx_assoc_doit(struct sk_buff *skb, struct genl_info *info); enum { PSP_NLGRP_MGMT, diff --git a/net/psp/psp.h b/net/psp/psp.h index 94d0cc31a61f..b4092936bc64 100644 --- a/net/psp/psp.h +++ b/net/psp/psp.h @@ -4,6 +4,7 @@ #define __PSP_PSP_H #include +#include #include #include #include @@ -17,15 +18,36 @@ int psp_dev_check_access(struct psp_dev *psd, struct net *net); void psp_nl_notify_dev(struct psp_dev *psd, u32 cmd); +struct psp_assoc *psp_assoc_create(struct psp_dev *psd); +struct psp_dev *psd_get_for_sock(struct sock *sk); +void psp_dev_tx_key_del(struct psp_dev *psd, struct psp_assoc *pas); +int psp_sock_assoc_set_rx(struct sock *sk, struct psp_assoc *pas, + struct psp_key_parsed *key, + struct netlink_ext_ack *extack); +int psp_sock_assoc_set_tx(struct sock *sk, struct psp_dev *psd, + u32 version, struct psp_key_parsed *key, + struct netlink_ext_ack *extack); + static inline void psp_dev_get(struct psp_dev *psd) { refcount_inc(&psd->refcnt); } +static inline bool psp_dev_tryget(struct psp_dev *psd) +{ + return refcount_inc_not_zero(&psd->refcnt); +} + static inline void psp_dev_put(struct psp_dev *psd) { if (refcount_dec_and_test(&psd->refcnt)) psp_dev_destroy(psd); } +static inline bool psp_dev_is_registered(struct psp_dev *psd) +{ + lockdep_assert_held(&psd->lock); + return !!psd->ops; +} + #endif /* __PSP_PSP_H */ diff --git a/net/psp/psp_main.c b/net/psp/psp_main.c index 536f50d82fb2..59066c4db048 100644 --- a/net/psp/psp_main.c +++ b/net/psp/psp_main.c @@ -55,7 +55,10 @@ psp_dev_create(struct net_device *netdev, if (WARN_ON(!psd_caps->versions || !psd_ops->set_config || - !psd_ops->key_rotate)) + !psd_ops->key_rotate || + !psd_ops->rx_spi_alloc || + !psd_ops->tx_key_add || + !psd_ops->tx_key_del)) return ERR_PTR(-EINVAL); psd = kzalloc(sizeof(*psd), GFP_KERNEL); @@ -68,6 +71,7 @@ psp_dev_create(struct net_device *netdev, psd->drv_priv = priv_ptr; mutex_init(&psd->lock); + INIT_LIST_HEAD(&psd->active_assocs); refcount_set(&psd->refcnt, 1); mutex_lock(&psp_devs_lock); @@ -103,6 +107,8 @@ void psp_dev_destroy(struct psp_dev *psd) */ void psp_dev_unregister(struct psp_dev *psd) { + struct psp_assoc *pas, *next; + mutex_lock(&psp_devs_lock); mutex_lock(&psd->lock); @@ -110,6 +116,9 @@ void psp_dev_unregister(struct psp_dev *psd) xa_erase(&psp_devs, psd->id); mutex_unlock(&psp_devs_lock); + list_for_each_entry_safe(pas, next, &psd->active_assocs, assocs_list) + psp_dev_tx_key_del(psd, pas); + rcu_assign_pointer(psd->main_netdev->psp_dev, NULL); psd->ops = NULL; diff --git a/net/psp/psp_nl.c b/net/psp/psp_nl.c index b7006e50dc87..58508e642185 100644 --- a/net/psp/psp_nl.c +++ b/net/psp/psp_nl.c @@ -79,9 +79,12 @@ void psp_device_unlock(const struct genl_split_ops *ops, struct sk_buff *skb, struct genl_info *info) { + struct socket *socket = info->user_ptr[1]; struct psp_dev *psd = info->user_ptr[0]; mutex_unlock(&psd->lock); + if (socket) + sockfd_put(socket); } static int @@ -261,3 +264,244 @@ int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info) nlmsg_free(rsp); return err; } + +/* Key etc. */ + +int psp_assoc_device_get_locked(const struct genl_split_ops *ops, + struct sk_buff *skb, struct genl_info *info) +{ + struct socket *socket; + struct psp_dev *psd; + struct nlattr *id; + struct sock *sk; + int fd, err; + + if (GENL_REQ_ATTR_CHECK(info, PSP_A_ASSOC_SOCK_FD)) + return -EINVAL; + + fd = nla_get_u32(info->attrs[PSP_A_ASSOC_SOCK_FD]); + socket = sockfd_lookup(fd, &err); + if (!socket) + return err; + + sk = socket->sk; + if (sk->sk_family != AF_INET && sk->sk_family != AF_INET6) { + NL_SET_ERR_MSG_ATTR(info->extack, + info->attrs[PSP_A_ASSOC_SOCK_FD], + "Unsupported socket family"); + err = -EOPNOTSUPP; + goto err_sock_put; + } + + psd = psd_get_for_sock(socket->sk); + if (psd) { + err = psp_dev_check_access(psd, genl_info_net(info)); + if (err) { + psp_dev_put(psd); + psd = NULL; + } + } + + if (!psd && GENL_REQ_ATTR_CHECK(info, PSP_A_ASSOC_DEV_ID)) { + err = -EINVAL; + goto err_sock_put; + } + + id = info->attrs[PSP_A_ASSOC_DEV_ID]; + if (psd) { + mutex_lock(&psd->lock); + if (id && psd->id != nla_get_u32(id)) { + mutex_unlock(&psd->lock); + NL_SET_ERR_MSG_ATTR(info->extack, id, + "Device id vs socket mismatch"); + err = -EINVAL; + goto err_psd_put; + } + + psp_dev_put(psd); + } else { + psd = psp_device_get_and_lock(genl_info_net(info), id); + if (IS_ERR(psd)) { + err = PTR_ERR(psd); + goto err_sock_put; + } + } + + info->user_ptr[0] = psd; + info->user_ptr[1] = socket; + + return 0; + +err_psd_put: + psp_dev_put(psd); +err_sock_put: + sockfd_put(socket); + return err; +} + +static unsigned int psp_nl_assoc_key_size(u32 version) +{ + switch (version) { + case PSP_VERSION_HDR0_AES_GCM_128: + case PSP_VERSION_HDR0_AES_GMAC_128: + return 16; + case PSP_VERSION_HDR0_AES_GCM_256: + case PSP_VERSION_HDR0_AES_GMAC_256: + return 32; + default: + /* Netlink policies should prevent us from getting here */ + WARN_ON_ONCE(1); + return 0; + } +} + +static int +psp_nl_parse_key(struct genl_info *info, u32 attr, struct psp_key_parsed *key, + unsigned int key_sz) +{ + struct nlattr *nest = info->attrs[attr]; + struct nlattr *tb[PSP_A_KEYS_SPI + 1]; + int err; + + err = nla_parse_nested(tb, ARRAY_SIZE(tb) - 1, nest, + psp_keys_nl_policy, info->extack); + if (err) + return err; + + if (NL_REQ_ATTR_CHECK(info->extack, nest, tb, PSP_A_KEYS_KEY) || + NL_REQ_ATTR_CHECK(info->extack, nest, tb, PSP_A_KEYS_SPI)) + return -EINVAL; + + if (nla_len(tb[PSP_A_KEYS_KEY]) != key_sz) { + NL_SET_ERR_MSG_ATTR(info->extack, tb[PSP_A_KEYS_KEY], + "incorrect key length"); + return -EINVAL; + } + + key->spi = cpu_to_be32(nla_get_u32(tb[PSP_A_KEYS_SPI])); + memcpy(key->key, nla_data(tb[PSP_A_KEYS_KEY]), key_sz); + + return 0; +} + +static int +psp_nl_put_key(struct sk_buff *skb, u32 attr, u32 version, + struct psp_key_parsed *key) +{ + int key_sz = psp_nl_assoc_key_size(version); + void *nest; + + nest = nla_nest_start(skb, attr); + + if (nla_put_u32(skb, PSP_A_KEYS_SPI, be32_to_cpu(key->spi)) || + nla_put(skb, PSP_A_KEYS_KEY, key_sz, key->key)) { + nla_nest_cancel(skb, nest); + return -EMSGSIZE; + } + + nla_nest_end(skb, nest); + + return 0; +} + +int psp_nl_rx_assoc_doit(struct sk_buff *skb, struct genl_info *info) +{ + struct socket *socket = info->user_ptr[1]; + struct psp_dev *psd = info->user_ptr[0]; + struct psp_key_parsed key; + struct psp_assoc *pas; + struct sk_buff *rsp; + u32 version; + int err; + + if (GENL_REQ_ATTR_CHECK(info, PSP_A_ASSOC_VERSION)) + return -EINVAL; + + version = nla_get_u32(info->attrs[PSP_A_ASSOC_VERSION]); + if (!(psd->caps->versions & (1 << version))) { + NL_SET_BAD_ATTR(info->extack, info->attrs[PSP_A_ASSOC_VERSION]); + return -EOPNOTSUPP; + } + + rsp = psp_nl_reply_new(info); + if (!rsp) + return -ENOMEM; + + pas = psp_assoc_create(psd); + if (!pas) { + err = -ENOMEM; + goto err_free_rsp; + } + pas->version = version; + pas->key_sz = psp_nl_assoc_key_size(version); + + err = psd->ops->rx_spi_alloc(psd, version, &key, info->extack); + if (err) + goto err_free_pas; + + if (nla_put_u32(rsp, PSP_A_ASSOC_DEV_ID, psd->id) || + nla_put_u32(rsp, PSP_A_ASSOC_VERSION, version) || + psp_nl_put_key(rsp, PSP_A_ASSOC_RX_KEY, version, &key)) { + err = -EMSGSIZE; + goto err_free_pas; + } + + err = psp_sock_assoc_set_rx(socket->sk, pas, &key, info->extack); + if (err) { + NL_SET_BAD_ATTR(info->extack, info->attrs[PSP_A_ASSOC_SOCK_FD]); + goto err_free_pas; + } + psp_assoc_put(pas); + + return psp_nl_reply_send(rsp, info); + +err_free_pas: + psp_assoc_put(pas); +err_free_rsp: + nlmsg_free(rsp); + return err; +} + +int psp_nl_tx_assoc_doit(struct sk_buff *skb, struct genl_info *info) +{ + struct socket *socket = info->user_ptr[1]; + struct psp_dev *psd = info->user_ptr[0]; + struct psp_key_parsed key; + struct sk_buff *rsp; + unsigned int key_sz; + u32 version; + int err; + + if (GENL_REQ_ATTR_CHECK(info, PSP_A_ASSOC_VERSION) || + GENL_REQ_ATTR_CHECK(info, PSP_A_ASSOC_TX_KEY)) + return -EINVAL; + + version = nla_get_u32(info->attrs[PSP_A_ASSOC_VERSION]); + if (!(psd->caps->versions & (1 << version))) { + NL_SET_BAD_ATTR(info->extack, info->attrs[PSP_A_ASSOC_VERSION]); + return -EOPNOTSUPP; + } + + key_sz = psp_nl_assoc_key_size(version); + if (!key_sz) + return -EINVAL; + + err = psp_nl_parse_key(info, PSP_A_ASSOC_TX_KEY, &key, key_sz); + if (err < 0) + return err; + + rsp = psp_nl_reply_new(info); + if (!rsp) + return -ENOMEM; + + err = psp_sock_assoc_set_tx(socket->sk, psd, version, &key, + info->extack); + if (err) + goto err_free_msg; + + return psp_nl_reply_send(rsp, info); + +err_free_msg: + nlmsg_free(rsp); + return err; +} diff --git a/net/psp/psp_sock.c b/net/psp/psp_sock.c new file mode 100644 index 000000000000..42b881e681b9 --- /dev/null +++ b/net/psp/psp_sock.c @@ -0,0 +1,255 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include +#include +#include + +#include +#include "psp.h" + +struct psp_dev *psd_get_for_sock(struct sock *sk) +{ + struct dst_entry *dst; + struct psp_dev *psd; + + dst = sk_dst_get(sk); + if (!dst) + return NULL; + + rcu_read_lock(); + psd = rcu_dereference(dst->dev->psp_dev); + if (psd && !psp_dev_tryget(psd)) + psd = NULL; + rcu_read_unlock(); + + dst_release(dst); + + return psd; +} + +static struct sk_buff * +psp_validate_xmit(struct sock *sk, struct net_device *dev, struct sk_buff *skb) +{ + struct psp_assoc *pas; + bool good; + + rcu_read_lock(); + pas = psp_skb_get_assoc_rcu(skb); + good = !pas || rcu_access_pointer(dev->psp_dev) == pas->psd; + rcu_read_unlock(); + if (!good) { + kfree_skb_reason(skb, SKB_DROP_REASON_PSP_OUTPUT); + return NULL; + } + + return skb; +} + +struct psp_assoc *psp_assoc_create(struct psp_dev *psd) +{ + struct psp_assoc *pas; + + lockdep_assert_held(&psd->lock); + + pas = kzalloc(struct_size(pas, drv_data, psd->caps->assoc_drv_spc), + GFP_KERNEL_ACCOUNT); + if (!pas) + return NULL; + + pas->psd = psd; + psp_dev_get(psd); + refcount_set(&pas->refcnt, 1); + + list_add_tail(&pas->assocs_list, &psd->active_assocs); + + return pas; +} + +static struct psp_assoc *psp_assoc_dummy(struct psp_assoc *pas) +{ + struct psp_dev *psd = pas->psd; + size_t sz; + + lockdep_assert_held(&psd->lock); + + sz = struct_size(pas, drv_data, psd->caps->assoc_drv_spc); + return kmemdup(pas, sz, GFP_KERNEL); +} + +static int psp_dev_tx_key_add(struct psp_dev *psd, struct psp_assoc *pas, + struct netlink_ext_ack *extack) +{ + return psd->ops->tx_key_add(psd, pas, extack); +} + +void psp_dev_tx_key_del(struct psp_dev *psd, struct psp_assoc *pas) +{ + if (pas->tx.spi) + psd->ops->tx_key_del(psd, pas); + list_del(&pas->assocs_list); +} + +static void psp_assoc_free(struct work_struct *work) +{ + struct psp_assoc *pas = container_of(work, struct psp_assoc, work); + struct psp_dev *psd = pas->psd; + + mutex_lock(&psd->lock); + if (psd->ops) + psp_dev_tx_key_del(psd, pas); + mutex_unlock(&psd->lock); + psp_dev_put(psd); + kfree(pas); +} + +static void psp_assoc_free_queue(struct rcu_head *head) +{ + struct psp_assoc *pas = container_of(head, struct psp_assoc, rcu); + + INIT_WORK(&pas->work, psp_assoc_free); + schedule_work(&pas->work); +} + +/** + * psp_assoc_put() - release a reference on a PSP association + * @pas: association to release + */ +void psp_assoc_put(struct psp_assoc *pas) +{ + if (pas && refcount_dec_and_test(&pas->refcnt)) + call_rcu(&pas->rcu, psp_assoc_free_queue); +} + +void psp_sk_assoc_free(struct sock *sk) +{ + rcu_read_lock(); + psp_assoc_put(rcu_dereference(sk->psp_assoc)); + rcu_assign_pointer(sk->psp_assoc, NULL); + rcu_read_unlock(); +} + +int psp_sock_assoc_set_rx(struct sock *sk, struct psp_assoc *pas, + struct psp_key_parsed *key, + struct netlink_ext_ack *extack) +{ + int err; + + memcpy(&pas->rx, key, sizeof(*key)); + + lock_sock(sk); + + if (psp_sk_assoc(sk)) { + NL_SET_ERR_MSG(extack, "Socket already has PSP state"); + err = -EBUSY; + goto exit_unlock; + } + + refcount_inc(&pas->refcnt); + rcu_assign_pointer(sk->psp_assoc, pas); + err = 0; + +exit_unlock: + release_sock(sk); + + return err; +} + +static int psp_sock_recv_queue_check(struct sock *sk) +{ + struct sk_buff *skb; + + skb_queue_walk(&sk->sk_receive_queue, skb) { + if (psp_sk_rx_policy_check(sk, skb)) + return -EBUSY; + } + return 0; +} + +int psp_sock_assoc_set_tx(struct sock *sk, struct psp_dev *psd, + u32 version, struct psp_key_parsed *key, + struct netlink_ext_ack *extack) +{ + struct psp_assoc *pas, *dummy; + int err; + + lock_sock(sk); + + pas = psp_sk_assoc(sk); + if (!pas) { + NL_SET_ERR_MSG(extack, "Socket has no Rx key"); + err = -EINVAL; + goto exit_unlock; + } + if (pas->psd != psd) { + NL_SET_ERR_MSG(extack, "Rx key from different device"); + err = -EINVAL; + goto exit_unlock; + } + if (pas->version != version) { + NL_SET_ERR_MSG(extack, + "PSP version mismatch with existing state"); + err = -EINVAL; + goto exit_unlock; + } + if (pas->tx.spi) { + NL_SET_ERR_MSG(extack, "Tx key already set"); + err = -EBUSY; + goto exit_unlock; + } + + WRITE_ONCE(pas->rx_required, 1); + err = psp_sock_recv_queue_check(sk); + if (err) { + NL_SET_ERR_MSG(extack, "Socket has incompatible segments already in the recv queue"); + goto exit_clear_rx; + } + + /* Pass a fake association to drivers to make sure they don't + * try to store pointers to it. For re-keying we'll need to + * re-allocate the assoc structures. + */ + dummy = psp_assoc_dummy(pas); + memcpy(&dummy->tx, key, sizeof(*key)); + err = psp_dev_tx_key_add(psd, dummy, extack); + if (err) + goto exit_free_dummy; + + memcpy(pas->drv_data, dummy->drv_data, psd->caps->assoc_drv_spc); + memcpy(&pas->tx, key, sizeof(*key)); + + WRITE_ONCE(sk->sk_validate_xmit_skb, psp_validate_xmit); + +exit_free_dummy: + kfree(dummy); +exit_clear_rx: + if (err) + WRITE_ONCE(pas->rx_required, 0); +exit_unlock: + release_sock(sk); + return err; +} + +void psp_twsk_init(struct tcp_timewait_sock *tw, struct sock *sk) +{ + struct psp_assoc *pas = psp_sk_assoc(sk); + + if (pas) + refcount_inc(&pas->refcnt); + rcu_assign_pointer(tw->psp_assoc, pas); +} + +void psp_twsk_assoc_free(struct tcp_timewait_sock *tw) +{ + rcu_read_lock(); + psp_assoc_put(rcu_dereference(tw->psp_assoc)); + rcu_assign_pointer(tw->psp_assoc, NULL); + rcu_read_unlock(); +} + +enum skb_drop_reason +psp_twsk_rx_policy_check(struct tcp_timewait_sock *tw, struct sk_buff *skb) +{ + return __psp_sk_rx_policy_check(skb_ext_find(skb, SKB_EXT_PSP), + rcu_dereference(tw->psp_assoc)); +} From patchwork Fri May 10 03:04:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660839 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A73BF149DF9 for ; Fri, 10 May 2024 03:04:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310289; cv=none; b=BB60RBjzaGLysO7fg2k7JnSlBz3BbhHpO76PDFtxgaqWoHbxYnTMx/TBaU9470iTvyBp3lt3787qiOEp+Udvnum16FvMPfOphg2J9xLWx5xeAuuBxUyroQ2qqdoNGhj7bDHGmonEM+jfSj6NT/JZy+aYlESeX0P/AJKu9VJ/tO0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310289; c=relaxed/simple; bh=qwX0S2IPTpnyMYSHB94y9FA9s2Cuw6juQRT9ywUW/uM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FKjkPgW8pF3B6qruX1XSU1agw2wAKiqihthbcCn5NwSLRAPcPS3nJaTpl51HNRYEpuIDU/EewblKCd0TUqzl4TbjUdWtojUhqhmxXf+yipTqiPGmNIBfM17x4Lu9HboAokxlxfXGPcW2sMGKAgcV9obc0eBbxAoL3h+F4qM29gU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KryGL4h4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KryGL4h4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D5338C4AF08; Fri, 10 May 2024 03:04:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310289; bh=qwX0S2IPTpnyMYSHB94y9FA9s2Cuw6juQRT9ywUW/uM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KryGL4h4EBf43dCXkP/84gSeqrEKXuPwuCWoLI8SQLRT0fsDLsctGU3F+rvgJdAwU 2Q55k0//NNSLoKClrlHha47c2mGt+Y4d14t+hek20yEULany3mTzjZsquhR9Gxeb1O GQLoP4ZUhSy4vRJ2Y3++cVjJWY4wK6Nx45nyfBYj5SHEJdr3zOle+KhEyFBW00R4NM Zn47DkCEPKKmuAN9HdX/VwGkezMELzSH3xsnhqz5NciL8ZzVY9djGqQ6bxZH616Ug8 kaV2dG1TDiFIqE7kJfGZJjk51+GhUaEQJbh+STxrNJiduezDzo3hP3uqbvTEGMr6Mo EwGPUjvNTHKqw== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Jakub Kicinski Subject: [RFC net-next 07/15] net: psp: update the TCP MSS to reflect PSP packet overhead Date: Thu, 9 May 2024 20:04:27 -0700 Message-ID: <20240510030435.120935-8-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC PSP eats 32B of header space. Adjust MSS appropriately. We can either modify tcp_mtu_to_mss() / tcp_mss_to_mtu() or reuse icsk_ext_hdr_len. The former option is more TCP specific and has runtime overhead. The latter is a bit of a hack as PSP is not an ext_hdr. If one squints hard enough, UDP encap is just a more practical version of IPv6 exthdr, so go with the latter. Happy to change. Signed-off-by: Jakub Kicinski --- include/net/psp/functions.h | 12 ++++++++++++ include/net/psp/types.h | 3 +++ net/ipv4/tcp_ipv4.c | 4 ++-- net/ipv6/ipv6_sockglue.c | 6 +++++- net/ipv6/tcp_ipv6.c | 12 ++++++------ net/psp/psp_sock.c | 5 +++++ 6 files changed, 33 insertions(+), 9 deletions(-) diff --git a/include/net/psp/functions.h b/include/net/psp/functions.h index ad81322dd4ed..be4f78dec425 100644 --- a/include/net/psp/functions.h +++ b/include/net/psp/functions.h @@ -90,6 +90,13 @@ static inline struct psp_assoc *psp_skb_get_assoc_rcu(struct sk_buff *skb) return NULL; return rcu_dereference(skb->sk->psp_assoc); } + +static inline unsigned int psp_sk_overhead(const struct sock *sk) +{ + bool has_psp = rcu_access_pointer(sk->psp_assoc); + + return has_psp ? PSP_HDR_SIZE + PSP_TRL_SIZE : 0; +} #else static inline void psp_sk_assoc_free(struct sock *sk) { } static inline void @@ -127,6 +134,11 @@ static inline struct psp_assoc *psp_skb_get_assoc_rcu(struct sk_buff *skb) { return NULL; } + +static inline unsigned int psp_sk_overhead(const struct sock *sk) +{ + return 0; +} #endif static inline unsigned long diff --git a/include/net/psp/types.h b/include/net/psp/types.h index e39abf10c03c..aad836c1c2ca 100644 --- a/include/net/psp/types.h +++ b/include/net/psp/types.h @@ -95,6 +95,9 @@ struct psp_dev_caps { #define PSP_V1_KEY 32 #define PSP_MAX_KEY 32 +#define PSP_HDR_SIZE 16 /* We don't support optional fields, yet */ +#define PSP_TRL_SIZE 16 /* AES-GCM/GMAC trailer size */ + struct psp_skb_ext { __be32 spi; /* generation and version are 8b but we don't want holes */ diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 9539c4a7b55d..2a602cf51009 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -279,9 +279,9 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) inet->inet_dport = usin->sin_port; sk_daddr_set(sk, daddr); - inet_csk(sk)->icsk_ext_hdr_len = 0; + inet_csk(sk)->icsk_ext_hdr_len = psp_sk_overhead(sk); if (inet_opt) - inet_csk(sk)->icsk_ext_hdr_len = inet_opt->opt.optlen; + inet_csk(sk)->icsk_ext_hdr_len += inet_opt->opt.optlen; tp->rx_opt.mss_clamp = TCP_MSS_DEFAULT; diff --git a/net/ipv6/ipv6_sockglue.c b/net/ipv6/ipv6_sockglue.c index d4c28ec1bc51..b4505bbb9e2c 100644 --- a/net/ipv6/ipv6_sockglue.c +++ b/net/ipv6/ipv6_sockglue.c @@ -49,6 +49,7 @@ #include #include #include +#include #include @@ -107,7 +108,10 @@ struct ipv6_txoptions *ipv6_update_options(struct sock *sk, !((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_CLOSE)) && inet_sk(sk)->inet_daddr != LOOPBACK4_IPV6) { struct inet_connection_sock *icsk = inet_csk(sk); - icsk->icsk_ext_hdr_len = opt->opt_flen + opt->opt_nflen; + + icsk->icsk_ext_hdr_len = + psp_sk_overhead(sk) + + opt->opt_flen + opt->opt_nflen; icsk->icsk_sync_mss(sk, icsk->icsk_pmtu_cookie); } } diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c index 6991464511c3..c67700fc49a1 100644 --- a/net/ipv6/tcp_ipv6.c +++ b/net/ipv6/tcp_ipv6.c @@ -299,10 +299,10 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, sk->sk_gso_type = SKB_GSO_TCPV6; ip6_dst_store(sk, dst, NULL, NULL); - icsk->icsk_ext_hdr_len = 0; + icsk->icsk_ext_hdr_len = psp_sk_overhead(sk); if (opt) - icsk->icsk_ext_hdr_len = opt->opt_flen + - opt->opt_nflen; + icsk->icsk_ext_hdr_len += opt->opt_flen + + opt->opt_nflen; tp->rx_opt.mss_clamp = IPV6_MIN_MTU - sizeof(struct tcphdr) - sizeof(struct ipv6hdr); @@ -1500,10 +1500,10 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff * opt = ipv6_dup_options(newsk, opt); RCU_INIT_POINTER(newnp->opt, opt); } - inet_csk(newsk)->icsk_ext_hdr_len = 0; + inet_csk(newsk)->icsk_ext_hdr_len = psp_sk_overhead(sk); if (opt) - inet_csk(newsk)->icsk_ext_hdr_len = opt->opt_nflen + - opt->opt_flen; + inet_csk(newsk)->icsk_ext_hdr_len += opt->opt_nflen + + opt->opt_flen; tcp_ca_openreq_child(newsk, dst); diff --git a/net/psp/psp_sock.c b/net/psp/psp_sock.c index 42b881e681b9..bcef042cb8a5 100644 --- a/net/psp/psp_sock.c +++ b/net/psp/psp_sock.c @@ -170,6 +170,7 @@ int psp_sock_assoc_set_tx(struct sock *sk, struct psp_dev *psd, u32 version, struct psp_key_parsed *key, struct netlink_ext_ack *extack) { + struct inet_connection_sock *icsk; struct psp_assoc *pas, *dummy; int err; @@ -220,6 +221,10 @@ int psp_sock_assoc_set_tx(struct sock *sk, struct psp_dev *psd, WRITE_ONCE(sk->sk_validate_xmit_skb, psp_validate_xmit); + icsk = inet_csk(sk); + icsk->icsk_ext_hdr_len += psp_sk_overhead(sk); + icsk->icsk_sync_mss(sk, icsk->icsk_pmtu_cookie); + exit_free_dummy: kfree(dummy); exit_clear_rx: From patchwork Fri May 10 03:04:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660840 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BAD4148840 for ; Fri, 10 May 2024 03:04:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310290; cv=none; b=utBuJ4QTPliZPZGVUHFCxuMDOOcODkphQb5htnrCNVVtdvWMHpYfrRqqvK+mqC0EFHuGtMWAOEgKe4uOJrLEJbG9lHmp8vPmwQkK5sgix+yE8Umym/7Iyi5oLibqTgKdqeAENu7aVnIjZbLYQSXN5jeSO5qFosUY5KmMN0npFc0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310290; c=relaxed/simple; bh=XvObPRnmyqwdGUB9ixy/MMsyTMFrwTNz3Ndw/Y6vDhA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=peY9jfdh0jQT3iK9Vd3+Jg5WsE0pg3XwkLux57LSzVRohpAjkoWzyOHqxHbEnpK1LqUFRoQzkP6NfSU/J8NUeia/63n6XbRGQ7WmhUEEiK32unYBc//TEL9RzyMsAL7JACfuRAYE0U8t2JeRyDOYtqHO88D/O9iavFvR7yyqxZE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PrDVB9d6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PrDVB9d6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E814C3277B; Fri, 10 May 2024 03:04:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310290; bh=XvObPRnmyqwdGUB9ixy/MMsyTMFrwTNz3Ndw/Y6vDhA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PrDVB9d6OC31LgBhux4hfgqqVtftoB22K11jc4G/I9EX4OgfHMJAv/DiAn8SUJQrq 0v1pzf26HzYnQDzHxK3+edGLgfC77dZcNDbiK+agP9bPQdKoQhs9SHQbXNZlDpylh1 rtne2AyJToLWK2orBlPK0YZun6mhTbcaQ604ocXVUbCBtJ9sLcQuZ1s/tek+HYWxO3 p6gLr7sGHarI+etCvBKiynNzN8E0Ey/22mB9jVddoMRbGYM27Pnas/e6bGwB0aftys c6lOXUtFaLI9ExfpPJScNcv7qosImCJNEjReDuNqecb3ox9xc8A99lDhGuceMcIUDY hvsHQvB8U/K1w== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Jakub Kicinski Subject: [RFC net-next 08/15] psp: track generations of secret state Date: Thu, 9 May 2024 20:04:28 -0700 Message-ID: <20240510030435.120935-9-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC There is a (somewhat theoretical in absence of multi-host support) possibility that another entity will rotate the key and we won't know. This may lead to accepting packets with matching SPI but which used different crypto keys than we expected. Maintain and compare "key generation" per PSP spec. Since we're tracking "key generations" more explicitly now, maintain different lists for associations from different generations. This way we can catch stale associations (the user space should listen to rotation notifications and change the keys). Drivers can "opt out" of generation tracking by setting the generation value to 0. Signed-off-by: Jakub Kicinski --- include/net/psp/types.h | 10 ++++++++++ net/psp/psp.h | 1 + net/psp/psp_main.c | 6 +++++- net/psp/psp_nl.c | 10 ++++++++++ net/psp/psp_sock.c | 16 ++++++++++++++++ 5 files changed, 42 insertions(+), 1 deletion(-) diff --git a/include/net/psp/types.h b/include/net/psp/types.h index aad836c1c2ca..a9e406c979a8 100644 --- a/include/net/psp/types.h +++ b/include/net/psp/types.h @@ -50,8 +50,12 @@ struct psp_dev_config { * @lock: instance lock, protects all fields * @refcnt: reference count for the instance * @id: instance id + * @generation: current generation of the secret state * @config: current device configuration * @active_assocs: list of registered associations + * @prev_assocs: associations which use old (but still usable) + * secret state + * @stale_assocs: associations which use a rotated out key * * @rcu: RCU head for freeing the structure */ @@ -67,13 +71,19 @@ struct psp_dev { u32 id; + u8 generation; + struct psp_dev_config config; struct list_head active_assocs; + struct list_head prev_assocs; + struct list_head stale_assocs; struct rcu_head rcu; }; +#define PSP_GEN_VALID_MASK 0x7f + /** * struct psp_dev_caps - PSP device capabilities */ diff --git a/net/psp/psp.h b/net/psp/psp.h index b4092936bc64..a511ec85e1c7 100644 --- a/net/psp/psp.h +++ b/net/psp/psp.h @@ -27,6 +27,7 @@ int psp_sock_assoc_set_rx(struct sock *sk, struct psp_assoc *pas, int psp_sock_assoc_set_tx(struct sock *sk, struct psp_dev *psd, u32 version, struct psp_key_parsed *key, struct netlink_ext_ack *extack); +void psp_assocs_key_rotated(struct psp_dev *psd); static inline void psp_dev_get(struct psp_dev *psd) { diff --git a/net/psp/psp_main.c b/net/psp/psp_main.c index 59066c4db048..f9c5ee57df78 100644 --- a/net/psp/psp_main.c +++ b/net/psp/psp_main.c @@ -72,6 +72,8 @@ psp_dev_create(struct net_device *netdev, mutex_init(&psd->lock); INIT_LIST_HEAD(&psd->active_assocs); + INIT_LIST_HEAD(&psd->prev_assocs); + INIT_LIST_HEAD(&psd->stale_assocs); refcount_set(&psd->refcnt, 1); mutex_lock(&psp_devs_lock); @@ -116,7 +118,9 @@ void psp_dev_unregister(struct psp_dev *psd) xa_erase(&psp_devs, psd->id); mutex_unlock(&psp_devs_lock); - list_for_each_entry_safe(pas, next, &psd->active_assocs, assocs_list) + list_splice_init(&psd->active_assocs, &psd->prev_assocs); + list_splice_init(&psd->prev_assocs, &psd->stale_assocs); + list_for_each_entry_safe(pas, next, &psd->stale_assocs, assocs_list) psp_dev_tx_key_del(psd, pas); rcu_assign_pointer(psd->main_netdev->psp_dev, NULL); diff --git a/net/psp/psp_nl.c b/net/psp/psp_nl.c index 58508e642185..7b8a1d390cde 100644 --- a/net/psp/psp_nl.c +++ b/net/psp/psp_nl.c @@ -230,6 +230,7 @@ int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info) struct psp_dev *psd = info->user_ptr[0]; struct genl_info ntf_info; struct sk_buff *ntf, *rsp; + u8 prev_gen; int err; rsp = psp_nl_reply_new(info); @@ -249,10 +250,19 @@ int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info) goto err_free_ntf; } + /* suggest the next gen number, driver can override */ + prev_gen = psd->generation; + psd->generation = (prev_gen + 1) & PSP_GEN_VALID_MASK; + err = psd->ops->key_rotate(psd, info->extack); if (err) goto err_free_ntf; + WARN_ON_ONCE((psd->generation && psd->generation == prev_gen) || + psd->generation & ~PSP_GEN_VALID_MASK); + + psp_assocs_key_rotated(psd); + nlmsg_end(ntf, (struct nlmsghdr *)ntf->data); genlmsg_multicast_netns(&psp_nl_family, dev_net(psd->main_netdev), ntf, 0, PSP_NLGRP_USE, GFP_KERNEL); diff --git a/net/psp/psp_sock.c b/net/psp/psp_sock.c index bcef042cb8a5..7a791703850c 100644 --- a/net/psp/psp_sock.c +++ b/net/psp/psp_sock.c @@ -58,6 +58,7 @@ struct psp_assoc *psp_assoc_create(struct psp_dev *psd) return NULL; pas->psd = psd; + pas->generation = psd->generation; psp_dev_get(psd); refcount_set(&pas->refcnt, 1); @@ -235,6 +236,21 @@ int psp_sock_assoc_set_tx(struct sock *sk, struct psp_dev *psd, return err; } +void psp_assocs_key_rotated(struct psp_dev *psd) +{ + struct psp_assoc *pas, *next; + + /* Mark the stale associations as invalid, they will no longer + * be able to Rx any traffic. + */ + list_for_each_entry_safe(pas, next, &psd->prev_assocs, assocs_list) + pas->generation |= ~PSP_GEN_VALID_MASK; + list_splice_init(&psd->prev_assocs, &psd->stale_assocs); + list_splice_init(&psd->active_assocs, &psd->prev_assocs); + + /* TODO: we should inform the sockets that got shut down */ +} + void psp_twsk_init(struct tcp_timewait_sock *tw, struct sock *sk) { struct psp_assoc *pas = psp_sk_assoc(sk); From patchwork Fri May 10 03:04:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660841 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7F2C14A088 for ; Fri, 10 May 2024 03:04:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310290; cv=none; b=gxxnos5YD7wFnqeNUSIicDOnfzWTob9+j8okaHAv9DZoKnXIB/VC0xQGhXzsnYXyO/c3JlP4X+pkOS1Cd+6PtLyBmEiDjrhbNnRjyjTWIrR/WT2ynuFVaK6/hJpwSUuROMuPwV4H+dvWgZFwUTF4zHBC8NmE6GkGPs/Yc5pPYBQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310290; c=relaxed/simple; bh=UmJPQY6q/vJGwfx+xf27dIf+FqQgMB7AfaBEjQ1Cxkc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RwEvzhdM/23Q4Eava5S2qK77tqQrVJYQiOnKMuTBqmn0/fUX+7dYXCwmAUD16wS6ZBEeT/GiPFJxvlZxz1vCZ9Q4oPC3RJmiuJYxcRlRfNhC4OeYD49dCX3BQSMcqd7+PQ4LUhAvGzKMiiAr9NZ1JezjLRXPZOPqn4mCjYM5le8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HCi5vrg/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HCi5vrg/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2E29FC116B1; Fri, 10 May 2024 03:04:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310290; bh=UmJPQY6q/vJGwfx+xf27dIf+FqQgMB7AfaBEjQ1Cxkc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HCi5vrg/deJcHpVfzeGMO9uwKxMiTxBk/dHwQD026srJL676FdPhiX+hWUC+CpsGM wAvJ+J5LnR2+/GKi86t4xdUXtFbQfVh9aNzDqg1CbwCcu9i0gsm+y3usW3IO3RV0+M K2Eupn8x61LHBIN1A0TqRA0hor+Tz+P+JyNtoa/xXKJJzF4EXSj/quqM0a7hAkozS6 guhX45UTpCr2dE4SjefXCoKJMJj/IQPiOWIVmGIh+ZJr1T3Vb6JHNEWeVkX2U/Xk33 SIXQN9kEz3Do6Iv894AWKYt2XF8+vy6lHQYMs+6f7IciMOgAYWPmi1coTAD79UnvCE 3AJ2t3LfIgitQ== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Raed Salem , Jakub Kicinski Subject: [RFC net-next 09/15] net/mlx5e: Support PSP offload functionality Date: Thu, 9 May 2024 20:04:29 -0700 Message-ID: <20240510030435.120935-10-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Raed Salem Add PSP offload related IFC structs, layouts, and enumerations. Implement .set_config and .rx_spi_alloc PSP device operations. Driver does not need to make use of the .set_config operation. Stub .assoc_add and .assoc_del PSP operations. Introduce the MLX5_EN_PSP configuration option for enabling PSP offload support on mlx5 devices. Signed-off-by: Raed Salem Signed-off-by: Rahul Rameshbabu Signed-off-by: Jakub Kicinski --- .../net/ethernet/mellanox/mlx5/core/Kconfig | 11 ++ .../net/ethernet/mellanox/mlx5/core/Makefile | 4 +- drivers/net/ethernet/mellanox/mlx5/core/en.h | 3 + .../ethernet/mellanox/mlx5/core/en/params.c | 4 +- .../mellanox/mlx5/core/en_accel/nisp.c | 149 ++++++++++++++++++ .../mellanox/mlx5/core/en_accel/nisp.h | 53 +++++++ .../mlx5/core/en_accel/nisp_offload.c | 52 ++++++ .../net/ethernet/mellanox/mlx5/core/en_main.c | 9 ++ drivers/net/ethernet/mellanox/mlx5/core/fw.c | 6 + .../net/ethernet/mellanox/mlx5/core/main.c | 5 + .../net/ethernet/mellanox/mlx5/core/nisp.c | 24 +++ .../net/ethernet/mellanox/mlx5/core/nisp.h | 15 ++ include/linux/mlx5/device.h | 4 + include/linux/mlx5/driver.h | 2 + include/linux/mlx5/mlx5_ifc.h | 98 +++++++++++- 15 files changed, 431 insertions(+), 8 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.h create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_offload.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/nisp.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/nisp.h diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig index 685335832a93..0dc9665a9557 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig +++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig @@ -197,3 +197,14 @@ config MLX5_DPLL help DPLL support in Mellanox Technologies ConnectX NICs. +config MLX5_EN_PSP + bool "Mellanox Technologies support for PSP cryptography-offload acceleration" + depends on INET_PSP + depends on MLX5_CORE_EN + default y + help + mlx5 device offload support for Google PSP Security Protocol offload. + Adds support for PSP encryption offload and for SPI and key generation + interfaces to PSP Stack which supports PSP crypto offload. + + If unsure, say Y. diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index 76dc5a9b9648..c17a5e343603 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -17,7 +17,7 @@ mlx5_core-y := main.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \ fs_counters.o fs_ft_pool.o rl.o lag/debugfs.o lag/lag.o dev.o events.o wq.o lib/gid.o \ lib/devcom.o lib/pci_vsc.o lib/dm.o lib/fs_ttc.o diag/fs_tracepoint.o \ diag/fw_tracer.o diag/crdump.o devlink.o diag/rsc_dump.o diag/reporter_vnic.o \ - fw_reset.o qos.o lib/tout.o lib/aso.o + fw_reset.o qos.o lib/tout.o lib/aso.o nisp.o # # Netdev basic @@ -109,6 +109,8 @@ mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/ktls_stats.o \ en_accel/fs_tcp.o en_accel/ktls.o en_accel/ktls_txrx.o \ en_accel/ktls_tx.o en_accel/ktls_rx.o +mlx5_core-$(CONFIG_MLX5_EN_PSP) += en_accel/nisp.o en_accel/nisp_offload.o + mlx5_core-$(CONFIG_MLX5_SW_STEERING) += steering/dr_domain.o steering/dr_table.o \ steering/dr_matcher.o steering/dr_rule.o \ steering/dr_icm_pool.o steering/dr_buddy.o \ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index f8bd9dbf59cd..b7ceb8011a92 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -941,6 +941,9 @@ struct mlx5e_priv { #ifdef CONFIG_MLX5_EN_IPSEC struct mlx5e_ipsec *ipsec; #endif +#ifdef CONFIG_MLX5_EN_PSP + struct mlx5e_nisp *nisp; +#endif #ifdef CONFIG_MLX5_EN_TLS struct mlx5e_tls *tls; #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c index ec819dfc98be..3141abb33ff7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c @@ -6,6 +6,7 @@ #include "en/port.h" #include "en_accel/en_accel.h" #include "en_accel/ipsec.h" +#include "en_accel/nisp.h" #include #include #include @@ -1005,7 +1006,8 @@ void mlx5e_build_sq_param(struct mlx5_core_dev *mdev, bool allow_swp; allow_swp = mlx5_geneve_tx_allowed(mdev) || - (mlx5_ipsec_device_caps(mdev) & MLX5_IPSEC_CAP_CRYPTO); + (mlx5_ipsec_device_caps(mdev) & MLX5_IPSEC_CAP_CRYPTO) || + mlx5_is_nisp_device(mdev); mlx5e_build_sq_param_common(mdev, param); MLX5_SET(wq, wq, log_wq_sz, params->log_sq_size); MLX5_SET(sqc, sqc, allow_swp, allow_swp); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c new file mode 100644 index 000000000000..eff7906b3764 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c @@ -0,0 +1,149 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ +#include +#include +#include +#include "mlx5_core.h" +#include "../nisp.h" +#include "lib/crypto.h" +#include "en_accel/nisp.h" + +static int +mlx5e_psp_set_config(struct psp_dev *psd, struct psp_dev_config *conf, + struct netlink_ext_ack *extack) +{ + return 0; /* TODO: this should actually do things to the device */ +} + +static int +mlx5e_psp_rx_spi_alloc(struct psp_dev *psd, u32 version, + struct psp_key_parsed *assoc, + struct netlink_ext_ack *extack) +{ + struct mlx5e_priv *priv = netdev_priv(psd->main_netdev); + enum mlx5_nisp_gen_spi_in_key_size keysz; + struct nisp_key_spi key_spi = {}; + u8 keysz_bytes; + int err; + + switch (version) { + case PSP_VERSION_HDR0_AES_GCM_128: + keysz = MLX5_NISP_GEN_SPI_IN_KEY_SIZE_128; + keysz_bytes = 16; + break; + case PSP_VERSION_HDR0_AES_GCM_256: + keysz = MLX5_NISP_GEN_SPI_IN_KEY_SIZE_256; + keysz_bytes = 32; + break; + default: + return -EINVAL; + } + + err = mlx5e_nisp_generate_key_spi(priv->mdev, keysz, keysz_bytes, + &key_spi); + if (err) + return err; + + assoc->spi = cpu_to_be32(key_spi.spi); + memcpy(assoc->key, key_spi.key, keysz_bytes); + return 0; +} + +static int mlx5e_psp_assoc_add(struct psp_dev *psd, struct psp_assoc *pas, + struct netlink_ext_ack *extack) +{ + struct mlx5e_priv *priv = netdev_priv(psd->main_netdev); + + mlx5_core_dbg(priv->mdev, "PSP assoc add: rx: %u, tx: %u\n", + be32_to_cpu(pas->rx.spi), be32_to_cpu(pas->tx.spi)); + + return -EINVAL; +} + +static void mlx5e_psp_assoc_del(struct psp_dev *psd, struct psp_assoc *tas) +{ +} + +static struct psp_dev_ops mlx5_psp_ops = { + .set_config = mlx5e_psp_set_config, + .rx_spi_alloc = mlx5e_psp_rx_spi_alloc, + .tx_key_add = mlx5e_psp_assoc_add, + .tx_key_del = mlx5e_psp_assoc_del, +}; + +void mlx5e_nisp_unregister(struct mlx5e_priv *priv) +{ + if (!priv->nisp || !priv->nisp->psp) + return; + + psp_dev_unregister(priv->nisp->psp); +} + +void mlx5e_nisp_register(struct mlx5e_priv *priv) +{ + /* FW Caps missing */ + if (!priv->nisp) + return; + + priv->nisp->caps.assoc_drv_spc = sizeof(u32); + priv->nisp->caps.versions = 1 << PSP_VERSION_HDR0_AES_GCM_128; + if (MLX5_CAP_NISP(priv->mdev, nisp_crypto_esp_aes_gcm_256_encrypt) && + MLX5_CAP_NISP(priv->mdev, nisp_crypto_esp_aes_gcm_256_decrypt)) + priv->nisp->caps.versions |= 1 << PSP_VERSION_HDR0_AES_GCM_256; + + priv->nisp->psp = psp_dev_create(priv->netdev, &mlx5_psp_ops, + &priv->nisp->caps, NULL); + if (IS_ERR(priv->nisp->psp)) + mlx5_core_err(priv->mdev, "PSP failed to register due to %pe\n", + priv->nisp->psp); +} + +int mlx5e_nisp_init(struct mlx5e_priv *priv) +{ + struct mlx5_core_dev *mdev = priv->mdev; + struct mlx5e_nisp *nisp; + + if (!mlx5_is_nisp_device(mdev)) { + mlx5_core_dbg(mdev, "NISP offload not supported\n"); + return -EOPNOTSUPP; + } + + if (!MLX5_CAP_ETH(mdev, swp)) { + mlx5_core_dbg(mdev, "SWP not supported\n"); + return -EOPNOTSUPP; + } + + if (!MLX5_CAP_ETH(mdev, swp_csum)) { + mlx5_core_dbg(mdev, "SWP checksum not supported\n"); + return -EOPNOTSUPP; + } + + if (!MLX5_CAP_ETH(mdev, swp_csum_l4_partial)) { + mlx5_core_dbg(mdev, "SWP L4 partial checksum not supported\n"); + return -EOPNOTSUPP; + } + + if (!MLX5_CAP_ETH(mdev, swp_lso)) { + mlx5_core_dbg(mdev, "NISP LSO not supported\n"); + return -EOPNOTSUPP; + } + + nisp = kzalloc(sizeof(*nisp), GFP_KERNEL); + if (!nisp) + return -ENOMEM; + + priv->nisp = nisp; + mlx5_core_dbg(priv->mdev, "NISP attached to netdevice\n"); + return 0; +} + +void mlx5e_nisp_cleanup(struct mlx5e_priv *priv) +{ + struct mlx5e_nisp *nisp = priv->nisp; + + if (!nisp) + return; + + priv->nisp = NULL; + kfree(nisp); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.h new file mode 100644 index 000000000000..93eaea8b6f77 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5E_ACCEL_NISP_H__ +#define __MLX5E_ACCEL_NISP_H__ +#if IS_ENABLED(CONFIG_MLX5_EN_PSP) +#include +#include "en.h" + +struct mlx5e_nisp { + struct psp_dev *psp; + struct psp_dev_caps caps; +}; + +struct nisp_key_spi { + u32 spi; + __be32 key[PSP_MAX_KEY / sizeof(u32)]; + u16 keysz; +}; + +static inline bool mlx5_is_nisp_device(struct mlx5_core_dev *mdev) +{ + if (!MLX5_CAP_GEN(mdev, psp)) + return false; + + if (!MLX5_CAP_NISP(mdev, nisp_crypto_esp_aes_gcm_128_encrypt) || + !MLX5_CAP_NISP(mdev, nisp_crypto_esp_aes_gcm_128_decrypt)) + return false; + + return true; +} + +void mlx5e_nisp_register(struct mlx5e_priv *priv); +void mlx5e_nisp_unregister(struct mlx5e_priv *priv); +int mlx5e_nisp_init(struct mlx5e_priv *priv); +void mlx5e_nisp_cleanup(struct mlx5e_priv *priv); +int mlx5e_nisp_rotate_key(struct mlx5_core_dev *mdev); +int mlx5e_nisp_generate_key_spi(struct mlx5_core_dev *mdev, + enum mlx5_nisp_gen_spi_in_key_size keysz, + unsigned int keysz_bytes, + struct nisp_key_spi *keys); +#else +static inline bool mlx5_is_nisp_device(struct mlx5_core_dev *mdev) +{ + return false; +} + +static inline void mlx5e_nisp_register(struct mlx5e_priv *priv) { } +static inline void mlx5e_nisp_unregister(struct mlx5e_priv *priv) { } +static inline int mlx5e_nisp_init(struct mlx5e_priv *priv) { return 0; } +static inline void mlx5e_nisp_cleanup(struct mlx5e_priv *priv) { } +#endif /* CONFIG_MLX5_EN_PSP */ +#endif /* __MLX5E_ACCEL_NISP_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_offload.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_offload.c new file mode 100644 index 000000000000..fc3268884dc9 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_offload.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ +#include +#include +#include "mlx5_core.h" +#include "en_accel/nisp.h" + +int mlx5e_nisp_rotate_key(struct mlx5_core_dev *mdev) +{ + u32 in[MLX5_ST_SZ_DW(nisp_rotate_key_in)] = {}; + u32 out[MLX5_ST_SZ_DW(nisp_rotate_key_out)]; + + MLX5_SET(nisp_rotate_key_in, in, opcode, + MLX5_CMD_OP_NISP_ROTATE_KEY); + + return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out)); +} + +int mlx5e_nisp_generate_key_spi(struct mlx5_core_dev *mdev, + enum mlx5_nisp_gen_spi_in_key_size keysz, + unsigned int keysz_bytes, + struct nisp_key_spi *keys) +{ + u32 in[MLX5_ST_SZ_DW(nisp_gen_spi_in)] = {}; + int err, outlen, i; + void *out, *outkey; + + WARN_ON_ONCE(keysz_bytes > PSP_MAX_KEY); + + outlen = MLX5_ST_SZ_BYTES(nisp_gen_spi_out) + MLX5_ST_SZ_BYTES(key_spi); + out = kzalloc(outlen, GFP_KERNEL); + if (!out) + return -ENOMEM; + + MLX5_SET(nisp_gen_spi_in, in, opcode, MLX5_CMD_OP_NISP_GEN_SPI); + MLX5_SET(nisp_gen_spi_in, in, key_size, keysz); + MLX5_SET(nisp_gen_spi_in, in, num_of_spi, 1); + err = mlx5_cmd_exec(mdev, in, sizeof(in), out, outlen); + if (err) + goto out; + + outkey = MLX5_ADDR_OF(nisp_gen_spi_out, out, key_spi); + keys->keysz = keysz_bytes * BITS_PER_BYTE; + keys->spi = MLX5_GET(key_spi, outkey, spi); + for (i = 0; i < keysz_bytes / sizeof(*keys->key); ++i) + keys->key[i] = cpu_to_be32(MLX5_GET(key_spi, + outkey + (32 - keysz_bytes), key[i])); + +out: + kfree(out); + return err; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index ffe8919494d5..4948e19c3f3f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -49,6 +49,7 @@ #include "en_tc.h" #include "en_rep.h" #include "en_accel/ipsec.h" +#include "en_accel/nisp.h" #include "en_accel/macsec.h" #include "en_accel/en_accel.h" #include "en_accel/ktls.h" @@ -5506,6 +5507,7 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev, if (take_rtnl) rtnl_lock(); + mlx5e_nisp_register(priv); /* update XDP supported features */ mlx5e_set_xdp_feature(netdev); @@ -5518,6 +5520,7 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev, static void mlx5e_nic_cleanup(struct mlx5e_priv *priv) { mlx5e_health_destroy_reporters(priv); + mlx5e_nisp_unregister(priv); mlx5e_ktls_cleanup(priv); mlx5e_fs_cleanup(priv->fs); debugfs_remove_recursive(priv->dfs_root); @@ -5645,6 +5648,10 @@ static void mlx5e_nic_enable(struct mlx5e_priv *priv) if (err) mlx5_core_err(mdev, "MACsec initialization failed, %d\n", err); + err = mlx5e_nisp_init(priv); + if (err) + mlx5_core_err(mdev, "PSP initialization failed, %d\n", err); + /* Marking the link as currently not needed by the Driver */ if (!netif_running(netdev)) mlx5e_modify_admin_state(mdev, MLX5_PORT_DOWN); @@ -5702,6 +5709,7 @@ static void mlx5e_nic_disable(struct mlx5e_priv *priv) mlx5e_disable_async_events(priv); mlx5_lag_remove_netdev(mdev, priv->netdev); mlx5_vxlan_reset_to_default(mdev->vxlan); + mlx5e_nisp_cleanup(priv); mlx5e_macsec_cleanup(priv); mlx5e_ipsec_cleanup(priv); } @@ -6337,6 +6345,7 @@ static void _mlx5e_remove(struct auxiliary_device *adev) mlx5_core_uplink_netdev_set(mdev, NULL); mlx5e_dcbnl_delete_app(priv); + mlx5e_nisp_unregister(priv); unregister_netdev(priv->netdev); _mlx5e_suspend(adev); priv->profile->cleanup(priv); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw.c b/drivers/net/ethernet/mellanox/mlx5/core/fw.c index 2d95a9b7b44e..eee318d64d98 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw.c @@ -280,6 +280,12 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev) return err; } + if (MLX5_CAP_GEN(dev, psp)) { + err = mlx5_core_get_caps(dev, MLX5_CAP_NISP); + if (err) + return err; + } + return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 331ce47f51a1..8625d593ed89 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -74,6 +74,7 @@ #include "mlx5_irq.h" #include "hwmon.h" #include "lag/lag.h" +#include "nisp.h" MODULE_AUTHOR("Eli Cohen "); MODULE_DESCRIPTION("Mellanox 5th generation network adapters (ConnectX series) core driver"); @@ -1013,6 +1014,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev) dev->vxlan = mlx5_vxlan_create(dev); dev->geneve = mlx5_geneve_create(dev); + dev->nisp = mlx5_nisp_create(dev); err = mlx5_init_rl_table(dev); if (err) { @@ -1095,6 +1097,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev) err_rl_cleanup: mlx5_cleanup_rl_table(dev); err_tables_cleanup: + mlx5_nisp_destroy(dev->nisp); mlx5_geneve_destroy(dev->geneve); mlx5_vxlan_destroy(dev->vxlan); mlx5_cleanup_clock(dev); @@ -1129,6 +1132,7 @@ static void mlx5_cleanup_once(struct mlx5_core_dev *dev) mlx5_sriov_cleanup(dev); mlx5_mpfs_cleanup(dev); mlx5_cleanup_rl_table(dev); + mlx5_nisp_destroy(dev->nisp); mlx5_geneve_destroy(dev->geneve); mlx5_vxlan_destroy(dev->vxlan); mlx5_cleanup_clock(dev); @@ -1763,6 +1767,7 @@ static const int types[] = { MLX5_CAP_VDPA_EMULATION, MLX5_CAP_IPSEC, MLX5_CAP_PORT_SELECTION, + MLX5_CAP_NISP, MLX5_CAP_MACSEC, MLX5_CAP_ADV_VIRTUALIZATION, MLX5_CAP_CRYPTO, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/nisp.c b/drivers/net/ethernet/mellanox/mlx5/core/nisp.c new file mode 100644 index 000000000000..f82734df8bf0 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/nisp.c @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#include "nisp.h" + +struct mlx5_nisp *mlx5_nisp_create(struct mlx5_core_dev *mdev) +{ + struct mlx5_nisp *nisp = kzalloc(sizeof(*nisp), GFP_KERNEL); + + if (!nisp) + return ERR_PTR(-ENOMEM); + + nisp->mdev = mdev; + + return nisp; +} + +void mlx5_nisp_destroy(struct mlx5_nisp *nisp) +{ + if (IS_ERR_OR_NULL(nisp)) + return; + + kfree(nisp); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/nisp.h b/drivers/net/ethernet/mellanox/mlx5/core/nisp.h new file mode 100644 index 000000000000..c15e50e7ada7 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/nisp.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5_NISP_H__ +#define __MLX5_NISP_H__ +#include + +struct mlx5_nisp { + struct mlx5_core_dev *mdev; +}; + +struct mlx5_nisp *mlx5_nisp_create(struct mlx5_core_dev *mdev); +void mlx5_nisp_destroy(struct mlx5_nisp *nisp); + +#endif /* __MLX5_NISP_H__ */ diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index d7bb31d9a446..6c00c78bad53 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -1223,6 +1223,7 @@ enum mlx5_cap_type { MLX5_CAP_DEV_EVENT = 0x14, MLX5_CAP_IPSEC, MLX5_CAP_CRYPTO = 0x1a, + MLX5_CAP_NISP = 0x1e, MLX5_CAP_MACSEC = 0x1f, MLX5_CAP_GENERAL_2 = 0x20, MLX5_CAP_PORT_SELECTION = 0x25, @@ -1435,6 +1436,9 @@ enum mlx5_qcam_feature_groups { #define MLX5_CAP_MACSEC(mdev, cap)\ MLX5_GET(macsec_cap, (mdev)->caps.hca[MLX5_CAP_MACSEC]->cur, cap) +#define MLX5_CAP_NISP(mdev, cap)\ + MLX5_GET(nisp_cap, (mdev)->caps.hca[MLX5_CAP_NISP]->cur, cap) + enum { MLX5_CMD_STAT_OK = 0x0, MLX5_CMD_STAT_INT_ERR = 0x1, diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index bf9324a31ae9..8d6060866163 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -515,6 +515,7 @@ struct mlx5_sf_dev_table; struct mlx5_sf_hw_table; struct mlx5_sf_table; struct mlx5_crypto_dek_priv; +struct mlx5_nisp; struct mlx5_rate_limit { u32 rate; @@ -824,6 +825,7 @@ struct mlx5_core_dev { #endif u64 num_ipsec_offloads; struct mlx5_sd *sd; + struct mlx5_nisp *nisp; }; struct mlx5_db { diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index f468763478ae..a48e6a293602 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -309,6 +309,8 @@ enum { MLX5_CMD_OP_CREATE_UMEM = 0xa08, MLX5_CMD_OP_DESTROY_UMEM = 0xa0a, MLX5_CMD_OP_SYNC_STEERING = 0xb00, + MLX5_CMD_OP_NISP_GEN_SPI = 0xb10, + MLX5_CMD_OP_NISP_ROTATE_KEY = 0xb11, MLX5_CMD_OP_QUERY_VHCA_STATE = 0xb0d, MLX5_CMD_OP_MODIFY_VHCA_STATE = 0xb0e, MLX5_CMD_OP_SYNC_CRYPTO = 0xb12, @@ -477,12 +479,14 @@ struct mlx5_ifc_flow_table_prop_layout_bits { u8 execute_aso[0x1]; u8 reserved_at_47[0x19]; - u8 reserved_at_60[0x2]; + u8 reformat_l2_to_l3_nisp_tunnel[0x1]; + u8 reformat_l3_nisp_tunnel_to_l2[0x1]; u8 reformat_insert[0x1]; u8 reformat_remove[0x1]; u8 macsec_encrypt[0x1]; u8 macsec_decrypt[0x1]; - u8 reserved_at_66[0x2]; + u8 nisp_encrypt[0x1]; + u8 nisp_decrypt[0x1]; u8 reformat_add_macsec[0x1]; u8 reformat_remove_macsec[0x1]; u8 reserved_at_6a[0xe]; @@ -670,7 +674,7 @@ struct mlx5_ifc_fte_match_set_misc2_bits { u8 metadata_reg_a[0x20]; - u8 reserved_at_1a0[0x8]; + u8 nisp_syndrome[0x8]; u8 macsec_syndrome[0x8]; u8 ipsec_syndrome[0x8]; @@ -1093,7 +1097,8 @@ struct mlx5_ifc_per_protocol_networking_offload_caps_bits { u8 tunnel_stateless_ip_over_ip_tx[0x1]; u8 reserved_at_2e[0x2]; u8 max_vxlan_udp_ports[0x8]; - u8 reserved_at_38[0x6]; + u8 swp_csum_l4_partial[0x1]; + u8 reserved_at_39[0x5]; u8 max_geneve_opt_len[0x1]; u8 tunnel_stateless_geneve_rx[0x1]; @@ -1390,6 +1395,19 @@ struct mlx5_ifc_macsec_cap_bits { u8 reserved_at_40[0x7c0]; }; +struct mlx5_ifc_nisp_cap_bits { + u8 reserved_at_0[0x1]; + u8 nisp_crypto_offload[0x1]; /* Set by the driver */ + u8 reserved_at_2[0x1]; + u8 nisp_crypto_esp_aes_gcm_256_encrypt[0x1]; + u8 nisp_crypto_esp_aes_gcm_128_encrypt[0x1]; + u8 nisp_crypto_esp_aes_gcm_256_decrypt[0x1]; + u8 nisp_crypto_esp_aes_gcm_128_decrypt[0x1]; + u8 reserved_at_7[0x4]; + u8 log_max_num_of_nisp_spi[0x5]; + u8 reserved_at_10[0x7f0]; +}; + enum { MLX5_WQ_TYPE_LINKED_LIST = 0x0, MLX5_WQ_TYPE_CYCLIC = 0x1, @@ -1521,7 +1539,7 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 reg_c_preserve[0x1]; u8 reserved_at_aa[0x1]; u8 log_max_srq[0x5]; - u8 reserved_at_b0[0x1]; + u8 reserved_at_b0[0x1]; u8 uplink_follow[0x1]; u8 ts_cqe_to_dest_cqn[0x1]; u8 reserved_at_b3[0x6]; @@ -1744,7 +1762,9 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 reserved_at_280[0x10]; u8 max_wqe_sz_sq[0x10]; - u8 reserved_at_2a0[0x10]; + u8 reserved_at_2a0[0xa]; + u8 psp[0x1]; + u8 reserved_at_2b1[0x5]; u8 max_wqe_sz_rq[0x10]; u8 max_flow_counter_31_16[0x10]; @@ -3519,6 +3539,7 @@ union mlx5_ifc_hca_cap_union_bits { struct mlx5_ifc_macsec_cap_bits macsec_cap; struct mlx5_ifc_crypto_cap_bits crypto_cap; struct mlx5_ifc_ipsec_cap_bits ipsec_cap; + struct mlx5_ifc_nisp_cap_bits nisp_cap; u8 reserved_at_0[0x8000]; }; @@ -3548,6 +3569,7 @@ enum { enum { MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_IPSEC = 0x0, MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_MACSEC = 0x1, + MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_NISP = 0x2, }; struct mlx5_ifc_vlan_bits { @@ -6747,6 +6769,8 @@ enum mlx5_reformat_ctx_type { MLX5_REFORMAT_TYPE_DEL_ESP_TRANSPORT_OVER_UDP = 0xa, MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_IPV6 = 0xb, MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_UDPV6 = 0xc, + MLX5_REFORMAT_TYPE_ADD_NISP_TUNNEL = 0xd, + MLX5_REFORMAT_TYPE_DEL_NISP_TUNNEL = 0xe, MLX5_REFORMAT_TYPE_INSERT_HDR = 0xf, MLX5_REFORMAT_TYPE_REMOVE_HDR = 0x10, MLX5_REFORMAT_TYPE_ADD_MACSEC = 0x11, @@ -6873,6 +6897,7 @@ enum { MLX5_ACTION_IN_FIELD_IPSEC_SYNDROME = 0x5D, MLX5_ACTION_IN_FIELD_OUT_EMD_47_32 = 0x6F, MLX5_ACTION_IN_FIELD_OUT_EMD_31_0 = 0x70, + MLX5_ACTION_IN_FIELD_NISP_SYNDROME = 0x71, }; struct mlx5_ifc_alloc_modify_header_context_out_bits { @@ -12452,6 +12477,7 @@ enum { MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_TLS = 0x1, MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_IPSEC = 0x2, MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_MACSEC = 0x4, + MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_NISP = 0x6, }; struct mlx5_ifc_tls_static_params_bits { @@ -12786,4 +12812,64 @@ struct mlx5_ifc_msees_reg_bits { u8 reserved_at_80[0x180]; }; +struct mlx5_ifc_nisp_rotate_key_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + + u8 reserved_at_40[0x40]; +}; + +struct mlx5_ifc_nisp_rotate_key_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + + u8 syndrome[0x20]; + + u8 reserved_at_40[0x40]; +}; + +enum mlx5_nisp_gen_spi_in_key_size { + MLX5_NISP_GEN_SPI_IN_KEY_SIZE_128 = 0x0, + MLX5_NISP_GEN_SPI_IN_KEY_SIZE_256 = 0x1, +}; + +struct mlx5_ifc_key_spi_bits { + u8 spi[0x20]; + + u8 reserved_at_20[0x60]; + + u8 key[8][0x20]; +}; + +struct mlx5_ifc_nisp_gen_spi_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + + u8 reserved_at_40[0x20]; + + u8 key_size[0x2]; + u8 reserved_at_62[0xe]; + u8 num_of_spi[0x10]; +}; + +struct mlx5_ifc_nisp_gen_spi_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + + u8 syndrome[0x20]; + + u8 reserved_at_40[0x10]; + u8 num_of_spi[0x10]; + + u8 reserved_at_60[0x20]; + + struct mlx5_ifc_key_spi_bits key_spi[0]; +}; + #endif /* MLX5_IFC_H */ From patchwork Fri May 10 03:04:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660842 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C01DA14A4EE for ; Fri, 10 May 2024 03:04:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310291; cv=none; b=LA0/64uW2J0KoTUxzm461M1rk7CDV+hB4m4LrrvkkH5cgQrjfv0J7BNP/S+DiY3NMJHCOK6GXMW2f+87nSJYrj//Cj4Y+sfDn8MZuNDJt86Qzg51WhD9amLGTtKfquftaP1sPiPzVXSX9Hf9/X4hZSzR5zXJz9KgYb8SPasAIxM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310291; c=relaxed/simple; bh=upOqaZJB+SbXIj2iAAB/XsbcajXrGP+lTOeYDUKikRs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pZzzKLCAM6sXypCjxM6oiI8L8TYi0WhQx2XNqv+rveuTbkHrTBrlcWz/MOodftMHHM1YUETajNxDJfndzvQnQ2yO0CS3LakGE8puPRl3w0NJjiHXIWGgCie1H+G7cam9DPa7jieCkA+X2+N5J4UrDIIlnBUdyPfImtkImVN/ROE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=r60+ibym; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="r60+ibym" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7F4EC3277B; Fri, 10 May 2024 03:04:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310291; bh=upOqaZJB+SbXIj2iAAB/XsbcajXrGP+lTOeYDUKikRs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r60+ibymnLBVoJrgw6FoYzXXXYypA0jDUOk1LfIfH6/gUulFzhZk/m//4ixJnYSeO Uf7RL+SQWxJSEv9cseoDfkDErU4beiDSjYLZ8mCmbBaVKA6a1OfkFntjZcpHlrkidS mM5wb02GQ56QFwUty/+Gh3Xw9iRW0RLOrBSLW7qfQYLKkjUtPt+SKpChuxuNu4o0ic 9qW2dHriEzalO0FnLdF49fH4tmwZDvjQJnyRq8qnZO/3VVE83sQ/qRjY/HxjUQnpxz fzOkc6mB6EAAKgc9aOim2Elxek2I7Iou5OOSClBj7I4EFVxEjt4gjMsYYnUerswqXV nPF+2lX33DUNw== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Raed Salem , Jakub Kicinski Subject: [RFC net-next 10/15] net/mlx5e: Implement PSP operations .assoc_add and .assoc_del Date: Thu, 9 May 2024 20:04:30 -0700 Message-ID: <20240510030435.120935-11-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Raed Salem Implement .assoc_add and .assoc_del PSP operations used in the tx control path. Allocate the relevant hardware resources when a new key is registered using .assoc_add. Destroy the key when .assoc_del is called. Use a atomic counter to keep track of the current number of keys being used by the device. Signed-off-by: Raed Salem Signed-off-by: Rahul Rameshbabu Signed-off-by: Jakub Kicinski --- .../net/ethernet/mellanox/mlx5/core/Makefile | 2 +- .../mellanox/mlx5/core/en_accel/en_accel.h | 8 + .../mellanox/mlx5/core/en_accel/nisp.c | 57 ++++- .../mellanox/mlx5/core/en_accel/nisp.h | 2 + .../mellanox/mlx5/core/en_accel/nisp_fs.c | 234 ++++++++++++++++++ .../mellanox/mlx5/core/en_accel/nisp_fs.h | 23 ++ .../net/ethernet/mellanox/mlx5/core/en_main.c | 10 +- .../ethernet/mellanox/mlx5/core/lib/crypto.h | 1 + 8 files changed, 327 insertions(+), 10 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.h diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index c17a5e343603..5ce78b84c763 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -109,7 +109,7 @@ mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/ktls_stats.o \ en_accel/fs_tcp.o en_accel/ktls.o en_accel/ktls_txrx.o \ en_accel/ktls_tx.o en_accel/ktls_rx.o -mlx5_core-$(CONFIG_MLX5_EN_PSP) += en_accel/nisp.o en_accel/nisp_offload.o +mlx5_core-$(CONFIG_MLX5_EN_PSP) += en_accel/nisp.o en_accel/nisp_offload.o en_accel/nisp_fs.o mlx5_core-$(CONFIG_MLX5_SW_STEERING) += steering/dr_domain.o steering/dr_table.o \ steering/dr_matcher.o steering/dr_rule.o \ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h index caa34b9c161e..c15e48b0724c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h @@ -42,6 +42,7 @@ #include #include "en.h" #include "en/txrx.h" +#include "en_accel/nisp_fs.h" #if IS_ENABLED(CONFIG_GENEVE) #include @@ -212,11 +213,18 @@ static inline void mlx5e_accel_cleanup_rx(struct mlx5e_priv *priv) static inline int mlx5e_accel_init_tx(struct mlx5e_priv *priv) { + int err; + + err = mlx5_accel_nisp_fs_init_tx_tables(priv); + if (err) + return err; + return mlx5e_ktls_init_tx(priv); } static inline void mlx5e_accel_cleanup_tx(struct mlx5e_priv *priv) { mlx5e_ktls_cleanup_tx(priv); + mlx5_accel_nisp_fs_cleanup_tx_tables(priv); } #endif /* __MLX5E_EN_ACCEL_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c index eff7906b3764..1131aa6e9b3d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c @@ -7,6 +7,12 @@ #include "../nisp.h" #include "lib/crypto.h" #include "en_accel/nisp.h" +#include "en_accel/nisp_fs.h" + +struct mlx5e_nisp_sa_entry { + struct mlx5e_accel_nisp_rule *nisp_rule; + u32 enc_key_id; +}; static int mlx5e_psp_set_config(struct psp_dev *psd, struct psp_dev_config *conf, @@ -49,19 +55,45 @@ mlx5e_psp_rx_spi_alloc(struct psp_dev *psd, u32 version, return 0; } +struct nisp_key { + u32 id; +}; + static int mlx5e_psp_assoc_add(struct psp_dev *psd, struct psp_assoc *pas, struct netlink_ext_ack *extack) { struct mlx5e_priv *priv = netdev_priv(psd->main_netdev); + struct mlx5_core_dev *mdev = priv->mdev; + struct mlx5e_nisp *nisp = priv->nisp; + struct psp_key_parsed *tx = &pas->tx; + struct nisp_key *nkey; + int err; - mlx5_core_dbg(priv->mdev, "PSP assoc add: rx: %u, tx: %u\n", - be32_to_cpu(pas->rx.spi), be32_to_cpu(pas->tx.spi)); + mdev = priv->mdev; + nkey = (struct nisp_key *)pas->drv_data; - return -EINVAL; + err = mlx5_create_encryption_key(mdev, tx->key, + pas->key_sz, + MLX5_ACCEL_OBJ_NISP_KEY, + &nkey->id); + if (err) { + mlx5_core_err(mdev, "Failed to create encryption key (err = %d)\n", err); + return err; + } + + atomic_inc(&nisp->tx_key_cnt); + return 0; } -static void mlx5e_psp_assoc_del(struct psp_dev *psd, struct psp_assoc *tas) +static void mlx5e_psp_assoc_del(struct psp_dev *psd, struct psp_assoc *pas) { + struct mlx5e_priv *priv = netdev_priv(psd->main_netdev); + struct mlx5e_nisp *nisp = priv->nisp; + struct nisp_key *nkey; + + nkey = (struct nisp_key *)pas->drv_data; + mlx5_destroy_encryption_key(priv->mdev, nkey->id); + atomic_dec(&nisp->tx_key_cnt); } static struct psp_dev_ops mlx5_psp_ops = { @@ -101,7 +133,9 @@ void mlx5e_nisp_register(struct mlx5e_priv *priv) int mlx5e_nisp_init(struct mlx5e_priv *priv) { struct mlx5_core_dev *mdev = priv->mdev; + struct mlx5e_nisp_fs *fs; struct mlx5e_nisp *nisp; + int err; if (!mlx5_is_nisp_device(mdev)) { mlx5_core_dbg(mdev, "NISP offload not supported\n"); @@ -133,8 +167,21 @@ int mlx5e_nisp_init(struct mlx5e_priv *priv) return -ENOMEM; priv->nisp = nisp; + fs = mlx5e_accel_nisp_fs_init(priv); + if (IS_ERR(fs)) { + err = PTR_ERR(fs); + goto out_err; + } + + nisp->fs = fs; + mlx5_core_dbg(priv->mdev, "NISP attached to netdevice\n"); return 0; + +out_err: + priv->nisp = NULL; + kfree(nisp); + return err; } void mlx5e_nisp_cleanup(struct mlx5e_priv *priv) @@ -144,6 +191,8 @@ void mlx5e_nisp_cleanup(struct mlx5e_priv *priv) if (!nisp) return; + WARN_ON(atomic_read(&nisp->tx_key_cnt)); + mlx5e_accel_nisp_fs_cleanup(nisp->fs); priv->nisp = NULL; kfree(nisp); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.h index 93eaea8b6f77..14e5813367a7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.h @@ -10,6 +10,8 @@ struct mlx5e_nisp { struct psp_dev *psp; struct psp_dev_caps caps; + struct mlx5e_nisp_fs *fs; + atomic_t tx_key_cnt; }; struct nisp_key_spi { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c new file mode 100644 index 000000000000..5d2ce83db7cc --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c @@ -0,0 +1,234 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#include +#include +#include "en.h" +#include "fs_core.h" +#include "en_accel/nisp_fs.h" +#include "en_accel/nisp.h" + +struct mlx5e_nisp_tx { + struct mlx5_flow_namespace *ns; + struct mlx5_flow_table *ft; + struct mlx5_flow_group *fg; + struct mlx5_flow_handle *rule; + struct mutex mutex; /* Protect NISP TX steering */ + u32 refcnt; +}; + +struct mlx5e_nisp_fs { + struct mlx5_core_dev *mdev; + struct mlx5e_nisp_tx *tx_fs; + struct mlx5e_flow_steering *fs; +}; + +enum accel_nisp_rule_action { + ACCEL_NISP_RULE_ACTION_ENCRYPT, +}; + +struct mlx5e_accel_nisp_rule { + struct mlx5_flow_handle *rule; + u8 action; +}; + +static void setup_fte_udp_psp(struct mlx5_flow_spec *spec, u16 udp_port) +{ + spec->match_criteria_enable |= MLX5_MATCH_OUTER_HEADERS; + MLX5_SET(fte_match_set_lyr_2_4, spec->match_criteria, udp_dport, 0xffff); + MLX5_SET(fte_match_set_lyr_2_4, spec->match_value, udp_dport, udp_port); + MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, spec->match_criteria, ip_protocol); + MLX5_SET(fte_match_set_lyr_2_4, spec->match_value, ip_protocol, IPPROTO_UDP); +} + +static int accel_nisp_fs_tx_create_ft_table(struct mlx5e_nisp_fs *fs) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + struct mlx5_flow_table_attr ft_attr = {}; + struct mlx5_core_dev *mdev = fs->mdev; + struct mlx5_flow_act flow_act = {}; + u32 *in, *mc, *outer_headers_c; + struct mlx5_flow_handle *rule; + struct mlx5_flow_spec *spec; + struct mlx5e_nisp_tx *tx_fs; + struct mlx5_flow_table *ft; + struct mlx5_flow_group *fg; + int err = 0; + + spec = kvzalloc(sizeof(*spec), GFP_KERNEL); + in = kvzalloc(inlen, GFP_KERNEL); + if (!spec || !in) { + err = -ENOMEM; + goto out; + } + + ft_attr.max_fte = 1; +#define MLX5E_NISP_PRIO 0 + ft_attr.prio = MLX5E_NISP_PRIO; +#define MLX5E_NISP_LEVEL 0 + ft_attr.level = MLX5E_NISP_LEVEL; + ft_attr.autogroup.max_num_groups = 1; + + tx_fs = fs->tx_fs; + ft = mlx5_create_flow_table(tx_fs->ns, &ft_attr); + if (IS_ERR(ft)) { + err = PTR_ERR(ft); + mlx5_core_err(mdev, "PSP: fail to add psp tx flow table, err = %d\n", err); + goto out; + } + + mc = MLX5_ADDR_OF(create_flow_group_in, in, match_criteria); + outer_headers_c = MLX5_ADDR_OF(fte_match_param, mc, outer_headers); + MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, outer_headers_c, ip_protocol); + MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, outer_headers_c, udp_dport); + MLX5_SET_CFG(in, match_criteria_enable, MLX5_MATCH_OUTER_HEADERS); + fg = mlx5_create_flow_group(ft, in); + if (IS_ERR(fg)) { + err = PTR_ERR(fg); + mlx5_core_err(mdev, "PSP: fail to add psp tx flow group, err = %d\n", err); + goto err_create_fg; + } + + setup_fte_udp_psp(spec, PSP_DEFAULT_UDP_PORT); + flow_act.crypto.type = MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_NISP; + flow_act.flags |= FLOW_ACT_NO_APPEND; + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_ALLOW | + MLX5_FLOW_CONTEXT_ACTION_CRYPTO_ENCRYPT; + rule = mlx5_add_flow_rules(ft, spec, &flow_act, NULL, 0); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + mlx5_core_err(mdev, "PSP: fail to add psp tx flow rule, err = %d\n", err); + goto err_add_flow_rule; + } + + tx_fs->ft = ft; + tx_fs->fg = fg; + tx_fs->rule = rule; + goto out; + +err_add_flow_rule: + mlx5_destroy_flow_group(fg); +err_create_fg: + mlx5_destroy_flow_table(ft); +out: + kvfree(in); + kvfree(spec); + return err; +} + +static void accel_nisp_fs_tx_destroy(struct mlx5e_nisp_tx *tx_fs) +{ + if (!tx_fs->ft) + return; + + mlx5_del_flow_rules(tx_fs->rule); + mlx5_destroy_flow_group(tx_fs->fg); + mlx5_destroy_flow_table(tx_fs->ft); +} + +static int accel_nisp_fs_tx_ft_get(struct mlx5e_nisp_fs *fs) +{ + struct mlx5e_nisp_tx *tx_fs = fs->tx_fs; + int err = 0; + + mutex_lock(&tx_fs->mutex); + if (tx_fs->refcnt++) + goto out; + + err = accel_nisp_fs_tx_create_ft_table(fs); + if (err) + tx_fs->refcnt--; +out: + mutex_unlock(&tx_fs->mutex); + return err; +} + +static void accel_nisp_fs_tx_ft_put(struct mlx5e_nisp_fs *fs) +{ + struct mlx5e_nisp_tx *tx_fs = fs->tx_fs; + + mutex_lock(&tx_fs->mutex); + if (--tx_fs->refcnt) + goto out; + + accel_nisp_fs_tx_destroy(tx_fs); +out: + mutex_unlock(&tx_fs->mutex); +} + +static void accel_nisp_fs_cleanup_tx(struct mlx5e_nisp_fs *fs) +{ + struct mlx5e_nisp_tx *tx_fs = fs->tx_fs; + + if (!tx_fs) + return; + + mutex_destroy(&tx_fs->mutex); + WARN_ON(tx_fs->refcnt); + kfree(tx_fs); + fs->tx_fs = NULL; +} + +static int accel_nisp_fs_init_tx(struct mlx5e_nisp_fs *fs) +{ + struct mlx5_flow_namespace *ns; + struct mlx5e_nisp_tx *tx_fs; + + ns = mlx5_get_flow_namespace(fs->mdev, + MLX5_FLOW_NAMESPACE_EGRESS_IPSEC); + if (!ns) + return -EOPNOTSUPP; + + tx_fs = kzalloc(sizeof(*tx_fs), GFP_KERNEL); + if (!tx_fs) + return -ENOMEM; + + mutex_init(&tx_fs->mutex); + tx_fs->ns = ns; + fs->tx_fs = tx_fs; + return 0; +} + +void mlx5_accel_nisp_fs_cleanup_tx_tables(struct mlx5e_priv *priv) +{ + if (!priv->nisp) + return; + + accel_nisp_fs_tx_ft_put(priv->nisp->fs); +} + +int mlx5_accel_nisp_fs_init_tx_tables(struct mlx5e_priv *priv) +{ + if (!priv->nisp) + return 0; + + return accel_nisp_fs_tx_ft_get(priv->nisp->fs); +} + +void mlx5e_accel_nisp_fs_cleanup(struct mlx5e_nisp_fs *fs) +{ + accel_nisp_fs_cleanup_tx(fs); + kfree(fs); +} + +struct mlx5e_nisp_fs *mlx5e_accel_nisp_fs_init(struct mlx5e_priv *priv) +{ + struct mlx5e_nisp_fs *fs; + int err = 0; + + fs = kzalloc(sizeof(*fs), GFP_KERNEL); + if (!fs) + return ERR_PTR(-ENOMEM); + + fs->mdev = priv->mdev; + err = accel_nisp_fs_init_tx(fs); + if (err) + goto err_tx; + + fs->fs = priv->fs; + + return fs; +err_tx: + kfree(fs); + return ERR_PTR(err); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.h new file mode 100644 index 000000000000..11cdc447a401 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5_NISP_FS_H__ +#define __MLX5_NISP_FS_H__ + +#ifdef CONFIG_MLX5_EN_PSP + +struct mlx5e_nisp_fs; + +struct mlx5e_nisp_fs *mlx5e_accel_nisp_fs_init(struct mlx5e_priv *priv); +void mlx5e_accel_nisp_fs_cleanup(struct mlx5e_nisp_fs *fs); +int mlx5_accel_nisp_fs_init_tx_tables(struct mlx5e_priv *priv); +void mlx5_accel_nisp_fs_cleanup_tx_tables(struct mlx5e_priv *priv); +#else +static inline int mlx5_accel_nisp_fs_init_tx_tables(struct mlx5e_priv *priv) +{ + return 0; +} + +static inline void mlx5_accel_nisp_fs_cleanup_tx_tables(struct mlx5e_priv *priv) { } +#endif /* CONFIG_MLX5_EN_PSP */ +#endif /* __MLX5_NISP_FS_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 4948e19c3f3f..38e0c4786b1c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -5495,6 +5495,10 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev, } priv->fs = fs; + err = mlx5e_nisp_init(priv); + if (err) + mlx5_core_err(mdev, "PSP initialization failed, %d\n", err); + err = mlx5e_ktls_init(priv); if (err) mlx5_core_err(mdev, "TLS initialization failed, %d\n", err); @@ -5522,6 +5526,7 @@ static void mlx5e_nic_cleanup(struct mlx5e_priv *priv) mlx5e_health_destroy_reporters(priv); mlx5e_nisp_unregister(priv); mlx5e_ktls_cleanup(priv); + mlx5e_nisp_cleanup(priv); mlx5e_fs_cleanup(priv->fs); debugfs_remove_recursive(priv->dfs_root); priv->fs = NULL; @@ -5648,10 +5653,6 @@ static void mlx5e_nic_enable(struct mlx5e_priv *priv) if (err) mlx5_core_err(mdev, "MACsec initialization failed, %d\n", err); - err = mlx5e_nisp_init(priv); - if (err) - mlx5_core_err(mdev, "PSP initialization failed, %d\n", err); - /* Marking the link as currently not needed by the Driver */ if (!netif_running(netdev)) mlx5e_modify_admin_state(mdev, MLX5_PORT_DOWN); @@ -5709,7 +5710,6 @@ static void mlx5e_nic_disable(struct mlx5e_priv *priv) mlx5e_disable_async_events(priv); mlx5_lag_remove_netdev(mdev, priv->netdev); mlx5_vxlan_reset_to_default(mdev->vxlan); - mlx5e_nisp_cleanup(priv); mlx5e_macsec_cleanup(priv); mlx5e_ipsec_cleanup(priv); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.h index c819c047bb9c..f257dfcf45d6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.h @@ -8,6 +8,7 @@ enum { MLX5_ACCEL_OBJ_TLS_KEY = MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_TLS, MLX5_ACCEL_OBJ_IPSEC_KEY = MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_IPSEC, MLX5_ACCEL_OBJ_MACSEC_KEY = MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_MACSEC, + MLX5_ACCEL_OBJ_NISP_KEY = MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_NISP, MLX5_ACCEL_OBJ_TYPE_KEY_NUM, }; From patchwork Fri May 10 03:04:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660843 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7135A14A612 for ; Fri, 10 May 2024 03:04:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310292; cv=none; b=fPSSo2gY/ivZc1/rFvjVHRlYzIxMna3g+DjB7rl9cZ0dNxwkDoLEUY3lJo1oOXWLHJNatMyC6XnT5ZRq2jSAtzgHeeDnU3m3LfdmIagEnKpv5guw437a/2xWFTAc8acOW3tr4hpY3vBburxqQiC2sjWyT9XNebX4FgFeazeYj28= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310292; c=relaxed/simple; bh=Ko7QWpB5UW2oDWL/PPysLgB3odYnHMO/QHDwLoXXCes=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mPIWXWc53n2b6MJV7UaOsxVjGEHTr7Kca9+PAYO+oMm6z8a/0E/e2K6FRvvUme+CIDhtXEyf/k6p8rP3J8IGejhmUAn/vCRM6iLECiOPyEp0U+WlS4c+mmXPpiWy5QsWDJPLNUV2+ULV0a1npvNxLf6Z9obwSMd2pkTjqWh0XDQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nwVdUNTB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nwVdUNTB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8FC36C2BBFC; Fri, 10 May 2024 03:04:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310292; bh=Ko7QWpB5UW2oDWL/PPysLgB3odYnHMO/QHDwLoXXCes=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nwVdUNTBqCicDcN9D7B1WIkBXUQfsHjA4RFJ+SRo4AIFb6jFEwmgLgp8rBfbq5aZ9 9FD4qZpOHh00lAurSGShSmu4XG9yIzwMtOtvnnTXD+w7FuvZRn8tjT+2/t9Mm8AlKo 6zchNVdMNe6Q/TE8oluJmtt7SKPOhK4mAtD5As25Vbws+5e5t4Xeyo1+fxV50PhyzV I5F8ieRzsYzyEJKQeRhGh+v12awaJPbXmwE+ReVB+Zbkjydr8DGC313vVvmdVDjEVy ayPnN8DKwPFkphFCh66m8m5MsECqmD+bE4Ixz8AxyskyRdMqf3REfSHMMriJcqwjVW Ac3Vsm6roudmA== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Raed Salem , Jakub Kicinski Subject: [RFC net-next 11/15] net/mlx5e: Implement PSP Tx data path Date: Thu, 9 May 2024 20:04:31 -0700 Message-ID: <20240510030435.120935-12-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Raed Salem Setup PSP offload on Tx data path based on whether skb indicates that it is intended for PSP or not. Support driver side encapsulation of the UDP headers, PSP headers, and PSP trailer for the PSP traffic that will be encrypted by the NIC. Signed-off-by: Raed Salem Signed-off-by: Rahul Rameshbabu Signed-off-by: Jakub Kicinski --- .../net/ethernet/mellanox/mlx5/core/Makefile | 3 +- drivers/net/ethernet/mellanox/mlx5/core/en.h | 4 +- .../mellanox/mlx5/core/en_accel/en_accel.h | 28 +++ .../mellanox/mlx5/core/en_accel/nisp_rxtx.c | 225 ++++++++++++++++++ .../mellanox/mlx5/core/en_accel/nisp_rxtx.h | 96 ++++++++ .../net/ethernet/mellanox/mlx5/core/en_tx.c | 10 +- .../mellanox/mlx5/core/lib/psp_defs.h | 28 +++ 7 files changed, 390 insertions(+), 4 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.h create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/lib/psp_defs.h diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index 5ce78b84c763..858ab2e7cb1f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -109,7 +109,8 @@ mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/ktls_stats.o \ en_accel/fs_tcp.o en_accel/ktls.o en_accel/ktls_txrx.o \ en_accel/ktls_tx.o en_accel/ktls_rx.o -mlx5_core-$(CONFIG_MLX5_EN_PSP) += en_accel/nisp.o en_accel/nisp_offload.o en_accel/nisp_fs.o +mlx5_core-$(CONFIG_MLX5_EN_PSP) += en_accel/nisp.o en_accel/nisp_offload.o en_accel/nisp_fs.o \ + en_accel/nisp_rxtx.o mlx5_core-$(CONFIG_MLX5_SW_STEERING) += steering/dr_domain.o steering/dr_table.o \ steering/dr_matcher.o steering/dr_rule.o \ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index b7ceb8011a92..92e2554d6271 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -47,6 +47,7 @@ #include #include #include +#include #include #include #include @@ -61,6 +62,7 @@ #include "en/rx_res.h" #include "en/selq.h" #include "lib/sd.h" +#include "lib/psp_defs.h" extern const struct net_device_ops mlx5e_netdev_ops; struct page_pool; @@ -68,7 +70,7 @@ struct page_pool; #define MLX5E_METADATA_ETHER_TYPE (0x8CE4) #define MLX5E_METADATA_ETHER_LEN 8 -#define MLX5E_ETH_HARD_MTU (ETH_HLEN + VLAN_HLEN + ETH_FCS_LEN) +#define MLX5E_ETH_HARD_MTU (ETH_HLEN + PSP_ENCAP_HLEN + PSP_TRL_SIZE + VLAN_HLEN + ETH_FCS_LEN) #define MLX5E_HW2SW_MTU(params, hwmtu) ((hwmtu) - ((params)->hard_mtu)) #define MLX5E_SW2HW_MTU(params, swmtu) ((swmtu) + ((params)->hard_mtu)) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h index c15e48b0724c..cea997847fa4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h @@ -43,6 +43,7 @@ #include "en.h" #include "en/txrx.h" #include "en_accel/nisp_fs.h" +#include "en_accel/nisp_rxtx.h" #if IS_ENABLED(CONFIG_GENEVE) #include @@ -114,6 +115,9 @@ struct mlx5e_accel_tx_state { #ifdef CONFIG_MLX5_EN_IPSEC struct mlx5e_accel_tx_ipsec_state ipsec; #endif +#ifdef CONFIG_MLX5_EN_PSP + struct mlx5e_accel_tx_nisp_state nisp_st; +#endif }; static inline bool mlx5e_accel_tx_begin(struct net_device *dev, @@ -132,6 +136,13 @@ static inline bool mlx5e_accel_tx_begin(struct net_device *dev, return false; #endif +#ifdef CONFIG_MLX5_EN_PSP + if (mlx5e_psp_is_offload(skb, dev)) { + if (unlikely(!mlx5e_nisp_handle_tx_skb(dev, skb, &state->nisp_st))) + return false; + } +#endif + #ifdef CONFIG_MLX5_EN_IPSEC if (test_bit(MLX5E_SQ_STATE_IPSEC, &sq->state) && xfrm_offload(skb)) { if (unlikely(!mlx5e_ipsec_handle_tx_skb(dev, skb, &state->ipsec))) @@ -152,8 +163,14 @@ static inline bool mlx5e_accel_tx_begin(struct net_device *dev, } static inline unsigned int mlx5e_accel_tx_ids_len(struct mlx5e_txqsq *sq, + struct sk_buff *skb, struct mlx5e_accel_tx_state *state) { +#ifdef CONFIG_MLX5_EN_PSP + if (mlx5e_psp_is_offload_state(&state->nisp_st)) + return mlx5e_nisp_tx_ids_len(&state->nisp_st); +#endif + #ifdef CONFIG_MLX5_EN_IPSEC if (test_bit(MLX5E_SQ_STATE_IPSEC, &sq->state)) return mlx5e_ipsec_tx_ids_len(&state->ipsec); @@ -167,8 +184,14 @@ static inline unsigned int mlx5e_accel_tx_ids_len(struct mlx5e_txqsq *sq, static inline void mlx5e_accel_tx_eseg(struct mlx5e_priv *priv, struct sk_buff *skb, + struct mlx5e_accel_tx_state *accel, struct mlx5_wqe_eth_seg *eseg, u16 ihs) { +#ifdef CONFIG_MLX5_EN_PSP + if (mlx5e_psp_is_offload_state(&accel->nisp_st)) + mlx5e_nisp_tx_build_eseg(priv, skb, &accel->nisp_st, eseg); +#endif + #ifdef CONFIG_MLX5_EN_IPSEC if (xfrm_offload(skb)) mlx5e_ipsec_tx_build_eseg(priv, skb, eseg); @@ -194,6 +217,11 @@ static inline void mlx5e_accel_tx_finish(struct mlx5e_txqsq *sq, mlx5e_ktls_handle_tx_wqe(&wqe->ctrl, &state->tls); #endif +#ifdef CONFIG_MLX5_EN_PSP + if (mlx5e_psp_is_offload_state(&state->nisp_st)) + mlx5e_nisp_handle_tx_wqe(wqe, &state->nisp_st, inlseg); +#endif + #ifdef CONFIG_MLX5_EN_IPSEC if (test_bit(MLX5E_SQ_STATE_IPSEC, &sq->state) && state->ipsec.xo && state->ipsec.tailen) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.c new file mode 100644 index 000000000000..c719b2916677 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.c @@ -0,0 +1,225 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#include +#include +#include +#include +#include +#include +#include + +#include "en.h" +#include "../nisp.h" +#include "en_accel/nisp_rxtx.h" +#include "en_accel/nisp.h" +#include "lib/psp_defs.h" + +static void mlx5e_nisp_set_swp(struct sk_buff *skb, + struct mlx5e_accel_tx_nisp_state *nisp_st, + struct mlx5_wqe_eth_seg *eseg) +{ + /* Tunnel Mode: + * SWP: OutL3 InL3 InL4 + * Pkt: MAC IP ESP IP L4 + * + * Transport Mode: + * SWP: OutL3 OutL4 + * Pkt: MAC IP ESP L4 + * + * Tunnel(VXLAN TCP/UDP) over Transport Mode + * SWP: OutL3 InL3 InL4 + * Pkt: MAC IP ESP UDP VXLAN IP L4 + */ + u8 inner_ipproto = 0; + struct ethhdr *eth; + + /* Shared settings */ + eseg->swp_outer_l3_offset = skb_network_offset(skb) / 2; + if (skb->protocol == htons(ETH_P_IPV6)) + eseg->swp_flags |= MLX5_ETH_WQE_SWP_OUTER_L3_IPV6; + + if (skb->inner_protocol_type == ENCAP_TYPE_IPPROTO) { + inner_ipproto = skb->inner_ipproto; + /* Set SWP additional flags for packet of type IP|UDP|PSP|[ TCP | UDP ] */ + switch (inner_ipproto) { + case IPPROTO_UDP: + eseg->swp_flags |= MLX5_ETH_WQE_SWP_INNER_L4_UDP; + fallthrough; + case IPPROTO_TCP: + eseg->swp_inner_l4_offset = skb_inner_transport_offset(skb) / 2; + break; + default: + break; + } + } else { + /* IP in IP tunneling like vxlan*/ + if (skb->inner_protocol_type != ENCAP_TYPE_ETHER) + return; + + eth = (struct ethhdr *)skb_inner_mac_header(skb); + switch (ntohs(eth->h_proto)) { + case ETH_P_IP: + inner_ipproto = ((struct iphdr *)((char *)skb->data + + skb_inner_network_offset(skb)))->protocol; + break; + case ETH_P_IPV6: + inner_ipproto = ((struct ipv6hdr *)((char *)skb->data + + skb_inner_network_offset(skb)))->nexthdr; + break; + default: + break; + } + + /* Tunnel(VXLAN TCP/UDP) over Transport Mode PSP i.e. PSP payload is vxlan tunnel */ + switch (inner_ipproto) { + case IPPROTO_UDP: + eseg->swp_flags |= MLX5_ETH_WQE_SWP_INNER_L4_UDP; + fallthrough; + case IPPROTO_TCP: + eseg->swp_inner_l3_offset = skb_inner_network_offset(skb) / 2; + eseg->swp_inner_l4_offset = + (skb->csum_start + skb->head - skb->data) / 2; + if (skb->protocol == htons(ETH_P_IPV6)) + eseg->swp_flags |= MLX5_ETH_WQE_SWP_INNER_L3_IPV6; + break; + default: + break; + } + + nisp_st->inner_ipproto = inner_ipproto; + } +} + +static bool mlx5e_nisp_set_state(struct mlx5e_priv *priv, + struct sk_buff *skb, + struct mlx5e_accel_tx_nisp_state *nisp_st) +{ + struct psp_assoc *pas; + bool ret = false; + + rcu_read_lock(); + pas = psp_skb_get_assoc_rcu(skb); + if (!pas) + goto out; + + ret = true; + nisp_st->tailen = PSP_TRL_SIZE; + nisp_st->spi = pas->tx.spi; + nisp_st->ver = pas->version; + memcpy(&nisp_st->keyid, pas->drv_data, sizeof(nisp_st->keyid)); + +out: + rcu_read_unlock(); + return ret; +} + +void mlx5e_nisp_tx_build_eseg(struct mlx5e_priv *priv, struct sk_buff *skb, + struct mlx5e_accel_tx_nisp_state *nisp_st, + struct mlx5_wqe_eth_seg *eseg) +{ + if (!mlx5_is_nisp_device(priv->mdev)) + return; + + if (unlikely(skb->protocol != htons(ETH_P_IP) && + skb->protocol != htons(ETH_P_IPV6))) + return; + + mlx5e_nisp_set_swp(skb, nisp_st, eseg); + /* Special WA for PSP LSO in ConnectX7 */ + eseg->swp_outer_l3_offset = 0; + eseg->swp_inner_l3_offset = 0; + + eseg->flow_table_metadata |= cpu_to_be32(nisp_st->keyid); + eseg->trailer |= cpu_to_be32(MLX5_ETH_WQE_INSERT_TRAILER) | + cpu_to_be32(MLX5_ETH_WQE_TRAILER_HDR_OUTER_L4_ASSOC); +} + +void mlx5e_nisp_handle_tx_wqe(struct mlx5e_tx_wqe *wqe, + struct mlx5e_accel_tx_nisp_state *nisp_st, + struct mlx5_wqe_inline_seg *inlseg) +{ + inlseg->byte_count = cpu_to_be32(nisp_st->tailen | MLX5_INLINE_SEG); +} + +static void psp_write_headers(struct net *net, struct sk_buff *skb, + __be32 spi, u8 ver, unsigned int udp_len, + __be16 sport) +{ + struct udphdr *uh = udp_hdr(skb); + struct psphdr *psph = (struct psphdr *)(uh + 1); + + uh->dest = htons(PSP_DEFAULT_UDP_PORT); + uh->source = udp_flow_src_port(net, skb, 0, 0, false); + uh->check = 0; + uh->len = htons(udp_len); + + psph->nexthdr = IPPROTO_TCP; + psph->hdrlen = PSP_HDRLEN_NOOPT; + psph->crypt_offset = 0; + psph->verfl = FIELD_PREP(PSPHDR_VERFL_VERSION, ver) | + FIELD_PREP(PSPHDR_VERFL_ONE, 1); + psph->spi = spi; + memset(&psph->iv, 0, sizeof(psph->iv)); +} + +/* Encapsulate a TCP packet with PSP by adding the UDP+PSP headers and filling + * them in. + */ +static bool psp_encapsulate(struct net *net, struct sk_buff *skb, + __be32 spi, u8 ver, __be16 sport) +{ + u32 network_len = skb_network_header_len(skb); + u32 ethr_len = skb_mac_header_len(skb); + u32 bufflen = ethr_len + network_len; + struct ipv6hdr *ip6; + + if (skb_cow_head(skb, PSP_ENCAP_HLEN)) + return false; + + skb_push(skb, PSP_ENCAP_HLEN); + skb->mac_header -= PSP_ENCAP_HLEN; + skb->network_header -= PSP_ENCAP_HLEN; + skb->transport_header -= PSP_ENCAP_HLEN; + memmove(skb->data, skb->data + PSP_ENCAP_HLEN, bufflen); + + ip6 = ipv6_hdr(skb); + skb_set_inner_ipproto(skb, IPPROTO_TCP); + ip6->nexthdr = IPPROTO_UDP; + be16_add_cpu(&ip6->payload_len, PSP_ENCAP_HLEN); + + skb_set_inner_transport_header(skb, skb_transport_offset(skb) + PSP_ENCAP_HLEN); + skb->encapsulation = 1; + psp_write_headers(net, skb, spi, ver, + skb->len - skb_transport_offset(skb), sport); + + return true; +} + +bool mlx5e_nisp_handle_tx_skb(struct net_device *netdev, + struct sk_buff *skb, + struct mlx5e_accel_tx_nisp_state *nisp_st) +{ + struct mlx5e_priv *priv = netdev_priv(netdev); + struct net *net = sock_net(skb->sk); + const struct ipv6hdr *ip6; + struct tcphdr *th; + + if (!mlx5e_nisp_set_state(priv, skb, nisp_st)) + return true; + + /* psp_encap of the packet */ + if (!psp_encapsulate(net, skb, nisp_st->spi, nisp_st->ver, 0)) { + kfree_skb_reason(skb, SKB_DROP_REASON_PSP_OUTPUT); + return false; + } + if (skb_is_gso(skb)) { + ip6 = ipv6_hdr(skb); + th = inner_tcp_hdr(skb); + + th->check = ~tcp_v6_check(skb_shinfo(skb)->gso_size + inner_tcp_hdrlen(skb), &ip6->saddr, + &ip6->daddr, 0); + } + + return true; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.h new file mode 100644 index 000000000000..1350a73c2019 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.h @@ -0,0 +1,96 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5E_NISP_RXTX_H__ +#define __MLX5E_NISP_RXTX_H__ + +#include +#include +#include +#include "en.h" +#include "en/txrx.h" + +struct mlx5e_accel_tx_nisp_state { + u32 tailen; + u32 keyid; + __be32 spi; + u8 inner_ipproto; + u8 ver; +}; + +#ifdef CONFIG_MLX5_EN_PSP +static inline bool mlx5e_psp_is_offload_state(struct mlx5e_accel_tx_nisp_state *nisp_state) +{ + return (nisp_state->tailen != 0); +} + +static inline bool mlx5e_psp_is_offload(struct sk_buff *skb, struct net_device *netdev) +{ + bool ret; + + rcu_read_lock(); + ret = !!psp_skb_get_assoc_rcu(skb); + rcu_read_unlock(); + return ret; +} + +bool mlx5e_nisp_handle_tx_skb(struct net_device *netdev, + struct sk_buff *skb, + struct mlx5e_accel_tx_nisp_state *nisp_st); + +void mlx5e_nisp_tx_build_eseg(struct mlx5e_priv *priv, struct sk_buff *skb, + struct mlx5e_accel_tx_nisp_state *nisp_st, + struct mlx5_wqe_eth_seg *eseg); + +void mlx5e_nisp_handle_tx_wqe(struct mlx5e_tx_wqe *wqe, + struct mlx5e_accel_tx_nisp_state *nisp_st, + struct mlx5_wqe_inline_seg *inlseg); + +static inline bool mlx5e_nisp_txwqe_build_eseg_csum(struct mlx5e_txqsq *sq, struct sk_buff *skb, + struct mlx5e_accel_tx_nisp_state *nisp_st, + struct mlx5_wqe_eth_seg *eseg) +{ + u8 inner_ipproto; + + if (!mlx5e_psp_is_offload_state(nisp_st)) + return false; + + inner_ipproto = nisp_st->inner_ipproto; + eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM; + if (inner_ipproto) { + eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM; + if (inner_ipproto == IPPROTO_TCP || inner_ipproto == IPPROTO_UDP) + eseg->cs_flags |= MLX5_ETH_WQE_L4_INNER_CSUM; + if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) + sq->stats->csum_partial_inner++; + } else if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) { + eseg->cs_flags |= MLX5_ETH_WQE_L4_INNER_CSUM; + sq->stats->csum_partial_inner++; + } + + return true; +} + +static inline unsigned int mlx5e_nisp_tx_ids_len(struct mlx5e_accel_tx_nisp_state *nisp_st) +{ + return nisp_st->tailen; +} +#else +static inline bool mlx5e_psp_is_offload_state(struct mlx5e_accel_tx_nisp_state *nisp_state) +{ + return false; +} + +static inline bool mlx5e_psp_is_offload(struct sk_buff *skb, struct net_device *netdev) +{ + return false; +} + +static inline bool mlx5e_nisp_txwqe_build_eseg_csum(struct mlx5e_txqsq *sq, struct sk_buff *skb, + struct mlx5e_accel_tx_nisp_state *nisp_st, + struct mlx5_wqe_eth_seg *eseg) +{ + return false; +} +#endif +#endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c index 099bf1078889..cc4d236a976f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c @@ -39,6 +39,7 @@ #include "ipoib/ipoib.h" #include "en_accel/en_accel.h" #include "en_accel/ipsec_rxtx.h" +#include "en_accel/nisp_rxtx.h" #include "en_accel/macsec.h" #include "en/ptp.h" #include @@ -120,6 +121,11 @@ mlx5e_txwqe_build_eseg_csum(struct mlx5e_txqsq *sq, struct sk_buff *skb, struct mlx5e_accel_tx_state *accel, struct mlx5_wqe_eth_seg *eseg) { +#ifdef CONFIG_MLX5_EN_PSP + if (unlikely(mlx5e_nisp_txwqe_build_eseg_csum(sq, skb, &accel->nisp_st, eseg))) + return; +#endif + if (unlikely(mlx5e_ipsec_txwqe_build_eseg_csum(sq, skb, eseg))) return; @@ -294,7 +300,7 @@ static void mlx5e_sq_xmit_prepare(struct mlx5e_txqsq *sq, struct sk_buff *skb, stats->packets++; } - attr->insz = mlx5e_accel_tx_ids_len(sq, accel); + attr->insz = mlx5e_accel_tx_ids_len(sq, skb, accel); stats->bytes += attr->num_bytes; } @@ -663,7 +669,7 @@ static void mlx5e_txwqe_build_eseg(struct mlx5e_priv *priv, struct mlx5e_txqsq * struct sk_buff *skb, struct mlx5e_accel_tx_state *accel, struct mlx5_wqe_eth_seg *eseg, u16 ihs) { - mlx5e_accel_tx_eseg(priv, skb, eseg, ihs); + mlx5e_accel_tx_eseg(priv, skb, accel, eseg, ihs); mlx5e_txwqe_build_eseg_csum(sq, skb, accel, eseg); if (unlikely(sq->ptpsq)) mlx5e_cqe_ts_id_eseg(sq->ptpsq, skb, eseg); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/psp_defs.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/psp_defs.h new file mode 100644 index 000000000000..7dd2aa90ed62 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/psp_defs.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef _LIB_PSP_DEFS_H +#define _LIB_PSP_DEFS_H + +/* PSP Security Payload (PSP) Header + * + * 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + * | Next Header | Hdr Ext Len | Crypt Offset | R |Version|V|1| + * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + * | Security Parameters Index (SPI) | + * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + * | | + * + Initialization Vector (IV) + + * | | + * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + * | Virtualization Key (VK) [Optional] | + * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + * | Pad to 8*N bytes [if needed] | + * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + */ + +/* total length of headers for PSP encapsulation (UDP + PSP) */ +#define PSP_ENCAP_HLEN (sizeof(struct udphdr) + sizeof(struct psphdr)) + +#endif /* _LIB_PSP_DEFS_H */ From patchwork Fri May 10 03:04:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660844 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3ACF914A4EE for ; Fri, 10 May 2024 03:04:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310293; cv=none; b=BaKJJKwQQUhWPMetTNdyrThzBPvDif33gw/dwamC5pneiC9nBxDrwWHDnkm9CEZP+1fHmNGgr5cFz+0hlOdocOAH5bwujGRsL4bBJMxfOzd3rnqgpz+xRtsP8SCzmC0IHw7mgv1Q3DA31J8HeDkhcbSuFAIp60BBL4SX06+1v8U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310293; c=relaxed/simple; bh=63DSVTkQGmXCvKdZNlH2ICLDQwES5+pYh9RQl/GBqHI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SF8cN1IVkaH0PvBtr349j84bu3a8uow9DhhL/SEvhQ//cBGvDuwjCqlr1lgY/HYkI+WjBug0olm3PbK0xPRjcwggv+KZtBXW8pyGX1WtSLaV+X6t7tpG+K8BE4uQgXD/2t6fTWqiq37gDobH+zXuyuXdxNEHcAUucj/9UB/beZw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GkaQUdDC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GkaQUdDC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4490DC32783; Fri, 10 May 2024 03:04:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310292; bh=63DSVTkQGmXCvKdZNlH2ICLDQwES5+pYh9RQl/GBqHI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GkaQUdDCVR/mu6vgtNCKzJZtFZOQOXydF4Xz4iDELtRQTpN9pkDLG5xYIqQZcRV8f SLfX6XHzf3LUiz6J8cyTBG0A9JOnuHNqYHmSy4BtMp0Q+Z+isJEQTD1ZFlzQ8QxzGR EH34VYL7gd4YPZgU1qs65UG07v2DF/A3zeS7GIKKhtecA/Y1xWUUTc4hRz0U4ApCsi 8PEZeo/OC12U6NXgagwfGZuCMBoXxuuI272sH5jdC1hgTtigGA7O99ZVH4+t3Lqx83 NtavyQ680lnzR4UysTc9oxujyFLuYi8mNNOmyVmC7M29IJchcGrcGuHRhBv4rsbKHt hXws6zeC/KB3A== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Raed Salem , Jakub Kicinski Subject: [RFC net-next 12/15] net/mlx5e: Add PSP steering in local NIC RX Date: Thu, 9 May 2024 20:04:32 -0700 Message-ID: <20240510030435.120935-13-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Raed Salem Introduce decrypt FT, the RX error FT, and the default rules. The PSP (PSP) RX decrypt flow table is pointed by the TTC (Traffic Type Classifier) UDP steering rules. The decrypt flow table has two flow groups. The first flow group keeps the decrypt steering rule programmed always when PSP packet is recognized using the dedicated udp destenation port number 1000, if packet is decrypted then a PSP marker is set in metadata_regB[30]. The second flow group has a default rule to forward all non-offloaded PSP packet to the TTC UDP default RSS TIR. The RX error flow table is the destination of the decrypt steering rules in the PSP RX decrypt flow table. It has two fixed rule one with single copy action that copies nisp_syndrome to metadata_regB[23:29]. The PSP marker and syndrome is used to filter out non-nisp packet and to return the PSP crypto offload status in Rx flow. The marker is used to identify such packet in driver so the driver could set SKB PSP metadata. The destination of RX error flow table is the TTC UDP default RSS TIR. The second rule will drop packets that failed to be decrypted (like in case illegal SPI or expired SPI is used). Signed-off-by: Raed Salem Signed-off-by: Rahul Rameshbabu Signed-off-by: Jakub Kicinski --- .../net/ethernet/mellanox/mlx5/core/en/fs.h | 2 +- .../mellanox/mlx5/core/en_accel/nisp_fs.c | 481 +++++++++++++++++- 2 files changed, 476 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h index 4d6225e0eec7..23af74e4f8c5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h @@ -83,7 +83,7 @@ enum { #ifdef CONFIG_MLX5_EN_ARFS MLX5E_ARFS_FT_LEVEL = MLX5E_INNER_TTC_FT_LEVEL + 1, #endif -#ifdef CONFIG_MLX5_EN_IPSEC +#if defined(CONFIG_MLX5_EN_IPSEC) || defined(CONFIG_MLX5_EN_PSP) MLX5E_ACCEL_FS_POL_FT_LEVEL = MLX5E_INNER_TTC_FT_LEVEL + 1, MLX5E_ACCEL_FS_ESP_FT_LEVEL, MLX5E_ACCEL_FS_ESP_FT_ERR_LEVEL, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c index 5d2ce83db7cc..11f583d13bdd 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c @@ -8,6 +8,12 @@ #include "en_accel/nisp_fs.h" #include "en_accel/nisp.h" +enum accel_fs_nisp_type { + ACCEL_FS_NISP4, + ACCEL_FS_NISP6, + ACCEL_FS_NISP_NUM_TYPES, +}; + struct mlx5e_nisp_tx { struct mlx5_flow_namespace *ns; struct mlx5_flow_table *ft; @@ -17,14 +23,15 @@ struct mlx5e_nisp_tx { u32 refcnt; }; -struct mlx5e_nisp_fs { - struct mlx5_core_dev *mdev; - struct mlx5e_nisp_tx *tx_fs; - struct mlx5e_flow_steering *fs; -}; - enum accel_nisp_rule_action { ACCEL_NISP_RULE_ACTION_ENCRYPT, + ACCEL_NISP_RULE_ACTION_DECRYPT, +}; + +enum accel_nisp_syndrome { + NISP_OK = 0, + NISP_ICV_FAIL, + NISP_BAD_TRAILER, }; struct mlx5e_accel_nisp_rule { @@ -32,6 +39,216 @@ struct mlx5e_accel_nisp_rule { u8 action; }; +struct mlx5e_nisp_rx_err { + struct mlx5_flow_table *ft; + struct mlx5_flow_handle *rule; + struct mlx5_flow_handle *drop_rule; + struct mlx5_modify_hdr *copy_modify_hdr; +}; + +struct mlx5e_accel_fs_nisp_prot { + struct mlx5_flow_table *ft; + struct mlx5_flow_group *miss_group; + struct mlx5_flow_handle *miss_rule; + struct mlx5_flow_destination default_dest; + struct mlx5e_nisp_rx_err rx_err; + u32 refcnt; + struct mutex prot_mutex; /* protect ESP4/ESP6 protocol */ + struct mlx5_flow_handle *def_rule; +}; + +struct mlx5e_accel_fs_nisp { + struct mlx5e_accel_fs_nisp_prot fs_prot[ACCEL_FS_NISP_NUM_TYPES]; +}; + +struct mlx5e_nisp_fs { + struct mlx5_core_dev *mdev; + struct mlx5e_nisp_tx *tx_fs; + /* Rx manage */ + struct mlx5e_flow_steering *fs; + struct mlx5e_accel_fs_nisp *rx_fs; +}; + +/* NISP RX flow steering */ +static enum mlx5_traffic_types fs_nisp2tt(enum accel_fs_nisp_type i) +{ + if (i == ACCEL_FS_NISP4) + return MLX5_TT_IPV4_UDP; + + return MLX5_TT_IPV6_UDP; +} + +static void accel_nisp_fs_rx_err_del_rules(struct mlx5e_nisp_fs *fs, + struct mlx5e_nisp_rx_err *rx_err) +{ + if (rx_err->drop_rule) { + mlx5_del_flow_rules(rx_err->drop_rule); + rx_err->drop_rule = NULL; + } + + if (rx_err->rule) { + mlx5_del_flow_rules(rx_err->rule); + rx_err->rule = NULL; + } + + if (rx_err->copy_modify_hdr) { + mlx5_modify_header_dealloc(fs->mdev, rx_err->copy_modify_hdr); + rx_err->copy_modify_hdr = NULL; + } +} + +static void accel_nisp_fs_rx_err_destroy_ft(struct mlx5e_nisp_fs *fs, + struct mlx5e_nisp_rx_err *rx_err) +{ + accel_nisp_fs_rx_err_del_rules(fs, rx_err); + + if (rx_err->ft) { + mlx5_destroy_flow_table(rx_err->ft); + rx_err->ft = NULL; + } +} + +static void accel_nisp_setup_syndrome_match(struct mlx5_flow_spec *spec, + enum accel_nisp_syndrome syndrome) +{ + void *misc_params_2; + + spec->match_criteria_enable |= MLX5_MATCH_MISC_PARAMETERS_2; + misc_params_2 = MLX5_ADDR_OF(fte_match_param, spec->match_criteria, misc_parameters_2); + MLX5_SET_TO_ONES(fte_match_set_misc2, misc_params_2, nisp_syndrome); + misc_params_2 = MLX5_ADDR_OF(fte_match_param, spec->match_value, misc_parameters_2); + MLX5_SET(fte_match_set_misc2, misc_params_2, nisp_syndrome, syndrome); +} + +static int accel_nisp_fs_rx_err_add_rule(struct mlx5e_nisp_fs *fs, + struct mlx5e_accel_fs_nisp_prot *fs_prot, + struct mlx5e_nisp_rx_err *rx_err) +{ + u8 action[MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto)] = {}; + struct mlx5_core_dev *mdev = fs->mdev; + struct mlx5_flow_act flow_act = {}; + struct mlx5_modify_hdr *modify_hdr; + struct mlx5_flow_handle *fte; + struct mlx5_flow_spec *spec; + int err = 0; + + spec = kzalloc(sizeof(*spec), GFP_KERNEL); + if (!spec) + return -ENOMEM; + + /* Action to copy 7 bit nisp_syndrome to regB[23:29] */ + MLX5_SET(copy_action_in, action, action_type, MLX5_ACTION_TYPE_COPY); + MLX5_SET(copy_action_in, action, src_field, MLX5_ACTION_IN_FIELD_NISP_SYNDROME); + MLX5_SET(copy_action_in, action, src_offset, 0); + MLX5_SET(copy_action_in, action, length, 7); + MLX5_SET(copy_action_in, action, dst_field, MLX5_ACTION_IN_FIELD_METADATA_REG_B); + MLX5_SET(copy_action_in, action, dst_offset, 23); + + modify_hdr = mlx5_modify_header_alloc(mdev, MLX5_FLOW_NAMESPACE_KERNEL, + 1, action); + if (IS_ERR(modify_hdr)) { + err = PTR_ERR(modify_hdr); + mlx5_core_err(mdev, + "fail to alloc nisp copy modify_header_id err=%d\n", err); + goto out_spec; + } + + accel_nisp_setup_syndrome_match(spec, NISP_OK); + /* create fte */ + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_MOD_HDR | + MLX5_FLOW_CONTEXT_ACTION_FWD_DEST; + flow_act.modify_hdr = modify_hdr; + fte = mlx5_add_flow_rules(rx_err->ft, spec, &flow_act, + &fs_prot->default_dest, 1); + if (IS_ERR(fte)) { + err = PTR_ERR(fte); + mlx5_core_err(mdev, "fail to add nisp rx err copy rule err=%d\n", err); + goto out; + } + rx_err->rule = fte; + + /* add default drop rule */ + memset(spec, 0, sizeof(*spec)); + memset(&flow_act, 0, sizeof(flow_act)); + /* create fte */ + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_DROP; + fte = mlx5_add_flow_rules(rx_err->ft, spec, &flow_act, NULL, 0); + if (IS_ERR(fte)) { + err = PTR_ERR(fte); + mlx5_core_err(mdev, "fail to add nisp rx err drop rule err=%d\n", err); + goto out_drop_rule; + } + rx_err->drop_rule = fte; + rx_err->copy_modify_hdr = modify_hdr; + + goto out_spec; + +out_drop_rule: + mlx5_del_flow_rules(rx_err->rule); + rx_err->rule = NULL; +out: + mlx5_modify_header_dealloc(mdev, modify_hdr); +out_spec: + kfree(spec); + return err; +} + +static int accel_nisp_fs_rx_err_create_ft(struct mlx5e_nisp_fs *fs, + struct mlx5e_accel_fs_nisp_prot *fs_prot, + struct mlx5e_nisp_rx_err *rx_err) +{ + struct mlx5_flow_namespace *ns = mlx5e_fs_get_ns(fs->fs, false); + struct mlx5_flow_table_attr ft_attr = {}; + struct mlx5_flow_table *ft; + int err; + + ft_attr.max_fte = 2; + ft_attr.autogroup.max_num_groups = 2; + ft_attr.level = MLX5E_ACCEL_FS_ESP_FT_ERR_LEVEL; // MLX5E_ACCEL_FS_TCP_FT_LEVEL + ft_attr.prio = MLX5E_NIC_PRIO; + ft = mlx5_create_auto_grouped_flow_table(ns, &ft_attr); + if (IS_ERR(ft)) { + err = PTR_ERR(ft); + mlx5_core_err(fs->mdev, "fail to create nisp rx inline ft err=%d\n", err); + return err; + } + + rx_err->ft = ft; + err = accel_nisp_fs_rx_err_add_rule(fs, fs_prot, rx_err); + if (err) + goto out_err; + + return 0; + +out_err: + mlx5_destroy_flow_table(ft); + rx_err->ft = NULL; + return err; +} + +static void accel_nisp_fs_rx_fs_destroy(struct mlx5e_accel_fs_nisp_prot *fs_prot) +{ + if (fs_prot->def_rule) { + mlx5_del_flow_rules(fs_prot->def_rule); + fs_prot->def_rule = NULL; + } + + if (fs_prot->miss_rule) { + mlx5_del_flow_rules(fs_prot->miss_rule); + fs_prot->miss_rule = NULL; + } + + if (fs_prot->miss_group) { + mlx5_destroy_flow_group(fs_prot->miss_group); + fs_prot->miss_group = NULL; + } + + if (fs_prot->ft) { + mlx5_destroy_flow_table(fs_prot->ft); + fs_prot->ft = NULL; + } +} + static void setup_fte_udp_psp(struct mlx5_flow_spec *spec, u16 udp_port) { spec->match_criteria_enable |= MLX5_MATCH_OUTER_HEADERS; @@ -41,6 +258,251 @@ static void setup_fte_udp_psp(struct mlx5_flow_spec *spec, u16 udp_port) MLX5_SET(fte_match_set_lyr_2_4, spec->match_value, ip_protocol, IPPROTO_UDP); } +static int accel_nisp_fs_rx_create_ft(struct mlx5e_nisp_fs *fs, + struct mlx5e_accel_fs_nisp_prot *fs_prot) +{ + struct mlx5_flow_namespace *ns = mlx5e_fs_get_ns(fs->fs, false); + u8 action[MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto)] = {}; + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + struct mlx5_modify_hdr *modify_hdr = NULL; + struct mlx5_flow_table_attr ft_attr = {}; + struct mlx5_flow_destination dest = {}; + struct mlx5_core_dev *mdev = fs->mdev; + struct mlx5_flow_group *miss_group; + MLX5_DECLARE_FLOW_ACT(flow_act); + struct mlx5_flow_handle *rule; + struct mlx5_flow_spec *spec; + struct mlx5_flow_table *ft; + u32 *flow_group_in; + int err = 0; + + flow_group_in = kvzalloc(inlen, GFP_KERNEL); + spec = kvzalloc(sizeof(*spec), GFP_KERNEL); + if (!flow_group_in || !spec) { + err = -ENOMEM; + goto out; + } + + /* Create FT */ + ft_attr.max_fte = 2; + ft_attr.level = MLX5E_ACCEL_FS_ESP_FT_LEVEL; + ft_attr.prio = MLX5E_NIC_PRIO; + ft_attr.autogroup.num_reserved_entries = 1; + ft_attr.autogroup.max_num_groups = 1; + ft = mlx5_create_auto_grouped_flow_table(ns, &ft_attr); + if (IS_ERR(ft)) { + err = PTR_ERR(ft); + mlx5_core_err(mdev, "fail to create nisp rx ft err=%d\n", err); + goto out_err; + } + fs_prot->ft = ft; + + /* Create miss_group */ + MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, ft->max_fte - 1); + MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, ft->max_fte - 1); + miss_group = mlx5_create_flow_group(ft, flow_group_in); + if (IS_ERR(miss_group)) { + err = PTR_ERR(miss_group); + mlx5_core_err(mdev, "fail to create nisp rx miss_group err=%d\n", err); + goto out_err; + } + fs_prot->miss_group = miss_group; + + /* Create miss rule */ + rule = mlx5_add_flow_rules(ft, spec, &flow_act, &fs_prot->default_dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + mlx5_core_err(mdev, "fail to create nisp rx miss_rule err=%d\n", err); + goto out_err; + } + fs_prot->miss_rule = rule; + + /* Add default Rx Nisp rule */ + setup_fte_udp_psp(spec, PSP_DEFAULT_UDP_PORT); + flow_act.crypto.type = MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_NISP; + /* Set bit[31, 30] NISP marker */ + /* Set bit[29-23] nisp_syndrome is set in error FT */ +#define MLX5E_NISP_MARKER_BIT (BIT(30) | BIT(31)) + MLX5_SET(set_action_in, action, action_type, MLX5_ACTION_TYPE_SET); + MLX5_SET(set_action_in, action, field, MLX5_ACTION_IN_FIELD_METADATA_REG_B); + MLX5_SET(set_action_in, action, data, MLX5E_NISP_MARKER_BIT); + MLX5_SET(set_action_in, action, offset, 0); + MLX5_SET(set_action_in, action, length, 32); + + modify_hdr = mlx5_modify_header_alloc(mdev, MLX5_FLOW_NAMESPACE_KERNEL, 1, action); + if (IS_ERR(modify_hdr)) { + err = PTR_ERR(modify_hdr); + mlx5_core_err(mdev, "fail to alloc nisp set modify_header_id err=%d\n", err); + modify_hdr = NULL; + goto out_err; + } + + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST | + MLX5_FLOW_CONTEXT_ACTION_CRYPTO_DECRYPT | + MLX5_FLOW_CONTEXT_ACTION_MOD_HDR; + flow_act.modify_hdr = modify_hdr; + dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; + dest.ft = fs_prot->rx_err.ft; + rule = mlx5_add_flow_rules(fs_prot->ft, spec, &flow_act, &dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + mlx5_core_err(mdev, + "fail to add nisp rule Rx dycrption, err=%d, flow_act.action = %#04X\n", + err, flow_act.action); + goto out_err; + } + + fs_prot->def_rule = rule; + goto out; + +out_err: + accel_nisp_fs_rx_fs_destroy(fs_prot); +out: + kvfree(flow_group_in); + kvfree(spec); + return err; +} + +static int accel_nisp_fs_rx_destroy(struct mlx5e_nisp_fs *fs, enum accel_fs_nisp_type type) +{ + struct mlx5e_accel_fs_nisp_prot *fs_prot; + struct mlx5e_accel_fs_nisp *accel_nisp; + + accel_nisp = fs->rx_fs; + + /* The netdev unreg already happened, so all offloaded rule are already removed */ + fs_prot = &accel_nisp->fs_prot[type]; + + accel_nisp_fs_rx_fs_destroy(fs_prot); + + accel_nisp_fs_rx_err_destroy_ft(fs, &fs_prot->rx_err); + + return 0; +} + +static int accel_nisp_fs_rx_create(struct mlx5e_nisp_fs *fs, enum accel_fs_nisp_type type) +{ + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs->fs, false); + struct mlx5e_accel_fs_nisp_prot *fs_prot; + struct mlx5e_accel_fs_nisp *accel_nisp; + int err; + + accel_nisp = fs->rx_fs; + fs_prot = &accel_nisp->fs_prot[type]; + + fs_prot->default_dest = mlx5_ttc_get_default_dest(ttc, fs_nisp2tt(type)); + + err = accel_nisp_fs_rx_err_create_ft(fs, fs_prot, &fs_prot->rx_err); + if (err) + return err; + + err = accel_nisp_fs_rx_create_ft(fs, fs_prot); + if (err) + accel_nisp_fs_rx_err_destroy_ft(fs, &fs_prot->rx_err); + + return err; +} + +static int accel_nisp_fs_rx_ft_get(struct mlx5e_nisp_fs *fs, enum accel_fs_nisp_type type) +{ + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs->fs, false); + struct mlx5e_accel_fs_nisp_prot *fs_prot; + struct mlx5_flow_destination dest = {}; + struct mlx5e_accel_fs_nisp *accel_nisp; + int err = 0; + + if (!fs || !fs->rx_fs) + return -EINVAL; + + accel_nisp = fs->rx_fs; + fs_prot = &accel_nisp->fs_prot[type]; + mutex_lock(&fs_prot->prot_mutex); + if (fs_prot->refcnt++) + goto out; + + /* create FT */ + err = accel_nisp_fs_rx_create(fs, type); + if (err) { + fs_prot->refcnt--; + goto out; + } + + /* connect */ + dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; + dest.ft = fs_prot->ft; + mlx5_ttc_fwd_dest(ttc, fs_nisp2tt(type), &dest); + +out: + mutex_unlock(&fs_prot->prot_mutex); + return err; +} + +static void accel_nisp_fs_rx_ft_put(struct mlx5e_nisp_fs *fs, enum accel_fs_nisp_type type) +{ + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs->fs, false); + struct mlx5e_accel_fs_nisp_prot *fs_prot; + struct mlx5e_accel_fs_nisp *accel_nisp; + + accel_nisp = fs->rx_fs; + fs_prot = &accel_nisp->fs_prot[type]; + mutex_lock(&fs_prot->prot_mutex); + if (--fs_prot->refcnt) + goto out; + + /* disconnect */ + mlx5_ttc_fwd_default_dest(ttc, fs_nisp2tt(type)); + + /* remove FT */ + accel_nisp_fs_rx_destroy(fs, type); + +out: + mutex_unlock(&fs_prot->prot_mutex); +} + +static void accel_nisp_fs_cleanup_rx(struct mlx5e_nisp_fs *fs) +{ + struct mlx5e_accel_fs_nisp_prot *fs_prot; + struct mlx5e_accel_fs_nisp *accel_nisp; + enum accel_fs_nisp_type i; + + if (!fs->rx_fs) + return; + + for (i = 0; i < ACCEL_FS_NISP_NUM_TYPES; i++) + accel_nisp_fs_rx_ft_put(fs, i); + + accel_nisp = fs->rx_fs; + for (i = 0; i < ACCEL_FS_NISP_NUM_TYPES; i++) { + fs_prot = &accel_nisp->fs_prot[i]; + mutex_destroy(&fs_prot->prot_mutex); + WARN_ON(fs_prot->refcnt); + } + kfree(fs->rx_fs); + fs->rx_fs = NULL; +} + +static int accel_nisp_fs_init_rx(struct mlx5e_nisp_fs *fs) +{ + struct mlx5e_accel_fs_nisp_prot *fs_prot; + struct mlx5e_accel_fs_nisp *accel_nisp; + enum accel_fs_nisp_type i; + + accel_nisp = kzalloc(sizeof(*accel_nisp), GFP_KERNEL); + if (!accel_nisp) + return -ENOMEM; + + for (i = 0; i < ACCEL_FS_NISP_NUM_TYPES; i++) { + fs_prot = &accel_nisp->fs_prot[i]; + mutex_init(&fs_prot->prot_mutex); + } + + for (i = 0; i < ACCEL_FS_NISP_NUM_TYPES; i++) + accel_nisp_fs_rx_ft_get(fs, ACCEL_FS_NISP4); + + fs->rx_fs = accel_nisp; + return 0; +} + static int accel_nisp_fs_tx_create_ft_table(struct mlx5e_nisp_fs *fs) { int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); @@ -207,6 +669,7 @@ int mlx5_accel_nisp_fs_init_tx_tables(struct mlx5e_priv *priv) void mlx5e_accel_nisp_fs_cleanup(struct mlx5e_nisp_fs *fs) { + accel_nisp_fs_cleanup_rx(fs); accel_nisp_fs_cleanup_tx(fs); kfree(fs); } @@ -226,8 +689,14 @@ struct mlx5e_nisp_fs *mlx5e_accel_nisp_fs_init(struct mlx5e_priv *priv) goto err_tx; fs->fs = priv->fs; + err = accel_nisp_fs_init_rx(fs); + if (err) + goto err_rx; return fs; + +err_rx: + accel_nisp_fs_cleanup_tx(fs); err_tx: kfree(fs); return ERR_PTR(err); From patchwork Fri May 10 03:04:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660845 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D29FE14A612 for ; Fri, 10 May 2024 03:04:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310293; cv=none; b=C4zKypxR2oGHYhJYIOj36TKfXUZ7rIr0PgMcf61czbt18H+/t0IAIe4qKMpf52Toc+B3lxLMEM4C934liQEt9oeLZ1KNzJMQvCjMSy+KA4MF17hjhDoiDKeymp6bVu+R3vbw1D+vtfz0mleyuoafHvViyku1oH4U1rVIqayyLYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310293; c=relaxed/simple; bh=AyNN56ie4UUGTQFe8agp412wqnS+yq5EDX7OkOfs9n8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=USg8iMDwszN4HSI0P/MTSRwsVjn6YkUNIS1lbmWbFlUIcTCz3UVsn8tJf4wTdztXAuEIWTrwDAgTYlnENx/+goa/vLMmD2pnavEbIw4LYMy+RS+UhzjuAaxRue6/rdV2+z/kFblL0xPPJYQfLnLysP+WjXXh4ExEqDDglYcdb3s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oiSt9SRH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oiSt9SRH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F08D4C3277B; Fri, 10 May 2024 03:04:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310293; bh=AyNN56ie4UUGTQFe8agp412wqnS+yq5EDX7OkOfs9n8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oiSt9SRHz9WaMX/07rB7HhYSSBrvOhGWKtFoupvA61E8byLEkCrVPhl6ggzgK7jS+ xIIzDURE7NkmRhp+4BO66MWHDqYP/UObNRgwJDqT8WUpOdX5rEJb/YkefZoH247V2Y 4CX4bqYXktV7aDMn+pRSqSS+UEzXJ1rr3JxKD2I55bdaE2msBjGABNon6u2BCQ2LIm mVBTQijqCAKYo+udgl10PmxX5bUzdIphwIGcRdxcYRmizT47YLmTcUeC6oumCJv5HQ rLbXUjTuaK7mv7uvaiWF9bihe5U2V8RAnpmB9hAKPyWGdxx7mYLKn6ZICWj4vqSEZ0 DG7lLlJIxMOUA== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Raed Salem , Jakub Kicinski Subject: [RFC net-next 13/15] net/mlx5e: Configure PSP Rx flow steering rules Date: Thu, 9 May 2024 20:04:33 -0700 Message-ID: <20240510030435.120935-14-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Raed Salem Set the Rx PSP flow steering rule where PSP packet is identified and decrypted using the dedicated UDP destination port number 1000. If packet is decrypted then a PSP marker and syndrome are added to metadata so SW can use it later on in Rx data path. The rule is set as part of init_rx netdev profile implementation. Signed-off-by: Raed Salem Signed-off-by: Rahul Rameshbabu Signed-off-by: Jakub Kicinski --- .../mellanox/mlx5/core/en_accel/en_accel.h | 14 +++++- .../mellanox/mlx5/core/en_accel/nisp_fs.c | 46 ++++++++++++++++--- .../mellanox/mlx5/core/en_accel/nisp_fs.h | 7 +++ 3 files changed, 60 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h index cea997847fa4..38d2c73a0f79 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h @@ -231,12 +231,24 @@ static inline void mlx5e_accel_tx_finish(struct mlx5e_txqsq *sq, static inline int mlx5e_accel_init_rx(struct mlx5e_priv *priv) { - return mlx5e_ktls_init_rx(priv); + int err; + + err = mlx5_accel_nisp_fs_init_rx_tables(priv); + if (err) + goto out; + + err = mlx5e_ktls_init_rx(priv); + if (err) + mlx5_accel_nisp_fs_cleanup_rx_tables(priv); + +out: + return err; } static inline void mlx5e_accel_cleanup_rx(struct mlx5e_priv *priv) { mlx5e_ktls_cleanup_rx(priv); + mlx5_accel_nisp_fs_cleanup_rx_tables(priv); } static inline int mlx5e_accel_init_tx(struct mlx5e_priv *priv) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c index 11f583d13bdd..0d80f646e5b8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c @@ -468,9 +468,6 @@ static void accel_nisp_fs_cleanup_rx(struct mlx5e_nisp_fs *fs) if (!fs->rx_fs) return; - for (i = 0; i < ACCEL_FS_NISP_NUM_TYPES; i++) - accel_nisp_fs_rx_ft_put(fs, i); - accel_nisp = fs->rx_fs; for (i = 0; i < ACCEL_FS_NISP_NUM_TYPES; i++) { fs_prot = &accel_nisp->fs_prot[i]; @@ -496,13 +493,50 @@ static int accel_nisp_fs_init_rx(struct mlx5e_nisp_fs *fs) mutex_init(&fs_prot->prot_mutex); } - for (i = 0; i < ACCEL_FS_NISP_NUM_TYPES; i++) - accel_nisp_fs_rx_ft_get(fs, ACCEL_FS_NISP4); - fs->rx_fs = accel_nisp; + return 0; } +void mlx5_accel_nisp_fs_cleanup_rx_tables(struct mlx5e_priv *priv) +{ + int i; + + if (!priv->nisp) + return; + + for (i = 0; i < ACCEL_FS_NISP_NUM_TYPES; i++) + accel_nisp_fs_rx_ft_put(priv->nisp->fs, i); +} + +int mlx5_accel_nisp_fs_init_rx_tables(struct mlx5e_priv *priv) +{ + enum accel_fs_nisp_type i; + struct mlx5e_nisp_fs *fs; + int err; + + if (!priv->nisp) + return 0; + + fs = priv->nisp->fs; + for (i = 0; i < ACCEL_FS_NISP_NUM_TYPES; i++) { + err = accel_nisp_fs_rx_ft_get(fs, i); + if (err) + goto out_err; + } + + return 0; + +out_err: + i--; + while (i >= 0) { + accel_nisp_fs_rx_ft_put(fs, i); + --i; + } + + return err; +} + static int accel_nisp_fs_tx_create_ft_table(struct mlx5e_nisp_fs *fs) { int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.h index 11cdc447a401..8c44dd51317c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.h @@ -12,6 +12,8 @@ struct mlx5e_nisp_fs *mlx5e_accel_nisp_fs_init(struct mlx5e_priv *priv); void mlx5e_accel_nisp_fs_cleanup(struct mlx5e_nisp_fs *fs); int mlx5_accel_nisp_fs_init_tx_tables(struct mlx5e_priv *priv); void mlx5_accel_nisp_fs_cleanup_tx_tables(struct mlx5e_priv *priv); +int mlx5_accel_nisp_fs_init_rx_tables(struct mlx5e_priv *priv); +void mlx5_accel_nisp_fs_cleanup_rx_tables(struct mlx5e_priv *priv); #else static inline int mlx5_accel_nisp_fs_init_tx_tables(struct mlx5e_priv *priv) { @@ -19,5 +21,10 @@ static inline int mlx5_accel_nisp_fs_init_tx_tables(struct mlx5e_priv *priv) } static inline void mlx5_accel_nisp_fs_cleanup_tx_tables(struct mlx5e_priv *priv) { } +static inline int mlx5_accel_nisp_fs_init_rx_tables(struct mlx5e_priv *priv) +{ + return 0; +} +static inline void mlx5_accel_nisp_fs_cleanup_rx_tables(struct mlx5e_priv *priv) { } #endif /* CONFIG_MLX5_EN_PSP */ #endif /* __MLX5_NISP_FS_H__ */ From patchwork Fri May 10 03:04:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660846 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9072614BF89 for ; Fri, 10 May 2024 03:04:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310294; cv=none; b=MJsIc/R9elNvrsBOWcl583Vd5JG/mi2xMoxgw/WUNASPD1T7Yv8JDVX7ODiC15AVl7kvI7lTbv2sN3w2xa3ZyCI/AoiFIDRFVWh4V8/PZPUZ+15Dsu9DDpjsz0dmEK1916UgFEGTX17NT3cQUSGCU2wf7UgmpgDfzQ0sQc2vs/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310294; c=relaxed/simple; bh=uqerNasafMvLz+XQH8HDuoRTvTgcSKRMJ5oF0hiHI6U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tqY5iUGGGgNz82MwANt2BusEUv4BbycfyGVr/oj49uicg/KzdyZlZjm52swpX5wr/lyrSHhEQeMc2hG0dbJO4HC5L9vqo+ZoNerH37iwQCg8PlQy2RuOezULZOkO4lhf7IsZ+aOrxjnVwNNtWhoEgO7ap0CWHgEPaEscPdyk41c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XYcBnEqf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XYcBnEqf" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A39B6C2BBFC; Fri, 10 May 2024 03:04:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310294; bh=uqerNasafMvLz+XQH8HDuoRTvTgcSKRMJ5oF0hiHI6U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XYcBnEqf30wGPQ17Xg4+Ky+43fl222rBwDvSqb2r+pRdTmi22H7iyggyG9tNgxk/r YxFh3AY1eRwiTOCeiuXBLgbLKKBgFBvDSJ9dpvE/zMajtZ7cRWHdm1Yj5gBDr+B6bq OJw2NTJuqjumdB93t9PJgiSSXHvhQiUT0IVbiCLkqNJr8w/6mKXiEIDUg999KKBu9/ 8NOY6e0izzUx9+pcxDMmrDst5yJYaS1jfYLLUQEcmzq2RRrpjaDBWhiJOwX7j61c4a hIx7HHUODc9aBqL39NyiX5OMlLf9YwnGsrEjuQQJwLRFDcgnC3UbcV2hjlA5sEN3nc R8qCnApxVvqBA== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Raed Salem , Jakub Kicinski Subject: [RFC net-next 14/15] net/mlx5e: Add Rx data path offload Date: Thu, 9 May 2024 20:04:34 -0700 Message-ID: <20240510030435.120935-15-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Raed Salem On receive flow inspect received packets for PSP offload indication using the cqe, for PSP offloaded packets set SKB PSP metadata i.e spi, header length and key generation number to stack for further processing. Signed-off-by: Raed Salem Signed-off-by: Rahul Rameshbabu Signed-off-by: Jakub Kicinski --- .../mellanox/mlx5/core/en_accel/ipsec_rxtx.h | 2 +- .../mellanox/mlx5/core/en_accel/nisp_rxtx.c | 79 +++++++++++++++++++ .../mellanox/mlx5/core/en_accel/nisp_rxtx.h | 28 +++++++ .../net/ethernet/mellanox/mlx5/core/en_rx.c | 10 +++ 4 files changed, 118 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h index 82064614846f..9f025c80a6ef 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h @@ -40,7 +40,7 @@ #include "en/txrx.h" /* Bit31: IPsec marker, Bit30: reserved, Bit29-24: IPsec syndrome, Bit23-0: IPsec obj id */ -#define MLX5_IPSEC_METADATA_MARKER(metadata) (((metadata) >> 31) & 0x1) +#define MLX5_IPSEC_METADATA_MARKER(metadata) ((((metadata) >> 30) & 0x3) == 0x2) #define MLX5_IPSEC_METADATA_SYNDROM(metadata) (((metadata) >> 24) & GENMASK(5, 0)) #define MLX5_IPSEC_METADATA_HANDLE(metadata) ((metadata) & GENMASK(23, 0)) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.c index c719b2916677..17f42b8d9fd8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.c @@ -15,6 +15,12 @@ #include "en_accel/nisp.h" #include "lib/psp_defs.h" +enum { + MLX5E_NISP_OFFLOAD_RX_SYNDROME_DECRYPTED, + MLX5E_NISP_OFFLOAD_RX_SYNDROME_AUTH_FAILED, + MLX5E_NISP_OFFLOAD_RX_SYNDROME_BAD_TRAILER, +}; + static void mlx5e_nisp_set_swp(struct sk_buff *skb, struct mlx5e_accel_tx_nisp_state *nisp_st, struct mlx5_wqe_eth_seg *eseg) @@ -114,6 +120,79 @@ static bool mlx5e_nisp_set_state(struct mlx5e_priv *priv, return ret; } +void mlx5e_nisp_csum_complete(struct net_device *netdev, struct sk_buff *skb) +{ + pskb_trim(skb, skb->len - PSP_TRL_SIZE); +} + +/* Receive handler for PSP packets. + * + * Presently it accepts only already-authenticated packets and does not + * support optional fields, such as virtualization cookies. + */ +static int psp_rcv(struct sk_buff *skb) +{ + const struct psphdr *psph; + int depth = 0, end_depth; + struct psp_skb_ext *pse; + struct ipv6hdr *ipv6h; + struct ethhdr *eth; + __be16 proto; + u32 spi; + + eth = (struct ethhdr *)(skb->data); + proto = __vlan_get_protocol(skb, eth->h_proto, &depth); + if (proto != htons(ETH_P_IPV6)) + return -EINVAL; + + ipv6h = (struct ipv6hdr *)(skb->data + depth); + depth += sizeof(*ipv6h); + end_depth = depth + sizeof(struct udphdr) + sizeof(struct psphdr); + + if (unlikely(end_depth > skb_headlen(skb))) + return -EINVAL; + + pse = skb_ext_add(skb, SKB_EXT_PSP); + if (!pse) + return -EINVAL; + + psph = (const struct psphdr *)(skb->data + depth + sizeof(struct udphdr)); + pse->spi = psph->spi; + spi = ntohl(psph->spi); + pse->generation = 0; + pse->version = FIELD_GET(PSPHDR_VERFL_VERSION, psph->verfl); + + ipv6h->nexthdr = psph->nexthdr; + ipv6h->payload_len = + htons(ntohs(ipv6h->payload_len) - PSP_ENCAP_HLEN - PSP_TRL_SIZE); + + memmove(skb->data + PSP_ENCAP_HLEN, skb->data, depth); + skb_pull(skb, PSP_ENCAP_HLEN); + + return 0; +} + +void mlx5e_nisp_offload_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb, + struct mlx5_cqe64 *cqe) +{ + u32 nisp_meta_data = be32_to_cpu(cqe->ft_metadata); + + /* TBD: report errors as SW counters to ethtool, any further handling ? */ + switch (MLX5_NISP_METADATA_SYNDROM(nisp_meta_data)) { + case MLX5E_NISP_OFFLOAD_RX_SYNDROME_DECRYPTED: + if (psp_rcv(skb)) + netdev_warn_once(netdev, "PSP handling failed"); + skb->decrypted = 1; + break; + case MLX5E_NISP_OFFLOAD_RX_SYNDROME_AUTH_FAILED: + break; + case MLX5E_NISP_OFFLOAD_RX_SYNDROME_BAD_TRAILER: + break; + default: + break; + } +} + void mlx5e_nisp_tx_build_eseg(struct mlx5e_priv *priv, struct sk_buff *skb, struct mlx5e_accel_tx_nisp_state *nisp_st, struct mlx5_wqe_eth_seg *eseg) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.h index 1350a73c2019..834481232b21 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.h @@ -10,6 +10,11 @@ #include "en.h" #include "en/txrx.h" +/* Bit30: NISP marker, Bit29-23: NISP syndrome, Bit22-0: NISP obj id */ +#define MLX5_NISP_METADATA_MARKER(metadata) ((((metadata) >> 30) & 0x3) == 0x3) +#define MLX5_NISP_METADATA_SYNDROM(metadata) (((metadata) >> 23) & GENMASK(6, 0)) +#define MLX5_NISP_METADATA_HANDLE(metadata) ((metadata) & GENMASK(22, 0)) + struct mlx5e_accel_tx_nisp_state { u32 tailen; u32 keyid; @@ -75,6 +80,16 @@ static inline unsigned int mlx5e_nisp_tx_ids_len(struct mlx5e_accel_tx_nisp_stat { return nisp_st->tailen; } + +static inline bool mlx5e_nisp_is_rx_flow(struct mlx5_cqe64 *cqe) +{ + return MLX5_NISP_METADATA_MARKER(be32_to_cpu(cqe->ft_metadata)); +} + +void mlx5e_nisp_offload_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb, + struct mlx5_cqe64 *cqe); + +void mlx5e_nisp_csum_complete(struct net_device *netdev, struct sk_buff *skb); #else static inline bool mlx5e_psp_is_offload_state(struct mlx5e_accel_tx_nisp_state *nisp_state) { @@ -92,5 +107,18 @@ static inline bool mlx5e_nisp_txwqe_build_eseg_csum(struct mlx5e_txqsq *sq, stru { return false; } + +static inline bool mlx5e_nisp_is_rx_flow(struct mlx5_cqe64 *cqe) +{ + return false; +} + +static inline void mlx5e_nisp_offload_handle_rx_skb(struct net_device *netdev, + struct sk_buff *skb, + struct mlx5_cqe64 *cqe) +{ +} + +static inline void mlx5e_nisp_csum_complete(struct net_device *netdev, struct sk_buff *skb) { } #endif #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index d601b5faaed5..41a4f8832f2f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -51,6 +51,7 @@ #include "ipoib/ipoib.h" #include "en_accel/ipsec.h" #include "en_accel/macsec.h" +#include "en_accel/nisp_rxtx.h" #include "en_accel/ipsec_rxtx.h" #include "en_accel/ktls_txrx.h" #include "en/xdp.h" @@ -1517,6 +1518,12 @@ static inline void mlx5e_handle_csum(struct net_device *netdev, skb->ip_summed = CHECKSUM_COMPLETE; skb->csum = csum_unfold((__force __sum16)cqe->check_sum); + if (unlikely(mlx5e_nisp_is_rx_flow(cqe))) { + /* TBD: PSP csum complete corrections for now chose csum_unnecessary path */ + mlx5e_nisp_csum_complete(netdev, skb); + goto csum_unnecessary; + } + if (test_bit(MLX5E_RQ_STATE_CSUM_FULL, &rq->state)) return; /* CQE csum covers all received bytes */ @@ -1559,6 +1566,9 @@ static inline void mlx5e_build_rx_skb(struct mlx5_cqe64 *cqe, if (unlikely(get_cqe_tls_offload(cqe))) mlx5e_ktls_handle_rx_skb(rq, skb, cqe, &cqe_bcnt); + if (unlikely(mlx5e_nisp_is_rx_flow(cqe))) + mlx5e_nisp_offload_handle_rx_skb(netdev, skb, cqe); + if (unlikely(mlx5_ipsec_is_rx_flow(cqe))) mlx5e_ipsec_offload_handle_rx_skb(netdev, skb, be32_to_cpu(cqe->ft_metadata)); From patchwork Fri May 10 03:04:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13660847 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE62C14BF8A for ; Fri, 10 May 2024 03:04:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310295; cv=none; b=j++sZ1fiZr4UohxLRiQhWxWj/+rXS2cQgOdgqcz9ow1om0/yOHDc+VQanvcK6XDu0jvwfJ73Hxi6xrYXfiUwvviXONFoYlNv0ganNRK0FvqxktC/ciSUsew06O3k+JU4iTs9x2FglRbeYNuMRn71M75KjVG0smSTbVfsvytgJFI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715310295; c=relaxed/simple; bh=SLhC/qzXGbm+7S3k+37urccU8JrEKDn1s9JBOl8cJCk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G9gF3XrzwFUFfUD+qbQBxDZtp1/qm2fzNh2mKKHKnAR2+LtKW1st1P4R+3iWwNYDztmWSkTYdLbjPl+UdHoUP+i3OnIqXOEEWi+mtholYIkvJ0wojNJnoZ1KmfZNBdnEHrKev9ccHstzxWgOWngC6YyGiF9cJ7TwtVmk1srGJgs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Kqs4aNqe; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Kqs4aNqe" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5423AC32783; Fri, 10 May 2024 03:04:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715310294; bh=SLhC/qzXGbm+7S3k+37urccU8JrEKDn1s9JBOl8cJCk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Kqs4aNqeCHe4aEA4f9z/6WY+gGVRpAMzMIte9neHGKCeja9K0XrFYIiCF7MzYutS0 oRXQdQiBIuDtQgKEKKEuDm6xRpO6kodNUcT+29gZ/7vVb+U5WDkiEEAo1ouVFeCZA8 ganUa4ISdTgbrOa7NhcB3lMevU4DOfDR5ifarbvy25mYTdJ5nAqTOhm0G1AdAbNdcu AcSI8HVeTK89cTQDUsXVZbHOP5U/fyQEmbdnVomkFDMjKmeAdoqS/efV/+g5oleG7S fVE17tW6S3IlIOPqsGShLjMBQizIgUkpv5hc/P9AiYqUH36dp5Dk19x7fB1JUh1tsX /QWH++WBjYm8w== From: Jakub Kicinski To: netdev@vger.kernel.org Cc: pabeni@redhat.com, willemdebruijn.kernel@gmail.com, borisp@nvidia.com, gal@nvidia.com, cratiu@nvidia.com, rrameshbabu@nvidia.com, steffen.klassert@secunet.com, tariqt@nvidia.com, Raed Salem , Jakub Kicinski Subject: [RFC net-next 15/15] net/mlx5e: Implement PSP key_rotate operation Date: Thu, 9 May 2024 20:04:35 -0700 Message-ID: <20240510030435.120935-16-kuba@kernel.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240510030435.120935-1-kuba@kernel.org> References: <20240510030435.120935-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Raed Salem Implement .key_rotate operation where when invoked will cause the HW to use a new master key to derive PSP spi/key pairs with complience with PSP spec. Signed-off-by: Raed Salem Signed-off-by: Rahul Rameshbabu Signed-off-by: Jakub Kicinski --- .../net/ethernet/mellanox/mlx5/core/en_accel/nisp.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c index 1131aa6e9b3d..cab4e79135d8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c @@ -96,11 +96,22 @@ static void mlx5e_psp_assoc_del(struct psp_dev *psd, struct psp_assoc *pas) atomic_dec(&nisp->tx_key_cnt); } +static int mlx5e_psp_key_rotate(struct psp_dev *psd, struct netlink_ext_ack *exack) +{ + struct mlx5e_priv *priv = netdev_priv(psd->main_netdev); + + /* no support for protecting against external rotations */ + psd->generation = 0; + + return mlx5e_nisp_rotate_key(priv->mdev); +} + static struct psp_dev_ops mlx5_psp_ops = { .set_config = mlx5e_psp_set_config, .rx_spi_alloc = mlx5e_psp_rx_spi_alloc, .tx_key_add = mlx5e_psp_assoc_add, .tx_key_del = mlx5e_psp_assoc_del, + .key_rotate = mlx5e_psp_key_rotate, }; void mlx5e_nisp_unregister(struct mlx5e_priv *priv)