From patchwork Sun Jan 5 01:25:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13926414 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF83735961 for ; Sun, 5 Jan 2025 01:25:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736040330; cv=none; b=OFzG4ht7eeJVavSlKu7E6uvj5I7oU3+T9FmCUIrV4y0s0dFWZn6LEx7vZ224gVHthT3KSRZF2qgpgFQRjp+brGpIGyZ6Z3vGXX8FYVpa1GXpGUOYzcz2vuTUthv416Cd2WO8I/AaAawJRjv0S2vVKDeakUlYaGW6syAb1+a9+EI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736040330; c=relaxed/simple; bh=6MonRFTi3F9rFmbR0sv9hYcpzmfgujw4rt8BgrxAWjU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jFNJBj6R+nVwE+rU5SYEVI38mV7un5Hypc77bgVBtLZQmlf5Jn3wUPMgIhY21urCERmKx14hUeO6WqLIREtk598jlrWX6Nv+hbnIOoOtPBFCTn1N9OqffN+NKClSz8BqPDDqw9R/ANwtXeqdF0S3KYyk+Inj0Rb3ooh5she/ncg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pS1emyfX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pS1emyfX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 224E9C4CEE0; Sun, 5 Jan 2025 01:25:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736040329; bh=6MonRFTi3F9rFmbR0sv9hYcpzmfgujw4rt8BgrxAWjU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pS1emyfXxxXGLWZA64TNETHDQQF/0k9/akXQYBdE9nA+sapbaLLDuKvkcCtSpPPhJ XhCUNGw7Hj0CxeRsfSz3HqTF9/XhLEG/0s3r+uNCQv1GOxKzeWYmkIwEPYkXpeeJml oB0pkC14vMt0iDROV6TJPzcxfLIsjOt1PNdCLq4qi/Ev+bdmsjKRgP2gKwhhuqMsEp C1bA/yh3Q4Xz3VV1VQuA6ze+FpEUWlb5IwKawEncJ8BKPYFLjsEar916wsZuUGJ4qb FfIh4rzOmFHDHS2FVzHhb9Bb93S2AIp5KhzAfSJVyCXJrze+4IqxutitMiGpdOZwtb /oEmwj9gzZC7Q== From: Jakub Kicinski To: davem@davemloft.net Cc: donald.hunter@gmail.com, netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, Jakub Kicinski Subject: [PATCH net-next 1/3] tools: ynl: correctly handle overrides of fields in subset Date: Sat, 4 Jan 2025 17:25:21 -0800 Message-ID: <20250105012523.1722231-2-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250105012523.1722231-1-kuba@kernel.org> References: <20250105012523.1722231-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org We stated in documentation [1] and previous discussions [2] that the need for overriding fields in members of subsets is anticipated. Implement it. [1] https://docs.kernel.org/next/userspace-api/netlink/specs.html#subset-of [2] https://lore.kernel.org/netdev/20231004171350.1f59cd1d@kernel.org/ Signed-off-by: Jakub Kicinski Reviewed-by: Donald Hunter --- tools/net/ynl/lib/nlspec.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tools/net/ynl/lib/nlspec.py b/tools/net/ynl/lib/nlspec.py index a745739655ad..314ec8007496 100644 --- a/tools/net/ynl/lib/nlspec.py +++ b/tools/net/ynl/lib/nlspec.py @@ -219,7 +219,10 @@ jsonschema = None else: real_set = family.attr_sets[self.subset_of] for elem in self.yaml['attributes']: - attr = real_set[elem['name']] + real_attr = real_set[elem['name']] + combined_elem = real_attr.yaml | elem + attr = self.new_attr(combined_elem, real_attr.value) + self.attrs[attr.name] = attr self.attrs_by_val[attr.value] = attr From patchwork Sun Jan 5 01:25:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13926415 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 529AA4204E for ; Sun, 5 Jan 2025 01:25:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736040330; cv=none; b=nc2vwKQ8ta7gTrXmxJDiWWqybdNemMECAtjcqsVEwv1Zzesz+Zi8V7z67CWn9x5rz8tIyj2ix+QMsilnN3K9q1WAJq88T2CHW6YA6wkaBMDDPvLZoKvuf3J6iQty6NrykqLZvizFJVwal7vR1OSwK1X7zDHd6jKKfum5FoRFVCw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736040330; c=relaxed/simple; bh=EMPpN9Lz77T6bvMY8U6BiqFQv63yYYxLq1+xJQjiHQA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ciXEnqP27F+jglVboXaBCIzELGgyXSn0x52UP7JHjeFMKaqWY/xE01QaaWI07dPEgSZ5nA/aXtXCgmYI3XS0qP6HKw1goFkKVlzwBrHvtGbgDDcZLxOL5+UaEMwT4gJloMmBZqa0H8pNYgTcakO9C/bKJrP2yk6TcsYQqCX9HZ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EnZyWypb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EnZyWypb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 94193C4CED1; Sun, 5 Jan 2025 01:25:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736040329; bh=EMPpN9Lz77T6bvMY8U6BiqFQv63yYYxLq1+xJQjiHQA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EnZyWypb4OaLduG2q7ncDPV19gHR0kUs3k1h5YegZdwp2wysRfP074H44v4a9BEsA J5AwdRU0jLsIhvNmeGFjOpELt9eLf62sQxjWJV3yqxjS60/R76kuf1/VpUwJnxZHcT fwoX4HqTE7mtq7rK2WuXQVwpc7zgAqyWOqWyD6OlwjDl3ghH6/Gebx9JFAQCNhhwDK 72pwoFApNhshb3TQABfqGGv70NvhERn7RDasSZlf5/QE4lmj5dq7nikMuHYh/vYPv7 aBiG0ombq3y5K5PXKc/WtPbMV3ElruJinKiW5eejA73QFlTAURbu6Lqdx6kOOKHXRj 2SgU6EdUetg/w== From: Jakub Kicinski To: davem@davemloft.net Cc: donald.hunter@gmail.com, netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, Jakub Kicinski Subject: [PATCH net-next 2/3] tools: ynl: print some information about attribute we can't parse Date: Sat, 4 Jan 2025 17:25:22 -0800 Message-ID: <20250105012523.1722231-3-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250105012523.1722231-1-kuba@kernel.org> References: <20250105012523.1722231-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org When parsing throws an exception one often has to figure out which attribute couldn't be parsed from first principles. For families with large message parsing trees like rtnetlink guessing the attribute can be hard. Print a bit of information as the exception travels out, e.g.: # when dumping rt links Error decoding 'flags' from 'linkinfo-ip6tnl-attrs' Error decoding 'data' from 'linkinfo-attrs' Error decoding 'linkinfo' from 'link-attrs' Traceback (most recent call last): File "/home/kicinski/linux/./tools/net/ynl/cli.py", line 119, in main() File "/home/kicinski/linux/./tools/net/ynl/cli.py", line 100, in main reply = ynl.dump(args.dump, attrs) File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 1064, in dump return self._op(method, vals, dump=True) File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 1058, in _op return self._ops(ops)[0] File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 1045, in _ops rsp_msg = self._decode(decoded.raw_attrs, op.attr_set.name) File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 738, in _decode subdict = self._decode(NlAttrs(attr.raw), attr_spec['nested-attributes'], search_attrs) File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 763, in _decode decoded = self._decode_sub_msg(attr, attr_spec, search_attrs) File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 714, in _decode_sub_msg subdict = self._decode(NlAttrs(attr.raw, offset), msg_format.attr_set) File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 749, in _decode decoded = attr.as_scalar(attr_spec['type'], attr_spec.byte_order) File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 147, in as_scalar return format.unpack(self.raw)[0] struct.error: unpack requires a buffer of 2 bytes The Traceback is what we would previously see, the "Error..." messages are new. We print a message per level (in the stack order). Printing single combined message gets tricky quickly given sub-messages etc. Signed-off-by: Jakub Kicinski Reviewed-by: Donald Hunter --- tools/net/ynl/lib/ynl.py | 72 +++++++++++++++++++++------------------- 1 file changed, 38 insertions(+), 34 deletions(-) diff --git a/tools/net/ynl/lib/ynl.py b/tools/net/ynl/lib/ynl.py index eea29359a899..08f8bf89cfc2 100644 --- a/tools/net/ynl/lib/ynl.py +++ b/tools/net/ynl/lib/ynl.py @@ -733,41 +733,45 @@ genl_family_name_to_id = None self._rsp_add(rsp, attr_name, None, self._decode_unknown(attr)) continue - if attr_spec["type"] == 'nest': - subdict = self._decode(NlAttrs(attr.raw), attr_spec['nested-attributes'], search_attrs) - decoded = subdict - elif attr_spec["type"] == 'string': - decoded = attr.as_strz() - elif attr_spec["type"] == 'binary': - decoded = self._decode_binary(attr, attr_spec) - elif attr_spec["type"] == 'flag': - decoded = True - elif attr_spec.is_auto_scalar: - decoded = attr.as_auto_scalar(attr_spec['type'], attr_spec.byte_order) - elif attr_spec["type"] in NlAttr.type_formats: - decoded = attr.as_scalar(attr_spec['type'], attr_spec.byte_order) - if 'enum' in attr_spec: - decoded = self._decode_enum(decoded, attr_spec) - elif attr_spec.display_hint: - decoded = self._formatted_string(decoded, attr_spec.display_hint) - elif attr_spec["type"] == 'indexed-array': - decoded = self._decode_array_attr(attr, attr_spec) - elif attr_spec["type"] == 'bitfield32': - value, selector = struct.unpack("II", attr.raw) - if 'enum' in attr_spec: - value = self._decode_enum(value, attr_spec) - selector = self._decode_enum(selector, attr_spec) - decoded = {"value": value, "selector": selector} - elif attr_spec["type"] == 'sub-message': - decoded = self._decode_sub_msg(attr, attr_spec, search_attrs) - elif attr_spec["type"] == 'nest-type-value': - decoded = self._decode_nest_type_value(attr, attr_spec) - else: - if not self.process_unknown: - raise Exception(f'Unknown {attr_spec["type"]} with name {attr_spec["name"]}') - decoded = self._decode_unknown(attr) + try: + if attr_spec["type"] == 'nest': + subdict = self._decode(NlAttrs(attr.raw), attr_spec['nested-attributes'], search_attrs) + decoded = subdict + elif attr_spec["type"] == 'string': + decoded = attr.as_strz() + elif attr_spec["type"] == 'binary': + decoded = self._decode_binary(attr, attr_spec) + elif attr_spec["type"] == 'flag': + decoded = True + elif attr_spec.is_auto_scalar: + decoded = attr.as_auto_scalar(attr_spec['type'], attr_spec.byte_order) + elif attr_spec["type"] in NlAttr.type_formats: + decoded = attr.as_scalar(attr_spec['type'], attr_spec.byte_order) + if 'enum' in attr_spec: + decoded = self._decode_enum(decoded, attr_spec) + elif attr_spec.display_hint: + decoded = self._formatted_string(decoded, attr_spec.display_hint) + elif attr_spec["type"] == 'indexed-array': + decoded = self._decode_array_attr(attr, attr_spec) + elif attr_spec["type"] == 'bitfield32': + value, selector = struct.unpack("II", attr.raw) + if 'enum' in attr_spec: + value = self._decode_enum(value, attr_spec) + selector = self._decode_enum(selector, attr_spec) + decoded = {"value": value, "selector": selector} + elif attr_spec["type"] == 'sub-message': + decoded = self._decode_sub_msg(attr, attr_spec, search_attrs) + elif attr_spec["type"] == 'nest-type-value': + decoded = self._decode_nest_type_value(attr, attr_spec) + else: + if not self.process_unknown: + raise Exception(f'Unknown {attr_spec["type"]} with name {attr_spec["name"]}') + decoded = self._decode_unknown(attr) - self._rsp_add(rsp, attr_spec["name"], attr_spec.is_multi, decoded) + self._rsp_add(rsp, attr_spec["name"], attr_spec.is_multi, decoded) + except: + print(f"Error decoding '{attr_spec.name}' from '{space}'") + raise return rsp From patchwork Sun Jan 5 01:25:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13926416 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B31735950 for ; Sun, 5 Jan 2025 01:25:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736040330; cv=none; b=P9EtokB2P+DuwvyI2kuMnZmUpZilkYOB5Iw9y7utvaPOd/cuzVqeUOwEv8LJxIDqeiS254spYsJygvkZ2quVBUGQOdf1ua7R+GCkMIijIV6Aw/qzvjDe/LLvNckeyyyIzyhI099G4hKfc8OaLYkyUwW6xL43GkfzKl7DUg22BLk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736040330; c=relaxed/simple; bh=fjkyiRAYREsBZoj2Q/oEApeDcuDbyoF5o5EZR5bWWhA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rigu4+EHlhlg1NvtLyGCpYpvMFHK9cYAkwfqsOwJAMgUM4Q37pJoeVId4HI2e+Y352MQeGPJ+9m0eD5giuYICOtQXRMSQUIdTyvxajui9VwrW4pPs3TY/BtOcHxrX9FjbHKTDbEQrbKhaWgtYBSTosNtWJPmrvzS22CZQob3ybk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NyAJzz9J; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NyAJzz9J" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15018C4CEE3; Sun, 5 Jan 2025 01:25:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736040330; bh=fjkyiRAYREsBZoj2Q/oEApeDcuDbyoF5o5EZR5bWWhA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NyAJzz9J882BIJQaInX9xkdIKjlNsPFDVcKky7fSQhzYy9c8Lf3xEfT3KXiryZrLb dHInJGr3p8TY4isS4LCQavi2nBOvW9bW1IoXs/AD2CUrGQShu8qkkPMT9HLM/a4VF9 8N97SWXbtBvShuZ4Y05M25DnmgyVU13TSohBMdy8PeI/aK9a8AGkDh1Bj5Q3OrvpVi 3SSYr6ZKoQThg/9YEisDfJESahl1fcllTKQfO2yjuq6fbQA50l5Jd+ZjpIk5r36e0R MhnrpFnGjKRuFi/k4daNOXne5vSAjLbYtHgdS2ZVysotRIHhJ/U/uCo86rOjue0buX rZUu9fy5Eb8bg== From: Jakub Kicinski To: davem@davemloft.net Cc: donald.hunter@gmail.com, netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, Jakub Kicinski Subject: [PATCH net-next 3/3] netlink: specs: rt_link: decode ip6tnl, vti and vti6 link attrs Date: Sat, 4 Jan 2025 17:25:23 -0800 Message-ID: <20250105012523.1722231-4-kuba@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250105012523.1722231-1-kuba@kernel.org> References: <20250105012523.1722231-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Some of our tests load vti and ip6tnl so not being able to decode the link attrs gets in the way of using Python YNL for testing. Decode link attributes for ip6tnl, vti and vti6. ip6tnl uses IFLA_IPTUN_FLAGS as u32, while ipv4 and sit expect a u16 attribute, so we have a (first?) subset type override... Signed-off-by: Jakub Kicinski Reviewed-by: Donald Hunter --- Documentation/netlink/specs/rt_link.yaml | 87 ++++++++++++++++++++++++ 1 file changed, 87 insertions(+) diff --git a/Documentation/netlink/specs/rt_link.yaml b/Documentation/netlink/specs/rt_link.yaml index 96465376d6fe..363c4d4f0779 100644 --- a/Documentation/netlink/specs/rt_link.yaml +++ b/Documentation/netlink/specs/rt_link.yaml @@ -1825,6 +1825,48 @@ protonum: 0 - name: erspan-hwid type: u16 + - + name: linkinfo-vti-attrs + name-prefix: ifla-vti- + attributes: + - + name: link + type: u32 + - + name: ikey + type: u32 + - + name: okey + type: u32 + - + name: local + type: binary + display-hint: ipv4 + - + name: remote + type: binary + display-hint: ipv4 + - + name: fwmark + type: u32 + - + name: linkinfo-vti6-attrs + subset-of: linkinfo-vti-attrs + attributes: + - + name: link + - + name: ikey + - + name: okey + - + name: local + display-hint: ipv6 + - + name: remote + display-hint: ipv6 + - + name: fwmark - name: linkinfo-geneve-attrs name-prefix: ifla-geneve- @@ -1941,6 +1983,42 @@ protonum: 0 - name: fwmark type: u32 + - + name: linkinfo-ip6tnl-attrs + subset-of: linkinfo-iptun-attrs + attributes: + - + name: link + - + name: local + display-hint: ipv6 + - + name: remote + display-hint: ipv6 + - + name: ttl + - + name: encap-limit + - + name: flowinfo + - + name: flags + # ip6tnl unlike ipip and sit has 32b flags + type: u32 + - + name: proto + - + name: encap-type + - + name: encap-flags + - + name: encap-sport + - + name: encap-dport + - + name: collect-metadata + - + name: fwmark - name: linkinfo-tun-attrs name-prefix: ifla-tun- @@ -2195,6 +2273,9 @@ protonum: 0 - value: ipip attribute-set: linkinfo-iptun-attrs + - + value: ip6tnl + attribute-set: linkinfo-ip6tnl-attrs - value: sit attribute-set: linkinfo-iptun-attrs @@ -2207,6 +2288,12 @@ protonum: 0 - value: vrf attribute-set: linkinfo-vrf-attrs + - + value: vti + attribute-set: linkinfo-vti-attrs + - + value: vti6 + attribute-set: linkinfo-vti6-attrs - value: netkit attribute-set: linkinfo-netkit-attrs