From patchwork Mon Oct 21 12:21:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13844011 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 349861F942B; Mon, 21 Oct 2024 12:21:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513319; cv=none; b=Vg8IwunvcWt2aS6WRa6r0hMpTX/fDfLNLUME4c9qS9kU98yNWSTi6NS9q7OEUb58BoOBnX0PrXnwwSqSKRmvhRrjIZD8zQHERKBLiZLfcxOy+JTCPZP5gHy8fFokQb0/kp1BZNDWl1YTJSmmaSiyA8KFtjHpL6Gm5oSLhxhHfz0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513319; c=relaxed/simple; bh=DksjoWr3zi3vNlUAvCVzy7Y3/lqTinQGR/Ze6AaMZC4=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jgsH+7h1Rb8rG4mC/sgQPPMutXvcpR8a8W8GXqyk5kBsHp7lyWNi5xTWA5nSnnsvZRVGfUVoWM7f+4+ffcd7kyJpu1fqGzn0LpeaKOTdSCpyYkfdNYmBHX0mcNv+6jBG0SboVaUL1bSEzWsGSuczbpb3FoF1UFWNz6qSbgyidbM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nHEQ1/Jq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nHEQ1/Jq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 67E0BC4CEC3; Mon, 21 Oct 2024 12:21:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729513318; bh=DksjoWr3zi3vNlUAvCVzy7Y3/lqTinQGR/Ze6AaMZC4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=nHEQ1/Jq4CEIn/6XFUcJ5UljN17WywRBZBPUwCPExk2/YXfvLhbx7jsTrL/5yA76S mUE6v/rv1LKMZ8kiHWDvGn5hc075ANSZQgMxLAh+LP5FWo0LYM9zDn0k4CJ6oZNgU6 bYqFFOutWCFWwbMcXMIiNjK7I+BYqp9khSA2hiVcHVTCy3g1PrUX+BTaNAYlVi/h8d vfN2pXpglXa+9KVGnf472OwHoRoKaNQ7YtDm46sRAmYGH6t5nrd6T2mKAICHZYH8xF GFuOyXgPh4RUfiRgY0e2KUp4Q6uoIe12F21/CAf7ZtPHLXpzr4m2loCAFDERDmupBT bobVeaSc/4f5w== From: Puranjay Mohan To: Albert Ou , Alexei Starovoitov , Andrew Morton , Andrii Nakryiko , bpf@vger.kernel.org, Daniel Borkmann , "David S. Miller" , Eduard Zingerman , Eric Dumazet , Hao Luo , Helge Deller , Jakub Kicinski , "James E.J. Bottomley" , Jiri Olsa , John Fastabend , KP Singh , linux-kernel@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, Martin KaFai Lau , Mykola Lysenko , netdev@vger.kernel.org, Palmer Dabbelt , Paolo Abeni , Paul Walmsley , Puranjay Mohan , Puranjay Mohan , Shuah Khan , Song Liu , Stanislav Fomichev , Yonghong Song Subject: [PATCH bpf-next 1/5] net: checksum: move from32to16() to generic header Date: Mon, 21 Oct 2024 12:21:08 +0000 Message-Id: <20241021122112.101513-2-puranjay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20241021122112.101513-1-puranjay@kernel.org> References: <20241021122112.101513-1-puranjay@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net from32to16() is used by lib/checksum.c and also by arch/parisc/lib/checksum.c. The next patch will use it in the bpf_csum_diff helper. Move from32to16() to the include/net/checksum.h as csum_from32to16() and remove other implementations. Signed-off-by: Puranjay Mohan Acked-by: Daniel Borkmann Reviewed-by: Toke Høiland-Jørgensen --- arch/parisc/lib/checksum.c | 13 ++----------- include/net/checksum.h | 6 ++++++ lib/checksum.c | 11 +---------- 3 files changed, 9 insertions(+), 21 deletions(-) diff --git a/arch/parisc/lib/checksum.c b/arch/parisc/lib/checksum.c index 4818f3db84a5c..59d8c15d81bd0 100644 --- a/arch/parisc/lib/checksum.c +++ b/arch/parisc/lib/checksum.c @@ -25,15 +25,6 @@ : "=r"(_t) \ : "r"(_r), "0"(_t)); -static inline unsigned short from32to16(unsigned int x) -{ - /* 32 bits --> 16 bits + carry */ - x = (x & 0xffff) + (x >> 16); - /* 16 bits + carry --> 16 bits including carry */ - x = (x & 0xffff) + (x >> 16); - return (unsigned short)x; -} - static inline unsigned int do_csum(const unsigned char * buff, int len) { int odd, count; @@ -85,7 +76,7 @@ static inline unsigned int do_csum(const unsigned char * buff, int len) } if (len & 1) result += le16_to_cpu(*buff); - result = from32to16(result); + result = csum_from32to16(result); if (odd) result = swab16(result); out: @@ -102,7 +93,7 @@ __wsum csum_partial(const void *buff, int len, __wsum sum) { unsigned int result = do_csum(buff, len); addc(result, sum); - return (__force __wsum)from32to16(result); + return (__force __wsum)csum_from32to16(result); } EXPORT_SYMBOL(csum_partial); diff --git a/include/net/checksum.h b/include/net/checksum.h index 1338cb92c8e72..0d082febfead4 100644 --- a/include/net/checksum.h +++ b/include/net/checksum.h @@ -151,6 +151,12 @@ static inline void csum_replace(__wsum *csum, __wsum old, __wsum new) *csum = csum_add(csum_sub(*csum, old), new); } +static inline __sum16 csum_from32to16(__wsum sum) +{ + sum += (sum >> 16) | (sum << 16); + return (__force __sum16)(sum >> 16); +} + struct sk_buff; void inet_proto_csum_replace4(__sum16 *sum, struct sk_buff *skb, __be32 from, __be32 to, bool pseudohdr); diff --git a/lib/checksum.c b/lib/checksum.c index 6860d6b05a171..025ba546e1ec6 100644 --- a/lib/checksum.c +++ b/lib/checksum.c @@ -34,15 +34,6 @@ #include #ifndef do_csum -static inline unsigned short from32to16(unsigned int x) -{ - /* add up 16-bit and 16-bit for 16+c bit */ - x = (x & 0xffff) + (x >> 16); - /* add up carry.. */ - x = (x & 0xffff) + (x >> 16); - return x; -} - static unsigned int do_csum(const unsigned char *buff, int len) { int odd; @@ -90,7 +81,7 @@ static unsigned int do_csum(const unsigned char *buff, int len) #else result += (*buff << 8); #endif - result = from32to16(result); + result = csum_from32to16(result); if (odd) result = ((result >> 8) & 0xff) | ((result & 0xff) << 8); out: From patchwork Mon Oct 21 12:21:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13844012 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D7E31E884B; Mon, 21 Oct 2024 12:22:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513323; cv=none; b=oF6OMma4+SyWfBDLQOzl/F01OKdC03HPxuqApwwR1qbAbV/PsOT7DzfJ/NMAcZ16DLC6HSW+sboxkj8qEUbQnQNzvawaFChO2qqYePP47AZh7IsC2kp2gERJevzEh+AlbNyFeXLiefSOEtTF3N99tiTVCP/EckmktRu+Uru6wHc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513323; c=relaxed/simple; bh=gJ2hfEV44PPxvxKXpZF+JkBcTX9tTBF4IaZx7SCOMZ8=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bchuinTiroAxS8JQuhlI0iqrCHezu6yKv2TQKrlS//TWFuTipA16PN5jgL0b404n3/WgPgz32MMaR/x/SbiwHcujrUCqhBhq5mOfThmaoyM76/HrHE3YL9YvO/km1Ynhqod8GjpIW8eEWBbzLcBXf4+voHWAhrsWBg/niTCbn+M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=H7u8hf1B; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="H7u8hf1B" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE604C4CEE4; Mon, 21 Oct 2024 12:22:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729513323; bh=gJ2hfEV44PPxvxKXpZF+JkBcTX9tTBF4IaZx7SCOMZ8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=H7u8hf1BlS2So8vQ1TSFGFD05iAr9Ps6b4H9FJhaHFS9M+hlJmUh9+4Os2jqFP/Q/ bdW2TEwQ6IxHTfkeXZwOtrm+b+57z5pFqqUi7KpQsoVGrI69x9j8DQ1ePmPrWMd3et RAhnxCxeEHjH3SoqZdayL+7a7ffLzisilXVatTyfXD39jrD/P8rZxi8OKIW00B/9H2 oUU9jI0zkAx42RmuN6Nya/uLvsYiNMKR1A5rAUCwNqvEVwJRK5JcwcQdmtgVbhuyrz YAh2aD+a1EAy5H4ucA6ux8jJyITT7IkQ/ZPolYVTZGf3iI8YRWfzg4UjRGupy9Bghv s1kjJbP9utYdw== From: Puranjay Mohan To: Albert Ou , Alexei Starovoitov , Andrew Morton , Andrii Nakryiko , bpf@vger.kernel.org, Daniel Borkmann , "David S. Miller" , Eduard Zingerman , Eric Dumazet , Hao Luo , Helge Deller , Jakub Kicinski , "James E.J. Bottomley" , Jiri Olsa , John Fastabend , KP Singh , linux-kernel@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, Martin KaFai Lau , Mykola Lysenko , netdev@vger.kernel.org, Palmer Dabbelt , Paolo Abeni , Paul Walmsley , Puranjay Mohan , Puranjay Mohan , Shuah Khan , Song Liu , Stanislav Fomichev , Yonghong Song Subject: [PATCH bpf-next 2/5] bpf: bpf_csum_diff: optimize and homogenize for all archs Date: Mon, 21 Oct 2024 12:21:09 +0000 Message-Id: <20241021122112.101513-3-puranjay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20241021122112.101513-1-puranjay@kernel.org> References: <20241021122112.101513-1-puranjay@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net 1. Optimization ------------ The current implementation copies the 'from' and 'to' buffers to a scratchpad and it takes the bitwise NOT of 'from' buffer while copying. In the next step csum_partial() is called with this scratchpad. so, mathematically, the current implementation is doing: result = csum(to - from) Here, 'to' and '~ from' are copied in to the scratchpad buffer, we need it in the scratchpad buffer because csum_partial() takes a single contiguous buffer and not two disjoint buffers like 'to' and 'from'. We can re write this equation to: result = csum(to) - csum(from) using the distributive property of csum(). this allows 'to' and 'from' to be at different locations and therefore this scratchpad and copying is not needed. This in C code will look like: result = csum_sub(csum_partial(to, to_size, seed), csum_partial(from, from_size, 0)); 2. Homogenization -------------- The bpf_csum_diff() helper calls csum_partial() which is implemented by some architectures like arm and x86 but other architectures rely on the generic implementation in lib/checksum.c The generic implementation in lib/checksum.c returns a 16 bit value but the arch specific implementations can return more than 16 bits, this works out in most places because before the result is used, it is passed through csum_fold() that turns it into a 16-bit value. bpf_csum_diff() directly returns the value from csum_partial() and therefore the returned values could be different on different architectures. see discussion in [1]: for the int value 28 the calculated checksums are: x86 : -29 : 0xffffffe3 generic (arm64, riscv) : 65507 : 0x0000ffe3 arm : 131042 : 0x0001ffe2 Pass the result of bpf_csum_diff() through from32to16() before returning to homogenize this result for all architectures. NOTE: from32to16() is used instead of csum_fold() because csum_fold() does from32to16() + bitwise NOT of the result, which is not what we want to do here. [1] https://lore.kernel.org/bpf/CAJ+HfNiQbOcqCLxFUP2FMm5QrLXUUaj852Fxe3hn_2JNiucn6g@mail.gmail.com/ Signed-off-by: Puranjay Mohan Acked-by: Daniel Borkmann Reviewed-by: Toke Høiland-Jørgensen --- net/core/filter.c | 37 +++++++++---------------------------- 1 file changed, 9 insertions(+), 28 deletions(-) diff --git a/net/core/filter.c b/net/core/filter.c index bd0d08bf76bb8..e00bec7de9edd 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -1654,18 +1654,6 @@ void sk_reuseport_prog_free(struct bpf_prog *prog) bpf_prog_destroy(prog); } -struct bpf_scratchpad { - union { - __be32 diff[MAX_BPF_STACK / sizeof(__be32)]; - u8 buff[MAX_BPF_STACK]; - }; - local_lock_t bh_lock; -}; - -static DEFINE_PER_CPU(struct bpf_scratchpad, bpf_sp) = { - .bh_lock = INIT_LOCAL_LOCK(bh_lock), -}; - static inline int __bpf_try_make_writable(struct sk_buff *skb, unsigned int write_len) { @@ -2022,11 +2010,6 @@ static const struct bpf_func_proto bpf_l4_csum_replace_proto = { BPF_CALL_5(bpf_csum_diff, __be32 *, from, u32, from_size, __be32 *, to, u32, to_size, __wsum, seed) { - struct bpf_scratchpad *sp = this_cpu_ptr(&bpf_sp); - u32 diff_size = from_size + to_size; - int i, j = 0; - __wsum ret; - /* This is quite flexible, some examples: * * from_size == 0, to_size > 0, seed := csum --> pushing data @@ -2035,19 +2018,17 @@ BPF_CALL_5(bpf_csum_diff, __be32 *, from, u32, from_size, * * Even for diffing, from_size and to_size don't need to be equal. */ - if (unlikely(((from_size | to_size) & (sizeof(__be32) - 1)) || - diff_size > sizeof(sp->diff))) - return -EINVAL; - local_lock_nested_bh(&bpf_sp.bh_lock); - for (i = 0; i < from_size / sizeof(__be32); i++, j++) - sp->diff[j] = ~from[i]; - for (i = 0; i < to_size / sizeof(__be32); i++, j++) - sp->diff[j] = to[i]; + if (from_size && to_size) + return csum_from32to16(csum_sub(csum_partial(to, to_size, seed), + csum_partial(from, from_size, 0))); + if (to_size) + return csum_from32to16(csum_partial(to, to_size, seed)); - ret = csum_partial(sp->diff, diff_size, seed); - local_unlock_nested_bh(&bpf_sp.bh_lock); - return ret; + if (from_size) + return csum_from32to16(~csum_partial(from, from_size, ~seed)); + + return seed; } static const struct bpf_func_proto bpf_csum_diff_proto = { From patchwork Mon Oct 21 12:21:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13844013 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 213D51FA262; Mon, 21 Oct 2024 12:22:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513327; cv=none; b=pAZNwXF+IX7FJxSkTzUuno6opPS8ZkvSa7432iHN9Q+6uP4fa0zsjbt8GC1cDfBRXA5NOfkCyNReh6UzU7Bu0WTvw8u8TicpRcpFxF4ql1FYj6cD4TdK9ylt96H4b5qVeX8nekPWe1wrAeeRRGsPSpt5nxnERMngBkbWFFN5cEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513327; c=relaxed/simple; bh=q9r41TUNdBdaXSTHRnEB4GOxd1xBEVJa+omNwYCjQ9g=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=M5eGNbKwrla3jAx5CuQaS6S1x+Mi7RUF3h+qNm3PH4XCJJioxchpU2ce7flMq+RIuhTObkQfKXxv3vSaj1GisAqcgwCwbh2TbiHV7+ak18mF+FGLpQWlpaI8F/Rqh9t7kgiXCBX8Pm7Lyr8NMX/yyHUEsepCzlihpGJk+CMC4ho= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=K2s47iFp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="K2s47iFp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4A248C4CEC3; Mon, 21 Oct 2024 12:22:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729513326; bh=q9r41TUNdBdaXSTHRnEB4GOxd1xBEVJa+omNwYCjQ9g=; h=From:To:Subject:Date:In-Reply-To:References:From; b=K2s47iFpNxrXBD19bu1MVAodC8qAnRBq0DYV2+zoAoelVjDzHq34KIcYHkvE/jHWi CDiDHlRCjvOZqIIjIZO0MEQCQZFiVIb63p7V8pfIRpzXQHXzM62S0hGnJMqmVhgQQY LuIVyNVIuMPtcui6PykJbw0VJs6Tg5+OpAldA/XTxeLVTvLcu+KkTXYNwGroOypmLJ AqHAzWmluEL6R1UHGwCvc7yAxP+XivHBshqAG9qRft8BKEwcwEwmKqMLJDS/6Zov3R GimBTQLOAe7/H/QRhRZwfgM7ha0qmbdQotEtL7otQ9AZOQRJRJYhcrtm0YW8ae7DeB +2rGIQLxsS3dg== From: Puranjay Mohan To: Albert Ou , Alexei Starovoitov , Andrew Morton , Andrii Nakryiko , bpf@vger.kernel.org, Daniel Borkmann , "David S. Miller" , Eduard Zingerman , Eric Dumazet , Hao Luo , Helge Deller , Jakub Kicinski , "James E.J. Bottomley" , Jiri Olsa , John Fastabend , KP Singh , linux-kernel@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, Martin KaFai Lau , Mykola Lysenko , netdev@vger.kernel.org, Palmer Dabbelt , Paolo Abeni , Paul Walmsley , Puranjay Mohan , Puranjay Mohan , Shuah Khan , Song Liu , Stanislav Fomichev , Yonghong Song Subject: [PATCH bpf-next 3/5] selftests/bpf: don't mask result of bpf_csum_diff() in test_verifier Date: Mon, 21 Oct 2024 12:21:10 +0000 Message-Id: <20241021122112.101513-4-puranjay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20241021122112.101513-1-puranjay@kernel.org> References: <20241021122112.101513-1-puranjay@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The bpf_csum_diff() helper has been fixed to return a 16-bit value for all archs, so now we don't need to mask the result. This commit is basically reverting the below: commit 6185266c5a85 ("selftests/bpf: Mask bpf_csum_diff() return value to 16 bits in test_verifier") Signed-off-by: Puranjay Mohan Acked-by: Daniel Borkmann Reviewed-by: Toke Høiland-Jørgensen --- tools/testing/selftests/bpf/progs/verifier_array_access.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/progs/verifier_array_access.c b/tools/testing/selftests/bpf/progs/verifier_array_access.c index 95d7ecc12963b..4195aa824ba55 100644 --- a/tools/testing/selftests/bpf/progs/verifier_array_access.c +++ b/tools/testing/selftests/bpf/progs/verifier_array_access.c @@ -368,8 +368,7 @@ __naked void a_read_only_array_2_1(void) r4 = 0; \ r5 = 0; \ call %[bpf_csum_diff]; \ -l0_%=: r0 &= 0xffff; \ - exit; \ +l0_%=: exit; \ " : : __imm(bpf_csum_diff), __imm(bpf_map_lookup_elem), From patchwork Mon Oct 21 12:21:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13844014 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 164831FAC3A; Mon, 21 Oct 2024 12:22:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513330; cv=none; b=TNEfMSUeHXE07YFMjJ/pBOw78/iqXfS97isKb1xprpO43JDlV8HyeOmdVDEqg5NJ+ahT1/Ig1eBl/PwR3d4zSn5gsfi4ZyO89rt8suwJbDKfH+TTMHLUSuZV5yuCgND4sYz03dkNxViGxVTmKsbD1pNYeIow3WHrijIML+wAk2Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513330; c=relaxed/simple; bh=Hoeel4AJqQq2OzQsQRr98YOviBg6J0xi4VZmo07O8iI=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=D/38qtquV04nKMQ36SflvWh9Cg3IqT//g72ltcfyEz5+pr63JbVzmw8EeZoPaAWYwxQgMCkxU1TnGQNnPrPKfL47YUU+8glMlhZmZ6sY79iQ1ArAJW5KC3Rk4j01fRmByQJWos/9ZqiJ+17emc487rVFPaC5lAOoO7tbDQgjOss= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=D1AYD7oA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="D1AYD7oA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2E900C4CEE4; Mon, 21 Oct 2024 12:22:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729513329; bh=Hoeel4AJqQq2OzQsQRr98YOviBg6J0xi4VZmo07O8iI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=D1AYD7oAoLtZtsFdvDU5FzWG4lsH1F/PM+Cdwy1cUfzVb3WYM4HMfuct2zqZy07z6 eksC6bLBWZaNPzqH7Bj+smPXujXXYwNn5GcPJyXqQfWhO3qOwnXVGJ1F1mVdv7bnNK lm03s8xuAuNaCx1t3HQjDMb2g+kwuzV1owdQ3c67TjrKEhgiQcqofWaM9+sdrG86uJ nTBsN9NKPAXFY0CxKMZARZijQLSvb+nGvVyChjPgtFiX+KdZnZfgijuyhlGYpwNbcY r4gqdSt+53SPeZtaN582mtK8arN455hJ+/aywirgG4X6sqlj9URJbha0fBv4ANYc/e VYkr8ABbyYz6Q== From: Puranjay Mohan To: Albert Ou , Alexei Starovoitov , Andrew Morton , Andrii Nakryiko , bpf@vger.kernel.org, Daniel Borkmann , "David S. Miller" , Eduard Zingerman , Eric Dumazet , Hao Luo , Helge Deller , Jakub Kicinski , "James E.J. Bottomley" , Jiri Olsa , John Fastabend , KP Singh , linux-kernel@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, Martin KaFai Lau , Mykola Lysenko , netdev@vger.kernel.org, Palmer Dabbelt , Paolo Abeni , Paul Walmsley , Puranjay Mohan , Puranjay Mohan , Shuah Khan , Song Liu , Stanislav Fomichev , Yonghong Song Subject: [PATCH bpf-next 4/5] selftests/bpf: Add benchmark for bpf_csum_diff() helper Date: Mon, 21 Oct 2024 12:21:11 +0000 Message-Id: <20241021122112.101513-5-puranjay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20241021122112.101513-1-puranjay@kernel.org> References: <20241021122112.101513-1-puranjay@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add a microbenchmark for bpf_csum_diff() helper. This benchmark works by filling a 4KB buffer with random data and calculating the internet checksum on different parts of this buffer using bpf_csum_diff(). Example run using ./benchs/run_bench_csum_diff.sh on x86_64: [bpf]$ ./benchs/run_bench_csum_diff.sh 4 2.296 ± 0.066M/s (drops 0.000 ± 0.000M/s) 8 2.320 ± 0.003M/s (drops 0.000 ± 0.000M/s) 16 2.315 ± 0.001M/s (drops 0.000 ± 0.000M/s) 20 2.318 ± 0.001M/s (drops 0.000 ± 0.000M/s) 32 2.308 ± 0.003M/s (drops 0.000 ± 0.000M/s) 40 2.300 ± 0.029M/s (drops 0.000 ± 0.000M/s) 64 2.286 ± 0.001M/s (drops 0.000 ± 0.000M/s) 128 2.250 ± 0.001M/s (drops 0.000 ± 0.000M/s) 256 2.173 ± 0.001M/s (drops 0.000 ± 0.000M/s) 512 2.023 ± 0.055M/s (drops 0.000 ± 0.000M/s) Signed-off-by: Puranjay Mohan Acked-by: Daniel Borkmann --- tools/testing/selftests/bpf/Makefile | 2 + tools/testing/selftests/bpf/bench.c | 4 + .../selftests/bpf/benchs/bench_csum_diff.c | 164 ++++++++++++++++++ .../bpf/benchs/run_bench_csum_diff.sh | 10 ++ .../selftests/bpf/progs/csum_diff_bench.c | 25 +++ 5 files changed, 205 insertions(+) create mode 100644 tools/testing/selftests/bpf/benchs/bench_csum_diff.c create mode 100755 tools/testing/selftests/bpf/benchs/run_bench_csum_diff.sh create mode 100644 tools/testing/selftests/bpf/progs/csum_diff_bench.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 28a76baa854d3..a0d86dd453e16 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -809,6 +809,7 @@ $(OUTPUT)/bench_ringbufs.o: $(OUTPUT)/ringbuf_bench.skel.h \ $(OUTPUT)/bench_bloom_filter_map.o: $(OUTPUT)/bloom_filter_bench.skel.h $(OUTPUT)/bench_bpf_loop.o: $(OUTPUT)/bpf_loop_bench.skel.h $(OUTPUT)/bench_strncmp.o: $(OUTPUT)/strncmp_bench.skel.h +$(OUTPUT)/bench_csum_diff.o: $(OUTPUT)/csum_diff_bench.skel.h $(OUTPUT)/bench_bpf_hashmap_full_update.o: $(OUTPUT)/bpf_hashmap_full_update_bench.skel.h $(OUTPUT)/bench_local_storage.o: $(OUTPUT)/local_storage_bench.skel.h $(OUTPUT)/bench_local_storage_rcu_tasks_trace.o: $(OUTPUT)/local_storage_rcu_tasks_trace_bench.skel.h @@ -829,6 +830,7 @@ $(OUTPUT)/bench: $(OUTPUT)/bench.o \ $(OUTPUT)/bench_bloom_filter_map.o \ $(OUTPUT)/bench_bpf_loop.o \ $(OUTPUT)/bench_strncmp.o \ + $(OUTPUT)/bench_csum_diff.o \ $(OUTPUT)/bench_bpf_hashmap_full_update.o \ $(OUTPUT)/bench_local_storage.o \ $(OUTPUT)/bench_local_storage_rcu_tasks_trace.o \ diff --git a/tools/testing/selftests/bpf/bench.c b/tools/testing/selftests/bpf/bench.c index 1bd403a5ef7b3..29bd6f4498ebc 100644 --- a/tools/testing/selftests/bpf/bench.c +++ b/tools/testing/selftests/bpf/bench.c @@ -278,6 +278,7 @@ extern struct argp bench_bpf_loop_argp; extern struct argp bench_local_storage_argp; extern struct argp bench_local_storage_rcu_tasks_trace_argp; extern struct argp bench_strncmp_argp; +extern struct argp bench_csum_diff_argp; extern struct argp bench_hashmap_lookup_argp; extern struct argp bench_local_storage_create_argp; extern struct argp bench_htab_mem_argp; @@ -290,6 +291,7 @@ static const struct argp_child bench_parsers[] = { { &bench_bpf_loop_argp, 0, "bpf_loop helper benchmark", 0 }, { &bench_local_storage_argp, 0, "local_storage benchmark", 0 }, { &bench_strncmp_argp, 0, "bpf_strncmp helper benchmark", 0 }, + { &bench_csum_diff_argp, 0, "bpf_csum_diff helper benchmark", 0 }, { &bench_local_storage_rcu_tasks_trace_argp, 0, "local_storage RCU Tasks Trace slowdown benchmark", 0 }, { &bench_hashmap_lookup_argp, 0, "Hashmap lookup benchmark", 0 }, @@ -539,6 +541,7 @@ extern const struct bench bench_hashmap_with_bloom; extern const struct bench bench_bpf_loop; extern const struct bench bench_strncmp_no_helper; extern const struct bench bench_strncmp_helper; +extern const struct bench bench_csum_diff; extern const struct bench bench_bpf_hashmap_full_update; extern const struct bench bench_local_storage_cache_seq_get; extern const struct bench bench_local_storage_cache_interleaved_get; @@ -599,6 +602,7 @@ static const struct bench *benchs[] = { &bench_bpf_loop, &bench_strncmp_no_helper, &bench_strncmp_helper, + &bench_csum_diff, &bench_bpf_hashmap_full_update, &bench_local_storage_cache_seq_get, &bench_local_storage_cache_interleaved_get, diff --git a/tools/testing/selftests/bpf/benchs/bench_csum_diff.c b/tools/testing/selftests/bpf/benchs/bench_csum_diff.c new file mode 100644 index 0000000000000..2c30c8b54d9bc --- /dev/null +++ b/tools/testing/selftests/bpf/benchs/bench_csum_diff.c @@ -0,0 +1,164 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright Amazon.com Inc. or its affiliates */ +#include +#include "bench.h" +#include "csum_diff_bench.skel.h" + +static struct csum_diff_ctx { + struct csum_diff_bench *skel; + int pfd; +} ctx; + +static struct csum_diff_args { + u32 buff_len; +} args = { + .buff_len = 32, +}; + +enum { + ARG_BUFF_LEN = 5000, +}; + +static const struct argp_option opts[] = { + { "buff-len", ARG_BUFF_LEN, "BUFF_LEN", 0, + "Set the length of the buffer" }, + {}, +}; + +static error_t csum_diff_parse_arg(int key, char *arg, struct argp_state *state) +{ + switch (key) { + case ARG_BUFF_LEN: + args.buff_len = strtoul(arg, NULL, 10); + if (!args.buff_len || + args.buff_len >= sizeof(ctx.skel->rodata->buff)) { + fprintf(stderr, "Invalid buff len (limit %zu)\n", + sizeof(ctx.skel->rodata->buff)); + argp_usage(state); + } + break; + default: + return ARGP_ERR_UNKNOWN; + } + + return 0; +} + +const struct argp bench_csum_diff_argp = { + .options = opts, + .parser = csum_diff_parse_arg, +}; + +static void csum_diff_validate(void) +{ + if (env.consumer_cnt != 0) { + fprintf(stderr, "csum_diff benchmark doesn't support consumer!\n"); + exit(1); + } +} + +static void csum_diff_setup(void) +{ + int err; + char *buff; + size_t i, sz; + + sz = sizeof(ctx.skel->rodata->buff); + + setup_libbpf(); + + ctx.skel = csum_diff_bench__open(); + if (!ctx.skel) { + fprintf(stderr, "failed to open skeleton\n"); + exit(1); + } + + srandom(time(NULL)); + buff = ctx.skel->rodata->buff; + + /* + * Set first 8 bytes of buffer to 0xdeadbeefdeadbeef, this is later used to verify the + * correctness of the helper by comparing the checksum result for 0xdeadbeefdeadbeef that + * should be 0x3b3b + */ + + *(u64 *)buff = 0xdeadbeefdeadbeef; + + for (i = 8; i < sz; i++) + buff[i] = '1' + random() % 9; + + ctx.skel->rodata->buff_len = args.buff_len; + + err = csum_diff_bench__load(ctx.skel); + if (err) { + fprintf(stderr, "failed to load skeleton\n"); + csum_diff_bench__destroy(ctx.skel); + exit(1); + } +} + +static void csum_diff_helper_setup(void) +{ + u8 tmp_out[64 << 2] = {}; + u8 tmp_in[64] = {}; + int err, saved_errno; + + LIBBPF_OPTS(bpf_test_run_opts, topts, + .data_in = tmp_in, + .data_size_in = sizeof(tmp_in), + .data_out = tmp_out, + .data_size_out = sizeof(tmp_out), + .repeat = 1, + ); + csum_diff_setup(); + ctx.pfd = bpf_program__fd(ctx.skel->progs.compute_checksum); + + err = bpf_prog_test_run_opts(ctx.pfd, &topts); + saved_errno = errno; + + if (err) { + fprintf(stderr, "failed to run setup prog: err %d, result %d, serror %d\n", + err, ctx.skel->bss->result, saved_errno); + csum_diff_bench__destroy(ctx.skel); + exit(1); + } + + /* Sanity check for correctness of helper */ + if (args.buff_len == 8 && ctx.skel->bss->result != 0x3b3b) { + fprintf(stderr, "csum_diff helper broken: buff: %lx, result: %x, expected: %x\n", + *(u64 *)ctx.skel->rodata->buff, ctx.skel->bss->result, 0x3b3b); + } +} + +static void *csum_diff_producer(void *unused) +{ + u8 tmp_out[64 << 2] = {}; + u8 tmp_in[64] = {}; + + LIBBPF_OPTS(bpf_test_run_opts, topts, + .data_in = tmp_in, + .data_size_in = sizeof(tmp_in), + .data_out = tmp_out, + .data_size_out = sizeof(tmp_out), + .repeat = 64, + ); + while (true) + (void)bpf_prog_test_run_opts(ctx.pfd, &topts); + return NULL; +} + +static void csum_diff_measure(struct bench_res *res) +{ + res->hits = atomic_swap(&ctx.skel->bss->hits, 0); +} + +const struct bench bench_csum_diff = { + .name = "csum-diff-helper", + .argp = &bench_csum_diff_argp, + .validate = csum_diff_validate, + .setup = csum_diff_helper_setup, + .producer_thread = csum_diff_producer, + .measure = csum_diff_measure, + .report_progress = hits_drops_report_progress, + .report_final = hits_drops_report_final, +}; diff --git a/tools/testing/selftests/bpf/benchs/run_bench_csum_diff.sh b/tools/testing/selftests/bpf/benchs/run_bench_csum_diff.sh new file mode 100755 index 0000000000000..c4e147fbf2f98 --- /dev/null +++ b/tools/testing/selftests/bpf/benchs/run_bench_csum_diff.sh @@ -0,0 +1,10 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +source ./benchs/run_common.sh + +set -eufo pipefail + +for s in 4 8 16 20 32 40 64 128 256 512; do + summarize ${s} "$($RUN_BENCH --buff-len=$s csum-diff-helper)" +done diff --git a/tools/testing/selftests/bpf/progs/csum_diff_bench.c b/tools/testing/selftests/bpf/progs/csum_diff_bench.c new file mode 100644 index 0000000000000..85245edd6f9dc --- /dev/null +++ b/tools/testing/selftests/bpf/progs/csum_diff_bench.c @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright Amazon.com Inc. or its affiliates */ +#include +#include +#include +#include + +#define BUFF_SZ 4096 + +/* Will be updated by benchmark before program loading */ +const char buff[BUFF_SZ]; +const volatile unsigned int buff_len = 4; + +long hits = 0; +short result; + +char _license[] SEC("license") = "GPL"; + +SEC("tc") +int compute_checksum(void *ctx) +{ + result = bpf_csum_diff(0, 0, (void *)buff, buff_len, 0); + __sync_add_and_fetch(&hits, 1); + return 0; +} From patchwork Mon Oct 21 12:21:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13844015 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7BCB1FB3C1; Mon, 21 Oct 2024 12:22:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513333; cv=none; b=PNi86J5KYEzQbiorVN5m+4aUNp53vM1Hr+TmYFikf8+kdi7hJCVt9SMnaIRc3vaUQ/4SyiijzYcyVZDwF/RRlQYIb73cqKPmpUH4ABl4nUQGBogpFCTxMIvGmvmt/Bx/7dhWZzkr4Im1LpUsjN1REbjShKhL7odTOHz2CG+Hclg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729513333; c=relaxed/simple; bh=c8L0VpdriMONmtoRlW2frKvu4VJmvuZBqyXENZuKGNQ=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SpVt9umXZX2F0CYrMC/UDoThHzE4xxkGeDXMFy74vXewIoyclmXt2r/IaRLwAnV0iq/lsOIUwMKY3we3QoICpmzULU6j0lNU6esTNyASfT8sP5GzKbX8cSHtMbVMErEjDp+f3oZX0s8epTcYH7SYRGqi+9svcO8pXP5u+0xv/Rk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nNHdQ0fG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nNHdQ0fG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F17C7C4CEE4; Mon, 21 Oct 2024 12:22:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729513332; bh=c8L0VpdriMONmtoRlW2frKvu4VJmvuZBqyXENZuKGNQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=nNHdQ0fGTbVmymWgcizfMY7q1QsD0sF12JAOVPKoGuq6hX7e57v+sBc+UxNwunBAD dOXDgrU6SBfbvt5PW04qoQZXeoUInUq4ie9jpi7kfNjtNpav/7OWrdBFkqHaePe7kn Nbs7T3X/2WICOqFOfoLYjSEQ04f0l9ddY+57yUH8KcTvy9qzRq4AiErk61F0fb+L58 PetUVz/TWk2V6OK4mz3PkxqPmnsnV5hKT9rFKlXGXZCh0yk7fR24OmruakqRHMPvh/ Kpa4SlIRe4B/NS/a3w6LA3vHUituYKv+HeZOsUJD6oEO2rSZaspwNbro7rKjWLwjep 2x/TxXL1Nn5rA== From: Puranjay Mohan To: Albert Ou , Alexei Starovoitov , Andrew Morton , Andrii Nakryiko , bpf@vger.kernel.org, Daniel Borkmann , "David S. Miller" , Eduard Zingerman , Eric Dumazet , Hao Luo , Helge Deller , Jakub Kicinski , "James E.J. Bottomley" , Jiri Olsa , John Fastabend , KP Singh , linux-kernel@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, Martin KaFai Lau , Mykola Lysenko , netdev@vger.kernel.org, Palmer Dabbelt , Paolo Abeni , Paul Walmsley , Puranjay Mohan , Puranjay Mohan , Shuah Khan , Song Liu , Stanislav Fomichev , Yonghong Song Subject: [PATCH bpf-next 5/5] selftests/bpf: Add a selftest for bpf_csum_diff() Date: Mon, 21 Oct 2024 12:21:12 +0000 Message-Id: <20241021122112.101513-6-puranjay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20241021122112.101513-1-puranjay@kernel.org> References: <20241021122112.101513-1-puranjay@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add a selftest for the bpf_csum_diff() helper. This selftests runs the helper in all three configurations(push, pull, and diff) and verifies its output. The correct results have been computed by hand and by the helper's older implementation. Signed-off-by: Puranjay Mohan Acked-by: Daniel Borkmann --- .../selftests/bpf/prog_tests/test_csum_diff.c | 408 ++++++++++++++++++ .../selftests/bpf/progs/csum_diff_test.c | 42 ++ 2 files changed, 450 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_csum_diff.c create mode 100644 tools/testing/selftests/bpf/progs/csum_diff_test.c diff --git a/tools/testing/selftests/bpf/prog_tests/test_csum_diff.c b/tools/testing/selftests/bpf/prog_tests/test_csum_diff.c new file mode 100644 index 0000000000000..107b20d43e839 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/test_csum_diff.c @@ -0,0 +1,408 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright Amazon.com Inc. or its affiliates */ +#include +#include "csum_diff_test.skel.h" + +#define BUFF_SZ 512 + +struct testcase { + unsigned long long to_buff[BUFF_SZ / 8]; + unsigned int to_buff_len; + unsigned long long from_buff[BUFF_SZ / 8]; + unsigned int from_buff_len; + unsigned short seed; + unsigned short result; +}; + +#define NUM_PUSH_TESTS 4 + +struct testcase push_tests[NUM_PUSH_TESTS] = { + { + .to_buff = { + 0xdeadbeefdeadbeef, + }, + .to_buff_len = 8, + .from_buff = {}, + .from_buff_len = 0, + .seed = 0, + .result = 0x3b3b + }, + { + .to_buff = { + 0xdeadbeefdeadbeef, + 0xbeefdeadbeefdead, + }, + .to_buff_len = 16, + .from_buff = {}, + .from_buff_len = 0, + .seed = 0x1234, + .result = 0x88aa + }, + { + .to_buff = { + 0xdeadbeefdeadbeef, + 0xbeefdeadbeefdead, + }, + .to_buff_len = 15, + .from_buff = {}, + .from_buff_len = 0, + .seed = 0x1234, +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ + .result = 0xcaa9 +#else + .result = 0x87fd +#endif + }, + { + .to_buff = { + 0x327b23c66b8b4567, + 0x66334873643c9869, + 0x19495cff74b0dc51, + 0x625558ec2ae8944a, + 0x46e87ccd238e1f29, + 0x507ed7ab3d1b58ba, + 0x41b71efb2eb141f2, + 0x7545e14679e2a9e3, + 0x5bd062c2515f007c, + 0x4db127f812200854, + 0x1f16e9e80216231b, + 0x66ef438d1190cde7, + 0x3352255a140e0f76, + 0x0ded7263109cf92e, + 0x1befd79f7fdcc233, + 0x6b68079a41a7c4c9, + 0x25e45d324e6afb66, + 0x431bd7b7519b500d, + 0x7c83e4583f2dba31, + 0x62bbd95a257130a3, + 0x628c895d436c6125, + 0x721da317333ab105, + 0x2d1d5ae92443a858, + 0x75a2a8d46763845e, + 0x79838cb208edbdab, + 0x0b03e0c64353d0cd, + 0x54e49eb4189a769b, + 0x2ca8861171f32454, + 0x02901d820836c40e, + 0x081386413a95f874, + 0x7c3dbd3d1e7ff521, + 0x6ceaf087737b8ddc, + 0x4516dde922221a70, + 0x614fd4a13006c83e, + 0x5577f8e1419ac241, + 0x05072367440badfc, + 0x77465f013804823e, + 0x5c482a977724c67e, + 0x5e884adc2463b9ea, + 0x2d51779651ead36b, + 0x153ea438580bd78f, + 0x70a64e2a3855585c, + 0x2a487cb06a2342ec, + 0x725a06fb1d4ed43b, + 0x57e4ccaf2cd89a32, + 0x4b588f547a6d8d3c, + 0x6de91b18542289ec, + 0x7644a45c38437fdb, + 0x684a481a32fff902, + 0x749abb43579478fe, + 0x1ba026fa3dc240fb, + 0x75c6c33a79a1deaa, + 0x70c6a52912e685fb, + 0x374a3fe6520eedd1, + 0x23f9c13c4f4ef005, + 0x275ac794649bb77c, + 0x1cf10fd839386575, + 0x235ba861180115be, + 0x354fe9f947398c89, + 0x741226bb15b5af5c, + 0x10233c990d34b6a8, + 0x615740953f6ab60f, + 0x77ae35eb7e0c57b1, + 0x310c50b3579be4f1, + }, + .to_buff_len = 512, + .from_buff = {}, + .from_buff_len = 0, + .seed = 0xffff, + .result = 0xca45 + }, +}; + +#define NUM_PULL_TESTS 4 + +struct testcase pull_tests[NUM_PULL_TESTS] = { + { + .from_buff = { + 0xdeadbeefdeadbeef, + }, + .from_buff_len = 8, + .to_buff = {}, + .to_buff_len = 0, + .seed = 0, + .result = 0xc4c4 + }, + { + .from_buff = { + 0xdeadbeefdeadbeef, + 0xbeefdeadbeefdead, + }, + .from_buff_len = 16, + .to_buff = {}, + .to_buff_len = 0, + .seed = 0x1234, + .result = 0x9bbd + }, + { + .from_buff = { + 0xdeadbeefdeadbeef, + 0xbeefdeadbeefdead, + }, + .from_buff_len = 15, + .to_buff = {}, + .to_buff_len = 0, + .seed = 0x1234, +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ + .result = 0x59be +#else + .result = 0x9c6a +#endif + }, + { + .from_buff = { + 0x327b23c66b8b4567, + 0x66334873643c9869, + 0x19495cff74b0dc51, + 0x625558ec2ae8944a, + 0x46e87ccd238e1f29, + 0x507ed7ab3d1b58ba, + 0x41b71efb2eb141f2, + 0x7545e14679e2a9e3, + 0x5bd062c2515f007c, + 0x4db127f812200854, + 0x1f16e9e80216231b, + 0x66ef438d1190cde7, + 0x3352255a140e0f76, + 0x0ded7263109cf92e, + 0x1befd79f7fdcc233, + 0x6b68079a41a7c4c9, + 0x25e45d324e6afb66, + 0x431bd7b7519b500d, + 0x7c83e4583f2dba31, + 0x62bbd95a257130a3, + 0x628c895d436c6125, + 0x721da317333ab105, + 0x2d1d5ae92443a858, + 0x75a2a8d46763845e, + 0x79838cb208edbdab, + 0x0b03e0c64353d0cd, + 0x54e49eb4189a769b, + 0x2ca8861171f32454, + 0x02901d820836c40e, + 0x081386413a95f874, + 0x7c3dbd3d1e7ff521, + 0x6ceaf087737b8ddc, + 0x4516dde922221a70, + 0x614fd4a13006c83e, + 0x5577f8e1419ac241, + 0x05072367440badfc, + 0x77465f013804823e, + 0x5c482a977724c67e, + 0x5e884adc2463b9ea, + 0x2d51779651ead36b, + 0x153ea438580bd78f, + 0x70a64e2a3855585c, + 0x2a487cb06a2342ec, + 0x725a06fb1d4ed43b, + 0x57e4ccaf2cd89a32, + 0x4b588f547a6d8d3c, + 0x6de91b18542289ec, + 0x7644a45c38437fdb, + 0x684a481a32fff902, + 0x749abb43579478fe, + 0x1ba026fa3dc240fb, + 0x75c6c33a79a1deaa, + 0x70c6a52912e685fb, + 0x374a3fe6520eedd1, + 0x23f9c13c4f4ef005, + 0x275ac794649bb77c, + 0x1cf10fd839386575, + 0x235ba861180115be, + 0x354fe9f947398c89, + 0x741226bb15b5af5c, + 0x10233c990d34b6a8, + 0x615740953f6ab60f, + 0x77ae35eb7e0c57b1, + 0x310c50b3579be4f1, + }, + .from_buff_len = 512, + .to_buff = {}, + .to_buff_len = 0, + .seed = 0xffff, + .result = 0x35ba + }, +}; + +#define NUM_DIFF_TESTS 4 + +struct testcase diff_tests[NUM_DIFF_TESTS] = { + { + .from_buff = { + 0xdeadbeefdeadbeef, + }, + .from_buff_len = 8, + .to_buff = { + 0xabababababababab, + }, + .to_buff_len = 8, + .seed = 0, + .result = 0x7373 + }, + { + .from_buff = { + 0xdeadbeefdeadbeef, + }, + .from_buff_len = 7, + .to_buff = { + 0xabababababababab, + }, + .to_buff_len = 7, + .seed = 0, +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ + .result = 0xa673 +#else + .result = 0x73b7 +#endif + }, + { + .from_buff = { + 0, + }, + .from_buff_len = 8, + .to_buff = { + 0xabababababababab, + }, + .to_buff_len = 8, + .seed = 0, + .result = 0xaeae + }, + { + .from_buff = { + 0xdeadbeefdeadbeef + }, + .from_buff_len = 8, + .to_buff = { + 0, + }, + .to_buff_len = 8, + .seed = 0xffff, + .result = 0xc4c4 + }, +}; + +#define NUM_EDGE_TESTS 4 + +struct testcase edge_tests[NUM_EDGE_TESTS] = { + { + .from_buff = {}, + .from_buff_len = 0, + .to_buff = {}, + .to_buff_len = 0, + .seed = 0, + .result = 0 + }, + { + .from_buff = { + 0x1234 + }, + .from_buff_len = 0, + .to_buff = { + 0x1234 + }, + .to_buff_len = 0, + .seed = 0, + .result = 0 + }, + { + .from_buff = {}, + .from_buff_len = 0, + .to_buff = {}, + .to_buff_len = 0, + .seed = 0x1234, + .result = 0x1234 + }, + { + .from_buff = {}, + .from_buff_len = 512, + .to_buff = {}, + .to_buff_len = 0, + .seed = 0xffff, + .result = 0xffff + }, +}; + +static unsigned short trigger_csum_diff(const struct csum_diff_test *skel) +{ + u8 tmp_out[64 << 2] = {}; + u8 tmp_in[64] = {}; + int err; + int pfd; + + LIBBPF_OPTS(bpf_test_run_opts, topts, + .data_in = tmp_in, + .data_size_in = sizeof(tmp_in), + .data_out = tmp_out, + .data_size_out = sizeof(tmp_out), + .repeat = 1, + ); + pfd = bpf_program__fd(skel->progs.compute_checksum); + err = bpf_prog_test_run_opts(pfd, &topts); + if (err) + return -1; + + return skel->bss->result; +} + +static void test_csum_diff(struct testcase *tests, int num_tests) +{ + struct csum_diff_test *skel; + unsigned short got; + int err; + + for (int i = 0; i < num_tests; i++) { + skel = csum_diff_test__open(); + if (!ASSERT_OK_PTR(skel, "csum_diff_test open")) + return; + + skel->rodata->to_buff_len = tests[i].to_buff_len; + skel->rodata->from_buff_len = tests[i].from_buff_len; + + err = csum_diff_test__load(skel); + if (!ASSERT_EQ(err, 0, "csum_diff_test load")) + goto out; + + memcpy(skel->bss->to_buff, tests[i].to_buff, tests[i].to_buff_len); + memcpy(skel->bss->from_buff, tests[i].from_buff, tests[i].from_buff_len); + skel->bss->seed = tests[i].seed; + + got = trigger_csum_diff(skel); + ASSERT_EQ(got, tests[i].result, "csum_diff result"); + + csum_diff_test__destroy(skel); + } + + return; +out: + csum_diff_test__destroy(skel); +} + +void test_test_csum_diff(void) +{ + if (test__start_subtest("csum_diff_push")) + test_csum_diff(push_tests, NUM_PUSH_TESTS); + if (test__start_subtest("csum_diff_pull")) + test_csum_diff(pull_tests, NUM_PULL_TESTS); + if (test__start_subtest("csum_diff_diff")) + test_csum_diff(diff_tests, NUM_DIFF_TESTS); + if (test__start_subtest("csum_diff_edge")) + test_csum_diff(edge_tests, NUM_EDGE_TESTS); +} diff --git a/tools/testing/selftests/bpf/progs/csum_diff_test.c b/tools/testing/selftests/bpf/progs/csum_diff_test.c new file mode 100644 index 0000000000000..9438f1773a589 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/csum_diff_test.c @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright Amazon.com Inc. or its affiliates */ +#include +#include +#include +#include + +#define BUFF_SZ 512 + +/* Will be updated by benchmark before program loading */ +char to_buff[BUFF_SZ]; +const volatile unsigned int to_buff_len = 0; +char from_buff[BUFF_SZ]; +const volatile unsigned int from_buff_len = 0; +unsigned short seed = 0; + +short result; + +char _license[] SEC("license") = "GPL"; + +SEC("tc") +int compute_checksum(void *ctx) +{ + int to_len_half = to_buff_len / 2; + int from_len_half = from_buff_len / 2; + short result2; + + /* Calculate checksum in one go */ + result2 = bpf_csum_diff((void *)from_buff, from_buff_len, + (void *)to_buff, to_buff_len, seed); + + /* Calculate checksum by concatenating bpf_csum_diff()*/ + result = bpf_csum_diff((void *)from_buff, from_buff_len - from_len_half, + (void *)to_buff, to_buff_len - to_len_half, seed); + + result = bpf_csum_diff((void *)from_buff + (from_buff_len - from_len_half), from_len_half, + (void *)to_buff + (to_buff_len - to_len_half), to_len_half, result); + + result = (result == result2) ? result : 0; + + return 0; +}