From patchwork Fri Nov 18 19:44:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048723 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB271C4332F for ; Fri, 18 Nov 2022 19:46:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240445AbiKRTqd (ORCPT ); Fri, 18 Nov 2022 14:46:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235483AbiKRTq3 (ORCPT ); Fri, 18 Nov 2022 14:46:29 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C54AD8D482 for ; Fri, 18 Nov 2022 11:46:26 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 7824AB824F8 for ; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 14DC5C433D6; Fri, 18 Nov 2022 19:46:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800784; bh=pld3XDt3BvGRUOAeWpGABORV9PhFrHCg0jaaS0wG4JY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ctclds9W9ZQRSH26nNSD/4Ztv5PPT1NqhIwr7+W5kWizkj4yfFlpOdLRj/psyZVSt k+EdrCpXYDx6ZsoGWYiZJJPYkPTNlE8vpZnWLQNDKu9N4cYokXEmJpJYqIhQBthvpp a1qZb2p+czXkPd4h+pRTXsJ1nwPkPFoR5NTZngEaM0IfsQho2kDdlLWgFUO8OegUFo NAg6JV8PTfqFhiqQS7xMNHdoPfrl7OG/PZle6XjzX6pNk1vEL5a7QaxnwJa71XrrEn 709qI+aFBuclXf9C3g/x11nAaBanv8tiJU5QQh+mWfWSmNJKqzNhqJNbFS1KBS0VQ4 S5N9IJkjWE/BA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 01/12] crypto: x86/aegis128 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:10 -0800 Message-Id: <20221118194421.160414-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers crypto_aegis128_aesni_enc(), crypto_aegis128_aesni_enc_tail(), crypto_aegis128_aesni_dec(), and crypto_aegis128_aesni_dec_tail() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/aegis128-aesni-asm.S | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/aegis128-aesni-asm.S b/arch/x86/crypto/aegis128-aesni-asm.S index b48ddebb47489..cdf3215ec272c 100644 --- a/arch/x86/crypto/aegis128-aesni-asm.S +++ b/arch/x86/crypto/aegis128-aesni-asm.S @@ -7,6 +7,7 @@ */ #include +#include #include #define STATE0 %xmm0 @@ -402,7 +403,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_ad) * void crypto_aegis128_aesni_enc(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_enc) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_enc) FRAME_BEGIN cmp $0x10, LEN @@ -499,7 +500,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_enc) * void crypto_aegis128_aesni_enc_tail(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_enc_tail) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_enc_tail) FRAME_BEGIN /* load the state: */ @@ -556,7 +557,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_enc_tail) * void crypto_aegis128_aesni_dec(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_dec) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_dec) FRAME_BEGIN cmp $0x10, LEN @@ -653,7 +654,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_dec) * void crypto_aegis128_aesni_dec_tail(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_dec_tail) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_dec_tail) FRAME_BEGIN /* load the state: */ From patchwork Fri Nov 18 19:44:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048727 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CB25C4321E for ; Fri, 18 Nov 2022 19:46:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235465AbiKRTqg (ORCPT ); Fri, 18 Nov 2022 14:46:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235496AbiKRTqa (ORCPT ); Fri, 18 Nov 2022 14:46:30 -0500 Received: from sin.source.kernel.org (sin.source.kernel.org [IPv6:2604:1380:40e1:4800::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D9538DA45 for ; Fri, 18 Nov 2022 11:46:28 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 742E2CE2232 for ; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 60099C43470; Fri, 18 Nov 2022 19:46:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800784; bh=1AOSU6xzPkyZPNezMVrPhaztVE44hJw1ys9zdKY6koM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A+ktCSy592Yjx/NCkOseT3ZiTEEtueqN9gCPefhFCsZ7WThNX6ly9+2KQIdYpupDm LtqZdxpsDM1ViXm7uZFXm+wdrZZcvqW0zs20zoPzblN39ubY7N+kspDUSqMSlc3hL7 S1YRE8ERG9kHtmcgr5c7dBVPzA/vu2Z6VTCDw9Nvl4DbWVOwZ8mEd1s44kfgC2V/e2 1KieEmygbTqVoBhn4Kg52pRrOevsUYdzYV+MGC92TnIk1kvsG4eg4HesH4qgNT1HtG ukjQIlxdwsolBskgNvVR0UHY9lMwAWMOfXrs91QHaF2GHMKrmtg4i7f0Drx/OvU1nm JCiFvhpfmOvTw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , Taehee Yoo , "Peter Zijlstra (Intel)" Subject: [PATCH v2 02/12] crypto: x86/aria - fix crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:11 -0800 Message-Id: <20221118194421.160414-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers aria_aesni_avx_encrypt_16way(), aria_aesni_avx_decrypt_16way(), aria_aesni_avx_ctr_crypt_16way(), aria_aesni_avx_gfni_encrypt_16way(), aria_aesni_avx_gfni_decrypt_16way(), and aria_aesni_avx_gfni_ctr_crypt_16way() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure. Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Cc: Taehee Yoo Signed-off-by: Eric Biggers --- arch/x86/crypto/aria-aesni-avx-asm_64.S | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/x86/crypto/aria-aesni-avx-asm_64.S b/arch/x86/crypto/aria-aesni-avx-asm_64.S index c75fd7d015ed8..03ae4cd1d976a 100644 --- a/arch/x86/crypto/aria-aesni-avx-asm_64.S +++ b/arch/x86/crypto/aria-aesni-avx-asm_64.S @@ -7,6 +7,7 @@ */ #include +#include #include /* struct aria_ctx: */ @@ -913,7 +914,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_crypt_16way) RET; SYM_FUNC_END(__aria_aesni_avx_crypt_16way) -SYM_FUNC_START(aria_aesni_avx_encrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_encrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -938,7 +939,7 @@ SYM_FUNC_START(aria_aesni_avx_encrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_encrypt_16way) -SYM_FUNC_START(aria_aesni_avx_decrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_decrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1039,7 +1040,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_ctr_gen_keystream_16way) RET; SYM_FUNC_END(__aria_aesni_avx_ctr_gen_keystream_16way) -SYM_FUNC_START(aria_aesni_avx_ctr_crypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_ctr_crypt_16way) /* input: * %rdi: ctx * %rsi: dst @@ -1208,7 +1209,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_gfni_crypt_16way) RET; SYM_FUNC_END(__aria_aesni_avx_gfni_crypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1233,7 +1234,7 @@ SYM_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_gfni_encrypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1258,7 +1259,7 @@ SYM_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_gfni_decrypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_ctr_crypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_ctr_crypt_16way) /* input: * %rdi: ctx * %rsi: dst From patchwork Fri Nov 18 19:44:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048722 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F60FC4321E for ; Fri, 18 Nov 2022 19:46:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235453AbiKRTqc (ORCPT ); Fri, 18 Nov 2022 14:46:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235220AbiKRTq3 (ORCPT ); Fri, 18 Nov 2022 14:46:29 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F31788CF0D for ; Fri, 18 Nov 2022 11:46:25 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7B5956276D for ; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B23D2C43140; Fri, 18 Nov 2022 19:46:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800784; bh=EjO3B77EIOTiWRygSBkRyIXiyATPysPcP7wmb2ToCDs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VHXo4sVaL3o8Zv5oH/58o8qIXTUaVmfy/xvsdGatFdPmp72sZHeiCQamxe+6zQLsv OrtAe1wOfSxzp+zI/VCoo9wVEipbCdEj5JmFl965KkqVSp35U4Se/YvdlPGFpqfpfO 29qfQnUPE6rtrRt3x6LORz70j1MH2sk7M1heQar6wUhUoRsIdcorRcrZYafkhOU46f aFzNTyKA2fJNvXLm+ETMq9XrfxIpnJh4iIS5V7uncd4KkSvJ+sZGsV3d//tn8BUU0/ SxmOROrRMkYf7tWReVw+SrizAR1o/plLqnHF/tF5BQ5HPjG/S4N6dJacB4XgPnGus0 lgPGT5ayQVuiw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 03/12] crypto: x86/nhpoly1305 - eliminate unnecessary CFI wrappers Date: Fri, 18 Nov 2022 11:44:12 -0800 Message-Id: <20221118194421.160414-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers Since the CFI implementation now supports indirect calls to assembly functions, take advantage of that rather than use wrapper functions. Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/nh-avx2-x86_64.S | 5 +++-- arch/x86/crypto/nh-sse2-x86_64.S | 5 +++-- arch/x86/crypto/nhpoly1305-avx2-glue.c | 11 ++--------- arch/x86/crypto/nhpoly1305-sse2-glue.c | 11 ++--------- 4 files changed, 10 insertions(+), 22 deletions(-) diff --git a/arch/x86/crypto/nh-avx2-x86_64.S b/arch/x86/crypto/nh-avx2-x86_64.S index 6a0b15e7196a8..ef73a3ab87263 100644 --- a/arch/x86/crypto/nh-avx2-x86_64.S +++ b/arch/x86/crypto/nh-avx2-x86_64.S @@ -8,6 +8,7 @@ */ #include +#include #define PASS0_SUMS %ymm0 #define PASS1_SUMS %ymm1 @@ -65,11 +66,11 @@ /* * void nh_avx2(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_avx2) +SYM_TYPED_FUNC_START(nh_avx2) vmovdqu 0x00(KEY), K0 vmovdqu 0x10(KEY), K1 diff --git a/arch/x86/crypto/nh-sse2-x86_64.S b/arch/x86/crypto/nh-sse2-x86_64.S index 34c567bbcb4fa..75fb994b6d177 100644 --- a/arch/x86/crypto/nh-sse2-x86_64.S +++ b/arch/x86/crypto/nh-sse2-x86_64.S @@ -8,6 +8,7 @@ */ #include +#include #define PASS0_SUMS %xmm0 #define PASS1_SUMS %xmm1 @@ -67,11 +68,11 @@ /* * void nh_sse2(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_sse2) +SYM_TYPED_FUNC_START(nh_sse2) movdqu 0x00(KEY), K0 movdqu 0x10(KEY), K1 diff --git a/arch/x86/crypto/nhpoly1305-avx2-glue.c b/arch/x86/crypto/nhpoly1305-avx2-glue.c index 8ea5ab0f1ca74..46b036204ed91 100644 --- a/arch/x86/crypto/nhpoly1305-avx2-glue.c +++ b/arch/x86/crypto/nhpoly1305-avx2-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_avx2(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_avx2(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_avx2(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_avx2_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_avx2_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_fpu_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_avx2); + crypto_nhpoly1305_update_helper(desc, src, n, nh_avx2); kernel_fpu_end(); src += n; srclen -= n; diff --git a/arch/x86/crypto/nhpoly1305-sse2-glue.c b/arch/x86/crypto/nhpoly1305-sse2-glue.c index 2b353d42ed13f..4a4970d751076 100644 --- a/arch/x86/crypto/nhpoly1305-sse2-glue.c +++ b/arch/x86/crypto/nhpoly1305-sse2-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_sse2(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_sse2(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_sse2(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_sse2_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_sse2_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_fpu_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_sse2); + crypto_nhpoly1305_update_helper(desc, src, n, nh_sse2); kernel_fpu_end(); src += n; srclen -= n; From patchwork Fri Nov 18 19:44:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048725 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBF85C433FE for ; Fri, 18 Nov 2022 19:46:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235379AbiKRTqi (ORCPT ); Fri, 18 Nov 2022 14:46:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235521AbiKRTqa (ORCPT ); Fri, 18 Nov 2022 14:46:30 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5F4B8D4A8 for ; Fri, 18 Nov 2022 11:46:27 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6C1E9B82504 for ; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B446C43145; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800785; bh=SJD1vbe3TbIhw+mwTcm5uBRYSsM1TzGphUBe1liqWrE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XVl2DMLVSs2HOx7cBmnsYJTM66kDDvXWPb6kNYxobQXvoNILZHP5QWdeeXp/xZX69 smMIgUE6p983K51FzNpAq24zTzSx+yVw1MMZc39NtTKCIdUkOb8MwdpH8QddBd+UCK aL7sM1o8a+PKgpxaxhSHsy/mIk8NQMdKyKLTVewufHX178cAhJ6ny04svwuucxOph/ uWYJncSByfA3ZtSqI5L5308goQSDGqKieLiK0bRgUj8RU+nUeRK5IyvyoOQE0IwGhx GOtXnFByE0zD+Qwmp24fzNwaP90B6VIDExux0LOgPgYRJq8KqBFsYtsg/TygwvGxA8 uOz10di6uwVgA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 04/12] crypto: x86/sha1 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:13 -0800 Message-Id: <20221118194421.160414-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sha1_transform_ssse3(), sha1_transform_avx(), and sha1_ni_transform() (but not sha1_transform_avx2()) are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/sha1_ni_asm.S | 3 ++- arch/x86/crypto/sha1_ssse3_asm.S | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/crypto/sha1_ni_asm.S b/arch/x86/crypto/sha1_ni_asm.S index 2f94ec0e763bf..3cae5a1bb3d6e 100644 --- a/arch/x86/crypto/sha1_ni_asm.S +++ b/arch/x86/crypto/sha1_ni_asm.S @@ -54,6 +54,7 @@ */ #include +#include #define DIGEST_PTR %rdi /* 1st arg */ #define DATA_PTR %rsi /* 2nd arg */ @@ -93,7 +94,7 @@ */ .text .align 32 -SYM_FUNC_START(sha1_ni_transform) +SYM_TYPED_FUNC_START(sha1_ni_transform) push %rbp mov %rsp, %rbp sub $FRAME_SIZE, %rsp diff --git a/arch/x86/crypto/sha1_ssse3_asm.S b/arch/x86/crypto/sha1_ssse3_asm.S index 263f916362e02..f54988c80eb40 100644 --- a/arch/x86/crypto/sha1_ssse3_asm.S +++ b/arch/x86/crypto/sha1_ssse3_asm.S @@ -25,6 +25,7 @@ */ #include +#include #define CTX %rdi // arg1 #define BUF %rsi // arg2 @@ -67,7 +68,7 @@ * param: function's name */ .macro SHA1_VECTOR_ASM name - SYM_FUNC_START(\name) + SYM_TYPED_FUNC_START(\name) push %rbx push %r12 From patchwork Fri Nov 18 19:44:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048721 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AD3BC433FE for ; Fri, 18 Nov 2022 19:46:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235446AbiKRTqb (ORCPT ); Fri, 18 Nov 2022 14:46:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235350AbiKRTq3 (ORCPT ); Fri, 18 Nov 2022 14:46:29 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83B708CF37 for ; Fri, 18 Nov 2022 11:46:26 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A99506276A for ; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5721AC43147; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800785; bh=qDEd6yjezAEKyeXhBgJqixcDjy7TkZZTMaumiYQZh3k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DBEPBR4DRSkF+Kb1ECCcGEn1Ubxv/dV/G+6aLXG4NvyXwplXEooeAdrG7XPZwaYKc j87GJp+ZztUXH6b6/DMu4YSbJ8g+K59EA+CgWzOtZmwy2FOtkx48JOpq4AP0jXjfQN UtvJ/BBhly38i9XphhacRGk0DiblV3Z0LH8hjnUfHIfI0Gh263SU4JMObQi6CM+hIq ZG7DoKNkPXOe+xxFWl5yy0iWYViMKuFKVcq1fPmKEXFbpHUEKCkDUHBQzOweqZDsnC nf14oYLw5fn3Szd9kQAPbtzdNC0zvehuAIhFTpydcbjgt8DcoAJB2GsZ2xvGZVieRa QjlqG1okzjFWA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 05/12] crypto: x86/sha256 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:14 -0800 Message-Id: <20221118194421.160414-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sha256_transform_ssse3(), sha256_transform_avx(), sha256_transform_rorx(), and sha256_ni_transform() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/sha256-avx-asm.S | 3 ++- arch/x86/crypto/sha256-avx2-asm.S | 3 ++- arch/x86/crypto/sha256-ssse3-asm.S | 3 ++- arch/x86/crypto/sha256_ni_asm.S | 3 ++- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/sha256-avx-asm.S b/arch/x86/crypto/sha256-avx-asm.S index 3baa1ec390974..06ea30c20828d 100644 --- a/arch/x86/crypto/sha256-avx-asm.S +++ b/arch/x86/crypto/sha256-avx-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define VMOVDQ vmovdqu @@ -346,7 +347,7 @@ a = TMP_ ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_avx) +SYM_TYPED_FUNC_START(sha256_transform_avx) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256-avx2-asm.S b/arch/x86/crypto/sha256-avx2-asm.S index 9bcdbc47b8b4b..2d2be531a11ed 100644 --- a/arch/x86/crypto/sha256-avx2-asm.S +++ b/arch/x86/crypto/sha256-avx2-asm.S @@ -49,6 +49,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define VMOVDQ vmovdqu @@ -523,7 +524,7 @@ STACK_SIZE = _CTX + _CTX_SIZE ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_rorx) +SYM_TYPED_FUNC_START(sha256_transform_rorx) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256-ssse3-asm.S b/arch/x86/crypto/sha256-ssse3-asm.S index c4a5db612c327..7db28839108dd 100644 --- a/arch/x86/crypto/sha256-ssse3-asm.S +++ b/arch/x86/crypto/sha256-ssse3-asm.S @@ -47,6 +47,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define MOVDQ movdqu @@ -355,7 +356,7 @@ a = TMP_ ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_ssse3) +SYM_TYPED_FUNC_START(sha256_transform_ssse3) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256_ni_asm.S b/arch/x86/crypto/sha256_ni_asm.S index 94d50dd27cb53..47f93937f798a 100644 --- a/arch/x86/crypto/sha256_ni_asm.S +++ b/arch/x86/crypto/sha256_ni_asm.S @@ -54,6 +54,7 @@ */ #include +#include #define DIGEST_PTR %rdi /* 1st arg */ #define DATA_PTR %rsi /* 2nd arg */ @@ -97,7 +98,7 @@ .text .align 32 -SYM_FUNC_START(sha256_ni_transform) +SYM_TYPED_FUNC_START(sha256_ni_transform) shl $6, NUM_BLKS /* convert to bytes */ jz .Ldone_hash From patchwork Fri Nov 18 19:44:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048724 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 572D1C43219 for ; Fri, 18 Nov 2022 19:46:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235483AbiKRTqe (ORCPT ); Fri, 18 Nov 2022 14:46:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235465AbiKRTq3 (ORCPT ); Fri, 18 Nov 2022 14:46:29 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83CAC8CFCC for ; Fri, 18 Nov 2022 11:46:26 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0220C62779 for ; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A245FC43146; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800785; bh=HdiSoWnZUTCZpmQjT2Un6tm8QTRonixjlwqY064EQSg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nFUVKDQu84QOhgaSio7dWBuVl7bRghkrifp3HXo62eW9ILO3xqm2T04vi0Z4xxAJ7 n6eojRyj2Kk68HPINIAorrqNaF+5kyNJXdXq5zWcK/cVfG3q8EeGiqARm3T+RGqSXW hOCeOmqUXt+q0H2UwCwDwK2UdriOXz8O3Kfae56ZkzjoiC84Hufd/jpwvuGCmhdjD7 rXfWtNsg8XXpyo6wg+EWQs1otRvV5cN3WJw1fQxhIxmj5KcJPRIAD/+S6hhnRGPdZE CW2MCG72y4aDWei6QXJTrQ9rpq+Z9Jx5R67rz7LByV3Ttuq0i5YfVccY2x73D16fhx mQZgIX+FDE7MA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 06/12] crypto: x86/sha512 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:15 -0800 Message-Id: <20221118194421.160414-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sha512_transform_ssse3(), sha512_transform_avx(), and sha512_transform_rorx() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/sha512-avx-asm.S | 3 ++- arch/x86/crypto/sha512-avx2-asm.S | 3 ++- arch/x86/crypto/sha512-ssse3-asm.S | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/arch/x86/crypto/sha512-avx-asm.S b/arch/x86/crypto/sha512-avx-asm.S index 1fefe6dd3a9e2..b0984f19fdb40 100644 --- a/arch/x86/crypto/sha512-avx-asm.S +++ b/arch/x86/crypto/sha512-avx-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include .text @@ -273,7 +274,7 @@ frame_size = frame_WK + WK_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks ######################################################################## -SYM_FUNC_START(sha512_transform_avx) +SYM_TYPED_FUNC_START(sha512_transform_avx) test msglen, msglen je nowork diff --git a/arch/x86/crypto/sha512-avx2-asm.S b/arch/x86/crypto/sha512-avx2-asm.S index 5cdaab7d69015..b1ca99055ef99 100644 --- a/arch/x86/crypto/sha512-avx2-asm.S +++ b/arch/x86/crypto/sha512-avx2-asm.S @@ -50,6 +50,7 @@ ######################################################################## #include +#include .text @@ -565,7 +566,7 @@ frame_size = frame_CTX + CTX_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks ######################################################################## -SYM_FUNC_START(sha512_transform_rorx) +SYM_TYPED_FUNC_START(sha512_transform_rorx) # Save GPRs push %rbx push %r12 diff --git a/arch/x86/crypto/sha512-ssse3-asm.S b/arch/x86/crypto/sha512-ssse3-asm.S index b84c22e06c5f7..c06afb5270e5f 100644 --- a/arch/x86/crypto/sha512-ssse3-asm.S +++ b/arch/x86/crypto/sha512-ssse3-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include .text @@ -274,7 +275,7 @@ frame_size = frame_WK + WK_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks. ######################################################################## -SYM_FUNC_START(sha512_transform_ssse3) +SYM_TYPED_FUNC_START(sha512_transform_ssse3) test msglen, msglen je nowork From patchwork Fri Nov 18 19:44:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048726 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D114C4167B for ; Fri, 18 Nov 2022 19:46:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241316AbiKRTqh (ORCPT ); Fri, 18 Nov 2022 14:46:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235467AbiKRTq3 (ORCPT ); Fri, 18 Nov 2022 14:46:29 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B15E88CFFB for ; Fri, 18 Nov 2022 11:46:26 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4C4966277E for ; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ED9CCC433D7; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800786; bh=kMW+VagvP+m/FfoCh7s6tC3GVUtFQvTJqaIKESW1UH4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JjeVGn8nldBI641WZBxQtxPLNepo8CQPS8YNSbUU43cGsQ3/2Ak+xN3DP1rAfDoEB JLzx4iADDlNubhwQTePqull+hV/RL/S6w1I6lt5C9kGxk5YI1DD4oAEW94eyE0hYLT ZAYE3smvPyy2nuJIh7wAUNwJk2etU9xN9S390jiSJC9A5VWDBKDWHdht9NHeSzRUl4 w5TVIhkvTvCg8a9vsG9SR7HKwc1MHeSpma80kkxCn7SKUgMAVp6OxkZLc33ifmiowD ySn88t/8XZEo950gVGmL5UkxFamcR0rGJA20hFtTe7n0DXetlJDzg8Lcv/bL3nFUd4 5iuZ5lz/Ji5Xg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 07/12] crypto: x86/sm3 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:16 -0800 Message-Id: <20221118194421.160414-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sm3_transform_avx() is called via indirect function calls. Therefore it needs to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause its type hash to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect call). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/sm3-avx-asm_64.S | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/crypto/sm3-avx-asm_64.S b/arch/x86/crypto/sm3-avx-asm_64.S index b12b9efb5ec51..8fc5ac681fd63 100644 --- a/arch/x86/crypto/sm3-avx-asm_64.S +++ b/arch/x86/crypto/sm3-avx-asm_64.S @@ -12,6 +12,7 @@ */ #include +#include #include /* Context structure */ @@ -328,7 +329,7 @@ * const u8 *data, int nblocks); */ .align 16 -SYM_FUNC_START(sm3_transform_avx) +SYM_TYPED_FUNC_START(sm3_transform_avx) /* input: * %rdi: ctx, CTX * %rsi: data (64*nblks bytes) From patchwork Fri Nov 18 19:44:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048728 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B695FC4332F for ; Fri, 18 Nov 2022 19:46:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235521AbiKRTqj (ORCPT ); Fri, 18 Nov 2022 14:46:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235567AbiKRTqa (ORCPT ); Fri, 18 Nov 2022 14:46:30 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84D85DEBC for ; Fri, 18 Nov 2022 11:46:28 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9065262782 for ; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 44E88C433C1; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800786; bh=pclA3LA8eoFmKTrqSAJh4mYcWrvybcXjZP/RNOkyKcY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=baeL6vobI7CZ5qsql2NjsgVd2NX3vsNjyXPqOkD9r/8ri1L43BvyGRYwqJ0GWLo8V bzIXh8dC3LswOmK2RxnFmlzWXc29fOD0mpjm+gOuALMNvDUI2SQi5Oxxk5uVnLtZbw o0W7thLe8AvO/Qwq36p6xejmKTQUFOlSj3ILSaaV3D3tYASC9TGKcG96t85shGigaL NQBOjzO2gixywvwmeeKbGmmQauoK1asj8ojR3LvtTspf3acolK5b+QljHMGT1nVNl2 3IJ7fMnrIsjJRzvB3MMY0vl8n4OZ7IVPQ5HVQUOsR+OSNbeKihX1YRizy/38SbXyJf V2ZCXa5Bu7EMQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH v2 08/12] crypto: x86/sm4 - fix crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:17 -0800 Message-Id: <20221118194421.160414-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sm4_aesni_avx_ctr_enc_blk8(), sm4_aesni_avx_cbc_dec_blk8(), sm4_aesni_avx_cfb_dec_blk8(), sm4_aesni_avx2_ctr_enc_blk16(), sm4_aesni_avx2_cbc_dec_blk16(), and sm4_aesni_avx2_cfb_dec_blk16() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure. (Or at least that should be the case. For some reason the CFI checks in sm4_avx_cbc_decrypt(), sm4_avx_cfb_decrypt(), and sm4_avx_ctr_crypt() are not always being generated, using current tip-of-tree clang. Anyway, this patch is a good idea anyway.) Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Signed-off-by: Eric Biggers --- arch/x86/crypto/sm4-aesni-avx-asm_64.S | 7 ++++--- arch/x86/crypto/sm4-aesni-avx2-asm_64.S | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/arch/x86/crypto/sm4-aesni-avx-asm_64.S b/arch/x86/crypto/sm4-aesni-avx-asm_64.S index 4767ab61ff489..22b6560eb9e1e 100644 --- a/arch/x86/crypto/sm4-aesni-avx-asm_64.S +++ b/arch/x86/crypto/sm4-aesni-avx-asm_64.S @@ -14,6 +14,7 @@ */ #include +#include #include #define rRIP (%rip) @@ -420,7 +421,7 @@ SYM_FUNC_END(sm4_aesni_avx_crypt8) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx_ctr_enc_blk8) +SYM_TYPED_FUNC_START(sm4_aesni_avx_ctr_enc_blk8) /* input: * %rdi: round key array, CTX * %rsi: dst (8 blocks) @@ -495,7 +496,7 @@ SYM_FUNC_END(sm4_aesni_avx_ctr_enc_blk8) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx_cbc_dec_blk8) +SYM_TYPED_FUNC_START(sm4_aesni_avx_cbc_dec_blk8) /* input: * %rdi: round key array, CTX * %rsi: dst (8 blocks) @@ -545,7 +546,7 @@ SYM_FUNC_END(sm4_aesni_avx_cbc_dec_blk8) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx_cfb_dec_blk8) +SYM_TYPED_FUNC_START(sm4_aesni_avx_cfb_dec_blk8) /* input: * %rdi: round key array, CTX * %rsi: dst (8 blocks) diff --git a/arch/x86/crypto/sm4-aesni-avx2-asm_64.S b/arch/x86/crypto/sm4-aesni-avx2-asm_64.S index 4732fe8bb65b6..23ee39a8ada8c 100644 --- a/arch/x86/crypto/sm4-aesni-avx2-asm_64.S +++ b/arch/x86/crypto/sm4-aesni-avx2-asm_64.S @@ -14,6 +14,7 @@ */ #include +#include #include #define rRIP (%rip) @@ -282,7 +283,7 @@ SYM_FUNC_END(__sm4_crypt_blk16) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx2_ctr_enc_blk16) +SYM_TYPED_FUNC_START(sm4_aesni_avx2_ctr_enc_blk16) /* input: * %rdi: round key array, CTX * %rsi: dst (16 blocks) @@ -395,7 +396,7 @@ SYM_FUNC_END(sm4_aesni_avx2_ctr_enc_blk16) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx2_cbc_dec_blk16) +SYM_TYPED_FUNC_START(sm4_aesni_avx2_cbc_dec_blk16) /* input: * %rdi: round key array, CTX * %rsi: dst (16 blocks) @@ -449,7 +450,7 @@ SYM_FUNC_END(sm4_aesni_avx2_cbc_dec_blk16) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx2_cfb_dec_blk16) +SYM_TYPED_FUNC_START(sm4_aesni_avx2_cfb_dec_blk16) /* input: * %rdi: round key array, CTX * %rsi: dst (16 blocks) From patchwork Fri Nov 18 19:44:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048730 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A626C4332F for ; Fri, 18 Nov 2022 19:46:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241649AbiKRTqm (ORCPT ); Fri, 18 Nov 2022 14:46:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235533AbiKRTqa (ORCPT ); Fri, 18 Nov 2022 14:46:30 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8522813D53 for ; Fri, 18 Nov 2022 11:46:28 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4D94F62784 for ; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89731C433D6; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800786; bh=X+7N1v7Ylhc12ODIGzGIdWud9IGuRB4PQ36hvFoPOac=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rtCymIaJdlKGgicPDjzScEGeZBPbUtIgcMmx+JdSB+XgIeGV2D9NzjsYFOZBKJxbC mTmt6DKomct3OOuoZhJivhmhqsmBxhXZlaGdj6nyQ9/vOcZHlV6EybyfhnEphUV45W UzW60U0Jo4lFOKdgseR19Ex2eWkHZscbJJdsANorpstc7QhK4MzZN5bqV6KeZTNCmC 4fFMhbXOX7C4u9+jxk4WeZ+y/lJ9fDhD76a4ZhGDLUk5uRVAmKB5ngqS1dTOHcsPeN 8hG6t1IxUJsakY7i/b20aE0Q8HjbaCGCpMcAX0zH/XZxEmcE0EisvfQcxC38pCPBlZ B1Q5hcLouLkaA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 09/12] crypto: arm64/nhpoly1305 - eliminate unnecessary CFI wrapper Date: Fri, 18 Nov 2022 11:44:18 -0800 Message-Id: <20221118194421.160414-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers Since the CFI implementation now supports indirect calls to assembly functions, take advantage of that rather than use a wrapper function. Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/arm64/crypto/nh-neon-core.S | 5 +++-- arch/arm64/crypto/nhpoly1305-neon-glue.c | 11 ++--------- 2 files changed, 5 insertions(+), 11 deletions(-) diff --git a/arch/arm64/crypto/nh-neon-core.S b/arch/arm64/crypto/nh-neon-core.S index 51c0a534ef87c..13eda08fda1e5 100644 --- a/arch/arm64/crypto/nh-neon-core.S +++ b/arch/arm64/crypto/nh-neon-core.S @@ -8,6 +8,7 @@ */ #include +#include KEY .req x0 MESSAGE .req x1 @@ -58,11 +59,11 @@ /* * void nh_neon(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_neon) +SYM_TYPED_FUNC_START(nh_neon) ld1 {K0.4s,K1.4s}, [KEY], #32 movi PASS0_SUMS.2d, #0 diff --git a/arch/arm64/crypto/nhpoly1305-neon-glue.c b/arch/arm64/crypto/nhpoly1305-neon-glue.c index c5405e6a6db76..cd882c35d9252 100644 --- a/arch/arm64/crypto/nhpoly1305-neon-glue.c +++ b/arch/arm64/crypto/nhpoly1305-neon-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_neon(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_neon(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_neon(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_neon_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_neon_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_neon_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_neon); + crypto_nhpoly1305_update_helper(desc, src, n, nh_neon); kernel_neon_end(); src += n; srclen -= n; From patchwork Fri Nov 18 19:44:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048731 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE3AFC433FE for ; Fri, 18 Nov 2022 19:46:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235220AbiKRTqo (ORCPT ); Fri, 18 Nov 2022 14:46:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235594AbiKRTqb (ORCPT ); Fri, 18 Nov 2022 14:46:31 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E1F08CFDE for ; Fri, 18 Nov 2022 11:46:29 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 50C1CB824C8 for ; Fri, 18 Nov 2022 19:46:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D64BCC433B5; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800787; bh=502XtfzzJCvn41Ye4mf8jH8Z9XzhNMTzAZtoz+ow1Mk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SvwBQUsI23XQ04sTI+nMVJ/WdNE5SRFV0xg5sQjzZmEWJ1B+c7rFPuO1rZ6ZiApwe oN0DQyrJnBsRXkaoO9X8r3laLNge0EW+fgfc6La6YGGQKvYuLZobjj3n8AkCMhC8mf gOpZJOhwNEVzfBoL7N5xSXxS6HEVgx6G/y5WmkR3LGvn79IeLRXprShuZwm2R4cnRB G8PqvC13bsWK+9I4PZkHV/TLQhlxmBxO7JVrraQ7AwrR5n2+PHMp5YwEAsTxy55WQw VT6e73YReQNO5UW1oa5/5CTixIuDmDQIHWoGgQiiMGWEe7EVzdvKrUIl6Kw8W45yry I5s/S5nTK7/1g== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 10/12] crypto: arm64/sm3 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:19 -0800 Message-Id: <20221118194421.160414-11-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sm3_neon_transform() is called via indirect function calls. Therefore it needs to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause its type hash to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect call). Fixes: c50d32859e70 ("arm64: Add types to indirect called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/arm64/crypto/sm3-neon-core.S | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/crypto/sm3-neon-core.S b/arch/arm64/crypto/sm3-neon-core.S index 3e3b4e5c736fc..4357e0e51be38 100644 --- a/arch/arm64/crypto/sm3-neon-core.S +++ b/arch/arm64/crypto/sm3-neon-core.S @@ -9,6 +9,7 @@ */ #include +#include #include /* Context structure */ @@ -351,7 +352,7 @@ */ .text .align 3 -SYM_FUNC_START(sm3_neon_transform) +SYM_TYPED_FUNC_START(sm3_neon_transform) ldp ra, rb, [RSTATE, #0] ldp rc, rd, [RSTATE, #8] ldp re, rf, [RSTATE, #16] From patchwork Fri Nov 18 19:44:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048729 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE8DDC43217 for ; Fri, 18 Nov 2022 19:46:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235528AbiKRTqk (ORCPT ); Fri, 18 Nov 2022 14:46:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235556AbiKRTqa (ORCPT ); Fri, 18 Nov 2022 14:46:30 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7BD458CFD3 for ; Fri, 18 Nov 2022 11:46:29 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8064562789 for ; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2DA40C43470; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800787; bh=Jktv+/Qu/uCCZJ2itmlKVBoDqxZ3t6fI03aW0+AnaOA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g0nHEAK6+gGwwvbGug2zU5o5c9AvD4J8LVD7A5I0bsJep+SZ+nLXLmWdCKLTHuToc gmz0GC4mEPhWbunktEF3A4pKjjUaxa0E/Ypzs2v+vJB/T8SaqYh57y1W3wFN/X7Nd/ 61MPbQJE/3yZUASVFukPNYvg50koFQy7Ied7Wye9UX5rUAA1rhBF25YYEi61gj/SId WoUsuILKqmHaT80VGqyHIjdSrLH/sZwobE6dKDuEZ18bLaex8VOv/Dl8HK+jZcRUYa nbR24R9KPBk8CLjzFvEvGg2CcDfQ26figvmVvtvbxwDp1yykkePPEj6IejOjWr75LD 8jqOfsYpb6JgQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 11/12] crypto: arm/nhpoly1305 - eliminate unnecessary CFI wrapper Date: Fri, 18 Nov 2022 11:44:20 -0800 Message-Id: <20221118194421.160414-12-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers The arm architecture doesn't support CFI yet, and even if it did, the new CFI implementation supports indirect calls to assembly functions. Therefore, there's no need to use a wrapper function for nh_neon(). Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/arm/crypto/nh-neon-core.S | 2 +- arch/arm/crypto/nhpoly1305-neon-glue.c | 11 ++--------- 2 files changed, 3 insertions(+), 10 deletions(-) diff --git a/arch/arm/crypto/nh-neon-core.S b/arch/arm/crypto/nh-neon-core.S index 434d80ab531c2..01620a0782ca9 100644 --- a/arch/arm/crypto/nh-neon-core.S +++ b/arch/arm/crypto/nh-neon-core.S @@ -69,7 +69,7 @@ /* * void nh_neon(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ diff --git a/arch/arm/crypto/nhpoly1305-neon-glue.c b/arch/arm/crypto/nhpoly1305-neon-glue.c index ffa8d73fe722c..e93e41ff26566 100644 --- a/arch/arm/crypto/nhpoly1305-neon-glue.c +++ b/arch/arm/crypto/nhpoly1305-neon-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_neon(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_neon(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_neon(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_neon_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_neon_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_neon_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_neon); + crypto_nhpoly1305_update_helper(desc, src, n, nh_neon); kernel_neon_end(); src += n; srclen -= n; From patchwork Fri Nov 18 19:44:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048732 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2860EC43217 for ; Fri, 18 Nov 2022 19:46:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239864AbiKRTqp (ORCPT ); Fri, 18 Nov 2022 14:46:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235463AbiKRTqb (ORCPT ); Fri, 18 Nov 2022 14:46:31 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7BCDC7818F for ; Fri, 18 Nov 2022 11:46:29 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CBBD662792 for ; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79032C43140; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800787; bh=kEiHXZili2H4efP5Zbt6QZIEDbUdz7NL5DjoHnDhWac=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ErSsA5S03M6uPJgYtoNYzpyGRsncDw5v9YTnjC2g1yf8Gtq/ce6QqMvNPysmNpyuo bYGdjDIE6a9BnUfpjq+NZRkN3g3HvNK9pXmR3dOf+OfKjs6qRu3O8UdW0Qu5ehCCHk ZLkprHysfZPvjQciMmGKDcwBqIeAyDMP0dBd+STirNqJjmpwjP39X1Mh+m/vvc+lYC NYU/FGL53zTFGOHqK/q7sxgZBkT80W3rnKbQ2ZTt0Kch3xciVjXRj5q2iPBMMJ6Muv RlkYp4IMfdIwuH2h5eNrdWftJWHyPRJDn2d+8xl6VxgpnLC1yB3C5xE209TQwl5ghF 1kjnVVuXiMbzw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 12/12] Revert "crypto: shash - avoid comparing pointers to exported functions under CFI" Date: Fri, 18 Nov 2022 11:44:21 -0800 Message-Id: <20221118194421.160414-13-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers This reverts commit 22ca9f4aaf431a9413dcc115dd590123307f274f because CFI no longer breaks cross-module function address equality, so crypto_shash_alg_has_setkey() can now be an inline function like before. This commit should not be backported to kernels that don't have the new CFI implementation. Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- crypto/shash.c | 18 +++--------------- include/crypto/internal/hash.h | 8 +++++++- 2 files changed, 10 insertions(+), 16 deletions(-) diff --git a/crypto/shash.c b/crypto/shash.c index 4c88e63b3350f..0f85431588267 100644 --- a/crypto/shash.c +++ b/crypto/shash.c @@ -20,24 +20,12 @@ static const struct crypto_type crypto_shash_type; -static int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, - unsigned int keylen) +int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, + unsigned int keylen) { return -ENOSYS; } - -/* - * Check whether an shash algorithm has a setkey function. - * - * For CFI compatibility, this must not be an inline function. This is because - * when CFI is enabled, modules won't get the same address for shash_no_setkey - * (if it were exported, which inlining would require) as the core kernel will. - */ -bool crypto_shash_alg_has_setkey(struct shash_alg *alg) -{ - return alg->setkey != shash_no_setkey; -} -EXPORT_SYMBOL_GPL(crypto_shash_alg_has_setkey); +EXPORT_SYMBOL_GPL(shash_no_setkey); static int shash_setkey_unaligned(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) diff --git a/include/crypto/internal/hash.h b/include/crypto/internal/hash.h index 25806141db591..0a288dddcf5be 100644 --- a/include/crypto/internal/hash.h +++ b/include/crypto/internal/hash.h @@ -75,7 +75,13 @@ void crypto_unregister_ahashes(struct ahash_alg *algs, int count); int ahash_register_instance(struct crypto_template *tmpl, struct ahash_instance *inst); -bool crypto_shash_alg_has_setkey(struct shash_alg *alg); +int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, + unsigned int keylen); + +static inline bool crypto_shash_alg_has_setkey(struct shash_alg *alg) +{ + return alg->setkey != shash_no_setkey; +} static inline bool crypto_shash_alg_needs_key(struct shash_alg *alg) {