From patchwork Fri Nov 18 09:02:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047859 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F018C4332F for ; Fri, 18 Nov 2022 09:04:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240803AbiKRJEL (ORCPT ); Fri, 18 Nov 2022 04:04:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241455AbiKRJEJ (ORCPT ); Fri, 18 Nov 2022 04:04:09 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AAF101143 for ; Fri, 18 Nov 2022 01:04:08 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3F0346230C for ; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7DCEAC433D7; Fri, 18 Nov 2022 09:04:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762247; bh=cS7/6QLD7TgC3YuEBW48zGmDobm+o6walAidsI/8OGs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NjtJJyf8H7nZ96eNiOfXsu5McNg8JK5zV0FckJdxHo8dDY1NKgZewBSzMQLrYjZf3 SQqLbctXfcuL4XCZjEr4uavRjOYolvNXe42Abn/F/cnd3krlFwnTV45bfevAgH6lz2 80E4RvUb7RkdTzZHOgTv9LEbCVnQYW0qTa5lV++HqJfze5XVfPfOnukxyMG4ZvZP2c bvzL0z1nrfFSUJBVT39wzLqEq/rW2tK3pLR02zpLG/l4Ft48+lkJZwRcrj2jM57h2g SYPkpj0gY3Jbwk2oLs2tE28qGg4wJ32KKjk+N+YhFwrqqWe0BdS2uJe0A5YH6UVcba jS5k7ZOOtX3ZA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 01/11] crypto: x86/aegis128 - fix crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:10 -0800 Message-Id: <20221118090220.398819-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers Some functions in aegis128-aesni-asm.S are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure. Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/aegis128-aesni-asm.S | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/aegis128-aesni-asm.S b/arch/x86/crypto/aegis128-aesni-asm.S index b48ddebb47489..cdf3215ec272c 100644 --- a/arch/x86/crypto/aegis128-aesni-asm.S +++ b/arch/x86/crypto/aegis128-aesni-asm.S @@ -7,6 +7,7 @@ */ #include +#include #include #define STATE0 %xmm0 @@ -402,7 +403,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_ad) * void crypto_aegis128_aesni_enc(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_enc) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_enc) FRAME_BEGIN cmp $0x10, LEN @@ -499,7 +500,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_enc) * void crypto_aegis128_aesni_enc_tail(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_enc_tail) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_enc_tail) FRAME_BEGIN /* load the state: */ @@ -556,7 +557,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_enc_tail) * void crypto_aegis128_aesni_dec(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_dec) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_dec) FRAME_BEGIN cmp $0x10, LEN @@ -653,7 +654,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_dec) * void crypto_aegis128_aesni_dec_tail(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_dec_tail) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_dec_tail) FRAME_BEGIN /* load the state: */ From patchwork Fri Nov 18 09:02:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047862 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B68ADC43217 for ; Fri, 18 Nov 2022 09:04:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241455AbiKRJES (ORCPT ); Fri, 18 Nov 2022 04:04:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241668AbiKRJEM (ORCPT ); Fri, 18 Nov 2022 04:04:12 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8870082BE3 for ; Fri, 18 Nov 2022 01:04:10 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 3BC78B82264 for ; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C2DACC4347C; Fri, 18 Nov 2022 09:04:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762247; bh=EUFYFfJ9l4ZaKrj6LXV97kZec7yb3+Tp3+VcEukdMMU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LdHq6u+FefAz6sqdRMGRP2sqZkl9VC4fP+bPiqnr1f1vd+rYrC802N2JZ+S05iTFj m5GDvxmjSEJ/6tNb26QRuPs88bWdriJPWqwD+hANG7EaL9ACOj+XsYnjH+PG0nFTXz fkkpn9Y2OyonbOGH7fy2UBbCJgrhRw3cZwh94a4aphQOwWNZ0DX0ia3A5A2MDqsO7n /4m1QSBvVOCncqq4T5p78r8pnT3RJIv+BJX0EuLX4dRoPlrfMiXCMgaGI/4mLnSV0m 2Zjo8aI+c/RsZfhgFIDLEDL5YwTW0KLyXnbBlRljLNTmh+6xYPqzIhVIBOaUq4tqY3 EdFVaxelRx4uA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , Taehee Yoo Subject: [PATCH 02/11] crypto: x86/aria - fix crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:11 -0800 Message-Id: <20221118090220.398819-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers Some functions in aria-aesni-avx-asm_64.S are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure. Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Cc: Taehee Yoo Signed-off-by: Eric Biggers --- arch/x86/crypto/aria-aesni-avx-asm_64.S | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/x86/crypto/aria-aesni-avx-asm_64.S b/arch/x86/crypto/aria-aesni-avx-asm_64.S index c75fd7d015ed8..03ae4cd1d976a 100644 --- a/arch/x86/crypto/aria-aesni-avx-asm_64.S +++ b/arch/x86/crypto/aria-aesni-avx-asm_64.S @@ -7,6 +7,7 @@ */ #include +#include #include /* struct aria_ctx: */ @@ -913,7 +914,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_crypt_16way) RET; SYM_FUNC_END(__aria_aesni_avx_crypt_16way) -SYM_FUNC_START(aria_aesni_avx_encrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_encrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -938,7 +939,7 @@ SYM_FUNC_START(aria_aesni_avx_encrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_encrypt_16way) -SYM_FUNC_START(aria_aesni_avx_decrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_decrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1039,7 +1040,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_ctr_gen_keystream_16way) RET; SYM_FUNC_END(__aria_aesni_avx_ctr_gen_keystream_16way) -SYM_FUNC_START(aria_aesni_avx_ctr_crypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_ctr_crypt_16way) /* input: * %rdi: ctx * %rsi: dst @@ -1208,7 +1209,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_gfni_crypt_16way) RET; SYM_FUNC_END(__aria_aesni_avx_gfni_crypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1233,7 +1234,7 @@ SYM_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_gfni_encrypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1258,7 +1259,7 @@ SYM_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_gfni_decrypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_ctr_crypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_ctr_crypt_16way) /* input: * %rdi: ctx * %rsi: dst From patchwork Fri Nov 18 09:02:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047860 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EBF4C4332F for ; Fri, 18 Nov 2022 09:04:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241714AbiKRJER (ORCPT ); Fri, 18 Nov 2022 04:04:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241703AbiKRJEL (ORCPT ); Fri, 18 Nov 2022 04:04:11 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEAED7C459 for ; Fri, 18 Nov 2022 01:04:09 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CC5FB623B5 for ; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1CD0DC43141; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762248; bh=zooE7+ejFNUPSNZGiDoSoSe4vxD5LeDR935uHu+9Mis=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Qf9w0v153TiLVCxBAem8j2g1CMJfdnoscvaKd6Y5RTaWZPbuSqqiy+cmiN9GykD8m V1JbktwvBI+56wX4b87xMLC1hMlu8FGwZDSvVetlpG4UPEZQCbN4FVkxhMVP0vTXfs Jz4g+0NbWdg8/bjv0VRNSDXqX/q/3iddolqedRes1J0id0S0fexAB1gX9w6bQiI9hc VHKDUCtnw4FtHCvhaoSeC0A4lfwpAtZHJdJI+p4oSVqm5KnhjPkoioi8s126xeS9t5 EQlKADqkpUDq3J9kt77B9rcbWsCumUFKb9O/+8XmjXb05p0rF6eKHrS9jGzz/em/UZ WbSiuLmxb9sxg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 03/11] crypto: x86/nhpoly1305 - eliminate unnecessary CFI wrappers Date: Fri, 18 Nov 2022 01:02:12 -0800 Message-Id: <20221118090220.398819-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers Since the CFI implementation now supports indirect calls to assembly functions, take advantage of that rather than use wrapper functions. Signed-off-by: Eric Biggers --- arch/x86/crypto/nh-avx2-x86_64.S | 5 +++-- arch/x86/crypto/nh-sse2-x86_64.S | 5 +++-- arch/x86/crypto/nhpoly1305-avx2-glue.c | 11 ++--------- arch/x86/crypto/nhpoly1305-sse2-glue.c | 11 ++--------- 4 files changed, 10 insertions(+), 22 deletions(-) diff --git a/arch/x86/crypto/nh-avx2-x86_64.S b/arch/x86/crypto/nh-avx2-x86_64.S index 6a0b15e7196a8..ef73a3ab87263 100644 --- a/arch/x86/crypto/nh-avx2-x86_64.S +++ b/arch/x86/crypto/nh-avx2-x86_64.S @@ -8,6 +8,7 @@ */ #include +#include #define PASS0_SUMS %ymm0 #define PASS1_SUMS %ymm1 @@ -65,11 +66,11 @@ /* * void nh_avx2(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_avx2) +SYM_TYPED_FUNC_START(nh_avx2) vmovdqu 0x00(KEY), K0 vmovdqu 0x10(KEY), K1 diff --git a/arch/x86/crypto/nh-sse2-x86_64.S b/arch/x86/crypto/nh-sse2-x86_64.S index 34c567bbcb4fa..75fb994b6d177 100644 --- a/arch/x86/crypto/nh-sse2-x86_64.S +++ b/arch/x86/crypto/nh-sse2-x86_64.S @@ -8,6 +8,7 @@ */ #include +#include #define PASS0_SUMS %xmm0 #define PASS1_SUMS %xmm1 @@ -67,11 +68,11 @@ /* * void nh_sse2(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_sse2) +SYM_TYPED_FUNC_START(nh_sse2) movdqu 0x00(KEY), K0 movdqu 0x10(KEY), K1 diff --git a/arch/x86/crypto/nhpoly1305-avx2-glue.c b/arch/x86/crypto/nhpoly1305-avx2-glue.c index 8ea5ab0f1ca74..46b036204ed91 100644 --- a/arch/x86/crypto/nhpoly1305-avx2-glue.c +++ b/arch/x86/crypto/nhpoly1305-avx2-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_avx2(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_avx2(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_avx2(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_avx2_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_avx2_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_fpu_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_avx2); + crypto_nhpoly1305_update_helper(desc, src, n, nh_avx2); kernel_fpu_end(); src += n; srclen -= n; diff --git a/arch/x86/crypto/nhpoly1305-sse2-glue.c b/arch/x86/crypto/nhpoly1305-sse2-glue.c index 2b353d42ed13f..4a4970d751076 100644 --- a/arch/x86/crypto/nhpoly1305-sse2-glue.c +++ b/arch/x86/crypto/nhpoly1305-sse2-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_sse2(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_sse2(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_sse2(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_sse2_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_sse2_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_fpu_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_sse2); + crypto_nhpoly1305_update_helper(desc, src, n, nh_sse2); kernel_fpu_end(); src += n; srclen -= n; From patchwork Fri Nov 18 09:02:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047865 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1214C43217 for ; Fri, 18 Nov 2022 09:04:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241530AbiKRJEV (ORCPT ); Fri, 18 Nov 2022 04:04:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241704AbiKRJEM (ORCPT ); Fri, 18 Nov 2022 04:04:12 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEF7382BC9 for ; Fri, 18 Nov 2022 01:04:09 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1BB78623BB for ; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 62593C433D7; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762248; bh=izas7t+6oGMs9+g/37vH55gFtFtlIMabHc0IYfz658Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Zf8+T9vZhJ4QB3orc2MRN4ra0Ac9rGJfEPJ53UqrkXAPvoo44QU3Vu7eUy/ch24I7 AHTqR9zvBWC12+JTY29n6NBcr/crrWNjbKxqztDHjwYgRxaSX0vApyVLgdTFYBvK9K u8ZUFQFlt3bCQZaUO3BbD1owS0hPOpx6hlKuuCg70BSYk4ueI5RtPQGXwn2RVzFox8 GzUw7yvAArNuDMULWjziqgd/Fppz6Z06e9j76DD8YaV215kQb8yzTnR2Kut3QniMxe nQ9MSxd0vciqlDwuWJoAHQnwgwQ5BxxvLWpg/DGL60PQ8I3KfUp8J0Ge6kvK9xzTsg 6iLV/zxHtLD2w== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 04/11] crypto: x86/sha1 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:13 -0800 Message-Id: <20221118090220.398819-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sha1_transform_ssse3(), sha1_transform_avx(), and sha1_ni_transform() (but not sha1_transform_avx2()) are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/sha1_ni_asm.S | 3 ++- arch/x86/crypto/sha1_ssse3_asm.S | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/crypto/sha1_ni_asm.S b/arch/x86/crypto/sha1_ni_asm.S index 2f94ec0e763bf..3cae5a1bb3d6e 100644 --- a/arch/x86/crypto/sha1_ni_asm.S +++ b/arch/x86/crypto/sha1_ni_asm.S @@ -54,6 +54,7 @@ */ #include +#include #define DIGEST_PTR %rdi /* 1st arg */ #define DATA_PTR %rsi /* 2nd arg */ @@ -93,7 +94,7 @@ */ .text .align 32 -SYM_FUNC_START(sha1_ni_transform) +SYM_TYPED_FUNC_START(sha1_ni_transform) push %rbp mov %rsp, %rbp sub $FRAME_SIZE, %rsp diff --git a/arch/x86/crypto/sha1_ssse3_asm.S b/arch/x86/crypto/sha1_ssse3_asm.S index 263f916362e02..f54988c80eb40 100644 --- a/arch/x86/crypto/sha1_ssse3_asm.S +++ b/arch/x86/crypto/sha1_ssse3_asm.S @@ -25,6 +25,7 @@ */ #include +#include #define CTX %rdi // arg1 #define BUF %rsi // arg2 @@ -67,7 +68,7 @@ * param: function's name */ .macro SHA1_VECTOR_ASM name - SYM_FUNC_START(\name) + SYM_TYPED_FUNC_START(\name) push %rbx push %r12 From patchwork Fri Nov 18 09:02:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047863 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70192C433FE for ; Fri, 18 Nov 2022 09:04:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241668AbiKRJET (ORCPT ); Fri, 18 Nov 2022 04:04:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241436AbiKRJEM (ORCPT ); Fri, 18 Nov 2022 04:04:12 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEECA82BC5 for ; Fri, 18 Nov 2022 01:04:09 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6890F623C0 for ; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7AB4C43147; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762248; bh=mJsH5L9LpAZtcysiSyHXhJEv09v85eGVUHWCSa6xvLc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sA1CiL1XzosLH61OUxoCSm3C1QMDTQZshxtbEDoz3+1vk8+AzfNScnLOcD23KoKI9 jwlUpcSmq8gdvTeTi9a6KxA/zMHVR3e4RquZ9PE1RaEpFHgAUvc2m6EFgsPERAmdME UUptYxUzbVxH0ID07b45ZnAGqEDvd1I+M4yKsuYMENsd0epuuXT148thCyt73NrWYn xqzYILvnzM0Nmng7KEa9BQfRu4POghIeyzzA0WBgxIMFnosIuCMgJviO9KeD0nUCeM 2hvBJ2oddwzRv6RciknvcNnRF0+VttvJ9EzVg9iqfeBJY/+30QevbwQ8x47FO9JG2G o+jSLeehssCSA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 05/11] crypto: x86/sha256 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:14 -0800 Message-Id: <20221118090220.398819-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sha256_transform_ssse3(), sha256_transform_avx(), sha256_transform_rorx(), and sha256_transform_ni() are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/sha256-avx-asm.S | 3 ++- arch/x86/crypto/sha256-avx2-asm.S | 3 ++- arch/x86/crypto/sha256-ssse3-asm.S | 3 ++- arch/x86/crypto/sha256_ni_asm.S | 3 ++- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/sha256-avx-asm.S b/arch/x86/crypto/sha256-avx-asm.S index 3baa1ec390974..06ea30c20828d 100644 --- a/arch/x86/crypto/sha256-avx-asm.S +++ b/arch/x86/crypto/sha256-avx-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define VMOVDQ vmovdqu @@ -346,7 +347,7 @@ a = TMP_ ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_avx) +SYM_TYPED_FUNC_START(sha256_transform_avx) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256-avx2-asm.S b/arch/x86/crypto/sha256-avx2-asm.S index 9bcdbc47b8b4b..2d2be531a11ed 100644 --- a/arch/x86/crypto/sha256-avx2-asm.S +++ b/arch/x86/crypto/sha256-avx2-asm.S @@ -49,6 +49,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define VMOVDQ vmovdqu @@ -523,7 +524,7 @@ STACK_SIZE = _CTX + _CTX_SIZE ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_rorx) +SYM_TYPED_FUNC_START(sha256_transform_rorx) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256-ssse3-asm.S b/arch/x86/crypto/sha256-ssse3-asm.S index c4a5db612c327..7db28839108dd 100644 --- a/arch/x86/crypto/sha256-ssse3-asm.S +++ b/arch/x86/crypto/sha256-ssse3-asm.S @@ -47,6 +47,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define MOVDQ movdqu @@ -355,7 +356,7 @@ a = TMP_ ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_ssse3) +SYM_TYPED_FUNC_START(sha256_transform_ssse3) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256_ni_asm.S b/arch/x86/crypto/sha256_ni_asm.S index 94d50dd27cb53..47f93937f798a 100644 --- a/arch/x86/crypto/sha256_ni_asm.S +++ b/arch/x86/crypto/sha256_ni_asm.S @@ -54,6 +54,7 @@ */ #include +#include #define DIGEST_PTR %rdi /* 1st arg */ #define DATA_PTR %rsi /* 2nd arg */ @@ -97,7 +98,7 @@ .text .align 32 -SYM_FUNC_START(sha256_ni_transform) +SYM_TYPED_FUNC_START(sha256_ni_transform) shl $6, NUM_BLKS /* convert to bytes */ jz .Ldone_hash From patchwork Fri Nov 18 09:02:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047861 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BE9BC433FE for ; Fri, 18 Nov 2022 09:04:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241719AbiKRJER (ORCPT ); Fri, 18 Nov 2022 04:04:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241455AbiKRJEM (ORCPT ); Fri, 18 Nov 2022 04:04:12 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEDDA7EC8F for ; Fri, 18 Nov 2022 01:04:09 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 431F1623BD for ; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC565C43146; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762249; bh=cvZrVNyaorZiFckp5I/DQO2Na69JdiQKH9GTwngz5kw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Txbmq1+8Os6ha8bRpkOEpMYl4yfovwRGORc/Q36bVbY0f/kt0jgLZs757o5lelPUR M4/GAy/RPij5/oInIsOl55+yuMlU2sQFJ05AcXKY4PudJAEycWR9iXNmcx1h5h3YNR 209MaRi+OoMvgNO4JI6rRGhf58knvpGrLNL6l5Kryv6wNaau6jo0dH5ItNYO++N6F1 PgC/p10LMij1YENAKkUG7CjntqP+312yJl+rZHiDardnnksxhPWLC1/rZoymXxqI5v dJORoyoJ2st87QAEJ/29Yi/9ygXONH/V34yN4BAKpsyZDJUpXzhspBzN7sSC19jbu0 KFZ777WpuVDDw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 06/11] crypto: x86/sha512 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:15 -0800 Message-Id: <20221118090220.398819-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sha512_transform_rorx(), sha512_transform_ssse3(), and sha512_transform_avx() are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/sha512-avx-asm.S | 3 ++- arch/x86/crypto/sha512-avx2-asm.S | 3 ++- arch/x86/crypto/sha512-ssse3-asm.S | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/arch/x86/crypto/sha512-avx-asm.S b/arch/x86/crypto/sha512-avx-asm.S index 1fefe6dd3a9e2..b0984f19fdb40 100644 --- a/arch/x86/crypto/sha512-avx-asm.S +++ b/arch/x86/crypto/sha512-avx-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include .text @@ -273,7 +274,7 @@ frame_size = frame_WK + WK_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks ######################################################################## -SYM_FUNC_START(sha512_transform_avx) +SYM_TYPED_FUNC_START(sha512_transform_avx) test msglen, msglen je nowork diff --git a/arch/x86/crypto/sha512-avx2-asm.S b/arch/x86/crypto/sha512-avx2-asm.S index 5cdaab7d69015..b1ca99055ef99 100644 --- a/arch/x86/crypto/sha512-avx2-asm.S +++ b/arch/x86/crypto/sha512-avx2-asm.S @@ -50,6 +50,7 @@ ######################################################################## #include +#include .text @@ -565,7 +566,7 @@ frame_size = frame_CTX + CTX_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks ######################################################################## -SYM_FUNC_START(sha512_transform_rorx) +SYM_TYPED_FUNC_START(sha512_transform_rorx) # Save GPRs push %rbx push %r12 diff --git a/arch/x86/crypto/sha512-ssse3-asm.S b/arch/x86/crypto/sha512-ssse3-asm.S index b84c22e06c5f7..c06afb5270e5f 100644 --- a/arch/x86/crypto/sha512-ssse3-asm.S +++ b/arch/x86/crypto/sha512-ssse3-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include .text @@ -274,7 +275,7 @@ frame_size = frame_WK + WK_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks. ######################################################################## -SYM_FUNC_START(sha512_transform_ssse3) +SYM_TYPED_FUNC_START(sha512_transform_ssse3) test msglen, msglen je nowork From patchwork Fri Nov 18 09:02:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047866 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1B7BC43219 for ; Fri, 18 Nov 2022 09:04:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241704AbiKRJEW (ORCPT ); Fri, 18 Nov 2022 04:04:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241705AbiKRJEM (ORCPT ); Fri, 18 Nov 2022 04:04:12 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5204879931 for ; Fri, 18 Nov 2022 01:04:10 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 89630623C1 for ; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3CE0BC433D6; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762249; bh=NnbwjD/ex8fXbifBK9kxmybIvw2mJxpjvji26vXujDE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jCfMsnWwuLZr2TZMX42ziZkf9YQUI/Z7eUjtM7+vmN99cHpcGLu8BoyYZNKSin29Q fGxnshtgoc/9/yDLFLEvCqu+dmCY44y3gUTofehfryzsGicymkxlJRvWcOzafSe7bR O+0+aHnGVahABEPC8PVOdy6wUJmDMFaMs3SG+yzlzgmtg9chTFSU5PY1zqX3oE6rn0 HdtJBfAadfp4QGOozG5olUgrRaR/UzcUM8rx+AQ7bFrPrWMzygCCcJWZ+WCj7WiEag xGkZ2t7gmMPfzrRqraVIdDjSncz8/T2PdRKGzXB65iuxcp2jE92LWeR3TAePTE2GgF +AvIqUqoixt4w== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 07/11] crypto: x86/sm3 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:16 -0800 Message-Id: <20221118090220.398819-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sm3_transform_avx() is called via indirect function calls. This function needs to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect call). Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/sm3-avx-asm_64.S | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/crypto/sm3-avx-asm_64.S b/arch/x86/crypto/sm3-avx-asm_64.S index b12b9efb5ec51..8fc5ac681fd63 100644 --- a/arch/x86/crypto/sm3-avx-asm_64.S +++ b/arch/x86/crypto/sm3-avx-asm_64.S @@ -12,6 +12,7 @@ */ #include +#include #include /* Context structure */ @@ -328,7 +329,7 @@ * const u8 *data, int nblocks); */ .align 16 -SYM_FUNC_START(sm3_transform_avx) +SYM_TYPED_FUNC_START(sm3_transform_avx) /* input: * %rdi: ctx, CTX * %rsi: data (64*nblks bytes) From patchwork Fri Nov 18 09:02:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047869 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35B30C433FE for ; Fri, 18 Nov 2022 09:04:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241702AbiKRJEs (ORCPT ); Fri, 18 Nov 2022 04:04:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241712AbiKRJEQ (ORCPT ); Fri, 18 Nov 2022 04:04:16 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4815E13D4F for ; Fri, 18 Nov 2022 01:04:12 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id DE6D1B82274 for ; Fri, 18 Nov 2022 09:04:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8187AC43146; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762249; bh=fc2JKYQatpTxXu11VT4a3V3fN6ivlyjI0KHBLOLHcEA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tWBHjlxK1blh/DEay6JaIGEL7wJIjBmtDxcPiGMfrHpdnkXk+LoFlctITaF3RXAYV 2yWhgt/dqlHzVPsnZthunPoK6jWayfFXHAD6LUeActAsOQ8ePYimylbS5Kz7BRh8oj hzDBNypHtok8kY0GHME3FQbOZZgA420/7KWbEY0uCwenBN4Kdp+SbSWcI2wcaITHuP 3vVdytMQFZYh14BrSbYisTfcmf8u34OleBaax7luuOy+9SX3OT7ImWs7FGdn2yoMBi KuLCzI2RD8DYd6sIbjyWH13Sa0UJqcfyybEMoAvCiY6CVmQ2BK2dHRazBnf4TQ4NEg pAKiN/iYuB9uA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 08/11] crypto: arm64/nhpoly1305 - eliminate unnecessary CFI wrapper Date: Fri, 18 Nov 2022 01:02:17 -0800 Message-Id: <20221118090220.398819-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers Since the CFI implementation now supports indirect calls to assembly functions, take advantage of that rather than use a wrapper function. Signed-off-by: Eric Biggers --- arch/arm64/crypto/nh-neon-core.S | 5 +++-- arch/arm64/crypto/nhpoly1305-neon-glue.c | 11 ++--------- 2 files changed, 5 insertions(+), 11 deletions(-) diff --git a/arch/arm64/crypto/nh-neon-core.S b/arch/arm64/crypto/nh-neon-core.S index 51c0a534ef87c..13eda08fda1e5 100644 --- a/arch/arm64/crypto/nh-neon-core.S +++ b/arch/arm64/crypto/nh-neon-core.S @@ -8,6 +8,7 @@ */ #include +#include KEY .req x0 MESSAGE .req x1 @@ -58,11 +59,11 @@ /* * void nh_neon(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_neon) +SYM_TYPED_FUNC_START(nh_neon) ld1 {K0.4s,K1.4s}, [KEY], #32 movi PASS0_SUMS.2d, #0 diff --git a/arch/arm64/crypto/nhpoly1305-neon-glue.c b/arch/arm64/crypto/nhpoly1305-neon-glue.c index c5405e6a6db76..cd882c35d9252 100644 --- a/arch/arm64/crypto/nhpoly1305-neon-glue.c +++ b/arch/arm64/crypto/nhpoly1305-neon-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_neon(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_neon(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_neon(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_neon_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_neon_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_neon_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_neon); + crypto_nhpoly1305_update_helper(desc, src, n, nh_neon); kernel_neon_end(); src += n; srclen -= n; From patchwork Fri Nov 18 09:02:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047870 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FB4BC4332F for ; Fri, 18 Nov 2022 09:04:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241734AbiKRJEt (ORCPT ); Fri, 18 Nov 2022 04:04:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241715AbiKRJER (ORCPT ); Fri, 18 Nov 2022 04:04:17 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 971278A17A for ; Fri, 18 Nov 2022 01:04:12 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2E9EDB82269 for ; Fri, 18 Nov 2022 09:04:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C609FC433D7; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762249; bh=wyK4erNXYZLb0gGezT+2lcL+zroMKoz8h7GycCvuO2k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NqABtUTYY1aVx9PfXGJq58Dbdt7rDMgcoP+hSxU5ThNarF3tRMyvrwQwAEkXDGUO5 tiJ5U6LMLR0lV4JumLMkHK8INpJW3VIBKbH25sTBOZ87643hheINT8JJZJRVNf2TJF Avmp31f8rb9nVNMm7VPa6EXaX/IeeMYxx0SdXLQ4veF50FCyygcXgJzJ0h/mNWUiz0 nXsrpByKTHAWcFIFVyfqlGKRd3pquE2LM5zke5eFjTEU3S7TFTNO9O/gQ3AjOeWI8Y mPsBOdZcHGDb2+C+DgTlFSdpfjQy/mk723T/7DJTGcpPCtgOYlXcdr21Vaz+4C3FCr kxX8ORTboM8rw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 09/11] crypto: arm64/sm3 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:18 -0800 Message-Id: <20221118090220.398819-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers sm3_neon_transform() is called via indirect function calls. This function needs to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect call). Fixes: c50d32859e70 ("arm64: Add types to indirect called assembly functions") Signed-off-by: Eric Biggers --- arch/arm64/crypto/sm3-neon-core.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/crypto/sm3-neon-core.S b/arch/arm64/crypto/sm3-neon-core.S index 3e3b4e5c736fc..8abea1d39ddd9 100644 --- a/arch/arm64/crypto/sm3-neon-core.S +++ b/arch/arm64/crypto/sm3-neon-core.S @@ -9,7 +9,7 @@ */ #include -#include +#include /* Context structure */ @@ -351,7 +351,7 @@ */ .text .align 3 -SYM_FUNC_START(sm3_neon_transform) +SYM_TYPED_FUNC_START(sm3_neon_transform) ldp ra, rb, [RSTATE, #0] ldp rc, rd, [RSTATE, #8] ldp re, rf, [RSTATE, #16] From patchwork Fri Nov 18 09:02:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047867 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56D50C43217 for ; Fri, 18 Nov 2022 09:04:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241426AbiKRJEX (ORCPT ); Fri, 18 Nov 2022 04:04:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241710AbiKRJEQ (ORCPT ); Fri, 18 Nov 2022 04:04:16 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61C3682BF5 for ; Fri, 18 Nov 2022 01:04:11 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C2822623B2 for ; Fri, 18 Nov 2022 09:04:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1685FC433D6; Fri, 18 Nov 2022 09:04:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762250; bh=ZkpnHZpH6zqpcaYDXylCMZLyYKMTqb6ZC8hlYT4Izo8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R2q5ZahIINyliBIF3hGitjFBaxgXe56Xs/ajq8hQ9BzHkYYdLsTgyQc1vuiSh9+II A4s2sslrbAetPUZYuXi5H+VrV0fLTlcyAv5NX1+nP3zQvorkEFN2+IR+VvgvrTGcb6 lqNzs3zcmptAv2UmuXiYpir0qL+WcvzbMaSQCoONYTaYYZrp2pTMnwLmfzURkEXpSW eCFl4arg6ka2UyoIGdcva1Pko+SP+kvRAwlpDC8W8fqixvLqQMiwyvmygshZHgys2C r0mc2YrT5iakwAQg1E8sbyR3VsmN8nSZ1EXoIzio9gKli3pshLwXWcINSXMRfSaMEx jE58Qto0vDAjw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 10/11] crypto: arm/nhpoly1305 - eliminate unnecessary CFI wrapper Date: Fri, 18 Nov 2022 01:02:19 -0800 Message-Id: <20221118090220.398819-11-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers The arm architecture doesn't support CFI yet, and even if it did, the new CFI implementation supports indirect calls to assembly functions. Therefore, there's no need to use a wrapper function for nh_neon(). Signed-off-by: Eric Biggers --- arch/arm/crypto/nh-neon-core.S | 2 +- arch/arm/crypto/nhpoly1305-neon-glue.c | 11 ++--------- 2 files changed, 3 insertions(+), 10 deletions(-) diff --git a/arch/arm/crypto/nh-neon-core.S b/arch/arm/crypto/nh-neon-core.S index 434d80ab531c2..01620a0782ca9 100644 --- a/arch/arm/crypto/nh-neon-core.S +++ b/arch/arm/crypto/nh-neon-core.S @@ -69,7 +69,7 @@ /* * void nh_neon(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ diff --git a/arch/arm/crypto/nhpoly1305-neon-glue.c b/arch/arm/crypto/nhpoly1305-neon-glue.c index ffa8d73fe722c..e93e41ff26566 100644 --- a/arch/arm/crypto/nhpoly1305-neon-glue.c +++ b/arch/arm/crypto/nhpoly1305-neon-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_neon(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_neon(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_neon(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_neon_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_neon_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_neon_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_neon); + crypto_nhpoly1305_update_helper(desc, src, n, nh_neon); kernel_neon_end(); src += n; srclen -= n; From patchwork Fri Nov 18 09:02:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047868 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 354E4C4332F for ; Fri, 18 Nov 2022 09:04:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241709AbiKRJEW (ORCPT ); Fri, 18 Nov 2022 04:04:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241426AbiKRJEQ (ORCPT ); Fri, 18 Nov 2022 04:04:16 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89BA182BFA for ; Fri, 18 Nov 2022 01:04:11 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 10514623B1 for ; Fri, 18 Nov 2022 09:04:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5B7DCC43141; Fri, 18 Nov 2022 09:04:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762250; bh=dN9lqJsqSknnJEVWfdbcH4PBv1COtt2elsbRPuqKWpA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=liqNp19oNyCipr9vwmUXYl94kRjIRI2R4Ks7FBDJWaZW/5/EuRAhr/i12ZO3dqwJP sIqNH+Jm/2M9dp86/CFzURzWn2glS35cKQ90JbEiRXZMJTCuw0I530QCO8Ky/pLakU 7PwI6ceXuN1oko89Q51kX8xlOEHPjAWuCrwXNpwKbkPJgppTkcii8c6fzUwxrE50SL 4rlviybpw4WjTC11URRRuqmpOza2fS6yhoHkrimgQPFWhu8jVzuWuUD5F1G/NIr9sn b93AtG0dxbXnWmfEoxmRkT4wW7raI2Wcmz1wMFy8WG+BQpTJpHTq1hVIFMbypx0MST U6EzSEKEO/KtA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 11/11] Revert "crypto: shash - avoid comparing pointers to exported functions under CFI" Date: Fri, 18 Nov 2022 01:02:20 -0800 Message-Id: <20221118090220.398819-12-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Eric Biggers This reverts commit 22ca9f4aaf431a9413dcc115dd590123307f274f because CFI no longer breaks cross-module function address equality, so crypto_shash_alg_has_setkey() can now be an inline function like before. This commit should not be backported to kernels that don't have the new CFI implementation. Signed-off-by: Eric Biggers --- crypto/shash.c | 18 +++--------------- include/crypto/internal/hash.h | 8 +++++++- 2 files changed, 10 insertions(+), 16 deletions(-) diff --git a/crypto/shash.c b/crypto/shash.c index 4c88e63b3350f..0f85431588267 100644 --- a/crypto/shash.c +++ b/crypto/shash.c @@ -20,24 +20,12 @@ static const struct crypto_type crypto_shash_type; -static int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, - unsigned int keylen) +int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, + unsigned int keylen) { return -ENOSYS; } - -/* - * Check whether an shash algorithm has a setkey function. - * - * For CFI compatibility, this must not be an inline function. This is because - * when CFI is enabled, modules won't get the same address for shash_no_setkey - * (if it were exported, which inlining would require) as the core kernel will. - */ -bool crypto_shash_alg_has_setkey(struct shash_alg *alg) -{ - return alg->setkey != shash_no_setkey; -} -EXPORT_SYMBOL_GPL(crypto_shash_alg_has_setkey); +EXPORT_SYMBOL_GPL(shash_no_setkey); static int shash_setkey_unaligned(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) diff --git a/include/crypto/internal/hash.h b/include/crypto/internal/hash.h index 25806141db591..0a288dddcf5be 100644 --- a/include/crypto/internal/hash.h +++ b/include/crypto/internal/hash.h @@ -75,7 +75,13 @@ void crypto_unregister_ahashes(struct ahash_alg *algs, int count); int ahash_register_instance(struct crypto_template *tmpl, struct ahash_instance *inst); -bool crypto_shash_alg_has_setkey(struct shash_alg *alg); +int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, + unsigned int keylen); + +static inline bool crypto_shash_alg_has_setkey(struct shash_alg *alg) +{ + return alg->setkey != shash_no_setkey; +} static inline bool crypto_shash_alg_needs_key(struct shash_alg *alg) {