From patchwork Fri Nov 18 09:02:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70D38C4332F for ; Fri, 18 Nov 2022 09:05:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=n3G0T3w+oPkW5pXk5QGJqJarg6JAivn/0egFZKMQ4vc=; b=lfDILqk+FRfbcs O/bJ+1xx5mSbHZNpl4meG8WjJ+YsyqR9UHzwQdNKWZaQuCHtALmj2yJaMEV35STALEtBulyMH620k wVna/kDg91Kw1leqd7bTIwUDghgrNpdHvIo/noBrw1/hi/S97xduDSiHgtFjUuu7Em84Q32RLUBGF lxbCyffb/J7hd2h07erXFqzSuscWZzD2RKILkkzeKXYLMgQThYhWbP5e5zwFa+GoYVBgv1O5roKQd RR0TrqvvgMGz4N/d1f4xUMqGoH225dBGnZG/kLXMDk2rQOmccRfYsMrSqxci3LJ2UWANGApSTbqGo wpZ8syMj2+/MZiSxKtXg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIP-002w2q-PA; Fri, 18 Nov 2022 09:04:25 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxI9-002vtt-6w for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:10 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 384C1623B2; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7DCEAC433D7; Fri, 18 Nov 2022 09:04:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762247; bh=cS7/6QLD7TgC3YuEBW48zGmDobm+o6walAidsI/8OGs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NjtJJyf8H7nZ96eNiOfXsu5McNg8JK5zV0FckJdxHo8dDY1NKgZewBSzMQLrYjZf3 SQqLbctXfcuL4XCZjEr4uavRjOYolvNXe42Abn/F/cnd3krlFwnTV45bfevAgH6lz2 80E4RvUb7RkdTzZHOgTv9LEbCVnQYW0qTa5lV++HqJfze5XVfPfOnukxyMG4ZvZP2c bvzL0z1nrfFSUJBVT39wzLqEq/rW2tK3pLR02zpLG/l4Ft48+lkJZwRcrj2jM57h2g SYPkpj0gY3Jbwk2oLs2tE28qGg4wJ32KKjk+N+YhFwrqqWe0BdS2uJe0A5YH6UVcba jS5k7ZOOtX3ZA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 01/11] crypto: x86/aegis128 - fix crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:10 -0800 Message-Id: <20221118090220.398819-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010409_332017_45131445 X-CRM114-Status: GOOD ( 12.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers Some functions in aegis128-aesni-asm.S are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure. Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/aegis128-aesni-asm.S | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/aegis128-aesni-asm.S b/arch/x86/crypto/aegis128-aesni-asm.S index b48ddebb47489..cdf3215ec272c 100644 --- a/arch/x86/crypto/aegis128-aesni-asm.S +++ b/arch/x86/crypto/aegis128-aesni-asm.S @@ -7,6 +7,7 @@ */ #include +#include #include #define STATE0 %xmm0 @@ -402,7 +403,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_ad) * void crypto_aegis128_aesni_enc(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_enc) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_enc) FRAME_BEGIN cmp $0x10, LEN @@ -499,7 +500,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_enc) * void crypto_aegis128_aesni_enc_tail(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_enc_tail) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_enc_tail) FRAME_BEGIN /* load the state: */ @@ -556,7 +557,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_enc_tail) * void crypto_aegis128_aesni_dec(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_dec) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_dec) FRAME_BEGIN cmp $0x10, LEN @@ -653,7 +654,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_dec) * void crypto_aegis128_aesni_dec_tail(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_dec_tail) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_dec_tail) FRAME_BEGIN /* load the state: */ From patchwork Fri Nov 18 09:02:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047874 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 695FEC433FE for ; Fri, 18 Nov 2022 09:05:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HD/MpSmqv0t5xQuzbFsjoFSEx4TsknkZZG/H92nJvh4=; b=Xn4+pK7rjiafza Vxk5d7Lc790oOWuDaVzcBRKRnV2r6UB5F9+t8IXqFqhxVED0FHNGdVOEozBwKlVqQH66jHAJ83RAU AXEk2CAEIj2TP5F7wMJG5INdmB8kNStiJ4lfd1eX7rZx8CJcRyEl5M0H1EfSsuyn4puDy573sSsB4 Wk1YkLVY33HzRouQHMKxktID1WJOPuguTs6Hl3vWMCUK4U1Yu1RCcGrmPayHFv+1c62MrizGpDDMN 5dVdccUy2S8vRMRyEIO2VGIAmEJLsFjm4qnSJ6AB4V+6Nd+JcGteCdqKzwvRrUO/mhM97/PXRbMih q3ofMQcXg6bBW1gZmP2A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIo-002w8g-Vy; Fri, 18 Nov 2022 09:04:51 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxI9-002vu1-KA for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:12 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8535C623B4; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C2DACC4347C; Fri, 18 Nov 2022 09:04:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762247; bh=EUFYFfJ9l4ZaKrj6LXV97kZec7yb3+Tp3+VcEukdMMU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LdHq6u+FefAz6sqdRMGRP2sqZkl9VC4fP+bPiqnr1f1vd+rYrC802N2JZ+S05iTFj m5GDvxmjSEJ/6tNb26QRuPs88bWdriJPWqwD+hANG7EaL9ACOj+XsYnjH+PG0nFTXz fkkpn9Y2OyonbOGH7fy2UBbCJgrhRw3cZwh94a4aphQOwWNZ0DX0ia3A5A2MDqsO7n /4m1QSBvVOCncqq4T5p78r8pnT3RJIv+BJX0EuLX4dRoPlrfMiXCMgaGI/4mLnSV0m 2Zjo8aI+c/RsZfhgFIDLEDL5YwTW0KLyXnbBlRljLNTmh+6xYPqzIhVIBOaUq4tqY3 EdFVaxelRx4uA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , Taehee Yoo Subject: [PATCH 02/11] crypto: x86/aria - fix crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:11 -0800 Message-Id: <20221118090220.398819-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010409_776949_72907436 X-CRM114-Status: GOOD ( 11.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers Some functions in aria-aesni-avx-asm_64.S are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure. Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Cc: Taehee Yoo Signed-off-by: Eric Biggers --- arch/x86/crypto/aria-aesni-avx-asm_64.S | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/x86/crypto/aria-aesni-avx-asm_64.S b/arch/x86/crypto/aria-aesni-avx-asm_64.S index c75fd7d015ed8..03ae4cd1d976a 100644 --- a/arch/x86/crypto/aria-aesni-avx-asm_64.S +++ b/arch/x86/crypto/aria-aesni-avx-asm_64.S @@ -7,6 +7,7 @@ */ #include +#include #include /* struct aria_ctx: */ @@ -913,7 +914,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_crypt_16way) RET; SYM_FUNC_END(__aria_aesni_avx_crypt_16way) -SYM_FUNC_START(aria_aesni_avx_encrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_encrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -938,7 +939,7 @@ SYM_FUNC_START(aria_aesni_avx_encrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_encrypt_16way) -SYM_FUNC_START(aria_aesni_avx_decrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_decrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1039,7 +1040,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_ctr_gen_keystream_16way) RET; SYM_FUNC_END(__aria_aesni_avx_ctr_gen_keystream_16way) -SYM_FUNC_START(aria_aesni_avx_ctr_crypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_ctr_crypt_16way) /* input: * %rdi: ctx * %rsi: dst @@ -1208,7 +1209,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_gfni_crypt_16way) RET; SYM_FUNC_END(__aria_aesni_avx_gfni_crypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1233,7 +1234,7 @@ SYM_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_gfni_encrypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1258,7 +1259,7 @@ SYM_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_gfni_decrypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_ctr_crypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_ctr_crypt_16way) /* input: * %rdi: ctx * %rsi: dst From patchwork Fri Nov 18 09:02:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047878 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F2FFBC433FE for ; Fri, 18 Nov 2022 09:07:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gOUYiP1EdDyxTSmFM2DI/Zt91jPCzrRaU+Z4eYInwFU=; b=Bss/hvHtkpI919 byKZQIzWziN67ddc3V/py8Ao/rHn7NDpNlcrXMNj25bFQzyr/n/nsGZmheeDCXUj8BiHFnGbkFpBu dWRdiSjys0XUWgM1r7hnKbTSQF+xLDa3xoXRu9OUCEYDUA0dJyp3taXdWPnUc2e1FWLDtwMjN9v1J wH9HuQgQqyY9uIOo8Lf93zqw51WomtrPlifkydGrwRf6OrZevEMcdH6pcMFMCpHo3Ubt4OyETiWqq G1I4FEybYTMMFihdB4YTTepXSa67+FXTcgrQXo7ZesRnzs1wAeWcHJSQE43SVV6r00/PM6KRuqodL ohbvmmiOFEqMuMWgouOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxK3-002wm7-Sr; Fri, 18 Nov 2022 09:06:08 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIA-002vvH-Ou for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:15 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6E9E3B8225A; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1CD0DC43141; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762248; bh=zooE7+ejFNUPSNZGiDoSoSe4vxD5LeDR935uHu+9Mis=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Qf9w0v153TiLVCxBAem8j2g1CMJfdnoscvaKd6Y5RTaWZPbuSqqiy+cmiN9GykD8m V1JbktwvBI+56wX4b87xMLC1hMlu8FGwZDSvVetlpG4UPEZQCbN4FVkxhMVP0vTXfs Jz4g+0NbWdg8/bjv0VRNSDXqX/q/3iddolqedRes1J0id0S0fexAB1gX9w6bQiI9hc VHKDUCtnw4FtHCvhaoSeC0A4lfwpAtZHJdJI+p4oSVqm5KnhjPkoioi8s126xeS9t5 EQlKADqkpUDq3J9kt77B9rcbWsCumUFKb9O/+8XmjXb05p0rF6eKHrS9jGzz/em/UZ WbSiuLmxb9sxg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 03/11] crypto: x86/nhpoly1305 - eliminate unnecessary CFI wrappers Date: Fri, 18 Nov 2022 01:02:12 -0800 Message-Id: <20221118090220.398819-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010411_193110_91B7A48F X-CRM114-Status: GOOD ( 11.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers Since the CFI implementation now supports indirect calls to assembly functions, take advantage of that rather than use wrapper functions. Signed-off-by: Eric Biggers --- arch/x86/crypto/nh-avx2-x86_64.S | 5 +++-- arch/x86/crypto/nh-sse2-x86_64.S | 5 +++-- arch/x86/crypto/nhpoly1305-avx2-glue.c | 11 ++--------- arch/x86/crypto/nhpoly1305-sse2-glue.c | 11 ++--------- 4 files changed, 10 insertions(+), 22 deletions(-) diff --git a/arch/x86/crypto/nh-avx2-x86_64.S b/arch/x86/crypto/nh-avx2-x86_64.S index 6a0b15e7196a8..ef73a3ab87263 100644 --- a/arch/x86/crypto/nh-avx2-x86_64.S +++ b/arch/x86/crypto/nh-avx2-x86_64.S @@ -8,6 +8,7 @@ */ #include +#include #define PASS0_SUMS %ymm0 #define PASS1_SUMS %ymm1 @@ -65,11 +66,11 @@ /* * void nh_avx2(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_avx2) +SYM_TYPED_FUNC_START(nh_avx2) vmovdqu 0x00(KEY), K0 vmovdqu 0x10(KEY), K1 diff --git a/arch/x86/crypto/nh-sse2-x86_64.S b/arch/x86/crypto/nh-sse2-x86_64.S index 34c567bbcb4fa..75fb994b6d177 100644 --- a/arch/x86/crypto/nh-sse2-x86_64.S +++ b/arch/x86/crypto/nh-sse2-x86_64.S @@ -8,6 +8,7 @@ */ #include +#include #define PASS0_SUMS %xmm0 #define PASS1_SUMS %xmm1 @@ -67,11 +68,11 @@ /* * void nh_sse2(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_sse2) +SYM_TYPED_FUNC_START(nh_sse2) movdqu 0x00(KEY), K0 movdqu 0x10(KEY), K1 diff --git a/arch/x86/crypto/nhpoly1305-avx2-glue.c b/arch/x86/crypto/nhpoly1305-avx2-glue.c index 8ea5ab0f1ca74..46b036204ed91 100644 --- a/arch/x86/crypto/nhpoly1305-avx2-glue.c +++ b/arch/x86/crypto/nhpoly1305-avx2-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_avx2(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_avx2(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_avx2(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_avx2_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_avx2_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_fpu_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_avx2); + crypto_nhpoly1305_update_helper(desc, src, n, nh_avx2); kernel_fpu_end(); src += n; srclen -= n; diff --git a/arch/x86/crypto/nhpoly1305-sse2-glue.c b/arch/x86/crypto/nhpoly1305-sse2-glue.c index 2b353d42ed13f..4a4970d751076 100644 --- a/arch/x86/crypto/nhpoly1305-sse2-glue.c +++ b/arch/x86/crypto/nhpoly1305-sse2-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_sse2(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_sse2(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_sse2(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_sse2_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_sse2_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_fpu_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_sse2); + crypto_nhpoly1305_update_helper(desc, src, n, nh_sse2); kernel_fpu_end(); src += n; srclen -= n; From patchwork Fri Nov 18 09:02:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 04FF7C4332F for ; Fri, 18 Nov 2022 09:05:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BjMIk0DeKrZ2TeVNkjvzwX6AsUk6WTVcFj4MoHeXPyc=; b=xAZU9yXC1hPyfY JYA1ExEb4PSnDCO4Bx6y66Bvu6Sunb7re7HwdLeKGyyc1fsVnb4YCiaKigD53f/84U2arjvha+BRM zAf3Zki378wTmT2UYUTX6FGj+M/0eeCA1oUn4aRvB4e+60xCzTtvMbY06Srqjx7ZTd9ynnrp+dRCh mb3pL1CeXvUqyyU7WUcb4VW2Npm2WiQnGg0fc/WNdVW5xTnwmFiWIVguzKdXwP5iZ0uSJNycnDm9i eEhx2FBDbDmfwOPoMTvL4YZgd3qKOZlnRwDi1L6zeW+sFEf2NCgrB3+tmZrbjFjYPzCCpLypCQHLP eiM3SapjtchrESGyaVBw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIZ-002w5J-Ki; Fri, 18 Nov 2022 09:04:35 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxI9-002vuK-Gp for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:12 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1E86B623BC; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 62593C433D7; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762248; bh=izas7t+6oGMs9+g/37vH55gFtFtlIMabHc0IYfz658Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Zf8+T9vZhJ4QB3orc2MRN4ra0Ac9rGJfEPJ53UqrkXAPvoo44QU3Vu7eUy/ch24I7 AHTqR9zvBWC12+JTY29n6NBcr/crrWNjbKxqztDHjwYgRxaSX0vApyVLgdTFYBvK9K u8ZUFQFlt3bCQZaUO3BbD1owS0hPOpx6hlKuuCg70BSYk4ueI5RtPQGXwn2RVzFox8 GzUw7yvAArNuDMULWjziqgd/Fppz6Z06e9j76DD8YaV215kQb8yzTnR2Kut3QniMxe nQ9MSxd0vciqlDwuWJoAHQnwgwQ5BxxvLWpg/DGL60PQ8I3KfUp8J0Ge6kvK9xzTsg 6iLV/zxHtLD2w== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 04/11] crypto: x86/sha1 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:13 -0800 Message-Id: <20221118090220.398819-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010409_621001_8648D5D2 X-CRM114-Status: GOOD ( 13.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sha1_transform_ssse3(), sha1_transform_avx(), and sha1_ni_transform() (but not sha1_transform_avx2()) are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/sha1_ni_asm.S | 3 ++- arch/x86/crypto/sha1_ssse3_asm.S | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/crypto/sha1_ni_asm.S b/arch/x86/crypto/sha1_ni_asm.S index 2f94ec0e763bf..3cae5a1bb3d6e 100644 --- a/arch/x86/crypto/sha1_ni_asm.S +++ b/arch/x86/crypto/sha1_ni_asm.S @@ -54,6 +54,7 @@ */ #include +#include #define DIGEST_PTR %rdi /* 1st arg */ #define DATA_PTR %rsi /* 2nd arg */ @@ -93,7 +94,7 @@ */ .text .align 32 -SYM_FUNC_START(sha1_ni_transform) +SYM_TYPED_FUNC_START(sha1_ni_transform) push %rbp mov %rsp, %rbp sub $FRAME_SIZE, %rsp diff --git a/arch/x86/crypto/sha1_ssse3_asm.S b/arch/x86/crypto/sha1_ssse3_asm.S index 263f916362e02..f54988c80eb40 100644 --- a/arch/x86/crypto/sha1_ssse3_asm.S +++ b/arch/x86/crypto/sha1_ssse3_asm.S @@ -25,6 +25,7 @@ */ #include +#include #define CTX %rdi // arg1 #define BUF %rsi // arg2 @@ -67,7 +68,7 @@ * param: function's name */ .macro SHA1_VECTOR_ASM name - SYM_FUNC_START(\name) + SYM_TYPED_FUNC_START(\name) push %rbx push %r12 From patchwork Fri Nov 18 09:02:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4629FC433FE for ; Fri, 18 Nov 2022 09:06:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2uXyv3gzQak2gWfEciVz9DbdE0FG2oYWCz24G3V4bO8=; b=R5CrBI1G1uZ+vS gtF3znHmDwWDmySMaU/YN05UrtzxkJOaeWhw3S0MBqUiC834lv/3pRFvvpQTmZ1cwwoFF9Lx72izG /29UniKhPrvq+aDvPZTYCMFRq8S1VxrNcUcFhTwqgW7mkDz2B9AywUd2MecFiKVLzPT7ArqcmJKlZ U3cSly7U1pAQmorlHqcpBWH3LBz9EXnhU37aEU4GbaunQd3NZN1PAiI5+aejymzMC5ddgPNEPpd7V 3+oH7JUphn/WunJfVu3zfkjroXruCPBusN9ooamtCp5ZzZ2i1YVNOn/09889CFY9ERz2aR9Xd4Irl 6snr3zz13LWW/l8Y2v3A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxJ6-002wFP-2I; Fri, 18 Nov 2022 09:05:08 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxI9-002vuG-G8 for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:12 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id F3E51623BA; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7AB4C43147; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762248; bh=mJsH5L9LpAZtcysiSyHXhJEv09v85eGVUHWCSa6xvLc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sA1CiL1XzosLH61OUxoCSm3C1QMDTQZshxtbEDoz3+1vk8+AzfNScnLOcD23KoKI9 jwlUpcSmq8gdvTeTi9a6KxA/zMHVR3e4RquZ9PE1RaEpFHgAUvc2m6EFgsPERAmdME UUptYxUzbVxH0ID07b45ZnAGqEDvd1I+M4yKsuYMENsd0epuuXT148thCyt73NrWYn xqzYILvnzM0Nmng7KEa9BQfRu4POghIeyzzA0WBgxIMFnosIuCMgJviO9KeD0nUCeM 2hvBJ2oddwzRv6RciknvcNnRF0+VttvJ9EzVg9iqfeBJY/+30QevbwQ8x47FO9JG2G o+jSLeehssCSA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 05/11] crypto: x86/sha256 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:14 -0800 Message-Id: <20221118090220.398819-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010409_642205_BCBD5FF2 X-CRM114-Status: GOOD ( 13.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sha256_transform_ssse3(), sha256_transform_avx(), sha256_transform_rorx(), and sha256_transform_ni() are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/sha256-avx-asm.S | 3 ++- arch/x86/crypto/sha256-avx2-asm.S | 3 ++- arch/x86/crypto/sha256-ssse3-asm.S | 3 ++- arch/x86/crypto/sha256_ni_asm.S | 3 ++- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/sha256-avx-asm.S b/arch/x86/crypto/sha256-avx-asm.S index 3baa1ec390974..06ea30c20828d 100644 --- a/arch/x86/crypto/sha256-avx-asm.S +++ b/arch/x86/crypto/sha256-avx-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define VMOVDQ vmovdqu @@ -346,7 +347,7 @@ a = TMP_ ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_avx) +SYM_TYPED_FUNC_START(sha256_transform_avx) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256-avx2-asm.S b/arch/x86/crypto/sha256-avx2-asm.S index 9bcdbc47b8b4b..2d2be531a11ed 100644 --- a/arch/x86/crypto/sha256-avx2-asm.S +++ b/arch/x86/crypto/sha256-avx2-asm.S @@ -49,6 +49,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define VMOVDQ vmovdqu @@ -523,7 +524,7 @@ STACK_SIZE = _CTX + _CTX_SIZE ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_rorx) +SYM_TYPED_FUNC_START(sha256_transform_rorx) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256-ssse3-asm.S b/arch/x86/crypto/sha256-ssse3-asm.S index c4a5db612c327..7db28839108dd 100644 --- a/arch/x86/crypto/sha256-ssse3-asm.S +++ b/arch/x86/crypto/sha256-ssse3-asm.S @@ -47,6 +47,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define MOVDQ movdqu @@ -355,7 +356,7 @@ a = TMP_ ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_ssse3) +SYM_TYPED_FUNC_START(sha256_transform_ssse3) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256_ni_asm.S b/arch/x86/crypto/sha256_ni_asm.S index 94d50dd27cb53..47f93937f798a 100644 --- a/arch/x86/crypto/sha256_ni_asm.S +++ b/arch/x86/crypto/sha256_ni_asm.S @@ -54,6 +54,7 @@ */ #include +#include #define DIGEST_PTR %rdi /* 1st arg */ #define DATA_PTR %rsi /* 2nd arg */ @@ -97,7 +98,7 @@ .text .align 32 -SYM_FUNC_START(sha256_ni_transform) +SYM_TYPED_FUNC_START(sha256_ni_transform) shl $6, NUM_BLKS /* convert to bytes */ jz .Ldone_hash From patchwork Fri Nov 18 09:02:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047882 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 31B54C433FE for ; Fri, 18 Nov 2022 09:07:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=djs5I/b1/vKy4wmCfQcHdEU6E6+UkxZnk7MrQTx+iW4=; b=L4yPRboBcA8yiE 2SmkGRSpkyKjDmvuWwLKFMqu3POzE3qNCAeR56+449jumNgS7pAgsJyrCng+R1qqT9GVcHmrYnKY7 ichV6T13ufD8VCS9KwSYyMVYtIpM2vEQRNKfxKaR1DAVkj+PCX6XYRCzZoBBj3UyklNGntHf7zGq2 KP6teGkfdpWm1hur+PrncTeSEUWnUuuqXEjJYfQmFQMu5HdaGwcm0qhhHORP46AEb9rMUAkhBbcW6 In/V7zUwvs+/dX7xs8OAvtr/7rB3+ixmhkD2NCcHtXSrR1NMXu5a4yuHGOzd4HailPX7NjFn1XCa/ xDQgg8R0XiKo7STv6JYg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxKb-002x3R-7Q; Fri, 18 Nov 2022 09:06:42 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIA-002vvo-Qw for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:15 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 43A88623BE; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC565C43146; Fri, 18 Nov 2022 09:04:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762249; bh=cvZrVNyaorZiFckp5I/DQO2Na69JdiQKH9GTwngz5kw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Txbmq1+8Os6ha8bRpkOEpMYl4yfovwRGORc/Q36bVbY0f/kt0jgLZs757o5lelPUR M4/GAy/RPij5/oInIsOl55+yuMlU2sQFJ05AcXKY4PudJAEycWR9iXNmcx1h5h3YNR 209MaRi+OoMvgNO4JI6rRGhf58knvpGrLNL6l5Kryv6wNaau6jo0dH5ItNYO++N6F1 PgC/p10LMij1YENAKkUG7CjntqP+312yJl+rZHiDardnnksxhPWLC1/rZoymXxqI5v dJORoyoJ2st87QAEJ/29Yi/9ygXONH/V34yN4BAKpsyZDJUpXzhspBzN7sSC19jbu0 KFZ777WpuVDDw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 06/11] crypto: x86/sha512 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:15 -0800 Message-Id: <20221118090220.398819-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010411_063101_F54BF322 X-CRM114-Status: GOOD ( 13.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sha512_transform_rorx(), sha512_transform_ssse3(), and sha512_transform_avx() are called via indirect function calls. These functions need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/sha512-avx-asm.S | 3 ++- arch/x86/crypto/sha512-avx2-asm.S | 3 ++- arch/x86/crypto/sha512-ssse3-asm.S | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/arch/x86/crypto/sha512-avx-asm.S b/arch/x86/crypto/sha512-avx-asm.S index 1fefe6dd3a9e2..b0984f19fdb40 100644 --- a/arch/x86/crypto/sha512-avx-asm.S +++ b/arch/x86/crypto/sha512-avx-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include .text @@ -273,7 +274,7 @@ frame_size = frame_WK + WK_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks ######################################################################## -SYM_FUNC_START(sha512_transform_avx) +SYM_TYPED_FUNC_START(sha512_transform_avx) test msglen, msglen je nowork diff --git a/arch/x86/crypto/sha512-avx2-asm.S b/arch/x86/crypto/sha512-avx2-asm.S index 5cdaab7d69015..b1ca99055ef99 100644 --- a/arch/x86/crypto/sha512-avx2-asm.S +++ b/arch/x86/crypto/sha512-avx2-asm.S @@ -50,6 +50,7 @@ ######################################################################## #include +#include .text @@ -565,7 +566,7 @@ frame_size = frame_CTX + CTX_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks ######################################################################## -SYM_FUNC_START(sha512_transform_rorx) +SYM_TYPED_FUNC_START(sha512_transform_rorx) # Save GPRs push %rbx push %r12 diff --git a/arch/x86/crypto/sha512-ssse3-asm.S b/arch/x86/crypto/sha512-ssse3-asm.S index b84c22e06c5f7..c06afb5270e5f 100644 --- a/arch/x86/crypto/sha512-ssse3-asm.S +++ b/arch/x86/crypto/sha512-ssse3-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include .text @@ -274,7 +275,7 @@ frame_size = frame_WK + WK_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks. ######################################################################## -SYM_FUNC_START(sha512_transform_ssse3) +SYM_TYPED_FUNC_START(sha512_transform_ssse3) test msglen, msglen je nowork From patchwork Fri Nov 18 09:02:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047876 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A318EC4332F for ; Fri, 18 Nov 2022 09:06:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Wj7PXnPyMwIBlPNKJWXZh/DH0zRUR8yLD7J+hPVVcnI=; b=1T4UGTmynmqDXJ hW+AmfTLfwQ+TKquDDm9fpfms0ix/SyeLJwuRyLy9tcRQZIWliZdgN7FUwY3NJJXjF7obIH0sKcCW NLXpUXKqVl+FiT3B34rN5ybJob2qW8n9Zkyiv8wwq9f6MLIi6Q66DIWnk6BYrhJ+giFAN1bzvjUMU Rh6c4ADWbB/f/sMShw2mntL/VUSTQt0RJ2br5jtSN10UWEHjnmHLWcdgwB9hJZsb6Bnop3LeulH7Q YshsvML7sYrbX2DfVbDn9m6bo0Lzdo/dYKpqk9frQduYQdE9e4dBNTSBIuS29ehe+cyPF9hley0xi nOjdIHOdFZbQIL5KBHgg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxJO-002wNf-J4; Fri, 18 Nov 2022 09:05:27 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIA-002vvl-RP for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:13 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 89D7F623BF; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3CE0BC433D6; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762249; bh=NnbwjD/ex8fXbifBK9kxmybIvw2mJxpjvji26vXujDE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jCfMsnWwuLZr2TZMX42ziZkf9YQUI/Z7eUjtM7+vmN99cHpcGLu8BoyYZNKSin29Q fGxnshtgoc/9/yDLFLEvCqu+dmCY44y3gUTofehfryzsGicymkxlJRvWcOzafSe7bR O+0+aHnGVahABEPC8PVOdy6wUJmDMFaMs3SG+yzlzgmtg9chTFSU5PY1zqX3oE6rn0 HdtJBfAadfp4QGOozG5olUgrRaR/UzcUM8rx+AQ7bFrPrWMzygCCcJWZ+WCj7WiEag xGkZ2t7gmMPfzrRqraVIdDjSncz8/T2PdRKGzXB65iuxcp2jE92LWeR3TAePTE2GgF +AvIqUqoixt4w== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 07/11] crypto: x86/sm3 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:16 -0800 Message-Id: <20221118090220.398819-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010411_003372_D4275E96 X-CRM114-Status: GOOD ( 13.47 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sm3_transform_avx() is called via indirect function calls. This function needs to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect call). Fixes: 3c516f89e17e ("x86: Add support for CONFIG_CFI_CLANG") Signed-off-by: Eric Biggers --- arch/x86/crypto/sm3-avx-asm_64.S | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/crypto/sm3-avx-asm_64.S b/arch/x86/crypto/sm3-avx-asm_64.S index b12b9efb5ec51..8fc5ac681fd63 100644 --- a/arch/x86/crypto/sm3-avx-asm_64.S +++ b/arch/x86/crypto/sm3-avx-asm_64.S @@ -12,6 +12,7 @@ */ #include +#include #include /* Context structure */ @@ -328,7 +329,7 @@ * const u8 *data, int nblocks); */ .align 16 -SYM_FUNC_START(sm3_transform_avx) +SYM_TYPED_FUNC_START(sm3_transform_avx) /* input: * %rdi: ctx, CTX * %rsi: data (64*nblks bytes) From patchwork Fri Nov 18 09:02:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F3542C4321E for ; Fri, 18 Nov 2022 09:08:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fPdPQ2EOMgd97GJ4j/ynKBwSTKbDgcqfCf6a51ZbXlE=; b=bBXOHhTSIRwuM0 3cqm3ggkSOJYFv29Yf9qRhgfvFTaU8zA/2QeKp0+7jD6ADiG3/u+86UXFW+CfK1yHlW6d6TY7gIZ7 4/CAwbQ7oZrxY1iCWOQ2qbhvIZVwlzHftWwFGVNa3S/pQcZMymt+oCAOS7utuy9SOBCvDEkPH+VkI YnuekqkLaUhRDFTpG4mg88ER+qT5LnT/0Aq3w20/kCZ8gezOjYPOndkyS63io9lS4xNiIr9RP4dDF nGv9H6jQx/6SugH3de63Cr3EUzOEvZ0Ackc3cy0kx9hbMk8jOYoS1BMZwSiqOvkMQNEo23U3LW7mS M/+Kfv6Wwq7/xMGICDXw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxL3-002xK4-Ho; Fri, 18 Nov 2022 09:07:10 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIB-002vvy-4F for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:16 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CD506623C2; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8187AC43146; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762249; bh=fc2JKYQatpTxXu11VT4a3V3fN6ivlyjI0KHBLOLHcEA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tWBHjlxK1blh/DEay6JaIGEL7wJIjBmtDxcPiGMfrHpdnkXk+LoFlctITaF3RXAYV 2yWhgt/dqlHzVPsnZthunPoK6jWayfFXHAD6LUeActAsOQ8ePYimylbS5Kz7BRh8oj hzDBNypHtok8kY0GHME3FQbOZZgA420/7KWbEY0uCwenBN4Kdp+SbSWcI2wcaITHuP 3vVdytMQFZYh14BrSbYisTfcmf8u34OleBaax7luuOy+9SX3OT7ImWs7FGdn2yoMBi KuLCzI2RD8DYd6sIbjyWH13Sa0UJqcfyybEMoAvCiY6CVmQ2BK2dHRazBnf4TQ4NEg pAKiN/iYuB9uA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 08/11] crypto: arm64/nhpoly1305 - eliminate unnecessary CFI wrapper Date: Fri, 18 Nov 2022 01:02:17 -0800 Message-Id: <20221118090220.398819-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010411_267637_317CD2F4 X-CRM114-Status: GOOD ( 12.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers Since the CFI implementation now supports indirect calls to assembly functions, take advantage of that rather than use a wrapper function. Signed-off-by: Eric Biggers --- arch/arm64/crypto/nh-neon-core.S | 5 +++-- arch/arm64/crypto/nhpoly1305-neon-glue.c | 11 ++--------- 2 files changed, 5 insertions(+), 11 deletions(-) diff --git a/arch/arm64/crypto/nh-neon-core.S b/arch/arm64/crypto/nh-neon-core.S index 51c0a534ef87c..13eda08fda1e5 100644 --- a/arch/arm64/crypto/nh-neon-core.S +++ b/arch/arm64/crypto/nh-neon-core.S @@ -8,6 +8,7 @@ */ #include +#include KEY .req x0 MESSAGE .req x1 @@ -58,11 +59,11 @@ /* * void nh_neon(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_neon) +SYM_TYPED_FUNC_START(nh_neon) ld1 {K0.4s,K1.4s}, [KEY], #32 movi PASS0_SUMS.2d, #0 diff --git a/arch/arm64/crypto/nhpoly1305-neon-glue.c b/arch/arm64/crypto/nhpoly1305-neon-glue.c index c5405e6a6db76..cd882c35d9252 100644 --- a/arch/arm64/crypto/nhpoly1305-neon-glue.c +++ b/arch/arm64/crypto/nhpoly1305-neon-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_neon(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_neon(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_neon(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_neon_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_neon_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_neon_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_neon); + crypto_nhpoly1305_update_helper(desc, src, n, nh_neon); kernel_neon_end(); src += n; srclen -= n; From patchwork Fri Nov 18 09:02:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3795FC4332F for ; Fri, 18 Nov 2022 09:06:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gzGin0NsiCeEZ79TLSi9BEGnbd0hpN7tY0r3govmUsk=; b=eECYM/FxAVbysM FTdNTDsPtSQOneD5K01KFDBm6E1+KHLoKCj2VML6OsOdT9dfCWsdWyiG1LAEnozVuwD5fuEEFav9d x+GisaX3UpEZRt6kTizmNABye9ebHKxyvwzXx+LTZQqSqvjcSrfcse7iND9O8k6WwVkGiKN5Ynkfa BzJSJ3TyK3/pKptSgreVXGoS2AA8bcmmsVp3TDdXsnjoWz71cH/mZSB5+taJtl2BGkWQD+unWmDf9 u4yWxcAwc1JFSBkByJafbcEtgyCTSYTU4+hsHEQ0b6JIBsDiMxOyHcbLCz7EB82RyfHmukruwgNTB 0KANV9kKzvCHxXk+Jf1g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxJc-002wWP-Sa; Fri, 18 Nov 2022 09:05:41 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIA-002vvz-WE for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:14 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1DF01623C4; Fri, 18 Nov 2022 09:04:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C609FC433D7; Fri, 18 Nov 2022 09:04:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762249; bh=wyK4erNXYZLb0gGezT+2lcL+zroMKoz8h7GycCvuO2k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NqABtUTYY1aVx9PfXGJq58Dbdt7rDMgcoP+hSxU5ThNarF3tRMyvrwQwAEkXDGUO5 tiJ5U6LMLR0lV4JumLMkHK8INpJW3VIBKbH25sTBOZ87643hheINT8JJZJRVNf2TJF Avmp31f8rb9nVNMm7VPa6EXaX/IeeMYxx0SdXLQ4veF50FCyygcXgJzJ0h/mNWUiz0 nXsrpByKTHAWcFIFVyfqlGKRd3pquE2LM5zke5eFjTEU3S7TFTNO9O/gQ3AjOeWI8Y mPsBOdZcHGDb2+C+DgTlFSdpfjQy/mk723T/7DJTGcpPCtgOYlXcdr21Vaz+4C3FCr kxX8ORTboM8rw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 09/11] crypto: arm64/sm3 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 01:02:18 -0800 Message-Id: <20221118090220.398819-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010411_140148_702E2630 X-CRM114-Status: GOOD ( 13.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sm3_neon_transform() is called via indirect function calls. This function needs to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect call). Fixes: c50d32859e70 ("arm64: Add types to indirect called assembly functions") Signed-off-by: Eric Biggers --- arch/arm64/crypto/sm3-neon-core.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/crypto/sm3-neon-core.S b/arch/arm64/crypto/sm3-neon-core.S index 3e3b4e5c736fc..8abea1d39ddd9 100644 --- a/arch/arm64/crypto/sm3-neon-core.S +++ b/arch/arm64/crypto/sm3-neon-core.S @@ -9,7 +9,7 @@ */ #include -#include +#include /* Context structure */ @@ -351,7 +351,7 @@ */ .text .align 3 -SYM_FUNC_START(sm3_neon_transform) +SYM_TYPED_FUNC_START(sm3_neon_transform) ldp ra, rb, [RSTATE, #0] ldp rc, rd, [RSTATE, #8] ldp re, rf, [RSTATE, #16] From patchwork Fri Nov 18 09:02:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047884 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AFF96C4332F for ; Fri, 18 Nov 2022 09:08:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GAzc2ZXeedTjA9veD9LiRDiIXKhC/FcGcp3GrNxOwsI=; b=XZVTg6zTRVFeZ8 opL544DIrkMKcrXx7fWcGqIDXu4pLn93VfLBGYtSKsU51kPYqBD/Pwh1maYKq0h3VgtOwEUkLWYdy jtWlaUUO4N0n7OUdaAYJ1VX/L+CMxeFN01dgOqXG0J3jd8km2yrqaP1DDyeIrtUZITkAdYqw2BYtO 6VZ/F3UE1ARfS2lvZ4oUWi3MQtNF3LHl1Ke2sYNVIB4UUomaaEtH8heSrKbQsEJWJ0gjor7yE2Vpc QI9KHrthOZAYvk/S5/YjS/TgzBxh0RnEKusYUXbtRDzKp/Ul1ALwTYSt6kwnjc7RU2x9ZVwzt14dp kDLOrZu9tHZKP0KWUpOQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxLP-002xVy-Ec; Fri, 18 Nov 2022 09:07:31 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIC-002vyT-ND for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:16 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 621386230C; Fri, 18 Nov 2022 09:04:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1685FC433D6; Fri, 18 Nov 2022 09:04:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762250; bh=ZkpnHZpH6zqpcaYDXylCMZLyYKMTqb6ZC8hlYT4Izo8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R2q5ZahIINyliBIF3hGitjFBaxgXe56Xs/ajq8hQ9BzHkYYdLsTgyQc1vuiSh9+II A4s2sslrbAetPUZYuXi5H+VrV0fLTlcyAv5NX1+nP3zQvorkEFN2+IR+VvgvrTGcb6 lqNzs3zcmptAv2UmuXiYpir0qL+WcvzbMaSQCoONYTaYYZrp2pTMnwLmfzURkEXpSW eCFl4arg6ka2UyoIGdcva1Pko+SP+kvRAwlpDC8W8fqixvLqQMiwyvmygshZHgys2C r0mc2YrT5iakwAQg1E8sbyR3VsmN8nSZ1EXoIzio9gKli3pshLwXWcINSXMRfSaMEx jE58Qto0vDAjw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 10/11] crypto: arm/nhpoly1305 - eliminate unnecessary CFI wrapper Date: Fri, 18 Nov 2022 01:02:19 -0800 Message-Id: <20221118090220.398819-11-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_010412_859031_DCDFDF96 X-CRM114-Status: GOOD ( 13.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers The arm architecture doesn't support CFI yet, and even if it did, the new CFI implementation supports indirect calls to assembly functions. Therefore, there's no need to use a wrapper function for nh_neon(). Signed-off-by: Eric Biggers --- arch/arm/crypto/nh-neon-core.S | 2 +- arch/arm/crypto/nhpoly1305-neon-glue.c | 11 ++--------- 2 files changed, 3 insertions(+), 10 deletions(-) diff --git a/arch/arm/crypto/nh-neon-core.S b/arch/arm/crypto/nh-neon-core.S index 434d80ab531c2..01620a0782ca9 100644 --- a/arch/arm/crypto/nh-neon-core.S +++ b/arch/arm/crypto/nh-neon-core.S @@ -69,7 +69,7 @@ /* * void nh_neon(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ diff --git a/arch/arm/crypto/nhpoly1305-neon-glue.c b/arch/arm/crypto/nhpoly1305-neon-glue.c index ffa8d73fe722c..e93e41ff26566 100644 --- a/arch/arm/crypto/nhpoly1305-neon-glue.c +++ b/arch/arm/crypto/nhpoly1305-neon-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_neon(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_neon(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_neon(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_neon_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_neon_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_neon_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_neon); + crypto_nhpoly1305_update_helper(desc, src, n, nh_neon); kernel_neon_end(); src += n; srclen -= n; From patchwork Fri Nov 18 09:02:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13047885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1294C433FE for ; Fri, 18 Nov 2022 09:09:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KtztKp2PzvIQgf3cP8vujAyuj3w95MWuewLEEtEHHAM=; b=VQsZx4dEx/M0RO IW7Wap3GHAWDQGT3zpE9QxjNPiAFLZSXw/kZ5XqFkfkE25vM6G+qH6ooQ6v8WWx5XIAD01v6d/F6e 8wwqvARlvtth1WX27KwRlMnu9xU2cJWjBuC/b5RxW+n+78BBL5hX//kcUM3AHI8Mjnyn+GH7H+wRn 4ktZREcbvyu2sxNsMVjQ4WRs6kp8YJWC48wmkCmf+Ie8HCj4j7XgD5fMh0fbeTjVh6dPq/LZRVWKV kdR8ffAJpx7IPY/vD4yJjVvLBtbC7oAsMphhry9VyvYGKJ/6WktVkpqdE5ljBzPh7gtuwHdshRUMn Goh7aV+bckskRZ9ZZxdg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxLn-002xjj-Sm; Fri, 18 Nov 2022 09:07:56 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIV-002w4M-4w for linux-arm-kernel@bombadil.infradead.org; Fri, 18 Nov 2022 09:04:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=1uaEfelK0/ZvXq/IuXnWefcz8Wg3/V/JgQXtRs4CXJ0=; b=hFriAaJES77qqGdN9Zancm/Dey fcS02eYPPR5ckOg/FVFYY+RtRvN6gz6A5pMRu+O5XS19j3LSiPDjtfmGU1hNJcCeCiUzsbKq8h6px /wY99DP2TzM9o7Uznm9LOdiknlyTUqucluSqCz+zysWQSwfvo9yTOOuPOIiTeWaXgO8EwMsZ1XCeQ mgbbuDIVmzZBFFR6PiNmnYPqC6Dqa1ozko/AXkctVHOwoZ2Qmq7hY1NO4Q839U+igUZshoohAiQhi e5gyYLY8tBZCTQw1atVKbWjF/EyoeC2A/sGqbJTxipQvndpzI6G0rAL7RJTScrmlsr0y5tlF4lTxM 7bvKfuLQ==; Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ovxIO-001vRO-5Z for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 09:04:28 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B5750B822CE; Fri, 18 Nov 2022 09:04:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5B7DCC43141; Fri, 18 Nov 2022 09:04:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668762250; bh=dN9lqJsqSknnJEVWfdbcH4PBv1COtt2elsbRPuqKWpA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=liqNp19oNyCipr9vwmUXYl94kRjIRI2R4Ks7FBDJWaZW/5/EuRAhr/i12ZO3dqwJP sIqNH+Jm/2M9dp86/CFzURzWn2glS35cKQ90JbEiRXZMJTCuw0I530QCO8Ky/pLakU 7PwI6ceXuN1oko89Q51kX8xlOEHPjAWuCrwXNpwKbkPJgppTkcii8c6fzUwxrE50SL 4rlviybpw4WjTC11URRRuqmpOza2fS6yhoHkrimgQPFWhu8jVzuWuUD5F1G/NIr9sn b93AtG0dxbXnWmfEoxmRkT4wW7raI2Wcmz1wMFy8WG+BQpTJpHTq1hVIFMbypx0MST U6EzSEKEO/KtA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH 11/11] Revert "crypto: shash - avoid comparing pointers to exported functions under CFI" Date: Fri, 18 Nov 2022 01:02:20 -0800 Message-Id: <20221118090220.398819-12-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118090220.398819-1-ebiggers@kernel.org> References: <20221118090220.398819-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_090424_359379_10AF162D X-CRM114-Status: GOOD ( 14.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers This reverts commit 22ca9f4aaf431a9413dcc115dd590123307f274f because CFI no longer breaks cross-module function address equality, so crypto_shash_alg_has_setkey() can now be an inline function like before. This commit should not be backported to kernels that don't have the new CFI implementation. Signed-off-by: Eric Biggers --- crypto/shash.c | 18 +++--------------- include/crypto/internal/hash.h | 8 +++++++- 2 files changed, 10 insertions(+), 16 deletions(-) diff --git a/crypto/shash.c b/crypto/shash.c index 4c88e63b3350f..0f85431588267 100644 --- a/crypto/shash.c +++ b/crypto/shash.c @@ -20,24 +20,12 @@ static const struct crypto_type crypto_shash_type; -static int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, - unsigned int keylen) +int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, + unsigned int keylen) { return -ENOSYS; } - -/* - * Check whether an shash algorithm has a setkey function. - * - * For CFI compatibility, this must not be an inline function. This is because - * when CFI is enabled, modules won't get the same address for shash_no_setkey - * (if it were exported, which inlining would require) as the core kernel will. - */ -bool crypto_shash_alg_has_setkey(struct shash_alg *alg) -{ - return alg->setkey != shash_no_setkey; -} -EXPORT_SYMBOL_GPL(crypto_shash_alg_has_setkey); +EXPORT_SYMBOL_GPL(shash_no_setkey); static int shash_setkey_unaligned(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) diff --git a/include/crypto/internal/hash.h b/include/crypto/internal/hash.h index 25806141db591..0a288dddcf5be 100644 --- a/include/crypto/internal/hash.h +++ b/include/crypto/internal/hash.h @@ -75,7 +75,13 @@ void crypto_unregister_ahashes(struct ahash_alg *algs, int count); int ahash_register_instance(struct crypto_template *tmpl, struct ahash_instance *inst); -bool crypto_shash_alg_has_setkey(struct shash_alg *alg); +int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, + unsigned int keylen); + +static inline bool crypto_shash_alg_has_setkey(struct shash_alg *alg) +{ + return alg->setkey != shash_no_setkey; +} static inline bool crypto_shash_alg_needs_key(struct shash_alg *alg) {