From patchwork Fri Nov 18 19:44:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6CD3BC433FE for ; Fri, 18 Nov 2022 21:02:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0Hy6zq6EkkM5cZUxKS7y0RkTt6KBqpv0kULHFMNeqsY=; b=gNMMhO19ETu/CL wuEFkVjLaSJZEoV7tAKk0NKgAmT6vTaiNXTCr/koUrqV7TUe5s6NsjC4EgTH451yFAKWzQGTgqh9G KebTGo8Td6HLI7W1DI/PKunZlTdO9XOBgXzMCi4i6o7cOWqXanZFdDF2lNjyfrGxH0HmCCghiJ8yN fY6VI2dqJ1BdzL/AKRYdAcG0w8s0dtBPn/n1gQbWjFBSzkBNgu218LHPd4OoNjrUYR0+LmN43vc+L tyyWonF35uQysfks4fFw8B+cs+hMvCRwpX/bhyFu7plyK7Y4Dj+wyPUsacdr5qD2NgWd+8Ax/JOGV amUUBmtYE72WIMkWmy+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow8UI-0084uM-Kb; Fri, 18 Nov 2022 21:01:26 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jh-007gwP-9r for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:28 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C90C362766; Fri, 18 Nov 2022 19:46:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 14DC5C433D6; Fri, 18 Nov 2022 19:46:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800784; bh=pld3XDt3BvGRUOAeWpGABORV9PhFrHCg0jaaS0wG4JY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ctclds9W9ZQRSH26nNSD/4Ztv5PPT1NqhIwr7+W5kWizkj4yfFlpOdLRj/psyZVSt k+EdrCpXYDx6ZsoGWYiZJJPYkPTNlE8vpZnWLQNDKu9N4cYokXEmJpJYqIhQBthvpp a1qZb2p+czXkPd4h+pRTXsJ1nwPkPFoR5NTZngEaM0IfsQho2kDdlLWgFUO8OegUFo NAg6JV8PTfqFhiqQS7xMNHdoPfrl7OG/PZle6XjzX6pNk1vEL5a7QaxnwJa71XrrEn 709qI+aFBuclXf9C3g/x11nAaBanv8tiJU5QQh+mWfWSmNJKqzNhqJNbFS1KBS0VQ4 S5N9IJkjWE/BA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 01/12] crypto: x86/aegis128 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:10 -0800 Message-Id: <20221118194421.160414-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_114625_505959_7B1D391F X-CRM114-Status: GOOD ( 12.47 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers crypto_aegis128_aesni_enc(), crypto_aegis128_aesni_enc_tail(), crypto_aegis128_aesni_dec(), and crypto_aegis128_aesni_dec_tail() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/aegis128-aesni-asm.S | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/aegis128-aesni-asm.S b/arch/x86/crypto/aegis128-aesni-asm.S index b48ddebb47489..cdf3215ec272c 100644 --- a/arch/x86/crypto/aegis128-aesni-asm.S +++ b/arch/x86/crypto/aegis128-aesni-asm.S @@ -7,6 +7,7 @@ */ #include +#include #include #define STATE0 %xmm0 @@ -402,7 +403,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_ad) * void crypto_aegis128_aesni_enc(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_enc) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_enc) FRAME_BEGIN cmp $0x10, LEN @@ -499,7 +500,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_enc) * void crypto_aegis128_aesni_enc_tail(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_enc_tail) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_enc_tail) FRAME_BEGIN /* load the state: */ @@ -556,7 +557,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_enc_tail) * void crypto_aegis128_aesni_dec(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_dec) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_dec) FRAME_BEGIN cmp $0x10, LEN @@ -653,7 +654,7 @@ SYM_FUNC_END(crypto_aegis128_aesni_dec) * void crypto_aegis128_aesni_dec_tail(void *state, unsigned int length, * const void *src, void *dst); */ -SYM_FUNC_START(crypto_aegis128_aesni_dec_tail) +SYM_TYPED_FUNC_START(crypto_aegis128_aesni_dec_tail) FRAME_BEGIN /* load the state: */ From patchwork Fri Nov 18 19:44:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 961A7C4332F for ; Fri, 18 Nov 2022 19:53:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HgfIcW48keyl3nCgkZ45XZvMVqyAzLPDxYHnSpfsAuE=; b=Oe7rmXNJlBF/GQ yEMeMfAXQa1IGDyS+hXgV4Nl8GeA1+sGYOX9d/mbEsDTRkZ55so7m9oUUPNouFufCFG76pH444XJh vQyoS4BFflVfVyyhhf4ZQofomG35oTwlgCRuESMasrHiK/1ZpKuKRXqadJG1m4zXj/XVhhK4tGryJ ZWxYyhdKyu9zFs3IfBjCHDnES4MBGIwlPm/pgKje0ZbClMRX6swfqRW3OizHNP8hlT4mKSg15qd7H hTGFh/WZPzmfoc5InrPskGEuYj4aSt7wVsQAIlwOAJ9qy/vJsJruI2CYpPHpxkqg+REIWQ6pNgwrd /HY4mj5ATYUZ8XCsXrsw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7PB-007jme-RX; Fri, 18 Nov 2022 19:52:06 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jq-007gzt-I8 for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:39 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 4574CCE2105; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 60099C43470; Fri, 18 Nov 2022 19:46:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800784; bh=1AOSU6xzPkyZPNezMVrPhaztVE44hJw1ys9zdKY6koM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A+ktCSy592Yjx/NCkOseT3ZiTEEtueqN9gCPefhFCsZ7WThNX6ly9+2KQIdYpupDm LtqZdxpsDM1ViXm7uZFXm+wdrZZcvqW0zs20zoPzblN39ubY7N+kspDUSqMSlc3hL7 S1YRE8ERG9kHtmcgr5c7dBVPzA/vu2Z6VTCDw9Nvl4DbWVOwZ8mEd1s44kfgC2V/e2 1KieEmygbTqVoBhn4Kg52pRrOevsUYdzYV+MGC92TnIk1kvsG4eg4HesH4qgNT1HtG ukjQIlxdwsolBskgNvVR0UHY9lMwAWMOfXrs91QHaF2GHMKrmtg4i7f0Drx/OvU1nm JCiFvhpfmOvTw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , Taehee Yoo , "Peter Zijlstra (Intel)" Subject: [PATCH v2 02/12] crypto: x86/aria - fix crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:11 -0800 Message-Id: <20221118194421.160414-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_114634_998085_0977D411 X-CRM114-Status: GOOD ( 11.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers aria_aesni_avx_encrypt_16way(), aria_aesni_avx_decrypt_16way(), aria_aesni_avx_ctr_crypt_16way(), aria_aesni_avx_gfni_encrypt_16way(), aria_aesni_avx_gfni_decrypt_16way(), and aria_aesni_avx_gfni_ctr_crypt_16way() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure. Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Cc: Taehee Yoo Signed-off-by: Eric Biggers --- arch/x86/crypto/aria-aesni-avx-asm_64.S | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/x86/crypto/aria-aesni-avx-asm_64.S b/arch/x86/crypto/aria-aesni-avx-asm_64.S index c75fd7d015ed8..03ae4cd1d976a 100644 --- a/arch/x86/crypto/aria-aesni-avx-asm_64.S +++ b/arch/x86/crypto/aria-aesni-avx-asm_64.S @@ -7,6 +7,7 @@ */ #include +#include #include /* struct aria_ctx: */ @@ -913,7 +914,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_crypt_16way) RET; SYM_FUNC_END(__aria_aesni_avx_crypt_16way) -SYM_FUNC_START(aria_aesni_avx_encrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_encrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -938,7 +939,7 @@ SYM_FUNC_START(aria_aesni_avx_encrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_encrypt_16way) -SYM_FUNC_START(aria_aesni_avx_decrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_decrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1039,7 +1040,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_ctr_gen_keystream_16way) RET; SYM_FUNC_END(__aria_aesni_avx_ctr_gen_keystream_16way) -SYM_FUNC_START(aria_aesni_avx_ctr_crypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_ctr_crypt_16way) /* input: * %rdi: ctx * %rsi: dst @@ -1208,7 +1209,7 @@ SYM_FUNC_START_LOCAL(__aria_aesni_avx_gfni_crypt_16way) RET; SYM_FUNC_END(__aria_aesni_avx_gfni_crypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1233,7 +1234,7 @@ SYM_FUNC_START(aria_aesni_avx_gfni_encrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_gfni_encrypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) /* input: * %rdi: ctx, CTX * %rsi: dst @@ -1258,7 +1259,7 @@ SYM_FUNC_START(aria_aesni_avx_gfni_decrypt_16way) RET; SYM_FUNC_END(aria_aesni_avx_gfni_decrypt_16way) -SYM_FUNC_START(aria_aesni_avx_gfni_ctr_crypt_16way) +SYM_TYPED_FUNC_START(aria_aesni_avx_gfni_ctr_crypt_16way) /* input: * %rdi: ctx * %rsi: dst From patchwork Fri Nov 18 19:44:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048734 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3EC37C4332F for ; Fri, 18 Nov 2022 19:48:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=e5F07OVwLgK4+HXbocy6v5i0tIF9rKxwX7vS+BrGQO4=; b=lWLahmTEv2npJ+ er/QMOD9Zmk67Ct27HgGCsjc6iQdqFVISFyGnepCeUmP5wrPP3HtRjd363bfi/8yohD1ghAPJ6LLz Newuz7s9FvEBc/OiXTeoskl/GnW4SHTcFcCorml2JprHXBEcOLObwRSCJlPI337YBOsLLLn5S5UPC beDB22w4wm8ajDgK1RitMIsM5JgrQFBDIyU5HzNaFRNw7MD4JoWyvtBB9XFSfiDUM8wrQZtkNHMR6 0L0tHD1lGJ5sFLZezgoTr2fjAj+HDSMoVE9wrKLXRBY+yFZW4LPrKFsw7427kbJ465k04w9Blc/E3 Eb1I0ASs5quzkNNybXdw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Kj-007hYg-7p; Fri, 18 Nov 2022 19:47:29 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jh-007gxB-SV for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:31 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6EDD462761; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B23D2C43140; Fri, 18 Nov 2022 19:46:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800784; bh=EjO3B77EIOTiWRygSBkRyIXiyATPysPcP7wmb2ToCDs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VHXo4sVaL3o8Zv5oH/58o8qIXTUaVmfy/xvsdGatFdPmp72sZHeiCQamxe+6zQLsv OrtAe1wOfSxzp+zI/VCoo9wVEipbCdEj5JmFl965KkqVSp35U4Se/YvdlPGFpqfpfO 29qfQnUPE6rtrRt3x6LORz70j1MH2sk7M1heQar6wUhUoRsIdcorRcrZYafkhOU46f aFzNTyKA2fJNvXLm+ETMq9XrfxIpnJh4iIS5V7uncd4KkSvJ+sZGsV3d//tn8BUU0/ SxmOROrRMkYf7tWReVw+SrizAR1o/plLqnHF/tF5BQ5HPjG/S4N6dJacB4XgPnGus0 lgPGT5ayQVuiw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 03/12] crypto: x86/nhpoly1305 - eliminate unnecessary CFI wrappers Date: Fri, 18 Nov 2022 11:44:12 -0800 Message-Id: <20221118194421.160414-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_114626_060958_C3ABDE7E X-CRM114-Status: GOOD ( 11.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers Since the CFI implementation now supports indirect calls to assembly functions, take advantage of that rather than use wrapper functions. Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/nh-avx2-x86_64.S | 5 +++-- arch/x86/crypto/nh-sse2-x86_64.S | 5 +++-- arch/x86/crypto/nhpoly1305-avx2-glue.c | 11 ++--------- arch/x86/crypto/nhpoly1305-sse2-glue.c | 11 ++--------- 4 files changed, 10 insertions(+), 22 deletions(-) diff --git a/arch/x86/crypto/nh-avx2-x86_64.S b/arch/x86/crypto/nh-avx2-x86_64.S index 6a0b15e7196a8..ef73a3ab87263 100644 --- a/arch/x86/crypto/nh-avx2-x86_64.S +++ b/arch/x86/crypto/nh-avx2-x86_64.S @@ -8,6 +8,7 @@ */ #include +#include #define PASS0_SUMS %ymm0 #define PASS1_SUMS %ymm1 @@ -65,11 +66,11 @@ /* * void nh_avx2(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_avx2) +SYM_TYPED_FUNC_START(nh_avx2) vmovdqu 0x00(KEY), K0 vmovdqu 0x10(KEY), K1 diff --git a/arch/x86/crypto/nh-sse2-x86_64.S b/arch/x86/crypto/nh-sse2-x86_64.S index 34c567bbcb4fa..75fb994b6d177 100644 --- a/arch/x86/crypto/nh-sse2-x86_64.S +++ b/arch/x86/crypto/nh-sse2-x86_64.S @@ -8,6 +8,7 @@ */ #include +#include #define PASS0_SUMS %xmm0 #define PASS1_SUMS %xmm1 @@ -67,11 +68,11 @@ /* * void nh_sse2(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_sse2) +SYM_TYPED_FUNC_START(nh_sse2) movdqu 0x00(KEY), K0 movdqu 0x10(KEY), K1 diff --git a/arch/x86/crypto/nhpoly1305-avx2-glue.c b/arch/x86/crypto/nhpoly1305-avx2-glue.c index 8ea5ab0f1ca74..46b036204ed91 100644 --- a/arch/x86/crypto/nhpoly1305-avx2-glue.c +++ b/arch/x86/crypto/nhpoly1305-avx2-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_avx2(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_avx2(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_avx2(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_avx2_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_avx2_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_fpu_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_avx2); + crypto_nhpoly1305_update_helper(desc, src, n, nh_avx2); kernel_fpu_end(); src += n; srclen -= n; diff --git a/arch/x86/crypto/nhpoly1305-sse2-glue.c b/arch/x86/crypto/nhpoly1305-sse2-glue.c index 2b353d42ed13f..4a4970d751076 100644 --- a/arch/x86/crypto/nhpoly1305-sse2-glue.c +++ b/arch/x86/crypto/nhpoly1305-sse2-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_sse2(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_sse2(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_sse2(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_sse2_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_sse2_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_fpu_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_sse2); + crypto_nhpoly1305_update_helper(desc, src, n, nh_sse2); kernel_fpu_end(); src += n; srclen -= n; From patchwork Fri Nov 18 19:44:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A277DC4332F for ; Fri, 18 Nov 2022 19:48:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KlEoLiP0pOtqyxMKfGRce/B87UWbBcwF+QCu3NHuO+M=; b=ZyEIm1d0ehj1CT g4EVY6Snq3QVw4+yJp+AVwGTruvYsneyI8fOiSOr/mmMr3D3RTAEed0/MAyTG9MKwjUEuBDubCiV0 KsY8WBJiK4/cYFsZ2BCKwjU0fXIEoS62E8q2kZnQJomLK3GB8dT3QEazEslp4EC4agCG4VDzsu3cX d1ZVkaSAdcj+3lGzfpl06xVhJ4aJFoaU/0x6OT7edwliV2fzE2YImwZ30NcqaSUrXZOXJ3sKBZdy6 lm7vTr69BD9x06/FOWyOWpldjRNr5IbaBbR6fA/Z7ZkilYhdkezDcLueyVwt49aAKtZUkRzELq4+Y GsqvaI+UQyM9MwZpxE7Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7K3-007hAM-Bg; Fri, 18 Nov 2022 19:46:48 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jh-007gx1-Qa for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:30 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5C22362769; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B446C43145; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800785; bh=SJD1vbe3TbIhw+mwTcm5uBRYSsM1TzGphUBe1liqWrE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XVl2DMLVSs2HOx7cBmnsYJTM66kDDvXWPb6kNYxobQXvoNILZHP5QWdeeXp/xZX69 smMIgUE6p983K51FzNpAq24zTzSx+yVw1MMZc39NtTKCIdUkOb8MwdpH8QddBd+UCK aL7sM1o8a+PKgpxaxhSHsy/mIk8NQMdKyKLTVewufHX178cAhJ6ny04svwuucxOph/ uWYJncSByfA3ZtSqI5L5308goQSDGqKieLiK0bRgUj8RU+nUeRK5IyvyoOQE0IwGhx GOtXnFByE0zD+Qwmp24fzNwaP90B6VIDExux0LOgPgYRJq8KqBFsYtsg/TygwvGxA8 uOz10di6uwVgA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 04/12] crypto: x86/sha1 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:13 -0800 Message-Id: <20221118194421.160414-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_114625_989586_FE39E092 X-CRM114-Status: GOOD ( 12.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sha1_transform_ssse3(), sha1_transform_avx(), and sha1_ni_transform() (but not sha1_transform_avx2()) are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/sha1_ni_asm.S | 3 ++- arch/x86/crypto/sha1_ssse3_asm.S | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/crypto/sha1_ni_asm.S b/arch/x86/crypto/sha1_ni_asm.S index 2f94ec0e763bf..3cae5a1bb3d6e 100644 --- a/arch/x86/crypto/sha1_ni_asm.S +++ b/arch/x86/crypto/sha1_ni_asm.S @@ -54,6 +54,7 @@ */ #include +#include #define DIGEST_PTR %rdi /* 1st arg */ #define DATA_PTR %rsi /* 2nd arg */ @@ -93,7 +94,7 @@ */ .text .align 32 -SYM_FUNC_START(sha1_ni_transform) +SYM_TYPED_FUNC_START(sha1_ni_transform) push %rbp mov %rsp, %rbp sub $FRAME_SIZE, %rsp diff --git a/arch/x86/crypto/sha1_ssse3_asm.S b/arch/x86/crypto/sha1_ssse3_asm.S index 263f916362e02..f54988c80eb40 100644 --- a/arch/x86/crypto/sha1_ssse3_asm.S +++ b/arch/x86/crypto/sha1_ssse3_asm.S @@ -25,6 +25,7 @@ */ #include +#include #define CTX %rdi // arg1 #define BUF %rsi // arg2 @@ -67,7 +68,7 @@ * param: function's name */ .macro SHA1_VECTOR_ASM name - SYM_FUNC_START(\name) + SYM_TYPED_FUNC_START(\name) push %rbx push %r12 From patchwork Fri Nov 18 19:44:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3CAB8C4332F for ; Fri, 18 Nov 2022 19:49:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=l4loUEVpqPih5md35snn7SHjBPbqkEyLNT63LAMJeDU=; b=rPnzHCTyEif7a1 QOi/JtflaF3ABIfhdF5CmS/xZbUcA9p18YLIKfq5Qj40FX1n/vSl81KVmI71ew1IVPih3tW+ukpjE FBC0fwT2nLRU0VWl4gobrpmhUUQQOKqkWLJNw0JvhKhDlr7ff4t9J312OwckW/buuq1VEx6FJFyQx qqly0pTh2fuqLRNiFlc38mOt26Ms0UrHGdluf9O98bokwWgZ3PM78Mw7t5PWwSl0TdcGkQfQ/zZ3F OErw0LxBC7aRLZlgV2t45We2w1J2A4CnnmJSMCO49drOSdEcolyJe4x/QPqVExxoaKz+TZPIK55qR 8WNH2nBx85ZS74q84QDw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7LS-007huD-5c; Fri, 18 Nov 2022 19:48:15 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Ji-007gxL-83 for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:31 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A832B6276F; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5721AC43147; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800785; bh=qDEd6yjezAEKyeXhBgJqixcDjy7TkZZTMaumiYQZh3k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DBEPBR4DRSkF+Kb1ECCcGEn1Ubxv/dV/G+6aLXG4NvyXwplXEooeAdrG7XPZwaYKc j87GJp+ZztUXH6b6/DMu4YSbJ8g+K59EA+CgWzOtZmwy2FOtkx48JOpq4AP0jXjfQN UtvJ/BBhly38i9XphhacRGk0DiblV3Z0LH8hjnUfHIfI0Gh263SU4JMObQi6CM+hIq ZG7DoKNkPXOe+xxFWl5yy0iWYViMKuFKVcq1fPmKEXFbpHUEKCkDUHBQzOweqZDsnC nf14oYLw5fn3Szd9kQAPbtzdNC0zvehuAIhFTpydcbjgt8DcoAJB2GsZ2xvGZVieRa QjlqG1okzjFWA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 05/12] crypto: x86/sha256 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:14 -0800 Message-Id: <20221118194421.160414-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_114626_416542_D660AE38 X-CRM114-Status: GOOD ( 12.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sha256_transform_ssse3(), sha256_transform_avx(), sha256_transform_rorx(), and sha256_ni_transform() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/sha256-avx-asm.S | 3 ++- arch/x86/crypto/sha256-avx2-asm.S | 3 ++- arch/x86/crypto/sha256-ssse3-asm.S | 3 ++- arch/x86/crypto/sha256_ni_asm.S | 3 ++- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/sha256-avx-asm.S b/arch/x86/crypto/sha256-avx-asm.S index 3baa1ec390974..06ea30c20828d 100644 --- a/arch/x86/crypto/sha256-avx-asm.S +++ b/arch/x86/crypto/sha256-avx-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define VMOVDQ vmovdqu @@ -346,7 +347,7 @@ a = TMP_ ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_avx) +SYM_TYPED_FUNC_START(sha256_transform_avx) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256-avx2-asm.S b/arch/x86/crypto/sha256-avx2-asm.S index 9bcdbc47b8b4b..2d2be531a11ed 100644 --- a/arch/x86/crypto/sha256-avx2-asm.S +++ b/arch/x86/crypto/sha256-avx2-asm.S @@ -49,6 +49,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define VMOVDQ vmovdqu @@ -523,7 +524,7 @@ STACK_SIZE = _CTX + _CTX_SIZE ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_rorx) +SYM_TYPED_FUNC_START(sha256_transform_rorx) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256-ssse3-asm.S b/arch/x86/crypto/sha256-ssse3-asm.S index c4a5db612c327..7db28839108dd 100644 --- a/arch/x86/crypto/sha256-ssse3-asm.S +++ b/arch/x86/crypto/sha256-ssse3-asm.S @@ -47,6 +47,7 @@ ######################################################################## #include +#include ## assume buffers not aligned #define MOVDQ movdqu @@ -355,7 +356,7 @@ a = TMP_ ## arg 3 : Num blocks ######################################################################## .text -SYM_FUNC_START(sha256_transform_ssse3) +SYM_TYPED_FUNC_START(sha256_transform_ssse3) .align 32 pushq %rbx pushq %r12 diff --git a/arch/x86/crypto/sha256_ni_asm.S b/arch/x86/crypto/sha256_ni_asm.S index 94d50dd27cb53..47f93937f798a 100644 --- a/arch/x86/crypto/sha256_ni_asm.S +++ b/arch/x86/crypto/sha256_ni_asm.S @@ -54,6 +54,7 @@ */ #include +#include #define DIGEST_PTR %rdi /* 1st arg */ #define DATA_PTR %rsi /* 2nd arg */ @@ -97,7 +98,7 @@ .text .align 32 -SYM_FUNC_START(sha256_ni_transform) +SYM_TYPED_FUNC_START(sha256_ni_transform) shl $6, NUM_BLKS /* convert to bytes */ jz .Ldone_hash From patchwork Fri Nov 18 19:44:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048736 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 373E9C4332F for ; Fri, 18 Nov 2022 19:49:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XbFazh5Rg3jgdM+mz6L05h+56sy+ZHUUvcV4YP1lrzY=; b=ZUpQcslEFF43l1 sFev+6QHDqhrkjf04SjZr9I77uqp5kY8znGriIaDblIudZA1a6f4DDey1Vox+Q01lcw5sUhwH6g/M FW8Le3rmNcVusNEgCMrmr0ubWSHUsmVmlSbHmvakf7n4uj4VUZjxF+9dyAIWyrH48R9wPS8yj1ATR RiYqAzt6MqcVDdYbCUbExhG8TrhW8w5qzQAImLyzcT0pLHqaWYjJTh3Qo+xiDz1hmBXvGlfUGiOo6 YRfOxqIwQqSVCSGBCQKSXfgheNjx6LFXpEcqFUe1EE6o4j0/GMifpRx5ECePOZBJRLD05s8lac2df nVE0rWntKQV/rAdRrdLw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7M8-007iEv-GE; Fri, 18 Nov 2022 19:48:56 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Ji-007gyt-I5 for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:31 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 01AA162770; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A245FC43146; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800785; bh=HdiSoWnZUTCZpmQjT2Un6tm8QTRonixjlwqY064EQSg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nFUVKDQu84QOhgaSio7dWBuVl7bRghkrifp3HXo62eW9ILO3xqm2T04vi0Z4xxAJ7 n6eojRyj2Kk68HPINIAorrqNaF+5kyNJXdXq5zWcK/cVfG3q8EeGiqARm3T+RGqSXW hOCeOmqUXt+q0H2UwCwDwK2UdriOXz8O3Kfae56ZkzjoiC84Hufd/jpwvuGCmhdjD7 rXfWtNsg8XXpyo6wg+EWQs1otRvV5cN3WJw1fQxhIxmj5KcJPRIAD/+S6hhnRGPdZE CW2MCG72y4aDWei6QXJTrQ9rpq+Z9Jx5R67rz7LByV3Ttuq0i5YfVccY2x73D16fhx mQZgIX+FDE7MA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 06/12] crypto: x86/sha512 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:15 -0800 Message-Id: <20221118194421.160414-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_114626_732305_6BE17D2B X-CRM114-Status: GOOD ( 12.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sha512_transform_ssse3(), sha512_transform_avx(), and sha512_transform_rorx() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect calls). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/sha512-avx-asm.S | 3 ++- arch/x86/crypto/sha512-avx2-asm.S | 3 ++- arch/x86/crypto/sha512-ssse3-asm.S | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/arch/x86/crypto/sha512-avx-asm.S b/arch/x86/crypto/sha512-avx-asm.S index 1fefe6dd3a9e2..b0984f19fdb40 100644 --- a/arch/x86/crypto/sha512-avx-asm.S +++ b/arch/x86/crypto/sha512-avx-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include .text @@ -273,7 +274,7 @@ frame_size = frame_WK + WK_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks ######################################################################## -SYM_FUNC_START(sha512_transform_avx) +SYM_TYPED_FUNC_START(sha512_transform_avx) test msglen, msglen je nowork diff --git a/arch/x86/crypto/sha512-avx2-asm.S b/arch/x86/crypto/sha512-avx2-asm.S index 5cdaab7d69015..b1ca99055ef99 100644 --- a/arch/x86/crypto/sha512-avx2-asm.S +++ b/arch/x86/crypto/sha512-avx2-asm.S @@ -50,6 +50,7 @@ ######################################################################## #include +#include .text @@ -565,7 +566,7 @@ frame_size = frame_CTX + CTX_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks ######################################################################## -SYM_FUNC_START(sha512_transform_rorx) +SYM_TYPED_FUNC_START(sha512_transform_rorx) # Save GPRs push %rbx push %r12 diff --git a/arch/x86/crypto/sha512-ssse3-asm.S b/arch/x86/crypto/sha512-ssse3-asm.S index b84c22e06c5f7..c06afb5270e5f 100644 --- a/arch/x86/crypto/sha512-ssse3-asm.S +++ b/arch/x86/crypto/sha512-ssse3-asm.S @@ -48,6 +48,7 @@ ######################################################################## #include +#include .text @@ -274,7 +275,7 @@ frame_size = frame_WK + WK_SIZE # of SHA512 message blocks. # "blocks" is the message length in SHA512 blocks. ######################################################################## -SYM_FUNC_START(sha512_transform_ssse3) +SYM_TYPED_FUNC_START(sha512_transform_ssse3) test msglen, msglen je nowork From patchwork Fri Nov 18 19:44:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 856DCC433FE for ; Fri, 18 Nov 2022 19:50:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0k8tlB8xVe0DE5QIU0zNMhJjd9FUCG9ALuw5wlma9dM=; b=R2FRAh8XBJ08l4 YKi7mMXhuoYGXdh0wduaxzqf4vDmtkAyK0IvMBZJfBvngNm6GpruNVw5ALVY51f5qTuu4pld5dnYN yS0EPzIOm1WlaMWTBTxNe34EO3/brlWcnofLOhy9KmClUFGmVunPY1JfP+U3LxuSYoZGU2UGuVkph Cu1CWITsHCoJ9/bLMB0CvaAYa51MIaDffo/eUYwa+G/GtlYYAgTo9pyaPk9aKydsEjbs1Q4/JC6oe sQ33RXHPYzoGSfJ4NnnHQZ0SrNYzaRBwiOFyX1rQQQnJfIQlgjal41w/GpkBQRLNDPwXOoW0F2EKP nTMSUrLW6ttjCxXKkNhg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Ml-007iVQ-GA; Fri, 18 Nov 2022 19:49:35 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jl-007h10-In for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:33 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4AB0E6277C; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ED9CCC433D7; Fri, 18 Nov 2022 19:46:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800786; bh=kMW+VagvP+m/FfoCh7s6tC3GVUtFQvTJqaIKESW1UH4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JjeVGn8nldBI641WZBxQtxPLNepo8CQPS8YNSbUU43cGsQ3/2Ak+xN3DP1rAfDoEB JLzx4iADDlNubhwQTePqull+hV/RL/S6w1I6lt5C9kGxk5YI1DD4oAEW94eyE0hYLT ZAYE3smvPyy2nuJIh7wAUNwJk2etU9xN9S390jiSJC9A5VWDBKDWHdht9NHeSzRUl4 w5TVIhkvTvCg8a9vsG9SR7HKwc1MHeSpma80kkxCn7SKUgMAVp6OxkZLc33ifmiowD ySn88t/8XZEo950gVGmL5UkxFamcR0rGJA20hFtTe7n0DXetlJDzg8Lcv/bL3nFUd4 5iuZ5lz/Ji5Xg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 07/12] crypto: x86/sm3 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:16 -0800 Message-Id: <20221118194421.160414-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_114629_724294_876549F6 X-CRM114-Status: GOOD ( 13.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sm3_transform_avx() is called via indirect function calls. Therefore it needs to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause its type hash to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect call). Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/x86/crypto/sm3-avx-asm_64.S | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/crypto/sm3-avx-asm_64.S b/arch/x86/crypto/sm3-avx-asm_64.S index b12b9efb5ec51..8fc5ac681fd63 100644 --- a/arch/x86/crypto/sm3-avx-asm_64.S +++ b/arch/x86/crypto/sm3-avx-asm_64.S @@ -12,6 +12,7 @@ */ #include +#include #include /* Context structure */ @@ -328,7 +329,7 @@ * const u8 *data, int nblocks); */ .align 16 -SYM_FUNC_START(sm3_transform_avx) +SYM_TYPED_FUNC_START(sm3_transform_avx) /* input: * %rdi: ctx, CTX * %rsi: data (64*nblks bytes) From patchwork Fri Nov 18 19:44:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048744 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F28E1C433FE for ; Fri, 18 Nov 2022 19:55:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LFGD0hl1RBXYkMcL2cCf9MqAtUTIw4mgQRb5knEx3b0=; b=GvSAtCorhBGpL+ WcU/eD8QrTSw3u8To4wHWeGGer61lmdvs8jccFVjbBoUZY6iw52uQbbEkhiRWhipByLPOZUxOi/5i m+85ArnWds6UxMYkcbulz37CTHDxWKINAk54X5Q98VDZxJlf5UtA2EXMbUFAjUAS4O1jEbBPKl7xY NWbN4NqXf9+4YP5kUHJWXpjXpX3voWlfxcKjHcD8jMoXsqVNz8mZ1tq9WHKZ2oJVOnj5kRDPf3HjK NpaJ6T7JEIL2yjPm2isnVF4fy8PgRbnDeW07DhIo9bvC2rH7j69EBCqYLyymNan3vGYzw2yp5br5G SKTpJIgD9ESbz73/NyPw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7RV-007kvd-FW; Fri, 18 Nov 2022 19:54:30 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jx-007hAX-R1 for linux-arm-kernel@bombadil.infradead.org; Fri, 18 Nov 2022 19:46:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=MYuXhy7xG99eR5h2kvs0G5inumlroE9AT6YzGEKJ5ew=; b=nj8CE+QiDMOsUmG2j0TB/5zZL8 VyORbX2TdkUZpzHzGrxgewWeTC0IyH8Y9grmzjmj3nyq3TYZpSd1qgzsfdWv9vRpAjBfWSwXa729g T98pZmPmLKVx3HYJlSe9gGkqIHDfPUak80zszQEtX5XqOozRdLqEeW/N3c5WDyL7dfmu2tkG4Qtdn AuPbHjmdabT2cxIKzN/gYMJv9UvHI/mcpV4PS0pwwO1gcy6scg6/4qsGdjWkjTcpQr+NmudQ6ycCZ PodA8oaJLOkZeVTIzj6kuYn3DYK1QFEm2aXd8t+6NSFqfDL9kIPN3Amio2JX5tio2xkP2u1ynlVHZ b3Af/Umg==; Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jr-0025TP-QF for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:39 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9005D62781; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 44E88C433C1; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800786; bh=pclA3LA8eoFmKTrqSAJh4mYcWrvybcXjZP/RNOkyKcY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=baeL6vobI7CZ5qsql2NjsgVd2NX3vsNjyXPqOkD9r/8ri1L43BvyGRYwqJ0GWLo8V bzIXh8dC3LswOmK2RxnFmlzWXc29fOD0mpjm+gOuALMNvDUI2SQi5Oxxk5uVnLtZbw o0W7thLe8AvO/Qwq36p6xejmKTQUFOlSj3ILSaaV3D3tYASC9TGKcG96t85shGigaL NQBOjzO2gixywvwmeeKbGmmQauoK1asj8ojR3LvtTspf3acolK5b+QljHMGT1nVNl2 3IJ7fMnrIsjJRzvB3MMY0vl8n4OZ7IVPQ5HVQUOsR+OSNbeKihX1YRizy/38SbXyJf V2ZCXa5Bu7EMQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen Subject: [PATCH v2 08/12] crypto: x86/sm4 - fix crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:17 -0800 Message-Id: <20221118194421.160414-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_194636_295484_A6DD4489 X-CRM114-Status: GOOD ( 13.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sm4_aesni_avx_ctr_enc_blk8(), sm4_aesni_avx_cbc_dec_blk8(), sm4_aesni_avx_cfb_dec_blk8(), sm4_aesni_avx2_ctr_enc_blk16(), sm4_aesni_avx2_cbc_dec_blk16(), and sm4_aesni_avx2_cfb_dec_blk16() are called via indirect function calls. Therefore they need to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause their type hashes to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure. (Or at least that should be the case. For some reason the CFI checks in sm4_avx_cbc_decrypt(), sm4_avx_cfb_decrypt(), and sm4_avx_ctr_crypt() are not always being generated, using current tip-of-tree clang. Anyway, this patch is a good idea anyway.) Fixes: ccace936eec7 ("x86: Add types to indirectly called assembly functions") Signed-off-by: Eric Biggers --- arch/x86/crypto/sm4-aesni-avx-asm_64.S | 7 ++++--- arch/x86/crypto/sm4-aesni-avx2-asm_64.S | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/arch/x86/crypto/sm4-aesni-avx-asm_64.S b/arch/x86/crypto/sm4-aesni-avx-asm_64.S index 4767ab61ff489..22b6560eb9e1e 100644 --- a/arch/x86/crypto/sm4-aesni-avx-asm_64.S +++ b/arch/x86/crypto/sm4-aesni-avx-asm_64.S @@ -14,6 +14,7 @@ */ #include +#include #include #define rRIP (%rip) @@ -420,7 +421,7 @@ SYM_FUNC_END(sm4_aesni_avx_crypt8) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx_ctr_enc_blk8) +SYM_TYPED_FUNC_START(sm4_aesni_avx_ctr_enc_blk8) /* input: * %rdi: round key array, CTX * %rsi: dst (8 blocks) @@ -495,7 +496,7 @@ SYM_FUNC_END(sm4_aesni_avx_ctr_enc_blk8) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx_cbc_dec_blk8) +SYM_TYPED_FUNC_START(sm4_aesni_avx_cbc_dec_blk8) /* input: * %rdi: round key array, CTX * %rsi: dst (8 blocks) @@ -545,7 +546,7 @@ SYM_FUNC_END(sm4_aesni_avx_cbc_dec_blk8) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx_cfb_dec_blk8) +SYM_TYPED_FUNC_START(sm4_aesni_avx_cfb_dec_blk8) /* input: * %rdi: round key array, CTX * %rsi: dst (8 blocks) diff --git a/arch/x86/crypto/sm4-aesni-avx2-asm_64.S b/arch/x86/crypto/sm4-aesni-avx2-asm_64.S index 4732fe8bb65b6..23ee39a8ada8c 100644 --- a/arch/x86/crypto/sm4-aesni-avx2-asm_64.S +++ b/arch/x86/crypto/sm4-aesni-avx2-asm_64.S @@ -14,6 +14,7 @@ */ #include +#include #include #define rRIP (%rip) @@ -282,7 +283,7 @@ SYM_FUNC_END(__sm4_crypt_blk16) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx2_ctr_enc_blk16) +SYM_TYPED_FUNC_START(sm4_aesni_avx2_ctr_enc_blk16) /* input: * %rdi: round key array, CTX * %rsi: dst (16 blocks) @@ -395,7 +396,7 @@ SYM_FUNC_END(sm4_aesni_avx2_ctr_enc_blk16) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx2_cbc_dec_blk16) +SYM_TYPED_FUNC_START(sm4_aesni_avx2_cbc_dec_blk16) /* input: * %rdi: round key array, CTX * %rsi: dst (16 blocks) @@ -449,7 +450,7 @@ SYM_FUNC_END(sm4_aesni_avx2_cbc_dec_blk16) * const u8 *src, u8 *iv) */ .align 8 -SYM_FUNC_START(sm4_aesni_avx2_cfb_dec_blk16) +SYM_TYPED_FUNC_START(sm4_aesni_avx2_cfb_dec_blk16) /* input: * %rdi: round key array, CTX * %rsi: dst (16 blocks) From patchwork Fri Nov 18 19:44:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8F102C4332F for ; Fri, 18 Nov 2022 19:56:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GqBFeMiK9vgR5QHkSYEiwMoGTqbgjpX+r6g3QbqQe8M=; b=dwcFkM3g+SLfwD rC76xHQkjd8ea0jMih+DV+Jlv/LuUdkCRROmO8v6KFCY0CWUcDDNiRc0KNRxoGZSkPpD0Qfn4/O2w 36kOVC5Hbr5nHBrPTRNdmps+3jkUac0uTTiII3k4hj7lv/F0g606CcrZITbY/dnkdHQuw34oiCcNW pK8WqskJ8xwOZv50H68P97OkxkDa4qu7NTTGFcKHTUbgOKpSWx9DHFeTHIj6i7KMk3VmFRiTfdjYJ 8HeSSkCQBQ8MVYAZP694MAeosy+m2RVIyt2FajkAqIan3Sx7iYa945I3Oz8XXwreCvzxDszzy1MJL fk1xT2GALmBV+NcsXIMQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7S7-007lEW-E3; Fri, 18 Nov 2022 19:55:07 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jx-007hAa-RA for linux-arm-kernel@bombadil.infradead.org; Fri, 18 Nov 2022 19:46:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Y8O+K/N6pvkZrlyn1gY8/RIBiTTqwQtsPlYaJ0T/FyY=; b=Lt4kfNqDIGHhNP8mYWQ4VaD/EQ yswXq6rUlPzCR1JTUj7kupxPEU10uLJxVeL2jbdgGfI3d+J69Gs4ZBtzU8BqqZLmShLRUdEZ5d2OQ IEBKWM0+x82KRY9HINm6sFPIIhSbf4GB3W2EP7Ccf1LbRG1MKihgKKL87eHHp7vadza5tGnNfJwpp S0+DK3ATUCdT2DWURVKH/Tuo2CP5tr+DkjxYCaPIRUbSM9l9ice1YcZMsLUq0XINsrVMeWzmFZqW3 5Rsk6FqVwP1UWsKHGPQcP3pGOhwwJhNIEL7Eyer3zbXTdxURoTJDUXgKZaREgcx1GrmX06WHYun6z G2iqDSwA==; Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jr-0025TT-QH for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:39 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 49FE96271F; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89731C433D6; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800786; bh=X+7N1v7Ylhc12ODIGzGIdWud9IGuRB4PQ36hvFoPOac=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rtCymIaJdlKGgicPDjzScEGeZBPbUtIgcMmx+JdSB+XgIeGV2D9NzjsYFOZBKJxbC mTmt6DKomct3OOuoZhJivhmhqsmBxhXZlaGdj6nyQ9/vOcZHlV6EybyfhnEphUV45W UzW60U0Jo4lFOKdgseR19Ex2eWkHZscbJJdsANorpstc7QhK4MzZN5bqV6KeZTNCmC 4fFMhbXOX7C4u9+jxk4WeZ+y/lJ9fDhD76a4ZhGDLUk5uRVAmKB5ngqS1dTOHcsPeN 8hG6t1IxUJsakY7i/b20aE0Q8HjbaCGCpMcAX0zH/XZxEmcE0EisvfQcxC38pCPBlZ B1Q5hcLouLkaA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 09/12] crypto: arm64/nhpoly1305 - eliminate unnecessary CFI wrapper Date: Fri, 18 Nov 2022 11:44:18 -0800 Message-Id: <20221118194421.160414-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_194636_102052_6C02198A X-CRM114-Status: GOOD ( 12.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers Since the CFI implementation now supports indirect calls to assembly functions, take advantage of that rather than use a wrapper function. Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/arm64/crypto/nh-neon-core.S | 5 +++-- arch/arm64/crypto/nhpoly1305-neon-glue.c | 11 ++--------- 2 files changed, 5 insertions(+), 11 deletions(-) diff --git a/arch/arm64/crypto/nh-neon-core.S b/arch/arm64/crypto/nh-neon-core.S index 51c0a534ef87c..13eda08fda1e5 100644 --- a/arch/arm64/crypto/nh-neon-core.S +++ b/arch/arm64/crypto/nh-neon-core.S @@ -8,6 +8,7 @@ */ #include +#include KEY .req x0 MESSAGE .req x1 @@ -58,11 +59,11 @@ /* * void nh_neon(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ -SYM_FUNC_START(nh_neon) +SYM_TYPED_FUNC_START(nh_neon) ld1 {K0.4s,K1.4s}, [KEY], #32 movi PASS0_SUMS.2d, #0 diff --git a/arch/arm64/crypto/nhpoly1305-neon-glue.c b/arch/arm64/crypto/nhpoly1305-neon-glue.c index c5405e6a6db76..cd882c35d9252 100644 --- a/arch/arm64/crypto/nhpoly1305-neon-glue.c +++ b/arch/arm64/crypto/nhpoly1305-neon-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_neon(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_neon(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_neon(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_neon_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_neon_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_neon_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_neon); + crypto_nhpoly1305_update_helper(desc, src, n, nh_neon); kernel_neon_end(); src += n; srclen -= n; From patchwork Fri Nov 18 19:44:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1342EC433FE for ; Fri, 18 Nov 2022 19:51:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HS1ZzeL3ZNgfOCLLBurBNdXlp+/BfQaPyC/SqQqSaLY=; b=cfjXS4AhhRX4KC 51WYfh4ROCeW2quhC7lcT7regysK3543SElhX+pL/9Fn+X7gAXK0xFpdJ9Ti0x8W/QWQai2UMKvLL vmyVbZuGmdahC8IYGi9T+AvIsb6fio9fOmE/yIhbZ1SLP3w24UWTd5qDwQ2fj/TBYdbFc6t47UbvF pCsv7YPRVQbM5yjesL0fZ1Mwh86TRKnCo2mMmiyXbYuvlLf+GCSF1AHvK/I6/RFPdu2bmwAnRN4pL vnDVS4b+lNYlorWz78jdD7+0rC8+B3wIdmyK5FYMM15mYT6wsKkBzt0cjvptMCWd5F+5z8vm/Gt1w NuwBtK41sjISAn3odTGA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7NV-007ir8-Vc; Fri, 18 Nov 2022 19:50:22 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jn-007h3g-Mz for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:35 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 94BE862785; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D64BCC433B5; Fri, 18 Nov 2022 19:46:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800787; bh=502XtfzzJCvn41Ye4mf8jH8Z9XzhNMTzAZtoz+ow1Mk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SvwBQUsI23XQ04sTI+nMVJ/WdNE5SRFV0xg5sQjzZmEWJ1B+c7rFPuO1rZ6ZiApwe oN0DQyrJnBsRXkaoO9X8r3laLNge0EW+fgfc6La6YGGQKvYuLZobjj3n8AkCMhC8mf gOpZJOhwNEVzfBoL7N5xSXxS6HEVgx6G/y5WmkR3LGvn79IeLRXprShuZwm2R4cnRB G8PqvC13bsWK+9I4PZkHV/TLQhlxmBxO7JVrraQ7AwrR5n2+PHMp5YwEAsTxy55WQw VT6e73YReQNO5UW1oa5/5CTixIuDmDQIHWoGgQiiMGWEe7EVzdvKrUIl6Kw8W45yry I5s/S5nTK7/1g== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 10/12] crypto: arm64/sm3 - fix possible crash with CFI enabled Date: Fri, 18 Nov 2022 11:44:19 -0800 Message-Id: <20221118194421.160414-11-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_114631_850853_BB6B9106 X-CRM114-Status: GOOD ( 12.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers sm3_neon_transform() is called via indirect function calls. Therefore it needs to use SYM_TYPED_FUNC_START instead of SYM_FUNC_START to cause its type hash to be emitted when the kernel is built with CONFIG_CFI_CLANG=y. Otherwise, the code crashes with a CFI failure (if the compiler didn't happen to optimize out the indirect call). Fixes: c50d32859e70 ("arm64: Add types to indirect called assembly functions") Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/arm64/crypto/sm3-neon-core.S | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/crypto/sm3-neon-core.S b/arch/arm64/crypto/sm3-neon-core.S index 3e3b4e5c736fc..4357e0e51be38 100644 --- a/arch/arm64/crypto/sm3-neon-core.S +++ b/arch/arm64/crypto/sm3-neon-core.S @@ -9,6 +9,7 @@ */ #include +#include #include /* Context structure */ @@ -351,7 +352,7 @@ */ .text .align 3 -SYM_FUNC_START(sm3_neon_transform) +SYM_TYPED_FUNC_START(sm3_neon_transform) ldp ra, rb, [RSTATE, #0] ldp rc, rd, [RSTATE, #8] ldp re, rf, [RSTATE, #16] From patchwork Fri Nov 18 19:44:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2677C433FE for ; Fri, 18 Nov 2022 19:52:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tYADT9ZJr2eqIsp2BKYQVsDQLkm4ydJMtJHglq8faTM=; b=NAU/xSptF5cZjg 5aGnYz/MkzQ22NN1lsMpwYuOYUyVfplOujFLmXwPoWh/fqAPZ0B8ygj669WKphY6uIPPpYAD8oBLD PSETgRXxtoLlKRV5nafx9Pp4r8Yz6Dbt0+ciEtnE2sUCJhm48mYMOaIWptPaA5dAQ19NU/QalAJa7 L7QVmu7dVvEnc9m12L5uTD4ofcTirjWVAxdFgoWLW+bT8Zh9j9KWnF8JlWULx+WUAJLpbg3ELRW88 bmY32WRvHc1Pc0t6J0WlkOVkL/vdoYjoLuRWN/Z4TlnAiozWDUJLXB2vpxCMnC9VzPwiwn2O4arOF Ch2s/f/XHgmOJyfUGNqA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7O9-007jB8-PS; Fri, 18 Nov 2022 19:51:02 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jm-007h2x-J5 for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:35 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7EF0C62788; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2DA40C43470; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800787; bh=Jktv+/Qu/uCCZJ2itmlKVBoDqxZ3t6fI03aW0+AnaOA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g0nHEAK6+gGwwvbGug2zU5o5c9AvD4J8LVD7A5I0bsJep+SZ+nLXLmWdCKLTHuToc gmz0GC4mEPhWbunktEF3A4pKjjUaxa0E/Ypzs2v+vJB/T8SaqYh57y1W3wFN/X7Nd/ 61MPbQJE/3yZUASVFukPNYvg50koFQy7Ied7Wye9UX5rUAA1rhBF25YYEi61gj/SId WoUsuILKqmHaT80VGqyHIjdSrLH/sZwobE6dKDuEZ18bLaex8VOv/Dl8HK+jZcRUYa nbR24R9KPBk8CLjzFvEvGg2CcDfQ26figvmVvtvbxwDp1yykkePPEj6IejOjWr75LD 8jqOfsYpb6JgQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 11/12] crypto: arm/nhpoly1305 - eliminate unnecessary CFI wrapper Date: Fri, 18 Nov 2022 11:44:20 -0800 Message-Id: <20221118194421.160414-12-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_114630_710681_AA973523 X-CRM114-Status: GOOD ( 13.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers The arm architecture doesn't support CFI yet, and even if it did, the new CFI implementation supports indirect calls to assembly functions. Therefore, there's no need to use a wrapper function for nh_neon(). Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- arch/arm/crypto/nh-neon-core.S | 2 +- arch/arm/crypto/nhpoly1305-neon-glue.c | 11 ++--------- 2 files changed, 3 insertions(+), 10 deletions(-) diff --git a/arch/arm/crypto/nh-neon-core.S b/arch/arm/crypto/nh-neon-core.S index 434d80ab531c2..01620a0782ca9 100644 --- a/arch/arm/crypto/nh-neon-core.S +++ b/arch/arm/crypto/nh-neon-core.S @@ -69,7 +69,7 @@ /* * void nh_neon(const u32 *key, const u8 *message, size_t message_len, - * u8 hash[NH_HASH_BYTES]) + * __le64 hash[NH_NUM_PASSES]) * * It's guaranteed that message_len % 16 == 0. */ diff --git a/arch/arm/crypto/nhpoly1305-neon-glue.c b/arch/arm/crypto/nhpoly1305-neon-glue.c index ffa8d73fe722c..e93e41ff26566 100644 --- a/arch/arm/crypto/nhpoly1305-neon-glue.c +++ b/arch/arm/crypto/nhpoly1305-neon-glue.c @@ -14,14 +14,7 @@ #include asmlinkage void nh_neon(const u32 *key, const u8 *message, size_t message_len, - u8 hash[NH_HASH_BYTES]); - -/* wrapper to avoid indirect call to assembly, which doesn't work with CFI */ -static void _nh_neon(const u32 *key, const u8 *message, size_t message_len, - __le64 hash[NH_NUM_PASSES]) -{ - nh_neon(key, message, message_len, (u8 *)hash); -} + __le64 hash[NH_NUM_PASSES]); static int nhpoly1305_neon_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) @@ -33,7 +26,7 @@ static int nhpoly1305_neon_update(struct shash_desc *desc, unsigned int n = min_t(unsigned int, srclen, SZ_4K); kernel_neon_begin(); - crypto_nhpoly1305_update_helper(desc, src, n, _nh_neon); + crypto_nhpoly1305_update_helper(desc, src, n, nh_neon); kernel_neon_end(); src += n; srclen -= n; From patchwork Fri Nov 18 19:44:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13048743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 77DEFC433FE for ; Fri, 18 Nov 2022 19:55:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wIxqV4Hse0OdhXw1tEJRtR3mogFXtwpdoZtqyQe4MaI=; b=Fggb5Pz5zdyeh5 TczgDOPAS9LgnWIIvY8SeOdsfFTjLgOAxuruKiOzBdOP8qQ1eFgYBSPGU++3SlxzITGbdeIUZDwef uWmiWgDiIzO1Jn1y1KBn95rq4rqhH8ryLKMuNwlhP0jY97SQPIx04U1RiJpE4sFT2xBI5At7W/evu p17DY0VrfcSwO2/pjh2KZJEw/BC2yL6FGTGOvHdeqCGbsg9rUx5VVFx5ilSycd0Wz8Ox3PepIl2E4 z8gzqsBf+Bmb8wg4UuA/ohg6dwwRo6bOarTorbMyc/B7Xej8brP9SL/UKvBfBEKK+Mi8y/dCrVJvk +oi8tIeO7Ibxv4ZYAwXQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Qw-007kfF-P9; Fri, 18 Nov 2022 19:53:55 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jx-007hAY-RC for linux-arm-kernel@bombadil.infradead.org; Fri, 18 Nov 2022 19:46:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=gh7914dEAA48kBeWLtVtjW7nNK6pvGJ2qkWHXI6JaNY=; b=Hr5LoFoj7nNyH02MsSl/wT/wqc 1Djr/gXCb6YBnhIw+ez05Kwxwc7iPHgNe8mNrll0PY9n4AQb8dGyThMQ7EQGOIZZLepkqasjiTmjn QCUXphbSW+Zjm7doyUnIEwWsR2RFqMeUR8Iv1NyvG/KGQzh9o7f4SnzIUC1hmbg9/L7E3rzYaOmOp PppyPSzqoSNmBTaH+C8VQ/tqxF+nZQqZ9nx3d7uS5wFpnmgaS0+vN1NBTYc1TYLW06Pu9UHx2j0Ms 4M97dO53bU8zMUYM29/7CIpzhh8mejLxiQg9aMkt1i9+pZC6cj8oWPmqsyj1Q6aI31rPzp4cERflR qa5AdKhA==; Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ow7Jr-0025TV-QE for linux-arm-kernel@lists.infradead.org; Fri, 18 Nov 2022 19:46:39 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CB67062790; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79032C43140; Fri, 18 Nov 2022 19:46:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668800787; bh=kEiHXZili2H4efP5Zbt6QZIEDbUdz7NL5DjoHnDhWac=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ErSsA5S03M6uPJgYtoNYzpyGRsncDw5v9YTnjC2g1yf8Gtq/ce6QqMvNPysmNpyuo bYGdjDIE6a9BnUfpjq+NZRkN3g3HvNK9pXmR3dOf+OfKjs6qRu3O8UdW0Qu5ehCCHk ZLkprHysfZPvjQciMmGKDcwBqIeAyDMP0dBd+STirNqJjmpwjP39X1Mh+m/vvc+lYC NYU/FGL53zTFGOHqK/q7sxgZBkT80W3rnKbQ2ZTt0Kch3xciVjXRj5q2iPBMMJ6Muv RlkYp4IMfdIwuH2h5eNrdWftJWHyPRJDn2d+8xl6VxgpnLC1yB3C5xE209TQwl5ghF 1kjnVVuXiMbzw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, Sami Tolvanen , "Peter Zijlstra (Intel)" Subject: [PATCH v2 12/12] Revert "crypto: shash - avoid comparing pointers to exported functions under CFI" Date: Fri, 18 Nov 2022 11:44:21 -0800 Message-Id: <20221118194421.160414-13-ebiggers@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118194421.160414-1-ebiggers@kernel.org> References: <20221118194421.160414-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221118_194636_292321_8D53CF00 X-CRM114-Status: GOOD ( 14.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Eric Biggers This reverts commit 22ca9f4aaf431a9413dcc115dd590123307f274f because CFI no longer breaks cross-module function address equality, so crypto_shash_alg_has_setkey() can now be an inline function like before. This commit should not be backported to kernels that don't have the new CFI implementation. Acked-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Signed-off-by: Eric Biggers --- crypto/shash.c | 18 +++--------------- include/crypto/internal/hash.h | 8 +++++++- 2 files changed, 10 insertions(+), 16 deletions(-) diff --git a/crypto/shash.c b/crypto/shash.c index 4c88e63b3350f..0f85431588267 100644 --- a/crypto/shash.c +++ b/crypto/shash.c @@ -20,24 +20,12 @@ static const struct crypto_type crypto_shash_type; -static int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, - unsigned int keylen) +int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, + unsigned int keylen) { return -ENOSYS; } - -/* - * Check whether an shash algorithm has a setkey function. - * - * For CFI compatibility, this must not be an inline function. This is because - * when CFI is enabled, modules won't get the same address for shash_no_setkey - * (if it were exported, which inlining would require) as the core kernel will. - */ -bool crypto_shash_alg_has_setkey(struct shash_alg *alg) -{ - return alg->setkey != shash_no_setkey; -} -EXPORT_SYMBOL_GPL(crypto_shash_alg_has_setkey); +EXPORT_SYMBOL_GPL(shash_no_setkey); static int shash_setkey_unaligned(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) diff --git a/include/crypto/internal/hash.h b/include/crypto/internal/hash.h index 25806141db591..0a288dddcf5be 100644 --- a/include/crypto/internal/hash.h +++ b/include/crypto/internal/hash.h @@ -75,7 +75,13 @@ void crypto_unregister_ahashes(struct ahash_alg *algs, int count); int ahash_register_instance(struct crypto_template *tmpl, struct ahash_instance *inst); -bool crypto_shash_alg_has_setkey(struct shash_alg *alg); +int shash_no_setkey(struct crypto_shash *tfm, const u8 *key, + unsigned int keylen); + +static inline bool crypto_shash_alg_has_setkey(struct shash_alg *alg) +{ + return alg->setkey != shash_no_setkey; +} static inline bool crypto_shash_alg_needs_key(struct shash_alg *alg) {