From patchwork Wed Feb 24 16:29:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102103 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8157C433DB for ; Wed, 24 Feb 2021 16:32:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7327F64F00 for ; Wed, 24 Feb 2021 16:32:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235085AbhBXQcL (ORCPT ); Wed, 24 Feb 2021 11:32:11 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:35307 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234450AbhBXQbF (ORCPT ); Wed, 24 Feb 2021 11:31:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184177; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CaIS24TekHR2zlXKqWJ9Ayyago1O42H+gYPmc+HqgA8=; b=S7FOgFpVwsfxQg0M9Aw4WaWEEKxps2eApXqf/ORUUkQnZOAoATB/xriP6oQhd9BB5TBUkQ VYH3tegLfdE1Qg216F0gaVWPCfEGIJHrL2lARHS1dN7sFqyNgOPoXv2XfkkEMbA9UUaBb/ tw5QnpM8yyKiWsNExKLzl0QSwJZRDsE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-284-XwQRX8d2P0avaRtWUQawiA-1; Wed, 24 Feb 2021 11:29:35 -0500 X-MC-Unique: XwQRX8d2P0avaRtWUQawiA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 83531803F47; Wed, 24 Feb 2021 16:29:33 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 49D7260862; Wed, 24 Feb 2021 16:29:32 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 01/13] objtool: Support asm jump tables Date: Wed, 24 Feb 2021 10:29:14 -0600 Message-Id: <460cf4dc675d64e1124146562cabd2c05aa322e8.1614182415.git.jpoimboe@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Objtool detection of asm jump tables would normally just work, except for the fact that asm retpolines use alternatives. Objtool thinks the alternative code path (a jump to the retpoline) is a sibling call. Don't treat alternative indirect branches as sibling calls when the original instruction has a jump table. Signed-off-by: Josh Poimboeuf --- tools/objtool/check.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 080a3d6cbd75..f95e406296f5 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -108,6 +108,18 @@ static struct instruction *prev_insn_same_sym(struct objtool_file *file, for (insn = next_insn_same_sec(file, insn); insn; \ insn = next_insn_same_sec(file, insn)) +static bool is_jump_table_jump(struct instruction *insn) +{ + struct alt_group *alt_group = insn->alt_group; + + if (insn->jump_table) + return true; + + /* Retpoline alternative for a jump table? */ + return alt_group && alt_group->orig_group && + alt_group->orig_group->first_insn->jump_table; +} + static bool is_sibling_call(struct instruction *insn) { /* @@ -120,7 +132,7 @@ static bool is_sibling_call(struct instruction *insn) /* An indirect jump is either a sibling call or a jump to a table. */ if (insn->type == INSN_JUMP_DYNAMIC) - return list_empty(&insn->alts); + return !is_jump_table_jump(insn); /* add_jump_destinations() sets insn->call_dest for sibling calls. */ return (is_static_jump(insn) && insn->call_dest); From patchwork Wed Feb 24 16:29:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102111 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1382FC43381 for ; Wed, 24 Feb 2021 16:33:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B87BB64EBA for ; Wed, 24 Feb 2021 16:33:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234708AbhBXQc5 (ORCPT ); Wed, 24 Feb 2021 11:32:57 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58889 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234658AbhBXQbU (ORCPT ); Wed, 24 Feb 2021 11:31:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184181; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wjYqxo2PcGDIUf3rOxDJ05fdBFc1pa3drtiPmkHIL9g=; b=SoCg0Yozd/7wfiNOtqRmh6nTQxpB/C60p/Z/cuyUKZonAGbEgAy9ZC5QoP1MJ9xCDli9Zf v6s+8DaKQtaILQR70E6ooM/i+MajTSVVy9RSF6SLq7y6X0NapGWWJfOi7HEffMxVSgJCUO dVWuAtY3k62HcUMnmq8mk9DIIO8GnhE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-241-wspEER2kNEKvO8bC-tS1ew-1; Wed, 24 Feb 2021 11:29:37 -0500 X-MC-Unique: wspEER2kNEKvO8bC-tS1ew-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EEBFE79EC7; Wed, 24 Feb 2021 16:29:34 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id B0F7D60862; Wed, 24 Feb 2021 16:29:33 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 02/13] x86/crypto/aesni-intel_avx: Remove unused macros Date: Wed, 24 Feb 2021 10:29:15 -0600 Message-Id: <53f7136ea93ebdbca399959e6d2991ecb46e733e.1614182415.git.jpoimboe@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org These macros are no longer used; remove them. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/aesni-intel_avx-x86_64.S | 8 -------- 1 file changed, 8 deletions(-) diff --git a/arch/x86/crypto/aesni-intel_avx-x86_64.S b/arch/x86/crypto/aesni-intel_avx-x86_64.S index 2cf8e94d986a..4fdf38e92d51 100644 --- a/arch/x86/crypto/aesni-intel_avx-x86_64.S +++ b/arch/x86/crypto/aesni-intel_avx-x86_64.S @@ -212,10 +212,6 @@ HashKey_8_k = 16*21 # store XOR of HashKey^8 <<1 mod poly here (for Karatsu #define arg4 %rcx #define arg5 %r8 #define arg6 %r9 -#define arg7 STACK_OFFSET+8*1(%r14) -#define arg8 STACK_OFFSET+8*2(%r14) -#define arg9 STACK_OFFSET+8*3(%r14) -#define arg10 STACK_OFFSET+8*4(%r14) #define keysize 2*15*16(arg1) i = 0 @@ -237,9 +233,6 @@ define_reg j %j .noaltmacro .endm -# need to push 4 registers into stack to maintain -STACK_OFFSET = 8*4 - TMP1 = 16*0 # Temporary storage for AAD TMP2 = 16*1 # Temporary storage for AES State 2 (State 1 is stored in an XMM register) TMP3 = 16*2 # Temporary storage for AES State 3 @@ -256,7 +249,6 @@ VARIABLE_OFFSET = 16*8 ################################ .macro FUNC_SAVE - #the number of pushes must equal STACK_OFFSET push %r12 push %r13 push %r14 From patchwork Wed Feb 24 16:29:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102113 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32BB7C4332B for ; Wed, 24 Feb 2021 16:33:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E0B4764EF1 for ; Wed, 24 Feb 2021 16:33:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234658AbhBXQdA (ORCPT ); Wed, 24 Feb 2021 11:33:00 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:52682 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234670AbhBXQbU (ORCPT ); Wed, 24 Feb 2021 11:31:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184182; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EoUyH4tB9klRBpxMdtA+3ioOLqqMzutoUlyl1zVvLUo=; b=FwHdKS8qAz0NdBiPfjhY0KbbLsiy3QLibsvAP2VSYQVSNshBA/O0F9MnldGRJD7McszHng j1MkNzxzOUTDiiUp1OlGUMi02fJ5BzHv/60BjMVUhZe66/i/490eJlyM6mmPX5RfNwYqnU z3vKdX0kXaINj0tSXupJlCFx+F7VcKE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-166-zYRW8OXmOjWnjx0U8NBX4w-1; Wed, 24 Feb 2021 11:29:38 -0500 X-MC-Unique: zYRW8OXmOjWnjx0U8NBX4w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 655BF801965; Wed, 24 Feb 2021 16:29:36 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2877960862; Wed, 24 Feb 2021 16:29:35 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 03/13] x86/crypto/aesni-intel_avx: Fix register usage comments Date: Wed, 24 Feb 2021 10:29:16 -0600 Message-Id: <8655d4513a0ed1eddec609165064153973010aa2.1614182415.git.jpoimboe@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Fix register usage comments to match reality. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/aesni-intel_avx-x86_64.S | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/crypto/aesni-intel_avx-x86_64.S b/arch/x86/crypto/aesni-intel_avx-x86_64.S index 4fdf38e92d51..188f1848a730 100644 --- a/arch/x86/crypto/aesni-intel_avx-x86_64.S +++ b/arch/x86/crypto/aesni-intel_avx-x86_64.S @@ -286,7 +286,7 @@ VARIABLE_OFFSET = 16*8 # combined for GCM encrypt and decrypt functions # clobbering all xmm registers -# clobbering r10, r11, r12, r13, r14, r15 +# clobbering r10, r11, r12, r13, r15, rax .macro GCM_ENC_DEC INITIAL_BLOCKS GHASH_8_ENCRYPT_8_PARALLEL GHASH_LAST_8 GHASH_MUL ENC_DEC REP vmovdqu AadHash(arg2), %xmm8 vmovdqu HashKey(arg2), %xmm13 # xmm13 = HashKey @@ -988,7 +988,7 @@ _partial_block_done_\@: ## num_initial_blocks = b mod 4# ## encrypt the initial num_initial_blocks blocks and apply ghash on the ciphertext ## r10, r11, r12, rax are clobbered -## arg1, arg3, arg4, r14 are used as a pointer only, not modified +## arg1, arg2, arg3, arg4 are used as pointers only, not modified .macro INITIAL_BLOCKS_AVX REP num_initial_blocks T1 T2 T3 T4 T5 CTR XMM1 XMM2 XMM3 XMM4 XMM5 XMM6 XMM7 XMM8 T6 T_key ENC_DEC i = (8-\num_initial_blocks) @@ -1223,7 +1223,7 @@ _initial_blocks_done\@: # encrypt 8 blocks at a time # ghash the 8 previously encrypted ciphertext blocks -# arg1, arg3, arg4 are used as pointers only, not modified +# arg1, arg2, arg3, arg4 are used as pointers only, not modified # r11 is the data offset value .macro GHASH_8_ENCRYPT_8_PARALLEL_AVX REP T1 T2 T3 T4 T5 T6 CTR XMM1 XMM2 XMM3 XMM4 XMM5 XMM6 XMM7 XMM8 T7 loop_idx ENC_DEC @@ -1936,7 +1936,7 @@ SYM_FUNC_END(aesni_gcm_finalize_avx_gen2) ## num_initial_blocks = b mod 4# ## encrypt the initial num_initial_blocks blocks and apply ghash on the ciphertext ## r10, r11, r12, rax are clobbered -## arg1, arg3, arg4, r14 are used as a pointer only, not modified +## arg1, arg2, arg3, arg4 are used as pointers only, not modified .macro INITIAL_BLOCKS_AVX2 REP num_initial_blocks T1 T2 T3 T4 T5 CTR XMM1 XMM2 XMM3 XMM4 XMM5 XMM6 XMM7 XMM8 T6 T_key ENC_DEC VER i = (8-\num_initial_blocks) @@ -2178,7 +2178,7 @@ _initial_blocks_done\@: # encrypt 8 blocks at a time # ghash the 8 previously encrypted ciphertext blocks -# arg1, arg3, arg4 are used as pointers only, not modified +# arg1, arg2, arg3, arg4 are used as pointers only, not modified # r11 is the data offset value .macro GHASH_8_ENCRYPT_8_PARALLEL_AVX2 REP T1 T2 T3 T4 T5 T6 CTR XMM1 XMM2 XMM3 XMM4 XMM5 XMM6 XMM7 XMM8 T7 loop_idx ENC_DEC From patchwork Wed Feb 24 16:29:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102107 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3FDCC433DB for ; Wed, 24 Feb 2021 16:33:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 41AAF64EBA for ; Wed, 24 Feb 2021 16:33:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231895AbhBXQcu (ORCPT ); Wed, 24 Feb 2021 11:32:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:26475 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234689AbhBXQbT (ORCPT ); Wed, 24 Feb 2021 11:31:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184183; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hGOdN7tSFAaU3AO1CqR7TyZfZzGj2VWJcbMfTD3exzQ=; b=IjYu2dXzv3kXsgoEt+zZ2pO1ecg7E9N7X/2Q6tlnRi8cjXgmjO5a00qzRhsZbNzf7EkJ8T /lcO63sLOqNgge3nPzaOJbaM3o1KKrOOCxvJ/DLGQnCQpO+V9/G8P5oU2D6bqnt6LVLD/a WTCniB0VuDsSv8sVqDsDrxrrvoiRB+I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-197-cRY8inFFN_OWfJ0gTvgbRw-1; Wed, 24 Feb 2021 11:29:40 -0500 X-MC-Unique: cRY8inFFN_OWfJ0gTvgbRw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0C42E1936B66; Wed, 24 Feb 2021 16:29:38 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8FCF360C5B; Wed, 24 Feb 2021 16:29:36 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 04/13] x86/crypto/aesni-intel_avx: Standardize stack alignment prologue Date: Wed, 24 Feb 2021 10:29:17 -0600 Message-Id: <02d00a0903a0959f4787e186e2a07d271e1f63d4.1614182415.git.jpoimboe@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Use RBP instead of R14 for saving the old stack pointer before realignment. This resembles what compilers normally do. This enables ORC unwinding by allowing objtool to understand the stack realignment. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/aesni-intel_avx-x86_64.S | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/crypto/aesni-intel_avx-x86_64.S b/arch/x86/crypto/aesni-intel_avx-x86_64.S index 188f1848a730..98e3552b6e03 100644 --- a/arch/x86/crypto/aesni-intel_avx-x86_64.S +++ b/arch/x86/crypto/aesni-intel_avx-x86_64.S @@ -251,22 +251,20 @@ VARIABLE_OFFSET = 16*8 .macro FUNC_SAVE push %r12 push %r13 - push %r14 push %r15 - mov %rsp, %r14 - - + push %rbp + mov %rsp, %rbp sub $VARIABLE_OFFSET, %rsp and $~63, %rsp # align rsp to 64 bytes .endm .macro FUNC_RESTORE - mov %r14, %rsp + mov %rbp, %rsp + pop %rbp pop %r15 - pop %r14 pop %r13 pop %r12 .endm From patchwork Wed Feb 24 16:29:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102127 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E25DC433E6 for ; Wed, 24 Feb 2021 16:34:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AEA4E64F02 for ; Wed, 24 Feb 2021 16:34:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234493AbhBXQeg (ORCPT ); Wed, 24 Feb 2021 11:34:36 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:49830 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234450AbhBXQcf (ORCPT ); Wed, 24 Feb 2021 11:32:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bBgJZbU8XJohBWogHzPnPQnzXHFLyp1Xf0dJOi3AIyQ=; b=Ug+m+ZEheT/Kg5bsr1sgdiwRRaUNkGA53w773Ml5BzYa7vw3JgEG9EdtHmfDxKGZqoyGUU eqPBZqlTkbH/0lP1GTTlSJGaanfy+FL9atH3WWHGAAT/EZZAQcU/M5+fFuJyZXOlORv2TL st81Uh12MiWgLwfj/pFkFQCggVJb6YM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-262-Xj6PoKWAOfu6p6elvNLZnA-1; Wed, 24 Feb 2021 11:29:42 -0500 X-MC-Unique: Xj6PoKWAOfu6p6elvNLZnA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D97DB107ACE6; Wed, 24 Feb 2021 16:29:39 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3A9B960862; Wed, 24 Feb 2021 16:29:38 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 05/13] x86/crypto/camellia-aesni-avx2: Unconditionally allocate stack buffer Date: Wed, 24 Feb 2021 10:29:18 -0600 Message-Id: <85ac96613ee5784b6239c18d3f68b1f3c509caa3.1614182415.git.jpoimboe@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org A conditional stack allocation violates traditional unwinding requirements when a single instruction can have differing stack layouts. There's no benefit in allocating the stack buffer conditionally. Just do it unconditionally. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/camellia-aesni-avx2-asm_64.S | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/x86/crypto/camellia-aesni-avx2-asm_64.S b/arch/x86/crypto/camellia-aesni-avx2-asm_64.S index 782e9712a1ec..706f70829a07 100644 --- a/arch/x86/crypto/camellia-aesni-avx2-asm_64.S +++ b/arch/x86/crypto/camellia-aesni-avx2-asm_64.S @@ -990,6 +990,7 @@ SYM_FUNC_START(camellia_cbc_dec_32way) * %rdx: src (32 blocks) */ FRAME_BEGIN + subq $(16 * 32), %rsp; vzeroupper; @@ -1002,7 +1003,6 @@ SYM_FUNC_START(camellia_cbc_dec_32way) %ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14, %ymm15, %rdx, (key_table)(CTX, %r8, 8)); - movq %rsp, %r10; cmpq %rsi, %rdx; je .Lcbc_dec_use_stack; @@ -1015,7 +1015,6 @@ SYM_FUNC_START(camellia_cbc_dec_32way) * dst still in-use (because dst == src), so use stack for temporary * storage. */ - subq $(16 * 32), %rsp; movq %rsp, %rax; .Lcbc_dec_continue: @@ -1025,7 +1024,6 @@ SYM_FUNC_START(camellia_cbc_dec_32way) vpxor %ymm7, %ymm7, %ymm7; vinserti128 $1, (%rdx), %ymm7, %ymm7; vpxor (%rax), %ymm7, %ymm7; - movq %r10, %rsp; vpxor (0 * 32 + 16)(%rdx), %ymm6, %ymm6; vpxor (1 * 32 + 16)(%rdx), %ymm5, %ymm5; vpxor (2 * 32 + 16)(%rdx), %ymm4, %ymm4; @@ -1047,6 +1045,7 @@ SYM_FUNC_START(camellia_cbc_dec_32way) vzeroupper; + addq $(16 * 32), %rsp; FRAME_END ret; SYM_FUNC_END(camellia_cbc_dec_32way) From patchwork Wed Feb 24 16:29:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102109 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B968CC433E6 for ; Wed, 24 Feb 2021 16:33:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 696F364EF1 for ; Wed, 24 Feb 2021 16:33:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234451AbhBXQcw (ORCPT ); Wed, 24 Feb 2021 11:32:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49072 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234708AbhBXQbT (ORCPT ); Wed, 24 Feb 2021 11:31:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184187; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BGP1D+fN8RYYb6r5ixsRX3SnpxjoEGyiGCXv8vP6/MA=; b=AczzPNyftfCwbLDC90SetT5H8nj4WjjNqt/A9Hb7otKshDaBUJp1agLN5P2Pn3oEIL6alz 1RNInWk+MVSEkhbL9Mb2xsfwn2lMCV9zIAJ+/Urah0olJ3MRWPGyWl4K74NtuKSRMOpBME eZJuTQC7hWqtqMK5ITyWXZIZs8tbFZA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-573-EpMdybTKMISHxWLyChJ06A-1; Wed, 24 Feb 2021 11:29:43 -0500 X-MC-Unique: EpMdybTKMISHxWLyChJ06A-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 67D3179EC3; Wed, 24 Feb 2021 16:29:41 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 13EF960CE7; Wed, 24 Feb 2021 16:29:39 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 06/13] x86/crypto/crc32c-pcl-intel: Standardize jump table Date: Wed, 24 Feb 2021 10:29:19 -0600 Message-Id: <5357a039def90b8ef6b5874ef12cda008ecf18ba.1614182415.git.jpoimboe@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Simplify the jump table code so that it resembles a compiler-generated table. This enables ORC unwinding by allowing objtool to follow all the potential code paths. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/crc32c-pcl-intel-asm_64.S | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/crypto/crc32c-pcl-intel-asm_64.S b/arch/x86/crypto/crc32c-pcl-intel-asm_64.S index 884dc767b051..ac1f303eed0f 100644 --- a/arch/x86/crypto/crc32c-pcl-intel-asm_64.S +++ b/arch/x86/crypto/crc32c-pcl-intel-asm_64.S @@ -53,7 +53,7 @@ .endm .macro JMPTBL_ENTRY i -.word crc_\i - crc_array +.quad crc_\i .endm .macro JNC_LESS_THAN j @@ -168,10 +168,7 @@ continue_block: xor crc2, crc2 ## branch into array - lea jump_table(%rip), %bufp - movzwq (%bufp, %rax, 2), len - lea crc_array(%rip), %bufp - lea (%bufp, len, 1), %bufp + mov jump_table(,%rax,8), %bufp JMP_NOSPEC bufp ################################################################ From patchwork Wed Feb 24 16:29:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102117 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D815EC433E0 for ; Wed, 24 Feb 2021 16:33:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 90D0164F00 for ; Wed, 24 Feb 2021 16:33:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234680AbhBXQcz (ORCPT ); Wed, 24 Feb 2021 11:32:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29740 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234717AbhBXQbU (ORCPT ); Wed, 24 Feb 2021 11:31:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184186; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eOHOjeFiMtzP9OGzSSHIXYUBmM19lsPz/JuM7QoJPBE=; b=TgLQCDy2bgsRVBHmbKeIYGLmStUd1Ogi9NQAC3rVtSH+4TgAP7LSwW4YytPKZu6lJd0yE5 kp2dHgzV08+385U7V0axuoodY9T9HAPCgEr3WcYHVU6gJGmDajbVs5PQeU0+O6zpdNQYlK 8Md705a3C2MuHgD5lU6ZycnO49JbqTE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-41-1h4Bbw7aMJuFblwKAqeTwQ-1; Wed, 24 Feb 2021 11:29:45 -0500 X-MC-Unique: 1h4Bbw7aMJuFblwKAqeTwQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0F3FF79EC1; Wed, 24 Feb 2021 16:29:43 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA4DA60862; Wed, 24 Feb 2021 16:29:41 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 07/13] x86/crypto/sha_ni: Standardize stack alignment prologue Date: Wed, 24 Feb 2021 10:29:20 -0600 Message-Id: <5033e1a79867dff1b18e1b4d0783c38897d3f223.1614182415.git.jpoimboe@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Use a more standard prologue for saving the stack pointer before realigning the stack. This enables ORC unwinding by allowing objtool to understand the stack realignment. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/sha1_ni_asm.S | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/sha1_ni_asm.S b/arch/x86/crypto/sha1_ni_asm.S index 11efe3a45a1f..5d8415f482bd 100644 --- a/arch/x86/crypto/sha1_ni_asm.S +++ b/arch/x86/crypto/sha1_ni_asm.S @@ -59,8 +59,6 @@ #define DATA_PTR %rsi /* 2nd arg */ #define NUM_BLKS %rdx /* 3rd arg */ -#define RSPSAVE %rax - /* gcc conversion */ #define FRAME_SIZE 32 /* space for 2x16 bytes */ @@ -96,7 +94,8 @@ .text .align 32 SYM_FUNC_START(sha1_ni_transform) - mov %rsp, RSPSAVE + push %rbp + mov %rsp, %rbp sub $FRAME_SIZE, %rsp and $~0xF, %rsp @@ -288,7 +287,8 @@ SYM_FUNC_START(sha1_ni_transform) pextrd $3, E0, 1*16(DIGEST_PTR) .Ldone_hash: - mov RSPSAVE, %rsp + mov %rbp, %rsp + pop %rbp ret SYM_FUNC_END(sha1_ni_transform) From patchwork Wed Feb 24 16:29:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102129 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 594FAC433E0 for ; Wed, 24 Feb 2021 16:36:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0390064EF5 for ; Wed, 24 Feb 2021 16:36:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235353AbhBXQfs (ORCPT ); Wed, 24 Feb 2021 11:35:48 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:57547 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235268AbhBXQdk (ORCPT ); Wed, 24 Feb 2021 11:33:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184324; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AuvWSaJ9jqWTQR6peCcPaZV/0nyJwPI7/KUXsD2VZRA=; b=RuLvmd2bph2Xgh7JoGsT0u9mrTZ8LUaDRgL0EzJ4keDSPBIo9c7+gU5K0WikovJ/Wgc4ip VwvSIyVehWh/97HxnHhCx//G3bSsSLKhVHnK3JcIKVayEOa1UDEvwe7X1JlhaqlUMOvhH9 gcJSs/1v01LiJyH7rMX07eFpiUwQiPg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-37-qtnbYkXpOa2FDxm45LwWDA-1; Wed, 24 Feb 2021 11:29:46 -0500 X-MC-Unique: qtnbYkXpOa2FDxm45LwWDA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 798931005501; Wed, 24 Feb 2021 16:29:44 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 40F5960862; Wed, 24 Feb 2021 16:29:43 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 08/13] x86/crypto/sha1_avx2: Standardize stack alignment prologue Date: Wed, 24 Feb 2021 10:29:21 -0600 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Use a more standard prologue for saving the stack pointer before realigning the stack. This enables ORC unwinding by allowing objtool to understand the stack realignment. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/sha1_avx2_x86_64_asm.S | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/sha1_avx2_x86_64_asm.S b/arch/x86/crypto/sha1_avx2_x86_64_asm.S index 1e594d60afa5..5eed620f4676 100644 --- a/arch/x86/crypto/sha1_avx2_x86_64_asm.S +++ b/arch/x86/crypto/sha1_avx2_x86_64_asm.S @@ -645,9 +645,9 @@ _loop3: RESERVE_STACK = (W_SIZE*4 + 8+24) /* Align stack */ - mov %rsp, %rbx + push %rbp + mov %rsp, %rbp and $~(0x20-1), %rsp - push %rbx sub $RESERVE_STACK, %rsp avx2_zeroupper @@ -665,8 +665,8 @@ _loop3: avx2_zeroupper - add $RESERVE_STACK, %rsp - pop %rsp + mov %rbp, %rsp + pop %rbp pop %r15 pop %r14 From patchwork Wed Feb 24 16:29:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102115 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56A02C433E9 for ; Wed, 24 Feb 2021 16:33:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 10FF164F00 for ; Wed, 24 Feb 2021 16:33:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234670AbhBXQdB (ORCPT ); Wed, 24 Feb 2021 11:33:01 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58316 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234875AbhBXQbU (ORCPT ); Wed, 24 Feb 2021 11:31:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184192; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gcsCAQPDZMD8BF6K4lOC+A/wlgHSo8oWdnRV/pLzh84=; b=W1t4VrutVd8I+D26/FVZmg8PFz1a8wBDtlHA8ltPDUZLoFQ92fEz7IQISgpBgQ0FTwZwBa uLjpeq6WQneqodWWSoWBcHnVJdwplMv9tNfFDhVYne5Bkoj9Qn4g7HOz1WBKdOf1UKkEm1 50ZsHLoTVcaE5sntXY/UHwnG7Lt3WsE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-89-N1u4lq1CMI2y_ddWYYqZAg-1; Wed, 24 Feb 2021 11:29:47 -0500 X-MC-Unique: N1u4lq1CMI2y_ddWYYqZAg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E80D9107ACE8; Wed, 24 Feb 2021 16:29:45 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id A5A7660862; Wed, 24 Feb 2021 16:29:44 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 09/13] x86/crypto/sha256-avx2: Standardize stack alignment prologue Date: Wed, 24 Feb 2021 10:29:22 -0600 Message-Id: <8048e7444c49a8137f05265262b83dc50f8fb7f3.1614182415.git.jpoimboe@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Use a more standard prologue for saving the stack pointer before realigning the stack. This enables ORC unwinding by allowing objtool to understand the stack realignment. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/sha256-avx2-asm.S | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/x86/crypto/sha256-avx2-asm.S b/arch/x86/crypto/sha256-avx2-asm.S index 11ff60c29c8b..4087f7432a7e 100644 --- a/arch/x86/crypto/sha256-avx2-asm.S +++ b/arch/x86/crypto/sha256-avx2-asm.S @@ -117,15 +117,13 @@ _XMM_SAVE_SIZE = 0 _INP_END_SIZE = 8 _INP_SIZE = 8 _CTX_SIZE = 8 -_RSP_SIZE = 8 _XFER = 0 _XMM_SAVE = _XFER + _XFER_SIZE _INP_END = _XMM_SAVE + _XMM_SAVE_SIZE _INP = _INP_END + _INP_END_SIZE _CTX = _INP + _INP_SIZE -_RSP = _CTX + _CTX_SIZE -STACK_SIZE = _RSP + _RSP_SIZE +STACK_SIZE = _CTX + _CTX_SIZE # rotate_Xs # Rotate values of symbols X0...X3 @@ -533,11 +531,11 @@ SYM_FUNC_START(sha256_transform_rorx) pushq %r14 pushq %r15 - mov %rsp, %rax + push %rbp + mov %rsp, %rbp + subq $STACK_SIZE, %rsp and $-32, %rsp # align rsp to 32 byte boundary - mov %rax, _RSP(%rsp) - shl $6, NUM_BLKS # convert to bytes jz done_hash @@ -704,7 +702,8 @@ only_one_block: done_hash: - mov _RSP(%rsp), %rsp + mov %rbp, %rsp + pop %rbp popq %r15 popq %r14 From patchwork Wed Feb 24 16:29:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102119 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85723C4332D for ; Wed, 24 Feb 2021 16:33:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 464F164EBA for ; Wed, 24 Feb 2021 16:33:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234875AbhBXQdC (ORCPT ); Wed, 24 Feb 2021 11:33:02 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:44163 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234878AbhBXQbV (ORCPT ); Wed, 24 Feb 2021 11:31:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I38yLyos8nKXeXwG/pZDKWvlhlgiM/lKSNW5bUlm4hQ=; b=gsRB0do+6lPRA2pjONY+G6PZcKeRrQADayUseL9bRte54WNIA1oJx4kveZj8x/WQMgiH3c S+Yop1PttFeOlNmjMs2y5a/lVyreWOuvKMnUqwO29uDSI7sNP9qGODcb84Gj9nHT5O0XfK YonFRFdf/6CIyBpDck0HOmmT/oDpEmk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-434-XTKiW_nXPWmbyaYOqBO0Hw-1; Wed, 24 Feb 2021 11:29:49 -0500 X-MC-Unique: XTKiW_nXPWmbyaYOqBO0Hw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8D1431936B71; Wed, 24 Feb 2021 16:29:47 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2154460CE7; Wed, 24 Feb 2021 16:29:46 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 10/13] x86/crypto/sha512-avx: Standardize stack alignment prologue Date: Wed, 24 Feb 2021 10:29:23 -0600 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Use a more standard prologue for saving the stack pointer before realigning the stack. This enables ORC unwinding by allowing objtool to understand the stack realignment. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/sha512-avx-asm.S | 41 +++++++++++++++----------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/arch/x86/crypto/sha512-avx-asm.S b/arch/x86/crypto/sha512-avx-asm.S index 684d58c8bc4f..3d8f0fd4eea8 100644 --- a/arch/x86/crypto/sha512-avx-asm.S +++ b/arch/x86/crypto/sha512-avx-asm.S @@ -76,14 +76,10 @@ tmp0 = %rax W_SIZE = 80*8 # W[t] + K[t] | W[t+1] + K[t+1] WK_SIZE = 2*8 -RSPSAVE_SIZE = 1*8 -GPRSAVE_SIZE = 5*8 frame_W = 0 frame_WK = frame_W + W_SIZE -frame_RSPSAVE = frame_WK + WK_SIZE -frame_GPRSAVE = frame_RSPSAVE + RSPSAVE_SIZE -frame_size = frame_GPRSAVE + GPRSAVE_SIZE +frame_size = frame_WK + WK_SIZE # Useful QWORD "arrays" for simpler memory references # MSG, DIGEST, K_t, W_t are arrays @@ -281,18 +277,18 @@ SYM_FUNC_START(sha512_transform_avx) test msglen, msglen je nowork + # Save GPRs + push %rbx + push %r12 + push %r13 + push %r14 + push %r15 + # Allocate Stack Space - mov %rsp, %rax + push %rbp + mov %rsp, %rbp sub $frame_size, %rsp and $~(0x20 - 1), %rsp - mov %rax, frame_RSPSAVE(%rsp) - - # Save GPRs - mov %rbx, frame_GPRSAVE(%rsp) - mov %r12, frame_GPRSAVE +8*1(%rsp) - mov %r13, frame_GPRSAVE +8*2(%rsp) - mov %r14, frame_GPRSAVE +8*3(%rsp) - mov %r15, frame_GPRSAVE +8*4(%rsp) updateblock: @@ -353,15 +349,16 @@ updateblock: dec msglen jnz updateblock - # Restore GPRs - mov frame_GPRSAVE(%rsp), %rbx - mov frame_GPRSAVE +8*1(%rsp), %r12 - mov frame_GPRSAVE +8*2(%rsp), %r13 - mov frame_GPRSAVE +8*3(%rsp), %r14 - mov frame_GPRSAVE +8*4(%rsp), %r15 - # Restore Stack Pointer - mov frame_RSPSAVE(%rsp), %rsp + mov %rbp, %rsp + pop %rbp + + # Restore GPRs + pop %r15 + pop %r14 + pop %r13 + pop %r12 + pop %rbx nowork: ret From patchwork Wed Feb 24 16:29:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102121 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AC58C4332E for ; Wed, 24 Feb 2021 16:33:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D212A64EF1 for ; Wed, 24 Feb 2021 16:33:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234878AbhBXQdD (ORCPT ); Wed, 24 Feb 2021 11:33:03 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29146 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234881AbhBXQb3 (ORCPT ); Wed, 24 Feb 2021 11:31:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184194; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a5xSosP/C+AktqnKcTC1zhdA3nQKp0XFHHslguVKp40=; b=geetHY4gZPx309/naBbC7EsfAQP1kq2hDd+CEjcCu8RzlqFrsF9g/3ZEfo73DKLN+1U17d Ayn2TYVA4Jgl8Ta34O5ouZjq+1vuzixMsQw+fMjjlwn0mh4OlGAKWvXw9r7Y8y8w6wYCPa 9SRmB2LOxjhJJqHoLlT5JK5jngXFYkg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-407-sHhUE2crP_ihPEziQ6xsKw-1; Wed, 24 Feb 2021 11:29:50 -0500 X-MC-Unique: sHhUE2crP_ihPEziQ6xsKw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 00382107ACE4; Wed, 24 Feb 2021 16:29:49 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id B8A5260CE7; Wed, 24 Feb 2021 16:29:47 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 11/13] x86/crypto/sha512-avx2: Standardize stack alignment prologue Date: Wed, 24 Feb 2021 10:29:24 -0600 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Use a more standard prologue for saving the stack pointer before realigning the stack. This enables ORC unwinding by allowing objtool to understand the stack realignment. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/sha512-avx2-asm.S | 42 +++++++++++++++---------------- 1 file changed, 20 insertions(+), 22 deletions(-) diff --git a/arch/x86/crypto/sha512-avx2-asm.S b/arch/x86/crypto/sha512-avx2-asm.S index 3a44bdcfd583..072cb0f0deae 100644 --- a/arch/x86/crypto/sha512-avx2-asm.S +++ b/arch/x86/crypto/sha512-avx2-asm.S @@ -102,17 +102,13 @@ SRND_SIZE = 1*8 INP_SIZE = 1*8 INPEND_SIZE = 1*8 CTX_SIZE = 1*8 -RSPSAVE_SIZE = 1*8 -GPRSAVE_SIZE = 5*8 frame_XFER = 0 frame_SRND = frame_XFER + XFER_SIZE frame_INP = frame_SRND + SRND_SIZE frame_INPEND = frame_INP + INP_SIZE frame_CTX = frame_INPEND + INPEND_SIZE -frame_RSPSAVE = frame_CTX + CTX_SIZE -frame_GPRSAVE = frame_RSPSAVE + RSPSAVE_SIZE -frame_size = frame_GPRSAVE + GPRSAVE_SIZE +frame_size = frame_CTX + CTX_SIZE ## assume buffers not aligned #define VMOVDQ vmovdqu @@ -570,18 +566,18 @@ frame_size = frame_GPRSAVE + GPRSAVE_SIZE # "blocks" is the message length in SHA512 blocks ######################################################################## SYM_FUNC_START(sha512_transform_rorx) + # Save GPRs + push %rbx + push %r12 + push %r13 + push %r14 + push %r15 + # Allocate Stack Space - mov %rsp, %rax + push %rbp + mov %rsp, %rbp sub $frame_size, %rsp and $~(0x20 - 1), %rsp - mov %rax, frame_RSPSAVE(%rsp) - - # Save GPRs - mov %rbx, 8*0+frame_GPRSAVE(%rsp) - mov %r12, 8*1+frame_GPRSAVE(%rsp) - mov %r13, 8*2+frame_GPRSAVE(%rsp) - mov %r14, 8*3+frame_GPRSAVE(%rsp) - mov %r15, 8*4+frame_GPRSAVE(%rsp) shl $7, NUM_BLKS # convert to bytes jz done_hash @@ -672,15 +668,17 @@ loop2: done_hash: -# Restore GPRs - mov 8*0+frame_GPRSAVE(%rsp), %rbx - mov 8*1+frame_GPRSAVE(%rsp), %r12 - mov 8*2+frame_GPRSAVE(%rsp), %r13 - mov 8*3+frame_GPRSAVE(%rsp), %r14 - mov 8*4+frame_GPRSAVE(%rsp), %r15 - # Restore Stack Pointer - mov frame_RSPSAVE(%rsp), %rsp + mov %rbp, %rsp + pop %rbp + + # Restore GPRs + pop %r15 + pop %r14 + pop %r13 + pop %r12 + pop %rbx + ret SYM_FUNC_END(sha512_transform_rorx) From patchwork Wed Feb 24 16:29:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102123 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60BCAC43333 for ; Wed, 24 Feb 2021 16:33:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 321A764F00 for ; Wed, 24 Feb 2021 16:33:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234934AbhBXQdX (ORCPT ); Wed, 24 Feb 2021 11:33:23 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:42784 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234915AbhBXQb3 (ORCPT ); Wed, 24 Feb 2021 11:31:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184196; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QHAXN62NKSYCiC1JTav+S2nx7912U3/cEpM7FBeWNPo=; b=AMIzG+hY/Lz6hpjfnJdpXeCE8Oznsq1t0wOEZdDjJaEFi+ea6vUzmJIdBL5BSWxfUL8Vok KqkLseqxZrt4TnCk7m8Fcb5Bd+9IOKkOJ5gc5N+AVypCBxGSkKjLzp8gfVZYRU5qY8v5gJ OCpxoNLIaAeOjMe2QljG24YE4eQxjp8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-482-W4AgpMZVPHycxmJilMLD3g-1; Wed, 24 Feb 2021 11:29:52 -0500 X-MC-Unique: W4AgpMZVPHycxmJilMLD3g-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9D1541936B60; Wed, 24 Feb 2021 16:29:50 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2E65260862; Wed, 24 Feb 2021 16:29:49 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 12/13] x86/crypto/sha512-ssse3: Standardize stack alignment prologue Date: Wed, 24 Feb 2021 10:29:25 -0600 Message-Id: <6ecaaac9f3828fbb903513bf90c34a08380a8e35.1614182415.git.jpoimboe@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Use a more standard prologue for saving the stack pointer before realigning the stack. This enables ORC unwinding by allowing objtool to understand the stack realignment. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/sha512-ssse3-asm.S | 41 ++++++++++++++---------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/arch/x86/crypto/sha512-ssse3-asm.S b/arch/x86/crypto/sha512-ssse3-asm.S index 50812af0b083..bd51c9070bed 100644 --- a/arch/x86/crypto/sha512-ssse3-asm.S +++ b/arch/x86/crypto/sha512-ssse3-asm.S @@ -74,14 +74,10 @@ tmp0 = %rax W_SIZE = 80*8 WK_SIZE = 2*8 -RSPSAVE_SIZE = 1*8 -GPRSAVE_SIZE = 5*8 frame_W = 0 frame_WK = frame_W + W_SIZE -frame_RSPSAVE = frame_WK + WK_SIZE -frame_GPRSAVE = frame_RSPSAVE + RSPSAVE_SIZE -frame_size = frame_GPRSAVE + GPRSAVE_SIZE +frame_size = frame_WK + WK_SIZE # Useful QWORD "arrays" for simpler memory references # MSG, DIGEST, K_t, W_t are arrays @@ -283,18 +279,18 @@ SYM_FUNC_START(sha512_transform_ssse3) test msglen, msglen je nowork + # Save GPRs + push %rbx + push %r12 + push %r13 + push %r14 + push %r15 + # Allocate Stack Space - mov %rsp, %rax + push %rbp + mov %rsp, %rbp sub $frame_size, %rsp and $~(0x20 - 1), %rsp - mov %rax, frame_RSPSAVE(%rsp) - - # Save GPRs - mov %rbx, frame_GPRSAVE(%rsp) - mov %r12, frame_GPRSAVE +8*1(%rsp) - mov %r13, frame_GPRSAVE +8*2(%rsp) - mov %r14, frame_GPRSAVE +8*3(%rsp) - mov %r15, frame_GPRSAVE +8*4(%rsp) updateblock: @@ -355,15 +351,16 @@ updateblock: dec msglen jnz updateblock - # Restore GPRs - mov frame_GPRSAVE(%rsp), %rbx - mov frame_GPRSAVE +8*1(%rsp), %r12 - mov frame_GPRSAVE +8*2(%rsp), %r13 - mov frame_GPRSAVE +8*3(%rsp), %r14 - mov frame_GPRSAVE +8*4(%rsp), %r15 - # Restore Stack Pointer - mov frame_RSPSAVE(%rsp), %rsp + mov %rbp, %rsp + pop %rbp + + # Restore GPRs + pop %r15 + pop %r14 + pop %r13 + pop %r12 + pop %rbx nowork: ret From patchwork Wed Feb 24 16:29:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 12102125 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 558DBC43332 for ; Wed, 24 Feb 2021 16:33:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0A6D164EFC for ; Wed, 24 Feb 2021 16:33:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235227AbhBXQdJ (ORCPT ); Wed, 24 Feb 2021 11:33:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28416 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234934AbhBXQb3 (ORCPT ); Wed, 24 Feb 2021 11:31:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614184198; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/DLjCfyPdWpsrA8C7WRrfmxuAmIfnze3qgJu059srp4=; b=E64oJ7T8mwld4VY1r4y45ENIU7KjE1caB11DXZ3x3+0pLU0qz5Tk0t1WLmxl6kVngTUx6Z 8zEfD6t0TRzu/7frMwg4sEXQJS3eRNFtyMoS1tes+5miJHR/xhURTe3mg60xML9XoCaJh3 fw9PXYQTmAL+bgC+2xPDHHnk0DUpXnU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-145-EE2OuDdWNTW4amn49x2nhw-1; Wed, 24 Feb 2021 11:29:54 -0500 X-MC-Unique: EE2OuDdWNTW4amn49x2nhw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 29115107ACE4; Wed, 24 Feb 2021 16:29:52 +0000 (UTC) Received: from treble.redhat.com (ovpn-118-134.rdu2.redhat.com [10.10.118.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id C9F8A60862; Wed, 24 Feb 2021 16:29:50 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org, Herbert Xu Cc: linux-kernel@vger.kernel.org, Miroslav Benes , Peter Zijlstra , Eric Biggers , Ard Biesheuvel , Sami Tolvanen , linux-crypto@vger.kernel.org, "David S . Miller" Subject: [PATCH 13/13] x86/crypto: Enable objtool in crypto code Date: Wed, 24 Feb 2021 10:29:26 -0600 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Now that all the stack alignment prologues have been cleaned up in the crypto code, enable objtool. Among other benefits, this will allow ORC unwinding to work. Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/Makefile | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/x86/crypto/Makefile b/arch/x86/crypto/Makefile index b28e36b7c96b..d0959e7b809f 100644 --- a/arch/x86/crypto/Makefile +++ b/arch/x86/crypto/Makefile @@ -2,8 +2,6 @@ # # x86 crypto algorithms -OBJECT_FILES_NON_STANDARD := y - obj-$(CONFIG_CRYPTO_TWOFISH_586) += twofish-i586.o twofish-i586-y := twofish-i586-asm_32.o twofish_glue.o obj-$(CONFIG_CRYPTO_TWOFISH_X86_64) += twofish-x86_64.o