From patchwork Tue Aug 29 18:05:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 9927875 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9B8CD602B9 for ; Tue, 29 Aug 2017 18:07:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 94D44289F1 for ; Tue, 29 Aug 2017 18:07:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 89832289F4; Tue, 29 Aug 2017 18:07:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F0A33289F1 for ; Tue, 29 Aug 2017 18:07:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751684AbdH2SHI (ORCPT ); Tue, 29 Aug 2017 14:07:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39772 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751514AbdH2SGb (ORCPT ); Tue, 29 Aug 2017 14:06:31 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C8C664DD4B; Tue, 29 Aug 2017 18:06:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C8C664DD4B Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=jpoimboe@redhat.com Received: from treble.redhat.com (ovpn-121-232.rdu2.redhat.com [10.10.121.232]) by smtp.corp.redhat.com (Postfix) with ESMTP id 78BE57D4F2; Tue, 29 Aug 2017 18:06:29 +0000 (UTC) From: Josh Poimboeuf To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, Tim Chen , Mathias Krause , Chandramouli Narayanan , Jussi Kivilinna , Peter Zijlstra , Herbert Xu , "David S. Miller" , linux-crypto@vger.kernel.org, Eric Biggers , Andy Lutomirski , Jiri Slaby Subject: [PATCH 11/12] x86/crypto: Fix RBP usage in sha512-avx2-asm.S Date: Tue, 29 Aug 2017 13:05:44 -0500 Message-Id: <9a0f9d601d4ee24f6896f4764f3e3d08c80fa806.1504029132.git.jpoimboe@redhat.com> In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 29 Aug 2017 18:06:31 +0000 (UTC) Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Using RBP as a temporary register breaks frame pointer convention and breaks stack traces when unwinding from an interrupt in the crypto code. Use R12 instead of RBP for the TBL register. Since R12 is also used as another temporary register (T1), it gets clobbered in each round of computation. So the TBL value needs to be freshly reloaded into R12 each time it's used. Since the value of TBL can change, store its permanent value on the stack at the frame_TBL offset. Also remove the unused y4 variable. Reported-by: Eric Biggers Reported-by: Peter Zijlstra Signed-off-by: Josh Poimboeuf --- arch/x86/crypto/sha512-avx2-asm.S | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/arch/x86/crypto/sha512-avx2-asm.S b/arch/x86/crypto/sha512-avx2-asm.S index 7f5f6c6ec72e..37cfc2004abd 100644 --- a/arch/x86/crypto/sha512-avx2-asm.S +++ b/arch/x86/crypto/sha512-avx2-asm.S @@ -81,7 +81,7 @@ d = %r8 e = %rdx y3 = %rsi -TBL = %rbp +TBL = %r12 # clobbered by T1 a = %rax b = %rbx @@ -96,11 +96,10 @@ y0 = %r13 y1 = %r14 y2 = %r15 -y4 = %r12 - # Local variables (stack frame) XFER_SIZE = 4*8 SRND_SIZE = 1*8 +TBL_SIZE = 1*8 INP_SIZE = 1*8 INPEND_SIZE = 1*8 RSPSAVE_SIZE = 1*8 @@ -108,7 +107,8 @@ GPRSAVE_SIZE = 6*8 frame_XFER = 0 frame_SRND = frame_XFER + XFER_SIZE -frame_INP = frame_SRND + SRND_SIZE +frame_TBL = frame_SRND + SRND_SIZE +frame_INP = frame_TBL + TBL_SIZE frame_INPEND = frame_INP + INP_SIZE frame_RSPSAVE = frame_INPEND + INPEND_SIZE frame_GPRSAVE = frame_RSPSAVE + RSPSAVE_SIZE @@ -601,7 +601,7 @@ ENTRY(sha512_transform_rorx) vmovdqa PSHUFFLE_BYTE_FLIP_MASK(%rip), BYTE_FLIP_MASK loop0: - lea K512(%rip), TBL + movq $K512, frame_TBL(%rsp) ## byte swap first 16 dwords COPY_YMM_AND_BSWAP Y_0, (INP), BYTE_FLIP_MASK @@ -616,39 +616,46 @@ loop0: .align 16 loop1: + mov frame_TBL(%rsp), TBL vpaddq (TBL), Y_0, XFER vmovdqa XFER, frame_XFER(%rsp) FOUR_ROUNDS_AND_SCHED + mov frame_TBL(%rsp), TBL vpaddq 1*32(TBL), Y_0, XFER vmovdqa XFER, frame_XFER(%rsp) FOUR_ROUNDS_AND_SCHED + mov frame_TBL(%rsp), TBL vpaddq 2*32(TBL), Y_0, XFER vmovdqa XFER, frame_XFER(%rsp) FOUR_ROUNDS_AND_SCHED + mov frame_TBL(%rsp), TBL vpaddq 3*32(TBL), Y_0, XFER vmovdqa XFER, frame_XFER(%rsp) - add $(4*32), TBL FOUR_ROUNDS_AND_SCHED + addq $(4*32), frame_TBL(%rsp) subq $1, frame_SRND(%rsp) jne loop1 movq $2, frame_SRND(%rsp) loop2: + mov frame_TBL(%rsp), TBL vpaddq (TBL), Y_0, XFER vmovdqa XFER, frame_XFER(%rsp) DO_4ROUNDS + + mov frame_TBL(%rsp), TBL vpaddq 1*32(TBL), Y_1, XFER vmovdqa XFER, frame_XFER(%rsp) - add $(2*32), TBL DO_4ROUNDS vmovdqa Y_2, Y_0 vmovdqa Y_3, Y_1 + add $(2*32), frame_TBL(%rsp) subq $1, frame_SRND(%rsp) jne loop2