From patchwork Fri Oct 11 00:40:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184745 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4A11917EE for ; Fri, 11 Oct 2019 00:41:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2B8B62190F for ; Fri, 11 Oct 2019 00:41:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727528AbfJKAlE (ORCPT ); Thu, 10 Oct 2019 20:41:04 -0400 Received: from mga18.intel.com ([134.134.136.126]:44342 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728094AbfJKAlD (ORCPT ); Thu, 10 Oct 2019 20:41:03 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 17:41:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,282,1566889200"; d="scan'208";a="198542990" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by orsmga006.jf.intel.com with ESMTP; 10 Oct 2019 17:41:01 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 1/5] x86/vdso: sgx: Fix misaligned stack bug when invoking exit handler Date: Thu, 10 Oct 2019 17:40:55 -0700 Message-Id: <20191011004059.28581-2-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191011004059.28581-1-sean.j.christopherson@intel.com> References: <20191011004059.28581-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Fix a recently introduced bug where an odd number of 8-byte parameters are pushed on the stack prior to invoking the userspace callback, which causes the CALL to execute with an unaligned stack and violate the x86_64 ABI. Reported-by: Cedric Xing Signed-off-by: Sean Christopherson --- arch/x86/entry/vdso/vsgx_enter_enclave.S | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S index fc5622dcd2fa..b63091818df1 100644 --- a/arch/x86/entry/vdso/vsgx_enter_enclave.S +++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S @@ -135,10 +135,12 @@ ENTRY(__vdso_sgx_enter_enclave) .Linvoke_userspace_handler: /* * Align stack per x86_64 ABI. Save the original %rsp in %rbx to be - * restored after the callback returns. + * restored after the callback returns. Note, %rsp needs to be 16-byte + * aligned _after_ pushing the three parameters on the stack. */ mov %rsp, %rbx and $-0x10, %rsp + sub $0x8, %rsp /* Push @e, u_rsp and @tcs as parameters to the callback. */ push 0x18(%rbp) From patchwork Fri Oct 11 00:40:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184743 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 233441575 for ; Fri, 11 Oct 2019 00:41:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 04FFC2190F for ; Fri, 11 Oct 2019 00:41:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727620AbfJKAlE (ORCPT ); Thu, 10 Oct 2019 20:41:04 -0400 Received: from mga18.intel.com ([134.134.136.126]:44340 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728095AbfJKAlD (ORCPT ); Thu, 10 Oct 2019 20:41:03 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 17:41:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,282,1566889200"; d="scan'208";a="198542993" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by orsmga006.jf.intel.com with ESMTP; 10 Oct 2019 17:41:01 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 2/5] selftests/x86/sgx: Add check to verify exit handler stack alignment Date: Thu, 10 Oct 2019 17:40:56 -0700 Message-Id: <20191011004059.28581-3-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191011004059.28581-1-sean.j.christopherson@intel.com> References: <20191011004059.28581-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Add an assembly trampoline to the basic exit handler to snapshot the pre-CALL %rsp in order to verify that the stack is 16-byte aligned as required by the x86_64 ABI. Signed-off-by: Sean Christopherson --- tools/testing/selftests/x86/sgx/defines.h | 1 + tools/testing/selftests/x86/sgx/main.c | 20 +++++++++++++++++--- 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/x86/sgx/defines.h b/tools/testing/selftests/x86/sgx/defines.h index 199a830e198a..8ddd3cfd1070 100644 --- a/tools/testing/selftests/x86/sgx/defines.h +++ b/tools/testing/selftests/x86/sgx/defines.h @@ -15,6 +15,7 @@ typedef uint64_t u64; #define __aligned(x) __attribute__((__aligned__(x))) #define __packed __attribute__((packed)) +#define __used __attribute__((__used__)) /* Derived from asm-generic/bitsperlong.h. */ #if __x86_64__ diff --git a/tools/testing/selftests/x86/sgx/main.c b/tools/testing/selftests/x86/sgx/main.c index 029502d81ac9..f46e5c8fdac4 100644 --- a/tools/testing/selftests/x86/sgx/main.c +++ b/tools/testing/selftests/x86/sgx/main.c @@ -331,14 +331,28 @@ static void test_vdso_no_exit_handler(struct sgx_secs *secs) ASSERT_EQ(exception.leaf, ENCLU_EENTER); } -static int basic_exit_handler(long rdi, long rsi, long rdx, int ret, - long r8, long r9, void *tcs, long ursp, - struct sgx_enclave_exception *e) +static int __used __basic_exit_handler(long rdi, long rsi, long rdx, int ret, + long r8, long r9, void *tcs, long ursp, + struct sgx_enclave_exception *e) { + TEST_ASSERT(!(r9 & 0xf), "Pre-CALL RSP not 16-byte aligned: %lx\n", r9); ASSERT_EQ(ret, 0); return 0; } +extern void *basic_exit_handler; + +static void __used basic_exit_handler_trampoline(void) +{ + /* Load the pre-CALL %rsp into %r9 to verify correct alignment. */ + asm volatile("1:\n\t" + "lea 0x8(%%rsp), %%r9\n\t" + "jmp __basic_exit_handler\n\t" + "basic_exit_handler: .quad 1b\n\t" + ".global basic_exit_handler" + ::: "memory"); +} + static int nr_page_faults; static int mprotect_exit_handler(long rdi, long rsi, long rdx, int ret, From patchwork Fri Oct 11 00:40:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184737 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D31A817EE for ; Fri, 11 Oct 2019 00:41:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BE0E62053B for ; Fri, 11 Oct 2019 00:41:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727581AbfJKAlD (ORCPT ); Thu, 10 Oct 2019 20:41:03 -0400 Received: from mga18.intel.com ([134.134.136.126]:44340 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727620AbfJKAlD (ORCPT ); Thu, 10 Oct 2019 20:41:03 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 17:41:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,282,1566889200"; d="scan'208";a="198542996" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by orsmga006.jf.intel.com with ESMTP; 10 Oct 2019 17:41:01 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 3/5] x86/vdso: sgx: Fix unwinder support Date: Thu, 10 Oct 2019 17:40:57 -0700 Message-Id: <20191011004059.28581-4-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191011004059.28581-1-sean.j.christopherson@intel.com> References: <20191011004059.28581-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Relocate the .cfi_endproc directive to the RET that actually returns from the vDSO function. During recent refactoring, it was inadvertantly left next to the RET used for the retpoline. Reported-by: Cedric Xing Signed-off-by: Sean Christopherson --- arch/x86/entry/vdso/vsgx_enter_enclave.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S index b63091818df1..3dd22780b7ef 100644 --- a/arch/x86/entry/vdso/vsgx_enter_enclave.S +++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S @@ -113,6 +113,7 @@ ENTRY(__vdso_sgx_enter_enclave) leave .cfi_def_cfa %rsp, 8 ret + .cfi_endproc .Linvalid_leaf: mov $(-EINVAL), %eax @@ -176,7 +177,6 @@ ENTRY(__vdso_sgx_enter_enclave) jmp 1b 2: mov %rax, (%rsp) ret - .cfi_endproc _ASM_VDSO_EXTABLE_HANDLE(.Lenclu_eenter_eresume, .Lhandle_exception) From patchwork Fri Oct 11 00:40:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184741 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58CE21920 for ; Fri, 11 Oct 2019 00:41:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 43CCE2190F for ; Fri, 11 Oct 2019 00:41:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728096AbfJKAlD (ORCPT ); Thu, 10 Oct 2019 20:41:03 -0400 Received: from mga18.intel.com ([134.134.136.126]:44340 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727528AbfJKAlC (ORCPT ); Thu, 10 Oct 2019 20:41:02 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 17:41:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,282,1566889200"; d="scan'208";a="198542999" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by orsmga006.jf.intel.com with ESMTP; 10 Oct 2019 17:41:01 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 4/5] x86/vdso: sgx: Reorder params to callback to improve readability Date: Thu, 10 Oct 2019 17:40:58 -0700 Message-Id: <20191011004059.28581-5-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191011004059.28581-1-sean.j.christopherson@intel.com> References: <20191011004059.28581-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Swap @ret and @ursp in the callback prototype so that the output from the vDSO itself, @ret and @e, are grouped together. Having the first N parameters all share a type also makes the prototype easier to parse by (some) humans. And, passing @ursp via register saves one whole MOV instruction! Signed-off-by: Sean Christopherson --- arch/x86/entry/vdso/vsgx_enter_enclave.S | 16 +++++++--------- arch/x86/include/uapi/asm/sgx.h | 10 +++++----- 2 files changed, 12 insertions(+), 14 deletions(-) diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S index 3dd22780b7ef..94f613b53b13 100644 --- a/arch/x86/entry/vdso/vsgx_enter_enclave.S +++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S @@ -134,23 +134,21 @@ ENTRY(__vdso_sgx_enter_enclave) jmp .Lhandle_exit .Linvoke_userspace_handler: + /* Pass the untrusted RSP (at exit) to the callback via %rcx. */ + mov %rsp, %rcx + /* - * Align stack per x86_64 ABI. Save the original %rsp in %rbx to be - * restored after the callback returns. Note, %rsp needs to be 16-byte - * aligned _after_ pushing the three parameters on the stack. + * Align stack per x86_64 ABI. Note, %rsp needs to be 16-byte aligned + * _after_ pushing the three parameters on the stack. */ - mov %rsp, %rbx and $-0x10, %rsp sub $0x8, %rsp - /* Push @e, u_rsp and @tcs as parameters to the callback. */ + /* Push @e, the "return" value and @tcs as params to the callback. */ push 0x18(%rbp) - push %rbx + push %rax push 0x10(%rbp) - /* Pass the "return" value to the callback via %rcx. */ - mov %eax, %ecx - /* Clear RFLAGS.DF per x86_64 ABI */ cld diff --git a/arch/x86/include/uapi/asm/sgx.h b/arch/x86/include/uapi/asm/sgx.h index 255392f5054f..95f69f938b58 100644 --- a/arch/x86/include/uapi/asm/sgx.h +++ b/arch/x86/include/uapi/asm/sgx.h @@ -88,16 +88,16 @@ struct sgx_enclave_exception { * @rdi: RDI at the time of enclave exit * @rsi: RSI at the time of enclave exit * @rdx: RDX at the time of enclave exit - * @ret: 0 on success (EEXIT), -EFAULT on an exception + * @ursp: RSP at the time of enclave exit (untrusted stack) * @r8: R8 at the time of enclave exit * @r9: R9 at the time of enclave exit * @tcs: Thread Control Structure used to enter enclave - * @ursp: RSP at the time of enclave exit + * @ret: 0 on success (EEXIT), -EFAULT on an exception * @e: Pointer to struct sgx_enclave_exception (as provided by caller) */ -typedef int (*sgx_enclave_exit_handler_t)(long rdi, long rsi, long rdx, int ret, - long r8, long r9, void *tcs, - long ursp, +typedef int (*sgx_enclave_exit_handler_t)(long rdi, long rsi, long rdx, + long ursp, long r8, long r9, + void *tcs, int ret, struct sgx_enclave_exception *e); #endif /* _UAPI_ASM_X86_SGX_H */ From patchwork Fri Oct 11 00:40:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184735 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8DFD1575 for ; Fri, 11 Oct 2019 00:41:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9E6AB2053B for ; Fri, 11 Oct 2019 00:41:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728093AbfJKAlC (ORCPT ); Thu, 10 Oct 2019 20:41:02 -0400 Received: from mga18.intel.com ([134.134.136.126]:44340 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728086AbfJKAlC (ORCPT ); Thu, 10 Oct 2019 20:41:02 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 17:41:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,282,1566889200"; d="scan'208";a="198543001" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by orsmga006.jf.intel.com with ESMTP; 10 Oct 2019 17:41:01 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 5/5] selftests/x86/sgx: Update the callbacks function parameters Date: Thu, 10 Oct 2019 17:40:59 -0700 Message-Id: <20191011004059.28581-6-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191011004059.28581-1-sean.j.christopherson@intel.com> References: <20191011004059.28581-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Swap @ursp and @ret in the exit handler callbacks to match a recent kernel change. Signed-off-by: Sean Christopherson --- tools/testing/selftests/x86/sgx/main.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/x86/sgx/main.c b/tools/testing/selftests/x86/sgx/main.c index f46e5c8fdac4..b84ffbf6e1e2 100644 --- a/tools/testing/selftests/x86/sgx/main.c +++ b/tools/testing/selftests/x86/sgx/main.c @@ -331,8 +331,8 @@ static void test_vdso_no_exit_handler(struct sgx_secs *secs) ASSERT_EQ(exception.leaf, ENCLU_EENTER); } -static int __used __basic_exit_handler(long rdi, long rsi, long rdx, int ret, - long r8, long r9, void *tcs, long ursp, +static int __used __basic_exit_handler(long rdi, long rsi, long rdx, long ursp, + long r8, long r9, void *tcs, int ret, struct sgx_enclave_exception *e) { TEST_ASSERT(!(r9 & 0xf), "Pre-CALL RSP not 16-byte aligned: %lx\n", r9); @@ -355,8 +355,8 @@ static void __used basic_exit_handler_trampoline(void) static int nr_page_faults; -static int mprotect_exit_handler(long rdi, long rsi, long rdx, int ret, - long r8, long r9, void *tcs, long ursp, +static int mprotect_exit_handler(long rdi, long rsi, long rdx, long ursp, + long r8, long r9, void *tcs, int ret, struct sgx_enclave_exception *e) { int prot, rc;