From patchwork Mon Dec 10 23:21:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 10722753 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37E9717FE for ; Mon, 10 Dec 2018 23:22:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 270862A710 for ; Mon, 10 Dec 2018 23:22:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1AEF42A71B; Mon, 10 Dec 2018 23:22:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C2372A710 for ; Mon, 10 Dec 2018 23:22:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729965AbeLJXWA (ORCPT ); Mon, 10 Dec 2018 18:22:00 -0500 Received: from mga04.intel.com ([192.55.52.120]:11277 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727919AbeLJXV7 (ORCPT ); Mon, 10 Dec 2018 18:21:59 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Dec 2018 15:21:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,340,1539673200"; d="scan'208";a="117684769" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.154]) by orsmga001.jf.intel.com with ESMTP; 10 Dec 2018 15:21:57 -0800 From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, Jarkko Sakkinen , Sean Christopherson , Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org, Andy Lutomirski , Josh Triplett , Haitao Huang , Jethro Beekman , "Dr . Greg Wettstein" Subject: [RFC PATCH v3 1/4] x86/sgx: Add a per-mm ENCLU exception fixup handler Date: Mon, 10 Dec 2018 15:21:38 -0800 Message-Id: <20181210232141.5425-2-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181210232141.5425-1-sean.j.christopherson@intel.com> References: <20181210232141.5425-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Intel Software Guard Extensions (SGX) introduces a new CPL3-only "enclave" mode that runs as a sort of black box shared object that is hosted by an untrusted "normal" CPl3 process. Entering an enclave can only be done through SGX-specific instructions, EENTER and ERESUME, and is a non-trivial process. Because of the complexity of transitioning to/from an enclave, the vast majority of enclaves are expected to utilize a library to handle the actual transitions. This is roughly analogous to how e.g. a libc and dynamic linker are used by most applications. Another crucial characteristic of SGX enclaves is that they can generate exceptions as part of their normal (at least as "normal" as SGX can be) operation that need to be handled *in* the enclave and/or are unique to SGX. And because they are essentially fancy shared objects, a process can host any number of enclaves, each of which can execute multiple threads simultaneously. Putting everything together, userspace enclaves will utilize a library that must be prepared to handle any and (almost) all exceptions any time at least one thread may be executing in an enclave. Leveraging signals to handle the enclave exceptions is unpleasant, to put it mildly, e.g. the SGX library must constantly (un)register its signal handler based on whether or not at least one thread is executing in an enclave, and filter and forward exceptions that aren't related to its enclaves. This becomes particularly nasty when using multiple levels of libraries that register signal handlers, e.g. running an enclave via cgo inside of the Go runtime. Add per-mm, i.e. per-process, exception fixup on ENCLU to so that the kernel can redirect unhandled exceptions, i.e. exceptions would otherwise generate a signal, to a user-provided exception handler. The exception handler ABI roughly follows the System V 64-bit ABI for function calls: - %rdi: trap number - %rsi: error code - %rdx: address Cc: Andy Lutomirski Cc: Jarkko Sakkinen Cc: Dave Hansen Cc: Josh Triplett Signed-off-by: Sean Christopherson --- arch/x86/include/asm/mmu.h | 4 ++++ arch/x86/include/asm/sgx.h | 13 +++++++++++++ arch/x86/kernel/cpu/sgx/main.c | 18 ++++++++++++++++++ 3 files changed, 35 insertions(+) diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h index 5ff3e8af2c20..1665c84e5844 100644 --- a/arch/x86/include/asm/mmu.h +++ b/arch/x86/include/asm/mmu.h @@ -54,6 +54,10 @@ typedef struct { /* address of the bounds directory */ void __user *bd_addr; #endif +#ifdef CONFIG_INTEL_SGX_CORE + unsigned long enclu_address; + unsigned long enclu_exception_handler; +#endif } mm_context_t; #define INIT_MM_CONTEXT(mm) \ diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h index d4f61d1c5c2a..bbf808a0ca91 100644 --- a/arch/x86/include/asm/sgx.h +++ b/arch/x86/include/asm/sgx.h @@ -309,6 +309,19 @@ static inline int __emodt(struct sgx_secinfo *secinfo, void *addr) return __encls_ret_2(SGX_EMODT, secinfo, addr); } +#ifdef CONFIG_INTEL_SGX_CORE +extern bool fixup_sgx_enclu_exception(struct pt_regs *regs, int trapnr, + unsigned long error_code, + unsigned long fault_addr); +#else +static inline bool fixup_sgx_enclu_exception(struct pt_regs *regs, int trapnr, + unsigned long error_code, + unsigned long fault_addr) +{ + return false; +} +#endif + struct sgx_epc_page *sgx_alloc_page(void *owner, bool reclaim); int __sgx_free_page(struct sgx_epc_page *page); void sgx_free_page(struct sgx_epc_page *page); diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index 30fd69f1fc07..1994e003581d 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -363,6 +363,24 @@ void sgx_page_reclaimable(struct sgx_epc_page *page) } EXPORT_SYMBOL_GPL(sgx_page_reclaimable); +bool fixup_sgx_enclu_exception(struct pt_regs *regs, int trapnr, + unsigned long error_code, + unsigned long fault_addr) +{ + if (current->mm->context.enclu_address != regs->ip) + return false; + + if (!current->mm->context.enclu_address && + !current->mm->context.enclu_exception_handler) + return false; + + regs->ip = current->mm->context.enclu_exception_handler; + regs->di = trapnr; + regs->si = error_code; + regs->dx = fault_addr; + return true; +} + static __init void sgx_free_epc_section(struct sgx_epc_section *section) { struct sgx_epc_page *page; From patchwork Mon Dec 10 23:21:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 10722757 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 85F9618E8 for ; Mon, 10 Dec 2018 23:22:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 762742A710 for ; Mon, 10 Dec 2018 23:22:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A39D2A711; Mon, 10 Dec 2018 23:22:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1210A2A71B for ; Mon, 10 Dec 2018 23:22:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729868AbeLJXWA (ORCPT ); Mon, 10 Dec 2018 18:22:00 -0500 Received: from mga04.intel.com ([192.55.52.120]:11277 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727852AbeLJXV7 (ORCPT ); Mon, 10 Dec 2018 18:21:59 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Dec 2018 15:21:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,340,1539673200"; d="scan'208";a="117684777" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.154]) by orsmga001.jf.intel.com with ESMTP; 10 Dec 2018 15:21:57 -0800 From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, Jarkko Sakkinen , Sean Christopherson , Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org, Andy Lutomirski , Josh Triplett , Haitao Huang , Jethro Beekman , "Dr . Greg Wettstein" Subject: [RFC PATCH v3 2/4] x86/fault: Attempt to fixup unhandled #PF on ENCLU before signaling Date: Mon, 10 Dec 2018 15:21:39 -0800 Message-Id: <20181210232141.5425-3-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181210232141.5425-1-sean.j.christopherson@intel.com> References: <20181210232141.5425-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Call fixup_sgx_enclu_exception() in the SIGSEGV and SIGBUS paths of the page fault handler immediately prior to signaling. If the fault is fixed, return cleanly and do not generate a signal. In the SIGSEGV flow, make sure the error code passed to userspace has been sanitized. Cc: Andy Lutomirski Cc: Jarkko Sakkinen Cc: Dave Hansen Cc: Josh Triplett Signed-off-by: Sean Christopherson --- arch/x86/mm/fault.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 7e8a7558ca07..d3903965334c 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -28,6 +28,7 @@ #include /* vma_pkey() */ #include /* efi_recover_from_page_fault()*/ #include /* store_idt(), ... */ +#include /* fixup_sgx_enclu_exception() */ #define CREATE_TRACE_POINTS #include @@ -928,6 +929,9 @@ __bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code, if (address >= TASK_SIZE_MAX) error_code |= X86_PF_PROT; + if (fixup_sgx_enclu_exception(regs, X86_TRAP_PF, error_code, address)) + return; + if (likely(show_unhandled_signals)) show_signal_msg(regs, error_code, address, tsk); @@ -1045,6 +1049,9 @@ do_sigbus(struct pt_regs *regs, unsigned long error_code, unsigned long address, if (is_prefetch(regs, error_code, address)) return; + if (fixup_sgx_enclu_exception(regs, X86_TRAP_PF, error_code, address)) + return; + set_signal_archinfo(address, error_code); #ifdef CONFIG_MEMORY_FAILURE From patchwork Mon Dec 10 23:21:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 10722755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C08713BF for ; Mon, 10 Dec 2018 23:22:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C0682A710 for ; Mon, 10 Dec 2018 23:22:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D0B72A720; Mon, 10 Dec 2018 23:22:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A23812A710 for ; Mon, 10 Dec 2018 23:22:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729963AbeLJXWA (ORCPT ); Mon, 10 Dec 2018 18:22:00 -0500 Received: from mga04.intel.com ([192.55.52.120]:11282 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729768AbeLJXV7 (ORCPT ); Mon, 10 Dec 2018 18:21:59 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Dec 2018 15:21:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,340,1539673200"; d="scan'208";a="117684783" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.154]) by orsmga001.jf.intel.com with ESMTP; 10 Dec 2018 15:21:57 -0800 From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, Jarkko Sakkinen , Sean Christopherson , Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org, Andy Lutomirski , Josh Triplett , Haitao Huang , Jethro Beekman , "Dr . Greg Wettstein" Subject: [RFC PATCH v3 3/4] x86/traps: Attempt to fixup exceptions in vDSO before signaling Date: Mon, 10 Dec 2018 15:21:40 -0800 Message-Id: <20181210232141.5425-4-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181210232141.5425-1-sean.j.christopherson@intel.com> References: <20181210232141.5425-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Call fixup_sgx_enclu_exception() in all trap flows that generate signals to userspace immediately prior to generating any such signal. If the exception is fixed, return cleanly and don't generate a signal. Cc: Andy Lutomirski Cc: Jarkko Sakkinen Cc: Dave Hansen Cc: Josh Triplett Signed-off-by: Sean Christopherson --- arch/x86/kernel/traps.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 9b7c4ca8f0a7..dd664503803b 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -61,6 +61,7 @@ #include #include #include +#include #ifdef CONFIG_X86_64 #include @@ -223,6 +224,10 @@ do_trap_no_signal(struct task_struct *tsk, int trapnr, const char *str, tsk->thread.error_code = error_code; tsk->thread.trap_nr = trapnr; + if (user_mode(regs) && + fixup_sgx_enclu_exception(regs, trapnr, error_code, 0)) + return 0; + return -1; } @@ -563,6 +568,9 @@ do_general_protection(struct pt_regs *regs, long error_code) tsk->thread.error_code = error_code; tsk->thread.trap_nr = X86_TRAP_GP; + if (fixup_sgx_enclu_exception(regs, X86_TRAP_GP, error_code, 0)) + return; + show_signal(tsk, SIGSEGV, "", desc, regs, error_code); force_sig(SIGSEGV, tsk); @@ -791,6 +799,10 @@ dotraplinkage void do_debug(struct pt_regs *regs, long error_code) goto exit; } + if (user_mode(regs) && + fixup_sgx_enclu_exception(regs, X86_TRAP_DB, error_code, 0)) + goto exit; + if (WARN_ON_ONCE((dr6 & DR_STEP) && !user_mode(regs))) { /* * Historical junk that used to handle SYSENTER single-stepping. @@ -854,6 +866,9 @@ static void math_error(struct pt_regs *regs, int error_code, int trapnr) if (!si_code) return; + if (fixup_sgx_enclu_exception(regs, trapnr, error_code, 0)) + return; + force_sig_fault(SIGFPE, si_code, (void __user *)uprobe_get_trap_addr(regs), task); } From patchwork Mon Dec 10 23:21:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 10722761 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A1C313BF for ; Mon, 10 Dec 2018 23:22:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EBA002A710 for ; Mon, 10 Dec 2018 23:22:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DFA852A71B; Mon, 10 Dec 2018 23:22:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 074422A710 for ; Mon, 10 Dec 2018 23:22:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727919AbeLJXWQ (ORCPT ); Mon, 10 Dec 2018 18:22:16 -0500 Received: from mga04.intel.com ([192.55.52.120]:11277 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729772AbeLJXWA (ORCPT ); Mon, 10 Dec 2018 18:22:00 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Dec 2018 15:21:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,340,1539673200"; d="scan'208";a="117684789" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.154]) by orsmga001.jf.intel.com with ESMTP; 10 Dec 2018 15:21:57 -0800 From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, Jarkko Sakkinen , Sean Christopherson , Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org, Andy Lutomirski , Josh Triplett , Haitao Huang , Jethro Beekman , "Dr . Greg Wettstein" Subject: [RFC PATCH v3 4/4] x86/sgx: Add an SGX IOCTL to register a per-mm ENCLU exception handler Date: Mon, 10 Dec 2018 15:21:41 -0800 Message-Id: <20181210232141.5425-5-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181210232141.5425-1-sean.j.christopherson@intel.com> References: <20181210232141.5425-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Intel Software Guard Extensions (SGX) SGX introduces a new CPL3-only enclave mode that runs as a sort of black box shared object that is hosted by an untrusted normal CPL3 process. Enclave transitions have semantics that are a lovely blend of SYCALL, SYSRET and VM-Exit. In a non-faulting scenario, entering and exiting an enclave can only be done through SGX-specific instructions, EENTER and EEXIT respectively. EENTER+EEXIT is analogous to SYSCALL+SYSRET, e.g. EENTER/SYSCALL load RCX with the next RIP and EEXIT/SYSRET load RIP from R{B,C}X. But in a faulting/interrupting scenario, enclave transitions act more like VM-Exit and VMRESUME. Maintaining the black box nature of the enclave means that hardware must automatically switch CPU context when an Asynchronous Exiting Event (AEE) occurs, an AEE being any interrupt or exception (exceptions are AEEs because asynchronous in this context is relative to the enclave and not CPU execution, e.g. the enclave doesn't get an opportunity to save/fuzz CPU state). Like VM-Exits, all AEEs jump to a common location, referred to as the Asynchronous Exiting Point (AEP). The AEP is specified at enclave entry via register passed to EENTER/ERESUME, similar to how the hypervisor specifies the VM-Exit point (via VMCS.HOST_RIP at VMLAUNCH/VMRESUME). Resuming the enclave/VM after the exiting event is handled is done via ERESUME/VMRESUME respectively. In SGX, AEEs that are handled by the kernel, e.g. INTR, NMI and most page faults, IRET will journey back to the AEP which then ERESUMEs th enclave. Enclaves also behave a bit like VMs in the sense that they can generate exceptions as part of their normal operation that for all intents and purposes need to handled in the enclave/VM. However, unlike VMX, SGX doesn't allow the host to modify its guest's, a.k.a. enclave's, state, as doing so would circumvent the enclave's security. So to handle an exception, the enclave must first be re-entered through the normal EENTER flow (SYSCALL/SYSRET behavior), and then resumed via ERESUME (VMRESUME behavior) after the source of the exception is resolved. All of the above is just the tip of the iceberg when it comes to running an enclave. But, SGX was designed in such a way that the host process can utilize an enclave agnostic library to build, launch and run an enclave. This is roughly analogous to how e.g. normal applications leverage libc implementations and a standardized dynamic linker so that the application on business logic instead of the gory details of system calls, vDSO functions, dynamic linking, etc... However, offloading the heavy lifting to a library comes with a rather large caveat. Because enclaves can generate *and* handle exceptions, SGX libraries must be prepared to handle nearly any exception whenever at least one thread is executing in an enclave. On Linux, this means the SGX library must register a signal handler in order to intercept relevant exceptions and forward them to the enclave (or in some cases, take action on behalf of the enclave). Unfortunately, Linux's signal mechanism doesn't mesh well with libraries, e.g. signal handlers are process wide, are difficult to chain, etc... This becomes particularly nasty when using multiple levels of libraries that register signal handlers, e.g. running an enclave via cgo inside of the Go runtime. Luckily, signals (due to exceptions) can be avoided entirely by taking advantage of several key properties of SGX/enclaves: - Enclaves can only be entered through SGX-specific instructions, and all CPL3 SGX instructions share a single umbrella opcode under the mnemonic ENCLU. - When an event/exception occurs in an enclave, hardware preps the post-exit state so that executing ENCLU will automagically ERESUME the enclave. This means that ENCLU[EENTER] and ENCLU[ERESUME] for an enclave can be the exact same ENCLU instruction. - Exceptions within the enclave appear to the kernel as if they occurred on the AEP, i.e. ENCLU[ERESUME]. - Enclaves are essentially just shared objects with a specialized dynamic linker, so it's not unreasonable to require a process to use a single loader and entry point, i.e. ENCLU, for all enclaves. So, to avoid forcing SGX libraries to juggle signal handlers, provide an IOCTL through /dev/sgx to allow a process to register an exception handler for a single per-mm, i.e. per-process, ENCLU instruction. If an unhandled exception occurs on the ENCLU, i.e. a signal would be generated, load DI, SI and DX with the trap number, error code and faulting address respectively in lieu of generating a signal. Softly enforce the use of the ENCLU handler mechanism by refusing to create enclaves for a process if it has not registered an ENCLU handler. In other words, the only ABI supported by the Linux kernel for handling exceptions on/in enclaves is to register an ENCLU exception handler. Obviously a process can register a dummy handler, but such behavior is NOT officially supported. Cc: Andy Lutomirski Cc: Jarkko Sakkinen Cc: Dave Hansen Cc: Josh Triplett Cc: Haitao Huang Cc: Jethro Beekman Cc: Dr. Greg Wettstein Signed-off-by: Sean Christopherson --- arch/x86/include/uapi/asm/sgx.h | 23 ++++++++++++++++++----- arch/x86/kernel/cpu/sgx/driver/encl.c | 6 ++++++ arch/x86/kernel/cpu/sgx/driver/ioctl.c | 20 ++++++++++++++++++++ 3 files changed, 44 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/uapi/asm/sgx.h b/arch/x86/include/uapi/asm/sgx.h index 266b813eefa1..63bd64e9535d 100644 --- a/arch/x86/include/uapi/asm/sgx.h +++ b/arch/x86/include/uapi/asm/sgx.h @@ -10,20 +10,33 @@ #define SGX_MAGIC 0xA4 +#define SGX_IOC_ENCLU_REGISTER \ + _IOW(SGX_MAGIC, 0x00, struct sgx_enclu_register) #define SGX_IOC_ENCLAVE_CREATE \ - _IOW(SGX_MAGIC, 0x00, struct sgx_enclave_create) + _IOW(SGX_MAGIC, 0x01, struct sgx_enclave_create) #define SGX_IOC_ENCLAVE_ADD_PAGE \ - _IOW(SGX_MAGIC, 0x01, struct sgx_enclave_add_page) + _IOW(SGX_MAGIC, 0x02, struct sgx_enclave_add_page) #define SGX_IOC_ENCLAVE_INIT \ - _IOW(SGX_MAGIC, 0x02, struct sgx_enclave_init) + _IOW(SGX_MAGIC, 0x03, struct sgx_enclave_init) #define SGX_IOC_ENCLAVE_REMOVE_PAGES \ - _IOW(SGX_MAGIC, 0x03, struct sgx_enclave_remove_pages) + _IOW(SGX_MAGIC, 0x04, struct sgx_enclave_remove_pages) #define SGX_IOC_ENCLAVE_MODIFY_PAGES \ - _IOW(SGX_MAGIC, 0x04, struct sgx_enclave_modify_pages) + _IOW(SGX_MAGIC, 0x05, struct sgx_enclave_modify_pages) /* IOCTL return values */ #define SGX_POWER_LOST_ENCLAVE 0x40000000 +/** + * struct sgx_enclu_register - parameter structure for the + * %SGX_IOC_ENCLU_REGISTER ioctl + * @enclu: address of the userspace process' ENCLU instruction + * @handler: address of the userspace process' ENCLU exception handler + */ +struct sgx_enclu_register { + __u64 enclu; + __u64 handler; +}; + /** * struct sgx_enclave_create - parameter structure for the * %SGX_IOC_ENCLAVE_CREATE ioctl diff --git a/arch/x86/kernel/cpu/sgx/driver/encl.c b/arch/x86/kernel/cpu/sgx/driver/encl.c index 61a14cc310f4..ed5df48fba63 100644 --- a/arch/x86/kernel/cpu/sgx/driver/encl.c +++ b/arch/x86/kernel/cpu/sgx/driver/encl.c @@ -525,6 +525,12 @@ int sgx_encl_create(struct sgx_encl *encl, struct sgx_secs *secs) } down_read(¤t->mm->mmap_sem); + if (!current->mm->context.enclu_address && + !current->mm->context.enclu_exception_handler) { + up_read(¤t->mm->mmap_sem); + return -EFAULT; + } + ret = sgx_encl_find(current->mm, secs->base, &vma); if (ret != -ENOENT) { if (!ret) diff --git a/arch/x86/kernel/cpu/sgx/driver/ioctl.c b/arch/x86/kernel/cpu/sgx/driver/ioctl.c index 44edfcd9a6ff..66f2aadd8f0a 100644 --- a/arch/x86/kernel/cpu/sgx/driver/ioctl.c +++ b/arch/x86/kernel/cpu/sgx/driver/ioctl.c @@ -11,6 +11,23 @@ #include #include "driver.h" +static long sgx_ioc_enclu_register(struct file *filep, unsigned int cmd, + unsigned long arg) +{ + struct sgx_enclu_register *reg = (struct sgx_enclu_register *)arg; + + if (reg->enclu == reg->handler) + return -EINVAL; + + if (down_write_killable(¤t->mm->mmap_sem)) + return -EINTR; + current->mm->context.enclu_address = reg->enclu; + current->mm->context.enclu_exception_handler = reg->handler; + up_write(¤t->mm->mmap_sem); + + return 0; +} + static int sgx_encl_get(unsigned long addr, struct sgx_encl **encl) { struct mm_struct *mm = current->mm; @@ -317,6 +334,9 @@ long sgx_ioctl(struct file *filep, unsigned int cmd, unsigned long arg) long ret; switch (cmd) { + case SGX_IOC_ENCLU_REGISTER: + handler = sgx_ioc_enclu_register; + break; case SGX_IOC_ENCLAVE_CREATE: handler = sgx_ioc_enclave_create; break;