From patchwork Sun Dec 4 18:40:41 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 9460105 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D462660231 for ; Sun, 4 Dec 2016 18:41:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA6F42465D for ; Sun, 4 Dec 2016 18:41:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BF93124B91; Sun, 4 Dec 2016 18:41:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7CA6A2465D for ; Sun, 4 Dec 2016 18:41:18 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 9334A81FC3 for ; Sun, 4 Dec 2016 10:41:18 -0800 (PST) X-Original-To: intel-sgx-kernel-dev@lists.01.org Delivered-To: intel-sgx-kernel-dev@lists.01.org Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id D6C7681FC3 for ; Sun, 4 Dec 2016 10:41:17 -0800 (PST) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP; 04 Dec 2016 10:41:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.33,743,1477983600"; d="scan'208"; a="1067835749" Received: from jcortizr-mobl1.ger.corp.intel.com (HELO localhost) ([10.252.2.177]) by orsmga001.jf.intel.com with ESMTP; 04 Dec 2016 10:41:15 -0800 From: Jarkko Sakkinen To: intel-sgx-kernel-dev@lists.01.org Date: Sun, 4 Dec 2016 20:40:41 +0200 Message-Id: <20161204184044.21031-6-jarkko.sakkinen@linux.intel.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20161204184044.21031-1-jarkko.sakkinen@linux.intel.com> References: <20161204184044.21031-1-jarkko.sakkinen@linux.intel.com> Subject: [intel-sgx-kernel-dev] [PATCH v6 5/8] intel_sgx: lock the enclave for the entire EPC eviction flow X-BeenThere: intel-sgx-kernel-dev@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Project: Intel® Software Guard Extensions for Linux*: https://01.org/intel-software-guard-extensions" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-sgx-kernel-dev-bounces@lists.01.org Sender: "intel-sgx-kernel-dev" X-Virus-Scanned: ClamAV using ClamSMTP From: Sean Christopherson The SGX architecture does not allow multiple ETRACK sequences to be in flight at the same time if the enclave being tracked has one or more active threads in the enclave. A second ETRACK will fail due to PREV_TRK_INCMPL if it is attempted before EWB is executed on all EBLOCKed pages (in the presence of active enclave threads). Currently the driver releases the enclave's lock after ETRACK and then reacquires the lock prior to EWB. Releasing the lock from the thread that performed ETRACK, T1, allows a different thread, T2, to acquire the lock for its own ETRACK. T2's ETRACK will fail due to the architectural restrictions if a third thread, T3, is executing inside the enclave whose pages are being swapped. T3: T1: lock_mutex(E1) T1: ETRACK(E1) T1: unlock_mutex(E1) T2: lock_mutex(E1) T2: ETRACK(E1) fails ETRACK failing is not fatal in and of itself, but the driver is not designed to handle an ETRACK failure (BUG_ON non-zero return code). Rather than modifying the driver to handle the failing path, e.g by kicking T3 out of the enclave with an IPI, simply lock the enclave for the entire ETRACK->EWB sequence to prevent multiple concurrent ETRACKs on a single enclave. Holding the lock for ETRACK->EWB does not negatively impact performance in any way. Signed-off-by: Sean Christopherson Reviewed-by: Jarkko Sakkinen Tested-by: Jarkko Sakkinen --- drivers/platform/x86/intel_sgx_page_cache.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/platform/x86/intel_sgx_page_cache.c b/drivers/platform/x86/intel_sgx_page_cache.c index 0f63060..f2a2ed1 100644 --- a/drivers/platform/x86/intel_sgx_page_cache.c +++ b/drivers/platform/x86/intel_sgx_page_cache.c @@ -280,15 +280,14 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) return; } - /* EBLOCK */ + mutex_lock(&encl->lock); + /* EBLOCK */ list_for_each_entry_safe(entry, tmp, src, load_list) { - mutex_lock(&encl->lock); evma = sgx_find_vma(encl, entry->addr); if (!evma) { list_del(&entry->load_list); sgx_free_encl_page(entry, encl, 0); - mutex_unlock(&encl->lock); continue; } @@ -297,25 +296,18 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) list_del(&entry->load_list); list_add_tail(&entry->load_list, &encl->load_list); entry->flags &= ~SGX_ENCL_PAGE_RESERVED; - mutex_unlock(&encl->lock); continue; } zap_vma_ptes(evma, entry->addr, PAGE_SIZE); sgx_eblock(entry->epc_page); cnt++; - mutex_unlock(&encl->lock); } /* ETRACK */ - - mutex_lock(&encl->lock); sgx_etrack(encl->secs_page.epc_page); - mutex_unlock(&encl->lock); /* EWB */ - - mutex_lock(&encl->lock); i = 0; while (!list_empty(src)) {