From patchwork Thu Dec 15 14:49:54 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 9476371 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C955C60825 for ; Thu, 15 Dec 2016 14:50:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C36842877D for ; Thu, 15 Dec 2016 14:50:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B6D722879D; Thu, 15 Dec 2016 14:50:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A1025287A6 for ; Thu, 15 Dec 2016 14:50:23 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 726A281D58 for ; Thu, 15 Dec 2016 06:50:23 -0800 (PST) X-Original-To: intel-sgx-kernel-dev@lists.01.org Delivered-To: intel-sgx-kernel-dev@lists.01.org Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 09FBD81FB1 for ; Thu, 15 Dec 2016 06:50:22 -0800 (PST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga102.fm.intel.com with ESMTP; 15 Dec 2016 06:50:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,352,1477983600"; d="scan'208";a="18524232" Received: from gstoettn-mobl2.ger.corp.intel.com (HELO localhost) ([10.252.55.163]) by orsmga002.jf.intel.com with ESMTP; 15 Dec 2016 06:50:19 -0800 From: Jarkko Sakkinen To: intel-sgx-kernel-dev@lists.01.org Date: Thu, 15 Dec 2016 16:49:54 +0200 Message-Id: <20161215144959.31245-6-jarkko.sakkinen@linux.intel.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20161215144959.31245-1-jarkko.sakkinen@linux.intel.com> References: <20161215144959.31245-1-jarkko.sakkinen@linux.intel.com> Subject: [intel-sgx-kernel-dev] [PATCH v9 05/10] intel_sgx: lock the enclave for the entire EPC eviction flow X-BeenThere: intel-sgx-kernel-dev@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Project: Intel® Software Guard Extensions for Linux*: https://01.org/intel-software-guard-extensions" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-sgx-kernel-dev-bounces@lists.01.org Sender: "intel-sgx-kernel-dev" X-Virus-Scanned: ClamAV using ClamSMTP From: Sean Christopherson The SGX architecture does not allow multiple ETRACK sequences to be in flight at the same time if the enclave being tracked has one or more active threads in the enclave. A second ETRACK will fail due to PREV_TRK_INCMPL if it is attempted before EWB is executed on all EBLOCKed pages (in the presence of active enclave threads). Currently the driver releases the enclave's lock after ETRACK and then reacquires the lock prior to EWB. Releasing the lock from the thread that performed ETRACK, T1, allows a different thread, T2, to acquire the lock for its own ETRACK. T2's ETRACK will fail due to the architectural restrictions if a third thread, T3, is executing inside the enclave whose pages are being swapped. T3: T1: lock_mutex(E1) T1: ETRACK(E1) T1: unlock_mutex(E1) T2: lock_mutex(E1) T2: ETRACK(E1) fails ETRACK failing is not fatal in and of itself, but the driver is not designed to handle an ETRACK failure (BUG_ON non-zero return code). Rather than modifying the driver to handle the failing path, e.g by kicking T3 out of the enclave with an IPI, simply lock the enclave for the entire ETRACK->EWB sequence to prevent multiple concurrent ETRACKs on a single enclave. Holding the lock for ETRACK->EWB does not negatively impact performance in any way. Signed-off-by: Sean Christopherson Reviewed-by: Jarkko Sakkinen Tested-by: Jarkko Sakkinen --- drivers/platform/x86/intel_sgx_page_cache.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/platform/x86/intel_sgx_page_cache.c b/drivers/platform/x86/intel_sgx_page_cache.c index 691c77a..d564f22 100644 --- a/drivers/platform/x86/intel_sgx_page_cache.c +++ b/drivers/platform/x86/intel_sgx_page_cache.c @@ -281,15 +281,14 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) return; } - /* EBLOCK */ + mutex_lock(&encl->lock); + /* EBLOCK */ list_for_each_entry_safe(entry, tmp, src, load_list) { - mutex_lock(&encl->lock); evma = sgx_find_vma(encl, entry->addr); if (!evma) { list_del(&entry->load_list); sgx_free_encl_page(entry, encl, 0); - mutex_unlock(&encl->lock); continue; } @@ -298,25 +297,18 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) list_del(&entry->load_list); list_add_tail(&entry->load_list, &encl->load_list); entry->flags &= ~SGX_ENCL_PAGE_RESERVED; - mutex_unlock(&encl->lock); continue; } zap_vma_ptes(evma, entry->addr, PAGE_SIZE); sgx_eblock(entry->epc_page); cnt++; - mutex_unlock(&encl->lock); } /* ETRACK */ - - mutex_lock(&encl->lock); sgx_etrack(encl->secs_page.epc_page); - mutex_unlock(&encl->lock); /* EWB */ - - mutex_lock(&encl->lock); i = 0; while (!list_empty(src)) {