From patchwork Wed Oct 19 19:14:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haitao Huang X-Patchwork-Id: 13012292 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EF98C4332F for ; Wed, 19 Oct 2022 19:14:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229822AbiJSTOW (ORCPT ); Wed, 19 Oct 2022 15:14:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229994AbiJSTOS (ORCPT ); Wed, 19 Oct 2022 15:14:18 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D40341CFC60 for ; Wed, 19 Oct 2022 12:14:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666206854; x=1697742854; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=00NZWSKJypMTFDjbk4E7RYg9qCKXQ8ThPEks/P3y6NA=; b=FMlwwC20MyUUk5YNdc5xEGikR0gFRkijIFxmyKBbDxJBqDJo3iz+fIVi TnWck5TWCff1+j9ob47Ka0ksVBkNNWfh9y+NXefvLt4kOyx6xz/PfT0ZB SB/5YSYbSkP4IFMb5TgnBNX68juGH83bFQ2fzK+yhdVlIeme/4k31X/xh p9HdvT/1RCGuVYBytJYO+1CDyb80TS2dTWwHL+ODo8J/dr0AwOQeDcrOc 3PNpBgaSrUZcJIzQI0sCucBnyU4FKRl9JFht+Dx06YQPL6g2COGubFTaq hxXkT8Osdklmbtir2tfgGn5yWpwkFVUWUK3eS6myM3IC+D35xmL9iN3Xk Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="286226059" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="286226059" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 12:14:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="874579972" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="874579972" Received: from b4969161e530.jf.intel.com ([10.165.56.46]) by fmsmga006.fm.intel.com with ESMTP; 19 Oct 2022 12:14:14 -0700 From: Haitao Huang To: linux-sgx@vger.kernel.org, jarkko@kernel.org, dave.hansen@linux.intel.com, reinette.chatre@intel.com, vijay.dhanraj@intel.com Subject: [RFC PATCH 1/4] x86/sgx: Export sgx_encl_eaug_page Date: Wed, 19 Oct 2022 12:14:10 -0700 Message-Id: <20221019191413.48752-2-haitao.huang@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019191413.48752-1-haitao.huang@linux.intel.com> References: <20221019191413.48752-1-haitao.huang@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Change return type so it can be reused later for fops->fadvise Signed-off-by: Haitao Huang --- arch/x86/kernel/cpu/sgx/encl.c | 46 ++++++++++++++++++++++------------ arch/x86/kernel/cpu/sgx/encl.h | 3 ++- 2 files changed, 32 insertions(+), 17 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c index 8bdeae2fc309..c57e60d5a0aa 100644 --- a/arch/x86/kernel/cpu/sgx/encl.c +++ b/arch/x86/kernel/cpu/sgx/encl.c @@ -305,11 +305,11 @@ struct sgx_encl_page *sgx_encl_load_page(struct sgx_encl *encl, * on a SGX2 system then the EPC can be added dynamically via the SGX2 * ENCLS[EAUG] instruction. * - * Returns: Appropriate vm_fault_t: VM_FAULT_NOPAGE when PTE was installed - * successfully, VM_FAULT_SIGBUS or VM_FAULT_OOM as error otherwise. + * Returns: 0 when PTE was installed successfully, -EBUSY for waiting on + * reclaimer to free EPC, -ENOMEM for out of RAM, -EFAULT as error otherwise. */ -static vm_fault_t sgx_encl_eaug_page(struct vm_area_struct *vma, - struct sgx_encl *encl, unsigned long addr) +int sgx_encl_eaug_page(struct vm_area_struct *vma, + struct sgx_encl *encl, unsigned long addr) { vm_fault_t vmret = VM_FAULT_SIGBUS; struct sgx_pageinfo pginfo = {0}; @@ -318,10 +318,10 @@ static vm_fault_t sgx_encl_eaug_page(struct vm_area_struct *vma, struct sgx_va_page *va_page; unsigned long phys_addr; u64 secinfo_flags; - int ret; + int ret = -EFAULT; if (!test_bit(SGX_ENCL_INITIALIZED, &encl->flags)) - return VM_FAULT_SIGBUS; + return -EFAULT; /* * Ignore internal permission checking for dynamically added pages. @@ -332,21 +332,21 @@ static vm_fault_t sgx_encl_eaug_page(struct vm_area_struct *vma, secinfo_flags = SGX_SECINFO_R | SGX_SECINFO_W | SGX_SECINFO_X; encl_page = sgx_encl_page_alloc(encl, addr - encl->base, secinfo_flags); if (IS_ERR(encl_page)) - return VM_FAULT_OOM; + return -ENOMEM; mutex_lock(&encl->lock); epc_page = sgx_alloc_epc_page(encl_page, false); if (IS_ERR(epc_page)) { if (PTR_ERR(epc_page) == -EBUSY) - vmret = VM_FAULT_NOPAGE; + ret = -EBUSY; goto err_out_unlock; } va_page = sgx_encl_grow(encl, false); if (IS_ERR(va_page)) { if (PTR_ERR(va_page) == -EBUSY) - vmret = VM_FAULT_NOPAGE; + ret = -EBUSY; goto err_out_epc; } @@ -359,16 +359,20 @@ static vm_fault_t sgx_encl_eaug_page(struct vm_area_struct *vma, * If ret == -EBUSY then page was created in another flow while * running without encl->lock */ - if (ret) + if (ret) { + ret = -EFAULT; goto err_out_shrink; + } pginfo.secs = (unsigned long)sgx_get_epc_virt_addr(encl->secs.epc_page); pginfo.addr = encl_page->desc & PAGE_MASK; pginfo.metadata = 0; ret = __eaug(&pginfo, sgx_get_epc_virt_addr(epc_page)); - if (ret) + if (ret) { + ret = -EFAULT; goto err_out; + } encl_page->encl = encl; encl_page->epc_page = epc_page; @@ -385,10 +389,10 @@ static vm_fault_t sgx_encl_eaug_page(struct vm_area_struct *vma, vmret = vmf_insert_pfn(vma, addr, PFN_DOWN(phys_addr)); if (vmret != VM_FAULT_NOPAGE) { mutex_unlock(&encl->lock); - return VM_FAULT_SIGBUS; + return -EFAULT; } mutex_unlock(&encl->lock); - return VM_FAULT_NOPAGE; + return 0; err_out: xa_erase(&encl->page_array, PFN_DOWN(encl_page->desc)); @@ -401,7 +405,7 @@ static vm_fault_t sgx_encl_eaug_page(struct vm_area_struct *vma, mutex_unlock(&encl->lock); kfree(encl_page); - return vmret; + return ret; } static vm_fault_t sgx_vma_fault(struct vm_fault *vmf) @@ -431,8 +435,18 @@ static vm_fault_t sgx_vma_fault(struct vm_fault *vmf) * enclave that will be checked for right away. */ if (cpu_feature_enabled(X86_FEATURE_SGX2) && - (!xa_load(&encl->page_array, PFN_DOWN(addr)))) - return sgx_encl_eaug_page(vma, encl, addr); + (!xa_load(&encl->page_array, PFN_DOWN(addr)))) { + switch (sgx_encl_eaug_page(vma, encl, addr)) { + case 0: + case -EBUSY: + return VM_FAULT_NOPAGE; + case -ENOMEM: + return VM_FAULT_OOM; + case -EFAULT: + default: + return VM_FAULT_SIGBUS; + } + } mutex_lock(&encl->lock); diff --git a/arch/x86/kernel/cpu/sgx/encl.h b/arch/x86/kernel/cpu/sgx/encl.h index a65a952116fd..36059d35e1bc 100644 --- a/arch/x86/kernel/cpu/sgx/encl.h +++ b/arch/x86/kernel/cpu/sgx/encl.h @@ -127,5 +127,6 @@ struct sgx_encl_page *sgx_encl_load_page(struct sgx_encl *encl, unsigned long addr); struct sgx_va_page *sgx_encl_grow(struct sgx_encl *encl, bool reclaim); void sgx_encl_shrink(struct sgx_encl *encl, struct sgx_va_page *va_page); - +int sgx_encl_eaug_page(struct vm_area_struct *vma, + struct sgx_encl *encl, unsigned long addr); #endif /* _X86_ENCL_H */ From patchwork Wed Oct 19 19:14:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haitao Huang X-Patchwork-Id: 13012294 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D71AC43217 for ; Wed, 19 Oct 2022 19:14:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230032AbiJSTOX (ORCPT ); Wed, 19 Oct 2022 15:14:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229852AbiJSTOS (ORCPT ); Wed, 19 Oct 2022 15:14:18 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69EDA1C712E for ; Wed, 19 Oct 2022 12:14:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666206855; x=1697742855; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=l0niIdzphR8AMTo2kfm6hvWbE8bEM/6edK/h826JwuE=; b=E2Cq/yiVnv3Ywg++XcPEpSrv+DgXfaWC6wN6aCLh71afozCNTYYYLLgE Xcg/SWCY/AjEgCGqJt0WThQ4g1/fDZjiiWCVrWtAvorDaE+LWIQQqIljP SquPOO3vInpiOHHK9UKZzpreD5N2UEcceqip2zJKGoVKlXzfXVm4rQfhh 5CKEg6GbxlyPcLdTQLo0RAPEAvnJIpPg1Cl47dYbaUeLGReYk4gcLuZLr GQmoPw+U+JV1iXHn0uiaFSIR5Llc5xjjdnJFWlwjr7Sk576JUsiAKTbb9 KU5wpHOA/Ucm5DL2UpTLPxmxvNZBMNqu/dMwXrYKuOC9/D6DTQ+K1aNIR w==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="286226061" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="286226061" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 12:14:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="874579978" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="874579978" Received: from b4969161e530.jf.intel.com ([10.165.56.46]) by fmsmga006.fm.intel.com with ESMTP; 19 Oct 2022 12:14:14 -0700 From: Haitao Huang To: linux-sgx@vger.kernel.org, jarkko@kernel.org, dave.hansen@linux.intel.com, reinette.chatre@intel.com, vijay.dhanraj@intel.com Subject: [RFC PATCH 2/4] x86/sgx: Implement support for MADV_WILLNEED Date: Wed, 19 Oct 2022 12:14:11 -0700 Message-Id: <20221019191413.48752-3-haitao.huang@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019191413.48752-2-haitao.huang@linux.intel.com> References: <20221019191413.48752-1-haitao.huang@linux.intel.com> <20221019191413.48752-2-haitao.huang@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Add support for madvise(..., MADV_WILLNEED) by adding pages with EAUG. Implement fops->fadvise() callback to achieve this behaviour. Note this is only done with best effort possible. If any errors encountered or EPC is under swapping, the operation will stop and return as normal. Signed-off-by: Haitao Huang --- arch/x86/kernel/cpu/sgx/driver.c | 81 ++++++++++++++++++++++++++++++++ 1 file changed, 81 insertions(+) diff --git a/arch/x86/kernel/cpu/sgx/driver.c b/arch/x86/kernel/cpu/sgx/driver.c index aa9b8b868867..54b24897605b 100644 --- a/arch/x86/kernel/cpu/sgx/driver.c +++ b/arch/x86/kernel/cpu/sgx/driver.c @@ -2,6 +2,7 @@ /* Copyright(c) 2016-20 Intel Corporation. */ #include +#include #include #include #include @@ -9,6 +10,7 @@ #include #include "driver.h" #include "encl.h" +#include "encls.h" u64 sgx_attributes_reserved_mask; u64 sgx_xfrm_reserved_mask = ~0x3; @@ -97,10 +99,88 @@ static int sgx_mmap(struct file *file, struct vm_area_struct *vma) vma->vm_ops = &sgx_vm_ops; vma->vm_flags |= VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP | VM_IO; vma->vm_private_data = encl; + /* Anchor vm_pgoff to the enclave base. + * So offset passed back to sgx_fadvise hook + * is relative to the enclave base + */ + vma->vm_pgoff = (vma->vm_start - encl->base) >> PAGE_SHIFT; return 0; } +/* + * Add new pages to the enclave sequentially with ENCLS[EAUG] for the WILLNEED advice. + * Only do this to existing VMAs in the same enclave and reject the request. + * Returns: 0 if EAUG done with best effort, -EINVAL if any sub-range given + * is not in the enclave, or enclave is not initialized.. + */ +static int sgx_fadvise(struct file *file, loff_t offset, loff_t len, int advice) +{ + struct sgx_encl *encl = file->private_data; + unsigned long start, end, pos; + int ret = -EINVAL; + struct vm_area_struct *vma = NULL; + + /* Only support WILLNEED */ + if (advice != POSIX_FADV_WILLNEED) + return -EINVAL; + if (!encl) + return -EINVAL; + if (!cpu_feature_enabled(X86_FEATURE_SGX2)) + return -EINVAL; + + if (offset + len < offset) + return -EINVAL; + if (encl->base + offset < encl->base) + return -EINVAL; + start = offset + encl->base; + end = start + len; + if (end < start) + return -EINVAL; + if (end > encl->base + encl->size) + return -EINVAL; + + /* EAUG works only for initialized enclaves. */ + if (!test_bit(SGX_ENCL_INITIALIZED, &encl->flags)) + return -EINVAL; + + mmap_read_lock(current->mm); + + vma = find_vma(current->mm, start); + if (!vma) + goto unlock; + if (vma->vm_private_data != encl) + goto unlock; + + pos = start; + if (pos < vma->vm_start || end > vma->vm_end) { + /* Don't allow any gaps */ + goto unlock; + } + /* Here: vm_start <= pos < end <= vm_end */ + while (pos < end) { + if (xa_load(&encl->page_array, PFN_DOWN(pos))) + continue; + if (signal_pending(current)) { + if (pos == start) + ret = -ERESTARTSYS; + else + ret = -EINTR; + goto unlock; + } + ret = sgx_encl_eaug_page(vma, encl, pos); + /* It's OK to not finish */ + if (ret) + break; + pos = pos + PAGE_SIZE; + cond_resched(); + } + ret = 0; +unlock: + mmap_read_unlock(current->mm); + return ret; +} + static unsigned long sgx_get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, @@ -133,6 +213,7 @@ static const struct file_operations sgx_encl_fops = { .compat_ioctl = sgx_compat_ioctl, #endif .mmap = sgx_mmap, + .fadvise = sgx_fadvise, .get_unmapped_area = sgx_get_unmapped_area, }; From patchwork Wed Oct 19 19:14:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haitao Huang X-Patchwork-Id: 13012293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C579DC43219 for ; Wed, 19 Oct 2022 19:14:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229923AbiJSTOW (ORCPT ); Wed, 19 Oct 2022 15:14:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230032AbiJSTOS (ORCPT ); Wed, 19 Oct 2022 15:14:18 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97D781D374E for ; Wed, 19 Oct 2022 12:14:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666206855; x=1697742855; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=U/iSupQd/iALKhWtingvKOvxMzE6VmW0vA3EATo7ydU=; b=jRT4EXTeYYjbF+yf8wSa0DLS3PnV5M68GvDdbX/s2BiyHUXXvUyYZhb7 AJeyaTzeqwX7HX1NS4feyu8I57M8Xsr0Ihzb7aNDJs2GHcDxAcGCiXUGP Hyl6zwfHWMoQT7v6V/ZOql2r6cm8KKh2Ij3+rFmc4eREj8xaQfm5vzjG8 Qs3SLqd2lsfm2c9x4P65qx0g4HljwuAC8VYrn4iAoWEtt0+eV1wQJXR5/ Omp2aYh69pHv4+fg1cWQ7vP7msziLNfDADD6HLw6EVI/jqjgib1XFDFKN M7xeXq0iSZjEpyOsQFSjjZ+b9vPb2XzMBpuaJCwob7t1u04yvG/vdfXBL A==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="286226064" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="286226064" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 12:14:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="874579984" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="874579984" Received: from b4969161e530.jf.intel.com ([10.165.56.46]) by fmsmga006.fm.intel.com with ESMTP; 19 Oct 2022 12:14:14 -0700 From: Haitao Huang To: linux-sgx@vger.kernel.org, jarkko@kernel.org, dave.hansen@linux.intel.com, reinette.chatre@intel.com, vijay.dhanraj@intel.com Subject: [RFC PATCH 3/4] selftests/sgx: add len field for EACCEPT op Date: Wed, 19 Oct 2022 12:14:12 -0700 Message-Id: <20221019191413.48752-4-haitao.huang@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019191413.48752-3-haitao.huang@linux.intel.com> References: <20221019191413.48752-1-haitao.huang@linux.intel.com> <20221019191413.48752-2-haitao.huang@linux.intel.com> <20221019191413.48752-3-haitao.huang@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org So we can EACCEPT multiple pages inside enclave without EEXIT, preparing for testing with MADV_WILLNEED for ranges bigger than a single page. Signed-off-by: Haitao Huang --- tools/testing/selftests/sgx/defines.h | 1 + tools/testing/selftests/sgx/test_encl.c | 20 ++++++++++++++------ 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/sgx/defines.h b/tools/testing/selftests/sgx/defines.h index d8587c971941..8578e773d3d8 100644 --- a/tools/testing/selftests/sgx/defines.h +++ b/tools/testing/selftests/sgx/defines.h @@ -60,6 +60,7 @@ struct encl_op_eaccept { struct encl_op_header header; uint64_t epc_addr; uint64_t flags; + uint64_t len; uint64_t ret; }; diff --git a/tools/testing/selftests/sgx/test_encl.c b/tools/testing/selftests/sgx/test_encl.c index c0d6397295e3..fc797385200b 100644 --- a/tools/testing/selftests/sgx/test_encl.c +++ b/tools/testing/selftests/sgx/test_encl.c @@ -35,14 +35,22 @@ static void do_encl_eaccept(void *_op) struct sgx_secinfo secinfo __aligned(sizeof(struct sgx_secinfo)) = {0}; struct encl_op_eaccept *op = _op; int rax; + if (op->len == 0) + op->len = 4096; secinfo.flags = op->flags; - - asm volatile(".byte 0x0f, 0x01, 0xd7" - : "=a" (rax) - : "a" (EACCEPT), - "b" (&secinfo), - "c" (op->epc_addr)); + for (uint64_t addr = op->epc_addr; + addr < op->epc_addr + op->len; addr += 4096) { + asm volatile(".byte 0x0f, 0x01, 0xd7" + : "=a" (rax) + : "a" (EACCEPT), + "b" (&secinfo), + "c" (addr)); + if (rax) { + op->ret = rax; + return; + } + } op->ret = rax; } From patchwork Wed Oct 19 19:14:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haitao Huang X-Patchwork-Id: 13012295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 029E0C4332F for ; Wed, 19 Oct 2022 19:14:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229785AbiJSTOZ (ORCPT ); Wed, 19 Oct 2022 15:14:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230071AbiJSTOT (ORCPT ); Wed, 19 Oct 2022 15:14:19 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D71D71D3773 for ; Wed, 19 Oct 2022 12:14:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666206855; x=1697742855; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=KWnJFJGkfA7A5GGsnzDAK63MfJBiJ4YYGRMcSk6UlFk=; b=ddxoSQu2asfu0AEQBxzIri06aJb70hAkVpvXZcg2ftHGdoXzXYpXVchl JpSHtO8rt5zksmi4Oy0oaqrWCLvtM+T+u9IRVR6s6fH2oTPKXb0fMiJ+e s8ks46QcA4rZNdzgwe8snhg5wbo8rbmMgBOz4g3AiGMTxpyo9WPGMEi6V 02PWNswFJyd4EdPVhwQmXakGyXlOl1vcbM0tEWz8ST1Y+IUbEXSQipVu5 N9vChimoJVA6WDuEWQphTJ7FI78E/stx08hEy3dlcRf5TZWdpQql0Qufl XFXxzjSEodd25GdSwRYxHq8h5SpYkBxRERW+158goyZamSPYWXwc+9xIK A==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="286226069" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="286226069" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 12:14:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="874579988" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="874579988" Received: from b4969161e530.jf.intel.com ([10.165.56.46]) by fmsmga006.fm.intel.com with ESMTP; 19 Oct 2022 12:14:15 -0700 From: Haitao Huang To: linux-sgx@vger.kernel.org, jarkko@kernel.org, dave.hansen@linux.intel.com, reinette.chatre@intel.com, vijay.dhanraj@intel.com Subject: [RFC PATCH 4/4] selftests/sgx: Add test for madvise(..., WILLNEED) Date: Wed, 19 Oct 2022 12:14:13 -0700 Message-Id: <20221019191413.48752-5-haitao.huang@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019191413.48752-4-haitao.huang@linux.intel.com> References: <20221019191413.48752-1-haitao.huang@linux.intel.com> <20221019191413.48752-2-haitao.huang@linux.intel.com> <20221019191413.48752-3-haitao.huang@linux.intel.com> <20221019191413.48752-4-haitao.huang@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Measure and compare run time for EAUG'ing different number of EPC pages with/without madvise(..., WILLNEED) call. Signed-off-by: Haitao Huang --- tools/testing/selftests/sgx/main.c | 167 +++++++++++++++++++++++++++++ 1 file changed, 167 insertions(+) diff --git a/tools/testing/selftests/sgx/main.c b/tools/testing/selftests/sgx/main.c index 48976bb7bd79..7b5f6705716d 100644 --- a/tools/testing/selftests/sgx/main.c +++ b/tools/testing/selftests/sgx/main.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -1358,6 +1359,172 @@ TEST_F_TIMEOUT(enclave, augment_via_eaccept_long, TIMEOUT_DEFAULT) munmap(addr, ENCL_DYNAMIC_SIZE_LONG); } +static int eaccept_range(struct _test_data_enclave *self, void *addr, + unsigned long size, uint64_t flags, + struct __test_metadata *_metadata) +{ + struct encl_op_eaccept eaccept_op; + + self->run.exception_vector = 0; + self->run.exception_error_code = 0; + self->run.exception_addr = 0; + + /* + * Run EACCEPT on every page to trigger the #PF->EAUG->EACCEPT(again + * without a #PF). All should be transparent to userspace. + */ + eaccept_op.flags = flags; + eaccept_op.ret = 0; + eaccept_op.header.type = ENCL_OP_EACCEPT; + eaccept_op.len = size; + eaccept_op.epc_addr = (uint64_t)(addr); + + EXPECT_EQ(ENCL_CALL(&eaccept_op, &self->run, true), 0); + + EXPECT_EQ(self->run.exception_vector, 0); + EXPECT_EQ(self->run.exception_error_code, 0); + EXPECT_EQ(self->run.exception_addr, 0); + ASSERT_EQ(eaccept_op.ret, 0); + ASSERT_EQ(self->run.function, EEXIT); + + return 0; +} + +static int trim_remove_range(struct _test_data_enclave *self, void *addr, + unsigned long size, struct __test_metadata *_metadata) +{ + int ret, errno_save; + struct sgx_enclave_remove_pages remove_ioc; + struct sgx_enclave_modify_types modt_ioc; + unsigned long offset; + unsigned long count; + + if ((uint64_t)addr <= self->encl.encl_base) + return -1; + offset = (uint64_t)addr - self->encl.encl_base; + + memset(&modt_ioc, 0, sizeof(modt_ioc)); + modt_ioc.offset = offset; + modt_ioc.length = size; + modt_ioc.page_type = SGX_PAGE_TYPE_TRIM; + count = 0; + do { + ret = ioctl(self->encl.fd, SGX_IOC_ENCLAVE_MODIFY_TYPES, &modt_ioc); + + errno_save = ret == -1 ? errno : 0; + if (errno_save != EAGAIN) + break; + EXPECT_EQ(modt_ioc.result, 0); + + count += modt_ioc.count; + modt_ioc.offset += modt_ioc.count; + modt_ioc.length -= modt_ioc.count; + modt_ioc.result = 0; + modt_ioc.count = 0; + } while (modt_ioc.length != 0); + + EXPECT_EQ(ret, 0); + EXPECT_EQ(errno_save, 0); + EXPECT_EQ(modt_ioc.result, 0); + count += modt_ioc.count; + EXPECT_EQ(count, size); + + EXPECT_EQ(eaccept_range(self, addr, size, + SGX_SECINFO_TRIM | SGX_SECINFO_MODIFIED, + _metadata), 0); + + /* Complete page removal. */ + memset(&remove_ioc, 0, sizeof(remove_ioc)); + remove_ioc.offset = offset; + remove_ioc.length = size; + count = 0; + do { + ret = ioctl(self->encl.fd, SGX_IOC_ENCLAVE_REMOVE_PAGES, &remove_ioc); + + errno_save = ret == -1 ? errno : 0; + if (errno_save != EAGAIN) + break; + + count += remove_ioc.count; + remove_ioc.offset += remove_ioc.count; + remove_ioc.length -= remove_ioc.count; + remove_ioc.count = 0; + } while (remove_ioc.length != 0); + + EXPECT_EQ(ret, 0); + EXPECT_EQ(errno_save, 0); + count += remove_ioc.count; + EXPECT_EQ(count, size); + + return 0; +} + +/* + * Compare performance with and without madvise call before EACCEPT'ing + * different size of regions. + */ +TEST_F_TIMEOUT(enclave, augment_via_madvise, TIMEOUT_DEFAULT) +{ + unsigned long advise_size = PAGE_SIZE; + unsigned long max_advise_size = get_total_epc_mem() * 3UL; + int speed_up_percent; + clock_t start; + double time_used1, time_used2; + size_t total_size = 0; + unsigned long i; + void *addr; + + if (!sgx2_supported()) + SKIP(return, "SGX2 not supported"); + + ASSERT_TRUE(setup_test_encl_dynamic(ENCL_HEAP_SIZE_DEFAULT, + max_advise_size, &self->encl, _metadata)); + + memset(&self->run, 0, sizeof(self->run)); + self->run.tcs = self->encl.encl_base; + + for (i = 0; i < self->encl.nr_segments; i++) { + struct encl_segment *seg = &self->encl.segment_tbl[i]; + + total_size += seg->size; + } + + for (i = 1; i < 52 && advise_size < max_advise_size; i++) { + addr = mmap((void *)self->encl.encl_base + total_size, advise_size, + PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, + self->encl.fd, 0); + EXPECT_NE(addr, MAP_FAILED); + + start = clock(); + EXPECT_EQ(eaccept_range(self, addr, advise_size, + SGX_SECINFO_R | SGX_SECINFO_W + | SGX_SECINFO_REG + | SGX_SECINFO_PENDING, + _metadata), 0); + time_used1 = (double)clock() - start; + + EXPECT_EQ(trim_remove_range(self, addr, advise_size, _metadata), 0); + + start = clock(); + EXPECT_EQ(madvise(addr, advise_size, MADV_WILLNEED), 0); + EXPECT_EQ(eaccept_range(self, addr, advise_size, + SGX_SECINFO_R | SGX_SECINFO_W + | SGX_SECINFO_REG + | SGX_SECINFO_PENDING, + _metadata), 0); + time_used2 = (double)clock() - start; + + speed_up_percent = (int)((time_used1 - time_used2) / time_used1 * 100); + TH_LOG("madvise speed up for eaug'ing %10ld pages: %d%%", + advise_size / PAGE_SIZE, speed_up_percent); + EXPECT_GE(speed_up_percent, 0); + EXPECT_EQ(trim_remove_range(self, addr, advise_size, _metadata), 0); + munmap(addr, advise_size); + advise_size = (advise_size << 1UL); + } + encl_delete(&self->encl); +} + /* * SGX2 page type modification test in two phases: * Phase 1: