| Message ID | e807033e3d56ede1177d7a1af34477678bfbfff9.1611634586.git.kai.huang@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | KVM SGX virtualization support | expand |
On Tue, Jan 26, 2021 at 10:31:06PM +1300, Kai Huang wrote: > From: Sean Christopherson <sean.j.christopherson@intel.com> > > The bare-metal kernel must intercept ECREATE to be able to impose policies > on guests. When it does this, the bare-metal kernel runs ECREATE against > the userspace mapping of the virtualized EPC. I guess Andy's earlier comment applies here, i.e. SGX driver? > > Provide wrappers around __ecreate() and __einit() to hide the ugliness > of overloading the ENCLS return value to encode multiple error formats > in a single int. KVM will trap-and-execute ECREATE and EINIT as part > of SGX virtualization, and on an exception, KVM needs the trapnr so that > it can inject the correct fault into the guest. > > Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com> > Signed-off-by: Kai Huang <kai.huang@intel.com> > --- > v2->v3: > > - Added kdoc for sgx_virt_ecreate() and sgx_virt_einit(), per Jarkko. > - Changed to use CONFIG_X86_SGX_KVM. > > --- > arch/x86/include/asm/sgx.h | 16 ++++++ > arch/x86/kernel/cpu/sgx/virt.c | 93 ++++++++++++++++++++++++++++++++++ > 2 files changed, 109 insertions(+) > create mode 100644 arch/x86/include/asm/sgx.h > > diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h > new file mode 100644 > index 000000000000..8a3ea3e1efbe > --- /dev/null > +++ b/arch/x86/include/asm/sgx.h > @@ -0,0 +1,16 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef _ASM_X86_SGX_H > +#define _ASM_X86_SGX_H > + > +#include <linux/types.h> > + > +#ifdef CONFIG_X86_SGX_KVM > +struct sgx_pageinfo; > + > +int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs, > + int *trapnr); > +int sgx_virt_einit(void __user *sigstruct, void __user *token, > + void __user *secs, u64 *lepubkeyhash, int *trapnr); > +#endif > + > +#endif /* _ASM_X86_SGX_H */ > diff --git a/arch/x86/kernel/cpu/sgx/virt.c b/arch/x86/kernel/cpu/sgx/virt.c > index e1ad7856d878..0f5b0e4e33dd 100644 > --- a/arch/x86/kernel/cpu/sgx/virt.c > +++ b/arch/x86/kernel/cpu/sgx/virt.c > @@ -252,3 +252,96 @@ int __init sgx_vepc_init(void) > > return misc_register(&sgx_vepc_dev); > } > + > +/** > + * sgx_virt_ecreate() - Run ECREATE on behalf of guest > + * @pageinfo: Pointer to PAGEINFO structure > + * @secs: Userspace pointer to SECS page > + * @trapnr: trap number injected to guest in case of ECREATE error > + * > + * Run ECREATE on behalf of guest after KVM traps ECREATE for the purpose > + * of enforcing policies of guest's enclaves, and return the trap number > + * which should be injected to guest in case of any ECREATE error. > + * > + * Return: > + * - 0: ECREATE was successful. > + * - -EFAULT: ECREATE returned error. > + */ > +int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs, > + int *trapnr) > +{ > + int ret; > + > + /* > + * @secs is userspace address, and it's not guaranteed @secs points at > + * an actual EPC page. It's also possible to generate a kernel mapping > + * to physical EPC page by resolving PFN but using __uaccess_xx() is > + * simpler. > + */ > + __uaccess_begin(); > + ret = __ecreate(pageinfo, (void *)secs); > + __uaccess_end(); > + > + if (encls_faulted(ret)) { > + *trapnr = ENCLS_TRAPNR(ret); > + return -EFAULT; > + } > + > + /* ECREATE doesn't return an error code, it faults or succeeds. */ > + WARN_ON_ONCE(ret); > + return 0; > +} > +EXPORT_SYMBOL_GPL(sgx_virt_ecreate); > + > +static int __sgx_virt_einit(void __user *sigstruct, void __user *token, > + void __user *secs) > +{ > + int ret; > + > + __uaccess_begin(); > + ret = __einit((void *)sigstruct, (void *)token, (void *)secs); > + __uaccess_end(); > + return ret; > +} > + > +/** > + * sgx_virt_ecreate() - Run EINIT on behalf of guest > + * @sigstruct: Userspace pointer to SIGSTRUCT structure > + * @token: Userspace pointer to EINITTOKEN structure > + * @secs: Userspace pointer to SECS page > + * @lepubkeyhash: Pointer to guest's *virtual* SGX_LEPUBKEYHASH MSR > + * values > + * @trapnr: trap number injected to guest in case of EINIT error > + * > + * Run EINIT on behalf of guest after KVM traps EINIT. If SGX_LC is available > + * in host, bare-metal driver may rewrite the hardware values, therefore KVM > + * needs to update hardware values to guest's virtual MSR values in order to > + * ensure EINIT is executed with expected hardware values. > + * > + * Return: > + * - 0: EINIT was successful. > + * - -EFAULT: EINIT returned error. > + */ > +int sgx_virt_einit(void __user *sigstruct, void __user *token, > + void __user *secs, u64 *lepubkeyhash, int *trapnr) > +{ > + int ret; > + > + if (!boot_cpu_has(X86_FEATURE_SGX_LC)) { > + ret = __sgx_virt_einit(sigstruct, token, secs); > + } else { > + preempt_disable(); > + > + sgx_update_lepubkeyhash(lepubkeyhash); > + > + ret = __sgx_virt_einit(sigstruct, token, secs); > + preempt_enable(); > + } > + > + if (encls_faulted(ret)) { > + *trapnr = ENCLS_TRAPNR(ret); > + return -EFAULT; > + } Empty line here before return. Applies also to sgx_virt_ecreate(). > + return ret; > +} > +EXPORT_SYMBOL_GPL(sgx_virt_einit); > -- > 2.29.2 Great work. I think this patch sets is shaping up. /Jarkko > >
On Sat, 30 Jan 2021 16:51:41 +0200 Jarkko Sakkinen wrote: > On Tue, Jan 26, 2021 at 10:31:06PM +1300, Kai Huang wrote: > > From: Sean Christopherson <sean.j.christopherson@intel.com> > > > > The bare-metal kernel must intercept ECREATE to be able to impose policies > > on guests. When it does this, the bare-metal kernel runs ECREATE against > > the userspace mapping of the virtualized EPC. > > I guess Andy's earlier comment applies here, i.e. SGX driver? Sure. [...] > > + } > > + > > + if (encls_faulted(ret)) { > > + *trapnr = ENCLS_TRAPNR(ret); > > + return -EFAULT; > > + } > > Empty line here before return. Applies also to sgx_virt_ecreate(). Yes I can remove, but I am just carious: isn't "having empty line before return" a good coding-style? Do you have any reference to the guideline? > > > + return ret; > > +} > > +EXPORT_SYMBOL_GPL(sgx_virt_einit); > > -- > > 2.29.2 > > Great work. I think this patch sets is shaping up. > > /Jarkko > > > >
On Mon, Feb 01, 2021 at 01:17:44PM +1300, Kai Huang wrote: > On Sat, 30 Jan 2021 16:51:41 +0200 Jarkko Sakkinen wrote: > > On Tue, Jan 26, 2021 at 10:31:06PM +1300, Kai Huang wrote: > > > From: Sean Christopherson <sean.j.christopherson@intel.com> > > > > > > The bare-metal kernel must intercept ECREATE to be able to impose policies > > > on guests. When it does this, the bare-metal kernel runs ECREATE against > > > the userspace mapping of the virtualized EPC. > > > > I guess Andy's earlier comment applies here, i.e. SGX driver? > > Sure. > > [...] > > > > + } > > > + > > > + if (encls_faulted(ret)) { > > > + *trapnr = ENCLS_TRAPNR(ret); Also here is an empty line needed. > > > + return -EFAULT; > > > + } > > > > Empty line here before return. Applies also to sgx_virt_ecreate(). > > Yes I can remove, but I am just carious: isn't "having empty line before return" > a good coding-style? Do you have any reference to the guideline? In the initial SGX patch set, this was the review feedback that I got from Boris, so I would presume it is tip tree convention. Also, looking at a random selection of files under arch/x86, it is commonly done this way. > > > > > > + return ret; > > > +} > > > +EXPORT_SYMBOL_GPL(sgx_virt_einit); > > > -- > > > 2.29.2 > > > > Great work. I think this patch sets is shaping up. > > > > /Jarkko > > > > > > > /Jarkko
On Tue, 2 Feb 2021 19:20:54 +0200 Jarkko Sakkinen wrote: > On Mon, Feb 01, 2021 at 01:17:44PM +1300, Kai Huang wrote: > > On Sat, 30 Jan 2021 16:51:41 +0200 Jarkko Sakkinen wrote: > > > On Tue, Jan 26, 2021 at 10:31:06PM +1300, Kai Huang wrote: > > > > From: Sean Christopherson <sean.j.christopherson@intel.com> > > > > > > > > The bare-metal kernel must intercept ECREATE to be able to impose policies > > > > on guests. When it does this, the bare-metal kernel runs ECREATE against > > > > the userspace mapping of the virtualized EPC. > > > > > > I guess Andy's earlier comment applies here, i.e. SGX driver? > > > > Sure. > > > > [...] > > > > > > + } > > > > + > > > > + if (encls_faulted(ret)) { > > > > + *trapnr = ENCLS_TRAPNR(ret); > > Also here is an empty line needed. I honestly don't like putting new line here, since it is just two lines of code. Adding new line is too sparse I think. > > > > > + return -EFAULT; > > > > + } > > > > > > Empty line here before return. Applies also to sgx_virt_ecreate(). > > > > Yes I can remove, but I am just carious: isn't "having empty line before return" > > a good coding-style? Do you have any reference to the guideline? > > In the initial SGX patch set, this was the review feedback that I got > from Boris, so I would presume it is tip tree convention. Also, looking > at a random selection of files under arch/x86, it is commonly done this > way. I'll add a new line here. Sorry I misunderstood your original comment. > > > > > > > > > > + return ret; > > > > +} > > > > +EXPORT_SYMBOL_GPL(sgx_virt_einit); > > > > -- > > > > 2.29.2 > > > > > > Great work. I think this patch sets is shaping up. > > > > > > /Jarkko > > > > > > > > > > > > /Jarkko
Hi Sean, Do you think is it reasonable to move this patch to KVM? sgx_virt_ecreate() can be merged to handle ECREATE patch, and sgx_virt_einit() can be merged to handle EINIT patch. W/o the context of that two patches, it doesn't makes too much sense to have them standalone under x86 here I think. And nobody except KVM will use them. On Tue, 2021-01-26 at 22:31 +1300, Kai Huang wrote: > From: Sean Christopherson <sean.j.christopherson@intel.com> > > The bare-metal kernel must intercept ECREATE to be able to impose policies > on guests. When it does this, the bare-metal kernel runs ECREATE against > the userspace mapping of the virtualized EPC. > > Provide wrappers around __ecreate() and __einit() to hide the ugliness > of overloading the ENCLS return value to encode multiple error formats > in a single int. KVM will trap-and-execute ECREATE and EINIT as part > of SGX virtualization, and on an exception, KVM needs the trapnr so that > it can inject the correct fault into the guest. > > Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com> > Signed-off-by: Kai Huang <kai.huang@intel.com> > --- > v2->v3: > > - Added kdoc for sgx_virt_ecreate() and sgx_virt_einit(), per Jarkko. > - Changed to use CONFIG_X86_SGX_KVM. > > --- > arch/x86/include/asm/sgx.h | 16 ++++++ > arch/x86/kernel/cpu/sgx/virt.c | 93 ++++++++++++++++++++++++++++++++++ > 2 files changed, 109 insertions(+) > create mode 100644 arch/x86/include/asm/sgx.h > > diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h > new file mode 100644 > index 000000000000..8a3ea3e1efbe > --- /dev/null > +++ b/arch/x86/include/asm/sgx.h > @@ -0,0 +1,16 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef _ASM_X86_SGX_H > +#define _ASM_X86_SGX_H > + > +#include <linux/types.h> > + > +#ifdef CONFIG_X86_SGX_KVM > +struct sgx_pageinfo; > + > +int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs, > + int *trapnr); > +int sgx_virt_einit(void __user *sigstruct, void __user *token, > + void __user *secs, u64 *lepubkeyhash, int *trapnr); > +#endif > + > +#endif /* _ASM_X86_SGX_H */ > diff --git a/arch/x86/kernel/cpu/sgx/virt.c b/arch/x86/kernel/cpu/sgx/virt.c > index e1ad7856d878..0f5b0e4e33dd 100644 > --- a/arch/x86/kernel/cpu/sgx/virt.c > +++ b/arch/x86/kernel/cpu/sgx/virt.c > @@ -252,3 +252,96 @@ int __init sgx_vepc_init(void) > > > return misc_register(&sgx_vepc_dev); > } > + > +/** > + * sgx_virt_ecreate() - Run ECREATE on behalf of guest > + * @pageinfo: Pointer to PAGEINFO structure > + * @secs: Userspace pointer to SECS page > + * @trapnr: trap number injected to guest in case of ECREATE error > + * > + * Run ECREATE on behalf of guest after KVM traps ECREATE for the purpose > + * of enforcing policies of guest's enclaves, and return the trap number > + * which should be injected to guest in case of any ECREATE error. > + * > + * Return: > + * - 0: ECREATE was successful. > + * - -EFAULT: ECREATE returned error. > + */ > +int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs, > + int *trapnr) > +{ > + int ret; > + > + /* > + * @secs is userspace address, and it's not guaranteed @secs points at > + * an actual EPC page. It's also possible to generate a kernel mapping > + * to physical EPC page by resolving PFN but using __uaccess_xx() is > + * simpler. > + */ > + __uaccess_begin(); > + ret = __ecreate(pageinfo, (void *)secs); > + __uaccess_end(); > + > + if (encls_faulted(ret)) { > + *trapnr = ENCLS_TRAPNR(ret); > + return -EFAULT; > + } > + > + /* ECREATE doesn't return an error code, it faults or succeeds. */ > + WARN_ON_ONCE(ret); > + return 0; > +} > +EXPORT_SYMBOL_GPL(sgx_virt_ecreate); > + > +static int __sgx_virt_einit(void __user *sigstruct, void __user *token, > + void __user *secs) > +{ > + int ret; > + > + __uaccess_begin(); > + ret = __einit((void *)sigstruct, (void *)token, (void *)secs); > + __uaccess_end(); > + return ret; > +} > + > +/** > + * sgx_virt_ecreate() - Run EINIT on behalf of guest > + * @sigstruct: Userspace pointer to SIGSTRUCT structure > + * @token: Userspace pointer to EINITTOKEN structure > + * @secs: Userspace pointer to SECS page > + * @lepubkeyhash: Pointer to guest's *virtual* SGX_LEPUBKEYHASH MSR > + * values > + * @trapnr: trap number injected to guest in case of EINIT error > + * > + * Run EINIT on behalf of guest after KVM traps EINIT. If SGX_LC is available > + * in host, bare-metal driver may rewrite the hardware values, therefore KVM > + * needs to update hardware values to guest's virtual MSR values in order to > + * ensure EINIT is executed with expected hardware values. > + * > + * Return: > + * - 0: EINIT was successful. > + * - -EFAULT: EINIT returned error. > + */ > +int sgx_virt_einit(void __user *sigstruct, void __user *token, > + void __user *secs, u64 *lepubkeyhash, int *trapnr) > +{ > + int ret; > + > + if (!boot_cpu_has(X86_FEATURE_SGX_LC)) { > + ret = __sgx_virt_einit(sigstruct, token, secs); > + } else { > + preempt_disable(); > + > + sgx_update_lepubkeyhash(lepubkeyhash); > + > + ret = __sgx_virt_einit(sigstruct, token, secs); > + preempt_enable(); > + } > + > + if (encls_faulted(ret)) { > + *trapnr = ENCLS_TRAPNR(ret); > + return -EFAULT; > + } > + return ret; > +} > +EXPORT_SYMBOL_GPL(sgx_virt_einit);
On Thu, Feb 04, 2021, Kai Huang wrote: > Hi Sean, > > Do you think is it reasonable to move this patch to KVM? sgx_virt_ecreate() can be > merged to handle ECREATE patch, and sgx_virt_einit() can be merged to handle EINIT > patch. W/o the context of that two patches, it doesn't makes too much sense to have > them standalone under x86 here I think. And nobody except KVM will use them. Short answer, no. To do that, nearly all of arch/x86/kernel/cpu/sgx/encls.h would need to be exposed via asm/sgx.h. The macro insanity and fault/error code shenanigans really should be kept as private crud in SGX. That's the primary motivation for putting these in sgx/virt.c instead of KVM, my changelog just did a really poor job of explaining that.
> On Thu, Feb 04, 2021, Kai Huang wrote: > > Hi Sean, > > > > Do you think is it reasonable to move this patch to KVM? > > sgx_virt_ecreate() can be merged to handle ECREATE patch, and > > sgx_virt_einit() can be merged to handle EINIT patch. W/o the context > > of that two patches, it doesn't makes too much sense to have them > standalone under x86 here I think. And nobody except KVM will use them. > > Short answer, no. To do that, nearly all of arch/x86/kernel/cpu/sgx/encls.h > would need to be exposed via asm/sgx.h. The macro insanity and fault/error > code shenanigans really should be kept as private crud in SGX. That's the > primary motivation for putting these in sgx/virt.c instead of KVM, my changelog > just did a really poor job of explaining that. OK.
diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h new file mode 100644 index 000000000000..8a3ea3e1efbe --- /dev/null +++ b/arch/x86/include/asm/sgx.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_SGX_H +#define _ASM_X86_SGX_H + +#include <linux/types.h> + +#ifdef CONFIG_X86_SGX_KVM +struct sgx_pageinfo; + +int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs, + int *trapnr); +int sgx_virt_einit(void __user *sigstruct, void __user *token, + void __user *secs, u64 *lepubkeyhash, int *trapnr); +#endif + +#endif /* _ASM_X86_SGX_H */ diff --git a/arch/x86/kernel/cpu/sgx/virt.c b/arch/x86/kernel/cpu/sgx/virt.c index e1ad7856d878..0f5b0e4e33dd 100644 --- a/arch/x86/kernel/cpu/sgx/virt.c +++ b/arch/x86/kernel/cpu/sgx/virt.c @@ -252,3 +252,96 @@ int __init sgx_vepc_init(void) return misc_register(&sgx_vepc_dev); } + +/** + * sgx_virt_ecreate() - Run ECREATE on behalf of guest + * @pageinfo: Pointer to PAGEINFO structure + * @secs: Userspace pointer to SECS page + * @trapnr: trap number injected to guest in case of ECREATE error + * + * Run ECREATE on behalf of guest after KVM traps ECREATE for the purpose + * of enforcing policies of guest's enclaves, and return the trap number + * which should be injected to guest in case of any ECREATE error. + * + * Return: + * - 0: ECREATE was successful. + * - -EFAULT: ECREATE returned error. + */ +int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs, + int *trapnr) +{ + int ret; + + /* + * @secs is userspace address, and it's not guaranteed @secs points at + * an actual EPC page. It's also possible to generate a kernel mapping + * to physical EPC page by resolving PFN but using __uaccess_xx() is + * simpler. + */ + __uaccess_begin(); + ret = __ecreate(pageinfo, (void *)secs); + __uaccess_end(); + + if (encls_faulted(ret)) { + *trapnr = ENCLS_TRAPNR(ret); + return -EFAULT; + } + + /* ECREATE doesn't return an error code, it faults or succeeds. */ + WARN_ON_ONCE(ret); + return 0; +} +EXPORT_SYMBOL_GPL(sgx_virt_ecreate); + +static int __sgx_virt_einit(void __user *sigstruct, void __user *token, + void __user *secs) +{ + int ret; + + __uaccess_begin(); + ret = __einit((void *)sigstruct, (void *)token, (void *)secs); + __uaccess_end(); + return ret; +} + +/** + * sgx_virt_ecreate() - Run EINIT on behalf of guest + * @sigstruct: Userspace pointer to SIGSTRUCT structure + * @token: Userspace pointer to EINITTOKEN structure + * @secs: Userspace pointer to SECS page + * @lepubkeyhash: Pointer to guest's *virtual* SGX_LEPUBKEYHASH MSR + * values + * @trapnr: trap number injected to guest in case of EINIT error + * + * Run EINIT on behalf of guest after KVM traps EINIT. If SGX_LC is available + * in host, bare-metal driver may rewrite the hardware values, therefore KVM + * needs to update hardware values to guest's virtual MSR values in order to + * ensure EINIT is executed with expected hardware values. + * + * Return: + * - 0: EINIT was successful. + * - -EFAULT: EINIT returned error. + */ +int sgx_virt_einit(void __user *sigstruct, void __user *token, + void __user *secs, u64 *lepubkeyhash, int *trapnr) +{ + int ret; + + if (!boot_cpu_has(X86_FEATURE_SGX_LC)) { + ret = __sgx_virt_einit(sigstruct, token, secs); + } else { + preempt_disable(); + + sgx_update_lepubkeyhash(lepubkeyhash); + + ret = __sgx_virt_einit(sigstruct, token, secs); + preempt_enable(); + } + + if (encls_faulted(ret)) { + *trapnr = ENCLS_TRAPNR(ret); + return -EFAULT; + } + return ret; +} +EXPORT_SYMBOL_GPL(sgx_virt_einit);