diff mbox series

[v2,07/25] x86/sgx: Initialize virtual EPC driver even when SGX driver is disabled

Message ID d2ebcffeb9193d26a1305e08fe1aa1347feb1c62.1615250634.git.kai.huang@intel.com (mailing list archive)
State New, archived
Headers show
Series KVM SGX virtualization support | expand

Commit Message

Huang, Kai March 9, 2021, 1:39 a.m. UTC
Modify sgx_init() to always try to initialize the virtual EPC driver,
even if the SGX driver is disabled.  The SGX driver might be disabled
if SGX Launch Control is in locked mode, or not supported in the
hardware at all.  This allows (non-Linux) guests that support non-LC
configurations to use SGX.

Acked-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Kai Huang <kai.huang@intel.com>
---
 arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

Comments

Sean Christopherson March 12, 2021, 9:44 p.m. UTC | #1
On Tue, Mar 09, 2021, Kai Huang wrote:
> Modify sgx_init() to always try to initialize the virtual EPC driver,
> even if the SGX driver is disabled.  The SGX driver might be disabled
> if SGX Launch Control is in locked mode, or not supported in the
> hardware at all.  This allows (non-Linux) guests that support non-LC
> configurations to use SGX.
> 
> Acked-by: Dave Hansen <dave.hansen@intel.com>
> Signed-off-by: Kai Huang <kai.huang@intel.com>
> ---
>  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> index 44fe91a5bfb3..8c922e68274d 100644
> --- a/arch/x86/kernel/cpu/sgx/main.c
> +++ b/arch/x86/kernel/cpu/sgx/main.c
> @@ -712,7 +712,15 @@ static int __init sgx_init(void)
>  		goto err_page_cache;
>  	}
>  
> -	ret = sgx_drv_init();
> +	/*
> +	 * Always try to initialize the native *and* KVM drivers.
> +	 * The KVM driver is less picky than the native one and
> +	 * can function if the native one is not supported on the
> +	 * current system or fails to initialize.
> +	 *
> +	 * Error out only if both fail to initialize.
> +	 */
> +	ret = !!sgx_drv_init() & !!sgx_vepc_init();

I love this code.

Reviewed-by: Sean Christopherson <seanjc@google.com>

>  	if (ret)
>  		goto err_kthread;
>  
> -- 
> 2.29.2
>
Jarkko Sakkinen March 13, 2021, 7:05 p.m. UTC | #2
On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> On Tue, Mar 09, 2021, Kai Huang wrote:
> > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > even if the SGX driver is disabled.  The SGX driver might be disabled
> > if SGX Launch Control is in locked mode, or not supported in the
> > hardware at all.  This allows (non-Linux) guests that support non-LC
> > configurations to use SGX.
> > 
> > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > ---
> >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> >  1 file changed, 9 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > index 44fe91a5bfb3..8c922e68274d 100644
> > --- a/arch/x86/kernel/cpu/sgx/main.c
> > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> >  		goto err_page_cache;
> >  	}
> >  
> > -	ret = sgx_drv_init();
> > +	/*
> > +	 * Always try to initialize the native *and* KVM drivers.
> > +	 * The KVM driver is less picky than the native one and
> > +	 * can function if the native one is not supported on the
> > +	 * current system or fails to initialize.
> > +	 *
> > +	 * Error out only if both fail to initialize.
> > +	 */
> > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> 
> I love this code.
> 
> Reviewed-by: Sean Christopherson <seanjc@google.com>

I'm still wondering why this code let's go through when sgx_drv_init()
succeeds and sgx_vepc_init() fails.

The inline comment explains only the mirrored case (which does make
sense).

/Jarkko
Jarkko Sakkinen March 13, 2021, 7:07 p.m. UTC | #3
On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > if SGX Launch Control is in locked mode, or not supported in the
> > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > configurations to use SGX.
> > > 
> > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > ---
> > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > index 44fe91a5bfb3..8c922e68274d 100644
> > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > >  		goto err_page_cache;
> > >  	}
> > >  
> > > -	ret = sgx_drv_init();
> > > +	/*
> > > +	 * Always try to initialize the native *and* KVM drivers.
> > > +	 * The KVM driver is less picky than the native one and
> > > +	 * can function if the native one is not supported on the
> > > +	 * current system or fails to initialize.
> > > +	 *
> > > +	 * Error out only if both fail to initialize.
> > > +	 */
> > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > 
> > I love this code.
> > 
> > Reviewed-by: Sean Christopherson <seanjc@google.com>
> 
> I'm still wondering why this code let's go through when sgx_drv_init()
> succeeds and sgx_vepc_init() fails.
> 
> The inline comment explains only the mirrored case (which does make
> sense).

I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
succeed. Why expect legitly anything else?

/Jarkko
Jarkko Sakkinen March 14, 2021, 3:25 p.m. UTC | #4
On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > configurations to use SGX.
> > > > 
> > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > ---
> > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > >  		goto err_page_cache;
> > > >  	}
> > > >  
> > > > -	ret = sgx_drv_init();
> > > > +	/*
> > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > +	 * The KVM driver is less picky than the native one and
> > > > +	 * can function if the native one is not supported on the
> > > > +	 * current system or fails to initialize.
> > > > +	 *
> > > > +	 * Error out only if both fail to initialize.
> > > > +	 */
> > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > 
> > > I love this code.
> > > 
> > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > 
> > I'm still wondering why this code let's go through when sgx_drv_init()
> > succeeds and sgx_vepc_init() fails.
> > 
> > The inline comment explains only the mirrored case (which does make
> > sense).
> 
> I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> succeed. Why expect legitly anything else?
 
Apologies coming with these ideas at this point, but here is what this
led me.

I think that the all this complexity comes from a bad code structure.

So, what is essentially happening here:

- We essentially want to make EPC always work.
- Driver optionally.

So what this sums to is something like:

        ret = sgx_epc_init();
        if (ret) {
                pr_err("EPC initialization failed.\n");
                return ret;
        }

        ret = sgx_drv_init();
        if (ret)
                pr_info("Driver could not be initialized.\n");

        /* continue */

I.e. I think there should be a single EPC init, which does both EPC
bootstrapping and vepc, and driver initialization comes after that.

/Jarkko
Jarkko Sakkinen March 14, 2021, 3:27 p.m. UTC | #5
On Sun, Mar 14, 2021 at 05:25:26PM +0200, Jarkko Sakkinen wrote:
> On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> > On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > > configurations to use SGX.
> > > > > 
> > > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > ---
> > > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > > >  		goto err_page_cache;
> > > > >  	}
> > > > >  
> > > > > -	ret = sgx_drv_init();
> > > > > +	/*
> > > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > > +	 * The KVM driver is less picky than the native one and
> > > > > +	 * can function if the native one is not supported on the
> > > > > +	 * current system or fails to initialize.
> > > > > +	 *
> > > > > +	 * Error out only if both fail to initialize.
> > > > > +	 */
> > > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > > 
> > > > I love this code.
> > > > 
> > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > 
> > > I'm still wondering why this code let's go through when sgx_drv_init()
> > > succeeds and sgx_vepc_init() fails.
> > > 
> > > The inline comment explains only the mirrored case (which does make
> > > sense).
> > 
> > I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> > succeed. Why expect legitly anything else?
>  
> Apologies coming with these ideas at this point, but here is what this
> led me.
> 
> I think that the all this complexity comes from a bad code structure.
> 
> So, what is essentially happening here:
> 
> - We essentially want to make EPC always work.
> - Driver optionally.
> 
> So what this sums to is something like:
> 
>         ret = sgx_epc_init();
>         if (ret) {
>                 pr_err("EPC initialization failed.\n");
>                 return ret;
>         }
> 
>         ret = sgx_drv_init();
>         if (ret)
>                 pr_info("Driver could not be initialized.\n");
> 
>         /* continue */
> 
> I.e. I think there should be a single EPC init, which does both EPC
> bootstrapping and vepc, and driver initialization comes after that.

In other words, from SGX point of view, the thing that KVM needs is
to cut out EPC and driver part into different islands. How this is now
implemented in the current patch set is half-way there but not yet what
it should be.

/Jarkko
Huang, Kai March 15, 2021, 3:13 a.m. UTC | #6
On Sun, 14 Mar 2021 17:27:18 +0200 Jarkko Sakkinen wrote:
> On Sun, Mar 14, 2021 at 05:25:26PM +0200, Jarkko Sakkinen wrote:
> > On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> > > On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > > > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > > > configurations to use SGX.
> > > > > > 
> > > > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > > ---
> > > > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > > > 
> > > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > > > >  		goto err_page_cache;
> > > > > >  	}
> > > > > >  
> > > > > > -	ret = sgx_drv_init();
> > > > > > +	/*
> > > > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > > > +	 * The KVM driver is less picky than the native one and
> > > > > > +	 * can function if the native one is not supported on the
> > > > > > +	 * current system or fails to initialize.
> > > > > > +	 *
> > > > > > +	 * Error out only if both fail to initialize.
> > > > > > +	 */
> > > > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > > > 
> > > > > I love this code.
> > > > > 
> > > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > > 
> > > > I'm still wondering why this code let's go through when sgx_drv_init()
> > > > succeeds and sgx_vepc_init() fails.
> > > > 
> > > > The inline comment explains only the mirrored case (which does make
> > > > sense).
> > > 
> > > I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> > > succeed. Why expect legitly anything else?
> >  
> > Apologies coming with these ideas at this point, but here is what this
> > led me.
> > 
> > I think that the all this complexity comes from a bad code structure.
> > 
> > So, what is essentially happening here:
> > 
> > - We essentially want to make EPC always work.
> > - Driver optionally.
> > 
> > So what this sums to is something like:
> > 
> >         ret = sgx_epc_init();
> >         if (ret) {
> >                 pr_err("EPC initialization failed.\n");
> >                 return ret;
> >         }
> > 
> >         ret = sgx_drv_init();
> >         if (ret)
> >                 pr_info("Driver could not be initialized.\n");
> > 
> >         /* continue */
> > 
> > I.e. I think there should be a single EPC init, which does both EPC
> > bootstrapping and vepc, and driver initialization comes after that.
> 
> In other words, from SGX point of view, the thing that KVM needs is
> to cut out EPC and driver part into different islands. How this is now
> implemented in the current patch set is half-way there but not yet what
> it should be.

Well conceptually, SGX virtualization and SGX driver are two independently
functionalities can be enabled separately, although they both requires some
come functionalities, such as /dev/sgx_provision, which we have moved to
sgx/main.c exactly for this purpose. THerefore, conceptually, it is bad to make
assumption that, if SGX virtualization initialization succeeded, SGX driver
must succeed -- we can potentially add more staff in SGX virtualization in the
future..

If the name sgx_vepc_init() confuses you, I can rename it to sgx_virt_init().

Hi Sean,

Please let us know if you have comments.
Jarkko Sakkinen March 15, 2021, 1:04 p.m. UTC | #7
On Mon, Mar 15, 2021 at 04:13:17PM +1300, Kai Huang wrote:
> On Sun, 14 Mar 2021 17:27:18 +0200 Jarkko Sakkinen wrote:
> > On Sun, Mar 14, 2021 at 05:25:26PM +0200, Jarkko Sakkinen wrote:
> > > On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> > > > On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > > > > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > > > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > > > > configurations to use SGX.
> > > > > > > 
> > > > > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > > > ---
> > > > > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > > > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > > > > 
> > > > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > > > > >  		goto err_page_cache;
> > > > > > >  	}
> > > > > > >  
> > > > > > > -	ret = sgx_drv_init();
> > > > > > > +	/*
> > > > > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > > > > +	 * The KVM driver is less picky than the native one and
> > > > > > > +	 * can function if the native one is not supported on the
> > > > > > > +	 * current system or fails to initialize.
> > > > > > > +	 *
> > > > > > > +	 * Error out only if both fail to initialize.
> > > > > > > +	 */
> > > > > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > > > > 
> > > > > > I love this code.
> > > > > > 
> > > > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > > > 
> > > > > I'm still wondering why this code let's go through when sgx_drv_init()
> > > > > succeeds and sgx_vepc_init() fails.
> > > > > 
> > > > > The inline comment explains only the mirrored case (which does make
> > > > > sense).
> > > > 
> > > > I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> > > > succeed. Why expect legitly anything else?
> > >  
> > > Apologies coming with these ideas at this point, but here is what this
> > > led me.
> > > 
> > > I think that the all this complexity comes from a bad code structure.
> > > 
> > > So, what is essentially happening here:
> > > 
> > > - We essentially want to make EPC always work.
> > > - Driver optionally.
> > > 
> > > So what this sums to is something like:
> > > 
> > >         ret = sgx_epc_init();
> > >         if (ret) {
> > >                 pr_err("EPC initialization failed.\n");
> > >                 return ret;
> > >         }
> > > 
> > >         ret = sgx_drv_init();
> > >         if (ret)
> > >                 pr_info("Driver could not be initialized.\n");
> > > 
> > >         /* continue */
> > > 
> > > I.e. I think there should be a single EPC init, which does both EPC
> > > bootstrapping and vepc, and driver initialization comes after that.
> > 
> > In other words, from SGX point of view, the thing that KVM needs is
> > to cut out EPC and driver part into different islands. How this is now
> > implemented in the current patch set is half-way there but not yet what
> > it should be.
> 
> Well conceptually, SGX virtualization and SGX driver are two independently
> functionalities can be enabled separately, although they both requires some
> come functionalities, such as /dev/sgx_provision, which we have moved to
> sgx/main.c exactly for this purpose. THerefore, conceptually, it is bad to make
> assumption that, if SGX virtualization initialization succeeded, SGX driver
> must succeed -- we can potentially add more staff in SGX virtualization in the
> future..
> 
> If the name sgx_vepc_init() confuses you, I can rename it to sgx_virt_init().

I don't understand what would be the bad thing here. Can you open that
up please? I'm neither capable of predicting the future...

> 
> 
> Please let us know if you have comments.
> 
/Jarkko
Jarkko Sakkinen March 15, 2021, 1:51 p.m. UTC | #8
On Mon, Mar 15, 2021 at 03:04:59PM +0200, Jarkko Sakkinen wrote:
> On Mon, Mar 15, 2021 at 04:13:17PM +1300, Kai Huang wrote:
> > On Sun, 14 Mar 2021 17:27:18 +0200 Jarkko Sakkinen wrote:
> > > On Sun, Mar 14, 2021 at 05:25:26PM +0200, Jarkko Sakkinen wrote:
> > > > On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> > > > > On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > > > > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > > > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > > > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > > > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > > > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > > > > > configurations to use SGX.
> > > > > > > > 
> > > > > > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > > > > ---
> > > > > > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > > > > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > > > > > 
> > > > > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > > > > > >  		goto err_page_cache;
> > > > > > > >  	}
> > > > > > > >  
> > > > > > > > -	ret = sgx_drv_init();
> > > > > > > > +	/*
> > > > > > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > > > > > +	 * The KVM driver is less picky than the native one and
> > > > > > > > +	 * can function if the native one is not supported on the
> > > > > > > > +	 * current system or fails to initialize.
> > > > > > > > +	 *
> > > > > > > > +	 * Error out only if both fail to initialize.
> > > > > > > > +	 */
> > > > > > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > > > > > 
> > > > > > > I love this code.
> > > > > > > 
> > > > > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > > > > 
> > > > > > I'm still wondering why this code let's go through when sgx_drv_init()
> > > > > > succeeds and sgx_vepc_init() fails.
> > > > > > 
> > > > > > The inline comment explains only the mirrored case (which does make
> > > > > > sense).
> > > > > 
> > > > > I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> > > > > succeed. Why expect legitly anything else?
> > > >  
> > > > Apologies coming with these ideas at this point, but here is what this
> > > > led me.
> > > > 
> > > > I think that the all this complexity comes from a bad code structure.
> > > > 
> > > > So, what is essentially happening here:
> > > > 
> > > > - We essentially want to make EPC always work.
> > > > - Driver optionally.
> > > > 
> > > > So what this sums to is something like:
> > > > 
> > > >         ret = sgx_epc_init();
> > > >         if (ret) {
> > > >                 pr_err("EPC initialization failed.\n");
> > > >                 return ret;
> > > >         }
> > > > 
> > > >         ret = sgx_drv_init();
> > > >         if (ret)
> > > >                 pr_info("Driver could not be initialized.\n");
> > > > 
> > > >         /* continue */
> > > > 
> > > > I.e. I think there should be a single EPC init, which does both EPC
> > > > bootstrapping and vepc, and driver initialization comes after that.
> > > 
> > > In other words, from SGX point of view, the thing that KVM needs is
> > > to cut out EPC and driver part into different islands. How this is now
> > > implemented in the current patch set is half-way there but not yet what
> > > it should be.
> > 
> > Well conceptually, SGX virtualization and SGX driver are two independently
> > functionalities can be enabled separately, although they both requires some
> > come functionalities, such as /dev/sgx_provision, which we have moved to
> > sgx/main.c exactly for this purpose. THerefore, conceptually, it is bad to make
> > assumption that, if SGX virtualization initialization succeeded, SGX driver
> > must succeed -- we can potentially add more staff in SGX virtualization in the
> > future..
> > 
> > If the name sgx_vepc_init() confuses you, I can rename it to sgx_virt_init().
> 
> I don't understand what would be the bad thing here. Can you open that
> up please? I'm neither capable of predicting the future...

Right, so since vepc_init() does only just device file initialization the
current function structure is fine. I totally forgot that sgx_drv_init()
does not call EPC initialization when I wrote the above :-) We refactored
during the inital cycle the driver so many times that I sometimes fix up
thing, sorry about.

To meld this into code:

        ret = sgx_vepc_init();
        if (ret != -ENODEV) {
                pr_err("vEPC initialization failed with %d.\n", ret);
                return ret;
        }

        ret = sgx_drv_init();
        if (ret != ENODEV)
                pr_info("Driver initialization failed %d.\n", ret);

This would also give more accurate information how far the initialization
went.

/Jarkko
Huang, Kai March 15, 2021, 8:48 p.m. UTC | #9
On Mon, 15 Mar 2021 15:51:17 +0200 Jarkko Sakkinen wrote:
> On Mon, Mar 15, 2021 at 03:04:59PM +0200, Jarkko Sakkinen wrote:
> > On Mon, Mar 15, 2021 at 04:13:17PM +1300, Kai Huang wrote:
> > > On Sun, 14 Mar 2021 17:27:18 +0200 Jarkko Sakkinen wrote:
> > > > On Sun, Mar 14, 2021 at 05:25:26PM +0200, Jarkko Sakkinen wrote:
> > > > > On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > > > > > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > > > > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > > > > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > > > > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > > > > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > > > > > > configurations to use SGX.
> > > > > > > > > 
> > > > > > > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > > > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > > > > > ---
> > > > > > > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > > > > > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > > > > > > 
> > > > > > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > > > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > > > > > > >  		goto err_page_cache;
> > > > > > > > >  	}
> > > > > > > > >  
> > > > > > > > > -	ret = sgx_drv_init();
> > > > > > > > > +	/*
> > > > > > > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > > > > > > +	 * The KVM driver is less picky than the native one and
> > > > > > > > > +	 * can function if the native one is not supported on the
> > > > > > > > > +	 * current system or fails to initialize.
> > > > > > > > > +	 *
> > > > > > > > > +	 * Error out only if both fail to initialize.
> > > > > > > > > +	 */
> > > > > > > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > > > > > > 
> > > > > > > > I love this code.
> > > > > > > > 
> > > > > > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > > > > > 
> > > > > > > I'm still wondering why this code let's go through when sgx_drv_init()
> > > > > > > succeeds and sgx_vepc_init() fails.
> > > > > > > 
> > > > > > > The inline comment explains only the mirrored case (which does make
> > > > > > > sense).
> > > > > > 
> > > > > > I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> > > > > > succeed. Why expect legitly anything else?
> > > > >  
> > > > > Apologies coming with these ideas at this point, but here is what this
> > > > > led me.
> > > > > 
> > > > > I think that the all this complexity comes from a bad code structure.
> > > > > 
> > > > > So, what is essentially happening here:
> > > > > 
> > > > > - We essentially want to make EPC always work.
> > > > > - Driver optionally.
> > > > > 
> > > > > So what this sums to is something like:
> > > > > 
> > > > >         ret = sgx_epc_init();
> > > > >         if (ret) {
> > > > >                 pr_err("EPC initialization failed.\n");
> > > > >                 return ret;
> > > > >         }
> > > > > 
> > > > >         ret = sgx_drv_init();
> > > > >         if (ret)
> > > > >                 pr_info("Driver could not be initialized.\n");
> > > > > 
> > > > >         /* continue */
> > > > > 
> > > > > I.e. I think there should be a single EPC init, which does both EPC
> > > > > bootstrapping and vepc, and driver initialization comes after that.
> > > > 
> > > > In other words, from SGX point of view, the thing that KVM needs is
> > > > to cut out EPC and driver part into different islands. How this is now
> > > > implemented in the current patch set is half-way there but not yet what
> > > > it should be.
> > > 
> > > Well conceptually, SGX virtualization and SGX driver are two independently
> > > functionalities can be enabled separately, although they both requires some
> > > come functionalities, such as /dev/sgx_provision, which we have moved to
> > > sgx/main.c exactly for this purpose. THerefore, conceptually, it is bad to make
> > > assumption that, if SGX virtualization initialization succeeded, SGX driver
> > > must succeed -- we can potentially add more staff in SGX virtualization in the
> > > future..
> > > 
> > > If the name sgx_vepc_init() confuses you, I can rename it to sgx_virt_init().
> > 
> > I don't understand what would be the bad thing here. Can you open that
> > up please? I'm neither capable of predicting the future...

Conceptually they are two different functionalities, and doesn't depend on each
other. Why calling SGX driver initialization only when SGX virtualization
succeeded?

We might want to add reclaiming EPC page (VMM EPC oversubscription) from KVM
guest in the future, which may bring more initialization staff sgx_vepc_init(),
and those new staff should not impact SGX driver.

I don't see your approach is any better, both from concept and flexibility.

Like I said, we can rename to sgx_virt_init() to be more generic, but I
strongly disagree your approach.

> 
> Right, so since vepc_init() does only just device file initialization the
> current function structure is fine. I totally forgot that sgx_drv_init()
> does not call EPC initialization when I wrote the above :-) We refactored
> during the inital cycle the driver so many times that I sometimes fix up
> thing, sorry about.
> 
> To meld this into code:
> 
>         ret = sgx_vepc_init();
>         if (ret != -ENODEV) {
>                 pr_err("vEPC initialization failed with %d.\n", ret);
>                 return ret;
>         }
> 
>         ret = sgx_drv_init();
>         if (ret != ENODEV)
>                 pr_info("Driver initialization failed %d.\n", ret);

Hmm.. Let's say an extreme case: misc_register() failed in sgx_vepc_init(), due
to -ENOMEM. Then OOM kill gets involved, and kills bunch of apps. And then In
this case, theoretically, misc_register() in sgx_drv_init() doesn't need to
fail.

The point is really SGX driver and SGX virt are two independent
functionalities, so don't make dependency on them, manually. Plus I don't see
any benefit of your approach, but only cons.

> 
> This would also give more accurate information how far the initialization
> went.
> 
> /Jarkko
Jarkko Sakkinen March 15, 2021, 11:05 p.m. UTC | #10
On Tue, Mar 16, 2021 at 09:48:59AM +1300, Kai Huang wrote:
> On Mon, 15 Mar 2021 15:51:17 +0200 Jarkko Sakkinen wrote:
> > On Mon, Mar 15, 2021 at 03:04:59PM +0200, Jarkko Sakkinen wrote:
> > > On Mon, Mar 15, 2021 at 04:13:17PM +1300, Kai Huang wrote:
> > > > On Sun, 14 Mar 2021 17:27:18 +0200 Jarkko Sakkinen wrote:
> > > > > On Sun, Mar 14, 2021 at 05:25:26PM +0200, Jarkko Sakkinen wrote:
> > > > > > On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > > On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > > > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > > > > > > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > > > > > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > > > > > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > > > > > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > > > > > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > > > > > > > configurations to use SGX.
> > > > > > > > > > 
> > > > > > > > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > > > > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > > > > > > ---
> > > > > > > > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > > > > > > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > > > > > > > 
> > > > > > > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > > > > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > > > > > > > >  		goto err_page_cache;
> > > > > > > > > >  	}
> > > > > > > > > >  
> > > > > > > > > > -	ret = sgx_drv_init();
> > > > > > > > > > +	/*
> > > > > > > > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > > > > > > > +	 * The KVM driver is less picky than the native one and
> > > > > > > > > > +	 * can function if the native one is not supported on the
> > > > > > > > > > +	 * current system or fails to initialize.
> > > > > > > > > > +	 *
> > > > > > > > > > +	 * Error out only if both fail to initialize.
> > > > > > > > > > +	 */
> > > > > > > > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > > > > > > > 
> > > > > > > > > I love this code.
> > > > > > > > > 
> > > > > > > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > > > > > > 
> > > > > > > > I'm still wondering why this code let's go through when sgx_drv_init()
> > > > > > > > succeeds and sgx_vepc_init() fails.
> > > > > > > > 
> > > > > > > > The inline comment explains only the mirrored case (which does make
> > > > > > > > sense).
> > > > > > > 
> > > > > > > I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> > > > > > > succeed. Why expect legitly anything else?
> > > > > >  
> > > > > > Apologies coming with these ideas at this point, but here is what this
> > > > > > led me.
> > > > > > 
> > > > > > I think that the all this complexity comes from a bad code structure.
> > > > > > 
> > > > > > So, what is essentially happening here:
> > > > > > 
> > > > > > - We essentially want to make EPC always work.
> > > > > > - Driver optionally.
> > > > > > 
> > > > > > So what this sums to is something like:
> > > > > > 
> > > > > >         ret = sgx_epc_init();
> > > > > >         if (ret) {
> > > > > >                 pr_err("EPC initialization failed.\n");
> > > > > >                 return ret;
> > > > > >         }
> > > > > > 
> > > > > >         ret = sgx_drv_init();
> > > > > >         if (ret)
> > > > > >                 pr_info("Driver could not be initialized.\n");
> > > > > > 
> > > > > >         /* continue */
> > > > > > 
> > > > > > I.e. I think there should be a single EPC init, which does both EPC
> > > > > > bootstrapping and vepc, and driver initialization comes after that.
> > > > > 
> > > > > In other words, from SGX point of view, the thing that KVM needs is
> > > > > to cut out EPC and driver part into different islands. How this is now
> > > > > implemented in the current patch set is half-way there but not yet what
> > > > > it should be.
> > > > 
> > > > Well conceptually, SGX virtualization and SGX driver are two independently
> > > > functionalities can be enabled separately, although they both requires some
> > > > come functionalities, such as /dev/sgx_provision, which we have moved to
> > > > sgx/main.c exactly for this purpose. THerefore, conceptually, it is bad to make
> > > > assumption that, if SGX virtualization initialization succeeded, SGX driver
> > > > must succeed -- we can potentially add more staff in SGX virtualization in the
> > > > future..
> > > > 
> > > > If the name sgx_vepc_init() confuses you, I can rename it to sgx_virt_init().
> > > 
> > > I don't understand what would be the bad thing here. Can you open that
> > > up please? I'm neither capable of predicting the future...
> 
> Conceptually they are two different functionalities, and doesn't depend on each
> other. Why calling SGX driver initialization only when SGX virtualization
> succeeded?
> 
> We might want to add reclaiming EPC page (VMM EPC oversubscription) from KVM
> guest in the future, which may bring more initialization staff sgx_vepc_init(),
> and those new staff should not impact SGX driver.
> 
> I don't see your approach is any better, both from concept and flexibility.
> 
> Like I said, we can rename to sgx_virt_init() to be more generic, but I
> strongly disagree your approach.
> 
> > 
> > Right, so since vepc_init() does only just device file initialization the
> > current function structure is fine. I totally forgot that sgx_drv_init()
> > does not call EPC initialization when I wrote the above :-) We refactored
> > during the inital cycle the driver so many times that I sometimes fix up
> > thing, sorry about.
> > 
> > To meld this into code:
> > 
> >         ret = sgx_vepc_init();
> >         if (ret != -ENODEV) {
> >                 pr_err("vEPC initialization failed with %d.\n", ret);
> >                 return ret;
> >         }
> > 
> >         ret = sgx_drv_init();
> >         if (ret != ENODEV)
> >                 pr_info("Driver initialization failed %d.\n", ret);
> 
> Hmm.. Let's say an extreme case: misc_register() failed in sgx_vepc_init(), due
> to -ENOMEM. Then OOM kill gets involved, and kills bunch of apps. And then In
> this case, theoretically, misc_register() in sgx_drv_init() doesn't need to
> fail.
> 
> The point is really SGX driver and SGX virt are two independent
> functionalities, so don't make dependency on them, manually. Plus I don't see
> any benefit of your approach, but only cons.

The way I've understood it is that given that KVM can support SGX
without FLC, vEPC should be available even if driver cannot be
enabled.

This is also exactly what the short summary states.

"Initialize virtual EPC driver even when SGX driver is disabled"

It *does not* state:

"Initialize SGX driver even when vEPC driver is disabled"

Also, this is how I interpret the inline comment.

All this considered, the other direction is undocumented functionality.


/Jarkko
Jarkko Sakkinen March 15, 2021, 11:08 p.m. UTC | #11
On Tue, Mar 16, 2021 at 01:05:05AM +0200, Jarkko Sakkinen wrote:
> On Tue, Mar 16, 2021 at 09:48:59AM +1300, Kai Huang wrote:
> > On Mon, 15 Mar 2021 15:51:17 +0200 Jarkko Sakkinen wrote:
> > > On Mon, Mar 15, 2021 at 03:04:59PM +0200, Jarkko Sakkinen wrote:
> > > > On Mon, Mar 15, 2021 at 04:13:17PM +1300, Kai Huang wrote:
> > > > > On Sun, 14 Mar 2021 17:27:18 +0200 Jarkko Sakkinen wrote:
> > > > > > On Sun, Mar 14, 2021 at 05:25:26PM +0200, Jarkko Sakkinen wrote:
> > > > > > > On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > > > On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > > > > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > > > > > > > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > > > > > > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > > > > > > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > > > > > > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > > > > > > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > > > > > > > > configurations to use SGX.
> > > > > > > > > > > 
> > > > > > > > > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > > > > > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > > > > > > > ---
> > > > > > > > > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > > > > > > > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > > > > > > > > 
> > > > > > > > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > > > > > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > > > > > > > > >  		goto err_page_cache;
> > > > > > > > > > >  	}
> > > > > > > > > > >  
> > > > > > > > > > > -	ret = sgx_drv_init();
> > > > > > > > > > > +	/*
> > > > > > > > > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > > > > > > > > +	 * The KVM driver is less picky than the native one and
> > > > > > > > > > > +	 * can function if the native one is not supported on the
> > > > > > > > > > > +	 * current system or fails to initialize.
> > > > > > > > > > > +	 *
> > > > > > > > > > > +	 * Error out only if both fail to initialize.
> > > > > > > > > > > +	 */
> > > > > > > > > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > > > > > > > > 
> > > > > > > > > > I love this code.
> > > > > > > > > > 
> > > > > > > > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > > > > > > > 
> > > > > > > > > I'm still wondering why this code let's go through when sgx_drv_init()
> > > > > > > > > succeeds and sgx_vepc_init() fails.
> > > > > > > > > 
> > > > > > > > > The inline comment explains only the mirrored case (which does make
> > > > > > > > > sense).
> > > > > > > > 
> > > > > > > > I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> > > > > > > > succeed. Why expect legitly anything else?
> > > > > > >  
> > > > > > > Apologies coming with these ideas at this point, but here is what this
> > > > > > > led me.
> > > > > > > 
> > > > > > > I think that the all this complexity comes from a bad code structure.
> > > > > > > 
> > > > > > > So, what is essentially happening here:
> > > > > > > 
> > > > > > > - We essentially want to make EPC always work.
> > > > > > > - Driver optionally.
> > > > > > > 
> > > > > > > So what this sums to is something like:
> > > > > > > 
> > > > > > >         ret = sgx_epc_init();
> > > > > > >         if (ret) {
> > > > > > >                 pr_err("EPC initialization failed.\n");
> > > > > > >                 return ret;
> > > > > > >         }
> > > > > > > 
> > > > > > >         ret = sgx_drv_init();
> > > > > > >         if (ret)
> > > > > > >                 pr_info("Driver could not be initialized.\n");
> > > > > > > 
> > > > > > >         /* continue */
> > > > > > > 
> > > > > > > I.e. I think there should be a single EPC init, which does both EPC
> > > > > > > bootstrapping and vepc, and driver initialization comes after that.
> > > > > > 
> > > > > > In other words, from SGX point of view, the thing that KVM needs is
> > > > > > to cut out EPC and driver part into different islands. How this is now
> > > > > > implemented in the current patch set is half-way there but not yet what
> > > > > > it should be.
> > > > > 
> > > > > Well conceptually, SGX virtualization and SGX driver are two independently
> > > > > functionalities can be enabled separately, although they both requires some
> > > > > come functionalities, such as /dev/sgx_provision, which we have moved to
> > > > > sgx/main.c exactly for this purpose. THerefore, conceptually, it is bad to make
> > > > > assumption that, if SGX virtualization initialization succeeded, SGX driver
> > > > > must succeed -- we can potentially add more staff in SGX virtualization in the
> > > > > future..
> > > > > 
> > > > > If the name sgx_vepc_init() confuses you, I can rename it to sgx_virt_init().
> > > > 
> > > > I don't understand what would be the bad thing here. Can you open that
> > > > up please? I'm neither capable of predicting the future...
> > 
> > Conceptually they are two different functionalities, and doesn't depend on each
> > other. Why calling SGX driver initialization only when SGX virtualization
> > succeeded?
> > 
> > We might want to add reclaiming EPC page (VMM EPC oversubscription) from KVM
> > guest in the future, which may bring more initialization staff sgx_vepc_init(),
> > and those new staff should not impact SGX driver.
> > 
> > I don't see your approach is any better, both from concept and flexibility.
> > 
> > Like I said, we can rename to sgx_virt_init() to be more generic, but I
> > strongly disagree your approach.
> > 
> > > 
> > > Right, so since vepc_init() does only just device file initialization the
> > > current function structure is fine. I totally forgot that sgx_drv_init()
> > > does not call EPC initialization when I wrote the above :-) We refactored
> > > during the inital cycle the driver so many times that I sometimes fix up
> > > thing, sorry about.
> > > 
> > > To meld this into code:
> > > 
> > >         ret = sgx_vepc_init();
> > >         if (ret != -ENODEV) {
> > >                 pr_err("vEPC initialization failed with %d.\n", ret);
> > >                 return ret;
> > >         }
> > > 
> > >         ret = sgx_drv_init();
> > >         if (ret != ENODEV)
> > >                 pr_info("Driver initialization failed %d.\n", ret);
> > 
> > Hmm.. Let's say an extreme case: misc_register() failed in sgx_vepc_init(), due
> > to -ENOMEM. Then OOM kill gets involved, and kills bunch of apps. And then In
> > this case, theoretically, misc_register() in sgx_drv_init() doesn't need to
> > fail.
> > 
> > The point is really SGX driver and SGX virt are two independent
> > functionalities, so don't make dependency on them, manually. Plus I don't see
> > any benefit of your approach, but only cons.
> 
> The way I've understood it is that given that KVM can support SGX
> without FLC, vEPC should be available even if driver cannot be
> enabled.
> 
> This is also exactly what the short summary states.
> 
> "Initialize virtual EPC driver even when SGX driver is disabled"
> 
> It *does not* state:
> 
> "Initialize SGX driver even when vEPC driver is disabled"
> 
> Also, this is how I interpret the inline comment.
> 
> All this considered, the other direction is undocumented functionality.

Also:

1. There is *zero* good practical reasons to support the "2nd direction".
   For KVM getting init'd with SGX, on the other hand, we have good
   practical reasons.
2. We can get something practically useful with simpler and more verbose
   code, i.e. better logging.

/Jarkko
Huang, Kai March 15, 2021, 11:49 p.m. UTC | #12
On Tue, 16 Mar 2021 01:08:44 +0200 Jarkko Sakkinen wrote:
> On Tue, Mar 16, 2021 at 01:05:05AM +0200, Jarkko Sakkinen wrote:
> > On Tue, Mar 16, 2021 at 09:48:59AM +1300, Kai Huang wrote:
> > > On Mon, 15 Mar 2021 15:51:17 +0200 Jarkko Sakkinen wrote:
> > > > On Mon, Mar 15, 2021 at 03:04:59PM +0200, Jarkko Sakkinen wrote:
> > > > > On Mon, Mar 15, 2021 at 04:13:17PM +1300, Kai Huang wrote:
> > > > > > On Sun, 14 Mar 2021 17:27:18 +0200 Jarkko Sakkinen wrote:
> > > > > > > On Sun, Mar 14, 2021 at 05:25:26PM +0200, Jarkko Sakkinen wrote:
> > > > > > > > On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > > > > On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > > > > > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > > > > > > > > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > > > > > > > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > > > > > > > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > > > > > > > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > > > > > > > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > > > > > > > > > configurations to use SGX.
> > > > > > > > > > > > 
> > > > > > > > > > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > > > > > > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > > > > > > > > ---
> > > > > > > > > > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > > > > > > > > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > > > > > > > > > 
> > > > > > > > > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > > > > > > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > > > > > > > > > >  		goto err_page_cache;
> > > > > > > > > > > >  	}
> > > > > > > > > > > >  
> > > > > > > > > > > > -	ret = sgx_drv_init();
> > > > > > > > > > > > +	/*
> > > > > > > > > > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > > > > > > > > > +	 * The KVM driver is less picky than the native one and
> > > > > > > > > > > > +	 * can function if the native one is not supported on the
> > > > > > > > > > > > +	 * current system or fails to initialize.
> > > > > > > > > > > > +	 *
> > > > > > > > > > > > +	 * Error out only if both fail to initialize.
> > > > > > > > > > > > +	 */
> > > > > > > > > > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > > > > > > > > > 
> > > > > > > > > > > I love this code.
> > > > > > > > > > > 
> > > > > > > > > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > > > > > > > > 
> > > > > > > > > > I'm still wondering why this code let's go through when sgx_drv_init()
> > > > > > > > > > succeeds and sgx_vepc_init() fails.
> > > > > > > > > > 
> > > > > > > > > > The inline comment explains only the mirrored case (which does make
> > > > > > > > > > sense).
> > > > > > > > > 
> > > > > > > > > I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> > > > > > > > > succeed. Why expect legitly anything else?
> > > > > > > >  
> > > > > > > > Apologies coming with these ideas at this point, but here is what this
> > > > > > > > led me.
> > > > > > > > 
> > > > > > > > I think that the all this complexity comes from a bad code structure.
> > > > > > > > 
> > > > > > > > So, what is essentially happening here:
> > > > > > > > 
> > > > > > > > - We essentially want to make EPC always work.
> > > > > > > > - Driver optionally.
> > > > > > > > 
> > > > > > > > So what this sums to is something like:
> > > > > > > > 
> > > > > > > >         ret = sgx_epc_init();
> > > > > > > >         if (ret) {
> > > > > > > >                 pr_err("EPC initialization failed.\n");
> > > > > > > >                 return ret;
> > > > > > > >         }
> > > > > > > > 
> > > > > > > >         ret = sgx_drv_init();
> > > > > > > >         if (ret)
> > > > > > > >                 pr_info("Driver could not be initialized.\n");
> > > > > > > > 
> > > > > > > >         /* continue */
> > > > > > > > 
> > > > > > > > I.e. I think there should be a single EPC init, which does both EPC
> > > > > > > > bootstrapping and vepc, and driver initialization comes after that.
> > > > > > > 
> > > > > > > In other words, from SGX point of view, the thing that KVM needs is
> > > > > > > to cut out EPC and driver part into different islands. How this is now
> > > > > > > implemented in the current patch set is half-way there but not yet what
> > > > > > > it should be.
> > > > > > 
> > > > > > Well conceptually, SGX virtualization and SGX driver are two independently
> > > > > > functionalities can be enabled separately, although they both requires some
> > > > > > come functionalities, such as /dev/sgx_provision, which we have moved to
> > > > > > sgx/main.c exactly for this purpose. THerefore, conceptually, it is bad to make
> > > > > > assumption that, if SGX virtualization initialization succeeded, SGX driver
> > > > > > must succeed -- we can potentially add more staff in SGX virtualization in the
> > > > > > future..
> > > > > > 
> > > > > > If the name sgx_vepc_init() confuses you, I can rename it to sgx_virt_init().
> > > > > 
> > > > > I don't understand what would be the bad thing here. Can you open that
> > > > > up please? I'm neither capable of predicting the future...
> > > 
> > > Conceptually they are two different functionalities, and doesn't depend on each
> > > other. Why calling SGX driver initialization only when SGX virtualization
> > > succeeded?
> > > 
> > > We might want to add reclaiming EPC page (VMM EPC oversubscription) from KVM
> > > guest in the future, which may bring more initialization staff sgx_vepc_init(),
> > > and those new staff should not impact SGX driver.
> > > 
> > > I don't see your approach is any better, both from concept and flexibility.
> > > 
> > > Like I said, we can rename to sgx_virt_init() to be more generic, but I
> > > strongly disagree your approach.
> > > 
> > > > 
> > > > Right, so since vepc_init() does only just device file initialization the
> > > > current function structure is fine. I totally forgot that sgx_drv_init()
> > > > does not call EPC initialization when I wrote the above :-) We refactored
> > > > during the inital cycle the driver so many times that I sometimes fix up
> > > > thing, sorry about.
> > > > 
> > > > To meld this into code:
> > > > 
> > > >         ret = sgx_vepc_init();
> > > >         if (ret != -ENODEV) {
> > > >                 pr_err("vEPC initialization failed with %d.\n", ret);
> > > >                 return ret;
> > > >         }
> > > > 
> > > >         ret = sgx_drv_init();
> > > >         if (ret != ENODEV)
> > > >                 pr_info("Driver initialization failed %d.\n", ret);
> > > 
> > > Hmm.. Let's say an extreme case: misc_register() failed in sgx_vepc_init(), due
> > > to -ENOMEM. Then OOM kill gets involved, and kills bunch of apps. And then In
> > > this case, theoretically, misc_register() in sgx_drv_init() doesn't need to
> > > fail.
> > > 
> > > The point is really SGX driver and SGX virt are two independent
> > > functionalities, so don't make dependency on them, manually. Plus I don't see
> > > any benefit of your approach, but only cons.
> > 
> > The way I've understood it is that given that KVM can support SGX
> > without FLC, vEPC should be available even if driver cannot be
> > enabled.
> > 
> > This is also exactly what the short summary states.
> > 
> > "Initialize virtual EPC driver even when SGX driver is disabled"
> > 
> > It *does not* state:
> > 
> > "Initialize SGX driver even when vEPC driver is disabled"

OK. The patch title can be improved. How about:

"Initialize SGX driver and virtual EPC driver independently"

?

> > 
> > Also, this is how I interpret the inline comment.
> > 
> > All this considered, the other direction is undocumented functionality.

OK. How about below?

/*
 * Always try to initialize the native *and* KVM drivers. They are independent
 * functionalities and one can be initialized even when the other is not
 * supported or fails to initialize.
 */

The explicit saying of "not supported or fails to initialize" was requested by
you -- you wanted to distinguish -ENODEV with other error codes.

> 
> Also:
> 
> 1. There is *zero* good practical reasons to support the "2nd direction".
>    For KVM getting init'd with SGX, on the other hand, we have good
>    practical reasons.

Why there's *zero* good practical reasons? With initializing them
independently, people don't need to worry about *internal* of
sgx_vepc_init() and sgx_drv_init(), but just need pay attention of the logic
that they are two independent functionalities. Being able to initialize them
independently is much more clear and easier to understand. And like I said, in
this way it is more flexible to extend -- for instance, we may add more staff
to support VMM EPC oversubscription. So why there is *zero* good practical
reasons?

Btw, there are customers that want to just use KVM SGX, but not SGX driver in
host, for which people may want to add separate CONFIG option, say,
CONFIG_X86_SGX_DRIVER, to be able to disable/enable SGX driver code, just like
CONFIG_X86_SGX_KVM. Make them independent logically  just make things more
clear.

> 2. We can get something practically useful with simpler and more verbose
>    code, i.e. better logging.

I can add error msg in sgx_vepc_init() upon misc_register() if you want.

> 
> /Jarkko
Sean Christopherson March 16, 2021, 1:13 a.m. UTC | #13
On Tue, Mar 16, 2021, Jarkko Sakkinen wrote:
> On Tue, Mar 16, 2021 at 01:05:05AM +0200, Jarkko Sakkinen wrote:
> > The way I've understood it is that given that KVM can support SGX
> > without FLC, vEPC should be available even if driver cannot be
> > enabled.
> > 
> > This is also exactly what the short summary states.
> > 
> > "Initialize virtual EPC driver even when SGX driver is disabled"
> > 
> > It *does not* state:
> > 
> > "Initialize SGX driver even when vEPC driver is disabled"
> > 
> > Also, this is how I interpret the inline comment.
> > 
> > All this considered, the other direction is undocumented functionality.
> 
> Also:
> 
> 1. There is *zero* good practical reasons to support the "2nd direction".

Uh, yes there is.  CONFIG_KVM_INTEL=n and X86_FEATURE_VMX=n, either of which
will cause vEPC initialization to fail.  The former is obvious, the latter is
possible via BIOS configuration.

>    For KVM getting init'd with SGX, on the other hand, we have good
>    practical reasons.
> 2. We can get something practically useful with simpler and more verbose
>    code, i.e. better logging.
> 
> /Jarkko
Jarkko Sakkinen March 16, 2021, 12:44 p.m. UTC | #14
On Tue, Mar 16, 2021 at 12:49:33PM +1300, Kai Huang wrote:
> On Tue, 16 Mar 2021 01:08:44 +0200 Jarkko Sakkinen wrote:
> > On Tue, Mar 16, 2021 at 01:05:05AM +0200, Jarkko Sakkinen wrote:
> > > On Tue, Mar 16, 2021 at 09:48:59AM +1300, Kai Huang wrote:
> > > > On Mon, 15 Mar 2021 15:51:17 +0200 Jarkko Sakkinen wrote:
> > > > > On Mon, Mar 15, 2021 at 03:04:59PM +0200, Jarkko Sakkinen wrote:
> > > > > > On Mon, Mar 15, 2021 at 04:13:17PM +1300, Kai Huang wrote:
> > > > > > > On Sun, 14 Mar 2021 17:27:18 +0200 Jarkko Sakkinen wrote:
> > > > > > > > On Sun, Mar 14, 2021 at 05:25:26PM +0200, Jarkko Sakkinen wrote:
> > > > > > > > > On Sat, Mar 13, 2021 at 09:07:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > > > > > On Sat, Mar 13, 2021 at 09:05:36PM +0200, Jarkko Sakkinen wrote:
> > > > > > > > > > > On Fri, Mar 12, 2021 at 01:44:58PM -0800, Sean Christopherson wrote:
> > > > > > > > > > > > On Tue, Mar 09, 2021, Kai Huang wrote:
> > > > > > > > > > > > > Modify sgx_init() to always try to initialize the virtual EPC driver,
> > > > > > > > > > > > > even if the SGX driver is disabled.  The SGX driver might be disabled
> > > > > > > > > > > > > if SGX Launch Control is in locked mode, or not supported in the
> > > > > > > > > > > > > hardware at all.  This allows (non-Linux) guests that support non-LC
> > > > > > > > > > > > > configurations to use SGX.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > Acked-by: Dave Hansen <dave.hansen@intel.com>
> > > > > > > > > > > > > Signed-off-by: Kai Huang <kai.huang@intel.com>
> > > > > > > > > > > > > ---
> > > > > > > > > > > > >  arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++-
> > > > > > > > > > > > >  1 file changed, 9 insertions(+), 1 deletion(-)
> > > > > > > > > > > > > 
> > > > > > > > > > > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > > > > index 44fe91a5bfb3..8c922e68274d 100644
> > > > > > > > > > > > > --- a/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > > > > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > > > > > > > > > > > > @@ -712,7 +712,15 @@ static int __init sgx_init(void)
> > > > > > > > > > > > >  		goto err_page_cache;
> > > > > > > > > > > > >  	}
> > > > > > > > > > > > >  
> > > > > > > > > > > > > -	ret = sgx_drv_init();
> > > > > > > > > > > > > +	/*
> > > > > > > > > > > > > +	 * Always try to initialize the native *and* KVM drivers.
> > > > > > > > > > > > > +	 * The KVM driver is less picky than the native one and
> > > > > > > > > > > > > +	 * can function if the native one is not supported on the
> > > > > > > > > > > > > +	 * current system or fails to initialize.
> > > > > > > > > > > > > +	 *
> > > > > > > > > > > > > +	 * Error out only if both fail to initialize.
> > > > > > > > > > > > > +	 */
> > > > > > > > > > > > > +	ret = !!sgx_drv_init() & !!sgx_vepc_init();
> > > > > > > > > > > > 
> > > > > > > > > > > > I love this code.
> > > > > > > > > > > > 
> > > > > > > > > > > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > > > > > > > > > 
> > > > > > > > > > > I'm still wondering why this code let's go through when sgx_drv_init()
> > > > > > > > > > > succeeds and sgx_vepc_init() fails.
> > > > > > > > > > > 
> > > > > > > > > > > The inline comment explains only the mirrored case (which does make
> > > > > > > > > > > sense).
> > > > > > > > > > 
> > > > > > > > > > I.e. if sgx_drv_init() succeeds, I'd expect that sgx_vepc_init() must
> > > > > > > > > > succeed. Why expect legitly anything else?
> > > > > > > > >  
> > > > > > > > > Apologies coming with these ideas at this point, but here is what this
> > > > > > > > > led me.
> > > > > > > > > 
> > > > > > > > > I think that the all this complexity comes from a bad code structure.
> > > > > > > > > 
> > > > > > > > > So, what is essentially happening here:
> > > > > > > > > 
> > > > > > > > > - We essentially want to make EPC always work.
> > > > > > > > > - Driver optionally.
> > > > > > > > > 
> > > > > > > > > So what this sums to is something like:
> > > > > > > > > 
> > > > > > > > >         ret = sgx_epc_init();
> > > > > > > > >         if (ret) {
> > > > > > > > >                 pr_err("EPC initialization failed.\n");
> > > > > > > > >                 return ret;
> > > > > > > > >         }
> > > > > > > > > 
> > > > > > > > >         ret = sgx_drv_init();
> > > > > > > > >         if (ret)
> > > > > > > > >                 pr_info("Driver could not be initialized.\n");
> > > > > > > > > 
> > > > > > > > >         /* continue */
> > > > > > > > > 
> > > > > > > > > I.e. I think there should be a single EPC init, which does both EPC
> > > > > > > > > bootstrapping and vepc, and driver initialization comes after that.
> > > > > > > > 
> > > > > > > > In other words, from SGX point of view, the thing that KVM needs is
> > > > > > > > to cut out EPC and driver part into different islands. How this is now
> > > > > > > > implemented in the current patch set is half-way there but not yet what
> > > > > > > > it should be.
> > > > > > > 
> > > > > > > Well conceptually, SGX virtualization and SGX driver are two independently
> > > > > > > functionalities can be enabled separately, although they both requires some
> > > > > > > come functionalities, such as /dev/sgx_provision, which we have moved to
> > > > > > > sgx/main.c exactly for this purpose. THerefore, conceptually, it is bad to make
> > > > > > > assumption that, if SGX virtualization initialization succeeded, SGX driver
> > > > > > > must succeed -- we can potentially add more staff in SGX virtualization in the
> > > > > > > future..
> > > > > > > 
> > > > > > > If the name sgx_vepc_init() confuses you, I can rename it to sgx_virt_init().
> > > > > > 
> > > > > > I don't understand what would be the bad thing here. Can you open that
> > > > > > up please? I'm neither capable of predicting the future...
> > > > 
> > > > Conceptually they are two different functionalities, and doesn't depend on each
> > > > other. Why calling SGX driver initialization only when SGX virtualization
> > > > succeeded?
> > > > 
> > > > We might want to add reclaiming EPC page (VMM EPC oversubscription) from KVM
> > > > guest in the future, which may bring more initialization staff sgx_vepc_init(),
> > > > and those new staff should not impact SGX driver.
> > > > 
> > > > I don't see your approach is any better, both from concept and flexibility.
> > > > 
> > > > Like I said, we can rename to sgx_virt_init() to be more generic, but I
> > > > strongly disagree your approach.
> > > > 
> > > > > 
> > > > > Right, so since vepc_init() does only just device file initialization the
> > > > > current function structure is fine. I totally forgot that sgx_drv_init()
> > > > > does not call EPC initialization when I wrote the above :-) We refactored
> > > > > during the inital cycle the driver so many times that I sometimes fix up
> > > > > thing, sorry about.
> > > > > 
> > > > > To meld this into code:
> > > > > 
> > > > >         ret = sgx_vepc_init();
> > > > >         if (ret != -ENODEV) {
> > > > >                 pr_err("vEPC initialization failed with %d.\n", ret);
> > > > >                 return ret;
> > > > >         }
> > > > > 
> > > > >         ret = sgx_drv_init();
> > > > >         if (ret != ENODEV)
> > > > >                 pr_info("Driver initialization failed %d.\n", ret);
> > > > 
> > > > Hmm.. Let's say an extreme case: misc_register() failed in sgx_vepc_init(), due
> > > > to -ENOMEM. Then OOM kill gets involved, and kills bunch of apps. And then In
> > > > this case, theoretically, misc_register() in sgx_drv_init() doesn't need to
> > > > fail.
> > > > 
> > > > The point is really SGX driver and SGX virt are two independent
> > > > functionalities, so don't make dependency on them, manually. Plus I don't see
> > > > any benefit of your approach, but only cons.
> > > 
> > > The way I've understood it is that given that KVM can support SGX
> > > without FLC, vEPC should be available even if driver cannot be
> > > enabled.
> > > 
> > > This is also exactly what the short summary states.
> > > 
> > > "Initialize virtual EPC driver even when SGX driver is disabled"
> > > 
> > > It *does not* state:
> > > 
> > > "Initialize SGX driver even when vEPC driver is disabled"
> 
> OK. The patch title can be improved. How about:
> 
> "Initialize SGX driver and virtual EPC driver independently"
> 
> ?
> 
> > > 
> > > Also, this is how I interpret the inline comment.
> > > 
> > > All this considered, the other direction is undocumented functionality.
> 
> OK. How about below?
> 
> /*
>  * Always try to initialize the native *and* KVM drivers. They are independent
>  * functionalities and one can be initialized even when the other is not
>  * supported or fails to initialize.
>  */
> 
> The explicit saying of "not supported or fails to initialize" was requested by
> you -- you wanted to distinguish -ENODEV with other error codes.
> 
> > 
> > Also:
> > 
> > 1. There is *zero* good practical reasons to support the "2nd direction".
> >    For KVM getting init'd with SGX, on the other hand, we have good
> >    practical reasons.
> 
> Why there's *zero* good practical reasons? With initializing them
> independently, people don't need to worry about *internal* of
> sgx_vepc_init() and sgx_drv_init(), but just need pay attention of the logic
> that they are two independent functionalities. Being able to initialize them
> independently is much more clear and easier to understand. And like I said, in
> this way it is more flexible to extend -- for instance, we may add more staff
> to support VMM EPC oversubscription. So why there is *zero* good practical
> reasons?

Then things would be reconsidered.

> 
> Btw, there are customers that want to just use KVM SGX, but not SGX driver in
> host, for which people may want to add separate CONFIG option, say,
> CONFIG_X86_SGX_DRIVER, to be able to disable/enable SGX driver code, just like
> CONFIG_X86_SGX_KVM. Make them independent logically  just make things more
> clear.

Why?

> 
> > 2. We can get something practically useful with simpler and more verbose
> >    code, i.e. better logging.
> 
> I can add error msg in sgx_vepc_init() upon misc_register() if you want.
> 
> > 
> > /Jarkko
> 

/Jarkko
Jarkko Sakkinen March 16, 2021, 12:46 p.m. UTC | #15
On Mon, Mar 15, 2021 at 06:13:53PM -0700, Sean Christopherson wrote:
> On Tue, Mar 16, 2021, Jarkko Sakkinen wrote:
> > On Tue, Mar 16, 2021 at 01:05:05AM +0200, Jarkko Sakkinen wrote:
> > > The way I've understood it is that given that KVM can support SGX
> > > without FLC, vEPC should be available even if driver cannot be
> > > enabled.
> > > 
> > > This is also exactly what the short summary states.
> > > 
> > > "Initialize virtual EPC driver even when SGX driver is disabled"
> > > 
> > > It *does not* state:
> > > 
> > > "Initialize SGX driver even when vEPC driver is disabled"
> > > 
> > > Also, this is how I interpret the inline comment.
> > > 
> > > All this considered, the other direction is undocumented functionality.
> > 
> > Also:
> > 
> > 1. There is *zero* good practical reasons to support the "2nd direction".
> 
> Uh, yes there is.  CONFIG_KVM_INTEL=n and X86_FEATURE_VMX=n, either of which
> will cause vEPC initialization to fail.  The former is obvious, the latter is
> possible via BIOS configuration.

Hmm... So you make the checks as if ret != -ENODEV? That's the sane way to
deal with that situation IMHO.

/Jarkko
Huang, Kai March 18, 2021, 12:04 a.m. UTC | #16
On Tue, 16 Mar 2021 14:46:05 +0200 Jarkko Sakkinen wrote:
> On Mon, Mar 15, 2021 at 06:13:53PM -0700, Sean Christopherson wrote:
> > On Tue, Mar 16, 2021, Jarkko Sakkinen wrote:
> > > On Tue, Mar 16, 2021 at 01:05:05AM +0200, Jarkko Sakkinen wrote:
> > > > The way I've understood it is that given that KVM can support SGX
> > > > without FLC, vEPC should be available even if driver cannot be
> > > > enabled.
> > > > 
> > > > This is also exactly what the short summary states.
> > > > 
> > > > "Initialize virtual EPC driver even when SGX driver is disabled"
> > > > 
> > > > It *does not* state:
> > > > 
> > > > "Initialize SGX driver even when vEPC driver is disabled"
> > > > 
> > > > Also, this is how I interpret the inline comment.
> > > > 
> > > > All this considered, the other direction is undocumented functionality.
> > > 
> > > Also:
> > > 
> > > 1. There is *zero* good practical reasons to support the "2nd direction".
> > 
> > Uh, yes there is.  CONFIG_KVM_INTEL=n and X86_FEATURE_VMX=n, either of which
> > will cause vEPC initialization to fail.  The former is obvious, the latter is
> > possible via BIOS configuration.
> 
> Hmm... So you make the checks as if ret != -ENODEV? That's the sane way to
> deal with that situation IMHO.
> 
> /Jarkko

OK. I actually wrote the code to show your idea:

-       ret = sgx_drv_init();
-       if (ret)
+       /*
+        * Only continue to initialize SGX driver when SGX virtualization
+        * initialization is successful, or is not supported (-ENODEV), since
+        * any other initialization failure means SGX driver is unlikely to be
+        * initialized successfully.
+        */
+       ret = sgx_vepc_init();
+       if (ret && ret != -ENODEV)
                goto err_kthread;
 
+       if (sgx_drv_init()) {
+               /*
+                * Cleanup when *both* SGX virtualization and SGX driver are
+                * not enabled, due to either not supported (-ENODEV), or
+                * somehow fail to initialize.
+                */
+               if (ret && ret != -ENODEV)
+                       goto err_kthread;
+       }
+
        return 0;
 
 err_kthread:


Is this the code you want?
diff mbox series

Patch

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 44fe91a5bfb3..8c922e68274d 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -712,7 +712,15 @@  static int __init sgx_init(void)
 		goto err_page_cache;
 	}
 
-	ret = sgx_drv_init();
+	/*
+	 * Always try to initialize the native *and* KVM drivers.
+	 * The KVM driver is less picky than the native one and
+	 * can function if the native one is not supported on the
+	 * current system or fails to initialize.
+	 *
+	 * Error out only if both fail to initialize.
+	 */
+	ret = !!sgx_drv_init() & !!sgx_vepc_init();
 	if (ret)
 		goto err_kthread;