| Message ID | 20191215021208.GB27378@us.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [V3,1/2] KVM: PPC: Add skip_page_out parameter | expand |
On Sat, Dec 14, 2019 at 06:12:08PM -0800, Sukadev Bhattiprolu wrote: > +unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm) > +{ > + int i; > + > + if (!(kvm->arch.secure_guest & KVMPPC_SECURE_INIT_START)) > + return H_UNSUPPORTED; > + > + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { > + struct kvm_memory_slot *memslot; > + struct kvm_memslots *slots = __kvm_memslots(kvm, i); > + > + if (!slots) > + continue; > + > + kvm_for_each_memslot(memslot, slots) > + kvmppc_uvmem_drop_pages(memslot, kvm, false); > + } You need to hold srcu_read_lock(&kvm->srcu) here. Regards, Bharata.
On Sat, Dec 14, 2019 at 06:12:08PM -0800, Sukadev Bhattiprolu wrote: > > Implement the H_SVM_INIT_ABORT hcall which the Ultravisor can use to > abort an SVM after it has issued the H_SVM_INIT_START and before the > H_SVM_INIT_DONE hcalls. This hcall could be used when Ultravisor > encounters security violations or other errors when starting an SVM. > > Note that this hcall is different from UV_SVM_TERMINATE ucall which > is used by HV to terminate/cleanup an VM that has becore secure. > > The H_SVM_INIT_ABORT should basically undo operations that were done > since the H_SVM_INIT_START hcall - i.e page-out all the VM pages back > to normal memory, and terminate the SVM. > > (If we do not bring the pages back to normal memory, the text/data > of the VM would be stuck in secure memory and since the SVM did not > go secure, its MSR_S bit will be clear and the VM wont be able to > access its pages even to do a clean exit). > > Based on patches and discussion with Paul Mackerras, Ram Pai and > Bharata Rao. > > Signed-off-by: Ram Pai <linuxram@linux.ibm.com> > Signed-off-by: Sukadev Bhattiprolu <sukadev@linux.ibm.com> > Signed-off-by: Bharata B Rao <bharata@linux.ibm.com> Minor comment below, but not a showstopper. Also, as Bharata noted you need to hold the srcu lock for reading. > + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { > + struct kvm_memory_slot *memslot; > + struct kvm_memslots *slots = __kvm_memslots(kvm, i); > + > + if (!slots) > + continue; > + > + kvm_for_each_memslot(memslot, slots) > + kvmppc_uvmem_drop_pages(memslot, kvm, false); > + } Since we use the default KVM_ADDRESS_SPACE_NUM, which is 1, this code isn't wrong but it is more verbose than it needs to be. It could be kvm_for_each_memslot(kvm_memslots(kvm), slots) kvmppc_uvmem_drop_pages(memslot, kvm, false); Paul.
Bharata B Rao [bharata@linux.ibm.com] wrote: > On Sat, Dec 14, 2019 at 06:12:08PM -0800, Sukadev Bhattiprolu wrote: > > +unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm) > > +{ > > + int i; > > + > > + if (!(kvm->arch.secure_guest & KVMPPC_SECURE_INIT_START)) > > + return H_UNSUPPORTED; > > + > > + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { > > + struct kvm_memory_slot *memslot; > > + struct kvm_memslots *slots = __kvm_memslots(kvm, i); > > + > > + if (!slots) > > + continue; > > + > > + kvm_for_each_memslot(memslot, slots) > > + kvmppc_uvmem_drop_pages(memslot, kvm, false); > > + } > > You need to hold srcu_read_lock(&kvm->srcu) here. Yes, thanks! Fixed in the next version. Sukadev
Paul Mackerras [paulus@ozlabs.org] wrote: > On Sat, Dec 14, 2019 at 06:12:08PM -0800, Sukadev Bhattiprolu wrote: > > > > Implement the H_SVM_INIT_ABORT hcall which the Ultravisor can use to > > abort an SVM after it has issued the H_SVM_INIT_START and before the > > H_SVM_INIT_DONE hcalls. This hcall could be used when Ultravisor > > encounters security violations or other errors when starting an SVM. > > > > Note that this hcall is different from UV_SVM_TERMINATE ucall which > > is used by HV to terminate/cleanup an VM that has becore secure. > > > > The H_SVM_INIT_ABORT should basically undo operations that were done > > since the H_SVM_INIT_START hcall - i.e page-out all the VM pages back > > to normal memory, and terminate the SVM. > > > > (If we do not bring the pages back to normal memory, the text/data > > of the VM would be stuck in secure memory and since the SVM did not > > go secure, its MSR_S bit will be clear and the VM wont be able to > > access its pages even to do a clean exit). > > > > Based on patches and discussion with Paul Mackerras, Ram Pai and > > Bharata Rao. > > > > Signed-off-by: Ram Pai <linuxram@linux.ibm.com> > > Signed-off-by: Sukadev Bhattiprolu <sukadev@linux.ibm.com> > > Signed-off-by: Bharata B Rao <bharata@linux.ibm.com> > > Minor comment below, but not a showstopper. Also, as Bharata noted > you need to hold the srcu lock for reading. Yes, I fixed that. > > > + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { > > + struct kvm_memory_slot *memslot; > > + struct kvm_memslots *slots = __kvm_memslots(kvm, i); > > + > > + if (!slots) > > + continue; > > + > > + kvm_for_each_memslot(memslot, slots) > > + kvmppc_uvmem_drop_pages(memslot, kvm, false); > > + } > > Since we use the default KVM_ADDRESS_SPACE_NUM, which is 1, this code > isn't wrong but it is more verbose than it needs to be. It could be > > kvm_for_each_memslot(kvm_memslots(kvm), slots) > kvmppc_uvmem_drop_pages(memslot, kvm, false); and simplified this. Thanks. Sukadev
On Sat, Dec 14, 2019 at 06:12:08PM -0800, Sukadev Bhattiprolu wrote: > > Implement the H_SVM_INIT_ABORT hcall which the Ultravisor can use to > abort an SVM after it has issued the H_SVM_INIT_START and before the > H_SVM_INIT_DONE hcalls. This hcall could be used when Ultravisor > encounters security violations or other errors when starting an SVM. > > Note that this hcall is different from UV_SVM_TERMINATE ucall which > is used by HV to terminate/cleanup an VM that has becore secure. > > The H_SVM_INIT_ABORT should basically undo operations that were done > since the H_SVM_INIT_START hcall - i.e page-out all the VM pages back > to normal memory, and terminate the SVM. > > (If we do not bring the pages back to normal memory, the text/data > of the VM would be stuck in secure memory and since the SVM did not > go secure, its MSR_S bit will be clear and the VM wont be able to > access its pages even to do a clean exit). > > Based on patches and discussion with Paul Mackerras, Ram Pai and > Bharata Rao. > > Signed-off-by: Ram Pai <linuxram@linux.ibm.com> > Signed-off-by: Sukadev Bhattiprolu <sukadev@linux.ibm.com> > Signed-off-by: Bharata B Rao <bharata@linux.ibm.com> > --- > Changelog[v3]: > - Rather than pass the NIP/MSR as parameters, load them into > SRR0/SRR1 (like we do with other registers) and terminate > the VM after paging out pages > - Move the code to add a skip_page_out parameter into a > separate patch. > > Changelog[v2]: > [Paul Mackerras] avoid returning to UV "one last time" after > the state is cleaned up. So, we now have H_SVM_INIT_ABORT: > - take the VM's NIP/MSR register states as parameters > - inherit the state of other registers as at UV_ESM call. > After cleaning up the partial state, HV uses these to return > directly to the VM with a failed UV_ESM call. > --- > Documentation/powerpc/ultravisor.rst | 57 +++++++++++++++++++++ > arch/powerpc/include/asm/hvcall.h | 1 + > arch/powerpc/include/asm/kvm_book3s_uvmem.h | 6 +++ > arch/powerpc/include/asm/kvm_host.h | 1 + > arch/powerpc/kvm/book3s_hv.c | 3 ++ > arch/powerpc/kvm/book3s_hv_uvmem.c | 24 +++++++++ > 6 files changed, 92 insertions(+) > > diff --git a/Documentation/powerpc/ultravisor.rst b/Documentation/powerpc/ultravisor.rst > index 730854f73830..8c114c071bfa 100644 > --- a/Documentation/powerpc/ultravisor.rst > +++ b/Documentation/powerpc/ultravisor.rst > @@ -948,6 +948,63 @@ Use cases > up its internal state for this virtual machine. > > > +H_SVM_INIT_ABORT > +---------------- > + > + Abort the process of securing an SVM. > + > +Syntax > +~~~~~~ > + > +.. code-block:: c > + > + uint64_t hypercall(const uint64_t H_SVM_INIT_ABORT) > + > +Return values > +~~~~~~~~~~~~~ > + > + One of the following values: > + > + * H_PARAMETER on successfully cleaning up the state, > + Hypervisor will return this value to the > + **guest**, to indicate that the underlying > + UV_ESM ultracall failed. > + > + * H_UNSUPPORTED if called from the wrong context (e.g. from > + an SVM or before an H_SVM_INIT_START hypercall). > + > +Description > +~~~~~~~~~~~ > + > + Abort the process of securing a virtual machine. This call must > + be made after a prior call to ``H_SVM_INIT_START`` hypercall and > + before a call to ``H_SVM_INIT_DONE``. > + > + On entry into this hypercall the non-volatile GPRs and FPRs are > + expected to contain the values they had at the time the VM issued > + the UV_ESM ultracall. Further ``SRR0`` is expected to contain the > + address of the instruction after the ``UV_ESM`` ultracall and ``SRR1`` > + the MSR value with which to return to the VM. > + > + This hypercall will cleanup any partial state that was established for > + the VM since the prior ``H_SVM_INIT_START`` hypercall, including paging > + out pages that were paged-into secure memory, and issue the > + ``UV_SVM_TERMINATE`` ultracall to terminate the VM. > + > + After the partial state is cleaned up, control returns to the VM > + (**not Ultravisor**), at the address specified in ``SRR0`` with the > + MSR values set to the value in ``SRR1``. > + > +Use cases > +~~~~~~~~~ > + > + If after a successful call to ``H_SVM_INIT_START``, the Ultravisor > + encounters an error while securing a virtual machine, either due > + to lack of resources or because the VM's security information could > + not be validated, Ultravisor informs the Hypervisor about it. > + Hypervisor should use this call to clean up any internal state for > + this virtual machine and return to the VM. > + > H_SVM_PAGE_IN > ------------- > > diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h > index 13bd870609c3..e90c073e437e 100644 > --- a/arch/powerpc/include/asm/hvcall.h > +++ b/arch/powerpc/include/asm/hvcall.h > @@ -350,6 +350,7 @@ > #define H_SVM_PAGE_OUT 0xEF04 > #define H_SVM_INIT_START 0xEF08 > #define H_SVM_INIT_DONE 0xEF0C > +#define H_SVM_INIT_ABORT 0xEF14 > > /* Values for 2nd argument to H_SET_MODE */ > #define H_SET_MODE_RESOURCE_SET_CIABR 1 > diff --git a/arch/powerpc/include/asm/kvm_book3s_uvmem.h b/arch/powerpc/include/asm/kvm_book3s_uvmem.h > index 3cf8425b9838..5a9834e0e2d1 100644 > --- a/arch/powerpc/include/asm/kvm_book3s_uvmem.h > +++ b/arch/powerpc/include/asm/kvm_book3s_uvmem.h > @@ -19,6 +19,7 @@ unsigned long kvmppc_h_svm_page_out(struct kvm *kvm, > unsigned long kvmppc_h_svm_init_start(struct kvm *kvm); > unsigned long kvmppc_h_svm_init_done(struct kvm *kvm); > int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn); > +unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm); > void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free, > struct kvm *kvm, bool skip_page_out); > #else > @@ -62,6 +63,11 @@ static inline unsigned long kvmppc_h_svm_init_done(struct kvm *kvm) > return H_UNSUPPORTED; > } > > +static inline unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm) > +{ > + return H_UNSUPPORTED; > +} > + > static inline int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn) > { > return -EFAULT; > diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h > index 577ca95fac7c..8310c0407383 100644 > --- a/arch/powerpc/include/asm/kvm_host.h > +++ b/arch/powerpc/include/asm/kvm_host.h > @@ -278,6 +278,7 @@ struct kvm_resize_hpt; > /* Flag values for kvm_arch.secure_guest */ > #define KVMPPC_SECURE_INIT_START 0x1 /* H_SVM_INIT_START has been called */ > #define KVMPPC_SECURE_INIT_DONE 0x2 /* H_SVM_INIT_DONE completed */ > +#define KVMPPC_SECURE_INIT_ABORT 0x4 /* H_SVM_INIT_ABORT issued */ > > struct kvm_arch { > unsigned int lpid; > diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c > index 66d5312be16b..1b22f2c7ad1b 100644 > --- a/arch/powerpc/kvm/book3s_hv.c > +++ b/arch/powerpc/kvm/book3s_hv.c > @@ -1099,6 +1099,9 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) > case H_SVM_INIT_DONE: > ret = kvmppc_h_svm_init_done(vcpu->kvm); > break; > + case H_SVM_INIT_ABORT: > + ret = kvmppc_h_svm_init_abort(vcpu->kvm); > + break; > > default: > return RESUME_HOST; > diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c > index 9a5bbad7d87e..f7df8c327468 100644 > --- a/arch/powerpc/kvm/book3s_hv_uvmem.c > +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c > @@ -287,6 +287,30 @@ void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free, > } > } > > +unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm) > +{ > + int i; > + > + if (!(kvm->arch.secure_guest & KVMPPC_SECURE_INIT_START)) > + return H_UNSUPPORTED; It should also return H_UNSUPPORTED when (kvm->arch.secure_guest & KVMPPC_SECURE_INIT_DONE) is true. RP
Ram Pai [linuxram@us.ibm.com] wrote: > > +unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm) > > +{ > > + int i; > > + > > + if (!(kvm->arch.secure_guest & KVMPPC_SECURE_INIT_START)) > > + return H_UNSUPPORTED; > > It should also return H_UNSUPPORTED when > (kvm->arch.secure_guest & KVMPPC_SECURE_INIT_DONE) is true. If KVMPPC_SECURE_INIT_DONE is set, KVMPPC_SECURE_INIT_START is also set - we never clear KVMPPC_SECURE_INIT_START right? Sukadev
On Thu, Jan 02, 2020 at 04:32:33PM -0800, Sukadev Bhattiprolu wrote: > Ram Pai [linuxram@us.ibm.com] wrote: > > > +unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm) > > > +{ > > > + int i; > > > + > > > + if (!(kvm->arch.secure_guest & KVMPPC_SECURE_INIT_START)) > > > + return H_UNSUPPORTED; > > > > It should also return H_UNSUPPORTED when > > (kvm->arch.secure_guest & KVMPPC_SECURE_INIT_DONE) is true. > > If KVMPPC_SECURE_INIT_DONE is set, KVMPPC_SECURE_INIT_START is also > set - we never clear KVMPPC_SECURE_INIT_START right? I am concerned about the case, where the VM has successfully transitioned into a SVM, where both KVMPPC_SECURE_INIT_DONE and KVMPPC_SECURE_INIT_START are set. In this scenario, if the UV makes a H_SVM_INIT_ABORT hcall, the Hypervisor will not return H_UNSUPPORTED, because KVMPPC_SECURE_INIT_START is set. That is the reason, I think, we need to add another check as below. if (kvm->arch.secure_guest & KVMPPC_SECURE_INIT_DONE) return H_UNSUPPORTED; > > Sukadev
diff --git a/Documentation/powerpc/ultravisor.rst b/Documentation/powerpc/ultravisor.rst index 730854f73830..8c114c071bfa 100644 --- a/Documentation/powerpc/ultravisor.rst +++ b/Documentation/powerpc/ultravisor.rst @@ -948,6 +948,63 @@ Use cases up its internal state for this virtual machine. +H_SVM_INIT_ABORT +---------------- + + Abort the process of securing an SVM. + +Syntax +~~~~~~ + +.. code-block:: c + + uint64_t hypercall(const uint64_t H_SVM_INIT_ABORT) + +Return values +~~~~~~~~~~~~~ + + One of the following values: + + * H_PARAMETER on successfully cleaning up the state, + Hypervisor will return this value to the + **guest**, to indicate that the underlying + UV_ESM ultracall failed. + + * H_UNSUPPORTED if called from the wrong context (e.g. from + an SVM or before an H_SVM_INIT_START hypercall). + +Description +~~~~~~~~~~~ + + Abort the process of securing a virtual machine. This call must + be made after a prior call to ``H_SVM_INIT_START`` hypercall and + before a call to ``H_SVM_INIT_DONE``. + + On entry into this hypercall the non-volatile GPRs and FPRs are + expected to contain the values they had at the time the VM issued + the UV_ESM ultracall. Further ``SRR0`` is expected to contain the + address of the instruction after the ``UV_ESM`` ultracall and ``SRR1`` + the MSR value with which to return to the VM. + + This hypercall will cleanup any partial state that was established for + the VM since the prior ``H_SVM_INIT_START`` hypercall, including paging + out pages that were paged-into secure memory, and issue the + ``UV_SVM_TERMINATE`` ultracall to terminate the VM. + + After the partial state is cleaned up, control returns to the VM + (**not Ultravisor**), at the address specified in ``SRR0`` with the + MSR values set to the value in ``SRR1``. + +Use cases +~~~~~~~~~ + + If after a successful call to ``H_SVM_INIT_START``, the Ultravisor + encounters an error while securing a virtual machine, either due + to lack of resources or because the VM's security information could + not be validated, Ultravisor informs the Hypervisor about it. + Hypervisor should use this call to clean up any internal state for + this virtual machine and return to the VM. + H_SVM_PAGE_IN ------------- diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h index 13bd870609c3..e90c073e437e 100644 --- a/arch/powerpc/include/asm/hvcall.h +++ b/arch/powerpc/include/asm/hvcall.h @@ -350,6 +350,7 @@ #define H_SVM_PAGE_OUT 0xEF04 #define H_SVM_INIT_START 0xEF08 #define H_SVM_INIT_DONE 0xEF0C +#define H_SVM_INIT_ABORT 0xEF14 /* Values for 2nd argument to H_SET_MODE */ #define H_SET_MODE_RESOURCE_SET_CIABR 1 diff --git a/arch/powerpc/include/asm/kvm_book3s_uvmem.h b/arch/powerpc/include/asm/kvm_book3s_uvmem.h index 3cf8425b9838..5a9834e0e2d1 100644 --- a/arch/powerpc/include/asm/kvm_book3s_uvmem.h +++ b/arch/powerpc/include/asm/kvm_book3s_uvmem.h @@ -19,6 +19,7 @@ unsigned long kvmppc_h_svm_page_out(struct kvm *kvm, unsigned long kvmppc_h_svm_init_start(struct kvm *kvm); unsigned long kvmppc_h_svm_init_done(struct kvm *kvm); int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn); +unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm); void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free, struct kvm *kvm, bool skip_page_out); #else @@ -62,6 +63,11 @@ static inline unsigned long kvmppc_h_svm_init_done(struct kvm *kvm) return H_UNSUPPORTED; } +static inline unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm) +{ + return H_UNSUPPORTED; +} + static inline int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn) { return -EFAULT; diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h index 577ca95fac7c..8310c0407383 100644 --- a/arch/powerpc/include/asm/kvm_host.h +++ b/arch/powerpc/include/asm/kvm_host.h @@ -278,6 +278,7 @@ struct kvm_resize_hpt; /* Flag values for kvm_arch.secure_guest */ #define KVMPPC_SECURE_INIT_START 0x1 /* H_SVM_INIT_START has been called */ #define KVMPPC_SECURE_INIT_DONE 0x2 /* H_SVM_INIT_DONE completed */ +#define KVMPPC_SECURE_INIT_ABORT 0x4 /* H_SVM_INIT_ABORT issued */ struct kvm_arch { unsigned int lpid; diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 66d5312be16b..1b22f2c7ad1b 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1099,6 +1099,9 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) case H_SVM_INIT_DONE: ret = kvmppc_h_svm_init_done(vcpu->kvm); break; + case H_SVM_INIT_ABORT: + ret = kvmppc_h_svm_init_abort(vcpu->kvm); + break; default: return RESUME_HOST; diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c index 9a5bbad7d87e..f7df8c327468 100644 --- a/arch/powerpc/kvm/book3s_hv_uvmem.c +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c @@ -287,6 +287,30 @@ void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free, } } +unsigned long kvmppc_h_svm_init_abort(struct kvm *kvm) +{ + int i; + + if (!(kvm->arch.secure_guest & KVMPPC_SECURE_INIT_START)) + return H_UNSUPPORTED; + + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { + struct kvm_memory_slot *memslot; + struct kvm_memslots *slots = __kvm_memslots(kvm, i); + + if (!slots) + continue; + + kvm_for_each_memslot(memslot, slots) + kvmppc_uvmem_drop_pages(memslot, kvm, false); + } + + kvm->arch.secure_guest = 0; + uv_svm_terminate(kvm->arch.lpid); + + return H_PARAMETER; +} + /* * Get a free device PFN from the pool *