| Message ID | 1371632965-20077-8-git-send-email-xiaoguangrong@linux.vnet.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Il 19/06/2013 11:09, Xiao Guangrong ha scritto: > Document it to Documentation/virtual/kvm/mmu.txt > > Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com> > --- > Documentation/virtual/kvm/mmu.txt | 25 +++++++++++++++++++++++++ > 1 file changed, 25 insertions(+) > > diff --git a/Documentation/virtual/kvm/mmu.txt b/Documentation/virtual/kvm/mmu.txt > index f5c4de9..9b7cfb3 100644 > --- a/Documentation/virtual/kvm/mmu.txt > +++ b/Documentation/virtual/kvm/mmu.txt > @@ -396,6 +396,31 @@ ensures the old pages are not used any more. The invalid-gen pages > (sp->mmu_valid_gen != kvm->arch.mmu_valid_gen) are zapped by using lock-break > technique. > > +Fast invalidate all mmio sptes > +=========== > +As mentioned in "Reaction to events" above, kvm will cache the mmio information > +to the last sptes so that we should zap all mmio sptes when the guest mmio info > +is changed. This will happen when a new memslot is added or the existing > +memslot is moved. > + > +Zapping mmio spte is also a scalability issue for the large memory and large > +vcpus guests since it needs to hold hot mmu-lock and walk all shadow pages to > +find all the mmio spte out. > + > +We fix this issue by using the similar way of "Fast invalidate all pages". > +The global mmio valid generation-number is stored in kvm->memslots.generation > +and every mmio spte stores the current global generation-number into his > +available bits when it is created > + > +The global mmio valid generation-number is increased whenever the gust memory > +info is changed. When guests do mmio access, kvm intercepts a MMIO #PF then it > +walks the shadow page table and get the mmio spte. If the generation-number on > +the spte does not equal the global generation-number, it will go to the normal > +#PF handler to update the mmio spte. > + > +Since 19 bits are used to store generation-number on mmio spte, we zap all pages > +when the number is round. As mentioned in "Reaction to events" above, kvm will cache MMIO information in leaf sptes. When a new memslot is added or an existing memslot is changed, this information may become stale and needs to be invalidated. This also needs to hold the MMU lock while walking all shadow pages, and is made more scalable with a similar technique. MMIO sptes have a few spare bits, which are used to store a generation number. The global generation number is stored in kvm_memslots(kvm)->generation, and increased whenever guest memory info changes. This generation number is distinct from the one described in the previous section. When KVM finds an MMIO spte, it checks the generation number of the spte. If the generation number of the spte does not equal the global generation number, it will ignore the cached MMIO information and handle the page fault through the slow path. Since only 19 bits are used to store generation-number on mmio spte, all pages are zapped when there is an overflow. I'm also adding this in the page fault section: - check for valid generation number in the spte (see "Fast invalidation of MMIO sptes" below) > Further reading > =============== > > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 06/19/2013 08:35 PM, Paolo Bonzini wrote: > Il 19/06/2013 11:09, Xiao Guangrong ha scritto: >> Document it to Documentation/virtual/kvm/mmu.txt >> >> Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com> >> --- >> Documentation/virtual/kvm/mmu.txt | 25 +++++++++++++++++++++++++ >> 1 file changed, 25 insertions(+) >> >> diff --git a/Documentation/virtual/kvm/mmu.txt b/Documentation/virtual/kvm/mmu.txt >> index f5c4de9..9b7cfb3 100644 >> --- a/Documentation/virtual/kvm/mmu.txt >> +++ b/Documentation/virtual/kvm/mmu.txt >> @@ -396,6 +396,31 @@ ensures the old pages are not used any more. The invalid-gen pages >> (sp->mmu_valid_gen != kvm->arch.mmu_valid_gen) are zapped by using lock-break >> technique. >> >> +Fast invalidate all mmio sptes >> +=========== >> +As mentioned in "Reaction to events" above, kvm will cache the mmio information >> +to the last sptes so that we should zap all mmio sptes when the guest mmio info >> +is changed. This will happen when a new memslot is added or the existing >> +memslot is moved. >> + >> +Zapping mmio spte is also a scalability issue for the large memory and large >> +vcpus guests since it needs to hold hot mmu-lock and walk all shadow pages to >> +find all the mmio spte out. >> + >> +We fix this issue by using the similar way of "Fast invalidate all pages". >> +The global mmio valid generation-number is stored in kvm->memslots.generation >> +and every mmio spte stores the current global generation-number into his >> +available bits when it is created >> + >> +The global mmio valid generation-number is increased whenever the gust memory >> +info is changed. When guests do mmio access, kvm intercepts a MMIO #PF then it >> +walks the shadow page table and get the mmio spte. If the generation-number on >> +the spte does not equal the global generation-number, it will go to the normal >> +#PF handler to update the mmio spte. >> + >> +Since 19 bits are used to store generation-number on mmio spte, we zap all pages >> +when the number is round. > > As mentioned in "Reaction to events" above, kvm will cache MMIO > information in leaf sptes. When a new memslot is added or an existing > memslot is changed, this information may become stale and needs to be > invalidated. This also needs to hold the MMU lock while walking all > shadow pages, and is made more scalable with a similar technique. > > MMIO sptes have a few spare bits, which are used to store a > generation number. The global generation number is stored in > kvm_memslots(kvm)->generation, and increased whenever guest memory info > changes. This generation number is distinct from the one described in > the previous section. > > When KVM finds an MMIO spte, it checks the generation number of the spte. > If the generation number of the spte does not equal the global generation > number, it will ignore the cached MMIO information and handle the page > fault through the slow path. > > Since only 19 bits are used to store generation-number on mmio spte, all > pages are zapped when there is an overflow. Really nice. > > > I'm also adding this in the page fault section: > > - check for valid generation number in the spte (see "Fast invalidation of > MMIO sptes" below) Oh, i forgot it in this section, thanks for your fix. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 06/19/2013 04:09:25 AM, Xiao Guangrong wrote:
> Document it to Documentation/virtual/kvm/mmu.txt
Why break a change to a single documentation file into 7 pieces.
Are we going to bisect the documentation?
Rob--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Il 20/06/2013 07:21, Rob Landley ha scritto: > On 06/19/2013 04:09:25 AM, Xiao Guangrong wrote: >> Document it to Documentation/virtual/kvm/mmu.txt > > Why break a change to a single documentation file into 7 pieces. > > Are we going to bisect the documentation? It is explaining 7 different optimizations, and each patch could have been submitted together with the corresponding optimization. So it makes sense---and it certainly helped review. Paolo -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/Documentation/virtual/kvm/mmu.txt b/Documentation/virtual/kvm/mmu.txt index f5c4de9..9b7cfb3 100644 --- a/Documentation/virtual/kvm/mmu.txt +++ b/Documentation/virtual/kvm/mmu.txt @@ -396,6 +396,31 @@ ensures the old pages are not used any more. The invalid-gen pages (sp->mmu_valid_gen != kvm->arch.mmu_valid_gen) are zapped by using lock-break technique. +Fast invalidate all mmio sptes +=========== +As mentioned in "Reaction to events" above, kvm will cache the mmio information +to the last sptes so that we should zap all mmio sptes when the guest mmio info +is changed. This will happen when a new memslot is added or the existing +memslot is moved. + +Zapping mmio spte is also a scalability issue for the large memory and large +vcpus guests since it needs to hold hot mmu-lock and walk all shadow pages to +find all the mmio spte out. + +We fix this issue by using the similar way of "Fast invalidate all pages". +The global mmio valid generation-number is stored in kvm->memslots.generation +and every mmio spte stores the current global generation-number into his +available bits when it is created + +The global mmio valid generation-number is increased whenever the gust memory +info is changed. When guests do mmio access, kvm intercepts a MMIO #PF then it +walks the shadow page table and get the mmio spte. If the generation-number on +the spte does not equal the global generation-number, it will go to the normal +#PF handler to update the mmio spte. + +Since 19 bits are used to store generation-number on mmio spte, we zap all pages +when the number is round. + Further reading ===============
Document it to Documentation/virtual/kvm/mmu.txt Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com> --- Documentation/virtual/kvm/mmu.txt | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)