diff mbox series

[v7,3/4] KVM: x86/mmu: count KVM mmu usage in secondary pagetable stats.

Message ID 20220823004639.2387269-4-yosryahmed@google.com (mailing list archive)
State New
Headers show
Series KVM: mm: count KVM mmu usage in memory stats | expand

Commit Message

Yosry Ahmed Aug. 23, 2022, 12:46 a.m. UTC 
Count the pages used by KVM mmu on x86 in memory stats under secondary
pagetable stats (e.g. "SecPageTables" in /proc/meminfo) to give better
visibility into the memory consumption of KVM mmu in a similar way to
how normal user page tables are accounted.

Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
Reviewed-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/mmu/mmu.c     | 16 ++++++++++++++--
 arch/x86/kvm/mmu/tdp_mmu.c | 12 ++++++++++++
 2 files changed, 26 insertions(+), 2 deletions(-)

Comments

Sean Christopherson Aug. 26, 2022, 8:20 p.m. UTC | #1 
On Tue, Aug 23, 2022, Yosry Ahmed wrote:
> Count the pages used by KVM mmu on x86 in memory stats under secondary
> pagetable stats (e.g. "SecPageTables" in /proc/meminfo) to give better
> visibility into the memory consumption of KVM mmu in a similar way to
> how normal user page tables are accounted.
> 
> Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
> Reviewed-by: Sean Christopherson <seanjc@google.com>
> ---
>  arch/x86/kvm/mmu/mmu.c     | 16 ++++++++++++++--
>  arch/x86/kvm/mmu/tdp_mmu.c | 12 ++++++++++++
>  2 files changed, 26 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index e418ef3ecfcb..4d38e4eba772 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -1665,6 +1665,18 @@ static inline void kvm_mod_used_mmu_pages(struct kvm *kvm, long nr)
>  	percpu_counter_add(&kvm_total_used_mmu_pages, nr);
>  }
>  
> +static void kvm_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
> +{
> +	kvm_mod_used_mmu_pages(kvm, +1);
> +	kvm_account_pgtable_pages((void *)sp->spt, +1);
> +}
> +
> +static void kvm_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
> +{
> +	kvm_mod_used_mmu_pages(kvm, -1);
> +	kvm_account_pgtable_pages((void *)sp->spt, -1);
> +}

Hrm, this is causing build on x86 issues for me.  AFAICT, modpost doesn't detect
that this creates a new module dependency on __mod_lruvec_page_state() and so doesn't
refresh vmlinux.symvers.

  ERROR: modpost: "__mod_lruvec_page_state" [arch/x86/kvm/kvm.ko] undefined!
  make[2]: *** [scripts/Makefile.modpost:128: modules-only.symvers] Error 1
  make[1]: *** [Makefile:1769: modules] Error 2
  make[1]: *** Waiting for unfinished jobs....
  Kernel: arch/x86/boot/bzImage is ready  (#128)
  make[1]: Leaving directory '/usr/local/google/home/seanjc/build/kernel/vm'
  make: *** [Makefile:222: __sub-make] Error 2

Both gcc and clang yield the same behavior, so I doubt it's the compiler doing
something odd.  Cleaning the build makes the problem go away, but that's a poor
band-aid.

If I squash this with the prior patch that adds kvm_account_pgtable_pages() to
kvm_host.h, modpost detects the need to refresh and all is well.

Given that ARM doesn't support building KVM as a module, i.e. can't run afoul
of whatever modpost weirdness I'm hitting, I'm inclined to squash this with the
previous patch and punt on the modpost issue so that we can get this merged.

Any objections?  Or thoughts on what's going wrong?
Yosry Ahmed Aug. 26, 2022, 9:25 p.m. UTC | #2 
On Fri, Aug 26, 2022 at 1:20 PM Sean Christopherson <seanjc@google.com> wrote:
>
> On Tue, Aug 23, 2022, Yosry Ahmed wrote:
> > Count the pages used by KVM mmu on x86 in memory stats under secondary
> > pagetable stats (e.g. "SecPageTables" in /proc/meminfo) to give better
> > visibility into the memory consumption of KVM mmu in a similar way to
> > how normal user page tables are accounted.
> >
> > Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
> > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > ---
> >  arch/x86/kvm/mmu/mmu.c     | 16 ++++++++++++++--
> >  arch/x86/kvm/mmu/tdp_mmu.c | 12 ++++++++++++
> >  2 files changed, 26 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> > index e418ef3ecfcb..4d38e4eba772 100644
> > --- a/arch/x86/kvm/mmu/mmu.c
> > +++ b/arch/x86/kvm/mmu/mmu.c
> > @@ -1665,6 +1665,18 @@ static inline void kvm_mod_used_mmu_pages(struct kvm *kvm, long nr)
> >       percpu_counter_add(&kvm_total_used_mmu_pages, nr);
> >  }
> >
> > +static void kvm_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
> > +{
> > +     kvm_mod_used_mmu_pages(kvm, +1);
> > +     kvm_account_pgtable_pages((void *)sp->spt, +1);
> > +}
> > +
> > +static void kvm_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
> > +{
> > +     kvm_mod_used_mmu_pages(kvm, -1);
> > +     kvm_account_pgtable_pages((void *)sp->spt, -1);
> > +}
>
> Hrm, this is causing build on x86 issues for me.  AFAICT, modpost doesn't detect
> that this creates a new module dependency on __mod_lruvec_page_state() and so doesn't
> refresh vmlinux.symvers.
>
>   ERROR: modpost: "__mod_lruvec_page_state" [arch/x86/kvm/kvm.ko] undefined!
>   make[2]: *** [scripts/Makefile.modpost:128: modules-only.symvers] Error 1
>   make[1]: *** [Makefile:1769: modules] Error 2
>   make[1]: *** Waiting for unfinished jobs....
>   Kernel: arch/x86/boot/bzImage is ready  (#128)
>   make[1]: Leaving directory '/usr/local/google/home/seanjc/build/kernel/vm'
>   make: *** [Makefile:222: __sub-make] Error 2
>
> Both gcc and clang yield the same behavior, so I doubt it's the compiler doing
> something odd.  Cleaning the build makes the problem go away, but that's a poor
> band-aid.
>
> If I squash this with the prior patch that adds kvm_account_pgtable_pages() to
> kvm_host.h, modpost detects the need to refresh and all is well.
>
> Given that ARM doesn't support building KVM as a module, i.e. can't run afoul
> of whatever modpost weirdness I'm hitting, I'm inclined to squash this with the
> previous patch and punt on the modpost issue so that we can get this merged.
>
> Any objections?  Or thoughts on what's going wrong?

I am not familiar at all with modpost so I have no idea what's going
wrong, but I have no problem with squashing the patches if you think
this works best.
Sean Christopherson Aug. 30, 2022, 9:52 p.m. UTC | #3 
On Fri, Aug 26, 2022, Sean Christopherson wrote:
> On Tue, Aug 23, 2022, Yosry Ahmed wrote:
> > Count the pages used by KVM mmu on x86 in memory stats under secondary
> > pagetable stats (e.g. "SecPageTables" in /proc/meminfo) to give better
> > visibility into the memory consumption of KVM mmu in a similar way to
> > how normal user page tables are accounted.
> > 
> > Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
> > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > ---
> >  arch/x86/kvm/mmu/mmu.c     | 16 ++++++++++++++--
> >  arch/x86/kvm/mmu/tdp_mmu.c | 12 ++++++++++++
> >  2 files changed, 26 insertions(+), 2 deletions(-)
> > 
> > diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> > index e418ef3ecfcb..4d38e4eba772 100644
> > --- a/arch/x86/kvm/mmu/mmu.c
> > +++ b/arch/x86/kvm/mmu/mmu.c
> > @@ -1665,6 +1665,18 @@ static inline void kvm_mod_used_mmu_pages(struct kvm *kvm, long nr)
> >  	percpu_counter_add(&kvm_total_used_mmu_pages, nr);
> >  }
> >  
> > +static void kvm_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
> > +{
> > +	kvm_mod_used_mmu_pages(kvm, +1);
> > +	kvm_account_pgtable_pages((void *)sp->spt, +1);
> > +}
> > +
> > +static void kvm_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
> > +{
> > +	kvm_mod_used_mmu_pages(kvm, -1);
> > +	kvm_account_pgtable_pages((void *)sp->spt, -1);
> > +}
> 
> Hrm, this is causing build on x86 issues for me.  AFAICT, modpost doesn't detect
> that this creates a new module dependency on __mod_lruvec_page_state() and so doesn't
> refresh vmlinux.symvers.
> 
>   ERROR: modpost: "__mod_lruvec_page_state" [arch/x86/kvm/kvm.ko] undefined!
>   make[2]: *** [scripts/Makefile.modpost:128: modules-only.symvers] Error 1
>   make[1]: *** [Makefile:1769: modules] Error 2
>   make[1]: *** Waiting for unfinished jobs....
>   Kernel: arch/x86/boot/bzImage is ready  (#128)
>   make[1]: Leaving directory '/usr/local/google/home/seanjc/build/kernel/vm'
>   make: *** [Makefile:222: __sub-make] Error 2
> 
> Both gcc and clang yield the same behavior, so I doubt it's the compiler doing
> something odd.  Cleaning the build makes the problem go away, but that's a poor
> band-aid.
> 
> If I squash this with the prior patch that adds kvm_account_pgtable_pages() to
> kvm_host.h, modpost detects the need to refresh and all is well.
> 
> Given that ARM doesn't support building KVM as a module, i.e. can't run afoul
> of whatever modpost weirdness I'm hitting, I'm inclined to squash this with the
> previous patch and punt on the modpost issue so that we can get this merged.
> 
> Any objections?  Or thoughts on what's going wrong?

Pushed the series with the squash to branch `for_paolo/6.1` at:

    https://github.com/sean-jc/linux.git

Unless you hear otherwise, it will make its way to kvm/queue "soon".

Please yell if there are objections.
Yosry Ahmed Aug. 30, 2022, 11:48 p.m. UTC | #4 
On Tue, Aug 30, 2022 at 2:52 PM Sean Christopherson <seanjc@google.com> wrote:
>
> On Fri, Aug 26, 2022, Sean Christopherson wrote:
> > On Tue, Aug 23, 2022, Yosry Ahmed wrote:
> > > Count the pages used by KVM mmu on x86 in memory stats under secondary
> > > pagetable stats (e.g. "SecPageTables" in /proc/meminfo) to give better
> > > visibility into the memory consumption of KVM mmu in a similar way to
> > > how normal user page tables are accounted.
> > >
> > > Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
> > > Reviewed-by: Sean Christopherson <seanjc@google.com>
> > > ---
> > >  arch/x86/kvm/mmu/mmu.c     | 16 ++++++++++++++--
> > >  arch/x86/kvm/mmu/tdp_mmu.c | 12 ++++++++++++
> > >  2 files changed, 26 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> > > index e418ef3ecfcb..4d38e4eba772 100644
> > > --- a/arch/x86/kvm/mmu/mmu.c
> > > +++ b/arch/x86/kvm/mmu/mmu.c
> > > @@ -1665,6 +1665,18 @@ static inline void kvm_mod_used_mmu_pages(struct kvm *kvm, long nr)
> > >     percpu_counter_add(&kvm_total_used_mmu_pages, nr);
> > >  }
> > >
> > > +static void kvm_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
> > > +{
> > > +   kvm_mod_used_mmu_pages(kvm, +1);
> > > +   kvm_account_pgtable_pages((void *)sp->spt, +1);
> > > +}
> > > +
> > > +static void kvm_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
> > > +{
> > > +   kvm_mod_used_mmu_pages(kvm, -1);
> > > +   kvm_account_pgtable_pages((void *)sp->spt, -1);
> > > +}
> >
> > Hrm, this is causing build on x86 issues for me.  AFAICT, modpost doesn't detect
> > that this creates a new module dependency on __mod_lruvec_page_state() and so doesn't
> > refresh vmlinux.symvers.
> >
> >   ERROR: modpost: "__mod_lruvec_page_state" [arch/x86/kvm/kvm.ko] undefined!
> >   make[2]: *** [scripts/Makefile.modpost:128: modules-only.symvers] Error 1
> >   make[1]: *** [Makefile:1769: modules] Error 2
> >   make[1]: *** Waiting for unfinished jobs....
> >   Kernel: arch/x86/boot/bzImage is ready  (#128)
> >   make[1]: Leaving directory '/usr/local/google/home/seanjc/build/kernel/vm'
> >   make: *** [Makefile:222: __sub-make] Error 2
> >
> > Both gcc and clang yield the same behavior, so I doubt it's the compiler doing
> > something odd.  Cleaning the build makes the problem go away, but that's a poor
> > band-aid.
> >
> > If I squash this with the prior patch that adds kvm_account_pgtable_pages() to
> > kvm_host.h, modpost detects the need to refresh and all is well.
> >
> > Given that ARM doesn't support building KVM as a module, i.e. can't run afoul
> > of whatever modpost weirdness I'm hitting, I'm inclined to squash this with the
> > previous patch and punt on the modpost issue so that we can get this merged.
> >
> > Any objections?  Or thoughts on what's going wrong?
>
> Pushed the series with the squash to branch `for_paolo/6.1` at:
>

Thanks Sean!

>     https://github.com/sean-jc/linux.git
>
> Unless you hear otherwise, it will make its way to kvm/queue "soon".
>
> Please yell if there are objections.
diff mbox series

Patch

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index e418ef3ecfcb..4d38e4eba772 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -1665,6 +1665,18 @@  static inline void kvm_mod_used_mmu_pages(struct kvm *kvm, long nr)
 	percpu_counter_add(&kvm_total_used_mmu_pages, nr);
 }
 
+static void kvm_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
+{
+	kvm_mod_used_mmu_pages(kvm, +1);
+	kvm_account_pgtable_pages((void *)sp->spt, +1);
+}
+
+static void kvm_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
+{
+	kvm_mod_used_mmu_pages(kvm, -1);
+	kvm_account_pgtable_pages((void *)sp->spt, -1);
+}
+
 static void kvm_mmu_free_shadow_page(struct kvm_mmu_page *sp)
 {
 	MMU_WARN_ON(!is_empty_shadow_page(sp->spt));
@@ -2122,7 +2134,7 @@  static struct kvm_mmu_page *kvm_mmu_alloc_shadow_page(struct kvm *kvm,
 	 */
 	sp->mmu_valid_gen = kvm->arch.mmu_valid_gen;
 	list_add(&sp->link, &kvm->arch.active_mmu_pages);
-	kvm_mod_used_mmu_pages(kvm, +1);
+	kvm_account_mmu_page(kvm, sp);
 
 	sp->gfn = gfn;
 	sp->role = role;
@@ -2456,7 +2468,7 @@  static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm,
 			list_add(&sp->link, invalid_list);
 		else
 			list_move(&sp->link, invalid_list);
-		kvm_mod_used_mmu_pages(kvm, -1);
+		kvm_unaccount_mmu_page(kvm, sp);
 	} else {
 		/*
 		 * Remove the active root from the active page list, the root
diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index bf2ccf9debca..672f0432d777 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -372,6 +372,16 @@  static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn,
 	}
 }
 
+static void tdp_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
+{
+	kvm_account_pgtable_pages((void *)sp->spt, +1);
+}
+
+static void tdp_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
+{
+	kvm_account_pgtable_pages((void *)sp->spt, -1);
+}
+
 /**
  * tdp_mmu_unlink_sp() - Remove a shadow page from the list of used pages
  *
@@ -384,6 +394,7 @@  static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn,
 static void tdp_mmu_unlink_sp(struct kvm *kvm, struct kvm_mmu_page *sp,
 			      bool shared)
 {
+	tdp_unaccount_mmu_page(kvm, sp);
 	if (shared)
 		spin_lock(&kvm->arch.tdp_mmu_pages_lock);
 	else
@@ -1132,6 +1143,7 @@  static int tdp_mmu_link_sp(struct kvm *kvm, struct tdp_iter *iter,
 	if (account_nx)
 		account_huge_nx_page(kvm, sp);
 	spin_unlock(&kvm->arch.tdp_mmu_pages_lock);
+	tdp_account_mmu_page(kvm, sp);
 
 	return 0;
 }