| Message ID | 20230811045127.3308641-1-rananta@google.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | KVM: arm64: Add support for FEAT_TLBIRANGE | expand |
On Fri, Aug 11, 2023, Raghavendra Rao Ananta wrote:
> The series is based off of upstream v6.5-rc1.
Lies! :-)
This is based off one of the kvmarm.git topic branches (I didn't bother to figure
out which one), not v6.5-rc1.
Please try to incorporate git format-patch's "--base" option into your workflow,
e.g. I do "git format-patch --base=HEAD~$nr" where $nr is the number of patches
I am posting.
It's not foolproof, e.g. my approach doesn't help if I have a local patch that
I'm not posting, but 99% of the time it Just Works and eliminates any ambuitity.
You can also do "--base=auto", but that only does the right thing if your series
has its upstream branch set to the base/tree that you want your patches applied
to (I use the upstream branch for a completely different purpose for my dev branches).
On Mon, Aug 14, 2023 at 4:16 PM Sean Christopherson <seanjc@google.com> wrote: > > On Fri, Aug 11, 2023, Raghavendra Rao Ananta wrote: > > The series is based off of upstream v6.5-rc1. > > Lies! :-) > > This is based off one of the kvmarm.git topic branches (I didn't bother to figure > out which one), not v6.5-rc1. > Sorry, what am I missing here? My git log is as follows: $ git log --oneline upstream_tlbi_range_v9 5025857507abe (upstream_tlbi_range_v9) KVM: arm64: Use TLBI range-based instructions for unmap 5c0291b99a8fc KVM: arm64: Invalidate the table entries upon a range 8c46b54d4aaec KVM: arm64: Flush only the memslot after write-protect 231abaeb7ffc2 KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range() 5ec291b863309 KVM: arm64: Define kvm_tlb_flush_vmid_range() 5bcd7a085c34e KVM: arm64: Implement __kvm_tlb_flush_vmid_range() ea08f9dff7e5b arm64: tlb: Implement __flush_s2_tlb_range_op() b3178687947c9 arm64: tlb: Refactor the core flush algorithm of __flush_tlb_range a4850fa988eef KVM: Move kvm_arch_flush_remote_tlbs_memslot() to common code 306dc4e6afd37 KVM: Allow range-based TLB invalidation from common code d02785a0a1e01 KVM: Remove CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL 136fa2d254537 KVM: arm64: Use kvm_arch_flush_remote_tlbs() e35c68a75170d KVM: Declare kvm_arch_flush_remote_tlbs() globally 5d592777b9bba KVM: Rename kvm_arch_flush_remote_tlb() to kvm_arch_flush_remote_tlbs() 06c2afb862f9d (tag: v6.5-rc1, tag: linux/v6.5-rc1) Linux 6.5-rc1 c192ac7357683 MAINTAINERS 2: Electric Boogaloo f71f64210d698 Merge tag 'dma-mapping-6.5-2023-07-09' of git://git.infradead.org/users/hch/dma-mapping ... Isn't the commit, 06c2afb862f9d (06c2afb862f9d (tag: v6.5-rc1, tag: linux/v6.5-rc1) Linux 6.5-rc1) the 'base' commit? Thank you. Raghavendra > Please try to incorporate git format-patch's "--base" option into your workflow, > e.g. I do "git format-patch --base=HEAD~$nr" where $nr is the number of patches > I am posting. > > It's not foolproof, e.g. my approach doesn't help if I have a local patch that > I'm not posting, but 99% of the time it Just Works and eliminates any ambuitity. > > You can also do "--base=auto", but that only does the right thing if your series > has its upstream branch set to the base/tree that you want your patches applied > to (I use the upstream branch for a completely different purpose for my dev branches).
On Mon, Aug 14, 2023, Raghavendra Rao Ananta wrote: > On Mon, Aug 14, 2023 at 4:16 PM Sean Christopherson <seanjc@google.com> wrote: > > > > On Fri, Aug 11, 2023, Raghavendra Rao Ananta wrote: > > > The series is based off of upstream v6.5-rc1. > > > > Lies! :-) > > > > This is based off one of the kvmarm.git topic branches (I didn't bother to figure > > out which one), not v6.5-rc1. > > > Sorry, what am I missing here? My git log is as follows: Hmm, not sure what's going on. Maybe I misinterpreted why `git am` was failing? I assumed it was because there were objects in kvmarm that I didn't have locally, and fetching kvmarm allowed am to complete, though with 3-way merges, which IIUC shouldn't have been necessary if I was using the exact same base. Or maybe I messed up and didn't actually reset to 6.5-rc1. > $ git log --oneline upstream_tlbi_range_v9 > 5025857507abe (upstream_tlbi_range_v9) KVM: arm64: Use TLBI > range-based instructions for unmap > 5c0291b99a8fc KVM: arm64: Invalidate the table entries upon a range > 8c46b54d4aaec KVM: arm64: Flush only the memslot after write-protect > 231abaeb7ffc2 KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range() > 5ec291b863309 KVM: arm64: Define kvm_tlb_flush_vmid_range() > 5bcd7a085c34e KVM: arm64: Implement __kvm_tlb_flush_vmid_range() > ea08f9dff7e5b arm64: tlb: Implement __flush_s2_tlb_range_op() > b3178687947c9 arm64: tlb: Refactor the core flush algorithm of __flush_tlb_range > a4850fa988eef KVM: Move kvm_arch_flush_remote_tlbs_memslot() to common code > 306dc4e6afd37 KVM: Allow range-based TLB invalidation from common code > d02785a0a1e01 KVM: Remove CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL > 136fa2d254537 KVM: arm64: Use kvm_arch_flush_remote_tlbs() > e35c68a75170d KVM: Declare kvm_arch_flush_remote_tlbs() globally > 5d592777b9bba KVM: Rename kvm_arch_flush_remote_tlb() to > kvm_arch_flush_remote_tlbs() > 06c2afb862f9d (tag: v6.5-rc1, tag: linux/v6.5-rc1) Linux 6.5-rc1 > c192ac7357683 MAINTAINERS 2: Electric Boogaloo > f71f64210d698 Merge tag 'dma-mapping-6.5-2023-07-09' of > git://git.infradead.org/users/hch/dma-mapping > ... > > Isn't the commit, 06c2afb862f9d (06c2afb862f9d (tag: v6.5-rc1, tag: > linux/v6.5-rc1) Linux 6.5-rc1) the 'base' commit? Ya, should be. Either way, even if this is PEBKAC on my end, using --base would be nice, e.g. then you can definitely say it's my fault ;-) > > Please try to incorporate git format-patch's "--base" option into your workflow, > > e.g. I do "git format-patch --base=HEAD~$nr" where $nr is the number of patches > > I am posting. > > > > It's not foolproof, e.g. my approach doesn't help if I have a local patch that > > I'm not posting, but 99% of the time it Just Works and eliminates any ambuitity. > > > > You can also do "--base=auto", but that only does the right thing if your series > > has its upstream branch set to the base/tree that you want your patches applied > > to (I use the upstream branch for a completely different purpose for my dev branches).
On Mon, Aug 14, 2023 at 5:52 PM Sean Christopherson <seanjc@google.com> wrote: > > On Mon, Aug 14, 2023, Raghavendra Rao Ananta wrote: > > On Mon, Aug 14, 2023 at 4:16 PM Sean Christopherson <seanjc@google.com> wrote: > > > > > > On Fri, Aug 11, 2023, Raghavendra Rao Ananta wrote: > > > > The series is based off of upstream v6.5-rc1. > > > > > > Lies! :-) > > > > > > This is based off one of the kvmarm.git topic branches (I didn't bother to figure > > > out which one), not v6.5-rc1. > > > > > Sorry, what am I missing here? My git log is as follows: > > Hmm, not sure what's going on. Maybe I misinterpreted why `git am` was failing? > I assumed it was because there were objects in kvmarm that I didn't have locally, > and fetching kvmarm allowed am to complete, though with 3-way merges, which IIUC > shouldn't have been necessary if I was using the exact same base. Or maybe I > messed up and didn't actually reset to 6.5-rc1. > > > $ git log --oneline upstream_tlbi_range_v9 > > 5025857507abe (upstream_tlbi_range_v9) KVM: arm64: Use TLBI > > range-based instructions for unmap > > 5c0291b99a8fc KVM: arm64: Invalidate the table entries upon a range > > 8c46b54d4aaec KVM: arm64: Flush only the memslot after write-protect > > 231abaeb7ffc2 KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range() > > 5ec291b863309 KVM: arm64: Define kvm_tlb_flush_vmid_range() > > 5bcd7a085c34e KVM: arm64: Implement __kvm_tlb_flush_vmid_range() > > ea08f9dff7e5b arm64: tlb: Implement __flush_s2_tlb_range_op() > > b3178687947c9 arm64: tlb: Refactor the core flush algorithm of __flush_tlb_range > > a4850fa988eef KVM: Move kvm_arch_flush_remote_tlbs_memslot() to common code > > 306dc4e6afd37 KVM: Allow range-based TLB invalidation from common code > > d02785a0a1e01 KVM: Remove CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL > > 136fa2d254537 KVM: arm64: Use kvm_arch_flush_remote_tlbs() > > e35c68a75170d KVM: Declare kvm_arch_flush_remote_tlbs() globally > > 5d592777b9bba KVM: Rename kvm_arch_flush_remote_tlb() to > > kvm_arch_flush_remote_tlbs() > > 06c2afb862f9d (tag: v6.5-rc1, tag: linux/v6.5-rc1) Linux 6.5-rc1 > > c192ac7357683 MAINTAINERS 2: Electric Boogaloo > > f71f64210d698 Merge tag 'dma-mapping-6.5-2023-07-09' of > > git://git.infradead.org/users/hch/dma-mapping > > ... > > > > Isn't the commit, 06c2afb862f9d (06c2afb862f9d (tag: v6.5-rc1, tag: > > linux/v6.5-rc1) Linux 6.5-rc1) the 'base' commit? > > Ya, should be. > > Either way, even if this is PEBKAC on my end, using --base would be nice, e.g. > then you can definitely say it's my fault ;-) > I'll consider this, and thanks for the confirmation. - Raghavendra > > > Please try to incorporate git format-patch's "--base" option into your workflow, > > > e.g. I do "git format-patch --base=HEAD~$nr" where $nr is the number of patches > > > I am posting. > > > > > > It's not foolproof, e.g. my approach doesn't help if I have a local patch that > > > I'm not posting, but 99% of the time it Just Works and eliminates any ambuitity. > > > > > > You can also do "--base=auto", but that only does the right thing if your series > > > has its upstream branch set to the base/tree that you want your patches applied > > > to (I use the upstream branch for a completely different purpose for my dev branches).
On Tue, Aug 15, 2023, Raghavendra Rao Ananta wrote: > On Mon, Aug 14, 2023 at 5:52 PM Sean Christopherson <seanjc@google.com> wrote: > > > > On Mon, Aug 14, 2023, Raghavendra Rao Ananta wrote: > > > On Mon, Aug 14, 2023 at 4:16 PM Sean Christopherson <seanjc@google.com> wrote: > > > > > > > > On Fri, Aug 11, 2023, Raghavendra Rao Ananta wrote: > > > > > The series is based off of upstream v6.5-rc1. > > > > > > > > Lies! :-) > > > > > > > > This is based off one of the kvmarm.git topic branches (I didn't bother to figure > > > > out which one), not v6.5-rc1. > > > > > > > Sorry, what am I missing here? My git log is as follows: > > > > Hmm, not sure what's going on. Maybe I misinterpreted why `git am` was failing? > > I assumed it was because there were objects in kvmarm that I didn't have locally, > > and fetching kvmarm allowed am to complete, though with 3-way merges, which IIUC > > shouldn't have been necessary if I was using the exact same base. Or maybe I > > messed up and didn't actually reset to 6.5-rc1. > > > > > $ git log --oneline upstream_tlbi_range_v9 > > > 5025857507abe (upstream_tlbi_range_v9) KVM: arm64: Use TLBI > > > range-based instructions for unmap > > > 5c0291b99a8fc KVM: arm64: Invalidate the table entries upon a range > > > 8c46b54d4aaec KVM: arm64: Flush only the memslot after write-protect > > > 231abaeb7ffc2 KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range() > > > 5ec291b863309 KVM: arm64: Define kvm_tlb_flush_vmid_range() > > > 5bcd7a085c34e KVM: arm64: Implement __kvm_tlb_flush_vmid_range() > > > ea08f9dff7e5b arm64: tlb: Implement __flush_s2_tlb_range_op() > > > b3178687947c9 arm64: tlb: Refactor the core flush algorithm of __flush_tlb_range > > > a4850fa988eef KVM: Move kvm_arch_flush_remote_tlbs_memslot() to common code > > > 306dc4e6afd37 KVM: Allow range-based TLB invalidation from common code > > > d02785a0a1e01 KVM: Remove CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL > > > 136fa2d254537 KVM: arm64: Use kvm_arch_flush_remote_tlbs() > > > e35c68a75170d KVM: Declare kvm_arch_flush_remote_tlbs() globally > > > 5d592777b9bba KVM: Rename kvm_arch_flush_remote_tlb() to > > > kvm_arch_flush_remote_tlbs() > > > 06c2afb862f9d (tag: v6.5-rc1, tag: linux/v6.5-rc1) Linux 6.5-rc1 > > > c192ac7357683 MAINTAINERS 2: Electric Boogaloo > > > f71f64210d698 Merge tag 'dma-mapping-6.5-2023-07-09' of > > > git://git.infradead.org/users/hch/dma-mapping > > > ... > > > > > > Isn't the commit, 06c2afb862f9d (06c2afb862f9d (tag: v6.5-rc1, tag: > > > linux/v6.5-rc1) Linux 6.5-rc1) the 'base' commit? > > > > Ya, should be. > > > > Either way, even if this is PEBKAC on my end, using --base would be nice, e.g. > > then you can definitely say it's my fault ;-) > > > I'll consider this, and thanks for the confirmation. Confirmed PEBKAC, I reset to v6.5-rc2, not rc1. My apologies, and thanks for being gentle.
On Fri, 11 Aug 2023 04:51:13 +0000, Raghavendra Rao Ananta wrote: > In certain code paths, KVM/ARM currently invalidates the entire VM's > page-tables instead of just invalidating a necessary range. For example, > when collapsing a table PTE to a block PTE, instead of iterating over > each PTE and flushing them, KVM uses 'vmalls12e1is' TLBI operation to > flush all the entries. This is inefficient since the guest would have > to refill the TLBs again, even for the addresses that aren't covered > by the table entry. The performance impact would scale poorly if many > addresses in the VM is going through this remapping. > > [...] Applied to next, thanks! [01/14] KVM: Rename kvm_arch_flush_remote_tlb() to kvm_arch_flush_remote_tlbs() commit: a1342c8027288e345cc5fd16c6800f9d4eb788ed [02/14] KVM: Declare kvm_arch_flush_remote_tlbs() globally commit: cfb0c08e80120928dda1e951718be135abd49bae [03/14] KVM: arm64: Use kvm_arch_flush_remote_tlbs() commit: 32121c813818a87ba7565b3afce93a9cc3610a22 [04/14] KVM: Remove CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL commit: eddd21481011008792f4e647a5244f6e15970abc [05/14] KVM: Allow range-based TLB invalidation from common code commit: d4788996051e3c07fadc6d9b214073fcf78810a8 [06/14] KVM: Move kvm_arch_flush_remote_tlbs_memslot() to common code commit: 619b5072443c05cf18c31b2c0320cdb42396d411 [07/14] arm64: tlb: Refactor the core flush algorithm of __flush_tlb_range commit: 360839027a6e4c022e8cbaa373dd747185f1e0a5 [08/14] arm64: tlb: Implement __flush_s2_tlb_range_op() commit: 4d73a9c13aaa78b149ac04b02f0ee7973f233bfa [09/14] KVM: arm64: Implement __kvm_tlb_flush_vmid_range() commit: 6354d15052ec88273c24beae4c99e31c3d3889b6 [10/14] KVM: arm64: Define kvm_tlb_flush_vmid_range() commit: 117940aa6e5f8308f1529e1313660980f1dae771 [11/14] KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range() commit: c42b6f0b1cde4dd19e6b5dd052e67b87cc331b01 [12/14] KVM: arm64: Flush only the memslot after write-protect commit: 3756b6f2bb3a242fef0867b39a23607f5aeca138 [13/14] KVM: arm64: Invalidate the table entries upon a range commit: defc8cc7abf0fcee8d73e440ee02827348d060e0 [14/14] KVM: arm64: Use TLBI range-based instructions for unmap commit: 7657ea920c54218f123ddc1b572821695b669c13 Cheers, M.
In certain code paths, KVM/ARM currently invalidates the entire VM's page-tables instead of just invalidating a necessary range. For example, when collapsing a table PTE to a block PTE, instead of iterating over each PTE and flushing them, KVM uses 'vmalls12e1is' TLBI operation to flush all the entries. This is inefficient since the guest would have to refill the TLBs again, even for the addresses that aren't covered by the table entry. The performance impact would scale poorly if many addresses in the VM is going through this remapping. For architectures that implement FEAT_TLBIRANGE, KVM can replace such inefficient paths by performing the invalidations only on the range of addresses that are in scope. This series tries to achieve the same in the areas of stage-2 map, unmap and write-protecting the pages. As suggested by Oliver in the original v5 of the series [1], I'm posting the series by including v2 of David Matlack's 'KVM: Add a common API for range-based TLB invalidation' series [2]. Patches 1-6 includes David M.'s patches 1, 2, 6, and 7 from [2], with minor modifications as per upstream comments. Patch-7 refactors the core arm64's __flush_tlb_range() to be used by other entities and patch-8 introduces a wrapper over it, __flush_s2_tlb_range_op(), more suited for stage-2 flushes. Patch-9,10 adds a range-based TLBI mechanism for KVM (VHE and nVHE). Patch-11 implements the kvm_arch_flush_remote_tlbs_range() for arm64. Patch-12 aims to flush only the memslot that undergoes a write-protect, instead of the entire VM. Patch-13 operates on stage2_try_break_pte() to use the range based TLBI instructions when collapsing a table entry. The map path is the immediate consumer of this when KVM remaps a table entry into a block. Patch-14 modifies the stage-2 unmap path in which, if the system supports FEAT_TLBIRANGE, the TLB invalidations are skipped during the page-table. walk. Instead it's done in one go after the entire walk is finished. The series is based off of upstream v6.5-rc1. The performance evaluation was done on a hardware that supports FEAT_TLBIRANGE, on a VHE configuration, using a modified kvm_page_table_test. The modified version updates the guest code in the ADJUST_MAPPINGS case to not only access this page but also to access up to 512 pages backwards for every new page it iterates through. This is done to test the effect of TLBI misses after KVM has handled a fault. The series captures the impact in the map and unmap paths as described above. $ kvm_page_table_test -m 2 -v 128 -s anonymous_hugetlb_2mb -b $i +--------+------------------------------+------------------------------+ | mem_sz | ADJUST_MAPPINGS (s) | Unmap VM (s) | | (GB) | Baseline | Baseline + series | Baseline | Baseline + series | +--------+----------|-------------------+------------------------------+ | 1 | 3.33 | 3.22 | 0.009 | 0.005 | | 2 | 7.39 | 7.32 | 0.012 | 0.006 | | 4 | 13.49 | 10.50 | 0.017 | 0.008 | | 8 | 21.60 | 21.50 | 0.027 | 0.011 | | 16 | 57.02 | 43.63 | 0.046 | 0.018 | | 32 | 95.92 | 83.26 | 0.087 | 0.030 | | 64 | 199.57 | 165.14 | 0.146 | 0.055 | | 128 | 423.65 | 349.37 | 0.280 | 0.100 | +--------+----------+-------------------+----------+-------------------+ $ kvm_page_table_test -m 2 -b 128G -s anonymous_hugetlb_2mb -v $i +--------+------------------------------+ | vCPUs | ADJUST_MAPPINGS (s) | | | Baseline | Baseline + series | +--------+----------|-------------------+ | 1 | 111.44 | 114.63 | | 2 | 102.88 | 74.64 | | 4 | 134.83 | 98.78 | | 8 | 98.81 | 95.01 | | 16 | 127.41 | 99.05 | | 32 | 105.35 | 91.75 | | 64 | 201.13 | 163.63 | | 128 | 423.65 | 349.37 | +--------+----------+-------------------+ For the ADJUST_MAPPINGS cases, which maps back the 4K table entries to 2M hugepages, the series sees an average improvement of ~15%. For unmapping 2M hugepages, we see a gain of 2x to 3x. $ kvm_page_table_test -m 2 -b $i +--------+------------------------------+ | mem_sz | Unmap VM (s) | | (GB) | Baseline | Baseline + series | +--------+------------------------------+ | 1 | 0.54 | 0.13 | | 2 | 1.07 | 0.25 | | 4 | 2.10 | 0.47 | | 8 | 4.19 | 0.92 | | 16 | 8.35 | 1.92 | | 32 | 16.66 | 3.61 | | 64 | 32.36 | 7.62 | | 128 | 64.65 | 14.39 | +--------+----------+-------------------+ The series sees an average gain of 4x when the guest backed by PAGE_SIZE (4K) pages. Other testing: - Booted on x86_64 and ran KVM selftests. - Build tested for MIPS and RISCV architectures with malta_kvm_defconfig and defconfig, respectively. Cc: David Matlack <dmatlack@google.com> v8: https://lore.kernel.org/all/20230808231330.3855936-1-rananta@google.com/ - Thanks for the suggestions from Gavin, Sean, and Shaoqin - Fix the build error reported by the kernel test robot in patch-11/14. - Fix typo in patch-9/14. (Gavin) - Renamed 'start_gfn' argument to 'gfn' in patch-5/14. (Gavin) v7: https://lore.kernel.org/all/20230722022251.3446223-1-rananta@google.com/ Thanks, Marc and Sean for the reviews and suggestions - Made the function declaration for kvm_arch_flush_remote_tlbs() global. (Marc, Sean) - Rename 'pages' to 'nr_pages' in TLBI functions that were introduced in the series. (Sean) - Define __flush_s2_tlb_range_op() as a wrapper over __flush_tlb_range_op(). (Marc) - Correct/improve the comments as suggested throughout the series. (Marc) - Get rid of WARN_ON() check and the 'struct stage2_unmap_data' as they were not necessary. (Marc) v6: https://lore.kernel.org/all/20230715005405.3689586-1-rananta@google.com/ Thank you, Philippe and Shaoqin for the reviews and suggestions - Split the patch-2/11 to separate the removal of CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL and arm64 to switch to using kvm_arch_flush_remote_tlbs(). (Philippe) - Align the 'pages' argument with 'kvm' in patch-3/11. (Shaoqin) - Call __tlb_switch_to_guest() before __flush_tlb_range_op() in the VHE's implementation of __kvm_tlb_flush_vmid_range(). (Shaoqin) v5 (RESEND): https://lore.kernel.org/all/20230621175002.2832640-1-rananta@google.com/ Thanks, Gavin for the suggestions: - Adjusted the comment on patch-2 to align with the code. - Fixed checkpatch.pl warning on patch-5. v5: https://lore.kernel.org/all/20230606192858.3600174-1-rananta@google.com/ Thank you, Marc and Oliver for the comments - Introduced a helper, kvm_tlb_flush_vmid_range(), to handle the decision of using range-based TLBI instructions or invalidating the entire VMID, rather than depending on __kvm_tlb_flush_vmid_range() for it. - kvm_tlb_flush_vmid_range() splits the range-based invalidations if the requested range exceeds MAX_TLBI_RANGE_PAGES. - All the users in need of invalidating the TLB upon a range now depends on kvm_tlb_flush_vmid_range() rather than directly on __kvm_tlb_flush_vmid_range(). - stage2_unmap_defer_tlb_flush() introduces a WARN_ON() to track if there's any change in TLBIRANGE or FWB support during the unmap process as the features are based on alternative patching and the TLBI operations solely depend on this check. - Corrected an incorrect hunk being present on v4's patch-3. - Updated the patches changelog and code comments as per the suggestions. v4: https://lore.kernel.org/all/20230519005231.3027912-1-rananta@google.com/ Thanks again, Oliver for all the comments - Updated the __kvm_tlb_flush_vmid_range() implementation for nVHE to adjust with the modified __tlb_switch_to_guest() that accepts a new 'bool nsh' arg. - Renamed stage2_put_pte() to stage2_unmap_put_pte() and removed the 'skip_flush' argument. - Defined stage2_unmap_defer_tlb_flush() to check if the PTE flushes can be deferred during the unmap table walk. It's being called from stage2_unmap_put_pte() and kvm_pgtable_stage2_unmap(). - Got rid of the 'struct stage2_unmap_data'. v3: https://lore.kernel.org/all/20230414172922.812640-1-rananta@google.com/ Thanks, Oliver for all the suggestions. - The core flush API (__kvm_tlb_flush_vmid_range()) now checks if the system support FEAT_TLBIRANGE or not, thus eliminating the redundancy in the upper layers. - If FEAT_TLBIRANGE is not supported, the implementation falls back to invalidating all the TLB entries with the VMID, instead of doing an iterative flush for the range. - The kvm_arch_flush_remote_tlbs_range() doesn't return -EOPNOTSUPP if the system doesn't implement FEAT_TLBIRANGE. It depends on __kvm_tlb_flush_vmid_range() to do take care of the decisions and return 0 regardless of the underlying feature support. - __kvm_tlb_flush_vmid_range() doesn't take 'level' as input to calculate the 'stride'. Instead, it always assumes PAGE_SIZE. - Fast unmap path is eliminated. Instead, the existing unmap walker is modified to skip the TLBIs during the walk, and do it all at once after the walk, using the range-based instructions. v2: https://lore.kernel.org/all/20230206172340.2639971-1-rananta@google.com/ - Rebased the series on top of David Matlack's series for common TLB invalidation API[1]. - Implement kvm_arch_flush_remote_tlbs_range() for arm64, by extending the support introduced by [1]. - Use kvm_flush_remote_tlbs_memslot() introduced by [1] to flush only the current memslot after write-protect. - Modified the __kvm_tlb_flush_range() macro to accepts 'level' as an argument to calculate the 'stride' instead of just using PAGE_SIZE. - Split the patch that introduces the range-based TLBI to KVM and the implementation of IPA-based invalidation into its own patches. - Dropped the patch that tries to optimize the mmu notifiers paths. - Rename the function kvm_table_pte_flush() to kvm_pgtable_stage2_flush_range(), and accept the range of addresses to flush. [Oliver] - Drop the 'tlb_level' argument for stage2_try_break_pte() and directly pass '0' as 'tlb_level' to kvm_pgtable_stage2_flush_range(). [Oliver] v1: https://lore.kernel.org/all/20230109215347.3119271-1-rananta@google.com/ Thank you. Raghavendra David Matlack (3): KVM: Rename kvm_arch_flush_remote_tlb() to kvm_arch_flush_remote_tlbs() KVM: Allow range-based TLB invalidation from common code KVM: Move kvm_arch_flush_remote_tlbs_memslot() to common code Raghavendra Rao Ananta (11): KVM: Declare kvm_arch_flush_remote_tlbs() globally KVM: arm64: Use kvm_arch_flush_remote_tlbs() KVM: Remove CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL arm64: tlb: Refactor the core flush algorithm of __flush_tlb_range arm64: tlb: Implement __flush_s2_tlb_range_op() KVM: arm64: Implement __kvm_tlb_flush_vmid_range() KVM: arm64: Define kvm_tlb_flush_vmid_range() KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range() KVM: arm64: Flush only the memslot after write-protect KVM: arm64: Invalidate the table entries upon a range KVM: arm64: Use TLBI range-based instructions for unmap arch/arm64/include/asm/kvm_asm.h | 3 + arch/arm64/include/asm/kvm_host.h | 4 + arch/arm64/include/asm/kvm_pgtable.h | 10 +++ arch/arm64/include/asm/tlbflush.h | 124 +++++++++++++++------------ arch/arm64/kvm/Kconfig | 1 - arch/arm64/kvm/arm.c | 6 -- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 +++ arch/arm64/kvm/hyp/nvhe/tlb.c | 30 +++++++ arch/arm64/kvm/hyp/pgtable.c | 63 ++++++++++++-- arch/arm64/kvm/hyp/vhe/tlb.c | 28 ++++++ arch/arm64/kvm/mmu.c | 16 +++- arch/mips/include/asm/kvm_host.h | 3 +- arch/mips/kvm/mips.c | 12 +-- arch/riscv/kvm/mmu.c | 6 -- arch/x86/include/asm/kvm_host.h | 6 +- arch/x86/kvm/mmu/mmu.c | 26 ++---- arch/x86/kvm/mmu/mmu_internal.h | 3 - arch/x86/kvm/x86.c | 2 +- include/linux/kvm_host.h | 24 ++++-- virt/kvm/Kconfig | 3 - virt/kvm/kvm_main.c | 35 ++++++-- 21 files changed, 286 insertions(+), 130 deletions(-)