[RFC,v5,07/38] KVM: arm64: Unmap unlocked memslot from stage 2 if kvm_mmu_has_pending_ops()

Commit Message

Alexandru Elisei Nov. 17, 2021, 3:38 p.m. UTC

KVM relies on doing the necessary maintenance operations for locked
memslots when the first VCPU is run. If the memslot has been locked, and
then unlocked before the first VCPU is run, the maintenance operations
won't be performed for the region described by the memslot, but the memory
remains mapped at stage 2. Which means that it is possible for a guest
running with the MMU off to read stale value from memory instead of the
newest values written by the host (and not written back to memory).

In this case, unmap the memslot from stage 2 to trigger stage 2 data
aborts, which will take care of any synchronisation requirements.

Signed-off-by: Alexandru Elisei <[e-mail]>
---
 Documentation/virt/kvm/api.rst |  7 +++++--
 arch/arm64/kvm/mmu.c           | 20 ++++++++++++++++++++
 2 files changed, 25 insertions(+), 2 deletions(-)

Patch

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 0ac12a730013..5a69b3b543c0 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6979,8 +6979,11 @@  write permissions are specified for a memslot which logs dirty pages.
 
 Enabling this capability causes the memory pinned when locking the memslot
 specified in args[0] to be unpinned, or, optionally, all memslots to be
-unlocked. The IPA range is not unmapped from stage 2. It is considered an error
-to attempt to unlock a memslot which is not locked.
+unlocked. If between the user memory region being locked and the same region
+being unlocked no VCPU has run, then unlocking the memory region also causes the
+corresponding IPA range to be unmapped from stage 2; otherwise, stage 2 is left
+unchanged. It is considered an error to attempt to unlock a memslot which is not
+locked.
 
 8. Other capabilities.
 ======================
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 2491e73e3d31..cd6f1bc7842d 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1632,6 +1632,14 @@  static void unlock_memslot(struct kvm *kvm, struct kvm_memory_slot *memslot)
 	bool writable = memslot->arch.flags & KVM_MEMSLOT_LOCK_WRITE;
 	unsigned long npages = memslot->npages;
 
+	/*
+	 * MMU maintenace operations aren't performed on an unlocked memslot.
+	 * Unmap it from stage 2 so the abort handler performs the necessary
+	 * operations.
+	 */
+	if (kvm_mmu_has_pending_ops(kvm))
+		kvm_arch_flush_shadow_memslot(kvm, memslot);
+
 	unpin_memslot_pages(memslot, writable);
 	account_locked_vm(current->mm, npages, false);
 
@@ -1642,6 +1650,7 @@  int kvm_mmu_unlock_memslot(struct kvm *kvm, u64 slot, u64 flags)
 {
 	bool unlock_all = flags & KVM_ARM_UNLOCK_MEM_ALL;
 	struct kvm_memory_slot *memslot;
+	bool has_locked_memslot;
 	int ret = 0;
 
 	if (!unlock_all && slot >= KVM_MEM_SLOTS_NUM)
@@ -1664,6 +1673,17 @@  int kvm_mmu_unlock_memslot(struct kvm *kvm, u64 slot, u64 flags)
 		unlock_memslot(kvm, memslot);
 	}
 
+	if (kvm_mmu_has_pending_ops(kvm)) {
+		has_locked_memslot = false;
+		kvm_for_each_memslot(memslot, kvm_memslots(kvm)) {
+			if (memslot_is_locked(memslot)) {
+				has_locked_memslot = true;
+			}
+		}
+		if (!has_locked_memslot)
+			kvm->arch.mmu_pending_ops = 0;
+	}
+
 out_unlock_slots:
 	mutex_unlock(&kvm->slots_lock);
 	return ret;