KVM: x86: Fix rmap allocation for very large memslots

Commit Message

Maciej S. Szmigiero Jan. 28, 2022, 9:36 p.m. UTC

From: "Maciej S. Szmigiero" <[e-mail]>

Commit 7661809d493b ("mm: don't allow oversized kvmalloc() calls") has
forbidden using kvmalloc() to make allocations larger than INT_MAX (2 GiB).

Unfortunately, adding a memslot exceeding 1 TiB in size will result in rmap
code trying to make an allocation exceeding this limit.
Besides failing this allocation, such operation will also trigger a
WARN_ON_ONCE() added by the aforementioned commit.

Since we probably still want to use kernel slab for small rmap allocations
let's only redirect such oversized allocations to vmalloc.

A possible alternative would be to add some kind of a __GFP_LARGE flag to
skip the INT_MAX check behind kvmalloc(), however this will impact the
common kernel memory allocation code, not just KVM.

Fixes: a7c3e901a4 ("mm: introduce kv[mz]alloc helpers")
Cc: [e-mail]
Signed-off-by: Maciej S. Szmigiero <[e-mail]>
---
 arch/x86/kvm/x86.c | 26 +++++++++++++++++++-------
 1 file changed, 19 insertions(+), 7 deletions(-)

Comments

Sean Christopherson Jan. 28, 2022, 9:47 p.m. UTC | #1

On Fri, Jan 28, 2022, Maciej S. Szmigiero wrote:
> From: "Maciej S. Szmigiero" <[e-mail]>
> 
> Commit 7661809d493b ("mm: don't allow oversized kvmalloc() calls") has
> forbidden using kvmalloc() to make allocations larger than INT_MAX (2 GiB).
> 
> Unfortunately, adding a memslot exceeding 1 TiB in size will result in rmap
> code trying to make an allocation exceeding this limit.
> Besides failing this allocation, such operation will also trigger a
> WARN_ON_ONCE() added by the aforementioned commit.
> 
> Since we probably still want to use kernel slab for small rmap allocations
> let's only redirect such oversized allocations to vmalloc.
> 
> A possible alternative would be to add some kind of a __GFP_LARGE flag to
> skip the INT_MAX check behind kvmalloc(), however this will impact the
> common kernel memory allocation code, not just KVM.

Paolo has a cleaner fix for this[1][2], but it appears to have stalled out somewhere.

Paolo???

[1] https://lore.kernel.org/all/[e-mail]
[2] https://lore.kernel.org/all/[e-mail]

Maciej S. Szmigiero Feb. 10, 2022, 9 p.m. UTC | #2

On 28.01.2022 22:47, Sean Christopherson wrote:
> On Fri, Jan 28, 2022, Maciej S. Szmigiero wrote:
>> From: "Maciej S. Szmigiero" <[e-mail]>
>>
>> Commit 7661809d493b ("mm: don't allow oversized kvmalloc() calls") has
>> forbidden using kvmalloc() to make allocations larger than INT_MAX (2 GiB).
>>
>> Unfortunately, adding a memslot exceeding 1 TiB in size will result in rmap
>> code trying to make an allocation exceeding this limit.
>> Besides failing this allocation, such operation will also trigger a
>> WARN_ON_ONCE() added by the aforementioned commit.
>>
>> Since we probably still want to use kernel slab for small rmap allocations
>> let's only redirect such oversized allocations to vmalloc.
>>
>> A possible alternative would be to add some kind of a __GFP_LARGE flag to
>> skip the INT_MAX check behind kvmalloc(), however this will impact the
>> common kernel memory allocation code, not just KVM.
> 
> Paolo has a cleaner fix for this[1][2], but it appears to have stalled out somewhere.
> 
> Paolo???
> 
> [1] https://lore.kernel.org/all/[e-mail]
> [2] https://lore.kernel.org/all/[e-mail]

So, what we do here?

Apparently the cleaner fix at [2] wasn't picked up despite Kees giving
it his "Reviewed-by".

Thanks,
Maciej

Patch

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 8033eca6f3a1..c64bac8614c7 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -11806,24 +11806,36 @@  void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot)
 
 int memslot_rmap_alloc(struct kvm_memory_slot *slot, unsigned long npages)
 {
-	const int sz = sizeof(*slot->arch.rmap[0]);
+	const size_t sz = sizeof(*slot->arch.rmap[0]);
 	int i;
 
 	for (i = 0; i < KVM_NR_PAGE_SIZES; ++i) {
 		int level = i + 1;
-		int lpages = __kvm_mmu_slot_lpages(slot, npages, level);
+		size_t lpages = __kvm_mmu_slot_lpages(slot, npages, level);
+		size_t rmap_size;
 
 		if (slot->arch.rmap[i])
 			continue;
 
-		slot->arch.rmap[i] = kvcalloc(lpages, sz, GFP_KERNEL_ACCOUNT);
-		if (!slot->arch.rmap[i]) {
-			memslot_rmap_free(slot);
-			return -ENOMEM;
-		}
+		if (unlikely(check_mul_overflow(lpages, sz, &rmap_size)))
+			goto ret_fail;
+
+		/* kvzalloc() only allows sizes up to INT_MAX */
+		if (unlikely(rmap_size > INT_MAX))
+			slot->arch.rmap[i] = __vmalloc(rmap_size,
+						       GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+		else
+			slot->arch.rmap[i] = kvzalloc(rmap_size, GFP_KERNEL_ACCOUNT);
+
+		if (!slot->arch.rmap[i])
+			goto ret_fail;
 	}
 
 	return 0;
+
+ret_fail:
+	memslot_rmap_free(slot);
+	return -ENOMEM;
 }
 
 static int kvm_alloc_memslot_metadata(struct kvm *kvm,