From patchwork Thu Jun 13 06:09:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 13696149 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9871637C; Thu, 13 Jun 2024 06:10:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259053; cv=none; b=lcVRtHrlceIWbEhQS8bdG41MOa1TNhbbzWM3PclEujPg9BrF/KdetCaGVuNRruJSXiSPpm1+BUHmJNVUKQZm4aXotW9Rcp6E2gjPMpA8G6W+ERChHUcvkAO7ZKu7NpxFXTQvBiqphRrGmEjUxd8PZrzeOnf2NUQaGHhh6vEneLs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259053; c=relaxed/simple; bh=GJW+aRbd3tyu56kqkS3E3JlIhZYvNb/UXv+q2wbBeig=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LurtYKSX6NxQknPSlBP/l8FuRJkAtTuQoUwHnioMbpqXyLQ0bjG9dneAPCCdMwzNDrXdhXPm9CeLrVucHYBsQ4GopEel0dH13Iwx9fXl37dZsalMrFA36VZYzocuX5eCS3ZGWhQNjwoRfitty5hdGfbFOgvzKG3mlRXYtK38JKs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jjkYfr3l; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jjkYfr3l" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1718259052; x=1749795052; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GJW+aRbd3tyu56kqkS3E3JlIhZYvNb/UXv+q2wbBeig=; b=jjkYfr3l8bex0Z3RutkC++l3gTJYwETsYSdrJ5naOqwfQwy1EuHWfjup IYQdvhdZQz3fr1BI6SmRhSe+VGj09AoaE1k91HAItQrwuQRxjsyT6/rnl Tw1i7F4jirRGCGUt6B+UqemA3sVlD61N44yhZqvMnGuMUhwef2KdMz+xi QzFLPJJxszmHLbNU/3jPL1s8suvY9qoFTE/1EOdKbgXFpgkn4mXSosfOE K9LlBIxQjjSkI0yfzd085KBvw6ng30zY+SMUmk41wBOHfdpImaaoMLwD1 1nHGkYvlM17Dl2GN4BBdYyl1prcM6qjkmEnoHzraxWrRpOe9HJiRnTlEh g==; X-CSE-ConnectionGUID: AO3LSsvuTvW21oYKgIECcA== X-CSE-MsgGUID: yOxrJQI7Q+OM5SkVUONUJg== X-IronPort-AV: E=McAfee;i="6700,10204,11101"; a="32598783" X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="32598783" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:10:51 -0700 X-CSE-ConnectionGUID: u17XlQ0tTfCNQ7FVqmgU9Q== X-CSE-MsgGUID: gRx5Dun+R2eTtjQ2cLBlew== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="44476513" Received: from yzhao56-desk.sh.intel.com ([10.239.159.62]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:10:48 -0700 From: Yan Zhao To: pbonzini@redhat.com, seanjc@google.com Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, isaku.yamahata@intel.com, dmatlack@google.com, sagis@google.com, erdemaktas@google.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Yan Zhao Subject: [PATCH 1/5] KVM: x86/mmu: Introduce a quirk to control memslot zap behavior Date: Thu, 13 Jun 2024 14:09:46 +0800 Message-ID: <20240613060946.11806-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240613060708.11761-1-yan.y.zhao@intel.com> References: <20240613060708.11761-1-yan.y.zhao@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce a quirk KVM_X86_QUIRK_SLOT_ZAP_ALL to allow users to choose between enabling the quirk to invalidate/zap all SPTEs when a memslot is moved/deleted and disabling the quirk to zap precisely/slowly of only leaf SPTEs that are within the range of moving/deleting memslot. The quirk is turned on by default. This is to work around a bug [1] where the changing the zapping behavior of memslot move/deletion would cause VM instability for VMs with an Nvidia GPU assigned. Users have the option to deactivate the quirk for specific VMs that are unaffected by this bug. Turning off the quirk enables a more precise zapping of SPTEs within the memory slot range, enhancing performance for certain scenarios [2] and meeting the functional requirements for TDX. In TDX, it is crucial that the root page of the private page table remains unchanged, with leaf entries being zapped before non-leaf entries. Additionally, any pages dropped in TDX would necessitate their re-acceptance by the guest. Previously, an attempt was made to introduce a per-VM capability [3] as a workaround for the bug. However, this approach was not preferred because the root cause of the bug remained unidentified. An alternative solution involving a per-memslot flag [4] was also considered but ultimately rejected. Sean and Paolo thereafter recommended the implementation of this quirk and explained that it's the least bad option [5]. For the quirk disabled case, add a new function kvm_mmu_zap_memslot_leafs() to zap leaf SPTEs within a memslot when moving/deleting a memslot. Rather than further calling kvm_unmap_gfn_range() for the actual zapping, this function bypasses the special handling to APIC_ACCESS_PAGE_PRIVATE_MEMSLOT. This is based on the considerations that 1) The APIC_ACCESS_PAGE_PRIVATE_MEMSLOT cannot be created by users, nor can it be moved. It is only deleted by KVM when APICv is permanently inhibited. 2) kvm_vcpu_reload_apic_access_page() effectively does nothing when APIC_ACCESS_PAGE_PRIVATE_MEMSLOT is deleted. 3) Avoid making all cpus request of KVM_REQ_APIC_PAGE_RELOAD can save on costly IPIs. Suggested-by: Kai Huang Suggested-by: Sean Christopherson Suggested-by: Paolo Bonzini Cc: Rick Edgecombe Link: https://patchwork.kernel.org/project/kvm/patch/20190205210137.1377-11-sean.j.christopherson@intel.com [1] Link: https://patchwork.kernel.org/project/kvm/patch/20190205210137.1377-11-sean.j.christopherson@intel.com/#25054908 [2] Link: https://lore.kernel.org/kvm/20200713190649.GE29725@linux.intel.com/T/#mabc0119583dacf621025e9d873c85f4fbaa66d5c [3] Link: https://lore.kernel.org/all/20240515005952.3410568-3-rick.p.edgecombe@intel.com [4] Link: https://lore.kernel.org/all/7df9032d-83e4-46a1-ab29-6c7973a2ab0b@redhat.com [5] Signed-off-by: Yan Zhao --- Documentation/virt/kvm/api.rst | 6 ++++++ arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/mmu/mmu.c | 36 ++++++++++++++++++++++++++++++++- 4 files changed, 44 insertions(+), 2 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index ebdf88078515..37b5ecb25778 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -8146,6 +8146,12 @@ KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS By default, KVM emulates MONITOR/MWAIT (if guest CPUID on writes to MISC_ENABLE if KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT is disabled. + +KVM_X86_QUIRK_SLOT_ZAP_ALL By default, KVM invalidates all SPTEs in + fast way when a memslot is deleted. + When this quirk is disabled, KVM zaps only + leaf SPTEs that are within the range of the + memslot being deleted. =================================== ============================================ 7.32 KVM_CAP_MAX_VCPU_ID diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index c9499e3b5915..8152b5259435 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2383,7 +2383,8 @@ int memslot_rmap_alloc(struct kvm_memory_slot *slot, unsigned long npages); KVM_X86_QUIRK_OUT_7E_INC_RIP | \ KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT | \ KVM_X86_QUIRK_FIX_HYPERCALL_INSN | \ - KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS) + KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS | \ + KVM_X86_QUIRK_SLOT_ZAP_ALL) /* * KVM previously used a u32 field in kvm_run to indicate the hypercall was diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index f64421c55266..c5d189f9ca34 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -438,6 +438,7 @@ struct kvm_sync_regs { #define KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT (1 << 4) #define KVM_X86_QUIRK_FIX_HYPERCALL_INSN (1 << 5) #define KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS (1 << 6) +#define KVM_X86_QUIRK_SLOT_ZAP_ALL (1 << 7) #define KVM_STATE_NESTED_FORMAT_VMX 0 #define KVM_STATE_NESTED_FORMAT_SVM 1 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1fd2f8ea6fab..6269fa315888 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -6987,10 +6987,44 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm) kvm_mmu_zap_all(kvm); } +/* + * Zapping leaf SPTEs with memslot range when a memslot is moved/deleted. + * + * Zapping non-leaf SPTEs, a.k.a. not-last SPTEs, isn't required, worst + * case scenario we'll have unused shadow pages lying around until they + * are recycled due to age or when the VM is destroyed. + */ +static void kvm_mmu_zap_memslot_leafs(struct kvm *kvm, struct kvm_memory_slot *slot) +{ + struct kvm_gfn_range range = { + .slot = slot, + .start = slot->base_gfn, + .end = slot->base_gfn + slot->npages, + .may_block = true, + }; + bool flush = false; + + write_lock(&kvm->mmu_lock); + + if (kvm_memslots_have_rmaps(kvm)) + flush = kvm_handle_gfn_range(kvm, &range, kvm_zap_rmap); + + if (tdp_mmu_enabled) + flush = kvm_tdp_mmu_unmap_gfn_range(kvm, &range, flush); + + if (flush) + kvm_flush_remote_tlbs_memslot(kvm, slot); + + write_unlock(&kvm->mmu_lock); +} + void kvm_arch_flush_shadow_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) { - kvm_mmu_zap_all_fast(kvm); + if (kvm_check_has_quirk(kvm, KVM_X86_QUIRK_SLOT_ZAP_ALL)) + kvm_mmu_zap_all_fast(kvm); + else + kvm_mmu_zap_memslot_leafs(kvm, slot); } void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen) From patchwork Thu Jun 13 06:10:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 13696171 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4624E1369A0; Thu, 13 Jun 2024 06:11:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259089; cv=none; b=lwTvBECMxGsCGk5pZPo7U46gXtt2CS+YwUfUVnF3W9FMVWcbo5PU8ovnQQJ48NRdQxjQk+T5T1o1ZcK1XVam2E4R+XYnHxHdv/uAwiVxVqDPTHCq5voP6iFG0HR3Yzn9wLkOEYtu3xMB0fZO7f+FR3d0yEUXNlHutOreSXqQ6uE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259089; c=relaxed/simple; bh=xELkib58GVsHFnJOyikK4lxSSfPcXfJatrw+PYgf+/A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=a5CFq3vttZcKoEBxIlfGvQms78EbOZmgwwK0s6AZf42g5zU6yf8OKxPySpqEbqsJldDzcfkCVLB+bfyGc2tC4HfZguV0W5WRUpPGEjBKfY9rNt0unkeVEq7vJDIbFKSqjWVMLezhaCa/CguTSB01G2k4MIKz7kidtrT/Ji74Z6Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=TLAMOb0d; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="TLAMOb0d" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1718259088; x=1749795088; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xELkib58GVsHFnJOyikK4lxSSfPcXfJatrw+PYgf+/A=; b=TLAMOb0d5LZPcHq/6sCKmMVyx1dh3zxh9bwYT7QsItMfR0MUSO/hy8F+ yyU8wDfZ5kyzTBbkfXHAsQDQn0HDaJ9VyrMBftjhhYGZC/ccCjAeG/JN3 aZUIWjGGOk/YchcffqW/wCbzIH7bjTv/9gXhl7nPHPigHj60kumEUIE+l Rq1wxUiAvUbPVNpSq0B07bBj5doTEk77E4VIrfXxMErthN5UKdGNdWlst JuMhmH7Ygg6nOtCZgdA61YeB+W/uyfytErozOv0vBUvnD56NPJcSry2YD mAhoVzOwPLsEf8Qx6kud+nnTRBIqViimMJ1FyiwIoQ1J6sAlVWd/BRDyR g==; X-CSE-ConnectionGUID: P/vIYiOGTiqjQWIrhblyPw== X-CSE-MsgGUID: gLnke7yRRo26umhDdbWvQQ== X-IronPort-AV: E=McAfee;i="6700,10204,11101"; a="14894744" X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="14894744" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:11:26 -0700 X-CSE-ConnectionGUID: prGISX2bT62Qm9w6RNU/0g== X-CSE-MsgGUID: vfjsAz2WTkyzrbiPqUcNqA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="44417196" Received: from yzhao56-desk.sh.intel.com ([10.239.159.62]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:11:23 -0700 From: Yan Zhao To: pbonzini@redhat.com, seanjc@google.com Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, isaku.yamahata@intel.com, dmatlack@google.com, sagis@google.com, erdemaktas@google.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Yan Zhao Subject: [PATCH 2/5] KVM: selftests: Test slot move/delete with slot zap quirk enabled/disabled Date: Thu, 13 Jun 2024 14:10:24 +0800 Message-ID: <20240613061024.11831-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240613060708.11761-1-yan.y.zhao@intel.com> References: <20240613060708.11761-1-yan.y.zhao@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Update set_memory_region_test to make sure memslot move and deletion function correctly both when slot zap quirk KVM_X86_QUIRK_SLOT_ZAP_ALL is enabled and disabled. Signed-off-by: Yan Zhao --- .../selftests/kvm/set_memory_region_test.c | 29 ++++++++++++++----- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/tools/testing/selftests/kvm/set_memory_region_test.c b/tools/testing/selftests/kvm/set_memory_region_test.c index bb8002084f52..a8267628e9ed 100644 --- a/tools/testing/selftests/kvm/set_memory_region_test.c +++ b/tools/testing/selftests/kvm/set_memory_region_test.c @@ -175,7 +175,7 @@ static void guest_code_move_memory_region(void) GUEST_DONE(); } -static void test_move_memory_region(void) +static void test_move_memory_region(bool disable_slot_zap_quirk) { pthread_t vcpu_thread; struct kvm_vcpu *vcpu; @@ -184,6 +184,9 @@ static void test_move_memory_region(void) vm = spawn_vm(&vcpu, &vcpu_thread, guest_code_move_memory_region); + if (disable_slot_zap_quirk) + vm_enable_cap(vm, KVM_CAP_DISABLE_QUIRKS2, KVM_X86_QUIRK_SLOT_ZAP_ALL); + hva = addr_gpa2hva(vm, MEM_REGION_GPA); /* @@ -266,7 +269,7 @@ static void guest_code_delete_memory_region(void) GUEST_ASSERT(0); } -static void test_delete_memory_region(void) +static void test_delete_memory_region(bool disable_slot_zap_quirk) { pthread_t vcpu_thread; struct kvm_vcpu *vcpu; @@ -276,6 +279,9 @@ static void test_delete_memory_region(void) vm = spawn_vm(&vcpu, &vcpu_thread, guest_code_delete_memory_region); + if (disable_slot_zap_quirk) + vm_enable_cap(vm, KVM_CAP_DISABLE_QUIRKS2, KVM_X86_QUIRK_SLOT_ZAP_ALL); + /* Delete the memory region, the guest should not die. */ vm_mem_region_delete(vm, MEM_REGION_SLOT); wait_for_vcpu(); @@ -553,7 +559,10 @@ int main(int argc, char *argv[]) { #ifdef __x86_64__ int i, loops; + int j, disable_slot_zap_quirk = 0; + if (kvm_check_cap(KVM_CAP_DISABLE_QUIRKS2) & KVM_X86_QUIRK_SLOT_ZAP_ALL) + disable_slot_zap_quirk = 1; /* * FIXME: the zero-memslot test fails on aarch64 and s390x because * KVM_RUN fails with ENOEXEC or EFAULT. @@ -579,13 +588,17 @@ int main(int argc, char *argv[]) else loops = 10; - pr_info("Testing MOVE of in-use region, %d loops\n", loops); - for (i = 0; i < loops; i++) - test_move_memory_region(); + for (j = 0; j <= disable_slot_zap_quirk; j++) { + pr_info("Testing MOVE of in-use region, %d loops, slot zap quirk %s\n", + loops, j ? "disabled" : "enabled"); + for (i = 0; i < loops; i++) + test_move_memory_region(!!j); - pr_info("Testing DELETE of in-use region, %d loops\n", loops); - for (i = 0; i < loops; i++) - test_delete_memory_region(); + pr_info("Testing DELETE of in-use region, %d loops, slot zap quirk %s\n", + loops, j ? "disabled" : "enabled"); + for (i = 0; i < loops; i++) + test_delete_memory_region(!!j); + } #endif return 0; From patchwork Thu Jun 13 06:10:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 13696172 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 035DC136E16; Thu, 13 Jun 2024 06:11:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259099; cv=none; b=hJxFw1j91EQwMMESdBXXNplWxfw2gr1moIBBNYYjRKeqA707NHyEHlE72rdykn/1924qi+7xE025GOCcpUUWThL78bsGZNddY8/Xcvf5HHvz1RbK29di9PXgroYcLWqHJb8g6sXF12+5ZCJsjvDPA/gpoOwAfZ1tgCOd87+58Vs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259099; c=relaxed/simple; bh=w/1rwRGbSd+x48cBrawS9ygwKtHv3Yd42T/+V4bBG5w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=akC6eGnC3CZaZlz6aLeqnTcMvxpJ4Lm8zNW3yrmWLTC04ornBB3ft/6EskBuIh9IWXLMHYfCKAdDk74VQz3BzOpbGYsZTsUAOnosXEfCiPmZkPhpwylblN3Hom5Itx6CF/1Q0GvBvFWPmQej8E5p5xSm0o86i9VmeTXePSkFdqI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=iEuQatd6; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iEuQatd6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1718259098; x=1749795098; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=w/1rwRGbSd+x48cBrawS9ygwKtHv3Yd42T/+V4bBG5w=; b=iEuQatd6rXdvzfKeRN9mWG9eF+xH5LsDXb07l5bPKszpG/Tx7rywlonT GGrWu7aZYRR3XzFBJ5e0rex0qnmB7MxQsTblkJawM1SKbPO9yS4D8qoFU c7zkT5RqURjH2HKzr6Lu+EnVU/Ift5dbKD+uI6SxyhPhnWQPTMs6ib6X0 A6u7w5vaMj4y6hGzW1aeV/npEeCUtWnDT16vwjFDuljLvUTe8CUc6RiBz Fkq0S8cev9ZD3Pcl8wMo/KkYGNU9R/uUdbfk0S5lYZTeC1MR/q2COexwa UuX4o+Egn7s4wtrN++jB9nSyhwsgDORE10l154jhe8jVUNhjHnsvo2PpX Q==; X-CSE-ConnectionGUID: 4xklyVFRTP2ZgflEnbZ1ow== X-CSE-MsgGUID: kdYSSuN4Q+CTJrMc2LWtYA== X-IronPort-AV: E=McAfee;i="6700,10204,11101"; a="14894761" X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="14894761" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:11:38 -0700 X-CSE-ConnectionGUID: kWon/JDkRTCTdkJOIhv5dg== X-CSE-MsgGUID: pGJo4d8WSECnjSFs1jk07w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="44417238" Received: from yzhao56-desk.sh.intel.com ([10.239.159.62]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:11:35 -0700 From: Yan Zhao To: pbonzini@redhat.com, seanjc@google.com Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, isaku.yamahata@intel.com, dmatlack@google.com, sagis@google.com, erdemaktas@google.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Yan Zhao Subject: [PATCH 3/5] KVM: selftests: Allow slot modification stress test with quirk disabled Date: Thu, 13 Jun 2024 14:10:37 +0800 Message-ID: <20240613061037.11847-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240613060708.11761-1-yan.y.zhao@intel.com> References: <20240613060708.11761-1-yan.y.zhao@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a new user option to memslot_modification_stress_test to allow testing with slot zap quirk KVM_X86_QUIRK_SLOT_ZAP_ALL disabled. Signed-off-by: Yan Zhao --- .../kvm/memslot_modification_stress_test.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/memslot_modification_stress_test.c b/tools/testing/selftests/kvm/memslot_modification_stress_test.c index 05fcf902e067..c6f22ded4c96 100644 --- a/tools/testing/selftests/kvm/memslot_modification_stress_test.c +++ b/tools/testing/selftests/kvm/memslot_modification_stress_test.c @@ -85,6 +85,7 @@ struct test_params { useconds_t delay; uint64_t nr_iterations; bool partition_vcpu_memory_access; + bool disable_slot_zap_quirk; }; static void run_test(enum vm_guest_mode mode, void *arg) @@ -95,6 +96,13 @@ static void run_test(enum vm_guest_mode mode, void *arg) vm = memstress_create_vm(mode, nr_vcpus, guest_percpu_mem_size, 1, VM_MEM_SRC_ANONYMOUS, p->partition_vcpu_memory_access); +#ifdef __x86_64__ + if (p->disable_slot_zap_quirk) + vm_enable_cap(vm, KVM_CAP_DISABLE_QUIRKS2, KVM_X86_QUIRK_SLOT_ZAP_ALL); + + pr_info("Memslot zap quirk %s\n", p->disable_slot_zap_quirk ? + "disabled" : "enabled"); +#endif pr_info("Finished creating vCPUs\n"); @@ -113,11 +121,12 @@ static void run_test(enum vm_guest_mode mode, void *arg) static void help(char *name) { puts(""); - printf("usage: %s [-h] [-m mode] [-d delay_usec]\n" + printf("usage: %s [-h] [-m mode] [-d delay_usec] [-q]\n" " [-b memory] [-v vcpus] [-o] [-i iterations]\n", name); guest_modes_help(); printf(" -d: add a delay between each iteration of adding and\n" " deleting a memslot in usec.\n"); + printf(" -q: Disable memslot zap quirk.\n"); printf(" -b: specify the size of the memory region which should be\n" " accessed by each vCPU. e.g. 10M or 3G.\n" " Default: 1G\n"); @@ -143,7 +152,7 @@ int main(int argc, char *argv[]) guest_modes_append_default(); - while ((opt = getopt(argc, argv, "hm:d:b:v:oi:")) != -1) { + while ((opt = getopt(argc, argv, "hm:d:qb:v:oi:")) != -1) { switch (opt) { case 'm': guest_modes_cmdline(optarg); @@ -166,6 +175,12 @@ int main(int argc, char *argv[]) case 'i': p.nr_iterations = atoi_positive("Number of iterations", optarg); break; + case 'q': + p.disable_slot_zap_quirk = true; + + TEST_REQUIRE(kvm_check_cap(KVM_CAP_DISABLE_QUIRKS2) & + KVM_X86_QUIRK_SLOT_ZAP_ALL); + break; case 'h': default: help(argv[0]); From patchwork Thu Jun 13 06:10:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 13696173 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06648136E2A; Thu, 13 Jun 2024 06:11:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259112; cv=none; b=d2RTaAPF9gFm/+Ej7BDZ/OLGyVobFxPl6NrkDPX5NCV5yTttCSB4UVplHt/uBGCuiRiZJJf3qcp8N28LoOeymydPy4xspcO1qf20ORyYWTAvQi1695AdBqGLlfuW0tON/QtmgulcIHZtJCI5GO9WyZCwYHuYkP5s3wAgoE8y4Sg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259112; c=relaxed/simple; bh=aI65XLqfvR7yKM7kV8fh7r8wiyY/a5ItqUizG+26xA4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=M4gJXqzA2ERZAMRcOnL3qknYbMPA19NQ+mMPo+k5ItIkgwiWWuhZKqLOhEwg8VHIb0kKFYcS8b8dubSYtLhlnQb7pipore0Ue2amiwlqK/nXfwQrM0rK7N4P43gKxIQCiGm8o9ynofn37w9oYRzZJi0MN+Qptxi5+qZcTnigdRY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DDlgTTuq; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DDlgTTuq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1718259111; x=1749795111; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aI65XLqfvR7yKM7kV8fh7r8wiyY/a5ItqUizG+26xA4=; b=DDlgTTuqqZFkosuanbg8GHKFR2fLzb6LS/evNbeN3akC0LiS4pulubzn dz1BhNwGBR7qztwyuNkr1oEbdnwPanyZuMPc0HH5nasN+PT/0/IFt1zY1 mOknu4ZGqRzCggDvVQrscIJQ4Evdj6zaSGST4ELkZAupPWidYLgfdWLCw bD2qt4jPeflRtDigv4VcRd6j5CDKiWKvwqhsctKgc4KaK3tMdSKt+d6er Nom+pj5HDtfznCDoqdAEfkZuoVUTxVFZ81ynVhZcK3JHC+ujwltexewFW L4N5uvXlkzosO8vmDazaMaR2tmTX+WZjuJ9krFasjEtirBWPd8Z+9GyNZ Q==; X-CSE-ConnectionGUID: xTAjIchzRyek/wqg+SdSfA== X-CSE-MsgGUID: RFZB+KN+TXqfWIvrLMJ1Qg== X-IronPort-AV: E=McAfee;i="6700,10204,11101"; a="25734247" X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="25734247" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:11:50 -0700 X-CSE-ConnectionGUID: mrFLflcqQZCc0+W/0STbVg== X-CSE-MsgGUID: 6xu47AzdQh+cjWAL95oB/w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="40669150" Received: from yzhao56-desk.sh.intel.com ([10.239.159.62]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:11:47 -0700 From: Yan Zhao To: pbonzini@redhat.com, seanjc@google.com Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, isaku.yamahata@intel.com, dmatlack@google.com, sagis@google.com, erdemaktas@google.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Yan Zhao Subject: [PATCH 4/5] KVM: selftests: Test memslot move in memslot_perf_test with quirk disabled Date: Thu, 13 Jun 2024 14:10:49 +0800 Message-ID: <20240613061049.11863-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240613060708.11761-1-yan.y.zhao@intel.com> References: <20240613060708.11761-1-yan.y.zhao@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a new user option to memslot_perf_test to allow testing memslot move with quirk KVM_X86_QUIRK_SLOT_ZAP_ALL disabled. Signed-off-by: Yan Zhao --- tools/testing/selftests/kvm/memslot_perf_test.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/memslot_perf_test.c b/tools/testing/selftests/kvm/memslot_perf_test.c index 579a64f97333..893366982f77 100644 --- a/tools/testing/selftests/kvm/memslot_perf_test.c +++ b/tools/testing/selftests/kvm/memslot_perf_test.c @@ -113,6 +113,7 @@ static_assert(ATOMIC_BOOL_LOCK_FREE == 2, "atomic bool is not lockless"); static sem_t vcpu_ready; static bool map_unmap_verify; +static bool disable_slot_zap_quirk; static bool verbose; #define pr_info_v(...) \ @@ -578,6 +579,9 @@ static bool test_memslot_move_prepare(struct vm_data *data, uint32_t guest_page_size = data->vm->page_size; uint64_t movesrcgpa, movetestgpa; + if (disable_slot_zap_quirk) + vm_enable_cap(data->vm, KVM_CAP_DISABLE_QUIRKS2, KVM_X86_QUIRK_SLOT_ZAP_ALL); + movesrcgpa = vm_slot2gpa(data, data->nslots - 1); if (isactive) { @@ -896,6 +900,7 @@ static void help(char *name, struct test_args *targs) pr_info(" -h: print this help screen.\n"); pr_info(" -v: enable verbose mode (not for benchmarking).\n"); pr_info(" -d: enable extra debug checks.\n"); + pr_info(" -q: Disable memslot zap quirk during memslot move.\n"); pr_info(" -s: specify memslot count cap (-1 means no cap; currently: %i)\n", targs->nslots); pr_info(" -f: specify the first test to run (currently: %i; max %zu)\n", @@ -954,7 +959,7 @@ static bool parse_args(int argc, char *argv[], uint32_t max_mem_slots; int opt; - while ((opt = getopt(argc, argv, "hvds:f:e:l:r:")) != -1) { + while ((opt = getopt(argc, argv, "hvdqs:f:e:l:r:")) != -1) { switch (opt) { case 'h': default: @@ -966,6 +971,11 @@ static bool parse_args(int argc, char *argv[], case 'd': map_unmap_verify = true; break; + case 'q': + disable_slot_zap_quirk = true; + TEST_REQUIRE(kvm_check_cap(KVM_CAP_DISABLE_QUIRKS2) & + KVM_X86_QUIRK_SLOT_ZAP_ALL); + break; case 's': targs->nslots = atoi_paranoid(optarg); if (targs->nslots <= 1 && targs->nslots != -1) { From patchwork Thu Jun 13 06:11:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 13696174 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EDB3135A4B; Thu, 13 Jun 2024 06:12:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259128; cv=none; b=M95oN+K1Bz+yPD6coopN4lwOFzopdWhusGFf2DOEaWbV3Qs7AMiNKPIKQ5M3LXs0wGzxOy9zDt5/OCF7O9fl0cdSOTPWUAuDN4/n7OLWoaTllvzFNLbP+Txd5wuwmFULXlf7I7RiTpgAq/5VDBAnErZB6coYqC42D824ycG4aNU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718259128; c=relaxed/simple; bh=200iro4Vmpjs5iDslEXl3WgKuWoTIPG9CIIZtZp9mVc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hB0se2jEPa6uI1G7Qnwighl8hyUcKOhVMOkN4wkFQNgSbC59jws3/dXfUv7PH6S47ZD2tyUiGpUSLVpXve80mqYe98ofVEUFGlBzNsM3uDCUeDehamM6k/0pWNNCHCv9Catvv+ewIqbRe/cSk5TBAY9XtX6R2cmVwwknzSV05fs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=KPVGPkHM; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KPVGPkHM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1718259127; x=1749795127; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=200iro4Vmpjs5iDslEXl3WgKuWoTIPG9CIIZtZp9mVc=; b=KPVGPkHMPHKJSPU4JslM3u65erR82kLl19jKoiZnUhWgVxytLd5Fq1+D WS9x810lH+wjbOj1zDBckHHU408jG8MPe4Z5xtZMaN7jvxwtNoxmaM/En ytQ4kqsVcQ5ETSkUtO92qO5kooZqQivyk8mYreQwPyjqPalc0RVmx9qJR P+A44FIgA7BFG8VIetjLsVBepxk/zfxg3SYpNC2l0QrwneKNLBevY8MBM a9bKs7C6uA0R79bOQ3dMyynNfM2hjNskuJ6tij9WNhyqvf1y0xugR6wBn b7KyEiRVo8E8wxgFxNuha2fPvM3j+LnC5xk8IQd/LlyfWdV5CHjKAAKR6 g==; X-CSE-ConnectionGUID: friMxZrJRLeaaBYMdFM7Qg== X-CSE-MsgGUID: DaDIW5QmTE6Q8uoJOXLZgg== X-IronPort-AV: E=McAfee;i="6700,10204,11101"; a="25734269" X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="25734269" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:12:07 -0700 X-CSE-ConnectionGUID: xvVCxiNORzShhmx7pEI/3g== X-CSE-MsgGUID: Jh2Q/CkXTNe0fI8K+ti5hQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,234,1712646000"; d="scan'208";a="40669223" Received: from yzhao56-desk.sh.intel.com ([10.239.159.62]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2024 23:12:03 -0700 From: Yan Zhao To: pbonzini@redhat.com, seanjc@google.com Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, isaku.yamahata@intel.com, dmatlack@google.com, sagis@google.com, erdemaktas@google.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Yan Zhao Subject: [PATCH 5/5] KVM: selftests: Test private access to deleted memslot with quirk disabled Date: Thu, 13 Jun 2024 14:11:05 +0800 Message-ID: <20240613061105.11880-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240613060708.11761-1-yan.y.zhao@intel.com> References: <20240613060708.11761-1-yan.y.zhao@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Update private_mem_kvm_exits_test to make sure private access to deleted memslot functional both when quirk KVM_X86_QUIRK_SLOT_ZAP_ALL is enabled and disabled. Signed-off-by: Yan Zhao --- .../selftests/kvm/x86_64/private_mem_kvm_exits_test.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/x86_64/private_mem_kvm_exits_test.c b/tools/testing/selftests/kvm/x86_64/private_mem_kvm_exits_test.c index 13e72fcec8dd..a4d304fce294 100644 --- a/tools/testing/selftests/kvm/x86_64/private_mem_kvm_exits_test.c +++ b/tools/testing/selftests/kvm/x86_64/private_mem_kvm_exits_test.c @@ -44,7 +44,7 @@ const struct vm_shape protected_vm_shape = { .type = KVM_X86_SW_PROTECTED_VM, }; -static void test_private_access_memslot_deleted(void) +static void test_private_access_memslot_deleted(bool disable_slot_zap_quirk) { struct kvm_vm *vm; struct kvm_vcpu *vcpu; @@ -55,6 +55,9 @@ static void test_private_access_memslot_deleted(void) vm = vm_create_shape_with_one_vcpu(protected_vm_shape, &vcpu, guest_repeatedly_read); + if (disable_slot_zap_quirk) + vm_enable_cap(vm, KVM_CAP_DISABLE_QUIRKS2, KVM_X86_QUIRK_SLOT_ZAP_ALL); + vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, EXITS_TEST_GPA, EXITS_TEST_SLOT, EXITS_TEST_NPAGES, @@ -115,6 +118,10 @@ int main(int argc, char *argv[]) { TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_VM)); - test_private_access_memslot_deleted(); + test_private_access_memslot_deleted(false); + + if (kvm_check_cap(KVM_CAP_DISABLE_QUIRKS2) & KVM_X86_QUIRK_SLOT_ZAP_ALL) + test_private_access_memslot_deleted(true); + test_private_access_memslot_not_private(); }