| Message ID | 20240906221824.491834-1-mlevitsk@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Relax canonical checks on some arch msrs | expand |
On Fri, 2024-09-06 at 18:18 -0400, Maxim Levitsky wrote: > Recently we came up upon a failure where likely the guest writes > 0xff4547ceb1600000 to MSR_KERNEL_GS_BASE and later on, qemu > sets this value via KVM_PUT_MSRS, and is rejected by the > kernel, likely due to not being canonical in 4 level paging. > > One of the way to trigger this is to make the guest enter SMM, > which causes paging to be disabled, which SMM bios re-enables > but not the whole 5 level. MSR_KERNEL_GS_BASE on the other > hand continues to contain old value. > > I did some reverse engineering and to my surprise I found out > that both Intel and AMD indeed ignore CR4.LA57 when doing > canonical checks on this and other msrs and/or other arch > registers (like GDT base) which contain linear addresses. > > V2: addressed a very good feedback from Chao Gao. Thanks! > > V3: also fix the nested VMX, and also fix the > MSR_IA32_SYSENTER_EIP / MSR_IA32_SYSENTER_ESP > > V4: > - added PT and PEBS msrs > - corrected emulation of SGDT/SIDT/STR/SLDT instructions > - corrected canonical checks for TLB invalidation instructions > > Best regards, > Maxim Levitsky > > Maxim Levitsky (4): > KVM: x86: drop x86.h include from cpuid.h > KVM: x86: implement emul_is_noncanonical_address using > is_noncanonical_address > KVM: x86: model canonical checks more precisely > KVM: nVMX: fix canonical check of vmcs12 HOST_RIP > > arch/x86/kvm/cpuid.h | 1 - > arch/x86/kvm/emulate.c | 15 ++++++----- > arch/x86/kvm/kvm_emulate.h | 5 ++++ > arch/x86/kvm/mmu.h | 1 + > arch/x86/kvm/mmu/mmu.c | 2 +- > arch/x86/kvm/vmx/hyperv.c | 1 + > arch/x86/kvm/vmx/nested.c | 35 +++++++++++++++++--------- > arch/x86/kvm/vmx/pmu_intel.c | 2 +- > arch/x86/kvm/vmx/sgx.c | 5 ++-- > arch/x86/kvm/vmx/vmx.c | 4 +-- > arch/x86/kvm/x86.c | 13 +++++++--- > arch/x86/kvm/x86.h | 49 ++++++++++++++++++++++++++++++++++-- > 12 files changed, 102 insertions(+), 31 deletions(-) > > -- > 2.26.3 > > Hi, A very gentle ping on this patch series. Best regards, Maxim Levitsky
On Wed, Oct 30, 2024, Maxim Levitsky wrote: > On Fri, 2024-09-06 at 18:18 -0400, Maxim Levitsky wrote: > > Recently we came up upon a failure where likely the guest writes > > 0xff4547ceb1600000 to MSR_KERNEL_GS_BASE and later on, qemu > > sets this value via KVM_PUT_MSRS, and is rejected by the > > kernel, likely due to not being canonical in 4 level paging. > > > > One of the way to trigger this is to make the guest enter SMM, > > which causes paging to be disabled, which SMM bios re-enables > > but not the whole 5 level. MSR_KERNEL_GS_BASE on the other > > hand continues to contain old value. > > > > I did some reverse engineering and to my surprise I found out > > that both Intel and AMD indeed ignore CR4.LA57 when doing > > canonical checks on this and other msrs and/or other arch > > registers (like GDT base) which contain linear addresses. > > > > V2: addressed a very good feedback from Chao Gao. Thanks! > > > > V3: also fix the nested VMX, and also fix the > > MSR_IA32_SYSENTER_EIP / MSR_IA32_SYSENTER_ESP > > > > V4: > > - added PT and PEBS msrs > > - corrected emulation of SGDT/SIDT/STR/SLDT instructions > > - corrected canonical checks for TLB invalidation instructions > > > > Best regards, > > Maxim Levitsky > > > > Maxim Levitsky (4): > > KVM: x86: drop x86.h include from cpuid.h > > KVM: x86: implement emul_is_noncanonical_address using > > is_noncanonical_address > > KVM: x86: model canonical checks more precisely > > KVM: nVMX: fix canonical check of vmcs12 HOST_RIP > > > > arch/x86/kvm/cpuid.h | 1 - > > arch/x86/kvm/emulate.c | 15 ++++++----- > > arch/x86/kvm/kvm_emulate.h | 5 ++++ > > arch/x86/kvm/mmu.h | 1 + > > arch/x86/kvm/mmu/mmu.c | 2 +- > > arch/x86/kvm/vmx/hyperv.c | 1 + > > arch/x86/kvm/vmx/nested.c | 35 +++++++++++++++++--------- > > arch/x86/kvm/vmx/pmu_intel.c | 2 +- > > arch/x86/kvm/vmx/sgx.c | 5 ++-- > > arch/x86/kvm/vmx/vmx.c | 4 +-- > > arch/x86/kvm/x86.c | 13 +++++++--- > > arch/x86/kvm/x86.h | 49 ++++++++++++++++++++++++++++++++++-- > > 12 files changed, 102 insertions(+), 31 deletions(-) > > > > -- > > 2.26.3 > > > > > > Hi, > A very gentle ping on this patch series. Heh, good timing, I literally (like, 2 seconds ago) applied this (still need to test before you'll see a "thank you" email).
On Wed, 2024-10-30 at 14:22 -0700, Sean Christopherson wrote: > On Wed, Oct 30, 2024, Maxim Levitsky wrote: > > On Fri, 2024-09-06 at 18:18 -0400, Maxim Levitsky wrote: > > > Recently we came up upon a failure where likely the guest writes > > > 0xff4547ceb1600000 to MSR_KERNEL_GS_BASE and later on, qemu > > > sets this value via KVM_PUT_MSRS, and is rejected by the > > > kernel, likely due to not being canonical in 4 level paging. > > > > > > One of the way to trigger this is to make the guest enter SMM, > > > which causes paging to be disabled, which SMM bios re-enables > > > but not the whole 5 level. MSR_KERNEL_GS_BASE on the other > > > hand continues to contain old value. > > > > > > I did some reverse engineering and to my surprise I found out > > > that both Intel and AMD indeed ignore CR4.LA57 when doing > > > canonical checks on this and other msrs and/or other arch > > > registers (like GDT base) which contain linear addresses. > > > > > > V2: addressed a very good feedback from Chao Gao. Thanks! > > > > > > V3: also fix the nested VMX, and also fix the > > > MSR_IA32_SYSENTER_EIP / MSR_IA32_SYSENTER_ESP > > > > > > V4: > > > - added PT and PEBS msrs > > > - corrected emulation of SGDT/SIDT/STR/SLDT instructions > > > - corrected canonical checks for TLB invalidation instructions > > > > > > Best regards, > > > Maxim Levitsky > > > > > > Maxim Levitsky (4): > > > KVM: x86: drop x86.h include from cpuid.h > > > KVM: x86: implement emul_is_noncanonical_address using > > > is_noncanonical_address > > > KVM: x86: model canonical checks more precisely > > > KVM: nVMX: fix canonical check of vmcs12 HOST_RIP > > > > > > arch/x86/kvm/cpuid.h | 1 - > > > arch/x86/kvm/emulate.c | 15 ++++++----- > > > arch/x86/kvm/kvm_emulate.h | 5 ++++ > > > arch/x86/kvm/mmu.h | 1 + > > > arch/x86/kvm/mmu/mmu.c | 2 +- > > > arch/x86/kvm/vmx/hyperv.c | 1 + > > > arch/x86/kvm/vmx/nested.c | 35 +++++++++++++++++--------- > > > arch/x86/kvm/vmx/pmu_intel.c | 2 +- > > > arch/x86/kvm/vmx/sgx.c | 5 ++-- > > > arch/x86/kvm/vmx/vmx.c | 4 +-- > > > arch/x86/kvm/x86.c | 13 +++++++--- > > > arch/x86/kvm/x86.h | 49 ++++++++++++++++++++++++++++++++++-- > > > 12 files changed, 102 insertions(+), 31 deletions(-) > > > > > > -- > > > 2.26.3 > > > > > > > > > > Hi, > > A very gentle ping on this patch series. > > Heh, good timing, I literally (like, 2 seconds ago) applied this (still need to > test before you'll see a "thank you" email). > Thank you! Best regards, Maxim Levitsky
On Fri, 06 Sep 2024 18:18:20 -0400, Maxim Levitsky wrote: > Recently we came up upon a failure where likely the guest writes > 0xff4547ceb1600000 to MSR_KERNEL_GS_BASE and later on, qemu > sets this value via KVM_PUT_MSRS, and is rejected by the > kernel, likely due to not being canonical in 4 level paging. > > One of the way to trigger this is to make the guest enter SMM, > which causes paging to be disabled, which SMM bios re-enables > but not the whole 5 level. MSR_KERNEL_GS_BASE on the other > hand continues to contain old value. > > [...] Applied to kvm-x86 misc, with some massaging (see responsed to individual patches). Thanks! [1/4] KVM: x86: drop x86.h include from cpuid.h https://github.com/kvm-x86/linux/commit/391bd0c520c1 [2/4] KVM: x86: implement emul_is_noncanonical_address using is_noncanonical_address https://github.com/kvm-x86/linux/commit/6c45d62536d0 [3/4] KVM: x86: model canonical checks more precisely https://github.com/kvm-x86/linux/commit/1b1336d1d858 [4/4] KVM: nVMX: fix canonical check of vmcs12 HOST_RIP https://github.com/kvm-x86/linux/commit/14a95598b6e7 -- https://github.com/kvm-x86/linux/tree/next
On Thu, Oct 31, 2024, Sean Christopherson wrote: > On Fri, 06 Sep 2024 18:18:20 -0400, Maxim Levitsky wrote: > > Recently we came up upon a failure where likely the guest writes > > 0xff4547ceb1600000 to MSR_KERNEL_GS_BASE and later on, qemu > > sets this value via KVM_PUT_MSRS, and is rejected by the > > kernel, likely due to not being canonical in 4 level paging. > > > > One of the way to trigger this is to make the guest enter SMM, > > which causes paging to be disabled, which SMM bios re-enables > > but not the whole 5 level. MSR_KERNEL_GS_BASE on the other > > hand continues to contain old value. > > > > [...] > > Applied to kvm-x86 misc, with some massaging (see responsed to individual > patches). Thanks! > > [1/4] KVM: x86: drop x86.h include from cpuid.h > https://github.com/kvm-x86/linux/commit/391bd0c520c1 > [2/4] KVM: x86: implement emul_is_noncanonical_address using is_noncanonical_address > https://github.com/kvm-x86/linux/commit/6c45d62536d0 > [3/4] KVM: x86: model canonical checks more precisely > https://github.com/kvm-x86/linux/commit/1b1336d1d858 > [4/4] KVM: nVMX: fix canonical check of vmcs12 HOST_RIP > https://github.com/kvm-x86/linux/commit/14a95598b6e7 FYI, I rebased misc to v6.12-rc5, as patches in another series had already been taken through the tip tree. New hashes: [1/5] KVM: x86: drop x86.h include from cpuid.h https://github.com/kvm-x86/linux/commit/e52ad1ddd0a3 [2/5] KVM: x86: Route non-canonical checks in emulator through emulate_ops https://github.com/kvm-x86/linux/commit/16ccadefa295 [3/5] KVM: x86: Add X86EMUL_F_MSR and X86EMUL_F_DT_LOAD to aid canonical checks https://github.com/kvm-x86/linux/commit/c534b37b7584 [4/5] KVM: x86: model canonical checks more precisely https://github.com/kvm-x86/linux/commit/9245fd6b8531 [5/5] KVM: nVMX: fix canonical check of vmcs12 HOST_RIP https://github.com/kvm-x86/linux/commit/90a877216e6b
Recently we came up upon a failure where likely the guest writes 0xff4547ceb1600000 to MSR_KERNEL_GS_BASE and later on, qemu sets this value via KVM_PUT_MSRS, and is rejected by the kernel, likely due to not being canonical in 4 level paging. One of the way to trigger this is to make the guest enter SMM, which causes paging to be disabled, which SMM bios re-enables but not the whole 5 level. MSR_KERNEL_GS_BASE on the other hand continues to contain old value. I did some reverse engineering and to my surprise I found out that both Intel and AMD indeed ignore CR4.LA57 when doing canonical checks on this and other msrs and/or other arch registers (like GDT base) which contain linear addresses. V2: addressed a very good feedback from Chao Gao. Thanks! V3: also fix the nested VMX, and also fix the MSR_IA32_SYSENTER_EIP / MSR_IA32_SYSENTER_ESP V4: - added PT and PEBS msrs - corrected emulation of SGDT/SIDT/STR/SLDT instructions - corrected canonical checks for TLB invalidation instructions Best regards, Maxim Levitsky Maxim Levitsky (4): KVM: x86: drop x86.h include from cpuid.h KVM: x86: implement emul_is_noncanonical_address using is_noncanonical_address KVM: x86: model canonical checks more precisely KVM: nVMX: fix canonical check of vmcs12 HOST_RIP arch/x86/kvm/cpuid.h | 1 - arch/x86/kvm/emulate.c | 15 ++++++----- arch/x86/kvm/kvm_emulate.h | 5 ++++ arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/vmx/hyperv.c | 1 + arch/x86/kvm/vmx/nested.c | 35 +++++++++++++++++--------- arch/x86/kvm/vmx/pmu_intel.c | 2 +- arch/x86/kvm/vmx/sgx.c | 5 ++-- arch/x86/kvm/vmx/vmx.c | 4 +-- arch/x86/kvm/x86.c | 13 +++++++--- arch/x86/kvm/x86.h | 49 ++++++++++++++++++++++++++++++++++-- 12 files changed, 102 insertions(+), 31 deletions(-)