From patchwork Fri Aug 1 23:54:52 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Matlack X-Patchwork-Id: 4664591 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id B167C9F2B8 for ; Fri, 1 Aug 2014 23:55:05 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B119820219 for ; Fri, 1 Aug 2014 23:55:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A659520218 for ; Fri, 1 Aug 2014 23:55:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751858AbaHAXzA (ORCPT ); Fri, 1 Aug 2014 19:55:00 -0400 Received: from mail-ie0-f175.google.com ([209.85.223.175]:62239 "EHLO mail-ie0-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750997AbaHAXy7 (ORCPT ); Fri, 1 Aug 2014 19:54:59 -0400 Received: by mail-ie0-f175.google.com with SMTP id x19so6868507ier.20 for ; Fri, 01 Aug 2014 16:54:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:to:cc:subject:date:message-id; bh=19oGmjNEsAUrMwjIid7v29/KoLQPPhjJIBLII4D4TXQ=; b=kCW0PVDyGfjn2IgqCEqaN18ZjIwCsg2NYkaNvUPi7nIi+dV08b3epDgneiLQxV9kiU xa4bTqUhDbyj/4LeOYVTy8/5YQFQ1KHhZxvvJaMVG0nszVOoaOFe41Earz0Chl7EqRQh npsCva/kBQqUcCoQAXf7B/4ciMqzk6w2dn26bFRsVcjnUSJop0wz7caGWSTQBSLmRo+t dSf9R6JNGo5OO797mtIrvt8hH48JyPGz25MmGaeXD3e6bFUQjJNOKUau6vngxHToLb/5 Y8HFWqVrWReJgVXIED42YSFTTLZhzldwdt8VmG4fINEih0OrTJ0sg/ZiQjXe7LGDS0bx 6QCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=19oGmjNEsAUrMwjIid7v29/KoLQPPhjJIBLII4D4TXQ=; b=WgEaxEJTcLN8Z/NhdIkO2s6C/CLx5Es3752hKi0eGgoZ+J1B/5KcMmr+UBL2sk6NoI NP5HOvgWz0zI3/Dj4eZPLILwFYGKE/ieK9FUGhJ6mPYe2YYczwYo5okwkqruCaC/f+KN nAWD9y95RvHn5/f18LQUtj8tpVkudIiCHOsQqTnlsSMSEeLAFJO3sBLJgTCKhrkIn67I JW1jY9ixoMyW+fmSRRdvPbxr3oYSm6iofuzRldKuIsgoaxZ/9xeiFvVohNhCoGbno+bw HUcjOkl7+UO/EABUertasKST2bzXfxFQAQFCH3GW9klpq6mqUvHnxOVw4dognUbF18hd 5+ng== X-Gm-Message-State: ALoCoQmjR8JyCyu9t26+pTNrlA7ifc49Mh3cnIRZRluKXdkp4hbGZ/6O4tz4zPPGyVCl+C9fw+vr X-Received: by 10.43.140.193 with SMTP id jb1mr9578577icc.15.1406937298095; Fri, 01 Aug 2014 16:54:58 -0700 (PDT) Received: from dmatlack-linux.kir.corp.google.com (dmatlack-linux.kir.corp.google.com [172.31.88.63]) by mx.google.com with ESMTPSA id d4sm15763783igc.5.2014.08.01.16.54.56 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 01 Aug 2014 16:54:57 -0700 (PDT) From: David Matlack To: Gleb Natapov , Paolo Bonzini , Xiao Guangrong , kvm@vger.kernel.org, x86@kernel.org Cc: Eric Northup , David Matlack Subject: [RFC][PATCH] kvm: x86: fix stale mmio cache bug Date: Fri, 1 Aug 2014 16:54:52 -0700 Message-Id: <1406937292-32033-1-git-send-email-dmatlack@google.com> X-Mailer: git-send-email 2.0.0.526.g5318336 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The following events can lead to an incorrect KVM_EXIT_MMIO bubbling up to userspace: (1) Guest accesses gpa X without a memory slot. The gfn is cached in struct kvm_vcpu_arch (mmio_gfn). On Intel EPT-enabled hosts, KVM sets the SPTE write-execute-noread so that future accesses cause EPT_MISCONFIGs. (2) Host userspace creates a memory slot via KVM_SET_USER_MEMORY_REGION covering the page just accessed. (3) Guest attempts to read or write to gpa X again. On Intel, this generates an EPT_MISCONFIG. The memory slot generation number that was incremented in (2) would normally take care of this but we fast path mmio faults through quickly_check_mmio_pf(), which only checks the per-vcpu mmio cache. Since we hit the cache, KVM passes a KVM_EXIT_MMIO up to userspace. This patch fixes the issue by clearing the mmio cache in the KVM_MR_CREATE code path. - introduce KVM_REQ_CLEAR_MMIO_CACHE for clearing all vcpu mmio caches. - extend vcpu_clear_mmio_info to clear mmio_gfn in addition to mmio_gva, since both can be used to fast path mmio faults. - issue KVM_REQ_CLEAR_MMIO_CACHE during memslot creation to flush the mmio cache. - in mmu_sync_roots, unconditionally clear the mmio cache since even direct_map (e.g. tdp) hosts use it. Signed-off-by: David Matlack --- arch/x86/kvm/mmu.c | 3 ++- arch/x86/kvm/x86.c | 5 +++++ arch/x86/kvm/x86.h | 8 +++++--- include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 10 +++++----- 5 files changed, 19 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index 9314678..8d50b84 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -3157,13 +3157,14 @@ static void mmu_sync_roots(struct kvm_vcpu *vcpu) int i; struct kvm_mmu_page *sp; + vcpu_clear_mmio_info(vcpu, MMIO_GVA_ANY); + if (vcpu->arch.mmu.direct_map) return; if (!VALID_PAGE(vcpu->arch.mmu.root_hpa)) return; - vcpu_clear_mmio_info(vcpu, ~0ul); kvm_mmu_audit(vcpu, AUDIT_PRE_SYNC); if (vcpu->arch.mmu.root_level == PT64_ROOT_LEVEL) { hpa_t root = vcpu->arch.mmu.root_hpa; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ef432f8..05b5629 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6001,6 +6001,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) kvm_deliver_pmi(vcpu); if (kvm_check_request(KVM_REQ_SCAN_IOAPIC, vcpu)) vcpu_scan_ioapic(vcpu); + + if (kvm_check_request(KVM_REQ_CLEAR_MMIO_CACHE, vcpu)) + vcpu_clear_mmio_info(vcpu, MMIO_GVA_ANY); } if (kvm_check_request(KVM_REQ_EVENT, vcpu) || req_int_win) { @@ -7281,6 +7284,8 @@ void kvm_arch_memslots_updated(struct kvm *kvm) * mmio generation may have reached its maximum value. */ kvm_mmu_invalidate_mmio_sptes(kvm); + + kvm_make_all_vcpus_request(kvm, KVM_REQ_CLEAR_MMIO_CACHE); } int kvm_arch_prepare_memory_region(struct kvm *kvm, diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 8c97bac..41ef197 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -81,15 +81,17 @@ static inline void vcpu_cache_mmio_info(struct kvm_vcpu *vcpu, } /* - * Clear the mmio cache info for the given gva, - * specially, if gva is ~0ul, we clear all mmio cache info. + * Clear the mmio cache info for the given gva. If gva is MMIO_GVA_ANY, + * unconditionally clear the mmio cache. */ +#define MMIO_GVA_ANY (~0ul) static inline void vcpu_clear_mmio_info(struct kvm_vcpu *vcpu, gva_t gva) { - if (gva != (~0ul) && vcpu->arch.mmio_gva != (gva & PAGE_MASK)) + if (gva != MMIO_GVA_ANY && vcpu->arch.mmio_gva != (gva & PAGE_MASK)) return; vcpu->arch.mmio_gva = 0; + vcpu->arch.mmio_gfn = 0; } static inline bool vcpu_match_mmio_gva(struct kvm_vcpu *vcpu, unsigned long gva) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ec4e3bd..e4edaff 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -136,6 +136,7 @@ static inline bool is_error_page(struct page *page) #define KVM_REQ_GLOBAL_CLOCK_UPDATE 22 #define KVM_REQ_ENABLE_IBS 23 #define KVM_REQ_DISABLE_IBS 24 +#define KVM_REQ_CLEAR_MMIO_CACHE 25 #define KVM_USERSPACE_IRQ_SOURCE_ID 0 #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID 1 @@ -591,6 +592,7 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *vcpu); void kvm_load_guest_fpu(struct kvm_vcpu *vcpu); void kvm_put_guest_fpu(struct kvm_vcpu *vcpu); +bool kvm_make_all_vcpus_request(struct kvm *kvm, unsigned int req); void kvm_flush_remote_tlbs(struct kvm *kvm); void kvm_reload_remote_mmus(struct kvm *kvm); void kvm_make_mclock_inprogress_request(struct kvm *kvm); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 4b6c01b..d09527a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -152,7 +152,7 @@ static void ack_flush(void *_completed) { } -static bool make_all_cpus_request(struct kvm *kvm, unsigned int req) +bool kvm_make_all_vcpus_request(struct kvm *kvm, unsigned int req) { int i, cpu, me; cpumask_var_t cpus; @@ -189,7 +189,7 @@ void kvm_flush_remote_tlbs(struct kvm *kvm) long dirty_count = kvm->tlbs_dirty; smp_mb(); - if (make_all_cpus_request(kvm, KVM_REQ_TLB_FLUSH)) + if (kvm_make_all_vcpus_request(kvm, KVM_REQ_TLB_FLUSH)) ++kvm->stat.remote_tlb_flush; cmpxchg(&kvm->tlbs_dirty, dirty_count, 0); } @@ -197,17 +197,17 @@ EXPORT_SYMBOL_GPL(kvm_flush_remote_tlbs); void kvm_reload_remote_mmus(struct kvm *kvm) { - make_all_cpus_request(kvm, KVM_REQ_MMU_RELOAD); + kvm_make_all_vcpus_request(kvm, KVM_REQ_MMU_RELOAD); } void kvm_make_mclock_inprogress_request(struct kvm *kvm) { - make_all_cpus_request(kvm, KVM_REQ_MCLOCK_INPROGRESS); + kvm_make_all_vcpus_request(kvm, KVM_REQ_MCLOCK_INPROGRESS); } void kvm_make_scan_ioapic_request(struct kvm *kvm) { - make_all_cpus_request(kvm, KVM_REQ_SCAN_IOAPIC); + kvm_make_all_vcpus_request(kvm, KVM_REQ_SCAN_IOAPIC); } int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id)