From patchwork Mon Jan 9 06:24:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jintack Lim X-Patchwork-Id: 9503857 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1BBDC60757 for ; Mon, 9 Jan 2017 06:28:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C4942811C for ; Mon, 9 Jan 2017 06:28:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 00B512838C; Mon, 9 Jan 2017 06:28:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89C042823E for ; Mon, 9 Jan 2017 06:28:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S942274AbdAIG2b (ORCPT ); Mon, 9 Jan 2017 01:28:31 -0500 Received: from outprodmail02.cc.columbia.edu ([128.59.72.51]:52462 "EHLO outprodmail02.cc.columbia.edu" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S939858AbdAIG0m (ORCPT ); Mon, 9 Jan 2017 01:26:42 -0500 Received: from hazelnut (hazelnut.cc.columbia.edu [128.59.213.250]) by outprodmail02.cc.columbia.edu (8.14.4/8.14.4) with ESMTP id v096PXDN005328 for ; Mon, 9 Jan 2017 01:26:34 -0500 Received: from hazelnut (localhost.localdomain [127.0.0.1]) by hazelnut (Postfix) with ESMTP id 2BF4780 for ; Mon, 9 Jan 2017 01:26:34 -0500 (EST) Received: from sendprodmail02.cc.columbia.edu (sendprodmail02.cc.columbia.edu [128.59.72.14]) by hazelnut (Postfix) with ESMTP id 0FF0A6D for ; Mon, 9 Jan 2017 01:26:34 -0500 (EST) Received: from mail-qt0-f199.google.com (mail-qt0-f199.google.com [209.85.216.199]) by sendprodmail02.cc.columbia.edu (8.14.4/8.14.4) with ESMTP id v096QX8q043076 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 9 Jan 2017 01:26:33 -0500 Received: by mail-qt0-f199.google.com with SMTP id k15so94311218qtg.5 for ; Sun, 08 Jan 2017 22:26:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pBoYn7AGcV6yIgQVRGmhf7m8PNt401Ix7tyOglmAo2o=; b=ZWfqrCf9M6Ies7rYYuRB2p+LKQ/v71lDrLXzl16bMf8XpjGkEdAg2Ng0pvTMzjkZAk r0NVGNidE1XALwcCPna6IdxJCwcIIl5nKk3CjeZj57aGbZ5dGEE2af/I6OP1N1sIbxL4 qqtwER3687XEDtwuq6GgEN5i80ucibEqDvWbApzPh72+0VbvToN7LykUhr5Fro5nyDqa pY4915IyabXg9n8uv91oJsjegzVyQ1it6IktozWdct4h6RQTxwDp4G3MAYWfODTNzgWg bzHkEYiG/uLMXcpcTVOh+MGiMfNVY12RsoFIrG6Z9h77dR7BNwaTn2P5hILwDAXEJTVm XO0A== X-Gm-Message-State: AIkVDXKIIe5LMMQWB/3vG6KE45UhbsF6ehxaeibuNysQaZETmRk12pjJcZmJffT3w5DdKaE0S0UjwWnszn8xZbkeab9rlB2zqvX7tNHGpXVZ/mSGTEl26iRc6nuCf2h214FM9m3+w8PmByA= X-Received: by 10.55.122.197 with SMTP id v188mr79430407qkc.120.1483943193326; Sun, 08 Jan 2017 22:26:33 -0800 (PST) X-Received: by 10.55.122.197 with SMTP id v188mr79430375qkc.120.1483943193148; Sun, 08 Jan 2017 22:26:33 -0800 (PST) Received: from jintack.cs.columbia.edu ([2001:18d8:ffff:16:21a:4aff:feaa:f900]) by smtp.gmail.com with ESMTPSA id h3sm8623257qtc.6.2017.01.08.22.26.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 08 Jan 2017 22:26:32 -0800 (PST) From: Jintack Lim To: christoffer.dall@linaro.org, marc.zyngier@arm.com, pbonzini@redhat.com, rkrcmar@redhat.com, linux@armlinux.org.uk, catalin.marinas@arm.com, will.deacon@arm.com, vladimir.murzin@arm.com, suzuki.poulose@arm.com, mark.rutland@arm.com, james.morse@arm.com, lorenzo.pieralisi@arm.com, kevin.brodsky@arm.com, wcohen@redhat.com, shankerd@codeaurora.org, geoff@infradead.org, andre.przywara@arm.com, eric.auger@redhat.com, anna-maria@linutronix.de, shihwei@cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: jintack@cs.columbia.edu Subject: [RFC 49/55] KVM: arm64: Fixes to toggle_cache for nesting Date: Mon, 9 Jan 2017 01:24:45 -0500 Message-Id: <1483943091-1364-50-git-send-email-jintack@cs.columbia.edu> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1483943091-1364-1-git-send-email-jintack@cs.columbia.edu> References: <1483943091-1364-1-git-send-email-jintack@cs.columbia.edu> X-No-Spam-Score: Local X-Scanned-By: MIMEDefang 2.78 on 128.59.72.14 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Christoffer Dall So far we were flushing almost the entire universe whenever a VM would load/unload the SCTLR_EL1 and the two versions of that register had different MMU enabled settings. This turned out to be so slow that it prevented forward progress for a nested VM, because a scheduler timer tick interrupt would always be pending when we reached the nested VM. To avoid this problem, we consider the SCTLR_EL2 when evaluating if caches are on or off when entering virtual EL2 (because this is the value that we end up shadowing onto the hardware EL1 register). We also reduce the scope of the flush operation to only flush shadow stage 2 page table state of the particular VCPU toggling the caches instead of the shadow stage 2 state of all possible VCPUs. Signed-off-by: Christoffer Dall Signed-off-by: Jintack Lim --- arch/arm/kvm/mmu.c | 31 ++++++++++++++++++++++++++++++- arch/arm64/include/asm/kvm_mmu.h | 7 ++++++- 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c index 68fc8e8..344bc01 100644 --- a/arch/arm/kvm/mmu.c +++ b/arch/arm/kvm/mmu.c @@ -422,6 +422,35 @@ static void stage2_flush_vm(struct kvm *kvm) srcu_read_unlock(&kvm->srcu, idx); } +/** + * Same as above but only flushed shadow state for specific vcpu + */ +static void stage2_flush_vcpu(struct kvm_vcpu *vcpu) +{ + struct kvm *kvm = vcpu->kvm; + struct kvm_memslots *slots; + struct kvm_memory_slot *memslot; + int idx; + struct kvm_nested_s2_mmu __maybe_unused *nested_mmu; + + idx = srcu_read_lock(&kvm->srcu); + spin_lock(&kvm->mmu_lock); + + slots = kvm_memslots(kvm); + kvm_for_each_memslot(memslot, slots) + stage2_flush_memslot(&kvm->arch.mmu, memslot); + +#ifdef CONFIG_KVM_ARM_NESTED_HYP + list_for_each_entry_rcu(nested_mmu, &vcpu->kvm->arch.nested_mmu_list, + list) { + kvm_stage2_flush_range(&nested_mmu->mmu, 0, KVM_PHYS_SIZE); + } +#endif + + spin_unlock(&kvm->mmu_lock); + srcu_read_unlock(&kvm->srcu, idx); +} + static void clear_hyp_pgd_entry(pgd_t *pgd) { pud_t *pud_table __maybe_unused = pud_offset(pgd, 0UL); @@ -2074,7 +2103,7 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled) * Clean + invalidate does the trick always. */ if (now_enabled != was_enabled) - stage2_flush_vm(vcpu->kvm); + stage2_flush_vcpu(vcpu); /* Caches are now on, stop trapping VM ops (until a S/W op) */ if (now_enabled) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 2086296..7754f3e 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -241,7 +241,12 @@ static inline bool kvm_page_empty(void *ptr) static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) { - return (vcpu_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101; + u32 mode = vcpu->arch.ctxt.gp_regs.regs.pstate & PSR_MODE_MASK; + + if (mode != PSR_MODE_EL2h && mode != PSR_MODE_EL2t) + return (vcpu_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101; + else + return (vcpu_el2_reg(vcpu, SCTLR_EL2) & 0b101) == 0b101; } static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu,