From patchwork Fri Jan 12 12:07:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 10160509 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D6DBE602D8 for ; Fri, 12 Jan 2018 12:13:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B9E7E289D2 for ; Fri, 12 Jan 2018 12:13:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AE6DD289D4; Fri, 12 Jan 2018 12:13:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0E9FA289D2 for ; Fri, 12 Jan 2018 12:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=SrCIi69jLM9X8hfEaKIQ+e287yQTePRgHm7dNrP5d3A=; b=R47blDlPxi6TgF0GE+Driyl+cz fN/413cWVzAJYYFI3xDgitd/uFbYEOrVOUQtvg5Wz0LSOW1ZDjMtJWJ6NL2LR8RUVyZbzFTjaEi1w YrUnjzYb9/Fvnnge/GsES40DiDt20Zp6zYMAzIHgTciIdm/7d7BgbOeguAc6BsUMwTKPSKHXDBUCg b5m53RuIjEkq4JX+KhAh+ZPzjefvtIRvNdv/pi0JGW88Q1mwCtkhtKU4Xf+mIRcniARW38qUra7ST 2Z2sIIq3NTTKYOfHjxZHNfPxHz+duikg22KJYd1MiGGit9BWPYaIPOET8A3USP7mCmYtX1/2HfSfW 8O2/2mXA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eZyDH-00085F-Ur; Fri, 12 Jan 2018 12:13:35 +0000 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eZy8S-0000nb-To for linux-arm-kernel@lists.infradead.org; Fri, 12 Jan 2018 12:09:00 +0000 Received: by mail-wm0-x244.google.com with SMTP id v123so2213193wmd.5 for ; Fri, 12 Jan 2018 04:08:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=n8yovchZ8ruI5wsHzziBvFcmgDRlInl2ve25FC3zoww=; b=P2J36YfLbcHfDTrcNHwJNEyTz5/Od/A2NX3WXT5KqatP7K267lTHsGm2UxBmO6csjn 1DIrvD06TVojZNHkOOIDRB2ewPqwZLYTPqIz7o9zLb3+kvpVt1YVsDQbYz4RN9U5/RKS KiLK6pQSGw6OE+QyDQEVa1+Fzq/X8zI/W020o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=n8yovchZ8ruI5wsHzziBvFcmgDRlInl2ve25FC3zoww=; b=VobWPrVnJGl0wlT9tn/IxV5vfHBDysqOibzoZPgSvmfe8z0ICRWYyNx70yXpH+Zi2T z3nkfBSeSIGFC5HdjmNKOS4inZ3q8sEDy90FS2gLD7Bg/2/5kot6gGD3XD/e5tN14r3n 5nxu5jE+E8DE8epOgx7fqfWFPMAx+fOy09PSpoSZk+iH58plIvHIdGtgKq7uwxtNHbx9 zgtvhWbOziOl+LgZNXXXEfOqNFJ6gqMLmdsDlYeqcrO78Q3xk8cJW0f9MsN2h2hZT+yf 0xg6pyCpoJSzDi46YJxsIZMhDCRerWAS9BuUORWU9yvhHWY2n2PlgSqSKIet4WGjEKZh yUAg== X-Gm-Message-State: AKGB3mL22mclZsEe8yEUfnWqru/hR8gjRvuZZRUGI6w/5I7H34Sz2IqC +V27mW2Rr9UK5YNaHN6tczMZQEV1w1A= X-Google-Smtp-Source: ACJfBotT/4HbegflMdYxrXlw9Y+0qBRwnkOBpgy78SPhv9w4WoIRfvDUn26luaMRhivshBdX4hh3bg== X-Received: by 10.80.144.241 with SMTP id d46mr34871622eda.263.1515758915428; Fri, 12 Jan 2018 04:08:35 -0800 (PST) Received: from localhost.localdomain (x50d2404e.cust.hiper.dk. [80.210.64.78]) by smtp.gmail.com with ESMTPSA id f16sm13489705edj.65.2018.01.12.04.08.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 12 Jan 2018 04:08:34 -0800 (PST) From: Christoffer Dall To: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 29/41] KVM: arm64: Defer saving/restoring 64-bit sysregs to vcpu load/put on VHE Date: Fri, 12 Jan 2018 13:07:35 +0100 Message-Id: <20180112120747.27999-30-christoffer.dall@linaro.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180112120747.27999-1-christoffer.dall@linaro.org> References: <20180112120747.27999-1-christoffer.dall@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180112_040837_409979_9AA41D18 X-CRM114-Status: GOOD ( 15.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Zyngier , Andrew Jones , Christoffer Dall , Shih-Wei Li , kvm@vger.kernel.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Some system registers do not affect the host kernel's execution and can therefore be loaded when we are about to run a VCPU and we don't have to restore the host state to the hardware before the time when we are actually about to return to userspace or schedule out the VCPU thread. The EL1 system registers and the userspace state registers only affecting EL0 execution do not need to be saved and restored on every switch between the VM and the host, because they don't affect the host kernel's execution. We mark all registers which are now deffered as such in the declarations in sys-regs.c to ensure the most up-to-date copy is always accessed. Note MPIDR_EL1 (controlled via VMPIDR_EL2) is accessed from other vcpu threads, for example via the GIC emulation, and therefore must be declared as immediate, which is fine as the guest cannot modify this value. The 32-bit sysregs can also be deferred but we do this in a separate patch as it requires a bit more infrastructure. Signed-off-by: Christoffer Dall --- arch/arm64/kvm/hyp/sysreg-sr.c | 37 +++++++++++++++++++++++++++++-------- arch/arm64/kvm/sys_regs.c | 40 ++++++++++++++++++++-------------------- 2 files changed, 49 insertions(+), 28 deletions(-) diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c index 1f2d5e9343b0..eabd35154232 100644 --- a/arch/arm64/kvm/hyp/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/sysreg-sr.c @@ -25,8 +25,12 @@ /* * Non-VHE: Both host and guest must save everything. * - * VHE: Host must save tpidr*_el0, actlr_el1, mdscr_el1, sp_el0, - * and guest must save everything. + * VHE: Host and guest must save mdscr_el1 and sp_el0 (and the PC and pstate, + * which are handled as part of the el2 return state) on every switch. + * tpidr_el0, tpidrro_el0, and actlr_el1 only need to be switched when going + * to host userspace or a different VCPU. EL1 registers only need to be + * switched when potentially going to run a different VCPU. The latter two + * classes are handled as part of kvm_arch_vcpu_load and kvm_arch_vcpu_put. */ static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt) @@ -90,14 +94,11 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt) void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt) { __sysreg_save_common_state(ctxt); - __sysreg_save_user_state(ctxt); } void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt) { - __sysreg_save_el1_state(ctxt); __sysreg_save_common_state(ctxt); - __sysreg_save_user_state(ctxt); __sysreg_save_el2_return_state(ctxt); } @@ -163,14 +164,11 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt) void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt) { __sysreg_restore_common_state(ctxt); - __sysreg_restore_user_state(ctxt); } void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt) { - __sysreg_restore_el1_state(ctxt); __sysreg_restore_common_state(ctxt); - __sysreg_restore_user_state(ctxt); __sysreg_restore_el2_return_state(ctxt); } @@ -236,6 +234,18 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu) */ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) { + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context; + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt; + + if (!has_vhe()) + return; + + __sysreg_save_user_state(host_ctxt); + + __sysreg_restore_user_state(guest_ctxt); + __sysreg_restore_el1_state(guest_ctxt); + + vcpu->arch.sysregs_loaded_on_cpu = true; } /** @@ -264,6 +274,17 @@ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs); vcpu->arch.guest_vfp_loaded = 0; } + + if (!has_vhe()) + return; + + __sysreg_save_el1_state(guest_ctxt); + __sysreg_save_user_state(guest_ctxt); + + /* Restore host user state */ + __sysreg_restore_user_state(host_ctxt); + + vcpu->arch.sysregs_loaded_on_cpu = false; } void __hyp_text __kvm_set_tpidr_el2(u64 tpidr_el2) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 9d353a6a55c9..8df651a8a36c 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -140,26 +140,26 @@ static void __default_write_sys_reg(struct kvm_vcpu *vcpu, int reg, u64 val) /* Ordered as in enum vcpu_sysreg */ DECLARE_IMMEDIATE_SR(MPIDR_EL1); -DECLARE_IMMEDIATE_SR(CSSELR_EL1); -DECLARE_IMMEDIATE_SR(SCTLR_EL1); -DECLARE_IMMEDIATE_SR(ACTLR_EL1); -DECLARE_IMMEDIATE_SR(CPACR_EL1); -DECLARE_IMMEDIATE_SR(TTBR0_EL1); -DECLARE_IMMEDIATE_SR(TTBR1_EL1); -DECLARE_IMMEDIATE_SR(TCR_EL1); -DECLARE_IMMEDIATE_SR(ESR_EL1); -DECLARE_IMMEDIATE_SR(AFSR0_EL1); -DECLARE_IMMEDIATE_SR(AFSR1_EL1); -DECLARE_IMMEDIATE_SR(FAR_EL1); -DECLARE_IMMEDIATE_SR(MAIR_EL1); -DECLARE_IMMEDIATE_SR(VBAR_EL1); -DECLARE_IMMEDIATE_SR(CONTEXTIDR_EL1); -DECLARE_IMMEDIATE_SR(TPIDR_EL0); -DECLARE_IMMEDIATE_SR(TPIDRRO_EL0); -DECLARE_IMMEDIATE_SR(TPIDR_EL1); -DECLARE_IMMEDIATE_SR(AMAIR_EL1); -DECLARE_IMMEDIATE_SR(CNTKCTL_EL1); -DECLARE_IMMEDIATE_SR(PAR_EL1); +DECLARE_DEFERRABLE_SR(CSSELR_EL1, SYS_CSSELR_EL1); +DECLARE_DEFERRABLE_SR(SCTLR_EL1, sctlr_EL12); +DECLARE_DEFERRABLE_SR(ACTLR_EL1, SYS_ACTLR_EL1); +DECLARE_DEFERRABLE_SR(CPACR_EL1, cpacr_EL12); +DECLARE_DEFERRABLE_SR(TTBR0_EL1, ttbr0_EL12); +DECLARE_DEFERRABLE_SR(TTBR1_EL1, ttbr1_EL12); +DECLARE_DEFERRABLE_SR(TCR_EL1, tcr_EL12); +DECLARE_DEFERRABLE_SR(ESR_EL1, esr_EL12); +DECLARE_DEFERRABLE_SR(AFSR0_EL1, afsr0_EL12); +DECLARE_DEFERRABLE_SR(AFSR1_EL1, afsr1_EL12); +DECLARE_DEFERRABLE_SR(FAR_EL1, far_EL12); +DECLARE_DEFERRABLE_SR(MAIR_EL1, mair_EL12); +DECLARE_DEFERRABLE_SR(VBAR_EL1, vbar_EL12); +DECLARE_DEFERRABLE_SR(CONTEXTIDR_EL1, contextidr_EL12); +DECLARE_DEFERRABLE_SR(TPIDR_EL0, SYS_TPIDR_EL0); +DECLARE_DEFERRABLE_SR(TPIDRRO_EL0, SYS_TPIDRRO_EL0); +DECLARE_DEFERRABLE_SR(TPIDR_EL1, SYS_TPIDR_EL1); +DECLARE_DEFERRABLE_SR(AMAIR_EL1, amair_EL12); +DECLARE_DEFERRABLE_SR(CNTKCTL_EL1, cntkctl_EL12); +DECLARE_DEFERRABLE_SR(PAR_EL1, SYS_PAR_EL1); DECLARE_IMMEDIATE_SR(MDSCR_EL1); DECLARE_IMMEDIATE_SR(MDCCINT_EL1); DECLARE_IMMEDIATE_SR(PMCR_EL0);