From patchwork Fri Feb 3 12:33:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 9554045 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1D727602B7 for ; Fri, 3 Feb 2017 12:33:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E20F2823D for ; Fri, 3 Feb 2017 12:33:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 013DF2846D; Fri, 3 Feb 2017 12:33:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,RCVD_IN_DNSWL_HI,RCVD_IN_SORBS_SPAM autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D01572823D for ; Fri, 3 Feb 2017 12:33:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752951AbdBCMdZ (ORCPT ); Fri, 3 Feb 2017 07:33:25 -0500 Received: from mail-wm0-f54.google.com ([74.125.82.54]:36543 "EHLO mail-wm0-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752388AbdBCMdX (ORCPT ); Fri, 3 Feb 2017 07:33:23 -0500 Received: by mail-wm0-f54.google.com with SMTP id c85so26675860wmi.1 for ; Fri, 03 Feb 2017 04:33:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=ah2pOG9Io/KV943zki0aTMamHLtC5IGVe7OV1Xgj9f8=; b=iwrGYdYRxS5WewOAdPebizvuTk9mMH4KEuvjl42ojT/I1Kikb/ZO449YdJxmLeL63N cjEedx978UR/bN2CGAw4HkKOugNHqb519NsiPhXf+HmOlGKt6omsHu7v7vLCZN7aCeHA u9gRHdgfgiP+Y+1ZXf1oSTGhRJU83cD9CopHM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=ah2pOG9Io/KV943zki0aTMamHLtC5IGVe7OV1Xgj9f8=; b=IeFyQFLKa0qLZUzC7Styk5OZFhqjQq7ArfySpSpJbE3vNqb1pv1ezKdDSkzxPO/FjN rGu1gNn58U66nv+FjdC4dqMqMXfhuSMWtb20PWStMFK+3LTQOUWkev7lRqBWNijCye2Z PCUPbhN39yWCnqqDh4KFgWVfp3lCcXolz0eR1OqP9b+/jaHEhetEcwaWiKPvrsD8QJ6e nEBUZ2lQv8hcVtGRdKLwiUKEEPHsM2anK7x1tawpBup50mBlj56uMc/Gb/Me2IFLCGIo DQyfYIqk/oym6S3T6ciibm/Ac8JkJrnuxrTTDLcqDWNCByUY9qBtnT+7AJVjZ4Dcfh18 9e7g== X-Gm-Message-State: AMke39k+PkXNdSaLESRm4jd+RHtNbhPjtBB9FCbYDuf70HU8/wV0ZoeoYqbfD//rlgo6QGX4 X-Received: by 10.28.98.194 with SMTP id w185mr1348634wmb.84.1486125201893; Fri, 03 Feb 2017 04:33:21 -0800 (PST) Received: from localhost ([217.61.220.40]) by smtp.gmail.com with ESMTPSA id a35sm45151940wra.21.2017.02.03.04.33.21 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Fri, 03 Feb 2017 04:33:21 -0800 (PST) Date: Fri, 3 Feb 2017 13:33:20 +0100 From: Christoffer Dall To: Jintack Lim Cc: Paolo Bonzini , Radim =?utf-8?B?S3LEjW3DocWZ?= , Christoffer Dall , Marc Zyngier , linux@armlinux.org.uk, Catalin Marinas , Will Deacon , Andre Przywara , KVM General , arm-mail-list , kvmarm@lists.cs.columbia.edu, lkml - Kernel Mailing List Subject: Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM Message-ID: <20170203123320.GK27852@cbox> References: <1485970990-13775-1-git-send-email-jintack@cs.columbia.edu> <20170202123104.GJ27852@cbox> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote: > On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall wrote: > > Hi Jintack, > > > > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote: > >> The ARM architecture defines the EL1 physical timer and the virtual timer, > >> and it is reasonable for an OS to expect to be able to access both. > >> However, the current KVM implementation does not provide the EL1 physical > >> timer to VMs but terminates VMs on access to the timer. > >> > >> This patch series enables VMs to use the EL1 physical timer through > >> trap-and-emulate. The KVM host emulates each EL1 physical timer register > >> access and sets up the background timer accordingly. When the background > >> timer expires, the KVM host injects EL1 physical timer interrupts to the > >> VM. Alternatively, it's also possible to allow VMs to access the EL1 > >> physical timer without trapping. However, this requires somehow using the > >> EL2 physical timer for the Linux host while running the VM instead of the > >> EL1 physical timer. Right now I just implemented trap-and-emulate because > >> this was straightforward to do, and I leave it to future work to determine > >> if transferring the EL1 physical timer state to the EL2 timer provides any > >> performance benefit. > >> > >> This feature will be useful for any OS that wishes to access the EL1 > >> physical timer. Nested virtualization is one of those use cases. A nested > >> hypervisor running inside a VM would think it has full access to the > >> hardware and naturally tries to use the EL1 physical timer as Linux would > >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen > >> would do, but supporting the EL2 physical timer to the VM is out of scope > >> of this patch series. This patch series will make it easy to add the EL2 > >> timer support in the future, though. > >> > >> Note that Linux VMs booting in EL1 will be unaffected by this patch series > >> and will continue to use only the virtual timer and this patch series will > >> therefore not introduce any performance degredation as a result of > >> trap-and-emulate. > >> > >> v2 => v3: > >> - Rebase on kvmarm/queue > >> - Take kvm->lock to synchronize cntvoff across all vtimers > >> - Remove unnecessary function parameters > >> - Add comments > > > > I just gave v3 a test run on my TC2 (32-bit platform) and my guest > > quickly locks up trying to run cyclictest or when booting the machine it > > stalls with RCU timeouts. > > Ok. It's my fault not to specify that the emulated physical timer is > supported/tested on arm64. > On 32-bit platform, it is supposed to show the same behavior as > before, but I haven't tested. > Were you using the physical timer or the virtual timer for the guest? > > > > > Could you have a look? > > Sure, I'll have a look. I don't have access to my Cubietruck today, > but I can work on that tomorrow. > Don't bother, I've figured this out for you. You need the following fixup to your patch: This is an amuzing one. Thanks, -Christoffer diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c index 93c811c..35d7100 100644 --- a/virt/kvm/arm/arch_timer.c +++ b/virt/kvm/arm/arch_timer.c @@ -410,14 +410,21 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu, } /* Make the updates of cntvoff for all vtimer contexts atomic */ -static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff) +static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff) { int i; - struct kvm_vcpu *vcpu; + struct kvm *kvm = vcpu->kvm; + struct kvm_vcpu *tmp; mutex_lock(&kvm->lock); - kvm_for_each_vcpu(i, vcpu, kvm) - vcpu_vtimer(vcpu)->cntvoff = cntvoff; + kvm_for_each_vcpu(i, tmp, kvm) + vcpu_vtimer(tmp)->cntvoff = cntvoff; + + /* + * When called from the vcpu create path, the CPU being created is not + * included in the loop above, so we just set it here as well. + */ + vcpu_vtimer(vcpu)->cntvoff = cntvoff; mutex_unlock(&kvm->lock); } @@ -426,7 +433,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu) struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu; /* Synchronize cntvoff across all vtimers of a VM. */ - update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read()); + update_vtimer_cntvoff(vcpu, kvm_phys_timer_read()); vcpu_ptimer(vcpu)->cntvoff = 0; INIT_WORK(&timer->expired, kvm_timer_inject_irq_work); @@ -448,7 +455,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value) vtimer->cnt_ctl = value; break; case KVM_REG_ARM_TIMER_CNT: - update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value); + update_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value); break; case KVM_REG_ARM_TIMER_CVAL: vtimer->cnt_cval = value;