From patchwork Thu Sep 16 18:15:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12499865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23D42C433EF for ; Thu, 16 Sep 2021 18:17:16 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D95E4611CA for ; Thu, 16 Sep 2021 18:17:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D95E4611CA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Mime-Version: Message-Id:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=cXmFgmaAJAb82BOPalIJtqVNYJ165HSCsMziWzcKNZQ=; b=rQF KOC8XNhIotd5pm4keZmyWEedCulvBQwVKjcp/DMbyceg+JmaHQKFcbvuWEs9rToTc8hXsH0jB5iMv dPTq8zESbbwj18peFnw31EfXhM2JDxkM2LMBDgy3QzBHBpyKw+qpM7tqIPlqHXcva/OFDxkZv+WaX 9CoEaEpwin6ioGv5dfMD+Laz2448YQlNiajsjRwBr48/nwrQa8oqkmP49rRBsAL0f3CYrXEOdxX0d v2rIh+QYVMNp/emMRyUFaVLI5IDlr6YRjGlrIbBzuCzrp48tXq7jCMjCrr1sN8GxCw/Bz8YX+2uh0 atrXGsV5OiTQnXAsjtrqWWOArxKlTTg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mQvvC-00C1wE-V6; Thu, 16 Sep 2021 18:15:43 +0000 Received: from mail-io1-xd4a.google.com ([2607:f8b0:4864:20::d4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mQvux-00C1sV-TX for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 18:15:29 +0000 Received: by mail-io1-xd4a.google.com with SMTP id z26-20020a05660200da00b005b86e36a1f4so13753241ioe.15 for ; Thu, 16 Sep 2021 11:15:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=btDL3L/+bb3DbBRXEgGoHsE8Zt71CBQL97fPH4EXk/A=; b=pH3ZPSHyuyqIdw7sVb6pv1UZzHZwd5tpk6oKk5pjSrUAGZjaI6gPpcnwmDwWIogj5j zj/3RnzxsG3WAEUhnmKI2g0W+tsSXBeox49VmFI5O+cpKSsRpG/6GdcRVsEh7H69XLxb XOCV8gStUGRy/QembchUDT3ZrwTt9voEZqdtsOVVh1EdYKxtygL1sfUnFlqfSlCayr3t HbiYWAEvdfR74JBhPty46cWzByaiwZHagXS7CSXSfkcVw7Uj5nHRYsUlPFFfU0JkmTYg HtQtxiOVYuCMtNh4+rjvCaFJCVWe0PvSpdy9xi6x/3YSDDF8pGUZYGKCaMNPV/IUnAkq mkcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=btDL3L/+bb3DbBRXEgGoHsE8Zt71CBQL97fPH4EXk/A=; b=xyezf5l7xiPDr37DbAKQBaVNMrDJUaB+K4Bsk8k4zQhNfoaW+Yqft22xcLbsFTlS3k DnJTDIiC9ViC/TGzUXd/McQJ34nSpOKKNmdHW4d5WW7I+mdc7Ed307vken/qIGpFiz6b JsRjhBdqFv2nyU5rzo/QZ7eKivzLo6GV75fI0hl349yvqAHqYbLGJc0sUrZnkB2chjMl eUfweWoFc3fEQfDzfYMNSXgfduHKQuctxuVT/cqTrh1BgIkVbU8sqpYZqHjsi7e48NL0 oGjisPPVnCZGRFlIpX9DCBL1pM3NhFKFxjiEKfe/sX0NHpGPH7cVvGkOFwWXpdaPj9Mm j++Q== X-Gm-Message-State: AOAM532xF2a4wM7iL0VoXjRIqbEXukj85Y47lWInwY0QXGoH1r1Etb1v hxFRyY2QY0jgneg/GnkuADxNWHbS/hI= X-Google-Smtp-Source: ABdhPJxFWTKFCHnmW32IKSDzUpsnORPNcQrlWReCXvncQJer79Nhv+dt9vMYUea9GMm6JAvZIYiN8o71iQI= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a92:90a:: with SMTP id y10mr5002860ilg.108.1631816123285; Thu, 16 Sep 2021 11:15:23 -0700 (PDT) Date: Thu, 16 Sep 2021 18:15:02 +0000 Message-Id: <20210916181510.963449-1-oupton@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog Subject: [PATCH v8 0/8] KVM: arm64: Add idempotent controls to migrate guest counter From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, Andrew Jones , Will Deacon , Catalin Marinas , Oliver Upton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_111528_039024_2E2CA8B0 X-CRM114-Status: GOOD ( 17.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, on KVM/arm64, we only allow a VMM to migrate the guest's virtual counter by-value. Saving and restoring the counter by value is problematic in the fact that the recorded state is not idempotent. Furthermore, we obfuscate from userspace the fact that the architecture actually provides offset-based controls. Another issue is that KVM/arm64 doesn't provide userspace with the controls of the physical counter-timer. This series aims to address both issues by adding offset-based controls for the virtual and physical counters. Patches 1-2 are refactor changes required to provide offset controls to userspace and putting in some generic plumbing to use for both physical and virtual offsets. Patch 3 is a minor refactor, creating a helper function to get the number of timer registers for a particular vCPU. Patch 4 exposes a vCPU's virtual offset through the KVM_*_ONE_REG ioctls. When NV support is added to KVM, CNTVOFF_EL2 will be considered a guest system register. So, it is safe to expose it now through that ioctl. Patch 5 adds a cpufeature bit to detect 'full' ECV implementations, providing EL2 with the ability to offset the physical counter-timer. Patch 6 exposes a vCPU's physical offset as a vCPU device attribute. This is deliberate, as the attribute is not architectural; KVM uses this attribute to track the host<->guest offset. Patch 7 is a prepatory change for the sake of physical offset emulation, as counter-timer traps must be configured separately for each vCPU. Patch 8 allows non-ECV hosts to support the physical offset vCPU device attribute, by trapping and emulating the physical counter registers. This series was tested on an Ampere Mt. Jade system (non-ECV, VHE and nVHE). I did not test this on the FVP, as I need to really figure out tooling for it on my workstation. Applies cleanly to v5.15-rc1 v7: http://lore.kernel.org/r/20210816001217.3063400-1-oupton@google.com v7 -> v8: - Only use ECV if !VHE - Only expose CNTVOFF_EL2 register to userspace with opt-in - Refer to the direct_ptimer explicitly Oliver Upton (8): KVM: arm64: Refactor update_vtimer_cntvoff() KVM: arm64: Separate guest/host counter offset values KVM: arm64: Make a helper function to get nr of timer regs KVM: arm64: Allow userspace to configure a vCPU's virtual offset arm64: cpufeature: Enumerate support for FEAT_ECV >= 0x2 KVM: arm64: Allow userspace to configure a guest's counter-timer offset KVM: arm64: Configure timer traps in vcpu_load() for VHE KVM: arm64: Emulate physical counter offsetting on non-ECV systems Documentation/arm64/booting.rst | 7 + Documentation/virt/kvm/api.rst | 23 +++ Documentation/virt/kvm/devices/vcpu.rst | 28 ++++ arch/arm64/include/asm/kvm_host.h | 3 + arch/arm64/include/asm/sysreg.h | 5 + arch/arm64/include/uapi/asm/kvm.h | 2 + arch/arm64/kernel/cpufeature.c | 10 ++ arch/arm64/kvm/arch_timer.c | 196 +++++++++++++++++++++--- arch/arm64/kvm/arm.c | 9 +- arch/arm64/kvm/guest.c | 28 +++- arch/arm64/kvm/hyp/include/hyp/switch.h | 32 ++++ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 11 +- arch/arm64/tools/cpucaps | 1 + include/clocksource/arm_arch_timer.h | 1 + include/kvm/arm_arch_timer.h | 14 +- include/uapi/linux/kvm.h | 1 + 16 files changed, 337 insertions(+), 34 deletions(-)