From patchwork Mon Aug 16 00:12:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12437511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A636AC4338F for ; Mon, 16 Aug 2021 00:16:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73EE961151 for ; Mon, 16 Aug 2021 00:16:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 73EE961151 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Mime-Version: Message-Id:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=GpKWlBLMsbm3YcwE6HxnDEBNLrEyq1cGaEr83Stnnks=; b=UF+ p65Zh4fZEig4r/cEEz5O6+4+DJMDk7wr/7f5iBYRavXu/eeUeaF+iQXQJh76flGb3VBPmV/vs84Op fQkYDUJ8I7GHbByhoElY4LwHzwPLYgFSBMs3dAQdMiI86O1HhV6lIWs9Da2OthIe0QGFO7JqZw8fp 71kQszJJG1m2jJwjJOyPnLAP+6RcjTM3RPGyQP6VgUspu2GOF2pNUgU1cQHXMXG9O6/ZKHYBwVGrA w4tJNZH2A/ZnhmOavT0jkYNw0CqqcgSYyIjR0zCuZ5+U+NppxRKgsKNpxn8h+Qf6sH3QFHL1qpd/P mrBwA2fC7cQYR/0IRKflkTDIV7B3wwQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mFQGi-00FlyL-SK; Mon, 16 Aug 2021 00:14:21 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mFQEo-00FlAV-Oc for linux-arm-kernel@lists.infradead.org; Mon, 16 Aug 2021 00:12:24 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id j21-20020a25d2150000b029057ac4b4e78fso14956836ybg.4 for ; Sun, 15 Aug 2021 17:12:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=5mrMQ5rgKSJDf7VPeRHLHNRCD8GR57cQzcrJ1T4+wG8=; b=lo6ullO6Kp1xDycruAhwYsKGgXPnqzWGDrHQLxnZckr1P61xPB1v7A1arHNGA4MA2f 1/FETdWS8qyfarlClNbsf8207Ps+zEM71kE4Wvu8mHn+/RXW2PIGPYzE7HAlLHCkmZuX 7BEbPqI0WNTpwuOxsjrIOB6oQaOpOPZtoRZvj7+X4FfK5TWmrM7Lxj9/LhvRAxyH9GOK C3QoxNvrgWXEN6Ql3hLHIUwcJrpB8LfE3es4CWc6TQtq8Yu+6PJnlYw7b3NNPF5vfb44 yL84b0/IGjXt4hHj2Gf4J2AfbgiQG4D0L0uC2VdiUsMmGaP4HujAqaXzCmhcsmo3l+dC AlrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=5mrMQ5rgKSJDf7VPeRHLHNRCD8GR57cQzcrJ1T4+wG8=; b=U9+U0Or2ECDpOlOWxXTOI4GkKvI4geLAwSlnjrYFM8jdkhZAbqdv/ykFAdeokTHTpL vJK6mx18HGvQQLw2NUXVAhPHrxs8KS57FEKOmq1A2n0DD9RDU+WS527fV+ReHtryUeE8 9FAS2uVZH+1qSPFcL8cz2DK8ws298E8sj7/j1QVfQz0oHEuJJ5Qn+Vpq8TNULBjOB6oK hSLQCUy4/3nm8HOa0LJRRk2+o5vGTP6wx8QV1Ocb0U/MNp7ChJNs3fZQAUBtMX9gfc/Q U8f29yziF/ogDfiXKsZQ6x0bcMh39CaRSodzv7+/RYqJTIiMdiEBkFrmLlhvD5BBm8ra i2WQ== X-Gm-Message-State: AOAM530CzvCLe+NTScOi6QMtH1mm+RRMJcd/opQVZ7bJvawQjcZgdrTw qWPOYaEtRt0ZSxsGoQHm7QxFG7Xg0e4= X-Google-Smtp-Source: ABdhPJxVRmLt9tUt3gGzv0/4JgNLL1IiSIdGK/E6P34z9I+oWRL6HpOccF2RgtNs7bOl/yPJOoA44nyOzDw= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a25:9241:: with SMTP id e1mr17406096ybo.38.1629072741082; Sun, 15 Aug 2021 17:12:21 -0700 (PDT) Date: Mon, 16 Aug 2021 00:12:10 +0000 Message-Id: <20210816001217.3063400-1-oupton@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.33.0.rc1.237.g0d66db33f3-goog Subject: [PATCH v7 0/7] KVM: arm64: Add idempotent controls to migrate guest counter From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, Andrew Jones , Will Deacon , Catalin Marinas , Oliver Upton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210815_171222_860907_9279C0AF X-CRM114-Status: GOOD ( 18.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, on KVM/arm64, we only allow a VMM to migrate the guest's virtual counter by-value. Saving and restoring the counter by value is problematic in the fact that the recorded state is not idempotent. Furthermore, we obfuscate from userspace the fact that the architecture actually provides offset-based controls. Another issue is that KVM/arm64 doesn't provide userspace with the controls of the physical counter-timer. This series aims to address both issues by adding offset-based controls for the virtual and physical counters. Patches 1-2 are refactor changes required to provide offset controls to userspace and putting in some generic plumbing to use for both physical and virtual offsets. Patch 3 exposes a vCPU's virtual offset through the KVM_*_ONE_REG ioctls. When NV support is added to KVM, CNTVOFF_EL2 will be considered a guest system register. So, it is safe to expose it now through that ioctl. Patch 4 adds a cpufeature bit to detect 'full' ECV implementations, providing EL2 with the ability to offset the physical counter-timer. Patch 5 exposes a vCPU's physical offset as a vCPU device attribute. This is deliberate, as the attribute is not architectural; KVM uses this attribute to track the host<->guest offset. Patch 6 is a prepatory change for the sake of physical offset emulation, as counter-timer traps must be configured separately for each vCPU. Patch 7 allows non-ECV hosts to support the physical offset vCPU device attribute, by trapping and emulating the physical counter registers. This series was tested on an Ampere Mt. Jade system (non-ECV, VHE and nVHE) as well as the ARM Base RevC FVP (ECV, VHE and nVHE). Patches apply to kvmarm/next at the following commit: ae280335cdb5 ("Merge branch kvm-arm64/mmu/el2-tracking into kvmarm-master/next") Selftests for these changes are being mailed as a separate series, since there exist dependencies betwen both x86 and arm64. v6: https://lore.kernel.org/r/20210804085819.846610-1-oupton@google.com v6 -> v7: - Fixed typo in documentation (Marc) - Clean up some unused variables (Drew) - Added trap configuration for ECV+nVHE (Marc) - Documented dependency on SCR_EL3.ECVEn (Marc) - wrap up ptimer_emulation_required() for use in hyp and kernel code (Drew) - check static branch condition first (Drew) - s/cpus_have_const_cap/cpus_have_final_cap/ (Marc) - s/ARM64_ECV/ARM64_HAS_ECV2/ - Emulate CNTPCTSS_EL2 if ECV2 not present (Marc) - Reordered the introduction of some functions to ensure that we don't have unused functions in the middle of the series. - Cleaned up the read side of CNTVOFF_EL2 (from userspace). Don't open-code the answer based on the difference of hardware offsets, just use the guest system register value we stashed on the write side. Oliver Upton (7): KVM: arm64: Refactor update_vtimer_cntvoff() KVM: arm64: Separate guest/host counter offset values KVM: arm64: Allow userspace to configure a vCPU's virtual offset arm64: cpufeature: Enumerate support for FEAT_ECV >= 0x2 KVM: arm64: Allow userspace to configure a guest's counter-timer offset KVM: arm64: Configure timer traps in vcpu_load() for VHE KVM: arm64: Emulate physical counter offsetting on non-ECV systems Documentation/arm64/booting.rst | 7 + Documentation/virt/kvm/api.rst | 10 ++ Documentation/virt/kvm/devices/vcpu.rst | 28 ++++ arch/arm64/include/asm/kvm_asm.h | 2 + arch/arm64/include/asm/sysreg.h | 5 + arch/arm64/include/uapi/asm/kvm.h | 2 + arch/arm64/kernel/cpufeature.c | 10 ++ arch/arm64/kvm/arch_timer.c | 196 +++++++++++++++++++++--- arch/arm64/kvm/arm.c | 4 +- arch/arm64/kvm/guest.c | 6 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 32 ++++ arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 + arch/arm64/kvm/hyp/nvhe/timer-sr.c | 20 ++- arch/arm64/kvm/hyp/vhe/timer-sr.c | 5 + arch/arm64/tools/cpucaps | 1 + include/clocksource/arm_arch_timer.h | 1 + include/kvm/arm_arch_timer.h | 9 +- 17 files changed, 315 insertions(+), 29 deletions(-)