From patchwork Tue Jun 8 15:48:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 934E0C4743F for ; Tue, 8 Jun 2021 15:55:50 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 659CC6134F for ; Tue, 8 Jun 2021 15:55:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 659CC6134F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YrrMgfX17lyE7nGAZKmo0Lg331oBTxQLqr/fFEsZx0E=; b=gBtvNpHf01xlK1 R+9nBsha5s9obRXPI5SnSyzib8WcYksuwZ0ReifWPIwoRuFAAHhgD91z8bOvjBDqeQG0KZZEhswa/ aZYGaIn1BWX/vrIGkM6kOY3U6pB98cREAEi4P4+yVpv5C6b/w9AvpM/cw/oV7PIF5rpg7ciF+WuOJ 82l5bz5OfiY0tW43shgVeZmXxxFsstEVIo2bHKGKicvSCXi9uFM1CTYag3U0Qzok52Dbg8543zuEh Q3YvYq3tjoAWug8jEFakncNfewCZHRkiWwFTDR+Zf8OzFa8DzR4Kvi7ZkuOnpZ/8kZemulxgwlM9x rkAaEnuV0/laUjIgDGdA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe3P-009O9E-AG; Tue, 08 Jun 2021 15:54:11 +0000 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe3K-009O7F-23 for linux-arm-kernel@lists.infradead.org; Tue, 08 Jun 2021 15:54:07 +0000 Received: by mail-wr1-x433.google.com with SMTP id r9so5434938wrz.10 for ; Tue, 08 Jun 2021 08:54:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=luZ2XBy4kIjKsUNN5YLr1hr2hYOSweOt5shsxweZY5c=; b=sbCxjbeI/oicVowr1ScpDXLBsVThc8WALV+maymt0+dWYJi+SdK1EjS3CBamvobctn oVj9jPoiFl+XFyqVbz7oy3/wgxvfgv4TUhXNsZtIJYnlHpvoajG5yUBb4YDZeadnQyZ6 FVMM9S8Byjv7mzoU+5SISyAl31bOpdFIzZ//vvWfgeVaD+uvShhJnQfPo+J+Wj1q9caq U3Hk1jw15VgUv/U1tleMfSt+xrJk0XKUTYZF7Qb+jytlId0YYKfFcJnS///J517/MU38 hcdfZin3+XvAGgVk0iRCkX8FCL1xBqvocAV1qiayCXWYGUiFRbLXitgw6SGtGQZndgP9 UYFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=luZ2XBy4kIjKsUNN5YLr1hr2hYOSweOt5shsxweZY5c=; b=Kufe9bDia1DvGkMb6F9+Sce4+iSCYHTcbkPMRyOpBgUaXHzeWKd9g73oPWxHBC8L7K hZEXy4p3o2v8JR0t5pZy4M0GsqFoearkx2VFpM8jyYvdwKyqiP7iDo8m/yExulVXpsqU xuWicRrFiYXTZeNVQl60aAXVcHgWajkJ5gVi+LqsXyW3tI4wK7VbmAcHYvsXU1q0QIq2 29FwRchrSQlKDRd9d730cP8gDk1TTKeFJ4KbQLOtRtEASZER7hIrsjWq8DhCdKlyfpR2 ywbseSba2abb27s2wA6ZcmYiCI+OhMmTP8tMQNmkVydnnGdGnL+scCmZzhzxo5NfH7kQ iMdg== X-Gm-Message-State: AOAM530HcWwgM/u9Ml3gxH3xYu00Mdvwvph2lZA6GZ6Prl6rLjGiL97T hIfyWzx82Tp9uMWrk1ZUkQJTmQ== X-Google-Smtp-Source: ABdhPJzkPP5/N5I0QEPYl+or03a+RJK+q3fbfrxy8aomfkBbcwKtQQDp8ZjdsnAAKwnqh5+OZC4YyA== X-Received: by 2002:a5d:4a4b:: with SMTP id v11mr23583559wrs.246.1623167642648; Tue, 08 Jun 2021 08:54:02 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:02 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 2/5] KVM: arm64: Move WFI execution to check_vcpu_requests() Date: Tue, 8 Jun 2021 17:48:03 +0200 Message-Id: <20210608154805.216869-3-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_085406_125936_264F661B X-CRM114-Status: GOOD ( 21.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Prepare for WFI requests from userspace, by adding a suspend request and moving the WFI execution into check_vcpu_requests(), next to the power-off logic. vcpu->arch.mp_state, previously only RUNNABLE or STOPPED, supports an additional state HALTED and two new state transitions: RUNNABLE -> HALTED from WFI or PSCI CPU_SUSPEND (same vCPU) HALTED -> RUNNABLE vGIC IRQ, pending timer, signal There shouldn't be any functional change with this patch, even though the KVM_GET_MP_STATE ioctl could now in theory return KVM_MP_STATE_HALTED, which would break some users' mp_state support. In practice it should not happen because we do not return to userspace with HALTED state. Both WFI and PSCI CPU_SUSPEND stay in the vCPU run loop until the suspend request is consumed. It does feel fragile though, maybe we should explicitly return RUNNABLE in KVM_GET_MP_STATE in place of HALTED, to prevent future breakage. Signed-off-by: Jean-Philippe Brucker --- arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/kvm/arm.c | 18 ++++++++++++++- arch/arm64/kvm/handle_exit.c | 3 +-- arch/arm64/kvm/psci.c | 37 +++++++++++++------------------ 4 files changed, 35 insertions(+), 25 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 55a04f4d5919..3ca732feb9a5 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -46,6 +46,7 @@ #define KVM_REQ_VCPU_RESET KVM_ARCH_REQ(2) #define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3) #define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4) +#define KVM_REQ_SUSPEND KVM_ARCH_REQ(5) #define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \ KVM_DIRTY_LOG_INITIALLY_SET) @@ -722,6 +723,7 @@ int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr); void kvm_arm_vcpu_power_off(struct kvm_vcpu *vcpu); bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu); +void kvm_arm_vcpu_suspend(struct kvm_vcpu *vcpu); /* Guest/host FPSIMD coordination helpers */ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index bcc24adb9c0a..d8cbaa0373c7 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -447,6 +447,12 @@ bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu) return vcpu->arch.mp_state == KVM_MP_STATE_STOPPED; } +void kvm_arm_vcpu_suspend(struct kvm_vcpu *vcpu) +{ + vcpu->arch.mp_state = KVM_MP_STATE_HALTED; + kvm_make_request(KVM_REQ_SUSPEND, vcpu); +} + int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, struct kvm_mp_state *mp_state) { @@ -667,6 +673,8 @@ static int kvm_vcpu_initialized(struct kvm_vcpu *vcpu) static void check_vcpu_requests(struct kvm_vcpu *vcpu) { + bool irq_pending; + if (kvm_request_pending(vcpu)) { if (kvm_check_request(KVM_REQ_SLEEP, vcpu)) vcpu_req_sleep(vcpu); @@ -678,7 +686,7 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu) * Clear IRQ_PENDING requests that were made to guarantee * that a VCPU sees new virtual interrupts. */ - kvm_check_request(KVM_REQ_IRQ_PENDING, vcpu); + irq_pending = kvm_check_request(KVM_REQ_IRQ_PENDING, vcpu); if (kvm_check_request(KVM_REQ_RECORD_STEAL, vcpu)) kvm_update_stolen_time(vcpu); @@ -690,6 +698,14 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu) vgic_v4_load(vcpu); preempt_enable(); } + + if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) { + if (!irq_pending) { + kvm_vcpu_block(vcpu); + kvm_clear_request(KVM_REQ_UNHALT, vcpu); + } + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; + } } } diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 6f48336b1d86..9717df3104cf 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -95,8 +95,7 @@ static int kvm_handle_wfx(struct kvm_vcpu *vcpu) } else { trace_kvm_wfx_arm64(*vcpu_pc(vcpu), false); vcpu->stat.wfi_exit_stat++; - kvm_vcpu_block(vcpu); - kvm_clear_request(KVM_REQ_UNHALT, vcpu); + kvm_arm_vcpu_suspend(vcpu); } kvm_incr_pc(vcpu); diff --git a/arch/arm64/kvm/psci.c b/arch/arm64/kvm/psci.c index 24b4a2265dbd..42a307ceb95f 100644 --- a/arch/arm64/kvm/psci.c +++ b/arch/arm64/kvm/psci.c @@ -31,27 +31,6 @@ static unsigned long psci_affinity_mask(unsigned long affinity_level) return 0; } -static unsigned long kvm_psci_vcpu_suspend(struct kvm_vcpu *vcpu) -{ - /* - * NOTE: For simplicity, we make VCPU suspend emulation to be - * same-as WFI (Wait-for-interrupt) emulation. - * - * This means for KVM the wakeup events are interrupts and - * this is consistent with intended use of StateID as described - * in section 5.4.1 of PSCI v0.2 specification (ARM DEN 0022A). - * - * Further, we also treat power-down request to be same as - * stand-by request as-per section 5.4.2 clause 3 of PSCI v0.2 - * specification (ARM DEN 0022A). This means all suspend states - * for KVM will preserve the register state. - */ - kvm_vcpu_block(vcpu); - kvm_clear_request(KVM_REQ_UNHALT, vcpu); - - return PSCI_RET_SUCCESS; -} - static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) { struct vcpu_reset_state *reset_state; @@ -227,7 +206,21 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) break; case PSCI_0_2_FN_CPU_SUSPEND: case PSCI_0_2_FN64_CPU_SUSPEND: - val = kvm_psci_vcpu_suspend(vcpu); + /* + * NOTE: For simplicity, we make VCPU suspend emulation to be + * same-as WFI (Wait-for-interrupt) emulation. + * + * This means for KVM the wakeup events are interrupts and this + * is consistent with intended use of StateID as described in + * section 5.4.1 of PSCI v0.2 specification (ARM DEN 0022A). + * + * Further, we also treat power-down request to be same as + * stand-by request as-per section 5.4.2 clause 3 of PSCI v0.2 + * specification (ARM DEN 0022A). This means all suspend states + * for KVM will preserve the register state. + */ + kvm_arm_vcpu_suspend(vcpu); + val = PSCI_RET_SUCCESS; break; case PSCI_0_2_FN_CPU_OFF: kvm_arm_vcpu_power_off(vcpu);