From patchwork Tue Aug 2 23:07:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mingwei Zhang X-Patchwork-Id: 12935086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64DFEC19F2C for ; Tue, 2 Aug 2022 23:07:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233568AbiHBXH0 (ORCPT ); Tue, 2 Aug 2022 19:07:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233050AbiHBXHZ (ORCPT ); Tue, 2 Aug 2022 19:07:25 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F393B357EF for ; Tue, 2 Aug 2022 16:07:23 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id q12-20020a17090aa00c00b001f228eb8b84so113006pjp.3 for ; Tue, 02 Aug 2022 16:07:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:from:to:cc; bh=wxekZ3ryQMo0zEK6TK2UEFmX6M8kBLcXtxqwenvhf90=; b=LsjTQbUVPs5ep6EeANhLZaoS+w+TGBfHkB0X3+hbHXpwwof50NPm5/gADNmNUPVGry jetfkpeTWJtAJf17ryFm6ijrEGTkAGEyuGbDWLM+Yw4GCOZlsF+enhh3kFIpZ3eG5YrO N8WRxuFAOjYf4gkrw+4AippD/Dr+fWGQFmX4g5vnfSn2RUXciqCq/17CWr1VFMtBLQtF 5vZZ7mXUsOowuCfyC41hylKc9PwdoeV2gUCnKbrPwKSiVEHSgmjSCxRkJoNvTxQiLi1p jrCbI7knR/gafVKVNkQWnXpSzbQS6mWSjK3qN9hM4G14oGIuzS1QbC6wGPhZIHl4aqMC zIZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc; bh=wxekZ3ryQMo0zEK6TK2UEFmX6M8kBLcXtxqwenvhf90=; b=lTPz4LQoCRU0TF+DDMq6bsL4tVt0ghbw+sS7CZTLTvbsQBIHDN0ROpow1spUnz9iNJ GKXdC2sW3o9X2u6gc43/o81q1coEbXr7cHC0Zbua+tVRud1IH5CKvyyEbr0Hs7ubMnvu cpnVjVgzogt6eT9uLBMsnamJ5zRNiiyxIj9r10mz7nICBxpxx1wwJvuBsxzEqo5BKOxa FpX99qKZ+5w6Ae98Qop1kAww5MiyPsuRQc7K0dfAF+nLRPRmcsoYJZjm/4Hr6HrRITVw zoZzDXWCAzEX0f0i7VE0/AYqJQ3MrIbiriQi652lwTbWzwHWYKKBQFaTaxxg7XYsuo0D JbHg== X-Gm-Message-State: ACgBeo22dGP4EQVuJf7t9ofYdYY2r1fYiYFiP/iiS+rkjSa/dwxJI+HA WUPbtIVVeqKKprrQsQ+/icR0I3UbldsU X-Google-Smtp-Source: AA6agR65A5j6bivlk0aePat9wLJNwHNc3cREL7Nm0VWN8wKalvs4bXoMwB3bdunep7rlIGdTrx9S1eAWFa9v X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a17:90b:1bc6:b0:1f5:313a:de4b with SMTP id oa6-20020a17090b1bc600b001f5313ade4bmr1871127pjb.4.1659481643558; Tue, 02 Aug 2022 16:07:23 -0700 (PDT) Reply-To: Mingwei Zhang Date: Tue, 2 Aug 2022 23:07:14 +0000 In-Reply-To: <20220802230718.1891356-1-mizhang@google.com> Message-Id: <20220802230718.1891356-2-mizhang@google.com> Mime-Version: 1.0 References: <20220802230718.1891356-1-mizhang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog Subject: [PATCH 1/5] KVM: x86: Get vmcs12 pages before checking pending interrupts From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Oliver Upton , Mingwei Zhang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Oliver Upton vmx_guest_apic_has_interrupts implicitly depends on the virtual APIC page being present + mapped into the kernel address space. However, with demand paging we break this dependency, as the KVM_REQ_GET_VMCS12_PAGES event isn't assessed before entering vcpu_block. Fix this by getting vmcs12 pages before inspecting the guest's APIC page. Note that upstream does not have this issue, as they will directly get the vmcs12 pages on vmlaunch/vmresume instead of relying on the event request mechanism. However, the upstream approach is problematic, as the vmcs12 pages will not be present if a live migration occurred before checking the virtual APIC page. Signed-off-by: Oliver Upton Signed-off-by: Mingwei Zhang --- arch/x86/kvm/x86.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 5366f884e9a7..1d3d8127aaea 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10599,6 +10599,23 @@ static inline int vcpu_block(struct kvm_vcpu *vcpu) { bool hv_timer; + /* + * We must first get the vmcs12 pages before checking for interrupts + * that might unblock the guest if L1 is using virtual-interrupt + * delivery. + */ + if (kvm_check_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu)) { + /* + * If we have to ask user-space to post-copy a page, + * then we have to keep trying to get all of the + * VMCS12 pages until we succeed. + */ + if (unlikely(!kvm_x86_ops.nested_ops->get_nested_state_pages(vcpu))) { + kvm_make_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu); + return 0; + } + } + if (!kvm_arch_vcpu_runnable(vcpu)) { /* * Switch to the software timer before halt-polling/blocking as From patchwork Tue Aug 2 23:07:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mingwei Zhang X-Patchwork-Id: 12935087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F39C3C00140 for ; Tue, 2 Aug 2022 23:07:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234149AbiHBXH1 (ORCPT ); Tue, 2 Aug 2022 19:07:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233660AbiHBXH0 (ORCPT ); Tue, 2 Aug 2022 19:07:26 -0400 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7092E357D9 for ; Tue, 2 Aug 2022 16:07:25 -0700 (PDT) Received: by mail-pl1-x64a.google.com with SMTP id d10-20020a170903230a00b0016d63e4112bso9586522plh.19 for ; Tue, 02 Aug 2022 16:07:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:from:to:cc; bh=5S5GfKU8cE9TZ3ZcnKyvYT+yBk+CiJ9ehorTUQPSZEs=; b=QOh/W0eFpTk0c/umwo+PMzRBkFjczKJXoIz6nthb4A/+VkLytCm4loft0i+5IH7YR1 yt+P5tSlmDfLKqZQExxwPCIGX0xsHYSCxYUOynqibr3CXnD9aTewZl0urmj+YXrRqqMz to6/NVOwXrBO88mpmYO5cxVgSx7SSIUJy8ci2SuYLZIyrxbAp6qtkettEXf6gWhxgKRz 4C5YbCxXkRwsrTBuelBgkhehrwk9/4qF1UZawB6gRJhZzwQ+8EiJKfnSma+6PZAe3YDA kCFu/J03N7jOVQ3TGauhhwz5tgoH99Hul2avVFPvPzUNgilv3WusAf3eNNiojtaOH3zs LRgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc; bh=5S5GfKU8cE9TZ3ZcnKyvYT+yBk+CiJ9ehorTUQPSZEs=; b=Pna2SFBXEoSy5y/L1Bw5pySO/tiB5GotTPbtMEBwdLGLP+KKIuCaDQVGsihYDvsp3G FvYNFZO1hDQ1pwxp5FoNFRnCnJoF2l85m0Klw9VIdK9U39H2OqIYRET7Io80ahWlFxA3 xjm/5ikB8SnMutsyRzh1sGGQxVqAaVjo2DtZYO0IIHhZ2TZnabmfRERD021mGjpCbANi ygWXWlZx+ZcS6noMpsecftV3zy+NOBd+E5zq7Jn75UQwrZ0MLuq73PbHKipvdkUOk1wr P07HOf4R8qhPn/YROaWw40NyPdKAJcaLkUMex3wZOO2SuYW7QBLrPRUyf/+wNjXcr44l b8/g== X-Gm-Message-State: AJIora9KnF8+QEmOgiaRc7RHGzdliJj7nHmHMQBSU4DMYVNlDalVZiFf EDjtQbWlCjmU+nCOEDptAOZ4QTFkHaFf X-Google-Smtp-Source: AGRyM1vXCBWGjOx32Vj2hcf3hruHK6Ry9qboVB6qc3W51oW0TCNZfcf7MhiIxH1CE/gH8cJdZpK74gGKbTOz X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a05:6a00:2185:b0:520:7276:6570 with SMTP id h5-20020a056a00218500b0052072766570mr23319476pfi.84.1659481645025; Tue, 02 Aug 2022 16:07:25 -0700 (PDT) Reply-To: Mingwei Zhang Date: Tue, 2 Aug 2022 23:07:15 +0000 In-Reply-To: <20220802230718.1891356-1-mizhang@google.com> Message-Id: <20220802230718.1891356-3-mizhang@google.com> Mime-Version: 1.0 References: <20220802230718.1891356-1-mizhang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog Subject: [PATCH 2/5] selftests: KVM/x86: Fix vcpu_{save,load}_state() by adding APIC state into kvm_x86_state From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Oliver Upton , Mingwei Zhang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Fix vcpu_{save,load}_state() by adding APIC state into kvm_x86_state and properly save/restore it in vcpu_{save,load}_state(). When vcpu resets, APIC state become software disabled in kernel and thus the corresponding vCPU is not able to receive posted interrupts [1]. So, add APIC save/restore in userspace in selftest library code. [1] commit 97222cc83163 ("KVM: Emulate local APIC in kernel"). Cc: Jim Mattson Signed-off-by: Mingwei Zhang --- tools/testing/selftests/kvm/include/kvm_util_base.h | 10 ++++++++++ tools/testing/selftests/kvm/include/x86_64/processor.h | 1 + tools/testing/selftests/kvm/lib/x86_64/processor.c | 2 ++ 3 files changed, 13 insertions(+) diff --git a/tools/testing/selftests/kvm/include/kvm_util_base.h b/tools/testing/selftests/kvm/include/kvm_util_base.h index 24fde97f6121..ac883b8eab57 100644 --- a/tools/testing/selftests/kvm/include/kvm_util_base.h +++ b/tools/testing/selftests/kvm/include/kvm_util_base.h @@ -457,6 +457,16 @@ static inline void vcpu_fpu_set(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) vcpu_ioctl(vcpu, KVM_SET_FPU, fpu); } +static inline void vcpu_apic_get(struct kvm_vcpu *vcpu, struct kvm_lapic_state *apic) +{ + vcpu_ioctl(vcpu, KVM_GET_LAPIC, apic); +} + +static inline void vcpu_apic_set(struct kvm_vcpu *vcpu, struct kvm_lapic_state *apic) +{ + vcpu_ioctl(vcpu, KVM_SET_LAPIC, apic); +} + static inline int __vcpu_get_reg(struct kvm_vcpu *vcpu, uint64_t id, void *addr) { struct kvm_one_reg reg = { .id = id, .addr = (uint64_t)addr }; diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h index 45edf45821d0..bf5f874709a4 100644 --- a/tools/testing/selftests/kvm/include/x86_64/processor.h +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h @@ -225,6 +225,7 @@ struct kvm_x86_state { struct kvm_nested_state nested; char nested_[16384]; }; + struct kvm_lapic_state apic; struct kvm_msrs msrs; }; diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c index f35626df1dea..d18da71654b6 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c @@ -980,6 +980,7 @@ struct kvm_x86_state *vcpu_save_state(struct kvm_vcpu *vcpu) vcpu_msrs_get(vcpu, &state->msrs); vcpu_debugregs_get(vcpu, &state->debugregs); + vcpu_apic_get(vcpu, &state->apic); return state; } @@ -997,6 +998,7 @@ void vcpu_load_state(struct kvm_vcpu *vcpu, struct kvm_x86_state *state) vcpu_mp_state_set(vcpu, &state->mp_state); vcpu_debugregs_set(vcpu, &state->debugregs); vcpu_regs_set(vcpu, &state->regs); + vcpu_apic_set(vcpu, &state->apic); if (state->nested.size) vcpu_nested_state_set(vcpu, &state->nested); From patchwork Tue Aug 2 23:07:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mingwei Zhang X-Patchwork-Id: 12935088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75635C19F2D for ; Tue, 2 Aug 2022 23:07:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234299AbiHBXH3 (ORCPT ); Tue, 2 Aug 2022 19:07:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234211AbiHBXH2 (ORCPT ); Tue, 2 Aug 2022 19:07:28 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2A833FA3A for ; Tue, 2 Aug 2022 16:07:26 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id y20-20020a170903011400b0016f06421a83so1713295plc.12 for ; Tue, 02 Aug 2022 16:07:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:from:to:cc; bh=+GFkeGjcI/kh+1Shtb4fxwNaREiAkX51PnmuuTAQK8A=; b=pOA9GnOT+nlyPEzAnIBAEih/PaG2I2w7V5WKhUFU6eQ5B8Fat2GYPJqmqhXHVNcxD6 p5gHVxB9cHa4DgK6uwW1wU7YMKq96dAGSpshNoelsEaVP8QTaC9nbRgsIuedG5zlB7tG FVzSMiSEhcVzUllG6do3/SuOGAgIAjcZtEyV2p0GSZ6K9IEn8eC9Zg7oKYOy14SYdaxE 5rcFsnxepDGJs6twRR4oVInr447mpWXw7X07CRek7XK3CuZBdqxq7nFH6EACkRWmDGiA GfX+aIHWEONmpu/l653qLK1m6bIr3LKBn7nj6H+cyd8EQdIA/Q6pJcXe16m84E59370q XvRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc; bh=+GFkeGjcI/kh+1Shtb4fxwNaREiAkX51PnmuuTAQK8A=; b=xVCxUujl3pB4Jyr1Vy0CkO2ItAfr61djBXTumheJGQ3CuEpBIT/etBkzHfOTCbsj6A VWPjAXLuP/jBQtmu/qarPaDhPcFJbW3BmIaZEHi6aY1Dp3nd7XZIJjPT+Qpr+eLhAXLV 5bM+iQqERS+2VHMtQku/x2Tr9L6yGbLt/mdcJ5M9KGpDP+sSGLkMn9v0hZz+ou5vkMF7 m+dGOxmoqVCLVLiCBfY/BTNZot9CeljkZrsfKfRVvD8dsUZDyt4F8obpPtkD+mVbsu87 d21wjQDBsps1fUGvYEKb/vKsWivv4nm+Bg340vyelnTex3tv3lsO153dCGoDhJwu9vGB X0Mw== X-Gm-Message-State: ACgBeo1ZOP3NbLngR30QqK61X439SvjJiMMFe3FY3HGJ9/9I+PhXBz0x uakGxs6qacIEsPzYK6xbwELtiO3P0E5q X-Google-Smtp-Source: AA6agR7baxuoDiumlwxT8og4KDOAqwSDO43YIdTmYpk4I7uSGbhlRnMu5fP0gocL3tClU2CNLVv8GL0cxZwz X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a17:90b:3e84:b0:1f0:3f92:8c91 with SMTP id rj4-20020a17090b3e8400b001f03f928c91mr1904559pjb.112.1659481646348; Tue, 02 Aug 2022 16:07:26 -0700 (PDT) Reply-To: Mingwei Zhang Date: Tue, 2 Aug 2022 23:07:16 +0000 In-Reply-To: <20220802230718.1891356-1-mizhang@google.com> Message-Id: <20220802230718.1891356-4-mizhang@google.com> Mime-Version: 1.0 References: <20220802230718.1891356-1-mizhang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog Subject: [PATCH 3/5] selftests: KVM: Introduce vcpu_run_interruptable() From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Oliver Upton , Mingwei Zhang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Introduce vcpu_run_interruptable() to allow selftests execute their own code when a vcpu is kicked out of KVM_RUN on receiving a POSIX signal. This function gives selftests the flexibility to execute their own logic especially when a vCPU is halted. Note that vcpu_run_complete_io() almost achieves the same effect. However, it explicitly disallows the case of returning to caller when errno is EINTR. Signed-off-by: Mingwei Zhang --- tools/testing/selftests/kvm/include/kvm_util_base.h | 2 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 11 +++++++++++ 2 files changed, 13 insertions(+) diff --git a/tools/testing/selftests/kvm/include/kvm_util_base.h b/tools/testing/selftests/kvm/include/kvm_util_base.h index ac883b8eab57..cfb91c63d8c3 100644 --- a/tools/testing/selftests/kvm/include/kvm_util_base.h +++ b/tools/testing/selftests/kvm/include/kvm_util_base.h @@ -398,6 +398,8 @@ static inline int __vcpu_run(struct kvm_vcpu *vcpu) return __vcpu_ioctl(vcpu, KVM_RUN, NULL); } +int vcpu_run_interruptable(struct kvm_vcpu *vcpu); + void vcpu_run_complete_io(struct kvm_vcpu *vcpu); struct kvm_reg_list *vcpu_get_reg_list(struct kvm_vcpu *vcpu); diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index 9889fe0d8919..aca418ce4e8c 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -1406,6 +1406,17 @@ void vm_create_irqchip(struct kvm_vm *vm) vm->has_irqchip = true; } +int vcpu_run_interruptable(struct kvm_vcpu *vcpu) +{ + int rc; + + rc = __vcpu_run(vcpu); + + vcpu->run->immediate_exit = 0; + + return rc; +} + int _vcpu_run(struct kvm_vcpu *vcpu) { int rc; From patchwork Tue Aug 2 23:07:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mingwei Zhang X-Patchwork-Id: 12935089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF325C00140 for ; Tue, 2 Aug 2022 23:07:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234895AbiHBXHk (ORCPT ); Tue, 2 Aug 2022 19:07:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234441AbiHBXH3 (ORCPT ); Tue, 2 Aug 2022 19:07:29 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0E0C4BD39 for ; Tue, 2 Aug 2022 16:07:28 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id 17-20020a631751000000b0041c70216fecso967049pgx.22 for ; Tue, 02 Aug 2022 16:07:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:from:to:cc; bh=NxWgyhLS/wEKEZsDnO9RgTHv9lubStrj0Vb693z0PP8=; b=E8gvKaUW/8c0xeISmmUx5b/6KR94RL17k2R745A/8QaEgsEoEyo6BULy+MHp1+Q1Hw 4feqOBmsj4yXeudFu7JS143gFbmxKlBZlF8LZU4mm2G2lLvfEM4MJRzxggTpPbtAEesn 26iizzQTo0Dv4aSY8aRg9/+cBeBvi3eEp2LajGe1eRp7fcOfwAHXjfWGZz1nk5K97iqA NooA5KSHe2Mz7GmTbLiDQ/05jYtf8mkpOcmTM/eNpcq9Q+J2V+KmUu9duJKfnS3OopZa yKxVjeYw57tjJGppY2vuFzW619KIVJRg+7bEWmq/9FMwMG/M24FiqUW4wOb3vLzh+1kn VR7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc; bh=NxWgyhLS/wEKEZsDnO9RgTHv9lubStrj0Vb693z0PP8=; b=bKSBO4q/6MRfZON/1qDnc83HEL7y3iBEFe5JP0OTn1oPQQ1/qn96/9sBRsjTbOQEd8 CTATigONvcQaqC4U9MACyQ/AOaNaynNNNk7dcSduAkv/wkv3ulTJ1HKJSawvM9CRALmI GVOy2E02SEppdpKamwDQ6pmILN2UilFkPuJPKg/BCyw6C7yZaVV0Yo/gih0DhXLciaiK EJu4+28O8kBLnU2GukN8j0nzqNisxIYXZPiZZ4FFQxJUyqdFuw37uDnAYX81fxQGfidX OW3656ospSy8bWAnG5yO09kemBXmQIasmI3dBzzObqcYcGfEe5Fy2eJb4MysOnULrmcA EMcg== X-Gm-Message-State: AJIora96R6yViTGsL1cc5QE2/mLvLpeeEz1iyEmvdDcBSUSAeoOR9Q1v J3aLNQnswWYKjQCAw6vsoLbL1FY6UUMy X-Google-Smtp-Source: AGRyM1syAf/a93tY4akFWCwnvKvWhSMYbJNZ5eFwOPqowcs0YDGMOnDQWpV7Ap88gyeED2TvtVHYITeBtchu X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a05:6a00:1a04:b0:52a:d4dc:5653 with SMTP id g4-20020a056a001a0400b0052ad4dc5653mr23019501pfv.69.1659481648254; Tue, 02 Aug 2022 16:07:28 -0700 (PDT) Reply-To: Mingwei Zhang Date: Tue, 2 Aug 2022 23:07:17 +0000 In-Reply-To: <20220802230718.1891356-1-mizhang@google.com> Message-Id: <20220802230718.1891356-5-mizhang@google.com> Mime-Version: 1.0 References: <20220802230718.1891356-1-mizhang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog Subject: [PATCH 4/5] selftests: KVM: Add support for posted interrupt handling in L2 From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Oliver Upton , Mingwei Zhang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add support for posted interrupt handling in L2. This is done by adding needed data structures in vmx_pages and APIs to allow an L2 receive posted interrupts Cc: Jim Mattson Signed-off-by: Mingwei Zhang --- tools/testing/selftests/kvm/include/x86_64/vmx.h | 10 ++++++++++ tools/testing/selftests/kvm/lib/x86_64/vmx.c | 16 ++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/tools/testing/selftests/kvm/include/x86_64/vmx.h b/tools/testing/selftests/kvm/include/x86_64/vmx.h index 7d8c980317f7..3449ae8ab282 100644 --- a/tools/testing/selftests/kvm/include/x86_64/vmx.h +++ b/tools/testing/selftests/kvm/include/x86_64/vmx.h @@ -579,6 +579,14 @@ struct vmx_pages { void *apic_access_hva; uint64_t apic_access_gpa; void *apic_access; + + void *virtual_apic_hva; + uint64_t virtual_apic_gpa; + void *virtual_apic; + + void *posted_intr_desc_hva; + uint64_t posted_intr_desc_gpa; + void *posted_intr_desc; }; union vmx_basic { @@ -622,5 +630,7 @@ void nested_identity_map_1g(struct vmx_pages *vmx, struct kvm_vm *vm, void prepare_eptp(struct vmx_pages *vmx, struct kvm_vm *vm, uint32_t eptp_memslot); void prepare_virtualize_apic_accesses(struct vmx_pages *vmx, struct kvm_vm *vm); +void prepare_virtual_apic(struct vmx_pages *vmx, struct kvm_vm *vm); +void prepare_posted_intr_desc(struct vmx_pages *vmx, struct kvm_vm *vm); #endif /* SELFTEST_KVM_VMX_H */ diff --git a/tools/testing/selftests/kvm/lib/x86_64/vmx.c b/tools/testing/selftests/kvm/lib/x86_64/vmx.c index 80a568c439b8..7d65bee37b9f 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/vmx.c +++ b/tools/testing/selftests/kvm/lib/x86_64/vmx.c @@ -556,3 +556,19 @@ void prepare_virtualize_apic_accesses(struct vmx_pages *vmx, struct kvm_vm *vm) vmx->apic_access_hva = addr_gva2hva(vm, (uintptr_t)vmx->apic_access); vmx->apic_access_gpa = addr_gva2gpa(vm, (uintptr_t)vmx->apic_access); } + +void prepare_virtual_apic(struct vmx_pages *vmx, struct kvm_vm *vm) +{ + vmx->virtual_apic = (void *)vm_vaddr_alloc_page(vm); + vmx->virtual_apic_hva = addr_gva2hva(vm, (uintptr_t)vmx->virtual_apic); + vmx->virtual_apic_gpa = addr_gva2gpa(vm, (uintptr_t)vmx->virtual_apic); +} + +void prepare_posted_intr_desc(struct vmx_pages *vmx, struct kvm_vm *vm) +{ + vmx->posted_intr_desc = (void *)vm_vaddr_alloc_page(vm); + vmx->posted_intr_desc_hva = + addr_gva2hva(vm, (uintptr_t)vmx->posted_intr_desc); + vmx->posted_intr_desc_gpa = + addr_gva2gpa(vm, (uintptr_t)vmx->posted_intr_desc); +} From patchwork Tue Aug 2 23:07:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mingwei Zhang X-Patchwork-Id: 12935090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F21CC00140 for ; Tue, 2 Aug 2022 23:07:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235193AbiHBXHo (ORCPT ); Tue, 2 Aug 2022 19:07:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234764AbiHBXHh (ORCPT ); Tue, 2 Aug 2022 19:07:37 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF9664F68D for ; Tue, 2 Aug 2022 16:07:30 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id z18-20020a63d012000000b0041b3478e9a9so6074423pgf.17 for ; Tue, 02 Aug 2022 16:07:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:from:to:cc; bh=QkdGTQkLJe04OzhDUqvLuS5XVeRvpbZXCtgImio6O8A=; b=OzSphegZ1ZknlH64i7UfsjWeZJTk9WrqkWEPugROrPfUOzMHLQLm+uQCZmOV9GVR59 tYhyP6MJ5xRrGXCzH5OKV2aikWfzBbdf9luApysy4G4MuorO54u6yVMwgJkfPLYlQLdk oa3Qn1bH24vwK9PsaD/Hbl9nh3MGudIDZDNSjJxiPXq6jWaWujYJLkddCQmYJMs5kbq0 oaFLJw18NwZHg3NQSpI/HREeSRpK7ieF1PDpNMigh1jqpVTo7EfvUq/IgP0GGPNWjO+F HOPOYBnqS/i/TBp5lrGBiDApbmBcwxQwRVuBCXh18CFdxTdTnk6r7bIxTBfhdKFCHC+s 1s5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc; bh=QkdGTQkLJe04OzhDUqvLuS5XVeRvpbZXCtgImio6O8A=; b=yxrAh5SL2FsJShJy9fmWXR+E2cLt37LgBB4ZjOd8pgwK/nio7bkmuxn1tu7D1as5Mt snqwT60dh0tMmKpErxdpa5dQ7uZjSQ8rzoDOcuEoWz7ymA3KSDZD78TCVb3Ap9SCwME7 HdJW0OMhMsRggyvexUGkffd2lVrecGa5eSMqMuy4JmQZ/eWA/zG9JowIVDhTXVl8kW2l DwbA1P4WtjQelRuMDUx/bqhBFXmHToHV15ZQIOuNTzWrkmcxtJIqksQoaU9i7jbDMwB6 lbo+zvmU8w141awbz/NI6pt52DcZgmnhqzmqvSljivNCPKdnHDNx9ygjs8etsKk1P0PS d9eg== X-Gm-Message-State: ACgBeo0I8GhhtcxG0H/gCl/+Wal6JsziMl4eg7iE4EerRQ8CioO86oKD TCn9TaEjqHFH581gsFeCfYu2tdNmgmfW X-Google-Smtp-Source: AA6agR547QKamGuY0F3IOJpsmXV7IB1tIgc1/YRZNcgkFzsbk9vm+zmpsbwzzwvMO+TFezDI2VsZNLZJNN6L X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a17:90a:249:b0:1e0:a8a3:3c6c with SMTP id t9-20020a17090a024900b001e0a8a33c6cmr162545pje.0.1659481649565; Tue, 02 Aug 2022 16:07:29 -0700 (PDT) Reply-To: Mingwei Zhang Date: Tue, 2 Aug 2022 23:07:18 +0000 In-Reply-To: <20220802230718.1891356-1-mizhang@google.com> Message-Id: <20220802230718.1891356-6-mizhang@google.com> Mime-Version: 1.0 References: <20220802230718.1891356-1-mizhang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog Subject: [PATCH 5/5] selftests: KVM: Test if posted interrupt delivery race with migration From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Oliver Upton , Mingwei Zhang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jim Mattson Test if posted interrupt delivery race with migration. Add a selftest to demonstrate a race condition between migration and posted interrupt for a nested VM. The consequence of this race condition causes the loss of a posted interrupt for a nested vCPU after migration and triggers a warning for unpatched kernel. The selftest demonstrates that if a L2 vCPU is in halted state before migration, then after migration, it is not able to receive a posted interrupt from another vCPU within the same VM. The fundamental problem is deeply buried in the kernel logic where vcpu_block() will directly check vmcs12 related mappings before having a valid vmcs12 ready. Because of that, it fails to process the posted interrupt and triggers the warning in vmx_guest_apic_has_interrupt() static bool vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) { ... if (WARN_ON_ONCE(!is_guest_mode(vcpu)) || !nested_cpu_has_vid(get_vmcs12(vcpu)) || WARN_ON_ONCE(!vmx->nested.virtual_apic_map.gfn)) <= HERE return false; ... } Signed-off-by: Jim Mattson Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../kvm/x86_64/vmx_migrate_pi_pending.c | 289 ++++++++++++++++++ 3 files changed, 291 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore index d625a3f83780..749b2be5b23c 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -51,6 +51,7 @@ /x86_64/vmx_exception_with_invalid_guest_state /x86_64/vmx_invalid_nested_guest_state /x86_64/vmx_msrs_test +/x86_64/vmx_migrate_pi_pending /x86_64/vmx_preemption_timer_test /x86_64/vmx_set_nested_state_test /x86_64/vmx_tsc_adjust_test diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 690b499c3471..2d32416237db 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -109,6 +109,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_exception_with_invalid_guest_state TEST_GEN_PROGS_x86_64 += x86_64/vmx_msrs_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_invalid_nested_guest_state +TEST_GEN_PROGS_x86_64 += x86_64/vmx_migrate_pi_pending TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_nested_tsc_scaling_test diff --git a/tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c b/tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file mode 100644 index 000000000000..f1498621eb9a --- /dev/null +++ b/tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c @@ -0,0 +1,289 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * vmx_migrate_pi_pending + * + * Copyright (C) 2022, Google, LLC. + * + * Deliver a nested posted interrupt between migration and the first + * KVM_RUN on the target. + */ + +#include "test_util.h" +#include "kvm_util.h" +#include "processor.h" +#include "vmx.h" + +#include +#include +#include +#include +#include + +#include "kselftest.h" + +#define VCPU_ID0 0 +#define VCPU_ID1 1 +#define PI_ON_BIT 256 +#define PI_NV 0x42 +#define L2_INTR 0x71 + +enum { + PORT_L0_EXIT = 0x2000, +}; + +/* The virtual machine object. */ +struct vmx_pages *vmx; + +static struct kvm_vm *vm; +static struct kvm_vcpu *vcpu0; +static struct kvm_vcpu *vcpu1; +bool vcpu0_can_run = true; +bool vcpu0_running; +bool pi_executed; +pthread_t pthread_cpu0; +pthread_t pthread_cpu1; + +static void vcpu0_ipi_handler(struct ex_regs *regs) +{ + asm volatile("inb %%dx, %%al" + : : [port] "d" (PORT_L0_EXIT) : "rax"); + asm volatile("vmcall"); +} + +static void l2_vcpu0_guest_code(void) +{ + asm volatile("cli"); + asm volatile("sti; nop; hlt"); +} + +static void l1_vcpu0_guest_code(struct vmx_pages *vmx_pages) +{ +#define L2_GUEST_STACK_SIZE 64 + unsigned long l2_vcpu0_guest_stack[L2_GUEST_STACK_SIZE]; + uint32_t control; + + x2apic_enable(); + + GUEST_ASSERT(prepare_for_vmx_operation(vmx_pages)); + GUEST_ASSERT(load_vmcs(vmx_pages)); + + /* Prepare the VMCS for L2 execution. */ + prepare_vmcs(vmx_pages, l2_vcpu0_guest_code, + &l2_vcpu0_guest_stack[L2_GUEST_STACK_SIZE]); + control = vmreadz(PIN_BASED_VM_EXEC_CONTROL); + control |= (PIN_BASED_EXT_INTR_MASK | + PIN_BASED_POSTED_INTR); + vmwrite(PIN_BASED_VM_EXEC_CONTROL, control); + control = vmreadz(CPU_BASED_VM_EXEC_CONTROL); + control |= (CPU_BASED_TPR_SHADOW | + CPU_BASED_ACTIVATE_SECONDARY_CONTROLS); + vmwrite(CPU_BASED_VM_EXEC_CONTROL, control); + control = vmreadz(SECONDARY_VM_EXEC_CONTROL); + control |= (SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE | + SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY); + vmwrite(SECONDARY_VM_EXEC_CONTROL, control); + control = vmreadz(VM_EXIT_CONTROLS); + control |= VM_EXIT_ACK_INTR_ON_EXIT; + vmwrite(VM_EXIT_CONTROLS, control); + vmwrite(VIRTUAL_APIC_PAGE_ADDR, vmx_pages->virtual_apic_gpa); + vmwrite(POSTED_INTR_DESC_ADDR, vmx_pages->posted_intr_desc_gpa); + vmwrite(POSTED_INTR_NV, PI_NV); + + GUEST_ASSERT(!vmlaunch()); + GUEST_ASSERT(vmreadz(VM_EXIT_REASON) == EXIT_REASON_VMCALL); + GUEST_ASSERT(!test_bit(PI_ON_BIT, (void *)vmx_pages->posted_intr_desc)); + GUEST_DONE(); +} + +static void post_intr(u8 vector, void *pi_desc) +{ + set_bit(vector, pi_desc); + set_bit(PI_ON_BIT, pi_desc); +} + +static void l1_vcpu1_guest_code(void *vcpu0_pi_desc) +{ + post_intr(L2_INTR, vcpu0_pi_desc); + x2apic_enable(); + x2apic_write_reg(APIC_ICR, ((u64)VCPU_ID0 << 32) | + APIC_DEST_PHYSICAL | APIC_DM_FIXED | PI_NV); + GUEST_DONE(); +} + +static void save_restore_vm(struct kvm_vm *vm) +{ + struct kvm_regs regs1 = {}, regs2 = {}; + struct kvm_x86_state *state; + + state = vcpu_save_state(vcpu0); + vcpu_regs_get(vcpu0, ®s1); + + kvm_vm_release(vm); + + /* Restore state in a new VM. */ + vcpu0 = vm_recreate_with_one_vcpu(vm); + vcpu_load_state(vcpu0, state); + kvm_x86_state_cleanup(state); + + vcpu_regs_get(vcpu0, ®s2); + TEST_ASSERT(!memcmp(®s1, ®s2, sizeof(regs2)), + "vcpu0: Unexpected register values after vcpu_load_state; rdi: %lx rsi: %lx", + (ulong) regs2.rdi, (ulong) regs2.rsi); +} + +void *create_and_run_vcpu1(void *arg) +{ + struct ucall uc; + struct kvm_run *run; + struct kvm_mp_state vcpu0_mp_state; + + pthread_cpu1 = pthread_self(); + + /* Keep trying to kick out vcpu0 until it is in halted state. */ + for (;;) { + WRITE_ONCE(vcpu0_can_run, true); + sleep(0.1); + WRITE_ONCE(vcpu0_can_run, false); + pthread_kill(pthread_cpu0, SIGUSR1); + printf("vcpu1: Sent SIGUSR1 to vcpu0\n"); + + while (READ_ONCE(vcpu0_running)) + ; + + vcpu_mp_state_get(vcpu0, &vcpu0_mp_state); + if (vcpu0_mp_state.mp_state == KVM_MP_STATE_HALTED) + break; + } + + printf("vcpu1: Kicked out vcpu0 and ensure vcpu0 is halted\n"); + + /* Use save_restore_vm() to simulate a VM migration. */ + save_restore_vm(vm); + + printf("vcpu1: Finished save and restore vm.\n"); + vcpu1 = vm_vcpu_add(vm, VCPU_ID1, l1_vcpu1_guest_code); + vcpu_args_set(vcpu1, 1, vmx->posted_intr_desc); + + /* Start an L1 in vcpu1 and send a posted interrupt to halted L2 in vcpu0. */ + for (;;) { + run = vcpu1->run; + vcpu_run(vcpu1); + + TEST_ASSERT(run->exit_reason == KVM_EXIT_IO, + "vcpu1: Got exit_reason other than KVM_EXIT_IO: %u (%s)\n", + run->exit_reason, + exit_reason_str(run->exit_reason)); + + switch (get_ucall(vcpu1, &uc)) { + case UCALL_ABORT: + TEST_FAIL("%s", (const char *)uc.args[0]); + /* NOT REACHED */ + case UCALL_DONE: + printf("vcpu1: Successfully send a posted interrupt to vcpu0\n"); + goto done; + default: + TEST_FAIL("vcpu1: Unknown ucall %lu", uc.cmd); + } + } + +done: + /* + * Allow vcpu0 resume execution from L0 userspace and check if the + * posted interrupt get executed. + */ + WRITE_ONCE(vcpu0_can_run, true); + sleep(1); + TEST_ASSERT(READ_ONCE(pi_executed), + "vcpu0 did not execute the posted interrupt.\n"); + + return NULL; +} + +void sig_handler(int signum, siginfo_t *info, void *context) +{ + TEST_ASSERT(pthread_self() == pthread_cpu0, + "Incorrect receiver of the signal, expect pthread_cpu0: " + "%lu, but get: %lu\n", pthread_cpu0, pthread_self()); + printf("vcpu0: Execute sighandler for signal: %d\n", signum); +} + +int main(int argc, char *argv[]) +{ + vm_vaddr_t vmx_pages_gva; + struct sigaction sig_action; + struct sigaction old_action; + + memset(&sig_action, 0, sizeof(sig_action)); + sig_action.sa_sigaction = sig_handler; + sig_action.sa_flags = SA_RESTART | SA_SIGINFO; + sigemptyset(&sig_action.sa_mask); + sigaction(SIGUSR1, &sig_action, &old_action); + + pthread_cpu0 = pthread_self(); + printf("vcpu0: Finish setup signal handler for SIGUSR1\n"); + + TEST_REQUIRE(kvm_has_cap(KVM_CAP_NESTED_STATE)); + TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_VMX)); + + vm = vm_create_with_one_vcpu(&vcpu0, (void *)l1_vcpu0_guest_code); + + vm_init_descriptor_tables(vm); + vcpu_init_descriptor_tables(vcpu0); + vm_install_exception_handler(vm, L2_INTR, vcpu0_ipi_handler); + + /* Allocate VMX pages and shared descriptors (vmx_pages). */ + vmx = vcpu_alloc_vmx(vm, &vmx_pages_gva); + prepare_virtual_apic(vmx, vm); + prepare_posted_intr_desc(vmx, vm); + vcpu_args_set(vcpu0, 1, vmx_pages_gva); + + pthread_create(&pthread_cpu1, NULL, create_and_run_vcpu1, NULL); + + for (;;) { + struct kvm_run *run = vcpu0->run; + struct ucall uc; + int rc; + + while (!READ_ONCE(vcpu0_can_run)) + ; + + WRITE_ONCE(vcpu0_running, true); + + rc = vcpu_run_interruptable(vcpu0); + + /* + * When vCPU is kicked out by a signal, ensure a consistent vCPU + * state to prepare for migration before setting the + * vcpu_running flag to false. + */ + if (rc == -1 && run->exit_reason == KVM_EXIT_INTR) { + vcpu_run_complete_io(vcpu0); + + WRITE_ONCE(vcpu0_running, false); + + continue; + } + + WRITE_ONCE(vcpu0_running, false); + + if (run->io.port == PORT_L0_EXIT) { + printf("vcpu0: Executed the posted interrupt\n"); + WRITE_ONCE(pi_executed, true); + continue; + } + + switch (get_ucall(vcpu0, &uc)) { + case UCALL_ABORT: + TEST_FAIL("%s", (const char *)uc.args[0]); + /* NOT REACHED */ + case UCALL_DONE: + goto done; + default: + TEST_FAIL("vcpu0: Unknown ucall %lu", uc.cmd); + } + } + +done: + kvm_vm_free(vm); + return 0; +}