From patchwork Fri Aug 2 19:51:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13751940 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3CB034D8A1 for ; Fri, 2 Aug 2024 19:51:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628287; cv=none; b=eDDqZwRrQK6A9HpvuF6V2vT/7tWROzCNM4NDHPdutmt1CEdxCCg/g/cDCiF1DfT8ZjEy0lukhTUOP79y2HgM4FEh0pxFJEnFVghINZCMBkFBC3wJT+KPYfvehpz8tj85UwEw8PN1Rj3ulR50Nrwo5JrJk44uRkL2GRsyA5GFgxM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628287; c=relaxed/simple; bh=zql+KU31olSPyMQGIOOmlA41hrQI7991hIRTrHnkF5g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uCruVV7DrnwPznyLwk+dbUllvtQeJMyGTc1LIeBnf3rk/f5cPPX45ST5rwt5dgwJxWTMnDSIMOs7YQuxCGjogN35gf5E1etZQEhnwYDYCbrAWc7m3BLPWwQdBS+gERyurATtPjAfUAZoYCzoBbkV7qc9s0/6fsEMhySeINNn2zk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RvfoDdEd; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RvfoDdEd" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e0ba318b1b2so8600275276.0 for ; Fri, 02 Aug 2024 12:51:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722628284; x=1723233084; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=j3MDuI39zu1XA4gRdGvyMxqjMSntoDT90qhT0kUFZqs=; b=RvfoDdEdubGsBZqZvhW7Mwi2ErJo/qjP2Dc430keGlJ4PbjVJ+396w4eXSD62UNmt2 WTJGpuLtEZJcKN0qUZF1J2e3l28yUH9j/TvNSDAtA843Z6W59goEPq8Kl60flLVuXCKW t1vaFZ6RArUF9ojfUtVUaKggx9Wz8T6gERL1oClnZD3FmuF4GED045x7NBdF3Ni07+Xm dUXCF24Jd43HLSqOuX8LS25M1UawDTHoZfSBdDR/Qq5w4et58tv84mPSVfvAzluVBwJx l0TGP1hVNtUJ+F9bC0hU2YVJykcivKxtTKQBDNlwynuYMN+Uv2HgCk0i6gOASzRkU+lu wbNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722628284; x=1723233084; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=j3MDuI39zu1XA4gRdGvyMxqjMSntoDT90qhT0kUFZqs=; b=mEWx1s0F76izPJo/IgOw9DqgNY0p6XgZTKwT1gY2R23CHo6uWp/XY4LK6FA1le0cim 9Txs4poJw2MV7zHHgD2I22QP/sR9NN3phwXcwt8Too8Lxin1bJStVjYtnTi4sPtIBzTG +NFQzjP8/7RxxxkJMAFMfW+wO8eAD/+kYlTh5nrlMBq7294AiOsX75xp7/1CQzilZbk+ w4gRHaFZ55g+vH3OOEj54Tg0u4pFRzIP7LbYavKHnpC11mpSNWqvmCfJFw39pf3cKHHO 9SRJwD2MetfpiAowrSirY79m/UR3CGvc5DUnmN7FA1woaPCaLCF3TeN0FY5hqT52PRPy U9ew== X-Gm-Message-State: AOJu0YwyH/bvPhFf6ORn2zJlvz9pXCPOmARO2QeAX6IB4dZ5+LWHKfhz bGsj2q9uUtKp/LHXB6jCw2YtK71pEBxHyu/VUOBPyl8ZAf1rPzSdcq4eNp5OddXCcG3mxag+Cce Wgw== X-Google-Smtp-Source: AGHT+IHXmtQ4fn/w4A9o1IxjByT19mNXJlLz8OGSPq2+ASik931Ksxoz9M3ugqUoQtYF2Bf10ojDjPLGf1M= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a5b:405:0:b0:e0b:f69b:da30 with SMTP id 3f1490d57ef6-e0bf69be1bdmr634276.9.1722628284272; Fri, 02 Aug 2024 12:51:24 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 2 Aug 2024 12:51:16 -0700 In-Reply-To: <20240802195120.325560-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240802195120.325560-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog Message-ID: <20240802195120.325560-2-seanjc@google.com> Subject: [PATCH 1/5] KVM: x86: Re-enter guest if WRMSR(X2APIC_ICR) fastpath is successful From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Re-enter the guest in the fastpath if WRMSR emulation for x2APIC's ICR is successful, as no additional work is needed, i.e. there is no code unique for WRMSR exits between the fastpath and the "!= EXIT_FASTPATH_NONE" check in __vmx_handle_exit(). Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index af6c8cf6a37a..cf397110953f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2173,7 +2173,7 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) data = kvm_read_edx_eax(vcpu); if (!handle_fastpath_set_x2apic_icr_irqoff(vcpu, data)) { kvm_skip_emulated_instruction(vcpu); - ret = EXIT_FASTPATH_EXIT_HANDLED; + ret = EXIT_FASTPATH_REENTER_GUEST; } break; case MSR_IA32_TSC_DEADLINE: From patchwork Fri Aug 2 19:51:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13751941 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC8DB6F068 for ; Fri, 2 Aug 2024 19:51:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628288; cv=none; b=Iq0NAjIzrBxQIDl2owFCvPHLDiNQt8NxmLClmJrNuZsik19xvRevgLn75VSgNCgTx5eqA0gQLy2Z7J1tHk/7hMMah0PE3ZPggL33QgckcvfNbSxM5d62eRiNOi4TetzAkzikMvSuGV/hBlYpkWraDCcaToB3gn3eJ9L7NwOTITA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628288; c=relaxed/simple; bh=TlVRiUICJX5iI1P2EisP0czzGlPcN/R9pgkv0nTLVyw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=d6ZQljIJEiMkFjG/C6maW03jBCpHOCMyEmbGH+1SSlCCoq9cPuBBM6BV/wqK+wgC9pUQSI7+nP8KCttEboS9C9u6USQS8aKCBvxC6957/JK8YYDTwqFNJHa7kP29ypy9RF5UKGluTsP1DrMSfXouqTrCYzqk/qLycO3ozW6RnuM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gRfxlCDL; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gRfxlCDL" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6886cd07673so47268007b3.3 for ; Fri, 02 Aug 2024 12:51:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722628286; x=1723233086; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=iMEFOq5Tk4KyJyaLBMXv5gnfBdOhdNIzblrVnJpsjo0=; b=gRfxlCDL95WQMFsto3I01rjyW+WQ3sYDWiGzxuUpkwFgpmdMEYgqPmYRE8VH7vYuTM Tq26s1pmLTmT8OUfMFkPFRmvu6l4P3ex/mPUobuH2Qnx/SIkWGQ8QztQjJCwNQ4BR0Di k5ybn97zdOfP717Id8ZKa1HsoNnqj32PU5ksWkqJwiZbtftrffS1AnGVksu/hoCaoYaF KZ2Q6xSJfNy9hDnq9FuSMEyPLsEw4gpJB001kcjNl5cgtFRHwuEwHwBmjQT6s4zitsX7 H1YftSItbuxZjBHHIazGIuezsJMoH1tGEyRiZ3TvqVNcjCfVYjQ2kL1422+h/G0sVsrN 5Vrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722628286; x=1723233086; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=iMEFOq5Tk4KyJyaLBMXv5gnfBdOhdNIzblrVnJpsjo0=; b=uA9weqa9lob3YRePxZDNsw4JbYFQ1ceThi6lbl7hVC2q0rvw2xVh6eFfa7v9QaNN2L eI9F3qPFth7Uj3F4/l4k2o/NfOw7wjymOR3Mv1CLPfxGbS4gA+uaE0Je8rWosbB1Boww QOBHi62gEeezaavsET1ayNNWx+lup7X138p9C6BBM9xuUn41ajgTTXiIjDEByA6uPavU SX26Xtt3JsZlR7jxD5OsWCcV5voObbXDsYZ/MlBTHfEr0X58FKp/vxAMPhv9N/M8CDif ZvMSBOfs+IVn7ot/t1BeHrF1Gc21USfuZdKU1Xu+i8MQ1G5x+e2mhUXAUBFpNMzWf5gA 7ULg== X-Gm-Message-State: AOJu0Yz2qEOkI3BTsxyARQ4zg8XJoKNPVaVHq1g5uTV55TM9MqW7KgRv dqpW1c0a/pZHFKhJ7CGVarGNwkEQ7d6SDWuA3B8pCkhWGSUUmxVzhklpCKP87nPqvNhNCoSKmvH dBA== X-Google-Smtp-Source: AGHT+IEMQIhmUdXTQxP5VzE0A3c9BCZw8dM8LR647TTNv36aIGWuOmIV2yy9svYvy4E20onflnwZ7K40qLM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:690c:113:b0:62c:f6fd:5401 with SMTP id 00721157ae682-68963bd9bb6mr1995527b3.6.1722628285936; Fri, 02 Aug 2024 12:51:25 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 2 Aug 2024 12:51:17 -0700 In-Reply-To: <20240802195120.325560-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240802195120.325560-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog Message-ID: <20240802195120.325560-3-seanjc@google.com> Subject: [PATCH 2/5] KVM: x86: Dedup fastpath MSR post-handling logic From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Now that the WRMSR fastpath for x2APIC_ICR and TSC_DEADLINE are identical, ignoring the backend MSR handling, consolidate the common bits of skipping the instruction and setting the return value. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index cf397110953f..332584476129 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2164,31 +2164,32 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) { u32 msr = kvm_rcx_read(vcpu); u64 data; - fastpath_t ret = EXIT_FASTPATH_NONE; + fastpath_t ret; + bool handled; kvm_vcpu_srcu_read_lock(vcpu); switch (msr) { case APIC_BASE_MSR + (APIC_ICR >> 4): data = kvm_read_edx_eax(vcpu); - if (!handle_fastpath_set_x2apic_icr_irqoff(vcpu, data)) { - kvm_skip_emulated_instruction(vcpu); - ret = EXIT_FASTPATH_REENTER_GUEST; - } + handled = !handle_fastpath_set_x2apic_icr_irqoff(vcpu, data); break; case MSR_IA32_TSC_DEADLINE: data = kvm_read_edx_eax(vcpu); - if (!handle_fastpath_set_tscdeadline(vcpu, data)) { - kvm_skip_emulated_instruction(vcpu); - ret = EXIT_FASTPATH_REENTER_GUEST; - } + handled = !handle_fastpath_set_tscdeadline(vcpu, data); break; default: + handled = false; break; } - if (ret != EXIT_FASTPATH_NONE) + if (handled) { + kvm_skip_emulated_instruction(vcpu); + ret = EXIT_FASTPATH_REENTER_GUEST; trace_kvm_msr_write(msr, data); + } else { + ret = EXIT_FASTPATH_NONE; + } kvm_vcpu_srcu_read_unlock(vcpu); From patchwork Fri Aug 2 19:51:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13751942 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 723257A13A for ; Fri, 2 Aug 2024 19:51:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628289; cv=none; b=k+h9AtwVCmnGMW5JG8ZeluwstJZ/Oa3S/8eRAnVel6DpL51Tt4eDn5StRDEa/CHtCyslJ5An5lC5rSzZp1jNnC6IEa36S6onpTsDUcAgwUj5aNtq7NBAKilEsFGAK06Jv2FYqaboH2T4A6vPZc8qzrnGqWtbHgIsh0m9XYN3p20= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628289; c=relaxed/simple; bh=kH7IYR7zBJzDgCb4mp8jhpOwUdQ4CJQGnGMEivOUcfI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Dy7bKb69ofKLU0yem/6i+5T6uxGA7zX7OoHi3qkPviHhaegoZwgIHzcYgrCOCx0ETL30cJGSSZltQc/NeaS8d/NvScyQFKwh5Jh7t0mtxrLlGUF8TPp0ZzPoWS4sRjOb/ye+aL93lYXppY4d6g/P8AYE62KytLjOBUJlOmpf7KQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=czdn7o28; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="czdn7o28" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7a242496897so6685102a12.2 for ; Fri, 02 Aug 2024 12:51:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722628288; x=1723233088; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0yZA/m2BmuooHYMrQtHZI/2LD/LZNOm46ViK83mQlGA=; b=czdn7o28UWBh1H7Rc6p1josjWAiFIKyOWzYuJM2avuxuo+DpalcHk4+DxQmYq9QssD 7RLFlgTi8v4amYOGS2pXruOfKlcHSJC7NiBzRuT3cOm+s2bXcv94999MsROqaXp1/7mp 6JGmDBuaQ15Ejef9ShW3Ek0b3wt+VyC8sjNCAIfwCjC4lGZ8/2guXXiQ3D7dTr2fYfY4 86SuLk8cpB7tKZxL/uwR09xcjPrI3u5t8kPpIaurYAp4kzyYxIVna4yu/Jxn3T8Du4pn pISa0m6s3nvpTORm0XhuTcX7mrA3pLhOfUf+z9Gq5b4n1SNR6Xil4KtFhDScqXMtzG2l 6NHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722628288; x=1723233088; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0yZA/m2BmuooHYMrQtHZI/2LD/LZNOm46ViK83mQlGA=; b=FG1+eL6a3UgQzaUMLdRWXssnmTsZ7f+kYd0wXltV4K3yvsqUenk72jip2YX2Xf9iyf K233uxdOrLF9/p1YMuV/5Ymi4GuKcLySV/zvRa5rdpsZ/GuZx/vZ8f1s6PSkhwwrm/aT 6ImT0DQzm18QtsKWCSN3GOzeYqNrndG+6Yovn84xdT5DM48h8khquk9UrsEbnioJ0v0t C/FRnb9kagvQGJ8Nao+n+bRRURtde5fDm7tgOdnLJcRYsiTx8b4R21kBltcxIj6TcXAM YBtWGnFSCFUWvizmVbqIyHA9dXyZHG62aAg6AC/HNijWLqW3/RAyWpa9c5K7IOGlnXzw ZRZg== X-Gm-Message-State: AOJu0YxI9upzrFC9dMVVerzttBtcRB2Atz3yIZEF6OfEns96sO2QZIm9 KQlOX7trWeIq5J4VSEhZl49kzCa5iXJ4Bfxf90z2ejnCE/aexi29Vgt3hT8sBTQt5E6wMtIZcGn tsA== X-Google-Smtp-Source: AGHT+IENPeRQ23Bb2K/Kq1zD91vd3dLeBsSlYk8jiLIo5PGRaYnIsL438VpxjIKwYaVrRR+p4LzacGFzdNU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a63:6307:0:b0:75c:49a4:2a2c with SMTP id 41be03b00d2f7-7b749d180e2mr7687a12.7.1722628287593; Fri, 02 Aug 2024 12:51:27 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 2 Aug 2024 12:51:18 -0700 In-Reply-To: <20240802195120.325560-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240802195120.325560-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog Message-ID: <20240802195120.325560-4-seanjc@google.com> Subject: [PATCH 3/5] KVM: x86: Exit to userspace if fastpath triggers one on instruction skip From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Exit to userspace if a fastpath handler triggers such an exit, which can happen when skipping the instruction, e.g. due to userspace single-stepping the guest via KVM_GUESTDBG_SINGLESTEP or because of an emulation failure. Fixes: 404d5d7bff0d ("KVM: X86: Introduce more exit_fastpath_completion enum values") Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 9 +++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 950a03e0181e..736dda300849 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -211,6 +211,7 @@ enum exit_fastpath_completion { EXIT_FASTPATH_NONE, EXIT_FASTPATH_REENTER_GUEST, EXIT_FASTPATH_EXIT_HANDLED, + EXIT_FASTPATH_EXIT_USERSPACE, }; typedef enum exit_fastpath_completion fastpath_t; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 332584476129..3c54a241696f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2184,8 +2184,10 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) } if (handled) { - kvm_skip_emulated_instruction(vcpu); - ret = EXIT_FASTPATH_REENTER_GUEST; + if (!kvm_skip_emulated_instruction(vcpu)) + ret = EXIT_FASTPATH_EXIT_USERSPACE; + else + ret = EXIT_FASTPATH_REENTER_GUEST; trace_kvm_msr_write(msr, data); } else { ret = EXIT_FASTPATH_NONE; @@ -11206,6 +11208,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (vcpu->arch.apic_attention) kvm_lapic_sync_from_vapic(vcpu); + if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) + return 0; + r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath); return r; From patchwork Fri Aug 2 19:51:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13751943 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B25D13049E for ; Fri, 2 Aug 2024 19:51:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628291; cv=none; b=luV0/NsaxjaKNNQmjwRZEwwlJ7hfONcWK3DHu5J2S1gS0ZJXsRtqOsdsWRFK6e41fvbrORDcdY0ZBsgB/1OGMZjMKB8zeU977dizZfQjyLN7lEQkfUQgccgOIJg0fz7M7CIF5OV3gA3dj2NtX7Se0QH8XnMNHCZPum8DxCVIXFQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628291; c=relaxed/simple; bh=GisDFGhm1yyVC6WE7YFlKal02q+ebK2NzRvmB0/ukH4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=eMXBchOaTCN+/OQfBb4eg+pacXfvdpoa8fkukiOu1wXHBHlvAVIAFyGbkWljhiFAiOZF0NBmH6/e0CUW7zpbQKNuGoFrJvvUu2xkykj8qwuJPq3WQ72gIzxx9p+Ei2FnzZp/ctHGnK1YEOPPGXeV8k9c3pYGJKPD/zt38NumObg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=A/6RYMB7; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="A/6RYMB7" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-1fd6d695662so88996495ad.0 for ; Fri, 02 Aug 2024 12:51:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722628289; x=1723233089; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=rTtTuEYhkp7ZliDdCROPWm4/aH5E9MbOqSEDHs65qFg=; b=A/6RYMB7DxMNCw0lA6At4tRxJqtHXS87PyI1lKNVoylXhdaogfKBdS2dDADgngHVai 1cRaUxXK0pzLO78vPRQy+FAlWJWGYzuw8xnPODS4CEQmZ0P7r+DiU7nOhmmYdSjpdD7l fPXfHPV1wPRjqZOu5GaZNqGUEZ6MbOIch/Nc2ysoyZOUTV15g7xbgIha8XNaN8OhgXHg HjSvNjYFW/2273E6AbZiWjE4daNO47Q36UjQbJZt+bq7i/ekAa/SyLZi/ROGyTkzrjmK ffTuq8BW1Dg67Ml+UqoTQVDcncPDpr7m2aWEQyZHjsIyZ7TqkEMdYE+GFZnEHB5s1TlF k/eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722628289; x=1723233089; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=rTtTuEYhkp7ZliDdCROPWm4/aH5E9MbOqSEDHs65qFg=; b=b3C8B+TRll5c9moyTTbHXZvRZ1aP7+1Ry5l9gNWv2W79aU5ENy0Y/ocmEcAPvnkNI6 qe5rH/3nL+tNooxftdsMbUDTTTlaikMznWaAGOSaLCNcNaLZHIGusC8abC2kPykRaU6O gRzWVhq0LFxy6AbIv0uRUnqAQe6A2IA88nbgDCJQyYsiC1ruTKs1j5XwBzG7tKtxFa1V nqSfFECC2P5GI9K5tCqmL2plj7cLADjmyc8f6M88SM4ywyiql2dhBk56VTBEU6SVHHeg /Q7wixx6GriM1iQy6yAsyPIFHJf4RSQhm0R+sVd71oFYNBxrIEmiaZiqW04DcqNw1Sao bhUg== X-Gm-Message-State: AOJu0YxV4bEvsKqSznrYBs3t4cEfOM6esr+nBZ2weMeR3x50o2Nk9H6e xYmz3cAF1Mjh/SbXlpysxzhB1iaWERUtihTKUivsGHAYr4Ww+aDsANaiZbc8bwks94bnhHXkl+C OWg== X-Google-Smtp-Source: AGHT+IFBUss19Xpkp75NzbgRP0olqSoTDLPxcqWGsDO3MC/8JqTWqabd+CJKuvDdDMWWc9kapW0n3uESCPc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:ec92:b0:1fb:f03:8935 with SMTP id d9443c01a7336-1ff573e3bc2mr3366305ad.7.1722628289493; Fri, 02 Aug 2024 12:51:29 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 2 Aug 2024 12:51:19 -0700 In-Reply-To: <20240802195120.325560-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240802195120.325560-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog Message-ID: <20240802195120.325560-5-seanjc@google.com> Subject: [PATCH 4/5] KVM: x86: Reorganize code in x86.c to co-locate vCPU blocking/running helpers From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Shuffle code around in x86.c so that the various helpers related to vCPU blocking/running logic are (a) located near each other and (b) ordered so that HLT emulation can use kvm_vcpu_has_events() in a future path. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 264 ++++++++++++++++++++++----------------------- 1 file changed, 132 insertions(+), 132 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3c54a241696f..46686504cd47 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9927,51 +9927,6 @@ void kvm_x86_vendor_exit(void) } EXPORT_SYMBOL_GPL(kvm_x86_vendor_exit); -static int __kvm_emulate_halt(struct kvm_vcpu *vcpu, int state, int reason) -{ - /* - * The vCPU has halted, e.g. executed HLT. Update the run state if the - * local APIC is in-kernel, the run loop will detect the non-runnable - * state and halt the vCPU. Exit to userspace if the local APIC is - * managed by userspace, in which case userspace is responsible for - * handling wake events. - */ - ++vcpu->stat.halt_exits; - if (lapic_in_kernel(vcpu)) { - vcpu->arch.mp_state = state; - return 1; - } else { - vcpu->run->exit_reason = reason; - return 0; - } -} - -int kvm_emulate_halt_noskip(struct kvm_vcpu *vcpu) -{ - return __kvm_emulate_halt(vcpu, KVM_MP_STATE_HALTED, KVM_EXIT_HLT); -} -EXPORT_SYMBOL_GPL(kvm_emulate_halt_noskip); - -int kvm_emulate_halt(struct kvm_vcpu *vcpu) -{ - int ret = kvm_skip_emulated_instruction(vcpu); - /* - * TODO: we might be squashing a GUESTDBG_SINGLESTEP-triggered - * KVM_EXIT_DEBUG here. - */ - return kvm_emulate_halt_noskip(vcpu) && ret; -} -EXPORT_SYMBOL_GPL(kvm_emulate_halt); - -int kvm_emulate_ap_reset_hold(struct kvm_vcpu *vcpu) -{ - int ret = kvm_skip_emulated_instruction(vcpu); - - return __kvm_emulate_halt(vcpu, KVM_MP_STATE_AP_RESET_HOLD, - KVM_EXIT_AP_RESET_HOLD) && ret; -} -EXPORT_SYMBOL_GPL(kvm_emulate_ap_reset_hold); - #ifdef CONFIG_X86_64 static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, unsigned long clock_type) @@ -11224,6 +11179,67 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) return r; } +static bool kvm_vcpu_running(struct kvm_vcpu *vcpu) +{ + return (vcpu->arch.mp_state == KVM_MP_STATE_RUNNABLE && + !vcpu->arch.apf.halted); +} + +static bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) +{ + if (!list_empty_careful(&vcpu->async_pf.done)) + return true; + + if (kvm_apic_has_pending_init_or_sipi(vcpu) && + kvm_apic_init_sipi_allowed(vcpu)) + return true; + + if (vcpu->arch.pv.pv_unhalted) + return true; + + if (kvm_is_exception_pending(vcpu)) + return true; + + if (kvm_test_request(KVM_REQ_NMI, vcpu) || + (vcpu->arch.nmi_pending && + kvm_x86_call(nmi_allowed)(vcpu, false))) + return true; + +#ifdef CONFIG_KVM_SMM + if (kvm_test_request(KVM_REQ_SMI, vcpu) || + (vcpu->arch.smi_pending && + kvm_x86_call(smi_allowed)(vcpu, false))) + return true; +#endif + + if (kvm_test_request(KVM_REQ_PMI, vcpu)) + return true; + + if (kvm_test_request(KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, vcpu)) + return true; + + if (kvm_arch_interrupt_allowed(vcpu) && kvm_cpu_has_interrupt(vcpu)) + return true; + + if (kvm_hv_has_stimer_pending(vcpu)) + return true; + + if (is_guest_mode(vcpu) && + kvm_x86_ops.nested_ops->has_events && + kvm_x86_ops.nested_ops->has_events(vcpu, false)) + return true; + + if (kvm_xen_has_pending_events(vcpu)) + return true; + + return false; +} + +int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu) +{ + return kvm_vcpu_running(vcpu) || kvm_vcpu_has_events(vcpu); +} + /* Called within kvm->srcu read side. */ static inline int vcpu_block(struct kvm_vcpu *vcpu) { @@ -11295,12 +11311,6 @@ static inline int vcpu_block(struct kvm_vcpu *vcpu) return 1; } -static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) -{ - return (vcpu->arch.mp_state == KVM_MP_STATE_RUNNABLE && - !vcpu->arch.apf.halted); -} - /* Called within kvm->srcu read side. */ static int vcpu_run(struct kvm_vcpu *vcpu) { @@ -11352,6 +11362,77 @@ static int vcpu_run(struct kvm_vcpu *vcpu) return r; } +static int __kvm_emulate_halt(struct kvm_vcpu *vcpu, int state, int reason) +{ + /* + * The vCPU has halted, e.g. executed HLT. Update the run state if the + * local APIC is in-kernel, the run loop will detect the non-runnable + * state and halt the vCPU. Exit to userspace if the local APIC is + * managed by userspace, in which case userspace is responsible for + * handling wake events. + */ + ++vcpu->stat.halt_exits; + if (lapic_in_kernel(vcpu)) { + vcpu->arch.mp_state = state; + return 1; + } else { + vcpu->run->exit_reason = reason; + return 0; + } +} + +int kvm_emulate_halt_noskip(struct kvm_vcpu *vcpu) +{ + return __kvm_emulate_halt(vcpu, KVM_MP_STATE_HALTED, KVM_EXIT_HLT); +} +EXPORT_SYMBOL_GPL(kvm_emulate_halt_noskip); + +int kvm_emulate_halt(struct kvm_vcpu *vcpu) +{ + int ret = kvm_skip_emulated_instruction(vcpu); + /* + * TODO: we might be squashing a GUESTDBG_SINGLESTEP-triggered + * KVM_EXIT_DEBUG here. + */ + return kvm_emulate_halt_noskip(vcpu) && ret; +} +EXPORT_SYMBOL_GPL(kvm_emulate_halt); + +int kvm_emulate_ap_reset_hold(struct kvm_vcpu *vcpu) +{ + int ret = kvm_skip_emulated_instruction(vcpu); + + return __kvm_emulate_halt(vcpu, KVM_MP_STATE_AP_RESET_HOLD, + KVM_EXIT_AP_RESET_HOLD) && ret; +} +EXPORT_SYMBOL_GPL(kvm_emulate_ap_reset_hold); + +bool kvm_arch_dy_has_pending_interrupt(struct kvm_vcpu *vcpu) +{ + return kvm_vcpu_apicv_active(vcpu) && + kvm_x86_call(dy_apicv_has_pending_interrupt)(vcpu); +} + +bool kvm_arch_vcpu_preempted_in_kernel(struct kvm_vcpu *vcpu) +{ + return vcpu->arch.preempted_in_kernel; +} + +bool kvm_arch_dy_runnable(struct kvm_vcpu *vcpu) +{ + if (READ_ONCE(vcpu->arch.pv.pv_unhalted)) + return true; + + if (kvm_test_request(KVM_REQ_NMI, vcpu) || +#ifdef CONFIG_KVM_SMM + kvm_test_request(KVM_REQ_SMI, vcpu) || +#endif + kvm_test_request(KVM_REQ_EVENT, vcpu)) + return true; + + return kvm_arch_dy_has_pending_interrupt(vcpu); +} + static inline int complete_emulated_io(struct kvm_vcpu *vcpu) { return kvm_emulate_instruction(vcpu, EMULTYPE_NO_DECODE); @@ -13163,87 +13244,6 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, kvm_arch_free_memslot(kvm, old); } -static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) -{ - if (!list_empty_careful(&vcpu->async_pf.done)) - return true; - - if (kvm_apic_has_pending_init_or_sipi(vcpu) && - kvm_apic_init_sipi_allowed(vcpu)) - return true; - - if (vcpu->arch.pv.pv_unhalted) - return true; - - if (kvm_is_exception_pending(vcpu)) - return true; - - if (kvm_test_request(KVM_REQ_NMI, vcpu) || - (vcpu->arch.nmi_pending && - kvm_x86_call(nmi_allowed)(vcpu, false))) - return true; - -#ifdef CONFIG_KVM_SMM - if (kvm_test_request(KVM_REQ_SMI, vcpu) || - (vcpu->arch.smi_pending && - kvm_x86_call(smi_allowed)(vcpu, false))) - return true; -#endif - - if (kvm_test_request(KVM_REQ_PMI, vcpu)) - return true; - - if (kvm_test_request(KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, vcpu)) - return true; - - if (kvm_arch_interrupt_allowed(vcpu) && kvm_cpu_has_interrupt(vcpu)) - return true; - - if (kvm_hv_has_stimer_pending(vcpu)) - return true; - - if (is_guest_mode(vcpu) && - kvm_x86_ops.nested_ops->has_events && - kvm_x86_ops.nested_ops->has_events(vcpu, false)) - return true; - - if (kvm_xen_has_pending_events(vcpu)) - return true; - - return false; -} - -int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu) -{ - return kvm_vcpu_running(vcpu) || kvm_vcpu_has_events(vcpu); -} - -bool kvm_arch_dy_has_pending_interrupt(struct kvm_vcpu *vcpu) -{ - return kvm_vcpu_apicv_active(vcpu) && - kvm_x86_call(dy_apicv_has_pending_interrupt)(vcpu); -} - -bool kvm_arch_vcpu_preempted_in_kernel(struct kvm_vcpu *vcpu) -{ - return vcpu->arch.preempted_in_kernel; -} - -bool kvm_arch_dy_runnable(struct kvm_vcpu *vcpu) -{ - if (READ_ONCE(vcpu->arch.pv.pv_unhalted)) - return true; - - if (kvm_test_request(KVM_REQ_NMI, vcpu) || -#ifdef CONFIG_KVM_SMM - kvm_test_request(KVM_REQ_SMI, vcpu) || -#endif - kvm_test_request(KVM_REQ_EVENT, vcpu)) - return true; - - return kvm_arch_dy_has_pending_interrupt(vcpu); -} - bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu) { if (vcpu->arch.guest_state_protected) From patchwork Fri Aug 2 19:51:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13751944 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3290D1369B0 for ; Fri, 2 Aug 2024 19:51:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628293; cv=none; b=ts83TpFfFFzn7w6C8K7H7C+mGgr4gcsW6zYO3QEsLUCfGFtrUETFEq/fqsZj0RC6QuzOPxtANH71qoAJsy06c7nRrh1Aqv40OuLkRsmkgn6P7E5npaSOHw2tik/JP3TY70RjhpPCXtUxXd+1RFIiVsezFb6UFSonbCCE1Eo4Ppk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722628293; c=relaxed/simple; bh=mqE0o9zRwOx1Mn/Z6tsZ+L0NFQbPg40KmGQ99XrGl84=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NVt9AIDSmUmdg3+L6gv7+fJX9YYZAUfmFyQfOyPotLS8wDsWy4Bu3bskCdqPWwmqGRRUyAiyuzE3dnBzKF4Z+e402Q/FnMtpuv+f0mvUU8nC1HdwH5AcjMxbEkBTl4lmVGd40AZIlfWLk5LLseIPTjI0hkfl55SM9ONZ5iyNm1k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QMmrmz1H; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QMmrmz1H" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-1fc6db23c74so85327765ad.0 for ; Fri, 02 Aug 2024 12:51:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722628291; x=1723233091; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=nbb8wMxDZTZlGNXuFp2fO6u+E/pERXdN8QklXoMJTsY=; b=QMmrmz1HJQVD/3afA/OZM4byS7A4t0tDwa96junvLXjETMtlfpsxI/k0o1g2EjAWmG ZRtz8sWCitc8VPrmvrpdbYMJGbcR0Ao2GogV8gD4TtWElUgSAIw+v+86xCeZuhEReUe8 GtAUaEuG7n1ux6Y8VgyPaxCu/R2IXLu3lMUjcO/gmsWTpyXgIV810ML+l+ZOqEgqeVj8 Vm8wyfKq724oBn9g21HxhKVnV633B02qrh60uzZIKynAzEbx4S1g5m/EthBtBsKE3OFB Q/VRIaBQuafAXKoXGM68fXkk1j9dL8J2CshKVfoKw62HeOQ7Ihmw3q2qzbvzS/C/s1oM amMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722628291; x=1723233091; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nbb8wMxDZTZlGNXuFp2fO6u+E/pERXdN8QklXoMJTsY=; b=ONDOf0NS07QvJXVUYroD7NPVLxGzWu3JSkBcIwwj/VNbvKLsIZZvYywMkHYlpztkFo jMI6ajKr2xjDwlGJTbGsfVs5QtoIL1wetId5MiwxFJrLHHu2S69885tzvvqzi491MG1t GYbUKDMZgbrgptRq6UQAAT1djBB4SOp9M8N6Q9Bs68jtMiWG8mtTixzFYdOQ/o+pZzud 5yVM2cfgNAl/57k/gQHosCD89lwbu64zfbIMy9aCfXEOC79e93lVlcsiRX06Sj2rxQ/V z0iXjeI/kgRjU6K+sAxNYnhAaDprIGR8w4pClynQrJ7s2XCqIM69EskdEyUoyTvXA9Az SV/A== X-Gm-Message-State: AOJu0YypHPCLOtqhDt24sHc38Kqg2GdmRtslGZHDCTx/VZHhj558PlbQ To5F6Go0oWlBaz4gRgnTL43oAAy5rw+RxuovQ5ZIwHl0+e3LgaqnM99cAuSq9XgM/Wcrp0msPYt fgg== X-Google-Smtp-Source: AGHT+IEeAJQqKfvBQKb2VjxpO0q2I+GKyZiMO8d1EwpY/xEJMhUHvJjVefCWVqCWyOQEcOEK5i7Edt5hsBM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:c38c:b0:1fb:7f2c:5642 with SMTP id d9443c01a7336-1ff5730306cmr2707885ad.4.1722628291438; Fri, 02 Aug 2024 12:51:31 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 2 Aug 2024 12:51:20 -0700 In-Reply-To: <20240802195120.325560-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240802195120.325560-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog Message-ID: <20240802195120.325560-6-seanjc@google.com> Subject: [PATCH 5/5] KVM: x86: Add fastpath handling of HLT VM-Exits From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Add a fastpath for HLT VM-Exits by immediately re-entering the guest if it has a pending wake event. When virtual interrupt delivery is enabled, i.e. when KVM doesn't need to manually inject interrupts, this allows KVM to stay in the fastpath run loop when a vIRQ arrives between the guest doing CLI and STI;HLT. Without AMD's Idle HLT-intercept support, the CPU generates a HLT VM-Exit even though KVM will immediately resume the guest. Note, on bare metal, it's relatively uncommon for a modern guest kernel to actually trigger this scenario, as the window between the guest checking for a wake event and committing to HLT is quite small. But in a nested environment, the timings change significantly, e.g. rudimentary testing showed that ~50% of HLT exits where HLT-polling was successful would be serviced by this fastpath, i.e. ~50% of the time that a nested vCPU gets a wake event before KVM schedules out the vCPU, the wake event was pending even before the VM-Exit. Link: https://lore.kernel.org/all/20240528041926.3989-3-manali.shukla@amd.com Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/svm.c | 13 +++++++++++-- arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 23 ++++++++++++++++++++++- arch/x86/kvm/x86.h | 1 + 4 files changed, 36 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index c115d26844f7..64381ff63034 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4144,12 +4144,21 @@ static int svm_vcpu_pre_run(struct kvm_vcpu *vcpu) static fastpath_t svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu) { + struct vcpu_svm *svm = to_svm(vcpu); + if (is_guest_mode(vcpu)) return EXIT_FASTPATH_NONE; - if (to_svm(vcpu)->vmcb->control.exit_code == SVM_EXIT_MSR && - to_svm(vcpu)->vmcb->control.exit_info_1) + switch (svm->vmcb->control.exit_code) { + case SVM_EXIT_MSR: + if (!svm->vmcb->control.exit_info_1) + break; return handle_fastpath_set_msr_irqoff(vcpu); + case SVM_EXIT_HLT: + return handle_fastpath_hlt(vcpu); + default: + break; + } return EXIT_FASTPATH_NONE; } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f18c2d8c7476..f6382750fbf0 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7265,6 +7265,8 @@ static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu, return handle_fastpath_set_msr_irqoff(vcpu); case EXIT_REASON_PREEMPTION_TIMER: return handle_fastpath_preemption_timer(vcpu, force_immediate_exit); + case EXIT_REASON_HLT: + return handle_fastpath_hlt(vcpu); default: return EXIT_FASTPATH_NONE; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 46686504cd47..eb5ea963698f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11373,7 +11373,10 @@ static int __kvm_emulate_halt(struct kvm_vcpu *vcpu, int state, int reason) */ ++vcpu->stat.halt_exits; if (lapic_in_kernel(vcpu)) { - vcpu->arch.mp_state = state; + if (kvm_vcpu_has_events(vcpu)) + vcpu->arch.pv.pv_unhalted = false; + else + vcpu->arch.mp_state = state; return 1; } else { vcpu->run->exit_reason = reason; @@ -11398,6 +11401,24 @@ int kvm_emulate_halt(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvm_emulate_halt); +fastpath_t handle_fastpath_hlt(struct kvm_vcpu *vcpu) +{ + int ret; + + kvm_vcpu_srcu_read_lock(vcpu); + ret = kvm_emulate_halt(vcpu); + kvm_vcpu_srcu_read_unlock(vcpu); + + if (!ret) + return EXIT_FASTPATH_EXIT_USERSPACE; + + if (kvm_vcpu_running(vcpu)) + return EXIT_FASTPATH_REENTER_GUEST; + + return EXIT_FASTPATH_EXIT_HANDLED; +} +EXPORT_SYMBOL_GPL(handle_fastpath_hlt); + int kvm_emulate_ap_reset_hold(struct kvm_vcpu *vcpu) { int ret = kvm_skip_emulated_instruction(vcpu); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 50596f6f8320..5185ab76fdd2 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -334,6 +334,7 @@ int x86_decode_emulated_instruction(struct kvm_vcpu *vcpu, int emulation_type, int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, int emulation_type, void *insn, int insn_len); fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu); +fastpath_t handle_fastpath_hlt(struct kvm_vcpu *vcpu); extern struct kvm_caps kvm_caps; extern struct kvm_host_values kvm_host;