From patchwork Tue Mar 19 16:43:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Colton Lewis X-Patchwork-Id: 13596917 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39672C54E71 for ; Tue, 19 Mar 2024 16:44:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=EIKisGJm1F5XmDqLzqmYgpbCTge6gSFSmAm6hPuQUYI=; b=jCq bL2ZpHy9sMAGHxcdLjiKILaYSoRvEKZJGe98SFIphgOzkgCLKTgSkx9Z/e/NgfVhWeEO7JPKEwNFR GlBC3szko+67cy+9jlDk/vy6G6GLw41481kcHqVtyTtBNyZE6cdFhvMYlsyEMub2J4VuzEPpwIVUd y01ocSNCoxtei3mTELlC0jk6UCegGqCn9kHocpG+FMKpwzG/j4d6PU4KQJBXOQFcKMfr8zzz47B8x UOK+E7ioYXdGCri7bBWqIhljNcsrcAosE8DJjTI3jUdYTnIMufgYjG3STTZgneRL2SB4CN8BNdKij QFVXQgj8DXkOUDmPc9xSYUSuCcvAYlg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rmcZd-0000000DSY7-0sm3; Tue, 19 Mar 2024 16:44:25 +0000 Received: from mail-io1-xd49.google.com ([2607:f8b0:4864:20::d49]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rmcZb-0000000DSW5-0MxF for linux-arm-kernel@lists.infradead.org; Tue, 19 Mar 2024 16:44:24 +0000 Received: by mail-io1-xd49.google.com with SMTP id ca18e2360f4ac-7cbf3ec6a96so4729739f.0 for ; Tue, 19 Mar 2024 09:44:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710866656; x=1711471456; darn=lists.infradead.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=55tADTKr/fKtx+P9Mx3gQHhaNaSTbLjPsk1QuvUf7uI=; b=FrjlmEMfgXYkgTNSnNVb7FsKB84yaElmo02fACHjqOYku3fRKhQSJSdpDfYkgOASws 6vab7DzsJIK25Psj9msuDKOzRUTLv6qpzQutTNPUL9buLm6BNAiIDW1iKImZmWqmOSH6 pC9VqyRrdunkh9NQNRYPL/NnNIgbQ3D4B/7jtC+5trbbfvo9tgC8QDWeiKp2NyQZvXhV LMGk6DIS7dH6veX0tuoeJmpz/C0aS198+zDv0K265Sc6Bg/E2bJyDCPNvJQzdzSVY0/q C5bkRxxwKadSiNkpiNBfY1ooNqBtY2u7nF9YnXxJsOqaLXnJqEqgbLFLSp3KPfgNLbRQ XO+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710866656; x=1711471456; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=55tADTKr/fKtx+P9Mx3gQHhaNaSTbLjPsk1QuvUf7uI=; b=nP61rHTsxJg9+v6pKlSC2dvHV99N9xdqHd9181fBzwOi+RJ7dtgOhtInvPw3QF2FZJ LcjDInBNTYotJygfkFTJPppMmKUiW4lZmgegLXRvHYvlqLhvi6nBcsw5VTMnc2mFbVvD +SCizFk8ZMyaPYCAzYMuURfL55juTRjp4Eaoik0SDIpcWGOP6ZYriqojF+3ucb++ushM JPYHAHaf90vA8ul6WKfTTSstmtMzO65Xi1NxTFMKZDXz/OxQy1ypyYERkPdk81WPNv/v S3Z2xgBUNBZyaZt01ZNUW/pD/w1F7xt35j4m9K1Ag6hvkwTPVODTdPBVfcSRMFvHLYsJ dB9g== X-Forwarded-Encrypted: i=1; AJvYcCWJ8cBrJLCk+H9aM42Bow4jZkdruD9EAmo8AehgDjXoazkcBgcBeIoUbZiQG8yD/BvDNpFVQkOJCqCxDfbkWECHH4oDHMV0K+sPFdNk+cGfV3lcDhI= X-Gm-Message-State: AOJu0YxFtH9+BcAOYLVXSii1LDIuOYRsLoMEUR9hFJ0b22a/98C3gQf0 VIE1mCXZ5jXqn7jVtE4OPShlvU+sdtW+uJU4v/fSvHwMqTfV0gssRIpAGoFdWu/xDnNRRedtDp4 IlgD9tKu5wWP1JwuhtJs0fg== X-Google-Smtp-Source: AGHT+IFRbk7J0ciLHbLLk9I3qhSS1omT3Bl07k9k3e57ZQB+uGmYZ+eVWm/uRTh6ZDfo3t25XVq4gku22P4yF8IxNA== X-Received: from coltonlewis-kvm.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:14ce]) (user=coltonlewis job=sendgmr) by 2002:a05:6602:1491:b0:7c8:264d:5e98 with SMTP id a17-20020a056602149100b007c8264d5e98mr145907iow.0.1710866655492; Tue, 19 Mar 2024 09:44:15 -0700 (PDT) Date: Tue, 19 Mar 2024 16:43:41 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.44.0.291.gc1ea87d7ee-goog Message-ID: <20240319164341.1674863-1-coltonlewis@google.com> Subject: [PATCH v2] KVM: arm64: Add KVM_CAP to control WFx trapping From: Colton Lewis To: kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , Paolo Bonzini , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, Colton Lewis X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240319_094423_157631_E3D402F7 X-CRM114-Status: GOOD ( 22.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a KVM_CAP to control WFx (WFI or WFE) trapping based on scheduler runqueue depth. This is so they can be passed through if the runqueue is shallow or the CPU has support for direct interrupt injection. They may be always trapped by setting this value to 0. Technically this means traps will be cleared when the runqueue depth is 0, but that implies nothing is running anyway so there is no reason to care. The default value is 1 to preserve previous behavior before adding this option. Think about his option as a threshold. The instruction will be trapped if the runqueue depth is higher than the threshold. Signed-off-by: Colton Lewis --- v2: The last version was exclusively a flag to enable unconditional wfx passthrough but there was feedback to make passthrough/trapping depend on runqueue depth. I asked the last thread if there were any preferences for the interface to accomplish this but I figured it's easier to show code than wait for people telling me what to do. v1: https://lore.kernel.org/kvmarm/20240129213918.3124494-1-coltonlewis@google.com/ arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/kvm/arm.c | 7 ++++++- include/linux/sched/stat.h | 1 + include/uapi/linux/kvm.h | 2 +- kernel/sched/core.c | 15 +++++++++++++-- 5 files changed, 22 insertions(+), 4 deletions(-) -- 2.44.0.291.gc1ea87d7ee-goog diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 21c57b812569..79f461efaa6c 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -317,6 +317,7 @@ struct kvm_arch { * the associated pKVM instance in the hypervisor. */ struct kvm_protected_vm pkvm; + u64 wfx_trap_runqueue_depth; }; struct kvm_vcpu_fault_info { diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index a25265aca432..419eed6e1814 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -116,6 +116,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, } mutex_unlock(&kvm->slots_lock); break; + case KVM_CAP_ARM_WFX_TRAP_RUNQUEUE_DEPTH: + kvm->arch.wfx_trap_runqueue_depth = cap->args[0]; + break; default: r = -EINVAL; break; @@ -176,6 +179,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES); + kvm->arch.wfx_trap_runqueue_depth = 1; return 0; err_free_cpumask: @@ -240,6 +244,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_ARM_SYSTEM_SUSPEND: case KVM_CAP_IRQFD_RESAMPLE: case KVM_CAP_COUNTER_OFFSET: + case KVM_CAP_ARM_WFX_TRAP_RUNQUEUE_DEPTH: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: @@ -456,7 +461,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) if (kvm_arm_is_pvtime_enabled(&vcpu->arch)) kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu); - if (single_task_running()) + if (nr_running_this_cpu() <= vcpu->kvm->arch.wfx_trap_runqueue_depth) vcpu_clear_wfx_traps(vcpu); else vcpu_set_wfx_traps(vcpu); diff --git a/include/linux/sched/stat.h b/include/linux/sched/stat.h index 0108a38bb64d..dc1541fcec56 100644 --- a/include/linux/sched/stat.h +++ b/include/linux/sched/stat.h @@ -18,6 +18,7 @@ extern int nr_threads; DECLARE_PER_CPU(unsigned long, process_counts); extern int nr_processes(void); extern unsigned int nr_running(void); +extern unsigned int nr_running_this_cpu(void); extern bool single_task_running(void); extern unsigned int nr_iowait(void); extern unsigned int nr_iowait_cpu(int cpu); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index c3308536482b..4c0ebf514c03 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1155,6 +1155,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_MEMORY_ATTRIBUTES 233 #define KVM_CAP_GUEST_MEMFD 234 #define KVM_CAP_VM_TYPES 235 +#define KVM_CAP_ARM_WFX_TRAP_RUNQUEUE_DEPTH 236 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 9116bcc90346..b18f29964648 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5420,7 +5420,7 @@ unsigned int nr_running(void) } /* - * Check if only the current task is running on the CPU. + * Return number of tasks running on this CPU. * * Caution: this function does not check that the caller has disabled * preemption, thus the result might have a time-of-check-to-time-of-use @@ -5432,9 +5432,20 @@ unsigned int nr_running(void) * * - in a loop with very short iterations (e.g. a polling loop) */ +unsigned int nr_running_this_cpu(void) +{ + return raw_rq()->nr_running; +} +EXPORT_SYMBOL(nr_running_this_cpu); + +/* + * Check if only the current task is running on the CPU. + * + * Caution: see warning for nr_running_this_cpu + */ bool single_task_running(void) { - return raw_rq()->nr_running == 1; + return nr_running_this_cpu() == 1; } EXPORT_SYMBOL(single_task_running);