From patchwork Thu Feb 27 02:18:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993546 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C1951A8F63 for ; Thu, 27 Feb 2025 02:19:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622748; cv=none; b=EYT9o0RT+H+pYgmsJJ4kZUkWJCNZ63qy9VC2NHHev2z9neVIiL3b/O72hvgIO3Q4Xzv4qWCYYZxInR7VilDrS8a8jbJ23gG4o7i9Wy84O6IJe5bz6Spqi+bhHPFb9IKuwrpeKzuRODMCE+Gc0uqtxX7knD0JravNR1k2awMkmDc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622748; c=relaxed/simple; bh=vr3NG+3TNhQXw/Ld9LNbvZ/lm9g+GH2tsC78kbtC4nE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=V4U/R5Gn/XUO1EHBLlt6I1NOjV6UkLZKCJGYoviaMIyBRiiTUMeWLvuvXg4RAI4MNtH/2SkGQn2FdpPKcMor2whaGe5RFEj00lEjymyGCcwpJn1b6Dje48xveMvzBYr4bVPCrkHe/gylwJNJ1+A89AKUUUR3fzpHYami+veZxtE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=U5Y3X53A; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="U5Y3X53A" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc1e7efdffso1555124a91.0 for ; Wed, 26 Feb 2025 18:19:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622744; x=1741227544; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KtBuHiug8RyBl7zdSfnpeCyQMsoAneIAJhrbW8v3FfQ=; b=U5Y3X53A2y1lQU6JZcZNk9HkzPtTid97/l/KmSjY0h0cAxtjcytG5ZjjcAEBtiFqLp PTsXBc5zeIj1yGbW2ecjlI+YpTlsIpD9ni9pvgISHOMJobsGUQqwbj9SSHbWUdzvYaUe ip36RXqRd5LhVlMNZfN8NzqjKVTXbEIVhe5RsPgKiWKIdWK+WonvNpxzm7eQ47tk9VY/ vCblbFtDKjSgTNT6UxtylyQx7LK/zpJTFogyTSNETtn4VbMKUJ9oLiYOIH9Z/VYm47Rj i2Ltiqj5MDHf3vgoMlPE+b6I7i27GjQrnnA6+4vfkej0hr27mnyA4RWheC9p8FvjsdTc NPWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622744; x=1741227544; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KtBuHiug8RyBl7zdSfnpeCyQMsoAneIAJhrbW8v3FfQ=; b=WCeDef4UbN89/S6F8J3/1vV1AOOAGrh5NtGXGypuJO7ItxneM51zsz7KATWUTkIfaP QmUGHeipB3fG1cypDbzSIOCqJrQWYAvvKkjpzOY0wqVtXWipESoafHucpZF4BJQWlNt/ 6vDWVddjDTu3joYBuInsXzFKGOb61LHTGnYTaofl23E8HOlBF62AiM2XlcBOKIKjoPth wbJPyuMH4fcotmd6oSmaAspGX8Zu/9FDEd1n9pRTzLOQ0cAZCYySvaKY+r6JI8MEMyB3 2fZ0pWfKkt4VJY9kvZDo+0gutNctSMsE3ri1suLS6x84+IRAEeXthfCJbUAa+DFJobaj 90cQ== X-Forwarded-Encrypted: i=1; AJvYcCXRtrhy6qVZICHF6Fr5K8cIGdqYsoqNoZGdQyuERR9xrChvqPUp9+IWdQwdehVM97JEnVs=@vger.kernel.org X-Gm-Message-State: AOJu0YwgGfS9DO3WYyXWQ5FE98EDYA5aVzwmfOHQPpuygu33Ak05nw4z CgQwYEfbyXPC900t+JDaN0MkF50mzf/GXmHM0g+8Bb29f/BbqF/FaqSXLSVS4b7EOf0B1CptvMY hsQ== X-Google-Smtp-Source: AGHT+IEZuLrSoCpVWZXJcy9TmAp9qIs68IJxs5ZGUPBqjbzVn9jX2mg4vMG9kljqKgFFkZhvGqcDyGJTu1M= X-Received: from pjbsn14.prod.google.com ([2002:a17:90b:2e8e:b0:2fc:15bf:92f6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1f8e:b0:2ee:b8ac:73b0 with SMTP id 98e67ed59e1d1-2fe68acd43fmr15933009a91.2.1740622744544; Wed, 26 Feb 2025 18:19:04 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:17 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-2-seanjc@google.com> Subject: [PATCH v2 01/38] x86/tsc: Add a standalone helpers for getting TSC info from CPUID.0x15 From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Extract retrieval of TSC frequency information from CPUID into standalone helpers so that TDX guest support and kvmlock can reuse the logic. Provide a version that includes the multiplier math as TDX in particular does NOT want to use native_calibrate_tsc()'s fallback logic that derives the TSC frequency based on CPUID.0x16 when the core crystal frequency isn't known. Opportunsitically drop native_calibrate_tsc()'s "== 0" and "!= 0" check in favor of the kernel's preferred style. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/tsc.h | 9 +++++ arch/x86/kernel/tsc.c | 67 +++++++++++++++++++++++++------------- 2 files changed, 53 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index 94408a784c8e..a4d84f721775 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -28,6 +28,15 @@ static inline cycles_t get_cycles(void) } #define get_cycles get_cycles +struct cpuid_tsc_info { + unsigned int denominator; + unsigned int numerator; + unsigned int crystal_khz; + unsigned int tsc_khz; +}; +extern int cpuid_get_tsc_info(struct cpuid_tsc_info *info); +extern int cpuid_get_tsc_freq(struct cpuid_tsc_info *info); + extern void tsc_early_init(void); extern void tsc_init(void); extern void mark_tsc_unstable(char *reason); diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 34dec0b72ea8..93713eb81f52 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -655,46 +655,67 @@ static unsigned long quick_pit_calibrate(void) return delta; } +int cpuid_get_tsc_info(struct cpuid_tsc_info *info) +{ + unsigned int ecx_hz, edx; + + memset(info, 0, sizeof(*info)); + + if (boot_cpu_data.cpuid_level < CPUID_LEAF_TSC) + return -ENOENT; + + /* CPUID 15H TSC/Crystal ratio, plus optionally Crystal Hz */ + cpuid(CPUID_LEAF_TSC, &info->denominator, &info->numerator, &ecx_hz, &edx); + + if (!info->denominator || !info->numerator) + return -ENOENT; + + /* + * Note, some CPUs provide the multiplier information, but not the core + * crystal frequency. The multiplier information is still useful for + * such CPUs, as the crystal frequency can be gleaned from CPUID.0x16. + */ + info->crystal_khz = ecx_hz / 1000; + return 0; +} + +int cpuid_get_tsc_freq(struct cpuid_tsc_info *info) +{ + if (cpuid_get_tsc_info(info) || !info->crystal_khz) + return -ENOENT; + + info->tsc_khz = info->crystal_khz * info->numerator / info->denominator; + return 0; +} + /** * native_calibrate_tsc - determine TSC frequency * Determine TSC frequency via CPUID, else return 0. */ unsigned long native_calibrate_tsc(void) { - unsigned int eax_denominator, ebx_numerator, ecx_hz, edx; - unsigned int crystal_khz; + struct cpuid_tsc_info info; if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) return 0; - if (boot_cpu_data.cpuid_level < CPUID_LEAF_TSC) + if (cpuid_get_tsc_info(&info)) return 0; - eax_denominator = ebx_numerator = ecx_hz = edx = 0; - - /* CPUID 15H TSC/Crystal ratio, plus optionally Crystal Hz */ - cpuid(CPUID_LEAF_TSC, &eax_denominator, &ebx_numerator, &ecx_hz, &edx); - - if (ebx_numerator == 0 || eax_denominator == 0) - return 0; - - crystal_khz = ecx_hz / 1000; - /* * Denverton SoCs don't report crystal clock, and also don't support * CPUID_LEAF_FREQ for the calculation below, so hardcode the 25MHz * crystal clock. */ - if (crystal_khz == 0 && - boot_cpu_data.x86_vfm == INTEL_ATOM_GOLDMONT_D) - crystal_khz = 25000; + if (!info.crystal_khz && boot_cpu_data.x86_vfm == INTEL_ATOM_GOLDMONT_D) + info.crystal_khz = 25000; /* * TSC frequency reported directly by CPUID is a "hardware reported" * frequency and is the most accurate one so far we have. This * is considered a known frequency. */ - if (crystal_khz != 0) + if (info.crystal_khz) setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); /* @@ -702,15 +723,15 @@ unsigned long native_calibrate_tsc(void) * clock, but we can easily calculate it to a high degree of accuracy * by considering the crystal ratio and the CPU speed. */ - if (crystal_khz == 0 && boot_cpu_data.cpuid_level >= CPUID_LEAF_FREQ) { + if (!info.crystal_khz && boot_cpu_data.cpuid_level >= CPUID_LEAF_FREQ) { unsigned int eax_base_mhz, ebx, ecx, edx; cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx, &ecx, &edx); - crystal_khz = eax_base_mhz * 1000 * - eax_denominator / ebx_numerator; + info.crystal_khz = eax_base_mhz * 1000 * + info.denominator / info.numerator; } - if (crystal_khz == 0) + if (!info.crystal_khz) return 0; /* @@ -727,10 +748,10 @@ unsigned long native_calibrate_tsc(void) * lapic_timer_period here to avoid having to calibrate the APIC * timer later. */ - lapic_timer_period = crystal_khz * 1000 / HZ; + lapic_timer_period = info.crystal_khz * 1000 / HZ; #endif - return crystal_khz * ebx_numerator / eax_denominator; + return info.crystal_khz * info.numerator / info.denominator; } static unsigned long cpu_khz_from_cpuid(void) From patchwork Thu Feb 27 02:18:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993547 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21FE314A4DF for ; Thu, 27 Feb 2025 02:19:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622748; cv=none; b=qmwDwGxJHZYY6Bor53DwwF2N/kXg0nXZoJHUWvl5HlRVkNG40o4QSnN6FfCf5rJyd40QyPqs4/Oxp68Jbu6AVLq4HIa9IJvTpvKhMmpKDXCWgExJikvUvEc49Iu03XASs0EUpXsJJ7fsIdGR2J8k4OajSRxPQEhs3KBsLLy2V0A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622748; c=relaxed/simple; bh=YIm02xTVNm0eSDO98sEQL1tUrXbpItcdQ7Va1EiMjyI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DwxtEbd2vws+ayPNfscGegw4gwa6MaVwroSYcRL7DxT7d8wFAmU7sdJOcmfyrn7A65yRLTmnZsqT9bGW5jpqJIFH1JbUYpy/JiALZ33wEzvqH7yaywk9g83TnqUpk6/pX1AlI0Rtt2hhroMuEkqeeN2fWJ1+oYy3AtmUx8q9teA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Zxx7mXR/; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Zxx7mXR/" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fc45101191so1079379a91.1 for ; Wed, 26 Feb 2025 18:19:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622746; x=1741227546; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Dwfv1w4tsOCmi3NCgULhIMD4IZcsOFfau2ULDYaMRLg=; b=Zxx7mXR/tph5hMWT+eMpiZUfB7LGnjo3vm++s+G/pOqzIlPRSB21YJk9W9ItALFEsc LDGTNDirm9YWggc4IQQxPUP/RG8Z17EYC9PV+t2EWJDiHFXh50A0CoWUK3ESCZxHm9kc /oQV/J94l19TNCfKU7fpofF0vP2Pj58nrs+8c8tHYJx7y5LQw4jG20WBvQveWqTLrWcv nOOmiQt48iS/OG4IrYpYzkOgd8OL2zvZaj1Pq0inJDoH86KwywJZ4UmvkDBphA7QYDpk taIKCdFuviMzaNpm9JaXKFYWdr8I/6psRCnzM4jBRT1DropZT+srUYkuL54B81g+vn2t a4fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622746; x=1741227546; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Dwfv1w4tsOCmi3NCgULhIMD4IZcsOFfau2ULDYaMRLg=; b=EH9aLHlXO3X3r+TnQjfJh/txxfzE4vHhmvuyE5BGVEx2iINn9uE5n/iufLDOU1ozGo 1pggyr1YOfQGmqUQfAhKKDIWUjX2HEmgVvDBdBPFAtbqi4mjQkbpy2wwlbAbZtx7qlZd EKYVZSoehXqkgT7B8SXGKS2Wz4DA90Zp5CMhQKhoIeZEGKjnb1J4WX+Eg8gB4hDktdDa fJ2wGwdgEM2D93rvm+1FUgMIlSXaf/BnteVO9fbsZDFg4Ef3Db0C3SWKYb1D9ZNSqujR gxi6zZs2JJQ6szL/OxpxNKX5gnIquaiGT3KxGaFgXWhtm0q2+vtv46b347NsGu4NNaDL I22A== X-Forwarded-Encrypted: i=1; AJvYcCXnPmo4iLrdC1uEOltBnDI690XBcw0wQgA+B3dKVHvNsbNilOtYJL2G+pIw8mtq9x8Hdhg=@vger.kernel.org X-Gm-Message-State: AOJu0YzXu69kL3Fg1zFzOsOegxttJ1RGIY9EfEfwFFFrsUuvGyGdRMos UU/A/sQAIe8BjOKoKlHEMre5MQsB7lRrji8rN+YZ65W2kVIqgTRmFQCxh3Zyov+oNMdDJ/evzl/ o6w== X-Google-Smtp-Source: AGHT+IH6wZ/mGewjHTQw2W+siQpDLDG/G2QvagXg40e21ebxNRSjKEBR8DQxe4UjV6eqsE0kkd3QFpygjvI= X-Received: from pjbsn14.prod.google.com ([2002:a17:90b:2e8e:b0:2fc:15bf:92f6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1d83:b0:2fc:3264:3666 with SMTP id 98e67ed59e1d1-2fce7b221c3mr36215820a91.30.1740622746375; Wed, 26 Feb 2025 18:19:06 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:18 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-3-seanjc@google.com> Subject: [PATCH v2 02/38] x86/tsc: Add standalone helper for getting CPU frequency from CPUID From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Extract the guts of cpu_khz_from_cpuid() to a standalone helper that doesn't restrict the usage to Intel CPUs. This will allow sharing the core logic with kvmclock, as (a) CPUID.0x16 may be enumerated alongside kvmclock, and (b) KVM generally doesn't restrict CPUID based on vendor. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/tsc.h | 1 + arch/x86/kernel/tsc.c | 37 +++++++++++++++++++++++-------------- 2 files changed, 24 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index a4d84f721775..c3a14df46327 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -36,6 +36,7 @@ struct cpuid_tsc_info { }; extern int cpuid_get_tsc_info(struct cpuid_tsc_info *info); extern int cpuid_get_tsc_freq(struct cpuid_tsc_info *info); +extern int cpuid_get_cpu_freq(unsigned int *cpu_khz); extern void tsc_early_init(void); extern void tsc_init(void); diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 93713eb81f52..bb4619148161 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -688,6 +688,24 @@ int cpuid_get_tsc_freq(struct cpuid_tsc_info *info) return 0; } +int cpuid_get_cpu_freq(unsigned int *cpu_khz) +{ + unsigned int eax_base_mhz, ebx, ecx, edx; + + *cpu_khz = 0; + + if (boot_cpu_data.cpuid_level < CPUID_LEAF_FREQ) + return -ENOENT; + + cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx, &ecx, &edx); + + if (!eax_base_mhz) + return -ENOENT; + + *cpu_khz = eax_base_mhz * 1000; + return 0; +} + /** * native_calibrate_tsc - determine TSC frequency * Determine TSC frequency via CPUID, else return 0. @@ -723,13 +741,8 @@ unsigned long native_calibrate_tsc(void) * clock, but we can easily calculate it to a high degree of accuracy * by considering the crystal ratio and the CPU speed. */ - if (!info.crystal_khz && boot_cpu_data.cpuid_level >= CPUID_LEAF_FREQ) { - unsigned int eax_base_mhz, ebx, ecx, edx; - - cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx, &ecx, &edx); - info.crystal_khz = eax_base_mhz * 1000 * - info.denominator / info.numerator; - } + if (!info.crystal_khz && !cpuid_get_cpu_freq(&cpu_khz)) + info.crystal_khz = cpu_khz * info.denominator / info.numerator; if (!info.crystal_khz) return 0; @@ -756,19 +769,15 @@ unsigned long native_calibrate_tsc(void) static unsigned long cpu_khz_from_cpuid(void) { - unsigned int eax_base_mhz, ebx_max_mhz, ecx_bus_mhz, edx; + unsigned int cpu_khz; if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) return 0; - if (boot_cpu_data.cpuid_level < CPUID_LEAF_FREQ) + if (cpuid_get_cpu_freq(&cpu_khz)) return 0; - eax_base_mhz = ebx_max_mhz = ecx_bus_mhz = edx = 0; - - cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx_max_mhz, &ecx_bus_mhz, &edx); - - return eax_base_mhz * 1000; + return cpu_khz; } /* From patchwork Thu Feb 27 02:18:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993548 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8A571AF0C5 for ; Thu, 27 Feb 2025 02:19:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622750; cv=none; b=WdENPH2s2+NUxS++e6y1j4aPLSKQ4QsPp6HPIiD2wOkGMoJPBNvsuaoM6QLI5w6XtUhASivwRJ8W5q1J90Ga0oBPazGH0Wu0/DOSKRFFIBKlAsrqCe4m53bXxCMpdQLZRBqgrM+AW2rVAKT7jYJ1udarfd+pkcRlmHWik9fSuzY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622750; c=relaxed/simple; bh=CRD6OYeqn6lfXavNqHsH0zbcrohxH95CePzgJTbqoqw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=X3ARApN2fnKtw0Ch0q1Tvks800VowGsQgdhUvt59D9pQU8O8tnryi222OpFgFHPZqK6o6KT7O4rMcG1G7+CC2CUr6AeMAIrtIeacYWu16sNfUZl+kvGIf5LvvbZ559Ej4+PXZn7MBZ/t8qDlNAPZ/bGp+VW5pJdUftZy1c+JCmU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=y4kztwTW; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="y4kztwTW" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2f816a85facso1036085a91.3 for ; Wed, 26 Feb 2025 18:19:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622748; x=1741227548; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=VvDX2+gOZciJHv0b670CNF4YevTYL2569oCfzMxNGMY=; b=y4kztwTWlstfeCIGCjXiC1tJ8K5F8ZDfCth+sTY4INMoviIwl6zHL1cJr/+F5tSTyi ci6jusT5E7cWUg2VQpv4lsCUnTZtMy/vhapuQ7o+vsoD/9/AmszcuhWYRXBATZPG+feZ SVZsVrzYHCVJ2GlHbKaSdrUhcFGjlyhVViYopwDV0V/6bhKXzUlZSq3xsm5kCKVsSA31 JU1Ul/80M4tFKXqx859DKFQnh8tUaj3DkNSEGp2WSty+3IB3WJEch1aO2YjCxYur0BXv 3eVLEtAuom23Sn6p92pDeGL6oYd9s1kzDw3bovq4F6aBKnhW/WN86NJZfz2NwFvTg7C7 Uq6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622748; x=1741227548; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VvDX2+gOZciJHv0b670CNF4YevTYL2569oCfzMxNGMY=; b=lUm4S85f5AxXtML9V/FMQKs8M32O0Ari3q+ObrzWg70R+orxRgv92zRf5YjRC3TlYU w4DGXXuKxYsHugrzrOPjcKHh951EwkJTnsC5vkRpoGYtAQ6sCqZrm0dwQUbMh1p/gSOQ tB+2W01droGhgBw3hfwwdkUSTZf8auzNn0NKMxYQUX5R7jslV9azD3g5prVExH2d7OZv sLWXDAtZLM5fblFfAAMV3vuAq9J+5p82zzZVOUw2916bLD8xnXLAyjlLak/5reverzSZ qXhw/YcSO4+x1FasZQJbht2dQDscf10MhnMrxvU2zGCRjCDie8lZdFrYpR0Y/TGG1cOF 8bcQ== X-Forwarded-Encrypted: i=1; AJvYcCUJFMOFNyiP6kvVVVNNxL9N+Q94xRZ1C2JBYbh4p/G7wVDjlHsxeFWD2ZKJoZytxnCajsI=@vger.kernel.org X-Gm-Message-State: AOJu0YyTLo+lehXlp2AIWwBDG8nky39e6Dy+R+NMdmQ19+L01j/3f27c H65gKQSgYF5xiUyXMgpvFN2GHf8KCXUqlz7aOwsvdb2TAeKaihO2tlPv0jXz98D18nZFI8FOUkS CLQ== X-Google-Smtp-Source: AGHT+IFmK8jVkmb1xHfpVWU6EZoqjhev83fKG44NRMRJnNlywOI5c93DmBzKNvUnwRZS9xQkuc4Cdr3SCGs= X-Received: from pjbsz5.prod.google.com ([2002:a17:90b:2d45:b0:2ea:5be5:da6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5148:b0:2ee:c9b6:4c42 with SMTP id 98e67ed59e1d1-2fce86cf0ebmr41779532a91.16.1740622748050; Wed, 26 Feb 2025 18:19:08 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:19 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-4-seanjc@google.com> Subject: [PATCH v2 03/38] x86/tsc: Add helper to register CPU and TSC freq calibration routines From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Add a helper to register non-native, i.e. PV and CoCo, CPU and TSC frequency calibration routines. This will allow consolidating handling of common TSC properties that are forced by hypervisor (PV routines), and will also allow adding sanity checks to guard against overriding a TSC calibration routine with a routine that is less robust/trusted. Make the CPU calibration routine optional, as Xen (very sanely) doesn't assume the CPU runs as the same frequency as the TSC. Wrap the helper in an #ifdef to document that the kernel overrides the native routines when running as a VM, and to guard against unwanted usage. Add a TODO to call out that AMD_MEM_ENCRYPT is a mess and doesn't depend on HYPERVISOR_GUEST because it gates both guest and host code. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Michael Kelley Tested-by: Michael Kelley --- arch/x86/coco/sev/core.c | 4 ++-- arch/x86/include/asm/tsc.h | 4 ++++ arch/x86/kernel/cpu/acrn.c | 4 ++-- arch/x86/kernel/cpu/mshyperv.c | 3 +-- arch/x86/kernel/cpu/vmware.c | 4 ++-- arch/x86/kernel/jailhouse.c | 4 ++-- arch/x86/kernel/kvmclock.c | 4 ++-- arch/x86/kernel/tsc.c | 17 +++++++++++++++++ arch/x86/xen/time.c | 2 +- 9 files changed, 33 insertions(+), 13 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index 82492efc5d94..684cef70edc1 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3291,6 +3291,6 @@ void __init snp_secure_tsc_init(void) rdmsrl(MSR_AMD64_GUEST_TSC_FREQ, tsc_freq_mhz); snp_tsc_freq_khz = (unsigned long)(tsc_freq_mhz * 1000); - x86_platform.calibrate_cpu = securetsc_get_tsc_khz; - x86_platform.calibrate_tsc = securetsc_get_tsc_khz; + tsc_register_calibration_routines(securetsc_get_tsc_khz, + securetsc_get_tsc_khz); } diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index c3a14df46327..9318c74e8d13 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -40,6 +40,10 @@ extern int cpuid_get_cpu_freq(unsigned int *cpu_khz); extern void tsc_early_init(void); extern void tsc_init(void); +#if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) +extern void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), + unsigned long (*calibrate_cpu)(void)); +#endif extern void mark_tsc_unstable(char *reason); extern int unsynchronized_tsc(void); extern int check_tsc_unstable(void); diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index 2c5b51aad91a..c1506cb87d8c 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -29,8 +29,8 @@ static void __init acrn_init_platform(void) /* Install system interrupt handler for ACRN hypervisor callback */ sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); - x86_platform.calibrate_tsc = acrn_get_tsc_khz; - x86_platform.calibrate_cpu = acrn_get_tsc_khz; + tsc_register_calibration_routines(acrn_get_tsc_khz, + acrn_get_tsc_khz); } static bool acrn_x2apic_available(void) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index f285757618fc..aa60491bf738 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -478,8 +478,7 @@ static void __init ms_hyperv_init_platform(void) if (ms_hyperv.features & HV_ACCESS_FREQUENCY_MSRS && ms_hyperv.misc_features & HV_FEATURE_FREQUENCY_MSRS_AVAILABLE) { - x86_platform.calibrate_tsc = hv_get_tsc_khz; - x86_platform.calibrate_cpu = hv_get_tsc_khz; + tsc_register_calibration_routines(hv_get_tsc_khz, hv_get_tsc_khz); setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); } diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index 00189cdeb775..d6f079a75f05 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -416,8 +416,8 @@ static void __init vmware_platform_setup(void) } vmware_tsc_khz = tsc_khz; - x86_platform.calibrate_tsc = vmware_get_tsc_khz; - x86_platform.calibrate_cpu = vmware_get_tsc_khz; + tsc_register_calibration_routines(vmware_get_tsc_khz, + vmware_get_tsc_khz); #ifdef CONFIG_X86_LOCAL_APIC /* Skip lapic calibration since we know the bus frequency. */ diff --git a/arch/x86/kernel/jailhouse.c b/arch/x86/kernel/jailhouse.c index cd8ed1edbf9e..b0a053692161 100644 --- a/arch/x86/kernel/jailhouse.c +++ b/arch/x86/kernel/jailhouse.c @@ -209,8 +209,6 @@ static void __init jailhouse_init_platform(void) x86_init.mpparse.parse_smp_cfg = jailhouse_parse_smp_config; x86_init.pci.arch_init = jailhouse_pci_arch_init; - x86_platform.calibrate_cpu = jailhouse_get_tsc; - x86_platform.calibrate_tsc = jailhouse_get_tsc; x86_platform.get_wallclock = jailhouse_get_wallclock; x86_platform.legacy.rtc = 0; x86_platform.legacy.warm_reset = 0; @@ -220,6 +218,8 @@ static void __init jailhouse_init_platform(void) machine_ops.emergency_restart = jailhouse_no_restart; + tsc_register_calibration_routines(jailhouse_get_tsc, jailhouse_get_tsc); + while (pa_data) { mapping = early_memremap(pa_data, sizeof(header)); memcpy(&header, mapping, sizeof(header)); diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 5b2c15214a6b..b898b95a7d50 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -320,8 +320,8 @@ void __init kvmclock_init(void) flags = pvclock_read_flags(&hv_clock_boot[0].pvti); kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); - x86_platform.calibrate_tsc = kvm_get_tsc_khz; - x86_platform.calibrate_cpu = kvm_get_tsc_khz; + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz); + x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; #ifdef CONFIG_X86_LOCAL_APIC diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index bb4619148161..d65e85929d3e 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1294,6 +1294,23 @@ static void __init check_system_tsc_reliable(void) tsc_disable_clocksource_watchdog(); } +/* + * TODO: Disentangle AMD_MEM_ENCRYPT and make SEV guest support depend on + * HYPERVISOR_GUEST. + */ +#if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) +void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), + unsigned long (*calibrate_cpu)(void)) +{ + if (WARN_ON_ONCE(!calibrate_tsc)) + return; + + x86_platform.calibrate_tsc = calibrate_tsc; + if (calibrate_cpu) + x86_platform.calibrate_cpu = calibrate_cpu; +} +#endif + /* * Make an educated guess if the TSC is trustworthy and synchronized * over all CPUs. diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 96521b1874ac..9e2e900dc0c7 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -566,7 +566,7 @@ static void __init xen_init_time_common(void) static_call_update(pv_steal_clock, xen_steal_clock); paravirt_set_sched_clock(xen_sched_clock); - x86_platform.calibrate_tsc = xen_tsc_khz; + tsc_register_calibration_routines(xen_tsc_khz, NULL); x86_platform.get_wallclock = xen_get_wallclock; } From patchwork Thu Feb 27 02:18:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993549 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EB83190497 for ; Thu, 27 Feb 2025 02:19:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622752; cv=none; b=nz9agBSUH74rqYDGxj6sUVeuIHQRdnoxb/q23kGwUq3JlpLcSvn4TAerLvs3j1zD3lIdf5RtgXPL12pjmp92WmyEd1rbkKXirMhZ6oBmhba0i2VSq5l8VKmovir6rloqQhLrFgXqRyHiWFc72df1uOEGrZ7SuX4YZy8MJbQQchc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622752; c=relaxed/simple; bh=cbbEYbCk5kmrOnFY4+pyu6dZHx8scVaupy5RtKUOy+Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lFynxlUWw0N+OegXqPba1EtfO2WUp+LZC7bAE1Hbh2iBhVEYTF3JKc6LUDO0RLWtmnLJ/mSh5BxrUTUGc6OecxfVug1/5qVG7jcv40oxOPcrgl+lkPacBUdQSmkwsib/2TyhKXeTD6D5kmuzlWRM/c/XsK1l973jTEgEGvTXqFs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ncjvrqUj; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ncjvrqUj" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fe8c5dbdb0so1061019a91.3 for ; Wed, 26 Feb 2025 18:19:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622750; x=1741227550; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=HbJ/1jLoPDWkVBe7cDWaMugTDenciqtu5nrTgQ9Zvm0=; b=ncjvrqUjmwekXY/o6eo006FvZWNuLjcP7sMTBFtsvpzy+wYAIYQmiUVOGEesPBXkgk qi8lChh/EZIbByPRtylkt5u29R8AVGqX3i65f/Oe79ryIK1SY4GA82w14jZFUp3IbDDQ 8FMOcu35hbdaLFWwFdVarqPuOHJY/s6oIGMvgBAoaUWcOFpD/xONrY2JWODhMmKBGGfY IlBDfDsysZ88OeBO1xK4XmvE8gUYGdBPunuO+cYQNY9cuIo32nmNdCnhzYDIyXGryHzk qIMgsmmfDJB68KhtxLWgHSkwsaZ8VdWlfyjFUU/gzDJDzJxrO0NSDuLzetvZDnhl9jeD 9LWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622750; x=1741227550; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HbJ/1jLoPDWkVBe7cDWaMugTDenciqtu5nrTgQ9Zvm0=; b=L8ZuJsI4DvGdkq/pId0HHkST8TcB4dCQAxWJAheZjzSSdKjtQoN2dukYKOeJ2CzN2B 5l1kUzSsXkz7/YWm6FqCKZKEVzMAdfr5+dcPwaQvWhfZ/n6Uyia/CgWFjNgzUnYs23kv TLDvK0omDm7WZzyW56Skjqcm5rDNXdZWZKzzAeY4cU15xI0F+K1GIUML2Kk/y72nhGTq VNA+GK/PGaEQCe+SHX31EzxKYFGK5lP6kJvFUNhlZ9DarogjDXkdJGGxNhCKdmhW6pW7 BMkpir6Ktxz1vt/uEXzEQsLhbq+7JrXLTb1k7KAEnbUWUXRiEK2xFU+KT2Rf0xbiESf0 HszA== X-Forwarded-Encrypted: i=1; AJvYcCUS94eRnMuUqnQ31q+75HmC6B5Iw6peQYibRvRW9n24ODAIPCYTfzxC6TvqoFnF/TGRmN0=@vger.kernel.org X-Gm-Message-State: AOJu0Yz+gfT4YmyZ7wzW+TTUzSSNOCv9okq3r5IfsREf4cdtN3ivOhDZ GZitfjhSZGtFEpqqQeO/KCRQAaFFrHfTLsNPX7142z4KPalDapaLiXpyCw8tydS9GQKzfa87EuL Utw== X-Google-Smtp-Source: AGHT+IFmgEsUJJ1z3bRk5Nf8mtm9mdRWiQdO+4qbvCwBd/hyjI3jP5eNnlt+3d6bfd4x/QDsZrTqSfxiFQY= X-Received: from pjbsw3.prod.google.com ([2002:a17:90b:2c83:b0:2fa:15aa:4d2b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:56cc:b0:2f1:2fa5:1924 with SMTP id 98e67ed59e1d1-2fe7e39f2afmr7133368a91.26.1740622749803; Wed, 26 Feb 2025 18:19:09 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:20 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-5-seanjc@google.com> Subject: [PATCH v2 04/38] x86/sev: Mark TSC as reliable when configuring Secure TSC From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move the code to mark the TSC as reliable from sme_early_init() to snp_secure_tsc_init(). The only reader of TSC_RELIABLE is the aptly named check_system_tsc_reliable(), which runs in tsc_init(), i.e. after snp_secure_tsc_init(). This will allow consolidating the handling of TSC_KNOWN_FREQ and TSC_RELIABLE when overriding the TSC calibration routine. Cc: Tom Lendacky Reviewed-by: Nikunj A Dadhania Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 2 ++ arch/x86/mm/mem_encrypt_amd.c | 3 --- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index 684cef70edc1..e6ce4ca72465 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3288,6 +3288,8 @@ void __init snp_secure_tsc_init(void) return; setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + rdmsrl(MSR_AMD64_GUEST_TSC_FREQ, tsc_freq_mhz); snp_tsc_freq_khz = (unsigned long)(tsc_freq_mhz * 1000); diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c index b56c5c073003..774f9677458f 100644 --- a/arch/x86/mm/mem_encrypt_amd.c +++ b/arch/x86/mm/mem_encrypt_amd.c @@ -541,9 +541,6 @@ void __init sme_early_init(void) * kernel mapped. */ snp_update_svsm_ca(); - - if (sev_status & MSR_AMD64_SNP_SECURE_TSC) - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); } void __init mem_encrypt_free_decrypted_mem(void) From patchwork Thu Feb 27 02:18:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993550 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67B701B653C for ; Thu, 27 Feb 2025 02:19:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622755; cv=none; b=jFbKgCJbk9DFtCyNNdbomJecRDVwedxDWdJqs2wOqtqU+oS8eq3x3lffpJfyE+VCxtizuCvqtXIb9irlq1ocGvQPyAhdbfcZzXrXb7rQ9IUUtfDQN388A67Yas2hE8h/X3P5C7B9CPAjuQeOWR3TWF3wPcoqRAWimdWwxmCzXoo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622755; c=relaxed/simple; bh=zouDyKCvgpfKjJTRdNmQrbmUt1upitIlri45/tL/V2s=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=O/ksbOS+xndi0/zlfH/MM7uL3VPRzrWZ/lylgOf6wWS4j93ot9jdWIRuJXGl0c5O+fmdLOE4X7GwmuG2O0l/fBSwXcPGplCV2Jynb2RUv3TRkPhnULnpp3i3q6aRJpjvgdkJa2e+z8N/QjXVE94ggZTm/44bDJNawhBVVx6iD4E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NYBCZS58; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NYBCZS58" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fc5a9f18afso1085285a91.1 for ; Wed, 26 Feb 2025 18:19:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622752; x=1741227552; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=JV0vL5ay0SjXSCwSvRRqMw5uYQ06F8yr+AS2fSIuB98=; b=NYBCZS58L0f/9PeWx7lqjHczttpcvhByhEy6/htzrG7ZDmbRs4hbvRKAcpiViwVFkp z/ACXB4U3NN5K5CSAcZTmLbUQETJVhXMbu6mKpoBpDewcv9usu16GoOTYsAiaBEUzNOk bW8f5iTVnyfNMEHPxcrkJ24qAJfi9rIocHPwjjklffa3yD7/2WExma/gQ/mT50f/Y4Jk XyZWkX3qHgOMVIYeOay1xgirs2AyZJUmScRp6r+AG+/UpKpXxKGi6VjNKGE3Q/t2OMhC JKXwadCWKCDJ2QXP7DkiIHl0BGP9c4W3B/+pVyY2qBPHf1oYQ9aoieGyGgLdZ8KvT72E ldXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622752; x=1741227552; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=JV0vL5ay0SjXSCwSvRRqMw5uYQ06F8yr+AS2fSIuB98=; b=Re818y6muG6UQfc18NHOUGJPPi9Lsxqjy1cPQ9a9fDjld6+keyLcYXOPCWVxWh6PDB NK62rHfjgLxEGFzIfB4hpIMboelpk9wNIOyrFjrMb+Ar5Z0fIQGdvE4iq4GLy+2rkySj CCVlwkJQM9zFkHfkzFWQe0w/PLSlzE7hvZJd/+glVePeKcn+pLS85lhPtL559TwMnt7x GQrDADCglxBbicL39XmmU+QtQDc8lGTOBGYdZ4PRMVToSSSBZZquOopk/ONb1WsQZnwH C/wG8BAskY4Nb4U4XN1Qi8pAvzbKpPYWNRK3H7h/guXQr9axE3e3oE5tX+v49Eavyj2D jbiw== X-Forwarded-Encrypted: i=1; AJvYcCUyqXqh4hhLq1QFrYTAPtj8BIhKofeSyuCwek/X/5AiTA2VAJyrYwgwU+anSW265Zx221o=@vger.kernel.org X-Gm-Message-State: AOJu0Yx051NAciUaDaX4GVMGPeQ9anQbMK8nDWUmoyEHEPIH8dMJq5Wg Bl6K8piLnGZZQoJhaP4dHed8Vaa8t1eit/UEITibiy+nywNFBoyZkXOSISrAKAeOXmAp/ngMy1m mJg== X-Google-Smtp-Source: AGHT+IEvNph7CiMpObG1u+jVz8JxqMtO0a8+Wef5S19XtPgcpuEdHq4SZt8ZBvsmjyd64opTzInKcYcJX7E= X-Received: from pjbsw14.prod.google.com ([2002:a17:90b:2c8e:b0:2fc:11a0:c546]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3b48:b0:2fc:9967:acd8 with SMTP id 98e67ed59e1d1-2fe7e3b327fmr9422757a91.33.1740622751672; Wed, 26 Feb 2025 18:19:11 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:21 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-6-seanjc@google.com> Subject: [PATCH v2 05/38] x86/sev: Move check for SNP Secure TSC support to tsc_early_init() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move the check on having a Secure TSC to the common tsc_early_init() so that it's obvious that having a Secure TSC is conditional, and to prepare for adding TDX to the mix (blindly initializing *both* SNP and TDX TSC logic looks especially weird). No functional change intended. Cc: Tom Lendacky Reviewed-by: Nikunj A Dadhania Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 3 --- arch/x86/kernel/tsc.c | 3 ++- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index e6ce4ca72465..dab386f782ce 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3284,9 +3284,6 @@ void __init snp_secure_tsc_init(void) { unsigned long long tsc_freq_mhz; - if (!cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)) - return; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index d65e85929d3e..6a011cd1ff94 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1563,7 +1563,8 @@ void __init tsc_early_init(void) if (is_early_uv_system()) return; - snp_secure_tsc_init(); + if (cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)) + snp_secure_tsc_init(); if (!determine_cpu_tsc_frequencies(true)) return; From patchwork Thu Feb 27 02:18:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993551 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43FBC1BEF9B for ; Thu, 27 Feb 2025 02:19:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622756; cv=none; b=LbVX0cdKgXXzS/fbpX46F/pb9dLdSW1X4TCfNlLwY0MGRuyVdLKY4Rsl9BEr/LDYyk86FWxVofPpzqrv3ZnaC/xILxyI2V+i3R0HKxHBMtth8wE1uHLjCewJ7GTB0PO/XmH8SXfZkjv+iv7utgUswIKvJ+6dqpnCs0rPxYhLXV8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622756; c=relaxed/simple; bh=B8tyPzR9PoDE2Pr64h/qBiff7qyIpoEROZEEit/hX78=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=r59Umboe7DF8W2p4C3E0nWmQP7B6NAoygMid+ZUfJmoZOVHEEqfNC7F+VdiL0ycYDViba7n4JBTAwtJclPPlJj9YmD/rB074DCeop/uTi4fPrBa/SB6Z14AqBwEhZrjE8nkIU+XhoLBYEJi6MLhSM2w1rUic950cgE5IDGJ92sA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iiBa51h5; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iiBa51h5" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-220cd43c75aso12649985ad.3 for ; Wed, 26 Feb 2025 18:19:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622753; x=1741227553; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=yramOYZYSvDmgSz8M8zEvByRhd31043Cze2FYqM0HuQ=; b=iiBa51h5gdzIrwfoODSWA2+hBs49NG43uvNXctmg/pM+le/SDpdXfSjNL+8wHp3E0C RVkNsxQJFyyo8GIQd9qQd+xYM27m4Wl/3gEuW0EGVgfwbox+nkSlTAmbPOGfmT7nL42F 1yat36p0E+mz7yuD+kBrdNDwmZG3pN8Y5xdkEDDyM1lJyZBKniA93uGUW03lUIRbrmvm knVtipFF+8jkLdAk2uGtgqJeVNXjOup1FxgYQ2nQrdgSWXTO5lHalAMHrF8K2rYQSwVX KP2J1Wzd2L+GFCmjQIgYcIaR4PeTQ30gkpuJUvU1sd5fcDfxk374FOK4nH81sIo8lfF5 BpHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622753; x=1741227553; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=yramOYZYSvDmgSz8M8zEvByRhd31043Cze2FYqM0HuQ=; b=khrO6CadaxQ5I8mghEOcoHb93XKbzyO1YnDBNdQ95ke70JWOuVxaoSYv82ixeKc2a8 ZCY4KQpqTRW7HeAybVeNIKLmtPjN7R0N4C2zbO+mk9CSovRV4R1/ScHi672mGBDqB3G5 y0RE438mQ43UOEAiprUrnTk+c3hpJZ0dfDhqA6oMhN67b2uX33h5XhHOg9ulmRwkLzD+ 9HPcqW/kEKnO4sKFhBI91NcYxL2HRex9XH+Gv5kFI1wUwzqA8jpo+PZ1W98axQkSZIcJ QfMPpzod2G7jecT0FgygWQ+dlNtxsgQzeVZPqtunHM/dc+cRgpX5TKTxUerVA6skYA4t Bx1w== X-Forwarded-Encrypted: i=1; AJvYcCUB3/NROoZNdWHSmOqCdAJX7Gw4Aa717ZvIbAqm2t71xQOlhkIpljdJ9cIPzMklvpgIyuQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzChP7Bcp6sCPhvUNDPWf+9AX78obsSS5xXnVS1Maz3Th4O+qHk +D0Kwd4LleLS3HTWp3lOd/u1uojJChwNEKS59vdONHHrlg8f6rd7Kg6L/XsVeH71Q/fQs/AbvA0 qXg== X-Google-Smtp-Source: AGHT+IEj/1REcONxv6g8r36LvNpCxOj7lRlC5Gm8UGTSETJJJpMqJ2Q+yEo6DBGNGx6TMjORq5yBRC/aLmE= X-Received: from pfbf4.prod.google.com ([2002:a05:6a00:ad84:b0:730:94db:d304]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:dacc:b0:220:d6ac:b5a3 with SMTP id d9443c01a7336-22320214b94mr83920905ad.51.1740622753535; Wed, 26 Feb 2025 18:19:13 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:22 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-7-seanjc@google.com> Subject: [PATCH v2 06/38] x86/tdx: Override PV calibration routines with CPUID-based calibration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When running as a TDX guest, explicitly override the TSC frequency calibration routine with CPUID-based calibration instead of potentially relying on a hypervisor-controlled PV routine. For TDX guests, CPUID.0x15 is always emulated by the TDX-Module, i.e. the information from CPUID is more trustworthy than the information provided by the hypervisor. To maintain backwards compatibility with TDX guest kernels that use native calibration, and because it's the least awful option, retain native_calibrate_tsc()'s stuffing of the local APIC bus period using the core crystal frequency. While it's entirely possible for the hypervisor to emulate the APIC timer at a different frequency than the core crystal frequency, the commonly accepted interpretation of Intel's SDM is that APIC timer runs at the core crystal frequency when that latter is enumerated via CPUID: The APIC timer frequency will be the processor’s bus clock or core crystal clock frequency (when TSC/core crystal clock ratio is enumerated in CPUID leaf 0x15). If the hypervisor is malicious and deliberately runs the APIC timer at the wrong frequency, nothing would stop the hypervisor from modifying the frequency at any time, i.e. attempting to manually calibrate the frequency out of paranoia would be futile. Deliberately leave the CPU frequency calibration routine as is, since the TDX-Module doesn't provide any guarantees with respect to CPUID.0x16. Opportunistically add a comment explaining that CoCo TSC initialization needs to come after hypervisor specific initialization. Cc: Kirill A. Shutemov Signed-off-by: Sean Christopherson --- arch/x86/coco/tdx/tdx.c | 30 +++++++++++++++++++++++++++--- arch/x86/include/asm/tdx.h | 2 ++ arch/x86/kernel/tsc.c | 8 ++++++++ 3 files changed, 37 insertions(+), 3 deletions(-) diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index 32809a06dab4..42cdaa98dc5e 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -1063,9 +1064,6 @@ void __init tdx_early_init(void) setup_force_cpu_cap(X86_FEATURE_TDX_GUEST); - /* TSC is the only reliable clock in TDX guest */ - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - cc_vendor = CC_VENDOR_INTEL; /* Configure the TD */ @@ -1122,3 +1120,29 @@ void __init tdx_early_init(void) tdx_announce(); } + +static unsigned long tdx_get_tsc_khz(void) +{ + struct cpuid_tsc_info info; + + if (WARN_ON_ONCE(cpuid_get_tsc_freq(&info))) + return 0; + + lapic_timer_period = info.crystal_khz * 1000 / HZ; + + return info.tsc_khz; +} + +void __init tdx_tsc_init(void) +{ + /* TSC is the only reliable clock in TDX guest */ + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + + /* + * Override the PV calibration routines (if set) with more trustworthy + * CPUID-based calibration. The TDX module emulates CPUID, whereas any + * PV information is provided by the hypervisor. + */ + tsc_register_calibration_routines(tdx_get_tsc_khz, NULL); +} diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index b4b16dafd55e..621fbdd101e2 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -53,6 +53,7 @@ struct ve_info { #ifdef CONFIG_INTEL_TDX_GUEST void __init tdx_early_init(void); +void __init tdx_tsc_init(void); void tdx_get_ve_info(struct ve_info *ve); @@ -72,6 +73,7 @@ void __init tdx_dump_td_ctls(u64 td_ctls); #else static inline void tdx_early_init(void) { }; +static inline void tdx_tsc_init(void) { } static inline void tdx_safe_halt(void) { }; static inline bool tdx_early_handle_ve(struct pt_regs *regs) { return false; } diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 6a011cd1ff94..472d6c71d3a5 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -32,6 +32,7 @@ #include #include #include +#include unsigned int __read_mostly cpu_khz; /* TSC clocks / usec, not used here */ EXPORT_SYMBOL(cpu_khz); @@ -1563,8 +1564,15 @@ void __init tsc_early_init(void) if (is_early_uv_system()) return; + /* + * Do CoCo specific "secure" TSC initialization *after* hypervisor + * platform initialization so that the secure variant can override the + * hypervisor's PV calibration routine with a more trusted method. + */ if (cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)) snp_secure_tsc_init(); + else if (boot_cpu_has(X86_FEATURE_TDX_GUEST)) + tdx_tsc_init(); if (!determine_cpu_tsc_frequencies(true)) return; From patchwork Thu Feb 27 02:18:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993552 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBE271C7004 for ; Thu, 27 Feb 2025 02:19:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622757; cv=none; b=JtdkObTdWn5J1KwKwvSsXesyLmIMQkuUZcbV/SFM3FsLlE6lzWTznJIZPSWBcL6qzBH4+HD3Z7lzpuBla4PczEzw33MfIpVWPGbRsvC5q8itpTkgu36+uUgiMdaYGr9nfnLmP+BZfjCkF0akcLA/eqAsDDTGA+bi6lRfTFH06gM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622757; c=relaxed/simple; bh=X5UL+eTktXYoDPfW4LhmVxWuD5wQW055Sxbi/QZyZe8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pj+a6FDK8XCidRGNwhaYhhkM1pu3IzMBVq/cy3qQjND08gcaRqraSiSXFNeDPQMvFepc95KwabBdsMwdPLg3HW8+bkn59H8jNxobXputB3QXYoU9EPU+mU0zfYaJtdS7yC3ivL9z3U4uHMcIGZTonPj7hPQUZbs+xBkH+HQA/7I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fef7FjLd; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fef7FjLd" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc1e7efdffso1555490a91.0 for ; Wed, 26 Feb 2025 18:19:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622755; x=1741227555; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YtBqucRQyVVjFtgUlGbh/U8C9xbSv2L/3t5kwpG/0fs=; b=fef7FjLdAGAJFHAV25KQWsC9t24sfQtyoidwN5PhExU2XZXQEHcBpReguAaITHeA2q euVkzCaUxnh9LnEd/DeVHueiSFgC48U8XOFiALyEo16At6ambnX0VOJMkcDU15wV4hF3 zk8x9Mv5sfkkIB69g9s0MCeaXB9p2k9Fa1M+dFW1pasT5Dy0RIciAUEmWfkcpicm/oY6 F3OJI5txJcOBanKjOnda7GPAL+1dGGB+UxbFzdFJ8pI9PhJ+2zZfIsCmEdqWbr3IPyQ/ zPMgJfwb6fabO18/j/NRfzi/j3VtftBgDU9aJ7Sds2DYxf9xs5dUqTcKuGe1gJOPiBEY XuYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622755; x=1741227555; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YtBqucRQyVVjFtgUlGbh/U8C9xbSv2L/3t5kwpG/0fs=; b=hnbVWbuJDhZl81mid1DLp7D32E3GwVG78+0i8Weh9xPOFrBeUNebv+S5zxkLl7J3SH No/Vvb1NdIHH7eJpz0rJjXuX0q+L8kKxWSA9yBIRZh6uUILUz0o7yjA3oCCj7XKhhDbm sPT7xQUU34ho7JZmogalyPwO9gzSkU2wuJ3ccTEYYez6zPZlS1u5/KlHX8mwBRrPAKAS NXlkc8Aq3cqk+wRBp3BidUMt0EPAhOc24BJDA288Z8H4S7Ahua+bLZS56vi1yTDQxgHb wf4VEC8l/hoz7V+gCNpwXuTRBIE6fw3YLUZePRZJcJ2pMTSmjr9iaZwo32yxM5o7MDzb 7GDQ== X-Forwarded-Encrypted: i=1; AJvYcCWnyMt0osUEgmSQIO/3DgzEDkJGFVEMAE24BBv1hT7zpHLQ1G/1QTxGA6gZKnxlRBSOCTY=@vger.kernel.org X-Gm-Message-State: AOJu0YzAv4oF4qcEsfM7K3cZXsdOTwy+NB7y+XUj8WnqWp25VcaqHOvp suijHegKBPj0Ksd1suUbWspCLa0gRaXRLmz4SZFNW8UfKGtd5rm2mpPPq3aJtzelncSO7D1SzCv gxA== X-Google-Smtp-Source: AGHT+IH9Y0SwBcAS02xHNq+a4IakpnWSLKqVw4P68iCQM2m6waYT1AwLAOPZPnrjzYdSpJiao+kBYkSTr34= X-Received: from pjb4.prod.google.com ([2002:a17:90b:2f04:b0:2ef:78ff:bc3b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2dc8:b0:2fa:ba3:5457 with SMTP id 98e67ed59e1d1-2fe68ae6c4fmr17520974a91.17.1740622755379; Wed, 26 Feb 2025 18:19:15 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:23 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-8-seanjc@google.com> Subject: [PATCH v2 07/38] x86/acrn: Mark TSC frequency as known when using ACRN for calibration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Mark the TSC frequency as known when using ACRN's PV CPUID information. Per commit 81a71f51b89e ("x86/acrn: Set up timekeeping") and common sense, the TSC freq is explicitly provided by the hypervisor. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/acrn.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index c1506cb87d8c..2da3de4d470e 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -29,6 +29,7 @@ static void __init acrn_init_platform(void) /* Install system interrupt handler for ACRN hypervisor callback */ sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); tsc_register_calibration_routines(acrn_get_tsc_khz, acrn_get_tsc_khz); } From patchwork Thu Feb 27 02:18:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993553 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACDFC1D63CC for ; Thu, 27 Feb 2025 02:19:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622759; cv=none; b=q7HFxVe7zYpoK7JA46jQ/Vr56MJvkFvyKUxjWVBkBnHofBtSTZnmcGChJLLOev8JcInVDQOau6pOtq4t0fEX0EfSLUlJ8CuIkeDpfaFtAYG8b0i4aE2j58M8p6dSszt9vm7KrRpvNB8XYcaVTlU0foDNYzoBqMpj7YanFfQXASs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622759; c=relaxed/simple; bh=EP6PsRCf87GzV1IBCG1heWk9cTGrnzInffXbkT0E3+Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=T7Xzi4mj2vZV/spPqWZX6Y/fgrwNlIqJn4m2vICpCCKnIGP89QuIzYsbWx0Ge0y14EKSphM+JHUurFZA10ZMRm41Wd7gGM8c+gHQcYE/ceFttRmxvNqw/wz7lvZ2hDsvmLBxtlY/N3buGk8n0Da3UvmneNAnu9D8uhAdeqsWDtk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NZeXZceS; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NZeXZceS" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc5a9f18afso1085380a91.1 for ; Wed, 26 Feb 2025 18:19:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622757; x=1741227557; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=xfyweVUhHjn5qLEM1jkmIOD/E67k0zFXQwB6BCCQHG8=; b=NZeXZceScV3GIGfgYEDo9NfJem4onudXg3iRAnNcFKD6lFQeaVgGALJK/BjOiWbPIh 9yS9s+jrMy7qBnGhkxMP5XpQHsNc0oIgW28xsqn6bNFU3Ih8QXaaBJkuJQ2vL/Korifg z87H+KjeqQxEq/BIffnglVcqEWlFL6V0ldXECM+HArU+yq8pWGvKk1yV26aOAr99bvsG BvR2IxqtY5p+4SIHDFogK5W3r9H6kVzFKXJv4sZ1LBiaL6BlGma0thlqXTAM7mFe5sAx WIVtTmTj0rde4eO3+9oCdaaLP4EqxSwJL7m1fV7vJUlFK2GvaSluBeXi0KNBvZGJa0Fz 3bkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622757; x=1741227557; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xfyweVUhHjn5qLEM1jkmIOD/E67k0zFXQwB6BCCQHG8=; b=VCo2Cv7eB6f4DOEgQHImg71MT6GyuRQgRa6HpZC1fqTa2ynG5WpVgcuOM+etH4oNqS SXeRQW+dbcOyRbe87TX9lRYNnstKT1K2uDucqK0g1+q/zrDmbAtyyghNl8jmjksIBIUW m0VTokvepjSTC5Zg82xWVs5bJWjbpZ05A/iig7MihjL8MGDUsPj4LZq6wmZwOoG1LGOZ ug1AHUL9D9AN2NTEhZ2CCpDcIhb7XHt9RKfkhhhjbtWYzvrP7Dlb0CtYqvTZDoQFT+mG S5nYq3eE1+dcW/ThRZDl7CaQ+5EM6gckHMJ1FjTnLIiXfJut2x0hDs+Xt7tn8zF2KkiA KNUw== X-Forwarded-Encrypted: i=1; AJvYcCVCkVh4xkXAD2Bvy4F122lIQz0LyQ00oA8jCKyVXfHVFxi4LorKD3/veb8T/hKYNzSLre0=@vger.kernel.org X-Gm-Message-State: AOJu0YyasOGj0AJX5M5MYt0l+DRUkN6WSIpPWZarQAbdU/YZf1iruf4g 9+8Zv74h8yynQ5UNFOMx5CrBxDDdZmjNvrN1pq8a+hA/5Qwweof9Yy4osPrXNltkSBW9BN59FMa CPg== X-Google-Smtp-Source: AGHT+IGK7T2uVtHCkzgg9idpaqI5CgjOHQ8lJUTz55dy8O8u0q+xXKgUvujnuayeBIrgMdHIgLk5Hrx+hJs= X-Received: from pjbta8.prod.google.com ([2002:a17:90b:4ec8:b0:2fa:1771:e276]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2d48:b0:2ee:d7d3:3019 with SMTP id 98e67ed59e1d1-2fe7e31f7c1mr10280973a91.12.1740622757149; Wed, 26 Feb 2025 18:19:17 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:24 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-9-seanjc@google.com> Subject: [PATCH v2 08/38] clocksource: hyper-v: Register sched_clock save/restore iff it's necessary From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Register the Hyper-V timer callbacks or saving/restoring its PV sched_clock if and only if the timer is actually being used for sched_clock. Currently, Hyper-V overrides the save/restore hooks if the reference TSC available, whereas the Hyper-V timer code only overrides sched_clock if the reference TSC is available *and* it's not invariant. The flaw is effectively papered over by invoking the "old" save/restore callbacks as part of save/restore, but that's unnecessary and fragile. To avoid introducing more complexity, and to allow for additional cleanups of the PV sched_clock code, move the save/restore hooks and logic into hyperv_timer.c and simply wire up the hooks when overriding sched_clock itself. Note, while the Hyper-V timer code is intended to be architecture neutral, CONFIG_PARAVIRT is firmly x86-only, i.e. adding a small amount of x86 specific code (which will be reduced in future cleanups) doesn't meaningfully pollute generic code. Signed-off-by: Sean Christopherson Reviewed-by: Michael Kelley Tested-by: Michael Kelley --- arch/x86/kernel/cpu/mshyperv.c | 58 ------------------------------ drivers/clocksource/hyperv_timer.c | 50 ++++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 58 deletions(-) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index aa60491bf738..174f6a71c899 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -223,63 +223,6 @@ static void hv_machine_crash_shutdown(struct pt_regs *regs) hyperv_cleanup(); } #endif /* CONFIG_CRASH_DUMP */ - -static u64 hv_ref_counter_at_suspend; -static void (*old_save_sched_clock_state)(void); -static void (*old_restore_sched_clock_state)(void); - -/* - * Hyper-V clock counter resets during hibernation. Save and restore clock - * offset during suspend/resume, while also considering the time passed - * before suspend. This is to make sure that sched_clock using hv tsc page - * based clocksource, proceeds from where it left off during suspend and - * it shows correct time for the timestamps of kernel messages after resume. - */ -static void save_hv_clock_tsc_state(void) -{ - hv_ref_counter_at_suspend = hv_read_reference_counter(); -} - -static void restore_hv_clock_tsc_state(void) -{ - /* - * Adjust the offsets used by hv tsc clocksource to - * account for the time spent before hibernation. - * adjusted value = reference counter (time) at suspend - * - reference counter (time) now. - */ - hv_adj_sched_clock_offset(hv_ref_counter_at_suspend - hv_read_reference_counter()); -} - -/* - * Functions to override save_sched_clock_state and restore_sched_clock_state - * functions of x86_platform. The Hyper-V clock counter is reset during - * suspend-resume and the offset used to measure time needs to be - * corrected, post resume. - */ -static void hv_save_sched_clock_state(void) -{ - old_save_sched_clock_state(); - save_hv_clock_tsc_state(); -} - -static void hv_restore_sched_clock_state(void) -{ - restore_hv_clock_tsc_state(); - old_restore_sched_clock_state(); -} - -static void __init x86_setup_ops_for_tsc_pg_clock(void) -{ - if (!(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE)) - return; - - old_save_sched_clock_state = x86_platform.save_sched_clock_state; - x86_platform.save_sched_clock_state = hv_save_sched_clock_state; - - old_restore_sched_clock_state = x86_platform.restore_sched_clock_state; - x86_platform.restore_sched_clock_state = hv_restore_sched_clock_state; -} #endif /* CONFIG_HYPERV */ static uint32_t __init ms_hyperv_platform(void) @@ -635,7 +578,6 @@ static void __init ms_hyperv_init_platform(void) /* Register Hyper-V specific clocksource */ hv_init_clocksource(); - x86_setup_ops_for_tsc_pg_clock(); hv_vtl_init_platform(); #endif /* diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index f00019b078a7..86a55167bf5d 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -534,10 +534,60 @@ static __always_inline void hv_setup_sched_clock(void *sched_clock) sched_clock_register(sched_clock, 64, NSEC_PER_SEC); } #elif defined CONFIG_PARAVIRT +static u64 hv_ref_counter_at_suspend; +static void (*old_save_sched_clock_state)(void); +static void (*old_restore_sched_clock_state)(void); + +/* + * Hyper-V clock counter resets during hibernation. Save and restore clock + * offset during suspend/resume, while also considering the time passed + * before suspend. This is to make sure that sched_clock using hv tsc page + * based clocksource, proceeds from where it left off during suspend and + * it shows correct time for the timestamps of kernel messages after resume. + */ +static void save_hv_clock_tsc_state(void) +{ + hv_ref_counter_at_suspend = hv_read_reference_counter(); +} + +static void restore_hv_clock_tsc_state(void) +{ + /* + * Adjust the offsets used by hv tsc clocksource to + * account for the time spent before hibernation. + * adjusted value = reference counter (time) at suspend + * - reference counter (time) now. + */ + hv_adj_sched_clock_offset(hv_ref_counter_at_suspend - hv_read_reference_counter()); +} +/* + * Functions to override save_sched_clock_state and restore_sched_clock_state + * functions of x86_platform. The Hyper-V clock counter is reset during + * suspend-resume and the offset used to measure time needs to be + * corrected, post resume. + */ +static void hv_save_sched_clock_state(void) +{ + old_save_sched_clock_state(); + save_hv_clock_tsc_state(); +} + +static void hv_restore_sched_clock_state(void) +{ + restore_hv_clock_tsc_state(); + old_restore_sched_clock_state(); +} + static __always_inline void hv_setup_sched_clock(void *sched_clock) { /* We're on x86/x64 *and* using PV ops */ paravirt_set_sched_clock(sched_clock); + + old_save_sched_clock_state = x86_platform.save_sched_clock_state; + x86_platform.save_sched_clock_state = hv_save_sched_clock_state; + + old_restore_sched_clock_state = x86_platform.restore_sched_clock_state; + x86_platform.restore_sched_clock_state = hv_restore_sched_clock_state; } #else /* !CONFIG_GENERIC_SCHED_CLOCK && !CONFIG_PARAVIRT */ static __always_inline void hv_setup_sched_clock(void *sched_clock) {} From patchwork Thu Feb 27 02:18:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993554 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28FA719E968 for ; Thu, 27 Feb 2025 02:19:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622760; cv=none; b=QTpQSRwbp1tFdTyWTmRNO2mTbSkrwCWxAEwKFIzFYC7LOgrnW0FkSO8dfgn3CrIpg5EL69tvGE1c2IJFS56pP5T9NIb4s3TMGIwcfYRg7URCc+d6VH2ySIrUIXVYn2HmXVf8rIX5E8ABeDFPNAfm5tl1alBWF1NnpjpCwTBA02I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622760; c=relaxed/simple; bh=M+VN5C0boRzeU19KeJ4QoMy1PkpWXwTcpvT2+bfg0Lk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=M1tee7xck3wGMJbSvjJSjHsGyaznwif66K1XuLXGLYRYSgI1mCf2iNv/7eHn8/+PVqE8p/5ZlpbdHylEvlHYMH8Hl8gidR9MWLQTJxLBlonC8W7pHRQiJ9KykX3N+rWjo/xko25q/re34CViOb8EDDT7e4lXCiaA0pbGUx2xQt4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PcQl/qQx; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PcQl/qQx" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2f83e54432dso1545389a91.2 for ; Wed, 26 Feb 2025 18:19:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622759; x=1741227559; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=M9J4vMOsFh+KFESDavQmkJXQ4sHD+OspEOtA7Gz9lOo=; b=PcQl/qQx/3b2oed4omjjhJihgSYATZ8Br5IF4lqggDiSUwykE5khbL2NWnA8RgmNpc xAtdkxs4jIgY/q5ISfp5AWXO/9gicxgI/RDLrWr7vQGu1HsVEFQttaWI71IIM+g6NICt 2ahJrsrNA4qHDYYB07k7utFv1wfcpPQMU2Vck6VZ4uG1LJAF6FLqZLk6XyKh6wbFmXrs A957TvtKJMWtRu2vjGUAkNsNsYWQUCGTyWILV+VAjZds94gVK9Marphz0lTP19i0HTJ9 GBlm9bjD6ymW2TwrfGx6c4YiONQD8knBMAtpTi8YGMMYO/HICap2/IfQbSK6bQdbWGSD qoaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622759; x=1741227559; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=M9J4vMOsFh+KFESDavQmkJXQ4sHD+OspEOtA7Gz9lOo=; b=OHmr0u5scthU2Zwfs3i20Tmlqcuhus/G6mIeSs5CDXogjGkGo54LAPrJnFyRfffEj/ 7hxKn6UHHKL74L6YvP5BXiYDeA03g5ZeNzvRvEjLEyUDH38nUOE2shtNTyYHCZ1CDd8q UAqdirFJpClqpwzYTRj8oTfMnoyZ8LoMEP7rMXn38y4yFd/G2Xq0xmqU9VUO0eyZT/Kp 6b3TdGd+QtUPPzchaTmgbnVprYEwp0zgx5Teor+4ysoTJS8crCi7pTCCU0ge2HKLdxU3 fVTLyC2EyFI8cPd6DXYF5baLMz52M3m3XF6AGxVawXrA7xEuztpVaOIu03oG8ne1l6gw gdNQ== X-Forwarded-Encrypted: i=1; AJvYcCVIkkCG31MRzqWEZr/Eu+UIzl9n24h84sMgNyG75bUzUigXQ23pZXSjDf2HDzlYE+kzxj0=@vger.kernel.org X-Gm-Message-State: AOJu0Yywxy7hFhalhr/p1MdM2tdYyh987s7HgYoiau56yBdYXsQcFbYW 1eKr6sXA0Urk9YxVw2haZGWrIBpBkKMpveUWigXd6kZX+0K7v0KtsUhJN2ThnIn2OTU9XfacbZM gYQ== X-Google-Smtp-Source: AGHT+IHGg2YNnpdN9W11waKGKjVdoMsbSztU341tlOE27g2wibRX2aXDt93sxEvHQRpF0p5gOf3qjCfl60c= X-Received: from pjbsn14.prod.google.com ([2002:a17:90b:2e8e:b0:2fc:15bf:92f6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2dc5:b0:2f6:be57:49d2 with SMTP id 98e67ed59e1d1-2fe7e32b7c8mr10195616a91.17.1740622758747; Wed, 26 Feb 2025 18:19:18 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:25 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-10-seanjc@google.com> Subject: [PATCH v2 09/38] clocksource: hyper-v: Drop wrappers to sched_clock save/restore helpers From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Now that all of the Hyper-V timer sched_clock code is located in a single file, drop the superfluous wrappers for the save/restore flows. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Michael Kelley Tested-by: Michael Kelley --- drivers/clocksource/hyperv_timer.c | 34 +++++------------------------- include/clocksource/hyperv_timer.h | 2 -- 2 files changed, 5 insertions(+), 31 deletions(-) diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index 86a55167bf5d..5a52d0acf31f 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -471,17 +471,6 @@ static void resume_hv_clock_tsc(struct clocksource *arg) hv_set_msr(HV_MSR_REFERENCE_TSC, tsc_msr.as_uint64); } -/* - * Called during resume from hibernation, from overridden - * x86_platform.restore_sched_clock_state routine. This is to adjust offsets - * used to calculate time for hv tsc page based sched_clock, to account for - * time spent before hibernation. - */ -void hv_adj_sched_clock_offset(u64 offset) -{ - hv_sched_clock_offset -= offset; -} - #ifdef HAVE_VDSO_CLOCKMODE_HVCLOCK static int hv_cs_enable(struct clocksource *cs) { @@ -545,12 +534,14 @@ static void (*old_restore_sched_clock_state)(void); * based clocksource, proceeds from where it left off during suspend and * it shows correct time for the timestamps of kernel messages after resume. */ -static void save_hv_clock_tsc_state(void) +static void hv_save_sched_clock_state(void) { + old_save_sched_clock_state(); + hv_ref_counter_at_suspend = hv_read_reference_counter(); } -static void restore_hv_clock_tsc_state(void) +static void hv_restore_sched_clock_state(void) { /* * Adjust the offsets used by hv tsc clocksource to @@ -558,23 +549,8 @@ static void restore_hv_clock_tsc_state(void) * adjusted value = reference counter (time) at suspend * - reference counter (time) now. */ - hv_adj_sched_clock_offset(hv_ref_counter_at_suspend - hv_read_reference_counter()); -} -/* - * Functions to override save_sched_clock_state and restore_sched_clock_state - * functions of x86_platform. The Hyper-V clock counter is reset during - * suspend-resume and the offset used to measure time needs to be - * corrected, post resume. - */ -static void hv_save_sched_clock_state(void) -{ - old_save_sched_clock_state(); - save_hv_clock_tsc_state(); -} + hv_sched_clock_offset -= (hv_ref_counter_at_suspend - hv_read_reference_counter()); -static void hv_restore_sched_clock_state(void) -{ - restore_hv_clock_tsc_state(); old_restore_sched_clock_state(); } diff --git a/include/clocksource/hyperv_timer.h b/include/clocksource/hyperv_timer.h index d48dd4176fd3..a4c81a60f53d 100644 --- a/include/clocksource/hyperv_timer.h +++ b/include/clocksource/hyperv_timer.h @@ -38,8 +38,6 @@ extern void hv_remap_tsc_clocksource(void); extern unsigned long hv_get_tsc_pfn(void); extern struct ms_hyperv_tsc_page *hv_get_tsc_page(void); -extern void hv_adj_sched_clock_offset(u64 offset); - static __always_inline bool hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg, u64 *cur_tsc, u64 *time) From patchwork Thu Feb 27 02:18:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993555 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C25F1BEF8C for ; Thu, 27 Feb 2025 02:19:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622762; cv=none; b=Hg4H/Hp2ad5ZSAZzkrBcZTvbhPmHENyFFl4r2Akruu6EvMErsMpVKvV1abhwmOI5KGkHEs7Q+C1JQdGC63L5NB+Vtu3MkAicvgIMByTQbJdh6eDvXSpmwohSo/hZGlRJGCvTO8N+VtZhfvb4/UYBl2/9Vue8DXvHmAkFrv0udKg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622762; c=relaxed/simple; bh=6BODDAoV1hIIhUhwPZ9d59Xbs0XOkFee+lu9N6Pq9I4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=s7KMQJcJbH+53gEoBDsVXpGRr9nj+4k3/ZYeOi/Cr6ZIB2DpY3M0LW35oL+PLs59e1jKDoeu+f5/PVjIEostJ6eV3U2NT0+fzvMlQ3orBJwu7Q86qs5AxwbRWRB7I//8vp6NyMNUeWrKOcMuiONrmIOQZkH+AYcNy1zvsKK74Ak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jpZ24PzQ; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jpZ24PzQ" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fe86c01e2bso1040524a91.2 for ; Wed, 26 Feb 2025 18:19:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622760; x=1741227560; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=yCluOSGlRhOy+WE1mTtSxuryf3jXdB/g/GtrHZOBhNw=; b=jpZ24PzQQs2wAgA+5FKc2U5L/UPpup1absHMLvmLlaMWDK5mcd0PUDvPqag7ZNil4Z Hk3epljU/B++4funWP4LGil39k0gkIaAtENVkATBszC/xwezk+e56SWxP3IPEue1V7dU w1JdLLb6rVk+kC1nKztfhGIM/hMykf7H0+oyTSiLC+8/3dSspd5rfPnzy77BoIgHBBo0 BowuYaKWSRpAR2oIcqLT475HBspQSNjgH8r3mSYkxCjY2NvyHLTdVRAAH5SRJRMyV24M eRR27I9S7EBMo0RNix1KTsjKkghixtpwm+6d7FH041nYfDhXWgfpLXXpqCAuU8v6arkv VxLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622760; x=1741227560; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yCluOSGlRhOy+WE1mTtSxuryf3jXdB/g/GtrHZOBhNw=; b=IJcghvxvahmMMjKJU64jntboajAdwpzbl5cF8lUnEI0VnSV39+DYWOJzPm+rpmW75n 4kxhBenWETeb1coM6CwQf+VgCZ8HuRIsKKDQfC1dCP659N00YA2qojiTEt6XrQtCmATR VPU/59IzC5V+M2B8798PpfPzBlxNHLM2wB00tvJA61urLtNj/6bBt5P0hn993qDB6e1/ Oz4EBN24SEp3heQYZLvoHkOgFx0z8r2i3fWi6a2MZB1MYn9QgFdPgEPGlSGJ35MzBWcK 6hfpO1OVkBeGZw52oH0NDpkdwe3q+UC6r0c/6gZOGry1FAuzelsPReDz94vdXxSNlKfO IyrA== X-Forwarded-Encrypted: i=1; AJvYcCUPMZQRNoYVQTrS4DTTOmHnniXfdVQEuPfJMWcfMb864vI65BjuoHLoHJYdwbWIiPYoyxo=@vger.kernel.org X-Gm-Message-State: AOJu0YygI8sPMqm1aO4kmXfzk/DK/PBGWncY6BgrsA+Y0nG9kFHJODrE 5oj9Gp+fB4ZoMwbdeUKqz6lCmLgvt4WA6ZOJMMcsIuP4x7XhQtpcrm4D+lDXjDkScjaPmABl/c7 y4g== X-Google-Smtp-Source: AGHT+IEg2qVczQMppIuK2WVUrcQpfIoWQRuX4zIj2kwQnn94y9fLsBQqATNYRMsqTAxp9OZvbzuNSvbgNPw= X-Received: from pgte20.prod.google.com ([2002:a65:6894:0:b0:ae1:49ef:10d4]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6300:8004:b0:1f0:e368:4a48 with SMTP id adf61e73a8af0-1f0e3684a6fmr22807696637.8.1740622760519; Wed, 26 Feb 2025 18:19:20 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:26 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-11-seanjc@google.com> Subject: [PATCH v2 10/38] clocksource: hyper-v: Don't save/restore TSC offset when using HV sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Now that Hyper-V overrides the sched_clock save/restore hooks if and only sched_clock itself is set to the Hyper-V timer, drop the invocation of the "old" save/restore callbacks. When the registration of the PV sched_clock was done separate from overriding the save/restore hooks, it was possible for Hyper-V to clobber the TSC save/restore callbacks without actually switching to the Hyper-V timer. Enabling a PV sched_clock is a one-way street, i.e. the kernel will never revert to using TSC for sched_clock, and so there is no need to invoke the TSC save/restore hooks (and if there was, it belongs in common PV code). Signed-off-by: Sean Christopherson Reviewed-by: Michael Kelley Tested-by: Michael Kelley --- drivers/clocksource/hyperv_timer.c | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index 5a52d0acf31f..4a21874e91b9 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -524,9 +524,6 @@ static __always_inline void hv_setup_sched_clock(void *sched_clock) } #elif defined CONFIG_PARAVIRT static u64 hv_ref_counter_at_suspend; -static void (*old_save_sched_clock_state)(void); -static void (*old_restore_sched_clock_state)(void); - /* * Hyper-V clock counter resets during hibernation. Save and restore clock * offset during suspend/resume, while also considering the time passed @@ -536,8 +533,6 @@ static void (*old_restore_sched_clock_state)(void); */ static void hv_save_sched_clock_state(void) { - old_save_sched_clock_state(); - hv_ref_counter_at_suspend = hv_read_reference_counter(); } @@ -550,8 +545,6 @@ static void hv_restore_sched_clock_state(void) * - reference counter (time) now. */ hv_sched_clock_offset -= (hv_ref_counter_at_suspend - hv_read_reference_counter()); - - old_restore_sched_clock_state(); } static __always_inline void hv_setup_sched_clock(void *sched_clock) @@ -559,10 +552,7 @@ static __always_inline void hv_setup_sched_clock(void *sched_clock) /* We're on x86/x64 *and* using PV ops */ paravirt_set_sched_clock(sched_clock); - old_save_sched_clock_state = x86_platform.save_sched_clock_state; x86_platform.save_sched_clock_state = hv_save_sched_clock_state; - - old_restore_sched_clock_state = x86_platform.restore_sched_clock_state; x86_platform.restore_sched_clock_state = hv_restore_sched_clock_state; } #else /* !CONFIG_GENERIC_SCHED_CLOCK && !CONFIG_PARAVIRT */ From patchwork Thu Feb 27 02:18:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993556 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB1F91A0730 for ; Thu, 27 Feb 2025 02:19:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622764; cv=none; b=sYA7tf7dirSj95kFtTcmDESShImj4aatmT/rj6vpVIL7WDt/1BTqvtyrXIUBozAVet9hWuNKH4RIuhBwmJ0gJ9yYq3pMsFHib7Lj20rnsTJmWqd4RU05rT1ZXMyg3/M8OBMsW3fBVqcMg1tu5UEdHwZ7I0HIVrnX1lE0husWX60= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622764; c=relaxed/simple; bh=YBBFB7GmxuAVG9YiGJjI2SvbfrD7XwOfpFXbtpwvuc8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fcRVqTnQYqZ3yBVr18OjRy/2s6lEDxYO3YxiVKpWvL9WW3e1HHR5l2r6WmtCJM4YL4yPERJ3tvS8PMzHTHfIGZVRCHGbl+9YqM2DwHvWtFnTgpd+rO2JGbVBv1JfXLe3jI3Id74O3Rg8iyz7u6QQxnaOT4CQAV0ciLCbqajjPmY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Mfl5tOLB; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Mfl5tOLB" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fc3e239675so1565643a91.0 for ; Wed, 26 Feb 2025 18:19:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622762; x=1741227562; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=yPGNLLFa/vXB2a0UM8EIYJpWd3IhBx8wcQJSj3RCAHc=; b=Mfl5tOLBxPPbZBxbuE6j27RikeaJMT8bJZcimZRek0AKjVfDveP7flohhWlEDHUCyl j0KsuPelGvo+36ZpDb/YCo0BwozOYknCCdKWkVXMRrNxweDfHuCbvm1XuN06Ak1Zkz9w blLrd7kaGYs7b6at1rblABhB6uZtnQvD2LUsQwVTFmUU+8+MHxPf7ftMmhHT1kJOzPdD m3S8bfaI1lB4xobOmyvZfiEiB1vh0V+DyKNmgB5Cc2zIFrBIVFQjNX2XvQcy9f6hJ3XJ bxGbSP5Mq1oIn5gQs3epb/NuKczYDNh1zwiE6OtGS5cBIIV6bO9bgopTtBMS1aO0nSUI iLjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622762; x=1741227562; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yPGNLLFa/vXB2a0UM8EIYJpWd3IhBx8wcQJSj3RCAHc=; b=Ipv0ftBcqe/Eydw5VciSZEYcaKd2o5I4ldEHNVvurjSKu4MFd2VJszCHL/UuDETHDH QmyAgyuFf04coUU/T206u1cZe5IPKWV5eE8cLImhFy2ZGfBHSLnlrju2lfsuvpml1kvg duIXhBrw+5JncpdBF0iOJnYI5TNCEFNk1LcEqlPGCfdO1sMqI9UZHhV+A26HT2Wl2tyR Rtw7xUnKD/2JCx6Nd3cMBbwTJ6M31XKwkCN4+n+X4BCrilAMcXujdNSWWuHSwfa+g8Cy zw3qSuSu9u3nG/jXccasEOW2pfh0YlaKB7o1kH/ZUap54mEMTbd+Cv0Ln3BKD17zgy/p Y05w== X-Forwarded-Encrypted: i=1; AJvYcCUIgtIW7+aE/HBXCgCL3EiTeM4cdswjyyUFib3y32wMQ27ONek9qIAryV3OeB5Jtn6PWOE=@vger.kernel.org X-Gm-Message-State: AOJu0Yx58fcJZkVktKq/hHW2nWCca3rnlzUG4v+mJsl9OzYQ3FoU0Q1S C+BGwoOoDOWXjRdr7t3Ijd0p1qOZvPPHZuchlL4rl29/skNFNtMzolnAYZMK+CqkFPwA5orKbk/ vQw== X-Google-Smtp-Source: AGHT+IGTG4bhtwk6xF9NkwSwRr4MZ0JMu1OW/uXbFyNTNSUBj4feLwfxPugt+I59qi4711mZsSVJuOvx1D0= X-Received: from pjbqn3.prod.google.com ([2002:a17:90b:3d43:b0:2ea:5469:76c2]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:fa14:b0:2fe:84d6:cdfc with SMTP id 98e67ed59e1d1-2fe84d6cf88mr6952534a91.35.1740622762299; Wed, 26 Feb 2025 18:19:22 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:27 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-12-seanjc@google.com> Subject: [PATCH v2 11/38] x86/kvmclock: Setup kvmclock for secondary CPUs iff CONFIG_SMP=y From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Gate kvmclock's secondary CPU code on CONFIG_SMP, not CONFIG_X86_LOCAL_APIC. Originally, kvmclock piggybacked PV APIC ops to setup secondary CPUs. When that wart was fixed by commit df156f90a0f9 ("x86: Introduce x86_cpuinit.early_percpu_clock_init hook"), the dependency on a local APIC got carried forward unnecessarily. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index b898b95a7d50..80d1a06609c8 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -186,7 +186,7 @@ static void kvm_restore_sched_clock_state(void) kvm_register_clock("primary cpu clock, resume"); } -#ifdef CONFIG_X86_LOCAL_APIC +#ifdef CONFIG_SMP static void kvm_setup_secondary_clock(void) { kvm_register_clock("secondary cpu clock"); @@ -324,7 +324,7 @@ void __init kvmclock_init(void) x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; -#ifdef CONFIG_X86_LOCAL_APIC +#ifdef CONFIG_SMP x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; #endif x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; From patchwork Thu Feb 27 02:18:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993557 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8FF5F1E5200 for ; Thu, 27 Feb 2025 02:19:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622766; cv=none; b=IJwZFxIxMURqA3EdEeWUmDAO2itlNLn6fOwcFq0FuqXO2CN/DY43HZC6wcoKDTWey3N0LWuUtMwSOk0iOy7JvzQj9OCVjkpMLUE1M968qHKjqBmwQIIaCTvUOPHU5gaxWCf2m84+PPZewvAlNsPeN51AdFSzVmSbBFJAdda1H98= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622766; c=relaxed/simple; bh=pVoicbIwuy1mW6yiJjZE1VBeAcXEO4G9pyLaBIqAzFg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LZHSP+qSOwMRvGqc+Ra5JHQomzxad1IazLB4clOXPj/pZRgdge6a7T1KyBONVGFfM98V1xTjfXYBs0pR/ahkIrNvbuKRtWxlOKXKa/GvE9JaLMv1xQplGM+a6K280+uA3Qwnvo+BBSqIpDvjtNPVwfLYNk81A4XvfP6UxrlBzxM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=aSZdGUdG; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="aSZdGUdG" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-220d8599659so8015615ad.0 for ; Wed, 26 Feb 2025 18:19:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622764; x=1741227564; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=pWgviwpmhKNHN5XYXpAWmYugB+1r1OlVvpsKGRVLlVs=; b=aSZdGUdG9PggLvyeJRebhiWHxzWA9PuNRcV4J86i7bawIEtnsWu5J4cddSAVay9h50 jbFCEnj+9zA2ksgeMg/JhZ2auP6arQvICtagr2OHWNBcmkFXDynRLlr8sR3S3MT0Vfhh TcNZjgMS8pk4TE71O66ZLt3iTwFaM9l2eWTtf0YLPo6VGe8xz1ROkOzLR1hV9aAMGQrI 5OuMlBDCcDS7yFtYHkE+GF9dao7CVhb0iYNqa6sQuPIjtU0PmXC004Ssvoo8+HjP4hv4 E5Q82fNKSyIaprKL4z1MRPldJC+NW0tYdlv+tDYphYLLf1hoKp7sHKo6U1Sb4Vm5bDeo yBCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622764; x=1741227564; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pWgviwpmhKNHN5XYXpAWmYugB+1r1OlVvpsKGRVLlVs=; b=E3nM1hrPLwawbrYuCO+st2WCgyNK1KauZ82TRVqEgquniCSmqBxoFa/YuLSP2hlK1+ sJ7rlYQXKvd8bIyW99SJRXnkdU3pLTJ5USvyRmlThRy/+95gBtkJ88eyefXuektSP3Zc lqPkRBwFwitqrek0WfCyECWOcVMOqeS52zW+bSI9MpU+XjfGnwN7NnPFS29Wq3iynI5W rw79FdfMVN3gYCBxMaSwA71fVJIi3Cd2HL9zy565BngIUBTm57cKgM9I1TOFSmmX2gWf muO7eckAzF0mTosa0uwakSOpv9HFw6jA5e6AZwtN0+0+NGSDmouv31Its2c2zyLqXMz5 4Svw== X-Forwarded-Encrypted: i=1; AJvYcCVrUUpdDTsfFLtY1glHD9G62AotyvfCK5N07scJxx4jL451QkBEg16T5QrkeIGSuD3NYgM=@vger.kernel.org X-Gm-Message-State: AOJu0Yz1i2d2wo/qYtuZdnlSy7kul1vfmzU19HtX3GEMCwER3SIoD7fi 1VYvYJwqCfWcWMrI8yQgIwRwO7bpYvxHrZJCRXkzLLzIZvaV+O/iQI15BKSIVBA5YE7lrRWWRWR SyA== X-Google-Smtp-Source: AGHT+IGQazOm0zxmui59w45JCda9qTDjxRSvZp9ylt3tyNT0rDqeQIE65vrdsSLJOsVjaLh1NupfPRbt/dc= X-Received: from plblo7.prod.google.com ([2002:a17:903:4347:b0:21f:429a:36ae]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ecca:b0:220:f06b:318 with SMTP id d9443c01a7336-22320080b32mr91766195ad.14.1740622763961; Wed, 26 Feb 2025 18:19:23 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:28 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-13-seanjc@google.com> Subject: [PATCH v2 12/38] x86/kvm: Don't disable kvmclock on BSP in syscore_suspend() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Don't disable kvmclock on the BSP during syscore_suspend(), as the BSP's clock is NOT restored during syscore_resume(), but is instead restored earlier via the sched_clock restore callback. If suspend is aborted, e.g. due to a late wakeup, the BSP will run without its clock enabled, which "works" only because KVM-the-hypervisor is kind enough to not clobber the shared memory when the clock is disabled. But over time, the BSP's view of time will drift from APs. Plumb in an "action" to KVM-as-a-guest and kvmclock code in preparation for additional cleanups to kvmclock's suspend/resume logic. Fixes: c02027b5742b ("x86/kvm: Disable kvmclock on all CPUs on shutdown") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_para.h | 8 +++++++- arch/x86/kernel/kvm.c | 15 ++++++++------- arch/x86/kernel/kvmclock.c | 31 +++++++++++++++++++++++++------ 3 files changed, 40 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 57bc74e112f2..8708598f5b8e 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -118,8 +118,14 @@ static inline long kvm_sev_hypercall3(unsigned int nr, unsigned long p1, } #ifdef CONFIG_KVM_GUEST +enum kvm_guest_cpu_action { + KVM_GUEST_BSP_SUSPEND, + KVM_GUEST_AP_OFFLINE, + KVM_GUEST_SHUTDOWN, +}; + void kvmclock_init(void); -void kvmclock_disable(void); +void kvmclock_cpu_action(enum kvm_guest_cpu_action action); bool kvm_para_available(void); unsigned int kvm_arch_para_features(void); unsigned int kvm_arch_para_hints(void); diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 7a422a6c5983..866b061ee0d9 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -447,7 +447,7 @@ static void __init sev_map_percpu_data(void) } } -static void kvm_guest_cpu_offline(bool shutdown) +static void kvm_guest_cpu_offline(enum kvm_guest_cpu_action action) { kvm_disable_steal_time(); if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) @@ -455,9 +455,10 @@ static void kvm_guest_cpu_offline(bool shutdown) if (kvm_para_has_feature(KVM_FEATURE_MIGRATION_CONTROL)) wrmsrl(MSR_KVM_MIGRATION_CONTROL, 0); kvm_pv_disable_apf(); - if (!shutdown) + if (action != KVM_GUEST_SHUTDOWN) apf_task_wake_all(); - kvmclock_disable(); + + kvmclock_cpu_action(action); } static int kvm_cpu_online(unsigned int cpu) @@ -713,7 +714,7 @@ static int kvm_cpu_down_prepare(unsigned int cpu) unsigned long flags; local_irq_save(flags); - kvm_guest_cpu_offline(false); + kvm_guest_cpu_offline(KVM_GUEST_AP_OFFLINE); local_irq_restore(flags); return 0; } @@ -724,7 +725,7 @@ static int kvm_suspend(void) { u64 val = 0; - kvm_guest_cpu_offline(false); + kvm_guest_cpu_offline(KVM_GUEST_BSP_SUSPEND); #ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL if (kvm_para_has_feature(KVM_FEATURE_POLL_CONTROL)) @@ -751,7 +752,7 @@ static struct syscore_ops kvm_syscore_ops = { static void kvm_pv_guest_cpu_reboot(void *unused) { - kvm_guest_cpu_offline(true); + kvm_guest_cpu_offline(KVM_GUEST_SHUTDOWN); } static int kvm_pv_reboot_notify(struct notifier_block *nb, @@ -775,7 +776,7 @@ static struct notifier_block kvm_pv_reboot_nb = { #ifdef CONFIG_CRASH_DUMP static void kvm_crash_shutdown(struct pt_regs *regs) { - kvm_guest_cpu_offline(true); + kvm_guest_cpu_offline(KVM_GUEST_SHUTDOWN); native_machine_crash_shutdown(regs); } #endif diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 80d1a06609c8..223e5297f5ee 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -177,8 +177,22 @@ static void kvm_register_clock(char *txt) pr_debug("kvm-clock: cpu %d, msr %llx, %s", smp_processor_id(), pa, txt); } +static void kvmclock_disable(void) +{ + if (msr_kvm_system_time) + native_write_msr(msr_kvm_system_time, 0, 0); +} + static void kvm_save_sched_clock_state(void) { + /* + * Stop host writes to kvmclock immediately prior to suspend/hibernate. + * If the system is hibernating, then kvmclock will likely reside at a + * different physical address when the system awakens, and host writes + * to the old address prior to reconfiguring kvmclock would clobber + * random memory. + */ + kvmclock_disable(); } static void kvm_restore_sched_clock_state(void) @@ -186,6 +200,17 @@ static void kvm_restore_sched_clock_state(void) kvm_register_clock("primary cpu clock, resume"); } +void kvmclock_cpu_action(enum kvm_guest_cpu_action action) +{ + /* + * Don't disable kvmclock on the BSP during suspend. If kvmclock is + * being used for sched_clock, then it needs to be kept alive until the + * last minute, and restored as quickly as possible after resume. + */ + if (action != KVM_GUEST_BSP_SUSPEND) + kvmclock_disable(); +} + #ifdef CONFIG_SMP static void kvm_setup_secondary_clock(void) { @@ -193,12 +218,6 @@ static void kvm_setup_secondary_clock(void) } #endif -void kvmclock_disable(void) -{ - if (msr_kvm_system_time) - native_write_msr(msr_kvm_system_time, 0, 0); -} - static void __init kvmclock_init_mem(void) { unsigned long ncpus; From patchwork Thu Feb 27 02:18:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993558 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6473B1E8334 for ; Thu, 27 Feb 2025 02:19:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622769; cv=none; b=d7oFFpNpibgL9sRanEMi7uPNeQaPJFvtJLtFUQa5PFjrGsJ/N5H5FK5jTsI6m9UD6I8Jvzt8mYyicChTvsfdgEXavTuRyChpjbDPTtO1larT38iWOdECtE4yx57Tguvg/E6dhfhp6t/qiFSW6qcKi3vmfs22TEUWnYImqGWt1a4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622769; c=relaxed/simple; bh=uls1GZWoryKNI+ZdGNLcku3CfoPFk/9lR0JDodIiYUM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Bl3QsYfVF1D3qoYre3six+M1GdFseUN+ija+yvdx31u/wr9QZj+VfUczMwZitx/npsSDmex5iW0QKwl1V695mng7bZAReaX+QP0o4msZ7DOLSvUTAXQp/rL7TW1BqugoleUJuXN7is+UiP+sRHZFmBxojdRZsYkbLIh0x3ncXTA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Nn0/JtXH; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Nn0/JtXH" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2233b764fc8so7602085ad.3 for ; Wed, 26 Feb 2025 18:19:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622766; x=1741227566; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=UfxMQF/rYWh5wJy03aYqaPrcV7f4BOaa6HmKd5c/EfY=; b=Nn0/JtXHxutzJ6n/fUVS4j4h157GScc0h43rNHp66lZcSO+UF+aA0aR3xT8RRpiCck 6uMGqJG9n4RiOb/sgRK1Gy1QspKYb2lPN2P4P64Xv3BdR2RM9FVZn46tIHqOCwlam9Vs TBIRrTc4J/4V3erKpGAiBinbXtFTNYMEDZpSmFkmN31/w1UwnAYOiNjxQmYLe9WiYLUs A2Ib1dGFrr8r0fAOF7AlWISJfwqVZHklIBEyPAPoQIJeNaPbLQqo+dFcMEGXpjdzAqHQ txyQfW6ygJjNTomZTYR/XH1BGRRsNtmJvbSUUMYgSptbbrJMlczObhbePO4HNnVGH8/Q HreA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622766; x=1741227566; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UfxMQF/rYWh5wJy03aYqaPrcV7f4BOaa6HmKd5c/EfY=; b=XobGMYvpiriNt/gdvAoKhfc549LeLuwQYpWmNRckcz+S4ODOHAnzeg9CuvOjpBYn2y ZZl+Fjv054/QabpaE+960KZBhXT+BXSPsIVLj0iPsAeLQM4bHtTCmgbHd6S0jKYpf4JW Hq5OiDiaENOKXuznJehAti5NOYlpVtxeNIIF53dqwaHjA3fCAg6skvVFrhC0NZ8d6J3n KWdwKvYrw98q9YkitRuSw62hkhIqPMCOBrbpjnDPGWCvjAcn2tjnXkdhU4alGAehBGfe AV9NPuZYEEgNFWRYlYjYPIYf0cjV1MIPQmrLMTUZ7cFMI5irs0DVGt+BUYfleLSr7b7A +B9Q== X-Forwarded-Encrypted: i=1; AJvYcCXbkMCA389Nw0mt57uBlIfUKkGdNpbRl0TK4fP/C8z3aw0dmF0/ofO533f3i9V9KtYItxs=@vger.kernel.org X-Gm-Message-State: AOJu0YyrMZxaVrWFvFPI6h/vCHaAhL9ZArYTtWIhkbWRSmjJ0RBAyUUF xHj/UzayC3eMYG+lK5NbH8TAIUm+Kr4yEDQw+e6S4IQ7aIal1nW90iol6HFghKrzpe5CjFqeMTA CDQ== X-Google-Smtp-Source: AGHT+IE7Xj/t7ND33DY7WpQVYCStd1wMVi/xBc5OTLhEqC2e0MCFeJOTyP9t5m80ApVOgGC3SmRws/tVvjE= X-Received: from pjuw3.prod.google.com ([2002:a17:90a:d603:b0:2fc:2b27:9d35]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ec8c:b0:21f:3e2d:7d2e with SMTP id d9443c01a7336-2219ffb8b65mr337766155ad.27.1740622765716; Wed, 26 Feb 2025 18:19:25 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:29 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-14-seanjc@google.com> Subject: [PATCH v2 13/38] x86/paravirt: Move handling of unstable PV clocks into paravirt_set_sched_clock() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move the handling of unstable PV clocks, of which kvmclock is the only example, into paravirt_set_sched_clock(). This will allow modifying paravirt_set_sched_clock() to keep using the TSC for sched_clock in certain scenarios without unintentionally marking the TSC-based clock as unstable. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 7 ++++++- arch/x86/kernel/kvmclock.c | 5 +---- arch/x86/kernel/paravirt.c | 6 +++++- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index 041aff51eb50..cfceabd5f7e1 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -28,7 +28,12 @@ u64 dummy_sched_clock(void); DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); -void paravirt_set_sched_clock(u64 (*func)(void)); +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable); + +static inline void paravirt_set_sched_clock(u64 (*func)(void)) +{ + __paravirt_set_sched_clock(func, true); +} static __always_inline u64 paravirt_sched_clock(void) { diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 223e5297f5ee..aae6fba21331 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -12,7 +12,6 @@ #include #include #include -#include #include #include #include @@ -93,10 +92,8 @@ static noinstr u64 kvm_sched_clock_read(void) static inline void kvm_sched_clock_init(bool stable) { - if (!stable) - clear_sched_clock_stable(); kvm_sched_clock_offset = kvm_clock_read(); - paravirt_set_sched_clock(kvm_sched_clock_read); + __paravirt_set_sched_clock(kvm_sched_clock_read, stable); pr_info("kvm-clock: using sched offset of %llu cycles", kvm_sched_clock_offset); diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 1ccaa3397a67..55c819673a9d 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -85,8 +86,11 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); -void paravirt_set_sched_clock(u64 (*func)(void)) +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable) { + if (!stable) + clear_sched_clock_stable(); + static_call_update(pv_sched_clock, func); } From patchwork Thu Feb 27 02:18:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993559 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BACA1EB1AE for ; Thu, 27 Feb 2025 02:19:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622770; cv=none; b=UOEPAbcc0x3oaLzhOSDHfymoaBAHGI9+eyWUQvUG02mr06syEhkU5RPEzjSnN6E3sE3CkwF6Iz+MDznO3SIPuf52M5go+N+HkbPVTNdw+CfuWeQximqKPRLb4/ePu7a/3IAW6iY6g8FVspxtZHJ19q6G1ezigOgwL2yN7GYstXc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622770; c=relaxed/simple; bh=tKdzx967TkUiaJBdJTLkl553UTH+9SxXCyvJrKuaAw4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=EezCqLlMQO60K8m9ZkhnOkhEsbrsuADE6cLs+YtjMeWzVeWKxNGVo/Eni8x1qMqKJam90Ux3klGhGDqzcQENyxrr7FVjxK2HJormgiC/TQGW2mjVF2Lj9KXqE/7+eDqrX0NvXsbJDEpj0wK5nQBnIjRoiHhdyFN2LC8u83wLp+w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YnC0VEy/; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YnC0VEy/" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc404aaed5so1539533a91.3 for ; Wed, 26 Feb 2025 18:19:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622767; x=1741227567; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=cUIL6SYLHU+yTwrAtTegZ21noDNMwlKuhwHKs3VPMSg=; b=YnC0VEy/pg4gKTdxVW67fJUo2KIQ7M+Pw0Mx6GI+1233vkqHZ0kWdUzXUk3duiwrRW EmwPUzkIr7x8A701mVGqOpmtwP1B6MFwTalSPDYDCYW+2IdJ6daiLUCSx5Qn75umzmUz jy8O4xmRdVj47i+qUDm/31NkEYZenTHTLaS7PQOxSMP19cZqCWbFUj5XPVKCPakZ4QtG T+bFBw7U0wr17PpbcrEzqro7Z4PdBD6McjXnf/x+YQp0XNA4dCkLF41+7sp4m0ha45ck qvygV4xVusaAA2AsripaXNDC4VAn+ueCOJ/e8RN8YRlamm5i2vRnL2uKKZ1mf27GkVan OQ+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622767; x=1741227567; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cUIL6SYLHU+yTwrAtTegZ21noDNMwlKuhwHKs3VPMSg=; b=Ij6le5ltYjnpQu2W0UTt95yM+T9La88taw+Ed6/a8s27CqJSmpUbTfnySd0v3bASKF CLGIuTmn8N/BcWfetjNDtuSvETRAA1vZxYETJIt9PefGD961dwwX3A0saTcZVtXoC+/y myVLqKGUOwOxvYDl9+CBzjDaeN/Urd9M7IuFh+f4LIjTOlgkksPQUBeCmD7N8I+OREsy 9P67ZdUoBdiATUidhNvRa5ezK+qZiGWo/yOgEc+kA9QZ5wmt0WILHiAT86Ls9Xe2qRag AA4W0SDRnp3srW9xTTbWcsZe4pmeeUMU7MQi0ucwq9IYnqLpqopu7QEiJJh7tH0l/gvA T8aQ== X-Forwarded-Encrypted: i=1; AJvYcCV7MkfHqGJ3a1O1Af1/2EVD4kyMmU/TPvdotAeeWG53nyDMjbD7araAnqAI+I3lgPRjVJk=@vger.kernel.org X-Gm-Message-State: AOJu0YzqCQxOdMAjNKn85zEYA7QvgLm1dyoQjOeRpd7Xvt7kyZKRkIG3 KYH3KKHG3b+LWTwwRHl90SPEVRliTIHm23bkDXJ2VxZtbm0Sj7azSHO4hAyL2Lqg82zSPRivM9r mZw== X-Google-Smtp-Source: AGHT+IHyh9LqedkCKW+Gv/tYRFdHEhi/V70ldbMXUa2l8chtlUNDSfd68q5Rf4dBrz6sq443DVBJXoiGfdY= X-Received: from pja6.prod.google.com ([2002:a17:90b:5486:b0:2ef:95f4:4619]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1d44:b0:2f6:f32e:90ac with SMTP id 98e67ed59e1d1-2fe7e2fe521mr9393579a91.11.1740622767576; Wed, 26 Feb 2025 18:19:27 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:30 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-15-seanjc@google.com> Subject: [PATCH v2 14/38] x86/kvmclock: Move sched_clock save/restore helpers up in kvmclock.c From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move kvmclock's sched_clock save/restore helper "up" so that they can (eventually) be referenced by kvm_sched_clock_init(). No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 108 ++++++++++++++++++------------------- 1 file changed, 54 insertions(+), 54 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index aae6fba21331..c78db52ae399 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -70,6 +70,25 @@ static int kvm_set_wallclock(const struct timespec64 *now) return -ENODEV; } +static void kvm_register_clock(char *txt) +{ + struct pvclock_vsyscall_time_info *src = this_cpu_hvclock(); + u64 pa; + + if (!src) + return; + + pa = slow_virt_to_phys(&src->pvti) | 0x01ULL; + wrmsrl(msr_kvm_system_time, pa); + pr_debug("kvm-clock: cpu %d, msr %llx, %s", smp_processor_id(), pa, txt); +} + +static void kvmclock_disable(void) +{ + if (msr_kvm_system_time) + native_write_msr(msr_kvm_system_time, 0, 0); +} + static u64 kvm_clock_read(void) { u64 ret; @@ -90,6 +109,30 @@ static noinstr u64 kvm_sched_clock_read(void) return pvclock_clocksource_read_nowd(this_cpu_pvti()) - kvm_sched_clock_offset; } +static void kvm_save_sched_clock_state(void) +{ + /* + * Stop host writes to kvmclock immediately prior to suspend/hibernate. + * If the system is hibernating, then kvmclock will likely reside at a + * different physical address when the system awakens, and host writes + * to the old address prior to reconfiguring kvmclock would clobber + * random memory. + */ + kvmclock_disable(); +} + +#ifdef CONFIG_SMP +static void kvm_setup_secondary_clock(void) +{ + kvm_register_clock("secondary cpu clock"); +} +#endif + +static void kvm_restore_sched_clock_state(void) +{ + kvm_register_clock("primary cpu clock, resume"); +} + static inline void kvm_sched_clock_init(bool stable) { kvm_sched_clock_offset = kvm_clock_read(); @@ -102,6 +145,17 @@ static inline void kvm_sched_clock_init(bool stable) sizeof(((struct pvclock_vcpu_time_info *)NULL)->system_time)); } +void kvmclock_cpu_action(enum kvm_guest_cpu_action action) +{ + /* + * Don't disable kvmclock on the BSP during suspend. If kvmclock is + * being used for sched_clock, then it needs to be kept alive until the + * last minute, and restored as quickly as possible after resume. + */ + if (action != KVM_GUEST_BSP_SUSPEND) + kvmclock_disable(); +} + /* * If we don't do that, there is the possibility that the guest * will calibrate under heavy load - thus, getting a lower lpj - @@ -161,60 +215,6 @@ static struct clocksource kvm_clock = { .enable = kvm_cs_enable, }; -static void kvm_register_clock(char *txt) -{ - struct pvclock_vsyscall_time_info *src = this_cpu_hvclock(); - u64 pa; - - if (!src) - return; - - pa = slow_virt_to_phys(&src->pvti) | 0x01ULL; - wrmsrl(msr_kvm_system_time, pa); - pr_debug("kvm-clock: cpu %d, msr %llx, %s", smp_processor_id(), pa, txt); -} - -static void kvmclock_disable(void) -{ - if (msr_kvm_system_time) - native_write_msr(msr_kvm_system_time, 0, 0); -} - -static void kvm_save_sched_clock_state(void) -{ - /* - * Stop host writes to kvmclock immediately prior to suspend/hibernate. - * If the system is hibernating, then kvmclock will likely reside at a - * different physical address when the system awakens, and host writes - * to the old address prior to reconfiguring kvmclock would clobber - * random memory. - */ - kvmclock_disable(); -} - -static void kvm_restore_sched_clock_state(void) -{ - kvm_register_clock("primary cpu clock, resume"); -} - -void kvmclock_cpu_action(enum kvm_guest_cpu_action action) -{ - /* - * Don't disable kvmclock on the BSP during suspend. If kvmclock is - * being used for sched_clock, then it needs to be kept alive until the - * last minute, and restored as quickly as possible after resume. - */ - if (action != KVM_GUEST_BSP_SUSPEND) - kvmclock_disable(); -} - -#ifdef CONFIG_SMP -static void kvm_setup_secondary_clock(void) -{ - kvm_register_clock("secondary cpu clock"); -} -#endif - static void __init kvmclock_init_mem(void) { unsigned long ncpus; From patchwork Thu Feb 27 02:18:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993560 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC1401EB5E8 for ; Thu, 27 Feb 2025 02:19:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622772; cv=none; b=h4vrn5awYTu76NKjppJoKux1Y12kz3NmT0u5BvkaHa5GqJJR3HszKS3h14ivZBt9Vq4tHxEWMcuwLcOQGqWg3hZIAEinZX7g9vFwpHvpCJ2VWS4lCISWKV+R4GiJ95KgMkuhdBoVETbSU3+lu5nOtpIB+GS9HPxWxONlkq3uv08= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622772; c=relaxed/simple; bh=qrmPmOcptAmLgQCle3N1kl8WrTK3PJ1AWETKMFIpJ6A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cKeqpnpbn86yAV12l+g0OQdTIgQ5FK2D8YdgYuQDtOkFFXLTNlEo0fGfND9EqkLVdpsBW5i4kFPMdhTdQR+ySHzWE/Dp/kqaYLRB+iJlChCFYNXtiS5rnGEEaYSlNQHNymg4dExRlQdtnlCgvad2YM2ISDfI40ubM5oNfmciGso= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YyyjItTo; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YyyjItTo" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fe8fa38f6eso1026734a91.2 for ; Wed, 26 Feb 2025 18:19:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622769; x=1741227569; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=4/yyaam3CnBmuwJ2ykq85CXFFo+ZsEjkVRvpD/uHncQ=; b=YyyjItToezQre9Sp01FKz3/7BsTNQhUdFKJMw2FCdGitER6cRnXEiuR7eMqIFgv4zC PLy6jO7yEgX0/h7b4T4480H5nNOVyoMeImsS77tMwMKZrrjMNqQIQfPKp0Iv7FumPaQK bWD/p7tGewBNhN/NL3DiD6yQcpmzqEIqrJvJE57/KdS0d2Msc1qMh2xcpYB7Gb+H22iV DUMuS3CztnCUtLtwLuv77Dpel0sFQDymf6jrKezXH3NftvLI46FWf3eLzgvzIpQMyhHU 0/HPDoqeCT1joGoj4juYfoSnNmkbGynN6PXOHEJPfg8feaXrpAMyeWDgvmmYxwEYLTrK VvoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622769; x=1741227569; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4/yyaam3CnBmuwJ2ykq85CXFFo+ZsEjkVRvpD/uHncQ=; b=G/OVngtsfOl2gGoq2dV/RBQMfA8DlBprWTIPc05DeL8VX5EAsz3I1agXxxZGdlB1ij WA4ptIGKTLwam50RdtR/ka2H92pUnAJ/aSNSa0aDCN9EBMsVMJsmFKde+IE9pakmQPaV op7NVZzciqfjgID/EK9NuUZPkVz7m7+S1tGaZ4S/e+DQ+LFbdsZWTL4C5p+Qt4P6yR8f oioIXJdLZNDmHIlnmgNIYXfAx7e6lzva/gvQ2A9vjjzGW0TluRtywVqOmh7zlPaY6c0B xr6gTswps5VZAR39tZcXEqm2UmkJnxD2sAoVa2Al3zOvnIxdtqLekd1n4eREEhnw2iJA mRxQ== X-Forwarded-Encrypted: i=1; AJvYcCUVziQ6VXkvllviZwD/3uB083yO5PVgGT43zF0+9owc9MTcXHwMtZcOATtOTHe59YmTz70=@vger.kernel.org X-Gm-Message-State: AOJu0YzD6YDTiz5lqXQ0fxWTwJje4eJqs0GsUWI9TVgyfVZtMdRpYbIv xYwAJxtjooVesJPQoyTEEU7rAu1cuDU/m6qzTtxqnlKS4aqFQutJa0L4xE9InhQUSMppmdsnV/r T7Q== X-Google-Smtp-Source: AGHT+IHB6t2yNhuRRLSjnQcCsiQss9r1P7c1nIVM64h4xrkl38FU8N4jisiF+hTwqGE4B/+REyTZDinhF2Y= X-Received: from pjbqb10.prod.google.com ([2002:a17:90b:280a:b0:2e0:915d:d594]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:280c:b0:2ee:5bc9:75c7 with SMTP id 98e67ed59e1d1-2fe68ac9330mr14041142a91.5.1740622769336; Wed, 26 Feb 2025 18:19:29 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:31 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-16-seanjc@google.com> Subject: [PATCH v2 15/38] x86/xen/time: Nullify x86_platform's sched_clock save/restore hooks From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Nullify the x86_platform sched_clock save/restore hooks when setting up Xen's PV clock to make it somewhat obvious the hooks aren't used when running as a Xen guest (Xen uses a paravirtualized suspend/resume flow). Signed-off-by: Sean Christopherson --- arch/x86/xen/time.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 9e2e900dc0c7..51eba986cd18 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -565,6 +565,12 @@ static void __init xen_init_time_common(void) xen_sched_clock_offset = xen_clocksource_read(); static_call_update(pv_steal_clock, xen_steal_clock); paravirt_set_sched_clock(xen_sched_clock); + /* + * Xen has paravirtualized suspend/resume and so doesn't use the common + * x86 sched_clock save/restore hooks. + */ + x86_platform.save_sched_clock_state = NULL; + x86_platform.restore_sched_clock_state = NULL; tsc_register_calibration_routines(xen_tsc_khz, NULL); x86_platform.get_wallclock = xen_get_wallclock; From patchwork Thu Feb 27 02:18:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993561 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A18031F583A for ; Thu, 27 Feb 2025 02:19:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622774; cv=none; b=hzX2+DmXmbGdMt6sPHDkPstUloKt52HVCSopGqtdy1OSNi70n0o3UI0U3zzMMwJ/1tFLIkfc+BbMRXhah8sHr9y6rFCcKrdf8Yh3IaTLjhwNaWAEvPtsyDzYEyAydej0INKrqs+elhBtPvcXOSG9ORPBX//59Ap4Rc6RZReksfY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622774; c=relaxed/simple; bh=Ct4a9oRRcb4uCHXZ3lSQwfqmCTAwLfdRFXjQ6iMkxFY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NvO5DnERc1wvYnD/xo3hQ64gVime/CIuM+eYTitCfVc8YrxDVbw3xlJJAxUTugpF81DV0X6UuWGa9Dwqe00hx6G2rmtKt/7QGnFVclEfYxStK3OtZb7Oo3yVEKQqiMXgrB4VgnEFdf6jR5s/krwx7g1PuQdEmGtjE5F6AnccbrY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WLccEHwF; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WLccEHwF" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc404aaed5so1539731a91.3 for ; Wed, 26 Feb 2025 18:19:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622771; x=1741227571; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Cdk6AVEMbYJ/6Avsp+2Egdk5E71fbrI/hFT7BN4L+X8=; b=WLccEHwFFmO1MV/Sx2o9uqoN4m/rnZV4iLX37pdEs5bXz0rgQ4vXgUq2/rLOUh1+oI cgbeDiBarE5LltOCTUEdyqVovJfe88UflAY3dbiRJ1MkUtlTiW9pKU84Y+WjDhA0HrkX nlUkRBI1tI1jKoC7zFN5rKNmoPL38BzvEfE8izv48MQuwfoOHObXX51lSh0v3pG5vK40 rhRbvu3uCCTBiv6nDMlZ3z4XRP7YeLVabhCqNNkQu4MW8kDpZnjkk+/tP02ML9+85/yq LuaY/eX4+szxEMqugecY4mopoDqZTXH41KtN7f4Ih/JMnoG8bsXtNfa8TrPVX/nnwnND qNrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622771; x=1741227571; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Cdk6AVEMbYJ/6Avsp+2Egdk5E71fbrI/hFT7BN4L+X8=; b=VsXQMcyRYqxVc4I34XKgCkz4S2aST218OCUInSW1YGrHFbivO0CXGMrnKaQMCmnk+5 N3OERAMtG0AWZNLSI9pzSpAYqY0MvykAZ/vUJlsmG6Xgi8bslm50wwKgBIBQOwI3sEvn uk7xO7m6o2nkjbY2Pt+pQw1Fjy97/thpl62fodkfm51Vy8NwxzxdwJc5sa1ybmeVFzIk BndRAYqJaRYGNeDEUl0mRGh0ej0jBKUUCZpNs6UVTIEvflS+MjszauSNtohlbveuBDDe uh0yrxDYU0FOh5RHN8aWPidX76QnxVnpPlprHnB27enw33arq6MlWa4enQT2GYAN6LQv Htww== X-Forwarded-Encrypted: i=1; AJvYcCVEf2TM10JeHiDWIjFqAX8bTtTlNB7mykXmhghwaZQhvjvHpF7xXSOuTeqBI4bju1+OQ+E=@vger.kernel.org X-Gm-Message-State: AOJu0YzN/znNgwhKiWXjUn5uWj7vkEYtPAiTWb44RIPP8QVkqSv8ojZE /nvjUwU8nncWYTsZgADypEsJwpiGWeQWk1+cnlwLowoBvessv9ZfwwKcJEyCDgS7L/0U5l4ICyl UQg== X-Google-Smtp-Source: AGHT+IGFILkiLdlR4RAAXuJxcaVfe/0bxh03RUlKL7+1zFXEwiB1qD3NnSdVhpPB8u60HxzWl62A5c233Wg= X-Received: from pjbdb4.prod.google.com ([2002:a17:90a:d644:b0:2fc:2ee0:d385]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5688:b0:2ee:a4f2:b307 with SMTP id 98e67ed59e1d1-2fe7e2eaca6mr8295061a91.4.1740622771119; Wed, 26 Feb 2025 18:19:31 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:32 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-17-seanjc@google.com> Subject: [PATCH v2 16/38] x86/vmware: Nullify save/restore hooks when using VMware's sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Nullify the sched_clock save/restore hooks when using VMware's version of sched_clock. This will allow extending paravirt_set_sched_clock() to set the save/restore hooks, without having to simultaneously change the behavior of VMware guests. Note, it's not at all obvious that it's safe/correct for VMware guests to do nothing on suspend/resume, but that's a pre-existing problem. Leave it for a VMware expert to sort out. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/vmware.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index d6f079a75f05..d6eadb5b37fd 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -344,8 +344,11 @@ static void __init vmware_paravirt_ops_setup(void) vmware_cyc2ns_setup(); - if (vmw_sched_clock) + if (vmw_sched_clock) { paravirt_set_sched_clock(vmware_sched_clock); + x86_platform.save_sched_clock_state = NULL; + x86_platform.restore_sched_clock_state = NULL; + } if (vmware_is_stealclock_available()) { has_steal_clock = true; From patchwork Thu Feb 27 02:18:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993562 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 66EB81F7904 for ; Thu, 27 Feb 2025 02:19:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622776; cv=none; b=rkSux4EP93lqNIYEMUZFxp0jiz+d9HAl4cH1RHa2JYqLGDV/W/Wtr38uEDoPj2ZcP4sMz7F125g0melYSd4XdoKlAEa7cIrMfs/smhxDEM6nLcu2oubquuxIkNdJGVniKAQN15E9ZSvZhChv3aEdo0a/2WbMTv/4HHTBAOEvRZE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622776; c=relaxed/simple; bh=P0O5mmkSnDe0SpsR7bTvnzJu0+xYte77F3QT24wbpm0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=M6KfPdHlJ9TGs4b9RlRhC4enfMKL73f7mFMu773upBam+F9zTpzMq4OQdrCCblSpu/gW6NNiohQwDKBvpaIkAVX1/1miy+cz7EWVpT67L+9pYVqtM9n3hrxmVM5JsoXUvJBQLKJBPeM4LImmt+woFf/32sf7H2oyCUGwobmwsFw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=u0Xr9RMN; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="u0Xr9RMN" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-220caea15ccso12391755ad.2 for ; Wed, 26 Feb 2025 18:19:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622773; x=1741227573; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=iUPGjlm6EABKAwlimnT9Yj0uy9Y0kc3vikfZk6zMyUM=; b=u0Xr9RMNsxVJE42Vyec4L7wv5hwQgKsHPCI3qt4/2eAdpgqeuhscM35oaMtq/TN3P2 0OL5Txkr+v59vkD/mQGlG90xMPNHr40hnGkbTwftTkLVtbOsVSgmfMBTB2/okKnbuO6U I08xs79VLD3mMj0ZX2U1dFjUshA+JEDeSWuZjbvha2ZGCyzE1VmYZVwWkUxNTfFlmEwW +CXuxQD+e14M/diL8RKMkb93imPLoOwAJz/t5B8BqysLfCdpBRmyYfe8JsKHa2ayQ0NZ oCSyKNzdol48LlKRhGIhJrUhnIssbmQX/do7B4kVJDZqguvFRCbsK1mAWt8mim4PFa3z P1mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622773; x=1741227573; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=iUPGjlm6EABKAwlimnT9Yj0uy9Y0kc3vikfZk6zMyUM=; b=kOhIPOwcIR4f6nyOsSbjbrWbfQYuy3/1LYPzpHlMYaGcvUXSqflFKZCteQahos9UW1 R773ZZKArO2wr1UvyQcpBGfkYktT4fiLbcwarCvKI7SnkEXL7cRB+TozUrFBq36kNxYw auhTDH1/dRyRwhXr8lArH7fe5Xdf5sv5TcYhzUt2n4dFs2v7g4qKy5QzXJP781aBd1B1 7SvPA5yRx49diibAA0Xfgnfrvs345XzLjGwTJnNSKXEzwNAiINlFzTsdrrZRwuRcFoxF 2WTOS1sjso3B+Eankubh94+tcqXAg1/Buj0BZxwLPXnHRa7XyPoFWrYrbgsSzE5ECkuJ HXhg== X-Forwarded-Encrypted: i=1; AJvYcCUh4q+kCvnXPOqctYGOvZOOLovyk/tRrChAsYSSDE+Z/i40BH+dLB23pcjv+dyt8RnDMbQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzC4nw8coyMmk8C8kH6MeH0Jiyx2+IlnDryFcCqUoIWmVOw3dz4 QMI8ixbNp+HYW3HMuY4D6041aH/Xr31sqD2osxb7xJiNjjnfCAa7kds2A4oQJw1TYpDNS+sstaT FkQ== X-Google-Smtp-Source: AGHT+IESbVD77xvo7C+ciGzHtrGqGL0Rj+DzLIK2/TaODo5QawXajXsWSJTUc7OSugj7U0H5GgKBaGLNEhI= X-Received: from pfld12.prod.google.com ([2002:a05:6a00:198c:b0:732:7e28:8f7d]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1813:b0:732:6221:7180 with SMTP id d2e1a72fcca58-73426c943c4mr38161657b3a.5.1740622772828; Wed, 26 Feb 2025 18:19:32 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:33 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-18-seanjc@google.com> Subject: [PATCH v2 17/38] x86/tsc: WARN if TSC sched_clock save/restore used with PV sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Now that all PV clocksources override the sched_clock save/restore hooks when overriding sched_clock, WARN if the "default" TSC hooks are invoked when using a PV sched_clock, e.g. to guard against regressions. Signed-off-by: Sean Christopherson --- arch/x86/kernel/tsc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 472d6c71d3a5..5501d76243c8 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -990,7 +990,7 @@ static unsigned long long cyc2ns_suspend; void tsc_save_sched_clock_state(void) { - if (!sched_clock_stable()) + if (!sched_clock_stable() || WARN_ON_ONCE(!using_native_sched_clock())) return; cyc2ns_suspend = sched_clock(); @@ -1010,7 +1010,7 @@ void tsc_restore_sched_clock_state(void) unsigned long flags; int cpu; - if (!sched_clock_stable()) + if (!sched_clock_stable() || WARN_ON_ONCE(!using_native_sched_clock())) return; local_irq_save(flags); From patchwork Thu Feb 27 02:18:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993563 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBEA32135AB for ; Thu, 27 Feb 2025 02:19:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622776; cv=none; b=Dh8GKVKorjnjxP2SObwso2sytyFVkCSZ4hj8KZZrQxPrkzIQYpEtguGPTwc82Rjkj3kUv/nzIoLhYeizFYXf2576g2mX7Mu3JuiqT50rVwnQMC/FK31B2VJOs03sYt/i8KT4Jni09nIgR2MDS5psx7B9XLPrc5gOIVsfn2GNXYU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622776; c=relaxed/simple; bh=MgbzTWQ5O3po+uvgZbmYvyZGhklKbSd0kRHueynvX0o=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QBzKf67yifHeQYhjCTpx6szeGPoTNI75QiEkYQPoIPp0dj+/zA3ZmtQXGTUyWWetwbBJ6XzFJaph5oO696IwLVYy87WXGORSERZvvgBqhO1AKzrdN6Nc/77grA99RoaLczvjXEPcehDucKBuq55rkGYk1hqThPBXvFiccZIw4EI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=esGgKbEa; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="esGgKbEa" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2f2a9f056a8so981929a91.2 for ; Wed, 26 Feb 2025 18:19:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622774; x=1741227574; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=+aPHxZSdvUlWxMuV2uSxypL8fcuaMUyvD+y52Dh+Bv8=; b=esGgKbEaExoCrTB4Y86lgDk+ECdQPl7Ki75ynOfIZdcx2Ni9dwYsEbKiljf5bhFpl8 yxy80BTNOVVSpjHjGxMtsviev/P+8XyqfMVX/8ajoH034qfASBFndqjFShDgrgsctm2q ULqHEFDLmK5own51FQQocur8XQ3q9XYQ7s0XytrLc+W28jICaVHjTUp11fGAVY+Trafh ar7hquqniruLHzG9HxKOqjS9cbx0OGSw5ne2HpQwEhjjwkBZ/WIe85cFhB8+eRNTs21Q WuhasTlOgzlep/Uy4bTrxpXYU5bfbnzMWjdDzZNKkKrSIhlA5SYUiDTaY1AogzQT2IX2 tjvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622774; x=1741227574; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+aPHxZSdvUlWxMuV2uSxypL8fcuaMUyvD+y52Dh+Bv8=; b=UHh3o8CTev/mDX7JukOCioIE22GsrQ+vq0fAOxKTLLbfpW1FS8qRF76qxIKCPVMf2g Ee3MBgZIWqW1uHXBIp2PdnO8o2xUTNZYjipOuvHStF+hYUHSYMR4vZ2/+d3cMSGxEI06 uQgo5iTig+ziBS5zKv7TZ+9tthxQD0ElwhipoMOKfK5BAfU5dm+4Tj6Uqh4DueKG7PWU l8Z7JgAqw/D8A7W2YCfCg4m21682oOG6MzYDKnzD6V3N7/Nt1VRRnHg3foh/PgnS/GEd Rjgv1fit/6srzkjhWyhWLIAVcjZxjFLvYRgsAtUmtO0koANq0p36/2GMZmy1p4STxvcU 04Ow== X-Forwarded-Encrypted: i=1; AJvYcCV8B0rF6FDqycz8PzU0WVP8c3elnGJBjHisakCnvIyaM1FPpPjL3uFwAJ058X35SxQTg9E=@vger.kernel.org X-Gm-Message-State: AOJu0Yx2nIdGVZRTVPAh3fpliKOkTs/8fR6TttmKad2wDeBZew0t+7+1 RTaCACG0IAc8zz+4PDs02Rtjs0uHrAmczQqW0bp9jyrb+FSi7DSia2Bf1YswRgMGrjdHtvq+jKM qgA== X-Google-Smtp-Source: AGHT+IGtJhSaBP2AlM0Njvk3g8X/xMhIC1fqVJrfdIV8pKqmQJT6ad+58HkgFke1VLmpWg41h8WiiOdBO04= X-Received: from pjbta3.prod.google.com ([2002:a17:90b:4ec3:b0:2ea:448a:8cd1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5483:b0:2ee:f076:20f1 with SMTP id 98e67ed59e1d1-2fe7e218ab9mr10711632a91.0.1740622774461; Wed, 26 Feb 2025 18:19:34 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:34 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-19-seanjc@google.com> Subject: [PATCH v2 18/38] x86/paravirt: Pass sched_clock save/restore helpers during registration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Pass in a PV clock's save/restore helpers when configuring sched_clock instead of relying on each PV clock to manually set the save/restore hooks. In addition to bringing sanity to the code, this will allow gracefully "rejecting" a PV sched_clock, e.g. when running as a CoCo guest that has access to a "secure" TSC. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 8 +++++--- arch/x86/kernel/cpu/vmware.c | 7 ++----- arch/x86/kernel/kvmclock.c | 5 ++--- arch/x86/kernel/paravirt.c | 5 ++++- arch/x86/xen/time.c | 5 ++--- drivers/clocksource/hyperv_timer.c | 6 ++---- 6 files changed, 17 insertions(+), 19 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index cfceabd5f7e1..dc26a3c26527 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -28,11 +28,13 @@ u64 dummy_sched_clock(void); DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); -void __paravirt_set_sched_clock(u64 (*func)(void), bool stable); +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)); -static inline void paravirt_set_sched_clock(u64 (*func)(void)) +static inline void paravirt_set_sched_clock(u64 (*func)(void), + void (*save)(void), void (*restore)(void)) { - __paravirt_set_sched_clock(func, true); + __paravirt_set_sched_clock(func, true, save, restore); } static __always_inline u64 paravirt_sched_clock(void) diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index d6eadb5b37fd..399cf3286a60 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -344,11 +344,8 @@ static void __init vmware_paravirt_ops_setup(void) vmware_cyc2ns_setup(); - if (vmw_sched_clock) { - paravirt_set_sched_clock(vmware_sched_clock); - x86_platform.save_sched_clock_state = NULL; - x86_platform.restore_sched_clock_state = NULL; - } + if (vmw_sched_clock) + paravirt_set_sched_clock(vmware_sched_clock, NULL, NULL); if (vmware_is_stealclock_available()) { has_steal_clock = true; diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index c78db52ae399..1ad3878cc1d9 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -136,7 +136,8 @@ static void kvm_restore_sched_clock_state(void) static inline void kvm_sched_clock_init(bool stable) { kvm_sched_clock_offset = kvm_clock_read(); - __paravirt_set_sched_clock(kvm_sched_clock_read, stable); + __paravirt_set_sched_clock(kvm_sched_clock_read, stable, + kvm_save_sched_clock_state, kvm_restore_sched_clock_state); pr_info("kvm-clock: using sched offset of %llu cycles", kvm_sched_clock_offset); @@ -343,8 +344,6 @@ void __init kvmclock_init(void) #ifdef CONFIG_SMP x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; #endif - x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; - x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; kvm_get_preset_lpj(); /* diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 55c819673a9d..9673cd3a3f0a 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -86,12 +86,15 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); -void __paravirt_set_sched_clock(u64 (*func)(void), bool stable) +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)) { if (!stable) clear_sched_clock_stable(); static_call_update(pv_sched_clock, func); + x86_platform.save_sched_clock_state = save; + x86_platform.restore_sched_clock_state = restore; } /* These are in entry.S */ diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 51eba986cd18..3179f850352d 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -564,13 +564,12 @@ static void __init xen_init_time_common(void) { xen_sched_clock_offset = xen_clocksource_read(); static_call_update(pv_steal_clock, xen_steal_clock); - paravirt_set_sched_clock(xen_sched_clock); + /* * Xen has paravirtualized suspend/resume and so doesn't use the common * x86 sched_clock save/restore hooks. */ - x86_platform.save_sched_clock_state = NULL; - x86_platform.restore_sched_clock_state = NULL; + paravirt_set_sched_clock(xen_sched_clock, NULL, NULL); tsc_register_calibration_routines(xen_tsc_khz, NULL); x86_platform.get_wallclock = xen_get_wallclock; diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index 4a21874e91b9..1c4ed9995cb2 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -550,10 +550,8 @@ static void hv_restore_sched_clock_state(void) static __always_inline void hv_setup_sched_clock(void *sched_clock) { /* We're on x86/x64 *and* using PV ops */ - paravirt_set_sched_clock(sched_clock); - - x86_platform.save_sched_clock_state = hv_save_sched_clock_state; - x86_platform.restore_sched_clock_state = hv_restore_sched_clock_state; + paravirt_set_sched_clock(sched_clock, hv_save_sched_clock_state, + hv_restore_sched_clock_state); } #else /* !CONFIG_GENERIC_SCHED_CLOCK && !CONFIG_PARAVIRT */ static __always_inline void hv_setup_sched_clock(void *sched_clock) {} From patchwork Thu Feb 27 02:18:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993564 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D87EB1AA1DA for ; Thu, 27 Feb 2025 02:19:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622779; cv=none; b=Hgcedyv06MwMPyaV8V3T6miYOldHGKpj34FHRl/ncqgRKCESDPFC7S1jWDHLQa0mfk0wY4s+aAg91miHZFSR38ikmrX7grnpo+IOLljgdM8Z+OGp9BRJAbMowD4uqYyPzOIagjZv6n0bsEhOYV7OSYkFM1LxXgkBtychnHrUrbY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622779; c=relaxed/simple; bh=6spaFA6jKPMfGFgmSBLcviB3IyQMP9osLaGs4X9wbRo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=UDcyWqNY5IK3yFwXIhIiJgSOwrTqrXpFJ+rqRB1QVcI+PS8uaaT/7ukqnIYTM0lNbWI2rhorvUqELknkuZLnIyQbDPJUVCTsZuL4nKtrZV6vTxOiOpd9cRasO/3kid071z1fj/G7DoDt/HU8HqguQlqPH5wVn8X2qnMNQwRAHiI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YNGEAb2o; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YNGEAb2o" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-22350e4b7baso5453205ad.3 for ; Wed, 26 Feb 2025 18:19:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622776; x=1741227576; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DqrQU03sQXOQDWLkbiIYCQHYscRa6LsfmxFd/y9lSQI=; b=YNGEAb2orJWT4DaRfs2+oc8Ryoe6Sgg9CpOjlncAktCi0oNOWV4j+zVPu4FHgHvPzm MisCMMOv0sElsSpzQpUX+g6iHQTJkRFheGKZHh2BzOgD/S3Uq5902KKHgqD1i+Hk8al4 sM/fyuLzXj+ruTxCJMwBauBzGZWHNzXuZxveUxZ6vWYnYlUshB+BV7DLon93RCYNc24Q YP0CT/PtIZNkCkUGjJ4l0oo/MmgyB6/Asef0aUoxqNDZ6x3ojL+AcB4fGzxLFgutur5j JPNBai+sqzfURvnZIuSvoO9pnDcmJ/ONdWvZtG97GPGw885SiWIuuzHlgQvT2wQDRk9c ndEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622776; x=1741227576; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DqrQU03sQXOQDWLkbiIYCQHYscRa6LsfmxFd/y9lSQI=; b=MT82X5gkkFV7GQtSmFDRT8QjyXrpNt40amq5ixKARfsewa3PzNnG9hwLamtVEGjPBP ifHVv4Hz6Eg8X1C3NAJvBUW3YQ5LBHUG0a2YVCbn+xs8pGf9czvyhuB82JwQYp11OYyY slkYw9hliiHjQ104omkgU8AU1FWDJ8wAc7JFLaRY4OSV5xMAu1TecoSq2TRei6+IOxBY vgt2Fa/oqSFg3GhgTqoWoz9WQOd8JVVHXNNWUAJVmeObDT0xZUzYn8Ku9lf3SNdf1pRl W/FBZSy5e5xMxbqWQrxlwiGz24rmYaiYBG+IT0hfsM1hrpIExDp3IlKJTLyy9PBtvPki Sq6Q== X-Forwarded-Encrypted: i=1; AJvYcCWJYIJyMhRywHj6w8gi7DA1g4N3SL+1t7S2avSHJz02dm5zDZbPNwz76n0fFYxL0u7+LRk=@vger.kernel.org X-Gm-Message-State: AOJu0YxFjEFj0W2xwJ+u6x2GDeIw6yrEaFkRmlyxaheoY5OgzlTxQabN NRcppLV5IsrtrQ9h/VlVob6S6Pel7vz2Tkks2fmIIyjgU4UfwLAALjFaD0i1X+Y1xKEP85uGVx9 7YA== X-Google-Smtp-Source: AGHT+IEOb0ihEjgZTJgsK50piH41vKgvY42fSlJ0VND1RngWC+UvT9KhzxC1r0flylu86Z/N4Su58D1alpU= X-Received: from pllg7.prod.google.com ([2002:a17:902:7407:b0:223:2747:3d22]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:d58e:b0:21f:140e:2929 with SMTP id d9443c01a7336-22307b52eb5mr158736135ad.15.1740622776273; Wed, 26 Feb 2025 18:19:36 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:35 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-20-seanjc@google.com> Subject: [PATCH v2 19/38] x86/kvmclock: Move kvm_sched_clock_init() down in kvmclock.c From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move kvm_sched_clock_init() "down" so that it can reference the global kvm_clock structure without needing a forward declaration. Opportunistically mark the helper as "__init" instead of "inline" to make its usage more obvious; modern compilers don't need a hint to inline a single-use function, and an extra CALL+RET pair during boot is a complete non-issue. And, if the compiler ignores the hint and does NOT inline the function, the resulting code may not get discarded after boot due lack of an __init annotation. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 1ad3878cc1d9..934ee4a4c6d4 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -133,19 +133,6 @@ static void kvm_restore_sched_clock_state(void) kvm_register_clock("primary cpu clock, resume"); } -static inline void kvm_sched_clock_init(bool stable) -{ - kvm_sched_clock_offset = kvm_clock_read(); - __paravirt_set_sched_clock(kvm_sched_clock_read, stable, - kvm_save_sched_clock_state, kvm_restore_sched_clock_state); - - pr_info("kvm-clock: using sched offset of %llu cycles", - kvm_sched_clock_offset); - - BUILD_BUG_ON(sizeof(kvm_sched_clock_offset) > - sizeof(((struct pvclock_vcpu_time_info *)NULL)->system_time)); -} - void kvmclock_cpu_action(enum kvm_guest_cpu_action action) { /* @@ -302,6 +289,19 @@ static int kvmclock_setup_percpu(unsigned int cpu) return p ? 0 : -ENOMEM; } +static void __init kvm_sched_clock_init(bool stable) +{ + kvm_sched_clock_offset = kvm_clock_read(); + __paravirt_set_sched_clock(kvm_sched_clock_read, stable, + kvm_save_sched_clock_state, kvm_restore_sched_clock_state); + + pr_info("kvm-clock: using sched offset of %llu cycles", + kvm_sched_clock_offset); + + BUILD_BUG_ON(sizeof(kvm_sched_clock_offset) > + sizeof(((struct pvclock_vcpu_time_info *)NULL)->system_time)); +} + void __init kvmclock_init(void) { u8 flags; From patchwork Thu Feb 27 02:18:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993565 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8505E2135AB for ; Thu, 27 Feb 2025 02:19:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622779; cv=none; b=i5KrfAV3f26IoE89vseDmpf9QrQ04Tj3lI0ezFsco4Pf9qh7P4rGK8nCA+spbJu1ascInLq6HJz7GX1/qHs3VZqIpEfzCbsnGmouxkjEhRoYW2MfVdmaBk4whb85tjUbAXHQDf4Wxm2Os7o5lKSMdeyyEwo/ihZivl9zSx/zzME= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622779; c=relaxed/simple; bh=EvFWgyIbvQaWWwWHzVo5uP3rkhFuLRC5lAG1scfhHho=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Lo8IvLS7wjEj1KNv5YgVBpx5VWp14KpBFzXFQoZtSdZo0YHsWPk3+9SPPoFFcoerS9wYC3Mvym4LKFEBEaiE0CeGS0DhXMhE0DN7raGJm9CUZtradL5Gtvefhhxc3w5LuzAfAXoUl43yNmgBrwGNxezNVUkNdnSK5uIWcJLgsuo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=X1zL7se1; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="X1zL7se1" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-220ec5c16e9so7653465ad.1 for ; Wed, 26 Feb 2025 18:19:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622778; x=1741227578; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Fan7xWhbYPUjoe6Gj73DFLAXBBtIUty3w4hEBXyAFt4=; b=X1zL7se16+Y15iD7qpRznCwf0+Gn1F8PWtxotjxAL9p3L6n4taifamPCuKE7/zxX3O DtNHfCZNcJO3cHqpHWFKjTjg3sSyZUCpEL63gb29qQpEgXkinrmfblDIsKPjMfxdZsW0 7ADNYtWcpjiGLiCq5qdtgsG1kSiCk0wL/0g1wX7zNzdiGi7ad7V605XJNP9oNdX5ofgI QVy6AmtBuIOQO5eBtoc6M+IGso7CXkBTP3NZYXC3PdskDEDwxU+jcTuwgEskwRNZ+LqL makxbBwWcd0SjHjffhMjHyK+tJtz7Ka+o51OQIifB9OkT4lIQCJkxzEg/JGJHclOpz1P ZcFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622778; x=1741227578; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Fan7xWhbYPUjoe6Gj73DFLAXBBtIUty3w4hEBXyAFt4=; b=v2rIwWZvwDldAsi8vDwnNwQJlYDZ7rD8NOtL1J46pFpNKVjkXfMiPgXAc1FXpXgTGk 0x+UxV9YjMX9igbofYbmCvKMuDocmSmyDBIXxBOtXkHGNbBsbZJs7SWSMsQ4UXc4ZVwW 78I26tCSPwJD3+Ftjx09m86dbKLpV3nUTov7nWznfnrCR+FkZw+SQX393Ekd+tAAnPYG OXcZ1N2unUcrbmNHEsQqXWPURDKSb807i15AqU62NYOFTKH6GW8QaivaX1AE80qAKXxe mNq6/UxRMQyEu1OQftAVe9nPEKHw+f6IL+dwhC8L5H2Ead5OWr+MPlmHduQoT4DS4gYB 42Rw== X-Forwarded-Encrypted: i=1; AJvYcCUdFDom0ziUS1Tq8AdgIea38BbZ6bBjC87R0SPF/UyqesUOcec2MTfZOa1YLCvPFMWmVRA=@vger.kernel.org X-Gm-Message-State: AOJu0YwR3QaYVgXZoeWeQz6t+KiAWNLUdNoucoFpsvx3lqMEsjk9NHTc /XxupCSNJbu2+B6L6ArbatxzuKb80SfeUBQhSPfqPwAaHPhAr6lcLL/ZTNdbxsIwIJsSHL3sTl6 aLg== X-Google-Smtp-Source: AGHT+IEIDOZ4TbuvBdNqbq0BRf/iBYYL2tXyd/xxECXyr6rznFcwd1mvvnFfD4EHjsDge5bCz6C8+YGYPK0= X-Received: from plbje3.prod.google.com ([2002:a17:903:2643:b0:223:4e55:d29a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e80d:b0:220:f5d7:6405 with SMTP id d9443c01a7336-221a0edbfc3mr358957175ad.16.1740622777972; Wed, 26 Feb 2025 18:19:37 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:36 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-21-seanjc@google.com> Subject: [PATCH v2 20/38] x86/xen/time: Mark xen_setup_vsyscall_time_info() as __init From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Annotate xen_setup_vsyscall_time_info() as being used only during kernel initialization; it's called only by xen_time_init(), which is already tagged __init. Signed-off-by: Sean Christopherson --- arch/x86/xen/time.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 3179f850352d..13e5888c4501 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -441,7 +441,7 @@ void xen_restore_time_memory_area(void) xen_sched_clock_offset = xen_clocksource_read() - xen_clock_value_saved; } -static void xen_setup_vsyscall_time_info(void) +static void __init xen_setup_vsyscall_time_info(void) { struct vcpu_register_time_memory_area t; struct pvclock_vsyscall_time_info *ti; From patchwork Thu Feb 27 02:18:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993566 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4090422B5B8 for ; Thu, 27 Feb 2025 02:19:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622782; cv=none; b=o/Zx1QkA6c6H7ykIXkLUpJpKzfT8nPrwhO8uftY0IBXAgYyxIB4cfOMtsfHL0x5oMfjPdwqubuFbr3LFSxMUEIZ4JPKpGXWE06+IMVAHlCVSMv9FPbizuxzAmnS0Oe/rcy/Q36Q2nWIp4Yc+TdO42L5l9Yp/P8sNZ5UT+/7UeFU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622782; c=relaxed/simple; bh=W77bDJioIxxlTCWtlgTvtUdQUjN+l9y/G05gekf8Apg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=B/lsceSUj0QXRFj3LnnOx4z+XeH5sStRWKo7XLIuuvF6gsGAgujyKipxPQhcd58qaAukjt+0EUxU6xPHSanS9nnN2aB8tLpvCxPdv5rejV2iRujPNytopW2DisfrjTJ/3QaRO8I1Bxh5iXEYGtw/go14k3oBhNMQk3Cev+a4f/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=U+ysUmz6; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="U+ysUmz6" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fe8fa38f6eso1027027a91.2 for ; Wed, 26 Feb 2025 18:19:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622780; x=1741227580; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=kU2lFcUz0RS/25JB9tsyegi1crsFJmJvQFST0gISHao=; b=U+ysUmz6K0zL0GFvBosFujLDT7MKwULHVNue4bITt0NaNhwzgJf1XdP1JpMp4qXUSA scRLReAuNFwgNPIf0JedSMaQ1FCmx9rei6uncVsB2FFh/RELhZdkb+b/omGFrZtvtE/M 8/p+IVNvUChpZQYYNnVDxTQFUG7+AzzePNgYcWrCEf9RDGpW21BDps2vKCpBj/kJy5/R s+j5mevY7qdNugYRay7mn5nvfG0kEaKtmUhgMDzuF3N3EVAQVG1+c4K5BJX60enue39p J5gDTOA1bsKhHm6tHDKbl2Ldsv0KJQ12MQJWgeiqABQj7tFnYhc10XK3f2uSNQpbe8bl hNDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622780; x=1741227580; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kU2lFcUz0RS/25JB9tsyegi1crsFJmJvQFST0gISHao=; b=HjgFRw+x0kS2AiE37rlKs5w9L1T9bZDtPXtT1Nzc0JDexDiZd7S9928qQLAtzFlg2/ ezuMq4xPle2e2zgMAzfwlZoLh1tEQ9Tic8XlcaF/bQwf0PNpaU8Y+zyyVq+1X2RpP0RV jEAAn0HRDy48ERnN+Pz9ejldUsAcHQLPpJ4elWO+BjlVGF5AqJopc0xjn75okKLyiPl+ Wa4Lise+PN3Z2WaVBibrPryDkN9zr486KZpLD2sxKnQHVauoZF3ngFLE/m0Wx9VS2QP2 5qRnloeMoTTPhhrWJ+F7wCgK1Ffvw5+ov6gUnaSn5sPx40fmme66Qlb16N66bKhFwIs9 /9XQ== X-Forwarded-Encrypted: i=1; AJvYcCU9vaQ7YgnW/QfCoB8y2xOz+peCQKnJrgMwzQlotJbRwIrwUjWHK0SRSR/wA0idlf0IsfY=@vger.kernel.org X-Gm-Message-State: AOJu0YwVpHM6xElRNVK7nCyCR+ni4HBlWwZflWCTDFRWuDReusDt/954 fPjKBUHTrgTO9IxkHxyXp+qC6RMxT7gbVnU3QzqJ8xPaHN6d25s6VQOtPCs6ioCyPOpoMDSx13W ccw== X-Google-Smtp-Source: AGHT+IGPqE+K3f21q2ak8FFNHYsAgDpuaP0E98T4gncRvFB5lVR5BPc4qf0azJ1lc37DAZgmd7FgPnilX0Q= X-Received: from pjbsh7.prod.google.com ([2002:a17:90b:5247:b0:2f2:e97a:e77f]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:fc44:b0:2f6:f107:fae6 with SMTP id 98e67ed59e1d1-2fe68cf3f5fmr12813327a91.23.1740622779809; Wed, 26 Feb 2025 18:19:39 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:37 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-22-seanjc@google.com> Subject: [PATCH v2 21/38] x86/pvclock: Mark setup helpers and related various as __init/__ro_after_init From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Now that Xen PV clock and kvmclock explicitly do setup only during init, tag the common PV clock flags/vsyscall variables and their mutators with __init. Signed-off-by: Sean Christopherson --- arch/x86/kernel/pvclock.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c index b3f81379c2fc..a51adce67f92 100644 --- a/arch/x86/kernel/pvclock.c +++ b/arch/x86/kernel/pvclock.c @@ -16,10 +16,10 @@ #include #include -static u8 valid_flags __read_mostly = 0; -static struct pvclock_vsyscall_time_info *pvti_cpu0_va __read_mostly; +static u8 valid_flags __ro_after_init = 0; +static struct pvclock_vsyscall_time_info *pvti_cpu0_va __ro_after_init; -void pvclock_set_flags(u8 flags) +void __init pvclock_set_flags(u8 flags) { valid_flags = flags; } @@ -153,7 +153,7 @@ void pvclock_read_wallclock(struct pvclock_wall_clock *wall_clock, set_normalized_timespec64(ts, now.tv_sec, now.tv_nsec); } -void pvclock_set_pvti_cpu0_va(struct pvclock_vsyscall_time_info *pvti) +void __init pvclock_set_pvti_cpu0_va(struct pvclock_vsyscall_time_info *pvti) { WARN_ON(vclock_was_used(VDSO_CLOCKMODE_PVCLOCK)); pvti_cpu0_va = pvti; From patchwork Thu Feb 27 02:18:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993567 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BE9022C34A for ; Thu, 27 Feb 2025 02:19:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622783; cv=none; b=mwa8bA648KKw25ZNFGzlYLAHCoyEYsNpV3P1JwDBSS8NaeEQB4RCt2q8NafmYOuVh++tYa36O+4Wga9T6e0GWSHGUTcqRbc/+bpTnRyWMHxQPMBnS8Z8bz8ZmDlJvHOm7MoefGNeeag6jkzuRDcPR/oX9tFFGWa2Gn2ch6KMhb4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622783; c=relaxed/simple; bh=xZNJmibFAKYDGISYNbW/hm8YjPy9RQVjkpaMVV1EGPI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fpAidrPfB/JEZAcm2gAxZ8hdf6z2f5/osPSK9sq9ASq9Yc7Ymac3T0PCnzW5D4XfDbne1OYU7ZINTXGpRI05b9uBPWZmss3BvAN+jFtZG1WbmhBdV39xn97wmMkZSgBatX99h+187mZZ2fOYyL5odnkmHaKGMopUvibUmcpRczs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=H6SdjLeU; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="H6SdjLeU" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc45101191so1080102a91.1 for ; Wed, 26 Feb 2025 18:19:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622781; x=1741227581; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=idfxGEE7w4MZK5vL28VhSjtN1o6ADW/NzBB0c7HM3Jg=; b=H6SdjLeU520UkDNHOZ//8MMHEiXmjYKiYsng1MVBe6W0d/x4XD19eza+Tm7qtpX5QC gEkpAbS5YN/zI2qMibEodzaquE4m8PxUZJo8qsoK2pWipEEngCBXIkwhM9RgonwL3NRD z++0ejZAJzx34aAMAGgrUT3fQJl7H+dwhh+egMvqqMiK9Jnd13gZWBxM79+zPY1wRIiv B0yUT+/9XyUO06TuvNE7aTwJnQ1OXhn08A0c0W+teCS0HmHP5CkF3GqPTGoTv9qdhP1P Pllgkn+I9zO4kfOBo73Vm14Emo7DHcUGCMZI63ifRJwoM9Y+0QR1mhYbAVUV2cmFovRn EnEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622781; x=1741227581; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=idfxGEE7w4MZK5vL28VhSjtN1o6ADW/NzBB0c7HM3Jg=; b=a94yhWZdaLlz0zQ9vONQgIsm4/PFbcdIvQQckZbXvLYKMfbpP+ODrgf7fPbwaYOQ4v 9p4BNZh9ExHcHEFkXJ3ExseMVsB5++6q6GAXiLwuLdJsTxVJ7/IEoSWr7jR6ktkvYhbT AIChdY8pQPcDrMUYeAU3CKTx4sVgusEhKJbuDsLL5UN5+ChK2anBwWuOKD3893YO3LUV YuYPvota6RrcwMCLZXAoT8cUg16041xDYjMnjZIR8l6Ti109V1ZdsdkA0A0PPb0YUvVE hQ30cL/8gSz9hEeOTUP1p6v6CLbGRRY0+aHpaVkkpO953WLnKdA4mGHPamaoFXUbbk7e RdKA== X-Forwarded-Encrypted: i=1; AJvYcCXSkz7I/h8EsbXUJDTn/78JvAPTnl/klxSlhW9YOynX08vSI4u7LuKKlbFfYFltc49LPJ8=@vger.kernel.org X-Gm-Message-State: AOJu0Yy0MJxu1XLSFWL5RZQa3P49Q9RFrFluiwnoVLuPSMN2HB/+miVw fEf/99zaKYm5eeUrHxS8ldsDjUDFB/CMxVVzzs8sVTPAFupSrSElByVdd0eTA6lFBE1TIyrx3fh EwA== X-Google-Smtp-Source: AGHT+IHCzPrRKHA3wP3HVgIKrQRSiWCMkFG4NMbhGFfjlCu1Pi9S6BKyZnSxtjm5Zx+28kwNIdNRFoi5Jdo= X-Received: from pjbqi7.prod.google.com ([2002:a17:90b:2747:b0:2f5:4762:e778]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4c04:b0:2fa:20f4:d27f with SMTP id 98e67ed59e1d1-2fce7b23bb5mr32633734a91.32.1740622781602; Wed, 26 Feb 2025 18:19:41 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:38 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-23-seanjc@google.com> Subject: [PATCH v2 22/38] x86/pvclock: WARN if pvclock's valid_flags are overwritten From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania WARN if the common PV clock valid_flags are overwritten; all PV clocks expect that they are the one and only PV clock, i.e. don't guard against another PV clock having modified the flags. Signed-off-by: Sean Christopherson --- arch/x86/kernel/pvclock.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c index a51adce67f92..8d098841a225 100644 --- a/arch/x86/kernel/pvclock.c +++ b/arch/x86/kernel/pvclock.c @@ -21,6 +21,7 @@ static struct pvclock_vsyscall_time_info *pvti_cpu0_va __ro_after_init; void __init pvclock_set_flags(u8 flags) { + WARN_ON(valid_flags); valid_flags = flags; } From patchwork Thu Feb 27 02:18:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993568 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D411C22D4D3 for ; Thu, 27 Feb 2025 02:19:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622785; cv=none; b=fiMLvr+r3k4Jj81V/jvhHfCJjn1FOu9ya0KD/04GCIbmyOIBjKISMBFck9RqEwyw23zZhMhGYGPrDlpcD6ESchq0zBr0T9bdV+D2r9+6RaQlSPkp8CTVeABt4xdqNWN2jaaWL2+Jac6L2l9hpvh4FhNZFTGMjJlFbxbumqbStdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622785; c=relaxed/simple; bh=c8XZYUiY0W1+X8puZluFjrjS+RyDGH52gngoWNRH5Ko=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rl8EhRDyUD6ibQ3lZtS0WL/HOo/UogpIQxCDbkJAESDLmssHWLx+cHGvLPuGvXk3+1TLAkMcRSagr7QxaSSUkTbUDFqtRCaqLANoKBbRsvf6SorYUeaURq8cS+JtG7bGfceI0GvYb8/zmRTf77MdXCZqqLgAILNyt0zcxDhvdOk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iNc5mDBO; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iNc5mDBO" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fc518f0564so1095861a91.2 for ; Wed, 26 Feb 2025 18:19:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622783; x=1741227583; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Za0nk5QeJovZveNmDgTJSzh3R0hb/7p0/gaKYZ4C/Vs=; b=iNc5mDBOH6iCXAVl7fjXBB9NgQAegd+oFXUTQf6c7PPrBAeGbAM/xfNDCJqf2qRxqw bEwsnT1BcjunEHggVxC4KaaBcADtyp41QpGbbZp9/85769HwwPFdK7phSzJxBg1u/xgZ YB9QzyOrvlejcioFW7BpLj298Kkm1gJMskSZ610tAKp7DEMKb4Kf1gpCnoZUupe/JVTC M/ADlrlXbMYVcCf1AuRlKMQbSQLeifzjE/jaZFtR92da7oNnNv44+sEK7wOgaK6ldpHj Ugef1n6qwdO/HftDAw5G6R1yxYGtOVczOGODa98lJye+UP4lLyOErt3pyr9/pJQKM/M/ KlTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622783; x=1741227583; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Za0nk5QeJovZveNmDgTJSzh3R0hb/7p0/gaKYZ4C/Vs=; b=vKxw/rWjDuUlVDKVXZWx6ZlHjkIF2GWPL8I0DxS74GpdieYFQBuOYXejPHqN4JjGVV zGoAWeIejEs+79Dolujqbh1pKO/BzrUKYu0bbqwGvIvNir86ve61/rGAgrHdkvBPLYNh UUR5q7cO5UN/9c4NHdZnFMrG1tzJ6bC1Znxo+mIYk9mTvwsxjLxa+5oICtlZtgEcSPr/ s73IQHvfFoPz1W1Qurze1jYoCokI9isN9wM7YsFkjgJUa5gyyAXkcFCEQQyiUF0aHen3 PTshnXG+GSkcl+VtpZJvMaapW4LZIq0ebWV7OJ2+B4sxZaPvvodtb+jntRT1HD2wT7+u oatw== X-Forwarded-Encrypted: i=1; AJvYcCXRDDLiOO4/bBhyma3XSuSOpDdzCmyLP86h7q8awne6G4ndUcDGOvIAPPhy+Y7n1GqEZOU=@vger.kernel.org X-Gm-Message-State: AOJu0YyLgqAtUxpTlSaxFmDBr1fjQ9kUPh6beRg1bZlsaUAHhR2YlzKO cIXIkxbodu4FNPho2Kk8sVdxQsuABcYNUXs7QmZpk+NcJyXkt1EugKbnVUY2uBlvWISs7Mrktvc 7HA== X-Google-Smtp-Source: AGHT+IHPEy5y9WPVcI/hX8J+CDDBk9ezIawKhLRmwSNwFlW/vj5Sb4uKTsOn6iseYydXRMp+Q/+8QjZLtQQ= X-Received: from pjbsm1.prod.google.com ([2002:a17:90b:2e41:b0:2f8:4024:b59a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:d2d0:b0:2ee:b6c5:1def with SMTP id 98e67ed59e1d1-2fe68ad9ef3mr15165448a91.8.1740622783395; Wed, 26 Feb 2025 18:19:43 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:39 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-24-seanjc@google.com> Subject: [PATCH v2 23/38] x86/kvmclock: Refactor handling of PVCLOCK_TSC_STABLE_BIT during kvmclock_init() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Clean up the setting of PVCLOCK_TSC_STABLE_BIT during kvmclock init to make it somewhat obvious that pvclock_read_flags() must be called *after* pvclock_set_flags(). Note, in theory, a different PV clock could have set PVCLOCK_TSC_STABLE_BIT in the supported flags, i.e. reading flags only if KVM_FEATURE_CLOCKSOURCE_STABLE_BIT is set could very, very theoretically result in a change in behavior. In practice, the kernel only supports a single PV clock. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 934ee4a4c6d4..0580fe1aefa0 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -304,7 +304,7 @@ static void __init kvm_sched_clock_init(bool stable) void __init kvmclock_init(void) { - u8 flags; + bool stable = false; if (!kvm_para_available() || !kvmclock) return; @@ -331,11 +331,18 @@ void __init kvmclock_init(void) kvm_register_clock("primary cpu clock"); pvclock_set_pvti_cpu0_va(hv_clock_boot); - if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT)) + if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT)) { pvclock_set_flags(PVCLOCK_TSC_STABLE_BIT); - flags = pvclock_read_flags(&hv_clock_boot[0].pvti); - kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); + /* + * Check if the clock is stable *after* marking TSC_STABLE as a + * valid flag. + */ + stable = pvclock_read_flags(&hv_clock_boot[0].pvti) & + PVCLOCK_TSC_STABLE_BIT; + } + + kvm_sched_clock_init(stable); tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz); From patchwork Thu Feb 27 02:18:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993569 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9504722D7BB for ; Thu, 27 Feb 2025 02:19:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622788; cv=none; b=OFVnOGpKYHqeddcl9nehtcrDVbOlC+5TmR5re88V9Jq51OI5Jc2yxlUQKhBCwmJvyah1DNUkqVk5MkTlSp5spQ9r6axDF3saPcPBw/K64zqgpdH/ar86Ij3lnnDRvmhilorRHIw/m0Loo2rcVVf0Ums1OKoFBUTMNAztX1AVWiE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622788; c=relaxed/simple; bh=2a8H0eAkRitgl8N78ADmzcHYgYnoAl+7GQjDO0K/gtg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uWv1BwSNgSJQrVsHpVz/uI3tQMK8t5yC4wuE8k2fifamcPPWqWURxq5pKqB4Rr2Q08IFgH9GV0ILx7DN31ANVSu+Ac5EqTYrwRhCTQF0Qfo9BDl4j7njDwmkQken9g3+c+yT5gDZHYG0LnDwmgGr97nKlcYYahZGFEeclLBMKBU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Nm8uHFYu; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Nm8uHFYu" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc1a4c14d4so1064240a91.0 for ; Wed, 26 Feb 2025 18:19:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622785; x=1741227585; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=qN8YI5wmqMG6yVEs+7cu8Sw/AfoU86Z3NaGBSlCYqu0=; b=Nm8uHFYuFCk8Bd2Qof4oqMWXSWGl8WVAxHjMxvQY4OFbVcZmBLVgG/N5xWGh8deco4 bt3AIBG6yct8eJW/F6RmyoZrfmDQLggUH6t+q/debx423fkAmEhwWRJuymWTmpuiiRC7 /P+yt6HaBQxc83xO5w35QbmGvQ55JUQCwglUDc7XDMI04DPOG3ZGPmz6qDLRsEmSuYDj KVUVzKlzVmbvAYldM1erDs/9pX4bSxKfB0thCPvgmlgeQp7OOzUpvNWUMA0gOXktNi7z JF8nv4+x1u1DMTwnkpV3RsVlQoqe4odhcfQenOF1luUdcA/2UCyrLsHh6+53/KrEu8Oi bHuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622785; x=1741227585; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qN8YI5wmqMG6yVEs+7cu8Sw/AfoU86Z3NaGBSlCYqu0=; b=qZgvfcHox60/Kp9ELACsxFqP+GglXNa7a8hmqQxDSXhi5eTGlq7azsxtk3R9JfxX7M UGt2vm9QQK7bjVIRzskcob1fMm19QKizuovnqGV40+eg5/QFEINKdDNM41DKw8Jex8Ea PYOwKuXvR+uKmRT+8fddeIVnY43ZSsxmOGyIVk9dY4DVLMLWKj8+EUsVKVXcVAQDcxjc 7XKZjwOP7qzDPV3idhE2JbLYCNzCLdpmHb2AxewwSTTh3dNEtjNZG+thizS1y8rvf4bH iTQH/S4LSxQmCGtrCfrYjqIGpZARm0S/7EwPGJdf67dD+0vVJPDUnCWYPUAIYBPFWkvz 5TtA== X-Forwarded-Encrypted: i=1; AJvYcCXUdkOvqwmgatVOZ3EU7uv1zrIrJRwaHVIW0DravdmSPA1MUd9TJX9BmCHDLVhIgyZ1/QA=@vger.kernel.org X-Gm-Message-State: AOJu0YxDm9zdDtWELkgWzFU0oznMA5pzbkt9SGBfb51YrQYXmgxl0Rsc 9vnlPg+sRFO546y7zR/Tu836/qIgBdbek+kQBfHsa0Unt3DbJMsI4EvzSWseSDK+NhwYD0yc/3+ Vwg== X-Google-Smtp-Source: AGHT+IEw7VZRln4iIrg1rWM/7IZnuYkNWSUUNxKKGfY9HmA7vrLySNMf4INY1k647TDU8xAK61s1yi1H724= X-Received: from pjbsw12.prod.google.com ([2002:a17:90b:2c8c:b0:2fa:284f:adae]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5243:b0:2f4:434d:c7f0 with SMTP id 98e67ed59e1d1-2fe68ada3e8mr17648463a91.12.1740622784911; Wed, 26 Feb 2025 18:19:44 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:40 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-25-seanjc@google.com> Subject: [PATCH v2 24/38] timekeeping: Resume clocksources before reading persistent clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When resuming timekeeping after suspend, restore clocksources prior to reading the persistent clock. Paravirt clocks, e.g. kvmclock, tie the validity of a PV persistent clock to a clocksource, i.e. reading the PV persistent clock will return garbage if the underlying PV clocksource hasn't been enabled. The flaw has gone unnoticed because kvmclock is a mess and uses its own suspend/resume hooks instead of the clocksource suspend/resume hooks, which happens to work by sheer dumb luck (the kvmclock resume hook runs before timekeeping_resume()). Note, there is no evidence that any clocksource supported by the kernel depends on a persistent clock. Signed-off-by: Sean Christopherson Reviewed-by: Thomas Gleixner --- kernel/time/timekeeping.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 1e67d076f195..332d053fa9ce 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1794,11 +1794,16 @@ void timekeeping_resume(void) u64 cycle_now, nsec; unsigned long flags; - read_persistent_clock64(&ts_new); - clockevents_resume(); clocksource_resume(); + /* + * Read persistent time after clocksources have been resumed. Paravirt + * clocks have a nasty habit of piggybacking a persistent clock on a + * system clock, and may return garbage if the system clock is suspended. + */ + read_persistent_clock64(&ts_new); + raw_spin_lock_irqsave(&tk_core.lock, flags); /* From patchwork Thu Feb 27 02:18:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993570 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3459822C35C for ; Thu, 27 Feb 2025 02:19:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622788; cv=none; b=CBybHZzqN5QUurMjOQBP2VZNCbkS2VFneQgXaG8NNMZNha+BFLHrPkw441mShqces+UqSpzPny9VN5HdopGYenlzaGuYa94Lar9mFLqx2cB2nErdi4BAoqT+7iec5PtGHNopKkvMe3Uj7Rwmf28gQLIe4/d5uwG/Fp2z1HALguM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622788; c=relaxed/simple; bh=zjjnD8AWfMJS2/xbCkWD8FS8YCODvYXqFbARM/Ts+/s=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PnQB6AdRplWUqMgMjwhQ34oyYDDYrKkZnCEJylOTo5tBOCgsNaMdDkSH0CpwTdthbvh9kMfdxyv4dIOV/3dV8Biw6KqeXXX9Y7m+naDb8Lk1ZSz9acBZmiwV7Bvu8rsIGWj/zHz4AXUq8xEcaByegdjytkr++7I1Qq2qT4mycmc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LGdP3V47; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LGdP3V47" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-22107b29ac3so7797505ad.1 for ; Wed, 26 Feb 2025 18:19:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622786; x=1741227586; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3WIlT5dxc8eqreSuMAUxshuXGBEM8C/NLPwcD1zy3Y0=; b=LGdP3V47zvBfuYhkNIl11AV1RfpjpLs+4ZoASXY5qRpzH0BbueuSYw41Iptz3ijd3v gtic5rc9c0YEZMvGsRCQDIrMPaDUL6uHWG5KOUwXvJmOOM58H//frQlF7FnmwGuft8Or xaAz8D6Spnx/Hx5xUKEmQxvRMvI8fliZDw55LCSQoM7Kni64zuQkGKnAziTtiFGFhtn/ KvjEcTKJbxSuG8OwRnkKPlkFJgyV2pOU5B4vJSVbGRBy2R64Zek5I9LgxpP61GR4lU8C gVLzI6H3fBqw/0SYdFT+kjqujcFYkM1DN+IkAhUFv3/B+J2ud4G8Jf48/cEHpguSI2sF L1HQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622786; x=1741227586; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3WIlT5dxc8eqreSuMAUxshuXGBEM8C/NLPwcD1zy3Y0=; b=bonGZEtjusMbTgAElOlli4+3gUK18eVY1En+xIYHmTDqogiSMrn3EjMRIxtjwYVEbQ 9NtEOkmbiygZhXN0LFiM5RTrMTmoCOFVA/n5yXrRF8JSkKA3wX8F6w70OyVmezqygYJb 4cr1oKsqoQI7gWJg+mEkACKgaj/WGMVKD76LAieYJYgxrNNMXHGaLLSRbgkV+Xv9W9iN nENGks1z1up4KFlBTfg6MBxsQBBAGRrGyrqY38hyZGVLJSx9HeLlU8iCIPTwW+Uo9LGF 9bArr/rrrL3Uq5RcordkTlBBBPs1HKOenHojOlL2ZsWOcoP+obxp8NxjalpH3E4rBDj5 lEyQ== X-Forwarded-Encrypted: i=1; AJvYcCUT7INh7y1KCTY8tWUiaMuC+L6lg8vxkut86aK/0JQs5edT9sjkihwhrcm836z74UIaxdM=@vger.kernel.org X-Gm-Message-State: AOJu0Yx8HAc44b21XaYzrVR3+lWB6WoeopTdHcd3mZAeAwo9zEz2ALSk tFratt3+XK7bWzZL+y1czuvswF5AOW0CkxUQeJ1rIC1UcNrMZnuiY6B8HAsElMD15SysitpLKKs yhw== X-Google-Smtp-Source: AGHT+IEKa+U3XmhHVLGnMc9mY3OxJMjJ1pR3+E2erpOOt4xnyyZwND9G3ZoTVgYNZ7eny0ZZLwJ5fe0HKa8= X-Received: from pjvf4.prod.google.com ([2002:a17:90a:da84:b0:2ea:3a1b:f493]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e80e:b0:21f:3d0d:2408 with SMTP id d9443c01a7336-2234a28af91mr27386885ad.10.1740622786458; Wed, 26 Feb 2025 18:19:46 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:41 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-26-seanjc@google.com> Subject: [PATCH v2 25/38] x86/kvmclock: Hook clocksource.suspend/resume when kvmclock isn't sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Save/restore kvmclock across suspend/resume via clocksource hooks when kvmclock isn't being used for sched_clock. This will allow using kvmclock as a clocksource (or for wallclock!) without also using it for sched_clock. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 0580fe1aefa0..319f8b2d0702 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -130,7 +130,17 @@ static void kvm_setup_secondary_clock(void) static void kvm_restore_sched_clock_state(void) { - kvm_register_clock("primary cpu clock, resume"); + kvm_register_clock("primary cpu, sched_clock resume"); +} + +static void kvmclock_suspend(struct clocksource *cs) +{ + kvmclock_disable(); +} + +static void kvmclock_resume(struct clocksource *cs) +{ + kvm_register_clock("primary cpu, clocksource resume"); } void kvmclock_cpu_action(enum kvm_guest_cpu_action action) @@ -201,6 +211,8 @@ static struct clocksource kvm_clock = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .id = CSID_X86_KVM_CLK, .enable = kvm_cs_enable, + .suspend = kvmclock_suspend, + .resume = kvmclock_resume, }; static void __init kvmclock_init_mem(void) @@ -295,6 +307,15 @@ static void __init kvm_sched_clock_init(bool stable) __paravirt_set_sched_clock(kvm_sched_clock_read, stable, kvm_save_sched_clock_state, kvm_restore_sched_clock_state); + /* + * The BSP's clock is managed via dedicated sched_clock save/restore + * hooks when kvmclock is used as sched_clock, as sched_clock needs to + * be kept alive until the very end of suspend entry, and restored as + * quickly as possible after resume. + */ + kvm_clock.suspend = NULL; + kvm_clock.resume = NULL; + pr_info("kvm-clock: using sched offset of %llu cycles", kvm_sched_clock_offset); From patchwork Thu Feb 27 02:18:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993571 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF4D622F386 for ; Thu, 27 Feb 2025 02:19:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622790; cv=none; b=RK2oH2/bY2PS4VzhqtMqIBFBfQWfrEeMURGJ7+hfdoSh927EyfvCL+hKWj7VL9LzN9DN1kd2QmzNoC2qZ3GzRdG9X8mFUc0QiF+PVtRn3ENSuAsNc8c+iDhexw7ykFHURnZM0NuJJaWQhuCG436WeQZ5JTU6x8Zox0Q7iqiBreI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622790; c=relaxed/simple; bh=R5OwNlXhbkGyFpfs6H/9FfbWD/QK2xYWDQUAUET1omM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qxXbA1E2sVz/0ht9C0CU2QK7bYjjaecC6jYBz16ijN+IWLH+Ja3atIqNuG/IKxYnINsrZTuPrw8VHemX2uaCCkUEoxE2LNPkyOtQbFPRuWjejxE/v2trSLxp3uJPynx0lGDIfeTfd7AasSYTj54I4AW0D9fZx6ScUKYiddUkJJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zuWy5JVs; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zuWy5JVs" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fe8de1297eso1024019a91.0 for ; Wed, 26 Feb 2025 18:19:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622788; x=1741227588; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=8l5U+hO6xQEeVCf+kI9O/UrrnIVeF1W04fkwqmR0RAk=; b=zuWy5JVsiYI1ZfzrJuCUUiLvMOgDT8++J5uxoTwFEt3DrW7QBR6I5wtKR4Wpn5IyhV v6U0951GdBh626qmBE1x4/B882Ni0Nke9dmrBN3Gwv2bLR3C7V5DRm/dT1R9SIb0ynaC IAAWN8DMFEYbfLZWOOZie/g9pC9T4C5IE7CSR/zTE2ycLlHcHedGGlRkeADw/a/UOGDx xRmEnOwzkthLbknuF/2X376Qw5fnPXBWviZTZ6e665FrWPHBJm6q3h4KE9yeqCZNJcht 7tlu2/+002Yf6s0tFnBtuHbUJ1zqGiWuANRsnq0UdceT1gZBO4E/cq0Z+G+2yi4hpvhZ VJvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622788; x=1741227588; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8l5U+hO6xQEeVCf+kI9O/UrrnIVeF1W04fkwqmR0RAk=; b=X8RkBgN11oOU1m0A26vSWgKVocEFRJ+nF73PtdrX9j9wjA+T+OEABgguZdWmoT6cEB l3eHHtgbo6VXojMzHE03/kDe/kpzemirEi2Id0cjF7LnRwp05QNZsqA5M+l25NFu9O3z GCVwWMgDnQqikcfJOa5lVbviOE7KLGDxVotmMQG9qoNMfGzvto0SK0oJyu2oboVWmdSQ H7PPTkkklMIc5800I+O2T+afHyusw9B4XhCWuE04nZuyh13oxvpQnl26Oqm4tL+F83uH 7uMrzL6jDL8NdoT2M83a1hA4F08NgZlpwZCKwqdyDbSdaAuwZgkGhWOvh0YPzyIyDxOW vp+A== X-Forwarded-Encrypted: i=1; AJvYcCWiiry7Jasnbk4pXCNFiR1YANsUIL5qCNK3QZjEN0hIuTSwzhiBNg/Tsgnnwl27019dVI0=@vger.kernel.org X-Gm-Message-State: AOJu0YzS4xKQZjGEGSBS7BWqHyI2WvELDhpNW4zruTfxaS3+dvQyHijX iIpDIhyfacLOOyZ0NO4EXSSLw5ShTFqevX84Kl2zNUJfIYjJJleeZxmdkx5naR/DzqfizPPttVI XWA== X-Google-Smtp-Source: AGHT+IGsGVabM6lqtL4epZ9oGORwajTYA23ku4RspaNu+4NGjN39FOibtOjpJfF6SNul6q4RPaHxkAISArM= X-Received: from pjbph15.prod.google.com ([2002:a17:90b:3bcf:b0:2ef:d136:17fc]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3901:b0:2fa:2268:1af4 with SMTP id 98e67ed59e1d1-2fea1299b06mr2509653a91.7.1740622788182; Wed, 26 Feb 2025 18:19:48 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:42 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-27-seanjc@google.com> Subject: [PATCH v2 26/38] x86/kvmclock: WARN if wall clock is read while kvmclock is suspended From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania WARN if kvmclock is still suspended when its wallclock is read, i.e. when the kernel reads its persistent clock. The wallclock subtly depends on the BSP's kvmclock being enabled, and returns garbage if kvmclock is disabled. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 319f8b2d0702..0ce23f862cbd 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -52,6 +52,8 @@ static struct pvclock_vsyscall_time_info *hvclock_mem; DEFINE_PER_CPU(struct pvclock_vsyscall_time_info *, hv_clock_per_cpu); EXPORT_PER_CPU_SYMBOL_GPL(hv_clock_per_cpu); +static bool kvmclock_suspended; + /* * The wallclock is the time of day when we booted. Since then, some time may * have elapsed since the hypervisor wrote the data. So we try to account for @@ -59,6 +61,7 @@ EXPORT_PER_CPU_SYMBOL_GPL(hv_clock_per_cpu); */ static void kvm_get_wallclock(struct timespec64 *now) { + WARN_ON_ONCE(kvmclock_suspended); wrmsrl(msr_kvm_wall_clock, slow_virt_to_phys(&wall_clock)); preempt_disable(); pvclock_read_wallclock(&wall_clock, this_cpu_pvti(), now); @@ -118,6 +121,7 @@ static void kvm_save_sched_clock_state(void) * to the old address prior to reconfiguring kvmclock would clobber * random memory. */ + kvmclock_suspended = true; kvmclock_disable(); } @@ -130,16 +134,19 @@ static void kvm_setup_secondary_clock(void) static void kvm_restore_sched_clock_state(void) { + kvmclock_suspended = false; kvm_register_clock("primary cpu, sched_clock resume"); } static void kvmclock_suspend(struct clocksource *cs) { + kvmclock_suspended = true; kvmclock_disable(); } static void kvmclock_resume(struct clocksource *cs) { + kvmclock_suspended = false; kvm_register_clock("primary cpu, clocksource resume"); } From patchwork Thu Feb 27 02:18:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993572 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4CF75230243 for ; Thu, 27 Feb 2025 02:19:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622792; cv=none; b=l5JBQjc4Eh2pXGrdYAu8aWav6bXM+nHjUbaqbcLTms2HGYbAeogTxjkEB/r2YREu7aYwtuGTXzP28l87sMLBlzBQTfKC7sPgNEodJzt1Ibd5Kt2HLmduAuXymy1ZQ4gZMUj2frrDvz6Xmb+MCGnxgGfiFMTm4aHQdF/bxpGucAw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622792; c=relaxed/simple; bh=qxvHP66lsjRniK4FyS4w/nVcINWk6vqDJhVEUffkgUc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=R7qWyjKcileF5Tn3Q7WeeqJ+rD/+MQeuNyIC+dYXki/TZ2WtpZ5YzgodWmGL74XPRI99pVfkHh+0pTxdRvvQOMgARen+Mj0THAbK59FNNDO+LskrHPq1VJh3P6RtBfKlQQvU2c9eMWI0luq6SFkuDRci6JYjrkQLOmbPvWV3vHk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=R0EEFHhd; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="R0EEFHhd" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc518f0564so1096062a91.2 for ; Wed, 26 Feb 2025 18:19:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622790; x=1741227590; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=QzXAit3+eiuwe2E434PEIcPVDj3bfcCBW02+u03qap4=; b=R0EEFHhdgeuv4rKnUYnpsWdwTNQIRkpZRtRfBBzT9Jm3kN6t6bcls+tw5pfyQQIpXv UFHx57GPzdTJj73JHnirodi210PKvnQraIlUFeXeO5aX8GIz6TNAPp7XefAqt8dfH6xd ZcKMh+Ilffll3RQM9LgPv1U/MgcLXZXYvntUSvss6ykD308PL8WXW5GIaGoNu4cCmNuE gJBoSBgJkfeEZwzqy0e5vbtwVbTtb8ZJ4CuL9ctlJvm5UL2sauRO0ZxvdfMHvNDnFfUA +ZD4mOHcy7JrwIiDO8X2qsnl8rc9Y8+NEr1NspiJhWElT33WWxGf8XBTLqOxXVARbD0N aH7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622790; x=1741227590; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QzXAit3+eiuwe2E434PEIcPVDj3bfcCBW02+u03qap4=; b=YqsNqVmcf9apg3hRJgzZ+fRetFjITunx3OsvF7XRoTvx+HsVnrwPTv/5UepBNCJSOr yKmFrRznxtaoyMJjDW4dZoyqx57Rw8wYkuYHoCcB8ErhweepmO+XwsgfaSJSGRRaf44Q 4ThOFLr+jIR5P/Rggr8afyk/YCMPgZGV9ErEIP5kuCUQHNdyYEAKnIF19DSS3q32HZN2 CFjDh03GecFhYgSddGHfBln/AgXl5qISYgbAdCZc3/i/ron+rnbBb/PR1moit/2MOJNb RqxkL1unSDED1jSgSX1i25v9fvp168cxAZG5Yyl4HM++Rt7IN+POiqmTPxVrs2ehaR0L uvKA== X-Forwarded-Encrypted: i=1; AJvYcCXTOj8QjYLPLMrmTf1R4Fe6tIMjvcHfKCbwAWpQScPvGM/w4Hmfhl+QSYppgUvZ3PYoWVs=@vger.kernel.org X-Gm-Message-State: AOJu0YxApZGxdZibU47XF3ee1T+DChVHMbcztJ668ZE9i7WQLybT4uBV lYhdX2djSErrpWzuy9oM/TB+xwQ/94LNEoyRoo5MUN/roIq51IYZfx2TscW+hWbYstHsCNrNOJr VWg== X-Google-Smtp-Source: AGHT+IEy19UXAGhpZ6BooyEkiFGbgvHpy57ZvxRHzoGzPaLhW9PSe6CovWEb42HIAZDNRdCcii7axkfnXfc= X-Received: from pjbsh13.prod.google.com ([2002:a17:90b:524d:b0:2ee:53fe:d0fc]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5385:b0:2f5:63a:449c with SMTP id 98e67ed59e1d1-2fe68cf4000mr14888818a91.28.1740622789852; Wed, 26 Feb 2025 18:19:49 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:43 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-28-seanjc@google.com> Subject: [PATCH v2 27/38] x86/kvmclock: Enable kvmclock on APs during onlining if kvmclock isn't sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania In anticipation of making x86_cpuinit.early_percpu_clock_init(), i.e. kvm_setup_secondary_clock(), a dedicated sched_clock hook that will be invoked if and only if kvmclock is set as sched_clock, ensure APs enable their kvmclock during CPU online. While a redundant write to the MSR is technically ok, skip the registration when kvmclock is sched_clock so that it's somewhat obvious that kvmclock *needs* to be enabled during early bringup when it's being used as sched_clock. Plumb in the BSP's resume path purely for documentation purposes. Both KVM (as-a-guest) and timekeeping/clocksource hook syscore_ops, and it's not super obvious that using KVM's hooks would be flawed. E.g. it would work today, because KVM's hooks happen to run after/before timekeeping's hooks during suspend/resume, but that's sheer dumb luck as the order in which syscore_ops are invoked depends entirely on when a subsystem is initialized and thus registers its hooks. Opportunsitically make the registration messages more precise to help debug issues where kvmclock is enabled too late. Opportunstically WARN in kvmclock_{suspend,resume}() to harden against future bugs. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_para.h | 2 ++ arch/x86/kernel/kvm.c | 24 +++++++++++------- arch/x86/kernel/kvmclock.c | 44 +++++++++++++++++++++++++++------ 3 files changed, 53 insertions(+), 17 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 8708598f5b8e..42d90bf8fde9 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -120,6 +120,8 @@ static inline long kvm_sev_hypercall3(unsigned int nr, unsigned long p1, #ifdef CONFIG_KVM_GUEST enum kvm_guest_cpu_action { KVM_GUEST_BSP_SUSPEND, + KVM_GUEST_BSP_RESUME, + KVM_GUEST_AP_ONLINE, KVM_GUEST_AP_OFFLINE, KVM_GUEST_SHUTDOWN, }; diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 866b061ee0d9..5f093190df17 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -461,18 +461,24 @@ static void kvm_guest_cpu_offline(enum kvm_guest_cpu_action action) kvmclock_cpu_action(action); } +static int __kvm_cpu_online(unsigned int cpu, enum kvm_guest_cpu_action action) +{ + unsigned long flags; + + local_irq_save(flags); + kvmclock_cpu_action(action); + kvm_guest_cpu_init(); + local_irq_restore(flags); + return 0; +} + +#ifdef CONFIG_SMP + static int kvm_cpu_online(unsigned int cpu) { - unsigned long flags; - - local_irq_save(flags); - kvm_guest_cpu_init(); - local_irq_restore(flags); - return 0; + return __kvm_cpu_online(cpu, KVM_GUEST_AP_ONLINE); } -#ifdef CONFIG_SMP - static DEFINE_PER_CPU(cpumask_var_t, __pv_cpu_mask); static bool pv_tlb_flush_supported(void) @@ -737,7 +743,7 @@ static int kvm_suspend(void) static void kvm_resume(void) { - kvm_cpu_online(raw_smp_processor_id()); + __kvm_cpu_online(raw_smp_processor_id(), KVM_GUEST_BSP_RESUME); #ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL if (kvm_para_has_feature(KVM_FEATURE_POLL_CONTROL) && has_guest_poll) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 0ce23f862cbd..76884dfc77f4 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -52,6 +52,7 @@ static struct pvclock_vsyscall_time_info *hvclock_mem; DEFINE_PER_CPU(struct pvclock_vsyscall_time_info *, hv_clock_per_cpu); EXPORT_PER_CPU_SYMBOL_GPL(hv_clock_per_cpu); +static bool kvmclock_is_sched_clock; static bool kvmclock_suspended; /* @@ -128,7 +129,7 @@ static void kvm_save_sched_clock_state(void) #ifdef CONFIG_SMP static void kvm_setup_secondary_clock(void) { - kvm_register_clock("secondary cpu clock"); + kvm_register_clock("secondary cpu, sched_clock setup"); } #endif @@ -140,25 +141,51 @@ static void kvm_restore_sched_clock_state(void) static void kvmclock_suspend(struct clocksource *cs) { + if (WARN_ON_ONCE(kvmclock_is_sched_clock)) + return; + kvmclock_suspended = true; kvmclock_disable(); } static void kvmclock_resume(struct clocksource *cs) { + if (WARN_ON_ONCE(kvmclock_is_sched_clock)) + return; + kvmclock_suspended = false; kvm_register_clock("primary cpu, clocksource resume"); } void kvmclock_cpu_action(enum kvm_guest_cpu_action action) { - /* - * Don't disable kvmclock on the BSP during suspend. If kvmclock is - * being used for sched_clock, then it needs to be kept alive until the - * last minute, and restored as quickly as possible after resume. - */ - if (action != KVM_GUEST_BSP_SUSPEND) + switch (action) { + /* + * The BSP's clock is managed via clocksource suspend/resume, + * to ensure it's enabled/disabled when timekeeping needs it + * to be, e.g. before reading wallclock (which uses kvmclock). + */ + case KVM_GUEST_BSP_SUSPEND: + case KVM_GUEST_BSP_RESUME: + break; + case KVM_GUEST_AP_ONLINE: + /* + * Secondary CPUs use dedicated sched_clock hooks to enable + * kvmclock early during bringup, there's nothing to be done + * when during CPU online. + */ + if (kvmclock_is_sched_clock) + break; + kvm_register_clock("secondary cpu, online"); + break; + case KVM_GUEST_AP_OFFLINE: + case KVM_GUEST_SHUTDOWN: kvmclock_disable(); + break; + default: + WARN_ON_ONCE(1); + break; + } } /* @@ -313,6 +340,7 @@ static void __init kvm_sched_clock_init(bool stable) kvm_sched_clock_offset = kvm_clock_read(); __paravirt_set_sched_clock(kvm_sched_clock_read, stable, kvm_save_sched_clock_state, kvm_restore_sched_clock_state); + kvmclock_is_sched_clock = true; /* * The BSP's clock is managed via dedicated sched_clock save/restore @@ -356,7 +384,7 @@ void __init kvmclock_init(void) msr_kvm_system_time, msr_kvm_wall_clock); this_cpu_write(hv_clock_per_cpu, &hv_clock_boot[0]); - kvm_register_clock("primary cpu clock"); + kvm_register_clock("primary cpu, online"); pvclock_set_pvti_cpu0_va(hv_clock_boot); if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT)) { From patchwork Thu Feb 27 02:18:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993573 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24A7D2327A7 for ; Thu, 27 Feb 2025 02:19:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622793; cv=none; b=J8iDdo38xsIa76oo6C40MZJiXa+jR71XEMAar0jwdYIIVM5+MR2WMrh+rvxdu6cHMy0WTIiRwbcH0sjU+Biuls1McpkbPIUY3c5SD7cjv9eBJyQhtELNVK7lRBJ1e7smYwnwGhFb/CpQCIm1jPUd6dJfEC0v/nxcqjiJo6JCIo4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622793; c=relaxed/simple; bh=Bgq2jKvITCPJGjInNMTg9hShy45LprwLheePx172YI0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LRc0oBx8Bd929JcK7Xza3tprBdThDGh1+Z3QCdL23dW0HtL8gdz9cw2KOduRMwtv1nEbX7BYD6Ab+YywqWRZxO9y0WOVG2F6emxuh8+Q0Il6O14ObgOW/TLi2e5rXff63TcWPuxbfu8d1fXEUwf0A2T3tXhFJ/9/wq3f+hTeqdk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ja7voLrO; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ja7voLrO" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fe870bc003so993484a91.1 for ; Wed, 26 Feb 2025 18:19:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622791; x=1741227591; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=w772IaQ5YRbZvSP+H3ufucBFIq7F/07nn6vrGLJgcY8=; b=ja7voLrOwYQdv1Sdy9Os0Q93S6Z8KcXth3ggO2jhvDNp+8SztQ1N1vNGd0QmO6Zd+j +tLpDz5lVBuh9fJbb90Z9RP9MeWHgAunRGw6VEK4R5r0DN6hkFZq8SNDJ8064NhC8S/o u3smzhjRcCOaJfi2Tn08Ij1AWLjp8ckJlkww5RghW2YzIDw1bV8fXxNO/JsIj75+sneO 2kgP6TkoDS8184soxCnxA3jygfgEL9hU/8VDIpsE/lPwq4jVIkvYOkEu+/Ph82GfkiBT fIUlG1X66B+jCqXRj5xPTcez7cJpA3PcQ4U7xeFafjEv14dEGceFjUyVGheNMb1AQ8Ih vVxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622791; x=1741227591; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=w772IaQ5YRbZvSP+H3ufucBFIq7F/07nn6vrGLJgcY8=; b=GD3Bov7Fz4xF8pRlXZc+JocOSzMWyFpKHZOfRHEAr/tQrpeoJcrGszHBM5ubK+hJAT IIwqUAL7FeNhmi6aRPn1OME9Yi7Qou0tPwTLJew0HkRVzmr1SRtgOdrss+NkADdFctZ3 zK9aoz4xyHtxJwWIEl59Fhujynevbsd6wiJ6WkxJ2ofL/uKiQ7TOkSuQfS4FQDkEjjyS 8HUFMkAWhlQSPIJqz5MH4gY/8UF6jvLcMeFEmJtyxSRPH1eL0W3W2iHh5R7YP/ymf9lx GPPo6Hqpy5XPN3+1ULqG4jxqYK1H2K536dsk0sx5wn3wmlC7RQL+OJePM0yBw1fBL96V 6cXg== X-Forwarded-Encrypted: i=1; AJvYcCUFUwKiaGWzIprBeGA8LOGMbRyjFAewwNbYUOVgUWYFxq1ccVrbKZJT6DTnS+8NznUbAXc=@vger.kernel.org X-Gm-Message-State: AOJu0YyRAl7pzwmAw7WX4ZyZi9RMVQdXj8VlfKIoIdBx8q8BDo5eIbY8 wmmzXD69UJN8ZKVIm4Y2/4IWgKzllburW+jbJ4xEuCCRi1ZjDf/QaoLfX97ZyBKoSqVulfl6WXj ymA== X-Google-Smtp-Source: AGHT+IFwaP3Gmshvc7uvq62wN2IryjydNwW4vN0WeuDaKx0968TrkTSYTEdNA9orJIr9gY0PmdxDWYBBhxM= X-Received: from pjbsm1.prod.google.com ([2002:a17:90b:2e41:b0:2f8:4024:b59a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2688:b0:2f9:d0cd:3403 with SMTP id 98e67ed59e1d1-2fea12fcb22mr2427211a91.16.1740622791492; Wed, 26 Feb 2025 18:19:51 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:44 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-29-seanjc@google.com> Subject: [PATCH v2 28/38] x86/paravirt: Mark __paravirt_set_sched_clock() as __init From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Annotate __paravirt_set_sched_clock() as __init, and make its wrapper __always_inline to ensure sanitizers don't result in a non-inline version hanging around. All callers run during __init, and changing sched_clock after boot would be all kinds of crazy. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 9 +++++---- arch/x86/kernel/paravirt.c | 4 ++-- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index dc26a3c26527..e6d5e77753c4 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -28,11 +28,12 @@ u64 dummy_sched_clock(void); DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); -void __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)); +void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)); -static inline void paravirt_set_sched_clock(u64 (*func)(void), - void (*save)(void), void (*restore)(void)) +static __always_inline void paravirt_set_sched_clock(u64 (*func)(void), + void (*save)(void), + void (*restore)(void)) { __paravirt_set_sched_clock(func, true, save, restore); } diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 9673cd3a3f0a..92bf831a63b1 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -86,8 +86,8 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); -void __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)) +void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)) { if (!stable) clear_sched_clock_stable(); From patchwork Thu Feb 27 02:18:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993574 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E28EF2343AE for ; Thu, 27 Feb 2025 02:19:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622795; cv=none; b=PchyynYswsuTqEz/zxbJQazDLLkSJdbn89gl1FhFR7KQsLQdurp17XAakjUgpLnlvWPWOmYOy7HNZI5VYF/TLLMLMlBDFWu2NA30E1csyAThCpMRS8iW/YVW4HL6fuwLQ3+gxV6E9NE4aqszbsNPN4lCeH875f6QunhMRYXlGr4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622795; c=relaxed/simple; bh=rD+knfVmlT64JQ1lkKWQu4cILK/xQrLi9CqFuuTpl9k=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QcBPOQxrqT5Xl8WUyPt0KZOvrtYVjC2vNfXOkivnKggsnBg/cyfCesG4Vq6e1kxeS5Tt2jW7rGxEqQTyoHzbVMOWaF+lF38oGPY9VUj1QLn1L9YxQZg2tMFDDaDq+13rx73GU42JWgkzvI5IHNpgo2LB3QiLN1VHIFax+4rebBg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=C2bEH7N8; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="C2bEH7N8" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2f83e54432dso1546807a91.2 for ; Wed, 26 Feb 2025 18:19:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622793; x=1741227593; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=P4tBmLUacP8DGxGQBI6X9mo8c2uhJ9G5VZle1I388Hk=; b=C2bEH7N8G98JK652NmlnCXoxwcXUVY16K/HXdHIWahTSHby9n9P6AOXN9dLq2mXCIr LX8pzyFqqseW3prLeqE0/H2ZA1wZLhXAirkzOaKV7ww3zKQFbqYkVxhE+/LSKkHBvfim ZQAG/zZAxu7sDeTN8F7M/NEXCk9QHRwiEJyDNuZ9LM2SK0VpXxcRIK7jBg/EC6lnf1+v JMpjTOp946M0745lysOM/GsjejPjE5Qg6vYqQ6A6ROrNCxegYKZjlwmTJgGoc399fUlw 18dv12Na02OedvCuMN9A5WM33WSUvbbnvWmXIR8AgExa1mSdRhO2fQgcKScXDzuj/Yzv QUmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622793; x=1741227593; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=P4tBmLUacP8DGxGQBI6X9mo8c2uhJ9G5VZle1I388Hk=; b=T2yMWTCYdr65JOsrJIsXQXzNHTarb5T0064suNzmOaYExS8g/GzCLf+Ol8PhtoPvuB YxltJW1EvW8tTDbRQWlXb7skz/MoB6AP/JSC7P2mTgFGjTFqwz80nVp5W8uPM6Lsxcdz hF1ZiJ2cVUdARygUoSohtE2o91KwTVF/I7/NKlQgWDuE2dqXaza0n1CDb1skUgRlHgbf Ee0CHUTDogzHXla/PtryOSC24mdH9yeSPkiIQ4qtBWErETB0XpADCS8qL5/mYMa1pKsQ PxncjyvBUx0pRzQ1jGE1uXrP1BOhOeXTsPdRQCC5wiYzelEzeq5WyDpOvORJqZvQoTPI DzoQ== X-Forwarded-Encrypted: i=1; AJvYcCXMuQ5ZYkZMCeZEL7XDtoPLUaMtF0ygCMVGK5uFcZwnFWon7iTkhAZ46T1GBLyy8yaN1Pk=@vger.kernel.org X-Gm-Message-State: AOJu0Yx73cF2TwQROD6WoswOYSM++Fr0Kly/QHVQADFTAuQdl60NpiYj YbuRWfvPHv8H124QNiE0poQ06hYh9zc/C22nZ1COlawvm/wVmIFg0V4VeSQXYnPWerPNjuwYuFN ISw== X-Google-Smtp-Source: AGHT+IGdVjOlq0BY70ioN6i4xVGmLOeDmFSQNBlkaPaHsOsARCADALqyiFXLpG70t8VDnA7M1OSlhaskPoo= X-Received: from pjbsw3.prod.google.com ([2002:a17:90b:2c83:b0:2fa:15aa:4d2b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:54cd:b0:2f2:8bdd:cd8b with SMTP id 98e67ed59e1d1-2fe7e3b1756mr8832651a91.29.1740622793310; Wed, 26 Feb 2025 18:19:53 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:45 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-30-seanjc@google.com> Subject: [PATCH v2 29/38] x86/paravirt: Plumb a return code into __paravirt_set_sched_clock() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Add a return code to __paravirt_set_sched_clock() so that the kernel can reject attempts to use a PV sched_clock without breaking the caller. E.g. when running as a CoCo VM with a secure TSC, using a PV clock is generally undesirable. Note, kvmclock is the only PV clock that does anything "extra" beyond simply registering itself as sched_clock, i.e. is the only caller that needs to check the new return value. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 6 +++--- arch/x86/kernel/kvmclock.c | 7 +++++-- arch/x86/kernel/paravirt.c | 5 +++-- 3 files changed, 11 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index e6d5e77753c4..5de31b22aa5f 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -28,14 +28,14 @@ u64 dummy_sched_clock(void); DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); -void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)); +int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)); static __always_inline void paravirt_set_sched_clock(u64 (*func)(void), void (*save)(void), void (*restore)(void)) { - __paravirt_set_sched_clock(func, true, save, restore); + (void)__paravirt_set_sched_clock(func, true, save, restore); } static __always_inline u64 paravirt_sched_clock(void) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 76884dfc77f4..1dbe12ecb26e 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -337,9 +337,12 @@ static int kvmclock_setup_percpu(unsigned int cpu) static void __init kvm_sched_clock_init(bool stable) { + if (__paravirt_set_sched_clock(kvm_sched_clock_read, stable, + kvm_save_sched_clock_state, + kvm_restore_sched_clock_state)) + return; + kvm_sched_clock_offset = kvm_clock_read(); - __paravirt_set_sched_clock(kvm_sched_clock_read, stable, - kvm_save_sched_clock_state, kvm_restore_sched_clock_state); kvmclock_is_sched_clock = true; /* diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 92bf831a63b1..a3a1359cfc26 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -86,8 +86,8 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); -void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)) +int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)) { if (!stable) clear_sched_clock_stable(); @@ -95,6 +95,7 @@ void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, static_call_update(pv_sched_clock, func); x86_platform.save_sched_clock_state = save; x86_platform.restore_sched_clock_state = restore; + return 0; } /* These are in entry.S */ From patchwork Thu Feb 27 02:18:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993575 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85C6E235346 for ; Thu, 27 Feb 2025 02:19:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622797; cv=none; b=L3QO54Gmqca+sKMUuOLKfPeKqUQv69gBnFb/CFSmR5j0rXZzctIHN/5Ip8xolTenEzDV/RSOg7v6mTAhs4KP8Lscu+9iEJr7Ssdk6MmE20Qc400l95+TIWCupzj1ajgAZUZAGyv/U2Pb0ZLauaoFMkr3Sy7BIQgLcGB5s846IEI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622797; c=relaxed/simple; bh=9PNQlaFr17QsW09OA3IEH/cFSTquMC7pYNcyhSay2n0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HYscxsRHDWYLP+zB4FwmDdfA7YTE6k0/Cg+dwPXyKse4Efzkm6YGFgXL7CvHdruGUaeGKwF0vm/OyT16hTDYVfQ3o5DIn90Ms0P/kz2lihRA9Ej83Kifn46OOocu2uu2hmfyWgezD3dIJV7whxE6xMKHWiftsQNEIykhpKDnG70= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UEbgBKOh; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UEbgBKOh" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fe86c01f5cso1042410a91.1 for ; Wed, 26 Feb 2025 18:19:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622795; x=1741227595; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3F6YIZD9bSIlkGuU2KG3Ao1xkSJhhKmqbmLiePXFSIM=; b=UEbgBKOhDxMYxMoDIIGBV5uWU/Tnmn8nsHyXriJImPHhu0J0q/06xJVRquOlcg1V5K jL538nlnF5PiO95x4DhiJAIz63+eT/PmtCiiQ1OCc0h5rXnN0XLrOdrTUtbA/OILqljq RURh6o/gjgofzE1fD0MgXZyG1aH7sr+ak88LGum+GhqLSKNGv62kStd+75zMIYXX7OV3 zhuPjdcwkYo9af3YLGjTgCEzColLONbqBDyHtBH98Kvrw6hNbu2M3Of2BEejO3D3Q6jV hU9NA3sLP3vXNJAStUWDxeNP5n/nyiWGVMaokES++xGuNfu6DmX4kagAdtwhrAF+Icq+ ZNgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622795; x=1741227595; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3F6YIZD9bSIlkGuU2KG3Ao1xkSJhhKmqbmLiePXFSIM=; b=w7KXPDbyhRHX8sSu6vzrzX2Q/PGNx8ujICvVFqwqyMjn865Y0sSKvSzgu4cE9Y6W1A hxO+owkLO2f4qsvzK6DIZwXLNO8qwSvKTUlbRWP6ei3QxX2LMl7chr5IB+yVQpXX7zhd mHmZkyqto7eU2XiWB2EBn4r6X93VDbMI3sL6+2PYkTeCt5LuQJn8YE6U5FiCWMV6TyCN eZUP7E/MKa0XwDC8hUmQB4V+YU+lw4wnYWn1o7Xk1Y7hHZEd14vAuIYuzvXzWkEpD9RA NLjkwN8W4T2wSvXkC1s3kY0D/p5VuLMSk9U7KeVhUwCou/65JLRTbB1ZN1JQOe6R7/iS D7CA== X-Forwarded-Encrypted: i=1; AJvYcCVlFm7asZW8x6cjXZx0Miw1/hXOLC79XCJFEoXkAeV0z5u2p52CQb1sCQMRKcDATDCYD7c=@vger.kernel.org X-Gm-Message-State: AOJu0YwhY1WlBfYYFR1Wjji5zgNu0+jIq7JmPv7AbqogEY60fb2fca0+ sNar0+xiQLPYc24Oa/KKmtJ4jzJdJG2euVoC1sMWJJjJbZXcFC7vly8KjTlXNAZXGJiJydmCe8r ZSg== X-Google-Smtp-Source: AGHT+IGJ6z3saIzRbkZlSBZLnZw7mlbnCfDCjtlOA/kyHI3FVoB/Y5gfMuQ1H6WQ1/ZAhh6gzh0XRDRpGl4= X-Received: from pfbfb4.prod.google.com ([2002:a05:6a00:2d84:b0:732:1ead:f8ac]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:1509:b0:1f2:e2b0:dd91 with SMTP id adf61e73a8af0-1f2e2b0ddb7mr3698859637.21.1740622795028; Wed, 26 Feb 2025 18:19:55 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:46 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-31-seanjc@google.com> Subject: [PATCH v2 30/38] x86/paravirt: Don't use a PV sched_clock in CoCo guests with trusted TSC From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Silently ignore attempts to switch to a paravirt sched_clock when running as a CoCo guest with trusted TSC. In hand-wavy theory, a misbehaving hypervisor could attack the guest by manipulating the PV clock to affect guest scheduling in some weird and/or predictable way. More importantly, reading TSC on such platforms is faster than any PV clock, and sched_clock is all about speed. Signed-off-by: Sean Christopherson --- arch/x86/kernel/paravirt.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index a3a1359cfc26..c538c608d9fb 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -89,6 +89,15 @@ DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, void (*save)(void), void (*restore)(void)) { + /* + * Don't replace TSC with a PV clock when running as a CoCo guest and + * the TSC is secure/trusted; PV clocks are emulated by the hypervisor, + * which isn't in the guest's TCB. + */ + if (cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC) || + boot_cpu_has(X86_FEATURE_TDX_GUEST)) + return -EPERM; + if (!stable) clear_sched_clock_stable(); From patchwork Thu Feb 27 02:18:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993576 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C0E0190470 for ; Thu, 27 Feb 2025 02:19:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622799; cv=none; b=MndOGa83TESvqfBgn4H8H5+LHtCZ/XD4ul5gBpMXvwc9oWqE8X6wYjXegC4w48Z5SI0c4oCKFdOOdgCtqkHruCaDUJxdxefb6mpWcfVmkzKzALp6eHwhSiVUrohMq55Xzp1oGbLv2GOo68YwjwkDF7qkGPaPnf1HT0WCRWIspm4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622799; c=relaxed/simple; bh=FqtvTL/qyi7OYXRbCV2e1dPSPCUMJ7/F3eHsImFUkdc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lOyLmI4rc0xo63hpR6hHZWMSDrwPK/k9Wy442FIPurc6XlD5ihySeRr9JtS0Q2GZWlEvGN2IrOVhfRLXjPvKheuxGKYXfqAcXntyseaK8oYfZ5cCXaXO9pNh2LiFmvPYvdYI8hSLbNWKld0Tp4PLcpg6IvRjZyestB9cSYuPxj8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vimbu3KW; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vimbu3KW" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fc1e7efdffso1557046a91.0 for ; Wed, 26 Feb 2025 18:19:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622797; x=1741227597; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0SoCRyyhI3eHKNBGNMMOnftZcsartKHSJO+ab+uvDMM=; b=vimbu3KWwirkfvG2z+F2RLzAMf3fleQVYftgs+b88GbRCqO0+ZJb5NFWIwoFHVRkeC XijiXmcioM5Qv5ZUanGznotAh1C8WnFWmMXVexEQxqqncbxDAodaukOCkf+sBAslObHB w1EOaJZfeEL14P4v0W8lOaHrDO3a3gr7GPd7J4ng4PAUfdl9xaNSLHyAFbuvZ76CRReN UeWGFCwVHBzRDvJuGzxyHvCaR7RcwyWA6T5jAGu5Z1zFlXanx8wavvPI7cnpXesjrzhy arZDhouFi575no1HyTHRC5ehVFyltEKH2d+DSxJVMI4nyBCS1B+1ttUfewWzQapcrLjv XNug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622797; x=1741227597; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0SoCRyyhI3eHKNBGNMMOnftZcsartKHSJO+ab+uvDMM=; b=Y4TsqQJBGTtmGsm/9USRqNzn+oTMLOUiMYsqLaOlqagGfWVk+9kEPdofsFHvDZJ0TG sXpaPumYecdBh86DWR6QAPuFerCfNASQe+JpJ/Xr3N02ZCbsD1RxApDTiuCpkn6/2l/M 98uZ3/Iqzeitfm42Njk8qIlx0/b3GoTHZ1mB1M9vFCtbOizqGqb1rPKaIBwdYW2ozLpb hP6PK/MMJPf7ua7b+NMlz2ZAszZby7owU9qaDJu8uilbjyF8/+z/f7ubSlcqgcCHZyLs o680pPohEc60lQHYCDUphQ9dBJ0pQ2CEG8ZbdqVQdh9RhyCvgwTLbM8NUnwYRjJJ+NUw mGVQ== X-Forwarded-Encrypted: i=1; AJvYcCVGwQoqjV/umYI4xWJgphiEvE2pqsltUAElruqFm1g2exRafX2Bwvtig5QF6R6PTEgoIoo=@vger.kernel.org X-Gm-Message-State: AOJu0Yw6dQ53JdtJLZ8MN51Tns6eFUhtkL8u7Bls9lEPs5dRF2RAk9st l+mmQfnHgZtLN0cvi96MWKB6FTXHYk5NxFgFvYmpO45xMHusmCHondluUo/NVOoQrhlWh4797N5 2BA== X-Google-Smtp-Source: AGHT+IGBOkpo7XivcprIZNXQBr8QlNDoqFx2IFaBrMW4afTfcx1rvnIfDlcWOduAP6/zUs4HYAwvxKFyq/w= X-Received: from pjboi12.prod.google.com ([2002:a17:90b:3a0c:b0:2fa:a101:755]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4ec6:b0:2ea:696d:732f with SMTP id 98e67ed59e1d1-2fe692c6ba8mr15211686a91.29.1740622796750; Wed, 26 Feb 2025 18:19:56 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:47 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-32-seanjc@google.com> Subject: [PATCH v2 31/38] x86/tsc: Pass KNOWN_FREQ and RELIABLE as params to registration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Add a "tsc_properties" set of flags and use it to annotate whether the TSC operates at a known and/or reliable frequency when registering a paravirtual TSC calibration routine. Currently, each PV flow manually sets the associated feature flags, but often in haphazard fashion that makes it difficult for unfamiliar readers to see the properties of the TSC when running under a particular hypervisor. The other, bigger issue with manually setting the feature flags is that it decouples the flags from the calibration routine. E.g. in theory, PV code could mark the TSC as having a known frequency, but then have its PV calibration discarded in favor of a method that doesn't use that known frequency. Passing the TSC properties along with the calibration routine will allow adding sanity checks to guard against replacing a "better" calibration routine with a "worse" routine. As a bonus, the flags also give developers working on new PV code a heads up that they should at least mark the TSC as having a known frequency. Signed-off-by: Sean Christopherson Reviewed-by: Michael Kelley Tested-by: Michael Kelley --- arch/x86/coco/sev/core.c | 6 ++---- arch/x86/coco/tdx/tdx.c | 7 ++----- arch/x86/include/asm/tsc.h | 8 +++++++- arch/x86/kernel/cpu/acrn.c | 4 ++-- arch/x86/kernel/cpu/mshyperv.c | 10 +++++++--- arch/x86/kernel/cpu/vmware.c | 7 ++++--- arch/x86/kernel/jailhouse.c | 4 ++-- arch/x86/kernel/kvmclock.c | 4 ++-- arch/x86/kernel/tsc.c | 8 +++++++- arch/x86/xen/time.c | 4 ++-- 10 files changed, 37 insertions(+), 25 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index dab386f782ce..29dd50552715 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3284,12 +3284,10 @@ void __init snp_secure_tsc_init(void) { unsigned long long tsc_freq_mhz; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - rdmsrl(MSR_AMD64_GUEST_TSC_FREQ, tsc_freq_mhz); snp_tsc_freq_khz = (unsigned long)(tsc_freq_mhz * 1000); tsc_register_calibration_routines(securetsc_get_tsc_khz, - securetsc_get_tsc_khz); + securetsc_get_tsc_khz, + TSC_FREQ_KNOWN_AND_RELIABLE); } diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index 42cdaa98dc5e..ca31560d0dd3 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -1135,14 +1135,11 @@ static unsigned long tdx_get_tsc_khz(void) void __init tdx_tsc_init(void) { - /* TSC is the only reliable clock in TDX guest */ - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); - /* * Override the PV calibration routines (if set) with more trustworthy * CPUID-based calibration. The TDX module emulates CPUID, whereas any * PV information is provided by the hypervisor. */ - tsc_register_calibration_routines(tdx_get_tsc_khz, NULL); + tsc_register_calibration_routines(tdx_get_tsc_khz, NULL, + TSC_FREQ_KNOWN_AND_RELIABLE); } diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index 9318c74e8d13..360f47610258 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -41,8 +41,14 @@ extern int cpuid_get_cpu_freq(unsigned int *cpu_khz); extern void tsc_early_init(void); extern void tsc_init(void); #if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) +enum tsc_properties { + TSC_FREQUENCY_KNOWN = BIT(0), + TSC_RELIABLE = BIT(1), + TSC_FREQ_KNOWN_AND_RELIABLE = TSC_FREQUENCY_KNOWN | TSC_RELIABLE, +}; extern void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), - unsigned long (*calibrate_cpu)(void)); + unsigned long (*calibrate_cpu)(void), + enum tsc_properties properties); #endif extern void mark_tsc_unstable(char *reason); extern int unsynchronized_tsc(void); diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index 2da3de4d470e..4f2f4f7ec334 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -29,9 +29,9 @@ static void __init acrn_init_platform(void) /* Install system interrupt handler for ACRN hypervisor callback */ sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); tsc_register_calibration_routines(acrn_get_tsc_khz, - acrn_get_tsc_khz); + acrn_get_tsc_khz, + TSC_FREQUENCY_KNOWN); } static bool acrn_x2apic_available(void) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 174f6a71c899..445ac3adfebc 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -421,8 +421,13 @@ static void __init ms_hyperv_init_platform(void) if (ms_hyperv.features & HV_ACCESS_FREQUENCY_MSRS && ms_hyperv.misc_features & HV_FEATURE_FREQUENCY_MSRS_AVAILABLE) { - tsc_register_calibration_routines(hv_get_tsc_khz, hv_get_tsc_khz); - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + enum tsc_properties tsc_properties = TSC_FREQUENCY_KNOWN; + + if (ms_hyperv.features & HV_ACCESS_TSC_INVARIANT) + tsc_properties = TSC_FREQ_KNOWN_AND_RELIABLE; + + tsc_register_calibration_routines(hv_get_tsc_khz, hv_get_tsc_khz, + tsc_properties); } if (ms_hyperv.priv_high & HV_ISOLATION) { @@ -525,7 +530,6 @@ static void __init ms_hyperv_init_platform(void) * is called. */ wrmsrl(HV_X64_MSR_TSC_INVARIANT_CONTROL, HV_EXPOSE_INVARIANT_TSC); - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); } /* diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index 399cf3286a60..a3a71309214c 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -385,10 +385,10 @@ static void __init vmware_paravirt_ops_setup(void) */ static void __init vmware_set_capabilities(void) { + /* TSC is non-stop and reliable even if the frequency isn't known. */ setup_force_cpu_cap(X86_FEATURE_CONSTANT_TSC); setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - if (vmware_tsc_khz) - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + if (vmware_hypercall_mode == CPUID_VMWARE_FEATURES_ECX_VMCALL) setup_force_cpu_cap(X86_FEATURE_VMCALL); else if (vmware_hypercall_mode == CPUID_VMWARE_FEATURES_ECX_VMMCALL) @@ -417,7 +417,8 @@ static void __init vmware_platform_setup(void) vmware_tsc_khz = tsc_khz; tsc_register_calibration_routines(vmware_get_tsc_khz, - vmware_get_tsc_khz); + vmware_get_tsc_khz, + TSC_FREQ_KNOWN_AND_RELIABLE); #ifdef CONFIG_X86_LOCAL_APIC /* Skip lapic calibration since we know the bus frequency. */ diff --git a/arch/x86/kernel/jailhouse.c b/arch/x86/kernel/jailhouse.c index b0a053692161..d73a4d0fb118 100644 --- a/arch/x86/kernel/jailhouse.c +++ b/arch/x86/kernel/jailhouse.c @@ -218,7 +218,8 @@ static void __init jailhouse_init_platform(void) machine_ops.emergency_restart = jailhouse_no_restart; - tsc_register_calibration_routines(jailhouse_get_tsc, jailhouse_get_tsc); + tsc_register_calibration_routines(jailhouse_get_tsc, jailhouse_get_tsc, + TSC_FREQUENCY_KNOWN); while (pa_data) { mapping = early_memremap(pa_data, sizeof(header)); @@ -256,7 +257,6 @@ static void __init jailhouse_init_platform(void) pr_debug("Jailhouse: PM-Timer IO Port: %#x\n", pmtmr_ioport); precalibrated_tsc_khz = setup_data.v1.tsc_khz; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); pci_probe = 0; diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 1dbe12ecb26e..ce676e735ced 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -199,7 +199,6 @@ void kvmclock_cpu_action(enum kvm_guest_cpu_action action) */ static unsigned long kvm_get_tsc_khz(void) { - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); return pvclock_tsc_khz(this_cpu_pvti()); } @@ -403,7 +402,8 @@ void __init kvmclock_init(void) kvm_sched_clock_init(stable); - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz); + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, + TSC_FREQUENCY_KNOWN); x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 5501d76243c8..be58df4fef66 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1301,11 +1301,17 @@ static void __init check_system_tsc_reliable(void) */ #if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), - unsigned long (*calibrate_cpu)(void)) + unsigned long (*calibrate_cpu)(void), + enum tsc_properties properties) { if (WARN_ON_ONCE(!calibrate_tsc)) return; + if (properties & TSC_FREQUENCY_KNOWN) + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + if (properties & TSC_RELIABLE) + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + x86_platform.calibrate_tsc = calibrate_tsc; if (calibrate_cpu) x86_platform.calibrate_cpu = calibrate_cpu; diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 13e5888c4501..4de06ea55397 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -40,7 +40,6 @@ static unsigned long xen_tsc_khz(void) struct pvclock_vcpu_time_info *info = &HYPERVISOR_shared_info->vcpu_info[0].time; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); return pvclock_tsc_khz(info); } @@ -571,7 +570,8 @@ static void __init xen_init_time_common(void) */ paravirt_set_sched_clock(xen_sched_clock, NULL, NULL); - tsc_register_calibration_routines(xen_tsc_khz, NULL); + tsc_register_calibration_routines(xen_tsc_khz, NULL, + TSC_FREQUENCY_KNOWN); x86_platform.get_wallclock = xen_get_wallclock; } From patchwork Thu Feb 27 02:18:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993577 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C6DD238D32 for ; Thu, 27 Feb 2025 02:19:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622802; cv=none; b=NqEGvsgOywZzY7sSehpCv6+Hct6Fq/hNnTtrC7hzZIvPkG6BiaPvAGgAoL27gUF3cZRMpRHJn4kmRl0Ryjp0NjIXEjCjirHvQ3vtVkwacZ2x3tWeL5ujZEdPXa2JeJApQSgBNibg3CkzwrZOrBNdXcZBP68c6ZngFhgHGfh/P7c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622802; c=relaxed/simple; bh=XdZRYBlPZyw/8jtawHaf0Gd25EjncrTXZSrOsMRCotM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=I4DNvzFXzi+rPhS5w5z8IyzGDyBGWJJUtK+Rih4NHzc3FoYOxmpOsEfY3XzLe5I+QZC3NqrM463AyRkMJtcfEn8h9p8r5mv6iAV5oMjsh3Zf1231EpLAOjbtXe5eeLbxFOjyjDglNSZy1BBA4tcyBMTPi5zQsRlLfXAstl4BPHM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=kL48CMa5; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kL48CMa5" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fe8de1297eso1024220a91.0 for ; Wed, 26 Feb 2025 18:19:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622799; x=1741227599; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=LZJJLySVHQgVgABAB7C0XZjDtBlLYwz9ES+Z1vP/8LE=; b=kL48CMa5HC14aNbI0Clsz2W/FXFsbTLSaq66y5swjaYLwy92G2/KOpyXHgPIdH1wA0 7pdF7gDgm1KFZsaLOrC3vhi/1/NU1Kds9jeCo857ZMxKJy43ucAAGlVApPm80cP7fyJ4 Ak3WGFAoZWP9p56PU/FtWgNi8x8LSfI5aSLZeENPtLkZJawX/5O0IB3pAhCL9FIW8JUw HQ5rSgm47il+rEQfdN6ul6S3Av+lh9JmyzPlpTfaPzPHACpvbCJxvWjLGa2SYgTuxu5F HYMmTbNr5hAw8O7aWmxnWa8YRxFwGyF9JTHnUZwtFjhqZpYsP1zTTe2hSkBq5qh4Ig0g 4Dxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622799; x=1741227599; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=LZJJLySVHQgVgABAB7C0XZjDtBlLYwz9ES+Z1vP/8LE=; b=O5tqeji56JcEiqYwu4NsjXrXl8OV1LKTaRVkwurBNGGRhJbgP8m1H3b9EniVjUry9S sPe1Vp6YBC+IPV9GfsHTvDlRqObBzMJiTleNZRzF/b1sFGWEnaRnH3CeLaCHBrhORnzi V4/WbxJpaUC9P876a5bD7sNjN5Sb/y/zrL0zRw0TlGGHcN59CLybm54Pcz736UEJSLZo GT63SPsfPEwwmmVEkzhuIRkqq32d52rSIncZ5EJlvoNvy+VVU2isV3f6U/zlUPjgrtGI aPolOnu7y3OyXQ/hGQsvznuQpm5cRuewfFuXbhKGGBssM/jaJ8WCoefUAEBBx4JTbUj1 /L+g== X-Forwarded-Encrypted: i=1; AJvYcCUNhNNM4IoMz/+iOjqrFY6vqOb63EuVp00/woNLD51fUfpBJqC8FXZe/F/FkVbrYU9Y/Ks=@vger.kernel.org X-Gm-Message-State: AOJu0YykTXxV2ybSNV/u+mIBPPH9/Cgu9uTuCsRjAi2XMCPQKYUzteKr gVuJ+FyAWRTtjRp4AH1s0um61nH61+Tnw1DOW/TrKiQ3X0KsH+ruHjhXnthTRnDsG5WHId7GGy8 UkQ== X-Google-Smtp-Source: AGHT+IGhmfOgzOy0w88MLax4aQJq7WSM6/RTmeD/HK48z8QB3SAzs8w4kTiAhr4wwBRMQkXZhflq/qsZ2GU= X-Received: from pjtq6.prod.google.com ([2002:a17:90a:c106:b0:2fc:11a0:c53f]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:570c:b0:2fa:2c61:3e5a with SMTP id 98e67ed59e1d1-2fea12c36b0mr2515446a91.10.1740622798574; Wed, 26 Feb 2025 18:19:58 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:48 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-33-seanjc@google.com> Subject: [PATCH v2 32/38] x86/tsc: Rejects attempts to override TSC calibration with lesser routine From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When registering a TSC frequency calibration routine, sanity check that the incoming routine is as robust as the outgoing routine, and reject the incoming routine if the sanity check fails. Because native calibration routines only mark the TSC frequency as known and reliable when they actually run, the effective progression of capabilities is: None (native) => Known and maybe Reliable (PV) => Known and Reliable (CoCo). Violating that progression for a PV override is relatively benign, but messing up the progression when CoCo is involved is more problematic, as it likely means a trusted source of information (hardware/firmware) is being discarded in favor of a less trusted source (hypervisor). Signed-off-by: Sean Christopherson --- arch/x86/kernel/tsc.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index be58df4fef66..ebcfaf7dcd38 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1309,8 +1309,13 @@ void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), if (properties & TSC_FREQUENCY_KNOWN) setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + else if (WARN_ON(boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ))) + return; + if (properties & TSC_RELIABLE) setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + else if (WARN_ON(boot_cpu_has(X86_FEATURE_TSC_RELIABLE))) + return; x86_platform.calibrate_tsc = calibrate_tsc; if (calibrate_cpu) From patchwork Thu Feb 27 02:18:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993578 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ECAF23AE7B for ; Thu, 27 Feb 2025 02:20:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622803; cv=none; b=YJKJKLHohXzgg5SFU9dxyqX0PPYGtt763NmzfYUj/X86wjzdmqNgR7q7UbNGi3q5piHN02e4P7Gp2jlFUm9Oq/q1QFWp+YMTeWU2Sl3MLFebnGZ7VvBZmo3z+1JH0TFJjKBxGvl+cbHX4O7NZ6/OpTfB7B3mEsA7Lc/JFg4zwD8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622803; c=relaxed/simple; bh=QPB0hO3DgGDIpWXharYozGSl0kpmrn7uptJliWGGUbA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Q4AJTMHB1K5oZJMIxLV36+jpUFoYBeY0i3S62layMWWv73RW3rk1otJj4uajFdtNc2xSQ2jo4j1syEO2C0pouK2CvxZCE3E0vkPjV0YC8V4cM6IxGWQXNfcuJ7Q+qskrupzC+EvvRj2q4TdvuQXO7KVX5iDwdQyDffffxALIDgw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=p/38eAlX; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="p/38eAlX" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fc45101191so1080492a91.1 for ; Wed, 26 Feb 2025 18:20:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622800; x=1741227600; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SUWURP6DS00BmWQuQkka/ewYMygd0uN/XlEQWE3xarE=; b=p/38eAlXVBlnzxMy4cgsnY0hH2hHTEIwimsxxtF21T0WkCtdNUw3WUnpEiNfrZFn9Z wFtDE33Y8Q6RhXb+kvOuUE80VfYmhRdlnLd49oet2ZCZyx55S9HQ+lsJiB6EGohCoCXs yLL8vgRDcZ6wUk8rxJ88gzM/Qb2l6l0lMsYQcfZKT5gQp9sEyP4GkQva5+PNY50SUmip XmHnWKq3pm6Btq73RCSWaagC3bIfp68w0vXxHzgNxh7z5yAuk4tVWLZCO1DIjZ2rfERI hq35r+/20Ic+WIA1yx9X+UhO8hvPB/Z6w5BYag0kiSpcpCLLBJ7aVZGWxDl1q8NxjtSx 6SLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622800; x=1741227600; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SUWURP6DS00BmWQuQkka/ewYMygd0uN/XlEQWE3xarE=; b=wMZn3FRtMhdHFixsGHj4Iqfxp/bDYbkyvAo5ToPP6zybxwNAhjb4Yl3KaV7cEKfiYg KdYA5EWm4t1Kar+mkLHOWQ7SlxXWq9FzSiTZmXsYvLaPVt1AngVL076D50E03g8TY49K zi4GivG+i7eHmGsLdmtGR4BuPo+yOBqeQu+HaPsnfC6btsQCru3ZsGmnnV6nemPeg5Hv JJ5Tf84GFRbyj/lghDAt/H+1b3X1ZCfTF/8CFpYsL74Y6JgQQ2lAxVMZKsCNK5DEfO51 QhvpZCB7nBmm2m2q2RElHZpsJepL9dJ48Vw1nm5ZtW/6vW5F82iPvqD8ePxkzDnzWHCh UBgw== X-Forwarded-Encrypted: i=1; AJvYcCWWQqGbdUOUlyz5WfYBd/4eV4A3xSYaWTVj2VLRY6ieiZptFLuejghzYcgpLm7lMBj8lx8=@vger.kernel.org X-Gm-Message-State: AOJu0YxbSXwXd5rMxp+NS3TT439HkTAa8J9cxgbrootfN227cSwSW7Ky gUYXhErm6acgMhLbvt/Qj8x8y+dCM5JoL+vMqtA0T/f3c/uxyhJiOtQ5IQNMegAiLxQiaj8vUwV kaw== X-Google-Smtp-Source: AGHT+IEaJegSwuz7gp17gUBIq0YV/a5p90hmpWu2RxMMtE/LMlRmTHKg61dycPOdBb/l8JsfSEdKtOPrkAY= X-Received: from pjboh8.prod.google.com ([2002:a17:90b:3a48:b0:2fc:2b96:2d4b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4fcf:b0:2f8:34df:5652 with SMTP id 98e67ed59e1d1-2fce78beb41mr33366155a91.21.1740622800327; Wed, 26 Feb 2025 18:20:00 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:49 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-34-seanjc@google.com> Subject: [PATCH v2 33/38] x86/kvmclock: Mark TSC as reliable when it's constant and nonstop From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Mark the TSC as reliable if the hypervisor (KVM) has enumerated the TSC as constant and nonstop, and the admin hasn't explicitly marked the TSC as unstable. Like most (all?) virtualization setups, any secondary clocksource that's used as a watchdog is guaranteed to be less reliable than a constant, nonstop TSC, as all clocksources the kernel uses as a watchdog are all but guaranteed to be emulated when running as a KVM guest. I.e. any observed discrepancies between the TSC and watchdog will be due to jitter in the watchdog. This is especially true for KVM, as the watchdog clocksource is usually emulated in host userspace, i.e. reading the clock incurs a roundtrip cost of thousands of cycles. Marking the TSC reliable addresses a flaw where the TSC will occasionally be marked unstable if the host is under moderate/heavy load. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index ce676e735ced..b924b19e8f0f 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -362,6 +362,7 @@ static void __init kvm_sched_clock_init(bool stable) void __init kvmclock_init(void) { + enum tsc_properties tsc_properties = TSC_FREQUENCY_KNOWN; bool stable = false; if (!kvm_para_available() || !kvmclock) @@ -400,18 +401,6 @@ void __init kvmclock_init(void) PVCLOCK_TSC_STABLE_BIT; } - kvm_sched_clock_init(stable); - - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, - TSC_FREQUENCY_KNOWN); - - x86_platform.get_wallclock = kvm_get_wallclock; - x86_platform.set_wallclock = kvm_set_wallclock; -#ifdef CONFIG_SMP - x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; -#endif - kvm_get_preset_lpj(); - /* * X86_FEATURE_NONSTOP_TSC is TSC runs at constant rate * with P/T states and does not stop in deep C-states. @@ -422,8 +411,22 @@ void __init kvmclock_init(void) */ if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && - !check_tsc_unstable()) + !check_tsc_unstable()) { kvm_clock.rating = 299; + tsc_properties = TSC_FREQ_KNOWN_AND_RELIABLE; + } + + kvm_sched_clock_init(stable); + + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, + tsc_properties); + + x86_platform.get_wallclock = kvm_get_wallclock; + x86_platform.set_wallclock = kvm_set_wallclock; +#ifdef CONFIG_SMP + x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; +#endif + kvm_get_preset_lpj(); clocksource_register_hz(&kvm_clock, NSEC_PER_SEC); pv_info.name = "KVM"; From patchwork Thu Feb 27 02:18:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993579 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9794C23BF9A for ; Thu, 27 Feb 2025 02:20:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622804; cv=none; b=UaxzIflor8pyMB+EMIa7J18vASo0lUet5H2jMQ5fuZNmO5WfD1+MG6bRy2hAiRmJIi3ErNblt+cpTHIjh6ubNvSJl5XxWwYmKmOSdGREP1Dst1owHjFyvKjg/EPtDroiBRDxcNCWr0ahrh7CcoLGrpA5jB4G3VDoTadkuvggJB4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622804; c=relaxed/simple; bh=3Om1yDQGTJbRQMaVDcoK4RL2pv7jGwTkVpwEklnlv5g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sW82hkl0sfqJxV6Rl04a6rhUmUiWl6FFPpDu6VG/tw8cntyqOuLUvRLarI+dtAMMSBT6AjX8Uj2Rcgez/pIZxfqgf+YW/D63LvqIDh1dH8mqK+DLuXHOAxqKx59MZC+uPGqAUgsUUB2eVuHNXXLwWiwqd6MM2xFLIsfny+ogFKQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=o5tSABgJ; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="o5tSABgJ" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fe9527c041so1101454a91.0 for ; Wed, 26 Feb 2025 18:20:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622802; x=1741227602; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=evC/AJYAKH4bntjSNStZX5d0W0KQw/JlJZazMUftcQU=; b=o5tSABgJASXFZUmKl+VdieZrjxfJcyhzhDWh7HjPF8xZS9Hl7IsB2i948WsiibIjVQ TWeQyAER81FznSdvJBoQlfwM25KeNy7fnOJwijqECLhe1ea6qF7tekPwy+e1vuK6uGml HGulWujjt5LEpnMHzmME1wj/cuQw19h2wIprAAP0taUS2J5FyGZSASuA5mhHMVwGFU2w vtRao6ZICuQ9J8OhLSMzjIxqZqbHJp39NYj6mmB+2Yio0MhQ/UiMbGWmtG1cJNE2hrPR tXBB7uLJWARTkguRjzRstizABnyZ1iGYh/OOTdz6yZ9P0dK+v1ZiA/G48M9cRd5CKH3E T/sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622802; x=1741227602; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=evC/AJYAKH4bntjSNStZX5d0W0KQw/JlJZazMUftcQU=; b=if0X2US7JXxBEdfdAp3stxn1WxamvBk/3ktq3fpvHh0XJVmDZjM69HUXToLWkcklBL 68J7Eo+VZK1rrH6elq+GiPhLSgOHhgmtVSVMbUz/i0i6Wlwi5AEeRbSdg88vQD65CSys cQGZ9PcI7OKtaBygtyEsggzgjBX3b1bSnCDR3p0WrVpDE10JBaDkkH6iTTkKyLmwGiBA 8PvsNZqEX2hfnVaFr0L+NPLUaRAuvl6uMR8NakQXAswpbJUIP2RoMWlGH/t6qcqryayr iOZ11azXvoQM1nmt4D9Hwq7bpixy3MrUaEqNYRVYO1cdOtb76mg3MUsRtjIBk4CQKWEH Rw4g== X-Forwarded-Encrypted: i=1; AJvYcCWyhn4hc8axM2/gQIXTP+gKLWhhDIUSZQH9Mf6mtk6nGVwAuQBa8/8OAUzKUfE1T8xz+QY=@vger.kernel.org X-Gm-Message-State: AOJu0Yzzu/en28tq8pQumOMyj7eChc8vd8tAM/i+Ez5vUPEbNQtvsqTv 9Ei/fam1yU6PY4TH2rkI/9WtEtBIWU6LiP+6u4kp//X+1YjF+E340PtTwQ2TfoteFO+CrpBHJth wAA== X-Google-Smtp-Source: AGHT+IGv/zpuplWrcklhWH3aZ9H+bWJRwaJtozM+byhaXyymURPHFd4oTcq5U1mZsa54ViZesa4/C8fD/pk= X-Received: from pjvb12.prod.google.com ([2002:a17:90a:d88c:b0:2ea:aa56:49c]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5101:b0:2fe:955d:cdb1 with SMTP id 98e67ed59e1d1-2fe955dd224mr3596029a91.23.1740622802051; Wed, 26 Feb 2025 18:20:02 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:50 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-35-seanjc@google.com> Subject: [PATCH v2 34/38] x86/kvmclock: Get CPU base frequency from CPUID when it's available From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania If CPUID.0x16 is present and valid, use the CPU frequency provided by CPUID instead of assuming that the virtual CPU runs at the same frequency as TSC and/or kvmclock. Back before constant TSCs were a thing, treating the TSC and CPU frequencies as one and the same was somewhat reasonable, but now it's nonsensical, especially if the hypervisor explicitly enumerates the CPU frequency. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index b924b19e8f0f..c45b321533e5 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -188,6 +188,20 @@ void kvmclock_cpu_action(enum kvm_guest_cpu_action action) } } +static unsigned long kvm_get_cpu_khz(void) +{ + unsigned int cpu_khz; + + /* + * Prefer CPUID over kvmclock when possible, as the base CPU frequency + * isn't necessarily the same as the kvmlock "TSC" frequency. + */ + if (!cpuid_get_cpu_freq(&cpu_khz)) + return cpu_khz; + + return pvclock_tsc_khz(this_cpu_pvti()); +} + /* * If we don't do that, there is the possibility that the guest * will calibrate under heavy load - thus, getting a lower lpj - @@ -418,7 +432,7 @@ void __init kvmclock_init(void) kvm_sched_clock_init(stable); - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_cpu_khz, tsc_properties); x86_platform.get_wallclock = kvm_get_wallclock; From patchwork Thu Feb 27 02:18:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993580 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7690D23958B for ; Thu, 27 Feb 2025 02:20:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622806; cv=none; b=SZO8DxlC6/t04MX2u7jR+Vahg+XAOqOIMW/jnOrpU0mK8UhBnm3y1euN/RX1xr5wTnSuXivGKoMTaPgm+ep05SKxqPAauV89lY10svlVESHuijeiXfn9DORmp6qhG23nt3azrsXbQOrDefufaqTC+u/rD3fZDrgEhSQVWExn67g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622806; c=relaxed/simple; bh=dSybtMuoZF9XQBwPnBkfo1UPLtlynUI+WZpyVl8WUu8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=AuPtZ7gtfNpFtfu2WrJOq1D79rkZZJ/a3mgFGcAhIVsR5h4hAJ5tapczdfKci/Z+L5JNtPZH2tszNvH+w3KK7Pwayne2UqSyseLBATcBewTrS+LpwM+HRwAzqC2h03AlbQeU7qwC5q5LHaA6iO1ptqe34ewNADOScp1vWwrLH3g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=yEKooXjh; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="yEKooXjh" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc3e239675so1567642a91.0 for ; Wed, 26 Feb 2025 18:20:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622804; x=1741227604; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=tqdKNUZqbb8d2CJvLoLFEQfT4OY26dkczDpR+rA3UIo=; b=yEKooXjhGutr0yoGJ9BYE8yYcdI1q/WwEB6uNmCnqyCCh1Ji3Yy7I1uCsFlMyj0iYr 0apXVa/6iHPLT1FwbgsWr3UTdCKOTD/fsK1AgTGNv09yQWAdM+/xlXhRSOrFJPBfzcQd KsggSannwx1q0Zp32G5uOTze8iItLZH2c2dncTTJVQXa9gTAeEmCKu23I0dU8uk3NaiN 0oLmwr6pKhoCw244suT0Yfn/hB8ujyqpW/kgRocDUqIKfrlcuMydOZyraUffXRh9gAV0 Brnr7nMVbq+xFNiCf29SLBBQviWQLaHewFSk8HdAdpb05XNhn37SEY6Z56xCv/q0iQ0q 0uVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622804; x=1741227604; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tqdKNUZqbb8d2CJvLoLFEQfT4OY26dkczDpR+rA3UIo=; b=ehxLE8b5dCv8KJtKOAfWIBX1uu33KrA8guUSFxEBLVfGPyyYrJCSe1UAcNz5ylCmbL rSyaCEwkecZEwT3d0BoGVnPIEFRt6bNDgvcSUKQIKBhlyQZoIKvWmWFdQzJEOyr2O50B CuCbn3o17Q42nodAG3hMS3QvEsUcdK8PO1i1lZRIWQlN5yKakTimqOXsoB2CNWKy3RIk JAqQV1bhsZgniWckv7/e6qroLtSGJgJyzv/V+ZG+k3GB/o+6Jr1fQ6fj1a+SHXX7C+Cm 6swcuLE+iiuyO+7mM7w35nxuMnpqbVVShvLNTMR/BN/8PaJzTecHdTbCZl2i5jbMoT/i IU2g== X-Forwarded-Encrypted: i=1; AJvYcCU+EJzWe4vWw/7g/0Ig3JwuPxJtRHgnUiBaadTQPR1rXjvwObfMhoSCv/sunqQV2owqeOc=@vger.kernel.org X-Gm-Message-State: AOJu0YzLa3D39vjzDJYdBCzJVyNOFIDCGuRtjPnUUl6hHrR/NBZIcolU X0Oa0wiv28YrPuULbqnK4tSLPtOuTpNuMGyaoTZpBBJERlxoO0PJLrZfVIl9Jow0BgXsJQ8kPEx yTg== X-Google-Smtp-Source: AGHT+IGMz3bH4S0o1xgsITZANeNn+Z/tGuVwcYyVyvqnl2NMWrRx99tESpyiBKte9PgB4aGO5q0vwXh4uHk= X-Received: from pjuj3.prod.google.com ([2002:a17:90a:d003:b0:2fc:b544:749e]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3c84:b0:2fc:c262:ef4b with SMTP id 98e67ed59e1d1-2fce86cf0e0mr42953377a91.18.1740622803772; Wed, 26 Feb 2025 18:20:03 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:51 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-36-seanjc@google.com> Subject: [PATCH v2 35/38] x86/kvmclock: Get TSC frequency from CPUID when its available From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When kvmclock and CPUID.0x15 are both present, use the TSC frequency from CPUID.0x15 instead of kvmclock's frequency. Barring a misconfigured setup, both sources should provide the same frequency, CPUID.0x15 is arguably a better source when using the TSC over kvmclock, and most importantly, using CPUID.0x15 will allow stuffing the local APIC timer frequency based on the core crystal frequency, i.e. will allow skipping APIC timer calibration. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index c45b321533e5..3efb837c7406 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -188,6 +188,16 @@ void kvmclock_cpu_action(enum kvm_guest_cpu_action action) } } +static unsigned long kvm_get_tsc_khz(void) +{ + struct cpuid_tsc_info info; + + if (!cpuid_get_tsc_freq(&info)) + return info.tsc_khz; + + return pvclock_tsc_khz(this_cpu_pvti()); +} + static unsigned long kvm_get_cpu_khz(void) { unsigned int cpu_khz; @@ -211,11 +221,6 @@ static unsigned long kvm_get_cpu_khz(void) * poll of guests can be running and trouble each other. So we preset * lpj here */ -static unsigned long kvm_get_tsc_khz(void) -{ - return pvclock_tsc_khz(this_cpu_pvti()); -} - static void __init kvm_get_preset_lpj(void) { unsigned long khz; From patchwork Thu Feb 27 02:18:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993581 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C295240604 for ; Thu, 27 Feb 2025 02:20:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622808; cv=none; b=UZT0baZ//hKymYViU5zPs0sRdsalWtdpsrApdkdBhBtYYElKab4WUct0Eo91kAGmZAt2/MqP8WBUVNIKjqYNeLXzZQfNEVLx93p9KEaIgoU68NgS1f0RWgb8gLlJTtteUsas2qHuBZ2hx0O2uayy0JESB5RHJ1e29Y32yWT9dv4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622808; c=relaxed/simple; bh=KkUJ82b9H7sinf/zXMFW+qV85JdkMspMIEMKpj9sh/I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=j8ZPesKIydY6xi+eSPJH92+RoETJB57tv2APukses1O9mZOOa9eGvwGqdJ1P7o8xte7BPemyt101a0AI3ON2vujCsiPlAoVJlznRMc43gmFZK+3WWpUuaa7i10Wkf/hLjnDxoyuSRAzosfmBIsPQtqVvlhuLpy5bTNCdq4Z+cAI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xDWDxp79; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xDWDxp79" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-220d8599659so8025015ad.0 for ; Wed, 26 Feb 2025 18:20:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622805; x=1741227605; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=pIteN/a7AZFre1KWgiuGeMT9S5HjiwYxYNebWBQLyrA=; b=xDWDxp79/1M3lU1GI0eNK0+ov4M4OQhhtamiQ2b5jpTvZt57VQVra8dae2Cq/IkdNu HldMFBMhLlBUsb7IsKR0lkNGVts9WVZT442m6HhowMFMIFPRZz/Y+zhoIDyJbWNPeG7E ssOTG8b83s48PlRX7a4JVDZCAsejjhW4sG1PHRqy80BFX1E4Wqc+Xuunr2/fNaCjJNYF CGzYEvzo2TQi483h25AE98d5KqQJE/LkSfbN/mcxL3XOr1fA0Hf96d4DBs2i0myXUQGu UQJN4ByTXK00xz7Ybmd11NvuwYT/HEgbZOrRVZzrpOdSPgaqd7r7+YIkl6+aWHh4M8YP 68mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622805; x=1741227605; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=pIteN/a7AZFre1KWgiuGeMT9S5HjiwYxYNebWBQLyrA=; b=MJ30B2AmXBK2EYd9vvoqy2VkRjU89Q6BJOSaPXWfhwOx5E/T3SS86wULaUW1Nwj1Gi 4XDE+CqHR6tMSa2lhoYkJVfAYwtn1oT5DaGzeQxqOIZ/yDVHhTVZvXZQAeWEZffRy5k6 fvq8WRbPaU8NKm4N/5y+5b/3Lv5a0CkBEw30e2CH3VXQcJKFZcKp9oGIxP5hGkDzW835 mMscRa+dRMz1uv2Xml/cI7Lio/wbux0nWXX11PvwnWe/XFXEy0zUiupqVmsXkEnvkeKk HIQfNWfZqITGGBLwRj57oGVO0x+gX6zoURHrU1t7LD1fywZr5KSNn8xAKyOdHxXd6V+j AHmA== X-Forwarded-Encrypted: i=1; AJvYcCWAdW6Z3MbL2t0BuJ5U1W7Q5Ka1g1VHx5su/JPqv2yr8No5hk/49pwRplvEl1UeeF5YNk8=@vger.kernel.org X-Gm-Message-State: AOJu0Yy3MQfg5HFYcMwLXTv1eTUITAwSmrXvRvZN0/GKgvZjM7jaeOZF UfGL9Xw4eWfPfVYM0P3okcTr9am0GHIqiXXuBjjLuCY4mBaiFJVeaOzIdSU5toGcE/REFHB0hMb uGg== X-Google-Smtp-Source: AGHT+IGP2cZX6cw4gCRMv2fLrFe/TXNI5NPzTOobUK44ADJodiEA9LQbSXwMRfyGc0rCQ4WOggdSpeUhPyc= X-Received: from pfhk13.prod.google.com ([2002:aa7:998d:0:b0:730:479d:3482]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2181:b0:731:ff1b:dd6a with SMTP id d2e1a72fcca58-7348be4c455mr8342588b3a.20.1740622805598; Wed, 26 Feb 2025 18:20:05 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:52 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-37-seanjc@google.com> Subject: [PATCH v2 36/38] x86/kvmclock: Stuff local APIC bus period when core crystal freq comes from CPUID From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When running as a KVM guest with kvmclock support enabled, stuff the APIC timer period/frequency with the core crystal frequency from CPUID.0x15 (if CPUID.0x15 is provided). KVM's ABI adheres to Intel's SDM, which states that the APIC timer runs at the core crystal frequency when said frequency is enumerated via CPUID.0x15. The APIC timer frequency will be the processor’s bus clock or core crystal clock frequency (when TSC/core crystal clock ratio is enumerated in CPUID leaf 0x15). Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 3efb837c7406..80d9c86e0671 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -192,8 +192,18 @@ static unsigned long kvm_get_tsc_khz(void) { struct cpuid_tsc_info info; - if (!cpuid_get_tsc_freq(&info)) + /* + * Prefer CPUID over kvmclock when possible, as CPUID also includes the + * core crystal frequency, i.e. the APIC timer frequency. When the core + * crystal frequency is enumerated in CPUID.0x15, KVM's ABI is that the + * (virtual) APIC BUS runs at the same frequency. + */ + if (!cpuid_get_tsc_freq(&info)) { +#ifdef CONFIG_X86_LOCAL_APIC + lapic_timer_period = info.crystal_khz * 1000 / HZ; +#endif return info.tsc_khz; + } return pvclock_tsc_khz(this_cpu_pvti()); } From patchwork Thu Feb 27 02:18:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993582 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27FD42405E3 for ; Thu, 27 Feb 2025 02:20:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622809; cv=none; b=jM3DTorLdnUDNTRXVwyguKzVkt/TGPej1/fVqrNb7U5mopzXNlTEx/04q6ZqFdR0YnVB9thfQmuBuk5rEzKXBRrBTlMEZIKaz764cWazfNfX94hMXnmiNFjtt4Ge7CWwiCLbjnzIqifbFq3W/gHVDK4idFMN+MjVcYmh8bhM/DQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622809; c=relaxed/simple; bh=8tMxUjf29DV5ZoAnNO0UTJZHJbBzjdRYlL7FtigAdUc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=iMvscfmeYqfTaAR58b4FHZevE343kOfzYijFJZpoSVICgsOrMU19evH/f2HO+YTJBFeDyqpfyT8DB+8eAyeb+U5qUFM6KcZlhADxrov4R/JHrWt/+CBZaFCtFcYSmPVkHIU0/bsMTnVqP6yANTyCiF6FpjwgTn3K4jkjZnXv0Ag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pBDqtGIv; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pBDqtGIv" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fe862ea448so1504311a91.3 for ; Wed, 26 Feb 2025 18:20:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622807; x=1741227607; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KI0queLIxZnghHgRHs7xWlDt+M3CNsR2JqWjVaFF+mE=; b=pBDqtGIvdseibNlSC2xxFsNKxOUMOixkWOZrHiSWF9VvxGM5wYxMz38pXHJiOSBfpp WtPOMpn/QEksEHUxgkwgxLjNTesfRvZqEX7p6nuBcY22WdtRpwoYaTfxLQg7mfk9Uo0/ 0arN2z7cxw2uT7H1OPlTYgcBDXgT5CMrgMfFTc6Ph1wZOL4uWrrDw6Mqr8xSxdQYcifH t9m1G+3dPqBu8k9U1wl+deJu1l6B2VsYmGNzGha6wShGAmHU6jOCR4mOjLPwxnxnGaZ5 7a5lhZGXnmKMz33HCEKO8LSlkOhW2vIjiLM4Vd8gvtnsUsVVWY/oKbT+3CpDmLf8SMz6 uJhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622807; x=1741227607; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KI0queLIxZnghHgRHs7xWlDt+M3CNsR2JqWjVaFF+mE=; b=kWMO5kn+6sA+Fse5hBLoH3IzmA7yQatneFPrOXjj3noeeEKq10a1KcKiLd1sC2amlX KjCis7r3MeoFo2l5e2JbT13T7okiTeQOuwvqG5LidINYgZkBufMI/i6cCarQOvo7nWi9 lGGu+7/V7vQyynnQ8WZaYflc6JkPA14XomE2xY/hVBpOPfCg6ivKKtcYQ0OmEWRtBcV5 yhRJG2A1Ofzi9sIwhcsqgMTWe04hWF8+0WBqHYg8K1vw9TxKcWusT25N7AyEpw/jzv3M jzx/jTZxSRinGNDXkvJ/VP1lYJSgrfHvTegkq+DG+BPZD3kwzBA0Mdw2UQ0X9MVe4DgN aypw== X-Forwarded-Encrypted: i=1; AJvYcCVyPQlqd26VKCYYGbaLTIjqX21uevY0n4RLATlv/kp9XyDt40c+uRVv8u/h6lREVR51uZg=@vger.kernel.org X-Gm-Message-State: AOJu0Yyo589Q8avcypcO1eQnY8f/blQbp6fEWPiTWqDzN/AFhlTEu7f6 iOtn3DgsDgiAIrH+0694rh6XxgJn2qQ22yTzJbOmBWnomUEdG4LEcrPZbAaIpu+1LIT6OYOOG3C UfQ== X-Google-Smtp-Source: AGHT+IFuRPRb2/rR+n2YIE9T1JgT09dE9pwcHYDYXuVdPUREy4VCpJNwziBTuQzC841m2IS5b8M/SH3vvo0= X-Received: from pjbsf13.prod.google.com ([2002:a17:90b:51cd:b0:2fc:e37d:85dc]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3d50:b0:2ee:9b2c:3253 with SMTP id 98e67ed59e1d1-2fe692c6c47mr14798005a91.30.1740622807422; Wed, 26 Feb 2025 18:20:07 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:53 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-38-seanjc@google.com> Subject: [PATCH v2 37/38] x86/kvmclock: Use TSC for sched_clock if it's constant and non-stop From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Prefer the TSC over kvmclock for sched_clock if the TSC is constant, nonstop, and not marked unstable via command line. I.e. use the same criteria as tweaking the clocksource rating so that TSC is preferred over kvmclock. Per the below comment from native_sched_clock(), sched_clock is more tolerant of slop than clocksource; using TSC for clocksource but not sched_clock makes little to no sense, especially now that KVM CoCo guests with a trusted TSC use TSC, not kvmclock. /* * Fall back to jiffies if there's no TSC available: * ( But note that we still use it if the TSC is marked * unstable. We do this because unlike Time Of Day, * the scheduler clock tolerates small errors and it's * very important for it to be as fast as the platform * can achieve it. ) */ The only advantage of using kvmclock is that doing so allows for early and common detection of PVCLOCK_GUEST_STOPPED, but that code has been broken for nearly two years with nary a complaint, i.e. it can't be _that_ valuable. And as above, certain types of KVM guests are losing the functionality regardless, i.e. acknowledging PVCLOCK_GUEST_STOPPED needs to be decoupled from sched_clock() no matter what. Link: https://lore.kernel.org/all/Z4hDK27OV7wK572A@google.com Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 80d9c86e0671..280bb964f30a 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -431,22 +431,22 @@ void __init kvmclock_init(void) } /* - * X86_FEATURE_NONSTOP_TSC is TSC runs at constant rate - * with P/T states and does not stop in deep C-states. - * - * Invariant TSC exposed by host means kvmclock is not necessary: - * can use TSC as clocksource. - * + * If the TSC counts at a constant frequency across P/T states, counts + * in deep C-states, and the TSC hasn't been marked unstable, prefer + * the TSC over kvmclock for sched_clock and drop kvmclock's rating so + * that TSC is chosen as the clocksource. Note, the TSC unstable check + * exists purely to honor the TSC being marked unstable via command + * line, any runtime detection of an unstable will happen after this. */ if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && !check_tsc_unstable()) { kvm_clock.rating = 299; tsc_properties = TSC_FREQ_KNOWN_AND_RELIABLE; + } else { + kvm_sched_clock_init(stable); } - kvm_sched_clock_init(stable); - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_cpu_khz, tsc_properties); From patchwork Thu Feb 27 02:18:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993583 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89D2724339C for ; Thu, 27 Feb 2025 02:20:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622811; cv=none; b=A0kq9t6kjEp/nalC80vvnq6l/4nqYASVxePZctiNp0I1ipG9V0fLLevzPotdcau+7XDewyF9mFYRsFwjGMpPRjhVGg6+bTyHaT9ItVJTYzGZCOr4SQ9kG6XMle7bpIpTTPflqi5FAAh3DFVlef3G6dSziyBhtSznpzS+MiVT3dw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740622811; c=relaxed/simple; bh=hxlTZC8hhB/2jCSFfc8NfglIDRkf8SaM4Ur2KmgRPXU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=UtnWH3aYhZf5AGvO+O9B7+3WGhDwNrcfNEPHFMIA3c10i4LdczUDkoVda89KvfkudzLSV14AGTsUOknuMHd7Ilxm4ntF8J3AUf8GbgyYIpH4e5m/7w7xueKI4vh67biym3odRYobrXtYdDCaXYBq6RrVh5qneDw5J1efg5YK31k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=kInpiMw5; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kInpiMw5" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2fc1cb0c2cbso1527805a91.1 for ; Wed, 26 Feb 2025 18:20:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622809; x=1741227609; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=7QXAixlOaLHnLrPcLZCTebtwCxVmNNMI9kZmebDzK/g=; b=kInpiMw5cFJpmKPKzQWqgAYgEFH7NA5HxAfNs+JX3x7c2qEI6Uk36skqLbrkXzYmFr +FCW/m70MEJIr4OxPaqOyTxBLImJt78g6Yj29Cxf3p6CMNbKr1a2exuVPPjOqmkVVA3u bE7ycJhluAYX2g/2mwoGSt778RLrsjxbfHnkx+BG6/xDYsp4CfXcMpuDcLhPWUUtDzWn LLJXrEMfC7nFwMOa6oSyKtMuKi1U5VcLIVIK+3LlVzBTwTAxMMzalNbG6DFwY2qDrvrj m1aqtvJtyh2cAzFSrndlfPwxuOCZZUNX5TTQZTG5QRHW68SfAJx6WLQ4XnOPDGyYCU+p Prqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622809; x=1741227609; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7QXAixlOaLHnLrPcLZCTebtwCxVmNNMI9kZmebDzK/g=; b=trl/dboAJfH/vS9wtIX2vUnqS4xbups3cx1oLme5XymbpdZBids6zGkwBEygpjFSM+ P4RjNUas8w0Y2AIUdllOmEyOQxBz67PJO3TXGAllvf0XIKjZQoW0r3bS78+TIiT7SNQS h+Ih64+HTby4ZT+trMlXTRC9AL4r3eKXroGawrFHz3OBfUvJsgQI6jIm0kQMwmWhpFRa 9MN6i2+kpDchwHyaJ3CvPL6aEpAAh4AIBOKrobEA0DC2J/I455mmpPCYEKORpm1OTo1J Gd6raUoikOeycOMdBsh0SHqHzCiLbJ3l5RiAsca+VEvodPb1f6aRMnlRfApjxH/BsLNz xoyQ== X-Forwarded-Encrypted: i=1; AJvYcCUNVQ1sn3UIA7DFaNzSwxQYeVRyAN8ZDu1Y6WiPL8Nhos3MmuFRHn59bsmlvBiRLU5ugFg=@vger.kernel.org X-Gm-Message-State: AOJu0YyC0BQUjKc6+8gzHVFh5P5YiTRZMDH5lLQ9QdBq5LplrEAXtz7T EsYNtd5B53GaAuBKUC1EC5zP1DN0Jm9u69mk7Pwja7kZus9jDEnCSZ1WiKgARrcnfgtA38tTw+K Faw== X-Google-Smtp-Source: AGHT+IHi4d1QOUfXEPfDRjR4DxqWNosFyXVnkFi4SDlf4Oxmtkdi3La4kP/2IDxMG83GHnzWpK766unKU5E= X-Received: from pjz6.prod.google.com ([2002:a17:90b:56c6:b0:2fc:3022:36b8]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5688:b0:2ee:d63f:d77 with SMTP id 98e67ed59e1d1-2fe68adec61mr16362401a91.9.1740622809153; Wed, 26 Feb 2025 18:20:09 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:54 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-39-seanjc@google.com> Subject: [PATCH v2 38/38] x86/paravirt: kvmclock: Setup kvmclock early iff it's sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Rework the seemingly generic x86_cpuinit_ops.early_percpu_clock_init hook into a dedicated PV sched_clock hook, as the only reason the hook exists is to allow kvmclock to enable its PV clock on secondary CPUs before the kernel tries to reference sched_clock, e.g. when grabbing a timestamp for printk. Rearranging the hook doesn't exactly reduce complexity; arguably it does the opposite. But as-is, it's practically impossible to understand *why* kvmclock needs to do early configuration. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 10 ++++++++-- arch/x86/include/asm/x86_init.h | 2 -- arch/x86/kernel/kvmclock.c | 13 ++++++------- arch/x86/kernel/paravirt.c | 18 +++++++++++++++++- arch/x86/kernel/smpboot.c | 2 +- arch/x86/kernel/x86_init.c | 1 - 6 files changed, 32 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index 5de31b22aa5f..8550262fc710 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -29,13 +29,14 @@ DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)); + void (*save)(void), void (*restore)(void), + void (*start_secondary)); static __always_inline void paravirt_set_sched_clock(u64 (*func)(void), void (*save)(void), void (*restore)(void)) { - (void)__paravirt_set_sched_clock(func, true, save, restore); + (void)__paravirt_set_sched_clock(func, true, save, restore, NULL); } static __always_inline u64 paravirt_sched_clock(void) @@ -43,6 +44,8 @@ static __always_inline u64 paravirt_sched_clock(void) return static_call(pv_sched_clock)(); } +void paravirt_sched_clock_start_secondary(void); + struct static_key; extern struct static_key paravirt_steal_enabled; extern struct static_key paravirt_steal_rq_enabled; @@ -756,6 +759,9 @@ void native_pv_lock_init(void) __init; static inline void native_pv_lock_init(void) { } +static inline void paravirt_sched_clock_start_secondary(void) +{ +} #endif #endif /* !CONFIG_PARAVIRT */ diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h index 213cf5379a5a..e3456def5aea 100644 --- a/arch/x86/include/asm/x86_init.h +++ b/arch/x86/include/asm/x86_init.h @@ -187,13 +187,11 @@ struct x86_init_ops { /** * struct x86_cpuinit_ops - platform specific cpu hotplug setups * @setup_percpu_clockev: set up the per cpu clock event device - * @early_percpu_clock_init: early init of the per cpu clock event device * @fixup_cpu_id: fixup function for cpuinfo_x86::topo.pkg_id * @parallel_bringup: Parallel bringup control */ struct x86_cpuinit_ops { void (*setup_percpu_clockev)(void); - void (*early_percpu_clock_init)(void); void (*fixup_cpu_id)(struct cpuinfo_x86 *c, int node); bool parallel_bringup; }; diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 280bb964f30a..11f078b91f22 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -126,12 +126,13 @@ static void kvm_save_sched_clock_state(void) kvmclock_disable(); } -#ifdef CONFIG_SMP -static void kvm_setup_secondary_clock(void) +static void kvm_setup_secondary_sched_clock(void) { + if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_SMP))) + return; + kvm_register_clock("secondary cpu, sched_clock setup"); } -#endif static void kvm_restore_sched_clock_state(void) { @@ -367,7 +368,8 @@ static void __init kvm_sched_clock_init(bool stable) { if (__paravirt_set_sched_clock(kvm_sched_clock_read, stable, kvm_save_sched_clock_state, - kvm_restore_sched_clock_state)) + kvm_restore_sched_clock_state, + kvm_setup_secondary_sched_clock)) return; kvm_sched_clock_offset = kvm_clock_read(); @@ -452,9 +454,6 @@ void __init kvmclock_init(void) x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; -#ifdef CONFIG_SMP - x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; -#endif kvm_get_preset_lpj(); clocksource_register_hz(&kvm_clock, NSEC_PER_SEC); diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index c538c608d9fb..f93278ddb1d2 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -86,8 +86,13 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); +#ifdef CONFIG_SMP +static void (*pv_sched_clock_start_secondary)(void) __ro_after_init; +#endif + int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)) + void (*save)(void), void (*restore)(void), + void (*start_secondary)) { /* * Don't replace TSC with a PV clock when running as a CoCo guest and @@ -104,9 +109,20 @@ int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, static_call_update(pv_sched_clock, func); x86_platform.save_sched_clock_state = save; x86_platform.restore_sched_clock_state = restore; +#ifdef CONFIG_SMP + pv_sched_clock_start_secondary = start_secondary; +#endif return 0; } +#ifdef CONFIG_SMP +void paravirt_sched_clock_start_secondary(void) +{ + if (pv_sched_clock_start_secondary) + pv_sched_clock_start_secondary(); +} +#endif + /* These are in entry.S */ static struct resource reserve_ioports = { .start = 0, diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index c10850ae6f09..e6fff67dd264 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -278,7 +278,7 @@ static void notrace start_secondary(void *unused) cpu_init(); fpu__init_cpu(); rcutree_report_cpu_starting(raw_smp_processor_id()); - x86_cpuinit.early_percpu_clock_init(); + paravirt_sched_clock_start_secondary(); ap_starting(); diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c index 0a2bbd674a6d..1d4cf071c74b 100644 --- a/arch/x86/kernel/x86_init.c +++ b/arch/x86/kernel/x86_init.c @@ -128,7 +128,6 @@ struct x86_init_ops x86_init __initdata = { }; struct x86_cpuinit_ops x86_cpuinit = { - .early_percpu_clock_init = x86_init_noop, .setup_percpu_clockev = setup_secondary_APIC_clock, .parallel_bringup = true, };