From patchwork Sat Feb 1 02:17:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956066 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEE4135950 for ; Sat, 1 Feb 2025 02:17:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376252; cv=none; b=utUmH8wxQjcXQyeKaNH3BVu7rKtGi72Lj+atK2f4oTivxWFD8Bn8+IcJDz79OfttGy+yiruN1PcAelKJfQe7VVVU/viGbZtQwUcywMZ8pNKPhQU5TzJ4i8Rj3nsK1q9rRc13awLQMVnCc0zUMIOob694a1LnGhdr34vrcRyrev8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376252; c=relaxed/simple; bh=bqOChW42KkISzQ0ttdKpX0A7QPv+hN2zNVexU8vjJYE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=iEnPgnrbsvijxJq9f+nTxBzYHM/qvjdvkFeP7mwIkWDDl5y8GBICXGQlTxPpcocX8L0GvaR+E6r41fSxmls9QQJTAZj9g/W+bC1U4FvLGobphMKuqCVhX5O9dFS/8mUY6YMlBP2Tge+l0TrsL/IbR6RUf+XV2AlaH7NAAuhUE44= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YYT3jl7Q; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YYT3jl7Q" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ef35de8901so4878768a91.3 for ; Fri, 31 Jan 2025 18:17:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376249; x=1738981049; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3q3VtWChJd6qluhOhoVrZUOxKE25zEDROrS3h+FyOz0=; b=YYT3jl7Qcm+YyvBH+OyECD5VrHlZTrVZJ3yzrUR8eD648UtiQlm1h/e4Pq3C8HyTow E6oNZXNX/OyHE/o0B6g5ak+AFK2677jIiTSmrnmBgw3SlwG998OvMbCa3gM8XZkoonVq XN5BtzQeiTHcmAew148U2Y+znB6qvjCYYkc1cZZF6y8Ar3PEzv0uJJBp1JG2XV74xi7t 9YInCNfIQ4txnuSd/KrBa0C5gphOoic0u0UnUJHAyN74Rforg+qCsrQW5uA2QgWQCh8T /zi8xXWadKjHSbFvdY3hxlbvbs2KKWIW43IVidIHmLlIFrUlBJI1Vr8eij+dxqyiwWoR haAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376249; x=1738981049; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3q3VtWChJd6qluhOhoVrZUOxKE25zEDROrS3h+FyOz0=; b=Rzxs0mIZQHuoaR3BAC4uJZkQDpMslVvIsqp5UFtxKdmK9t8Iaqy/F4nJHd6ORznI41 ozlHQqxCfO4KIOJSpkby8toBUD+Y1Z+L3Fu87pDEwQkwwe5SPnZn8ph9ZdKpaNt5d7x0 GlEFxwW7Q0S9bDpvoD0FW/KEfNGHnVwaIok7/eRfo2D9PdgyQYRarkNmoAjTNMWecH7a i9nenEYEVLAh60WBsxXUBx3mJ39tGojpMyRGdQPjRsaG2AL8AN9/zvCta6A9jz7oxUDv u/FuXx9Y6C7x9Pw59NAuwAjEhD8alueHR9fbpgj3lkoN0saVPbZrZMLZnn9LDBVudYP0 rMaw== X-Forwarded-Encrypted: i=1; AJvYcCUAlFJnUcjw+FOmP+kCbFzFwBIzCjWih+lE9p2kaUfxaRAr9QMwxbO8ckypKtyngddtg4s=@vger.kernel.org X-Gm-Message-State: AOJu0YxViVxIodHco7HrBiAmK7/qNldVBf7WJBZs8Pln5WIQUdtrHYQM +hRt38c43IDMN7MEzXqYfNfkll+b0roOAQe/lNh1ihoy0JLplUSiZ3PaDXyq9NqEXkNRTadCEoG H7A== X-Google-Smtp-Source: AGHT+IGlJL+gh3dhTIPqGXIyfYqBO171NEVUqAKwIwg+mryw0iUEaDc1VWsrQ1YFiTTJEYVX1rRFTepD4XE= X-Received: from pjbpx11.prod.google.com ([2002:a17:90b:270b:b0:2e9:5043:f55b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:fc46:b0:2ee:af31:a7b3 with SMTP id 98e67ed59e1d1-2f83ab8c3edmr21370350a91.7.1738376249247; Fri, 31 Jan 2025 18:17:29 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:03 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-2-seanjc@google.com> Subject: [PATCH 01/16] x86/tsc: Add a standalone helpers for getting TSC info from CPUID.0x15 From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Extract retrieval of TSC frequency information from CPUID into standalone helpers so that TDX guest support and kvmlock can reuse the logic. Provide a version that includes the multiplier math as TDX in particular does NOT want to use native_calibrate_tsc()'s fallback logic that derives the TSC frequency based on CPUID.0x16 when the core crystal frequency isn't known. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/tsc.h | 41 ++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/tsc.c | 14 ++----------- 2 files changed, 43 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index 94408a784c8e..14a81a66b37c 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -28,6 +28,47 @@ static inline cycles_t get_cycles(void) } #define get_cycles get_cycles +static inline int cpuid_get_tsc_info(unsigned int *crystal_khz, + unsigned int *denominator, + unsigned int *numerator) +{ + unsigned int ecx_hz, edx; + + if (boot_cpu_data.cpuid_level < CPUID_LEAF_TSC) + return -ENOENT; + + *crystal_khz = *denominator = *numerator = ecx_hz = edx = 0; + + /* CPUID 15H TSC/Crystal ratio, plus optionally Crystal Hz */ + cpuid(CPUID_LEAF_TSC, denominator, numerator, &ecx_hz, &edx); + + if (!*denominator || !*numerator) + return -ENOENT; + + /* + * Note, some CPUs provide the multiplier information, but not the core + * crystal frequency. The multiplier information is still useful for + * such CPUs, as the crystal frequency can be gleaned from CPUID.0x16. + */ + *crystal_khz = ecx_hz / 1000; + return 0; +} + +static inline int cpuid_get_tsc_freq(unsigned int *tsc_khz, + unsigned int *crystal_khz) +{ + unsigned int denominator, numerator; + + if (cpuid_get_tsc_info(tsc_khz, &denominator, &numerator)) + return -ENOENT; + + if (!*crystal_khz) + return -ENOENT; + + *tsc_khz = *crystal_khz * numerator / denominator; + return 0; +} + extern void tsc_early_init(void); extern void tsc_init(void); extern void mark_tsc_unstable(char *reason); diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 34dec0b72ea8..e3faa2b36910 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -661,25 +661,15 @@ static unsigned long quick_pit_calibrate(void) */ unsigned long native_calibrate_tsc(void) { - unsigned int eax_denominator, ebx_numerator, ecx_hz, edx; + unsigned int eax_denominator, ebx_numerator; unsigned int crystal_khz; if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) return 0; - if (boot_cpu_data.cpuid_level < CPUID_LEAF_TSC) + if (cpuid_get_tsc_info(&crystal_khz, &eax_denominator, &ebx_numerator)) return 0; - eax_denominator = ebx_numerator = ecx_hz = edx = 0; - - /* CPUID 15H TSC/Crystal ratio, plus optionally Crystal Hz */ - cpuid(CPUID_LEAF_TSC, &eax_denominator, &ebx_numerator, &ecx_hz, &edx); - - if (ebx_numerator == 0 || eax_denominator == 0) - return 0; - - crystal_khz = ecx_hz / 1000; - /* * Denverton SoCs don't report crystal clock, and also don't support * CPUID_LEAF_FREQ for the calculation below, so hardcode the 25MHz From patchwork Sat Feb 1 02:17:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956067 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68361446A1 for ; Sat, 1 Feb 2025 02:17:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376252; cv=none; b=dvbERAmhXBj60jYBAONnptL60lc0sJfJE+oQs9NW4OPw02OqWB9SM8SfZ7B6JOtfkrzMH4wV/yDEhEHEy13RzxupnA7WcC+Qj1exkDMpj0b9hqndT+KmlnigTk/w0qLBYnLH0Q/2euhUib5kUK/1BaTS/p/GEjtoXyVmvp5lBn4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376252; c=relaxed/simple; bh=TyPGGnI+qvESlCqTdnx/2r6KTTCSxp5Z/qlgDjzTlP0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gUJ4tjdXApVRah41WuQRvzIIvQTlm3Rzb8KPXN9d1cnve2ov2Pp49etme5s3O8x50V+yKcENoBIdcWulk/Bvtv9U+8cwAHJbSUOFoSUHSj/NOgsPgbhKavwj+qURNyWgs68TzJIuDAb8sGGS7rjKvNqH/0FyaLFVbUDDwbkJjto= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=k8gbP36v; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="k8gbP36v" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-216405eea1fso52301635ad.0 for ; Fri, 31 Jan 2025 18:17:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376251; x=1738981051; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=E0C6rmJeK4TduAmeKIdyYwLof9bH1hCIL8jVX3VQ6HM=; b=k8gbP36vFJT0lD4k/FBbCT4q4E2h6FpPE418yfW/LP7RkR/W18L6zr+JEDqAryPMit xkpWbUrL+xIe5w11xBdlicJ6OOW1TKZYi+baWGQGGdGZ49y52KFXLA/4rGzn48CjdU9B 3O5JHOq2+ixlSUeKvauPJ423LNWcksqaAUf4vz0pglxtkbvZIZDADZrPRYlMVgxQIqLE xYTUz7856a0NSIFYRBiTF0XRrpT8VXr28iRMREqmYzW6OqElo9bL5N0Ez3wHqqwtrnkE 23QOTt4jRs95o3YoixUG6NXmMMHHlyi0YU1+htD9GlIoBFkxQDdSuJ+4CNG71hxm6+ti kAGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376251; x=1738981051; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=E0C6rmJeK4TduAmeKIdyYwLof9bH1hCIL8jVX3VQ6HM=; b=gGDTQparh67kGAFeYcA7bl1l95WEdJMvxkqpuXrgCoWNs1ueSH0deLvS47KCxucLTy 99Q7T5ATNyfohI5qEfTL+/tSR5wZu3blmDp5XjKyF6fOMw0S5VufWrRLM+3YzH7xNuiz AGdjjhDH5jAdIQu+uTHl0xYE4/TQJtamvVu3NPJTa9AD+QBc9yfTwfF0yOj1sqHXEg2s jfjGargfu7wTJa4oKATw0tJWtzoHrST1AoMEZMRAjIMnKBb4vIpgSpnj5IISVUhTTItD DPmJ6kShXfEA4jQ+ql0CogQdilHubNhTHzxXMom0NnEvrNq+Ugaq8h1nVs26hCEUATXh m/zg== X-Forwarded-Encrypted: i=1; AJvYcCUWuaw4RTHLEbcENeas92nqMqe2VizJlQwS9rJty8hJ+JndACsXCCq8xq40MvlW8ZZQB/I=@vger.kernel.org X-Gm-Message-State: AOJu0Ywt2j1bXxpFro4raI1CteXffzlfSd8VSJ8n2JU7N0jMiZHEfhdu JNuqOnF8C5owbl6y+DfWmvwXi+NCP/BTdXkHiDZizbYqX9mfDqf9krf+eHThXgaXXylB+5syl2m uKg== X-Google-Smtp-Source: AGHT+IFlvWiOED1ZYXsClK6j8veie9a9mJfZCZQPkiq0i15PJ6F5P2kJWMXEZplyquyEAP3l8GbEgk3YB0s= X-Received: from pghg16.prod.google.com ([2002:a63:e610:0:b0:ad0:f8ff:b90d]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6300:6713:b0:1e0:d766:8da1 with SMTP id adf61e73a8af0-1ed7a6e12f1mr21226332637.39.1738376250763; Fri, 31 Jan 2025 18:17:30 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:04 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-3-seanjc@google.com> Subject: [PATCH 02/16] x86/tsc: Add standalone helper for getting CPU frequency from CPUID From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Extract the guts of cpu_khz_from_cpuid() to a standalone helper that doesn't restrict the usage to Intel CPUs. This will allow sharing the core logic with kvmclock, as (a) CPUID.0x16 may be enumerated alongside kvmclock, and (b) KVM generally doesn't restrict CPUID based on vendor. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/tsc.h | 16 ++++++++++++++++ arch/x86/kernel/tsc.c | 21 ++++++--------------- 2 files changed, 22 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index 14a81a66b37c..540e2a31c87d 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -69,6 +69,22 @@ static inline int cpuid_get_tsc_freq(unsigned int *tsc_khz, return 0; } +static inline int cpuid_get_cpu_freq(unsigned int *cpu_khz) +{ + unsigned int eax_base_mhz, ebx, ecx, edx; + + if (boot_cpu_data.cpuid_level < CPUID_LEAF_FREQ) + return -ENOENT; + + cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx, &ecx, &edx); + + if (!eax_base_mhz) + return -ENOENT; + + *cpu_khz = eax_base_mhz * 1000; + return 0; +} + extern void tsc_early_init(void); extern void tsc_init(void); extern void mark_tsc_unstable(char *reason); diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index e3faa2b36910..4fc633ac5873 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -662,7 +662,7 @@ static unsigned long quick_pit_calibrate(void) unsigned long native_calibrate_tsc(void) { unsigned int eax_denominator, ebx_numerator; - unsigned int crystal_khz; + unsigned int crystal_khz, cpu_khz; if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) return 0; @@ -692,13 +692,8 @@ unsigned long native_calibrate_tsc(void) * clock, but we can easily calculate it to a high degree of accuracy * by considering the crystal ratio and the CPU speed. */ - if (crystal_khz == 0 && boot_cpu_data.cpuid_level >= CPUID_LEAF_FREQ) { - unsigned int eax_base_mhz, ebx, ecx, edx; - - cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx, &ecx, &edx); - crystal_khz = eax_base_mhz * 1000 * - eax_denominator / ebx_numerator; - } + if (crystal_khz == 0 && !cpuid_get_cpu_freq(&cpu_khz)) + crystal_khz = cpu_khz * eax_denominator / ebx_numerator; if (crystal_khz == 0) return 0; @@ -725,19 +720,15 @@ unsigned long native_calibrate_tsc(void) static unsigned long cpu_khz_from_cpuid(void) { - unsigned int eax_base_mhz, ebx_max_mhz, ecx_bus_mhz, edx; + unsigned int cpu_khz; if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) return 0; - if (boot_cpu_data.cpuid_level < CPUID_LEAF_FREQ) + if (cpuid_get_cpu_freq(&cpu_khz)) return 0; - eax_base_mhz = ebx_max_mhz = ecx_bus_mhz = edx = 0; - - cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx_max_mhz, &ecx_bus_mhz, &edx); - - return eax_base_mhz * 1000; + return cpu_khz; } /* From patchwork Sat Feb 1 02:17:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956068 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 191EC84D0F for ; Sat, 1 Feb 2025 02:17:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376254; cv=none; b=uu5ENt3pQlSnQobCDWGPmaQLW8yOQoZXTJ82wwsbpnhNomKMRekakMlOa6ImVfy9pa7QP6+mkqf9khHnnJkZ3/a1iMNdEKYAd9UiJscssDEta3WKqx2p6kXXubvuOewP4lIcGcow41btdgi7y4+URAGxWtb2Hk0snJW2yHiGSz4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376254; c=relaxed/simple; bh=jYmI/eXpvsg2cQw9fM1WNGkxu+qGdy0jshAF8Oe8hpc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZgZ1Y9Kuj9ZcHIpa0m/RJeFQEQkzFAtEtgGpSHuLbt3MkCVquymaz57pUv6rV0utMqRWhNM05SyL3hKhAIu6c5sQS1PnWtsdIxSqH2GPcbJsAMZXdZ8Jm764ZOFI5UOcyv+wxTZXEpwEb448FdjvM37UDSSfaYqUTnVnka7la0s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MuvGHiMl; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MuvGHiMl" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ef9da03117so7118129a91.1 for ; Fri, 31 Jan 2025 18:17:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376252; x=1738981052; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=jT3QYFJ8LBBvWgIRRoR6lTexg7BmHolHr3L1o5rQg1s=; b=MuvGHiMluEw4cTn+5fuVHv799QGTYc1ZAF60q8gekw5CCCrbsI/VYuY+I7AXurCVMy v+QsMPAi/B75dMxtfYhWMURQXJUzeehiFoCJMAkdvVapTu4jqu7wWBdsyjzuAJrGL8Cq aluwRRVHrlgcFvHPuD0Vpx84Khyu8OiFgKew6kVmoGE6Xs+kemM89kr6z7jCgeR30SCO wsgOxbig9rX1Y2DlcM7s3i6ho+NfAKW1XjZXT5CGw47jMtmM0lWGNI7/CEMt6BFXUqUf r5EjQQBHBr6jTwvfdilXF8gEs3OzD9QQsfb4MMtLsH0od1EAdOYR9yJYPPNgbRPOmFtz qm0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376252; x=1738981052; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jT3QYFJ8LBBvWgIRRoR6lTexg7BmHolHr3L1o5rQg1s=; b=IKFezbI62poytchpYe1IPeMO6lAAjAPEz0PaTzE9+Z4b5As1TUd8jzuMKR736OPcxg Q3O1HXJj7N7BwAYrQdZDUxrUiqR3g71shEgK2rECPmuUGcDOR9MtVB0Jbnv3C5cbZre0 NBFTpImRToIoHJXS+2W+x4v0COe6HsfrLrYrw/v/ci8lC9lkGVkwGuzB3NgPl1ZiZZTS g4vuYR/Vtd4DEFN/4mYRABONMJwOMTMPj7r5+845MHJNcmAHs8qXN2shRd4kAJKmkzWR /bJJAWNOlZMj2np9MxX7CuXPwHjl/rKnir/7/Nh+a2ZaDmvVsA8dacUYzv+yKphv64HL Q2Lg== X-Forwarded-Encrypted: i=1; AJvYcCWuwRRZSFjghNrgJsxvq83+Ab71/gLrYura9pGQnJ9HI2MW+j8X2XIVTpi9TYluUGTzq3E=@vger.kernel.org X-Gm-Message-State: AOJu0YzBHn1RvBPXcyWinv/sOwTCXbODxmx0BhWK2vpCaVY73DrNGg4L Mv1DfbcwVneR3stoj7alI1T24zbFC4m7tSLYozB14xsRgtG7hHjjaMvJXIUyFebS87SljM5ayc0 0gA== X-Google-Smtp-Source: AGHT+IENOAlD/a9C1yscB8pt52goTvVVex1rnSy0sEVpxDmpsSGOR0udgFhGxI9toBaldalY7yVRQJOWBrI= X-Received: from pjbqj6.prod.google.com ([2002:a17:90b:28c6:b0:2ef:6fb0:55fb]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:c887:b0:2ee:a744:a4fe with SMTP id 98e67ed59e1d1-2f83ac5cbf4mr18887576a91.25.1738376252521; Fri, 31 Jan 2025 18:17:32 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:05 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-4-seanjc@google.com> Subject: [PATCH 03/16] x86/tsc: Add helper to register CPU and TSC freq calibration routines From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Add a helper to register non-native, i.e. PV and CoCo, CPU and TSC frequency calibration routines. This will allow consolidating handling of common TSC properties that are forced by hypervisor (PV routines), and will also allow adding sanity checks to guard against overriding a TSC calibration routine with a routine that is less robust/trusted. Make the CPU calibration routine optional, as Xen (very sanely) doesn't assume the CPU runs as the same frequency as the TSC. Wrap the helper in an #ifdef to document that the kernel overrides the native routines when running as a VM, and to guard against unwanted usage. Add a TODO to call out that AMD_MEM_ENCRYPT is a mess and doesn't depend on HYPERVISOR_GUEST because it gates both guest and host code. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 4 ++-- arch/x86/include/asm/tsc.h | 4 ++++ arch/x86/kernel/cpu/acrn.c | 4 ++-- arch/x86/kernel/cpu/mshyperv.c | 3 +-- arch/x86/kernel/cpu/vmware.c | 4 ++-- arch/x86/kernel/jailhouse.c | 4 ++-- arch/x86/kernel/kvmclock.c | 4 ++-- arch/x86/kernel/tsc.c | 17 +++++++++++++++++ arch/x86/xen/time.c | 2 +- 9 files changed, 33 insertions(+), 13 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index 82492efc5d94..684cef70edc1 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3291,6 +3291,6 @@ void __init snp_secure_tsc_init(void) rdmsrl(MSR_AMD64_GUEST_TSC_FREQ, tsc_freq_mhz); snp_tsc_freq_khz = (unsigned long)(tsc_freq_mhz * 1000); - x86_platform.calibrate_cpu = securetsc_get_tsc_khz; - x86_platform.calibrate_tsc = securetsc_get_tsc_khz; + tsc_register_calibration_routines(securetsc_get_tsc_khz, + securetsc_get_tsc_khz); } diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index 540e2a31c87d..82a6cc27cafb 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -87,6 +87,10 @@ static inline int cpuid_get_cpu_freq(unsigned int *cpu_khz) extern void tsc_early_init(void); extern void tsc_init(void); +#if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) +extern void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), + unsigned long (*calibrate_cpu)(void)); +#endif extern void mark_tsc_unstable(char *reason); extern int unsynchronized_tsc(void); extern int check_tsc_unstable(void); diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index 2c5b51aad91a..c1506cb87d8c 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -29,8 +29,8 @@ static void __init acrn_init_platform(void) /* Install system interrupt handler for ACRN hypervisor callback */ sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); - x86_platform.calibrate_tsc = acrn_get_tsc_khz; - x86_platform.calibrate_cpu = acrn_get_tsc_khz; + tsc_register_calibration_routines(acrn_get_tsc_khz, + acrn_get_tsc_khz); } static bool acrn_x2apic_available(void) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index f285757618fc..aa60491bf738 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -478,8 +478,7 @@ static void __init ms_hyperv_init_platform(void) if (ms_hyperv.features & HV_ACCESS_FREQUENCY_MSRS && ms_hyperv.misc_features & HV_FEATURE_FREQUENCY_MSRS_AVAILABLE) { - x86_platform.calibrate_tsc = hv_get_tsc_khz; - x86_platform.calibrate_cpu = hv_get_tsc_khz; + tsc_register_calibration_routines(hv_get_tsc_khz, hv_get_tsc_khz); setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); } diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index 00189cdeb775..d6f079a75f05 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -416,8 +416,8 @@ static void __init vmware_platform_setup(void) } vmware_tsc_khz = tsc_khz; - x86_platform.calibrate_tsc = vmware_get_tsc_khz; - x86_platform.calibrate_cpu = vmware_get_tsc_khz; + tsc_register_calibration_routines(vmware_get_tsc_khz, + vmware_get_tsc_khz); #ifdef CONFIG_X86_LOCAL_APIC /* Skip lapic calibration since we know the bus frequency. */ diff --git a/arch/x86/kernel/jailhouse.c b/arch/x86/kernel/jailhouse.c index cd8ed1edbf9e..b0a053692161 100644 --- a/arch/x86/kernel/jailhouse.c +++ b/arch/x86/kernel/jailhouse.c @@ -209,8 +209,6 @@ static void __init jailhouse_init_platform(void) x86_init.mpparse.parse_smp_cfg = jailhouse_parse_smp_config; x86_init.pci.arch_init = jailhouse_pci_arch_init; - x86_platform.calibrate_cpu = jailhouse_get_tsc; - x86_platform.calibrate_tsc = jailhouse_get_tsc; x86_platform.get_wallclock = jailhouse_get_wallclock; x86_platform.legacy.rtc = 0; x86_platform.legacy.warm_reset = 0; @@ -220,6 +218,8 @@ static void __init jailhouse_init_platform(void) machine_ops.emergency_restart = jailhouse_no_restart; + tsc_register_calibration_routines(jailhouse_get_tsc, jailhouse_get_tsc); + while (pa_data) { mapping = early_memremap(pa_data, sizeof(header)); memcpy(&header, mapping, sizeof(header)); diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 5b2c15214a6b..b898b95a7d50 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -320,8 +320,8 @@ void __init kvmclock_init(void) flags = pvclock_read_flags(&hv_clock_boot[0].pvti); kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); - x86_platform.calibrate_tsc = kvm_get_tsc_khz; - x86_platform.calibrate_cpu = kvm_get_tsc_khz; + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz); + x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; #ifdef CONFIG_X86_LOCAL_APIC diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 4fc633ac5873..5a16271b7a5c 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1245,6 +1245,23 @@ static void __init check_system_tsc_reliable(void) tsc_disable_clocksource_watchdog(); } +/* + * TODO: Disentangle AMD_MEM_ENCRYPT and make SEV guest support depend on + * HYPERVISOR_GUEST. + */ +#if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) +void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), + unsigned long (*calibrate_cpu)(void)) +{ + if (WARN_ON_ONCE(!calibrate_tsc)) + return; + + x86_platform.calibrate_tsc = calibrate_tsc; + if (calibrate_cpu) + x86_platform.calibrate_cpu = calibrate_cpu; +} +#endif + /* * Make an educated guess if the TSC is trustworthy and synchronized * over all CPUs. diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 96521b1874ac..9e2e900dc0c7 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -566,7 +566,7 @@ static void __init xen_init_time_common(void) static_call_update(pv_steal_clock, xen_steal_clock); paravirt_set_sched_clock(xen_sched_clock); - x86_platform.calibrate_tsc = xen_tsc_khz; + tsc_register_calibration_routines(xen_tsc_khz, NULL); x86_platform.get_wallclock = xen_get_wallclock; } From patchwork Sat Feb 1 02:17:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956069 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA5EE13D504 for ; Sat, 1 Feb 2025 02:17:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376256; cv=none; b=tB0TzMEiPgneGa3X/mAe1vym9sBd6MVr/sBwTbGdfLF9bJj1S/+Dm1rovtkRaqjfVjxC2UbVmZVlhHbCFNI13FzPBURdRDqVxSA3HD6NumwW60/f+0rSsmPIGY7nO/MwDCu6PasEGLMvRNVR4pndNC44GTfq94nOlmLfcC0ORKo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376256; c=relaxed/simple; bh=IAZ81L9edk4QC0pX5hltlfX8CufP2WccrKBU/+/H8Yk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ubuNaWkLms4AIlJ7rTW9VG/w2NAl+fcInFdof9KhesmWIC726r5bobmkTNw2lr6dnRkHCqWISnt5rj6J7ik7R4Bvj6AU1olDfeSMuSj9yLofVPCpKqSI/hpUhbQK7iv+w4C8PNC/4T4GsQ+UMXSTAnFiMXR1MniBb8PeYJNo3rQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=x+SGPj+C; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="x+SGPj+C" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2164861e1feso51137615ad.1 for ; Fri, 31 Jan 2025 18:17:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376254; x=1738981054; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=MDyJubTgvsqHYl0zNK5/PK8+bagsI5N8h9xCVnMunLg=; b=x+SGPj+CSE/9RBE7h9++TQuAkVhapJKCZHFKt2FLSgdf6DwZ3fSG4QHxobS8Wu7hZY WwtA0Ohu71rZO+cNk1YknOaXUi3t8bhHBUkBIGTt4sOXCRrzKIzoLCbiA7+RdTXxX26i y4I4eW1ev6FGy4SW9UnX9/wPKKdPmhQqrMro8RofUZxNI4I6SWAL17qGvWJOWv5dzHMi RIhvkJQhqnOLx0vRuRtBMeap+CY+zFb4lNwGukW8ESuroD+Sn3qbmEPDRgO1YZjIHxvr xaQOoVnTMOLSHTNjNIrYpdUutK+kAHFRx6D4nJYkAsp4mYQuhsz24YAUERYYsmjiBlS6 k5NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376254; x=1738981054; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=MDyJubTgvsqHYl0zNK5/PK8+bagsI5N8h9xCVnMunLg=; b=D1HoUhQj4WWlW9VtPilezA34Zd+l+NSxWR2fvdDjshAgpGYZ5dOpZM0UH2cRrBEx5A JY/Z7kJheUZT30WwR84lCu/kl8+zqadOikFttuRrX0o3EM6XD0qkaGUXs0MKd6V/7LU0 ZbLJSlBfaUL0nc6zkYQq4xmWsXB1zFA24WuhN+e3SZLSi0GU99qsp5IK8zH+UBZk7isG hO67GvOs8Wq4W5Uoqda2K15LbmnuObKaxWnidq4zeYSFzl1zoiG5xsH+2hpPgznIIrio TeDaTnOpGO6sRdOT/ttdMJPiKyKIuoPXYQ2P05045R0bpnZUOa/kz3iLZiGt+lI41jnW QmOg== X-Forwarded-Encrypted: i=1; AJvYcCUYAK+qoukmD9R0UVcs27KOZDxkL0TSJ45EzqTvbn9p86e9xpVqxwoa6ohPyFoXfyHpmKg=@vger.kernel.org X-Gm-Message-State: AOJu0YzUYGWQDlFFtv9Ldo1ifiAqUROX2fGv9h9MyDamW34v/ppOxEn8 ywrv3IOd1b/FC7WhXjrt5Zn3wNFo6lAsVYrckYibqkWUpyFQLA3l2okEp6gH11AY1tNL6oP1x9Z Log== X-Google-Smtp-Source: AGHT+IGkjSAAWxmwHiu0qCv61xkb4iGGZ5alFNn1UCDa8mCUkr+gLXYmTCgtlrE3otTrUQeGakGB8oHEBGw= X-Received: from pjj16.prod.google.com ([2002:a17:90b:5550:b0:2e5:8726:a956]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:db04:b0:216:4c88:d93a with SMTP id d9443c01a7336-21dd7dd7356mr228949515ad.48.1738376254204; Fri, 31 Jan 2025 18:17:34 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:06 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-5-seanjc@google.com> Subject: [PATCH 04/16] x86/sev: Mark TSC as reliable when configuring Secure TSC From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Move the code to mark the TSC as reliable from sme_early_init() to snp_secure_tsc_init(). The only reader of TSC_RELIABLE is the aptly named check_system_tsc_reliable(), which runs in tsc_init(), i.e. after snp_secure_tsc_init(). This will allow consolidating the handling of TSC_KNOWN_FREQ and TSC_RELIABLE when overriding the TSC calibration routine. Cc: Nikunj A Dadhania Cc: Tom Lendacky Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 2 ++ arch/x86/mm/mem_encrypt_amd.c | 3 --- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index 684cef70edc1..e6ce4ca72465 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3288,6 +3288,8 @@ void __init snp_secure_tsc_init(void) return; setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + rdmsrl(MSR_AMD64_GUEST_TSC_FREQ, tsc_freq_mhz); snp_tsc_freq_khz = (unsigned long)(tsc_freq_mhz * 1000); diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c index b56c5c073003..774f9677458f 100644 --- a/arch/x86/mm/mem_encrypt_amd.c +++ b/arch/x86/mm/mem_encrypt_amd.c @@ -541,9 +541,6 @@ void __init sme_early_init(void) * kernel mapped. */ snp_update_svsm_ca(); - - if (sev_status & MSR_AMD64_SNP_SECURE_TSC) - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); } void __init mem_encrypt_free_decrypted_mem(void) From patchwork Sat Feb 1 02:17:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956070 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 94EC6148310 for ; Sat, 1 Feb 2025 02:17:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376258; cv=none; b=L8ZJXJJX+mWoIzmIjNTzEgWRjhQKBMVLnf9JFVLM5mfZVirwW4uGOkyFlHktMBDbqrqqQhzNp7wrIpkJf/l5oUptRwTJdwMhcP3Rf6/qIq8fecMqPA3gOeAYZPANc6MqRZcuX7mCDSWwdWHQJNF0DPYrK/vOeej3D39a06ueAUc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376258; c=relaxed/simple; bh=0aeg6fmcALkQCcWUWMu3ttM1q1g3Byn2iNg3RXSwUWU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=EhQb5JVfO9FVSBMznau3Nj0Ur4veUct0D6jm7RuVrV8e8krUOznIQApw2ZNIVtoloWC1JgNyrQlXTi3jfxSZ6QZWHK25pSDwiVv93Bwa0m+HYWclyyR3WaamwXU9RpfY39xJAume85WkouYee/yTQXcaMBxDgb4kHvG6tQ456p4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eTadbpQK; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eTadbpQK" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ef79403c5eso7332318a91.0 for ; Fri, 31 Jan 2025 18:17:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376256; x=1738981056; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Jjc0FDt1SDUJCzq8fzQuxH7Yk0FUL+T40LhhOPtkSdY=; b=eTadbpQK5e4nyjwAylxGFM6pa3E/SzIUhokZGI/v/ncORphfsvy27/SlNAbRH9UexZ AB9FDjV6Ki4upJpqrut+PSokrmU0BW47bVdDZMfK+YjPbgMv8m9CHR7WvdKplqlw6WkW FzElUSDmePGf0+IHoDAw5fBgpUn38BfS1isjdcm8/dkUo537lzyWtBdvyq2wVF5OVaDq oz0Vb3FAMRemfryAHiVY/gK4rMBfUtnhgcF9Yi96QTOmTZvR1SBjxphsBS7gUWgvhW5x eLVvg6v3N/S4wdegNjIhxanagDKo47OTcD0fp2OVn2YsRhHvw4iOgnjLapPSnsfrbT51 VEaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376256; x=1738981056; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Jjc0FDt1SDUJCzq8fzQuxH7Yk0FUL+T40LhhOPtkSdY=; b=cZJg+JeNwXbnbR3FfzKr93xF6Invg7drep6J1d0j3xW66K+BiDvJPVla36r2BmoecI 71PSJp0QJF+cLwvnw2GcPeGLAJ943qT4JBoLNrVP6zcan6KHhvUKzupR5eq3+DNL23Y7 l4edPG55uDvgW0liYTL/1KxWYmXczFNe1qHumMXEj0yLdwXaTQHU1m0F+8oYwD+efCoh idIh0hyXsKoJ/qwNG1Cw0Ta6QL4DxItNtjrPlyUOZ4PxLLa+Cy6NLgK3b1Ib2XwCto3B +Rqtf5tBUmE5wHNdVLQ7scTVpOW4qjJVmkwWa+6qm6Vm5PARaKsx8iDmLn5E13q8d6bz thKQ== X-Forwarded-Encrypted: i=1; AJvYcCVKNQQnuytgDona8DP4B1Q7NpupFuLUdcgTT3t/3iLKvUnLjOjyfbmD2icGsyy1NaIe9Zw=@vger.kernel.org X-Gm-Message-State: AOJu0Yz/10CfuaH4N/oJ1EECZxNz6+riK6F2ExHzNU2fhtOQD1CiJWl3 zUU6Ly+ALjuCPR7GqPkDj+oJTSry1cAXmgq9+uvChmvWs31lpFb4lYqhBNbya+79ogEXDBKUcKf rnQ== X-Google-Smtp-Source: AGHT+IHNY7VvQQRT0KSmjAH4FP26r3GzVfaLj1GKx7+F1yhKWmFMaX6hkP/CltwMml7T2BlNCnyUm1qLLO4= X-Received: from pjbsv5.prod.google.com ([2002:a17:90b:5385:b0:2f4:3ea1:9033]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1c03:b0:2ee:d18c:7d84 with SMTP id 98e67ed59e1d1-2f83ac17ebemr18755620a91.20.1738376255998; Fri, 31 Jan 2025 18:17:35 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:07 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-6-seanjc@google.com> Subject: [PATCH 05/16] x86/sev: Move check for SNP Secure TSC support to tsc_early_init() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Move the check on having a Secure TSC to the common tsc_early_init() so that it's obvious that having a Secure TSC is conditional, and to prepare for adding TDX to the mix (blindly initializing *both* SNP and TDX TSC logic looks especially weird). No functional change intended. Cc: Nikunj A Dadhania Cc: Tom Lendacky Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 3 --- arch/x86/kernel/tsc.c | 3 ++- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index e6ce4ca72465..dab386f782ce 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3284,9 +3284,6 @@ void __init snp_secure_tsc_init(void) { unsigned long long tsc_freq_mhz; - if (!cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)) - return; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 5a16271b7a5c..09ca0cbd4f31 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1514,7 +1514,8 @@ void __init tsc_early_init(void) if (is_early_uv_system()) return; - snp_secure_tsc_init(); + if (cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)) + snp_secure_tsc_init(); if (!determine_cpu_tsc_frequencies(true)) return; From patchwork Sat Feb 1 02:17:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956071 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D76514AD38 for ; Sat, 1 Feb 2025 02:17:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376260; cv=none; b=pHhzgDXn5PDL/g0HGSkW+y8E7o0iNlU5Bj+IN28s3bQMHUZoN9D2woDlNssRT0B/11l+Q3oi0rgjsHryOfkWHzucfiWs5ZWYvvVLCn2rsO8PKp1xPjEzzzr+OuZu+H/izb1DXEmChXL7ql22pmTHIfDNRzsw4Ll7kt9xOmWaXUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376260; c=relaxed/simple; bh=28sb199ir3z+fqMIFiQCff2Q1bkmQ/ZCLy7LVrgv3wA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HLPVFruwlYC4TkTcS9rHD6TQMGBlMombYrDO0hWtm51VAdDOJkN4dCw5qfsOF6emV7X3KCpv3jeraoBqCetyOuWJl6yKHixO1OBDHQgz61bYT0oMC0pmJv0N6Wl4p7ngYqA/PF1EII3L2aX7L54BeWQVIOvF/a7fJzs70YXuFZY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=c2W9PjL6; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="c2W9PjL6" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ef9864e006so7282866a91.2 for ; Fri, 31 Jan 2025 18:17:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376258; x=1738981058; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=OZIVWnX4ZIR8qTULzG2lFgPLNUNOiBx5bmZBm2lc3bA=; b=c2W9PjL6Mj/K7LoRXwArZm/ri4tzHjPHZq2+N8nrF59kXkOhZPSSKA1g13HGrsI3M5 OQ2rsiUViTmgfFS6383FVxIMd7a6LdAOw5JzhdaYkSdQT3n6QOx+NuRrOLxSGS6Fzmig sHmUM4r+/TiWk9VPAhxlDsp7WgYvEQ/WHzgtlsHpQx7eAq2vK+XDFQDFR6qy6jWybPb6 5M5pXtf9SJyP/13Z1pOsxhMq+H9m6M/J5aQMYFzLy/SN8uRbq5uElae1mt+xdooV/Zh/ 0l5i3zZijDp2HrjvE13ap5eEJmp1h4Aq26QR0a1ngcMP3Rc9KlfnnXREu8r6dVw0DsR0 ZTVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376258; x=1738981058; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=OZIVWnX4ZIR8qTULzG2lFgPLNUNOiBx5bmZBm2lc3bA=; b=HsKf9bdLGtHs8Te6HzRQP+N+jbI6AJvyOHQU3QbrZQDpJbDflVJX6glXV0Mjgw+7CJ iSsESxtjojnsaO4w3LsxLB+9HxD6NVABvCFh8fuEgq859NHeH30f9xN3huMbEdZZY5y6 yUki8OLWtlUdVwDyJPAdp0VY4S387ZbjXr4Zme+8fSmmn+K1EOcbhoKj5PPM/NDrNDtG qDD7drz3iWvJHzFUwr+9oudUWOdn/6c7YIIP+aQsikwaGdqml2sXoiggLwhCwopgI/D4 Ft1FFFuc97p44YpQfe7qMP0Zc+LLgg30vtI9z0/1NLUIx7xQ/ny/CxU+rvzKHJRgxh9W mKag== X-Forwarded-Encrypted: i=1; AJvYcCXefPYioIWpEiY/dKpAg00eZP22IWWkHUqxsynD1O4zx8cMhEFthQ4xLJmq3B/pLtWFTaw=@vger.kernel.org X-Gm-Message-State: AOJu0Yx5qNmaRMZ1i1LFL5rAPaTjuhj2P4rm1Ggjas9pt/KX7gvSTHwF 5tsvVzyfInRc9+JiprpyDSF+6jCq8EzS11VHhAHEbOq/GTLI0JO3NnOmrDyheRnm0QlEK5eub96 VFg== X-Google-Smtp-Source: AGHT+IFcMeXA8VIBFL831iVKjhY+AV/GUGMaW5llrGZfmojSdj8zqHZ4SMmI2Ir2VJD2NBPugN1JemRWHpg= X-Received: from pjtu6.prod.google.com ([2002:a17:90a:c886:b0:2ef:9b30:69d3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:3dc1:b0:2f9:9ddd:689c with SMTP id 98e67ed59e1d1-2f99ddd6bcfmr3721003a91.25.1738376257808; Fri, 31 Jan 2025 18:17:37 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:08 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-7-seanjc@google.com> Subject: [PATCH 06/16] x86/tdx: Override PV calibration routines with CPUID-based calibration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky When running as a TDX guest, explicitly override the TSC frequency calibration routine with CPUID-based calibration instead of potentially relying on a hypervisor-controlled PV routine. For TDX guests, CPUID.0x15 is always emulated by the TDX-Module, i.e. the information from CPUID is more trustworthy than the information provided by the hypervisor. To maintain backwards compatibility with TDX guest kernels that use native calibration, and because it's the least awful option, retain native_calibrate_tsc()'s stuffing of the local APIC bus period using the core crystal frequency. While it's entirely possible for the hypervisor to emulate the APIC timer at a different frequency than the core crystal frequency, the commonly accepted interpretation of Intel's SDM is that APIC timer runs at the core crystal frequency when that latter is enumerated via CPUID: The APIC timer frequency will be the processor’s bus clock or core crystal clock frequency (when TSC/core crystal clock ratio is enumerated in CPUID leaf 0x15). If the hypervisor is malicious and deliberately runs the APIC timer at the wrong frequency, nothing would stop the hypervisor from modifying the frequency at any time, i.e. attempting to manually calibrate the frequency out of paranoia would be futile. Deliberately leave the CPU frequency calibration routine as is, since the TDX-Module doesn't provide any guarantees with respect to CPUID.0x16. Opportunistically add a comment explaining that CoCo TSC initialization needs to come after hypervisor specific initialization. Cc: Kirill A. Shutemov Signed-off-by: Sean Christopherson --- arch/x86/coco/tdx/tdx.c | 30 +++++++++++++++++++++++++++--- arch/x86/include/asm/tdx.h | 2 ++ arch/x86/kernel/tsc.c | 8 ++++++++ 3 files changed, 37 insertions(+), 3 deletions(-) diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index 32809a06dab4..9d95dc713331 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -1063,9 +1064,6 @@ void __init tdx_early_init(void) setup_force_cpu_cap(X86_FEATURE_TDX_GUEST); - /* TSC is the only reliable clock in TDX guest */ - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - cc_vendor = CC_VENDOR_INTEL; /* Configure the TD */ @@ -1122,3 +1120,29 @@ void __init tdx_early_init(void) tdx_announce(); } + +static unsigned long tdx_get_tsc_khz(void) +{ + unsigned int __tsc_khz, crystal_khz; + + if (WARN_ON_ONCE(cpuid_get_tsc_freq(&__tsc_khz, &crystal_khz))) + return 0; + + lapic_timer_period = crystal_khz * 1000 / HZ; + + return __tsc_khz; +} + +void __init tdx_tsc_init(void) +{ + /* TSC is the only reliable clock in TDX guest */ + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + + /* + * Override the PV calibration routines (if set) with more trustworthy + * CPUID-based calibration. The TDX module emulates CPUID, whereas any + * PV information is provided by the hypervisor. + */ + tsc_register_calibration_routines(tdx_get_tsc_khz, NULL); +} diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index b4b16dafd55e..621fbdd101e2 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -53,6 +53,7 @@ struct ve_info { #ifdef CONFIG_INTEL_TDX_GUEST void __init tdx_early_init(void); +void __init tdx_tsc_init(void); void tdx_get_ve_info(struct ve_info *ve); @@ -72,6 +73,7 @@ void __init tdx_dump_td_ctls(u64 td_ctls); #else static inline void tdx_early_init(void) { }; +static inline void tdx_tsc_init(void) { } static inline void tdx_safe_halt(void) { }; static inline bool tdx_early_handle_ve(struct pt_regs *regs) { return false; } diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 09ca0cbd4f31..922003059101 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -32,6 +32,7 @@ #include #include #include +#include unsigned int __read_mostly cpu_khz; /* TSC clocks / usec, not used here */ EXPORT_SYMBOL(cpu_khz); @@ -1514,8 +1515,15 @@ void __init tsc_early_init(void) if (is_early_uv_system()) return; + /* + * Do CoCo specific "secure" TSC initialization *after* hypervisor + * platform initialization so that the secure variant can override the + * hypervisor's PV calibration routine with a more trusted method. + */ if (cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)) snp_secure_tsc_init(); + else if (boot_cpu_has(X86_FEATURE_TDX_GUEST)) + tdx_tsc_init(); if (!determine_cpu_tsc_frequencies(true)) return; From patchwork Sat Feb 1 02:17:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956072 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 433CA15383C for ; Sat, 1 Feb 2025 02:17:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376261; cv=none; b=q5jLDxTToi+CioDdOixDr8Yj8ZeciQ8Kgp+IZbhxhx7JcufNAITbfD9w2n9X9hYqLZk0av7LOXn2kOLZPX7F58Yd2VhviCeNnIS8AkNc1EC3HYKqL9rIy6HELUielTI2GDPpLqnpE8PgHjeGk4vOXbIsgBBD5pzkQdATvJbAEXQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376261; c=relaxed/simple; bh=E6Iy1BiM9FEEtqnIX6/l7kPa8oyhyUi3aCV4SUPddsQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sWFdgMHMRDkP3W67+Z3nNT+07aLVCNWuTcWTtQei0yRDAQZJYw7uOMQDhROaI7Zkz/IAAolRqMYEkz1k/pvdpHASXb5rHPFpJ2ODjBdfn14g6uLOdRRoqjZrZVzwVtJB54o0Y87nlfFLhA4//SwuQiJ262ezM3umeeaAKcKrhnc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Lc4rnG6D; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Lc4rnG6D" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ee9f66cb12so5228861a91.1 for ; Fri, 31 Jan 2025 18:17:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376259; x=1738981059; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=IW8kJcAcrKvt4c0vMOL7jWV1OXPEd6lfnHsLFZy4QXA=; b=Lc4rnG6Dm6msGUFDJy1doz8yyGyZL+0sW/L70hr/9AcdgDF+5ljhpV2WFFPRn59xG2 JMdnkczVc3HsnISv79/GB/CFVO+fAEtoxVlJpu+kZF/gVUeb07RsEQ8nNNZfBAYosltb ZjFcPv6ISf6yg71vzNJ8W8xp5kUwNuFdgsrZiZWdwexHES8U+Lriigo1NV6MKdS2hmn6 jdNhdPLfKyBJtILm3vcJMJgtyQybH50lEl5sRn78WFp376vxMXH4j3Kk41GdU1i9xYoV Dm3HaUG2Ig/4PLg+3w8X+jSA5loW/lZxaUezO2K+TY0aPfhsC+4yI73UpQjUpt3SCxKG L42A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376259; x=1738981059; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=IW8kJcAcrKvt4c0vMOL7jWV1OXPEd6lfnHsLFZy4QXA=; b=aKbgZN9AFuI7OeWXY0ZDdPjMj+JfLeQZ05VtNH04W++43e0rcW9nQs15hELinkAH8H CrvmPUvwVEKgUmnXTZ3KNgnXkv7zCoZUO2iTHLrXJnOiDKP+3IIz70AyZ17ZR456ktg3 nKE7uWLSWTTQW+79FRcN5a8rORXpFsbcxXVYgmz9I+/zFpAtNh4i7KeAFHF1Q9T/QrgB ZIHyBlsiJURdwh4mOcn5fACTBexPP6nLHIvqbzNassqwk48plBm/x3hloO/8fVHQ4arZ VW4+DHeOuDACoDoujVGoPymWzA+hNR1YWYQeKtUAzR0p5UYrGdhOD5EAEryNXxprZYJq ru8A== X-Forwarded-Encrypted: i=1; AJvYcCUJgRLEMPezge0lCNRBqRA/GXYHr2er2igSJXvRULVrRjuqcsg0u2wSlpyNo7K8FaEtKuM=@vger.kernel.org X-Gm-Message-State: AOJu0Yx4pmSxmKmTeaNP0FhoQBFL+CX4m7WR+9gSUOp/cgKXoxLpd6WX DeZxTQSlSapDrjp+Pp8dx58ZfGWd8YOp67bhyTBWpqZaG01h+6RHk/e0/KZcicHxcROHIqk31TB 1+Q== X-Google-Smtp-Source: AGHT+IF5rIJ4SRqkvzrTVMPpN2jn4obzhnrXzWmKwhKWojp8QTZ0kuVpAxPsfycQKPuD0sAxN2zkeFmWkds= X-Received: from pjbsb15.prod.google.com ([2002:a17:90b:50cf:b0:2ef:701e:21c1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3a08:b0:2ea:37b4:5373 with SMTP id 98e67ed59e1d1-2f83abe2135mr21011714a91.10.1738376259599; Fri, 31 Jan 2025 18:17:39 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:09 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-8-seanjc@google.com> Subject: [PATCH 07/16] x86/acrn: Mark TSC frequency as known when using ACRN for calibration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Mark the TSC frequency as known when using ACRN's PV CPUID information. Per commit 81a71f51b89e ("x86/acrn: Set up timekeeping") and common sense, the TSC freq is explicitly provided by the hypervisor. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/acrn.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index c1506cb87d8c..2da3de4d470e 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -29,6 +29,7 @@ static void __init acrn_init_platform(void) /* Install system interrupt handler for ACRN hypervisor callback */ sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); tsc_register_calibration_routines(acrn_get_tsc_khz, acrn_get_tsc_khz); } From patchwork Sat Feb 1 02:17:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956073 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41FD916F858 for ; Sat, 1 Feb 2025 02:17:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376264; cv=none; b=YBNRkfWm6PdEiOUiLYO+BCwHI/2dRwuwwVARbT+zon5ThyPKgKnlan796+ddh6w9X1nJeAV7dxaDzewfqkj3yfRDy3h4ZeMUEjtlkxdXCRm0TL6vjhsb3z7KeAqcYyDlG4CiXToZr+Jw+/ug/iWWnpJXh/6RXEZiFVwOzZr3Nig= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376264; c=relaxed/simple; bh=ASwUqN95BYwqinjLF99Tq16sgCdUDLFn1N7MLc7S61U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mDSoTtb1AKJ52uM5kDnmPFL0u26NygCFbBT/cU6KZJ7u7ag/4Fy09ROX7Xq5wdK7P+FKSXc8LA+3rx/WBvfl8XRcjoWtgvgk+iDKA0y74KYxvxZy1faxrDOZ6uKrQCxRnwwm6WbHdrZ3h69zxeJ1oNr7XK5Q3iTmecDoW7JXrvc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3shTZLaq; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3shTZLaq" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-216728b170cso54750045ad.2 for ; Fri, 31 Jan 2025 18:17:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376261; x=1738981061; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=fjq2RAKOugqtigb0Q/zSm7wR8ZLcl6wCuY/1ydRYBE0=; b=3shTZLaqphM/mtbPc7pIH2NrzZwiLXL3ON8pYb9vIwnXE7Ra4p0GfyN0isHnwO9wlV wm/NX7dJ8eS2YR6sJt8x2PrrwTrz5C+gXlgvb+BT2Jr2By7XMg7GUfggX4BAvdaQqA3C a/iFdhKQKJi6+p+dP1bCFYX1ZTnthkPJ4COqxq5fFBABsYyzqxw6R09o0GD5s77xg1KO nw5DCVrxTDH2NLAq/MoqrKHBZfQzaKg0Y9ExX4NClZCZwWvrHWLkIGfVTEB2C2vnwnUl YbFQrSbAmNFVsG8kqQ4NeKSfuYne8UnU02I+VWFLf76c/WSRBBbAoi3tnXvIYqBRcyY5 JpOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376261; x=1738981061; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=fjq2RAKOugqtigb0Q/zSm7wR8ZLcl6wCuY/1ydRYBE0=; b=VhQ5fOWBiinlb8sCCaBGw4R9cv5r74bMvfl3LEdUwSzC2ajiJm8YHykZa8N0tnf9e5 l1IPPjILK2LNXQMDba7F/iBRGXK40SyjmUSRez8LWUMyPKRBxEn1rHEK3222PyH6ZYxx q3nAKLRRhqZkhewC118OVLpzIh2L2D14vuxLShQzhc5TOWvpW6yNpKd5HWM06xg6dEOU CslzafTwOaRM6h/7uK0rTxDUQpG9Wy+tEHeDRco4dLt5B3b3hMYZuNuYvfWqoQg57P38 9JAY38wL2xolwXRfvZbmMcP6Ruy4dlrrySR1F7cPgeVFE4ld3prVyw/fljTIESSe2pfN yN5g== X-Forwarded-Encrypted: i=1; AJvYcCVAPCRKPo8AmsMx26K4aa7ZJKMuOGcM8WPGBrtCW8y4JtoWIJo7H6rO4d6aGF8GgtIGe18=@vger.kernel.org X-Gm-Message-State: AOJu0YxlguWVUfYRWg7dXgzdclxxnZJnOQah1hvx/hm4pBfXi2/45sjt 1caybdsdG9U475TsxBDysf/qDvMKOnwcURGXpQ05NDgj3uqivnJDYwPTg5+qdxtGeh+8UzEFXwr ObA== X-Google-Smtp-Source: AGHT+IEzM58dX9KoHCeq7XuQ3ZLuiQUuTP7JBFBKc9/3u2bcW6PA+cqHRPb1Z08cdz7ppSJkxA+MEfhtD78= X-Received: from pjbpx11.prod.google.com ([2002:a17:90b:270b:b0:2e9:5043:f55b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ce8b:b0:215:e98c:c5c1 with SMTP id d9443c01a7336-21dd7d78f78mr169836355ad.30.1738376261457; Fri, 31 Jan 2025 18:17:41 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:10 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-9-seanjc@google.com> Subject: [PATCH 08/16] x86/tsc: Pass KNOWN_FREQ and RELIABLE as params to registration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Add a "tsc_properties" set of flags and use it to annotate whether the TSC operates at a known and/or reliable frequency when registering a paravirtual TSC calibration routine. Currently, each PV flow manually sets the associated feature flags, but often in haphazard fashion that makes it difficult for unfamiliar readers to see the properties of the TSC when running under a particular hypervisor. The other, bigger issue with manually setting the feature flags is that it decouples the flags from the calibration routine. E.g. in theory, PV code could mark the TSC as having a known frequency, but then have its PV calibration discarded in favor of a method that doesn't use that known frequency. Passing the TSC properties along with the calibration routine will allow adding sanity checks to guard against replacing a "better" calibration routine with a "worse" routine. As a bonus, the flags also give developers working on new PV code a heads up that they should at least mark the TSC as having a known frequency. Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 6 ++---- arch/x86/coco/tdx/tdx.c | 7 ++----- arch/x86/include/asm/tsc.h | 8 +++++++- arch/x86/kernel/cpu/acrn.c | 4 ++-- arch/x86/kernel/cpu/mshyperv.c | 10 +++++++--- arch/x86/kernel/cpu/vmware.c | 7 ++++--- arch/x86/kernel/jailhouse.c | 4 ++-- arch/x86/kernel/kvmclock.c | 4 ++-- arch/x86/kernel/tsc.c | 8 +++++++- arch/x86/xen/time.c | 4 ++-- 10 files changed, 37 insertions(+), 25 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index dab386f782ce..29dd50552715 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3284,12 +3284,10 @@ void __init snp_secure_tsc_init(void) { unsigned long long tsc_freq_mhz; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - rdmsrl(MSR_AMD64_GUEST_TSC_FREQ, tsc_freq_mhz); snp_tsc_freq_khz = (unsigned long)(tsc_freq_mhz * 1000); tsc_register_calibration_routines(securetsc_get_tsc_khz, - securetsc_get_tsc_khz); + securetsc_get_tsc_khz, + TSC_FREQ_KNOWN_AND_RELIABLE); } diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index 9d95dc713331..b1e3cca091b3 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -1135,14 +1135,11 @@ static unsigned long tdx_get_tsc_khz(void) void __init tdx_tsc_init(void) { - /* TSC is the only reliable clock in TDX guest */ - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); - /* * Override the PV calibration routines (if set) with more trustworthy * CPUID-based calibration. The TDX module emulates CPUID, whereas any * PV information is provided by the hypervisor. */ - tsc_register_calibration_routines(tdx_get_tsc_khz, NULL); + tsc_register_calibration_routines(tdx_get_tsc_khz, NULL, + TSC_FREQ_KNOWN_AND_RELIABLE); } diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index 82a6cc27cafb..e99966f10594 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -88,8 +88,14 @@ static inline int cpuid_get_cpu_freq(unsigned int *cpu_khz) extern void tsc_early_init(void); extern void tsc_init(void); #if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) +enum tsc_properties { + TSC_FREQUENCY_KNOWN = BIT(0), + TSC_RELIABLE = BIT(1), + TSC_FREQ_KNOWN_AND_RELIABLE = TSC_FREQUENCY_KNOWN | TSC_RELIABLE, +}; extern void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), - unsigned long (*calibrate_cpu)(void)); + unsigned long (*calibrate_cpu)(void), + enum tsc_properties properties); #endif extern void mark_tsc_unstable(char *reason); extern int unsynchronized_tsc(void); diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index 2da3de4d470e..4f2f4f7ec334 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -29,9 +29,9 @@ static void __init acrn_init_platform(void) /* Install system interrupt handler for ACRN hypervisor callback */ sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); tsc_register_calibration_routines(acrn_get_tsc_khz, - acrn_get_tsc_khz); + acrn_get_tsc_khz, + TSC_FREQUENCY_KNOWN); } static bool acrn_x2apic_available(void) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index aa60491bf738..607a3c51eddf 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -478,8 +478,13 @@ static void __init ms_hyperv_init_platform(void) if (ms_hyperv.features & HV_ACCESS_FREQUENCY_MSRS && ms_hyperv.misc_features & HV_FEATURE_FREQUENCY_MSRS_AVAILABLE) { - tsc_register_calibration_routines(hv_get_tsc_khz, hv_get_tsc_khz); - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + enum tsc_properties tsc_properties = TSC_FREQUENCY_KNOWN; + + if (ms_hyperv.features & HV_ACCESS_TSC_INVARIANT) + tsc_properties = TSC_FREQ_KNOWN_AND_RELIABLE; + + tsc_register_calibration_routines(hv_get_tsc_khz, hv_get_tsc_khz, + tsc_properties); } if (ms_hyperv.priv_high & HV_ISOLATION) { @@ -582,7 +587,6 @@ static void __init ms_hyperv_init_platform(void) * is called. */ wrmsrl(HV_X64_MSR_TSC_INVARIANT_CONTROL, HV_EXPOSE_INVARIANT_TSC); - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); } /* diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index d6f079a75f05..6e4a2053857c 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -385,10 +385,10 @@ static void __init vmware_paravirt_ops_setup(void) */ static void __init vmware_set_capabilities(void) { + /* TSC is non-stop and reliable even if the frequency isn't known. */ setup_force_cpu_cap(X86_FEATURE_CONSTANT_TSC); setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - if (vmware_tsc_khz) - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + if (vmware_hypercall_mode == CPUID_VMWARE_FEATURES_ECX_VMCALL) setup_force_cpu_cap(X86_FEATURE_VMCALL); else if (vmware_hypercall_mode == CPUID_VMWARE_FEATURES_ECX_VMMCALL) @@ -417,7 +417,8 @@ static void __init vmware_platform_setup(void) vmware_tsc_khz = tsc_khz; tsc_register_calibration_routines(vmware_get_tsc_khz, - vmware_get_tsc_khz); + vmware_get_tsc_khz, + TSC_FREQ_KNOWN_AND_RELIABLE); #ifdef CONFIG_X86_LOCAL_APIC /* Skip lapic calibration since we know the bus frequency. */ diff --git a/arch/x86/kernel/jailhouse.c b/arch/x86/kernel/jailhouse.c index b0a053692161..d73a4d0fb118 100644 --- a/arch/x86/kernel/jailhouse.c +++ b/arch/x86/kernel/jailhouse.c @@ -218,7 +218,8 @@ static void __init jailhouse_init_platform(void) machine_ops.emergency_restart = jailhouse_no_restart; - tsc_register_calibration_routines(jailhouse_get_tsc, jailhouse_get_tsc); + tsc_register_calibration_routines(jailhouse_get_tsc, jailhouse_get_tsc, + TSC_FREQUENCY_KNOWN); while (pa_data) { mapping = early_memremap(pa_data, sizeof(header)); @@ -256,7 +257,6 @@ static void __init jailhouse_init_platform(void) pr_debug("Jailhouse: PM-Timer IO Port: %#x\n", pmtmr_ioport); precalibrated_tsc_khz = setup_data.v1.tsc_khz; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); pci_probe = 0; diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index b898b95a7d50..b41ac7f27b9f 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -116,7 +116,6 @@ static inline void kvm_sched_clock_init(bool stable) */ static unsigned long kvm_get_tsc_khz(void) { - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); return pvclock_tsc_khz(this_cpu_pvti()); } @@ -320,7 +319,8 @@ void __init kvmclock_init(void) flags = pvclock_read_flags(&hv_clock_boot[0].pvti); kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz); + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, + TSC_FREQUENCY_KNOWN); x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 922003059101..47776f450720 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1252,11 +1252,17 @@ static void __init check_system_tsc_reliable(void) */ #if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), - unsigned long (*calibrate_cpu)(void)) + unsigned long (*calibrate_cpu)(void), + enum tsc_properties properties) { if (WARN_ON_ONCE(!calibrate_tsc)) return; + if (properties & TSC_FREQUENCY_KNOWN) + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + if (properties & TSC_RELIABLE) + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + x86_platform.calibrate_tsc = calibrate_tsc; if (calibrate_cpu) x86_platform.calibrate_cpu = calibrate_cpu; diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 9e2e900dc0c7..e7429f3cffc6 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -40,7 +40,6 @@ static unsigned long xen_tsc_khz(void) struct pvclock_vcpu_time_info *info = &HYPERVISOR_shared_info->vcpu_info[0].time; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); return pvclock_tsc_khz(info); } @@ -566,7 +565,8 @@ static void __init xen_init_time_common(void) static_call_update(pv_steal_clock, xen_steal_clock); paravirt_set_sched_clock(xen_sched_clock); - tsc_register_calibration_routines(xen_tsc_khz, NULL); + tsc_register_calibration_routines(xen_tsc_khz, NULL, + TSC_FREQUENCY_KNOWN); x86_platform.get_wallclock = xen_get_wallclock; } From patchwork Sat Feb 1 02:17:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956074 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9438C189F20 for ; Sat, 1 Feb 2025 02:17:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376265; cv=none; b=KHRsi3/q/mi6Ds8dduveyVCwmO6Bpug3PHcadnxLCxDIrTGZlvpz539/MIb4ahjY40OvEsnEDpkGtPOZLgUuJT7FAQoMqRGz6H3CQc3jpsBrydlU/wVwe2M+oKWy1Tu9NiFezAi9xYd7tIBuMFlxGgOc/xSMSLXJhRlysf88Osg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376265; c=relaxed/simple; bh=SMb+ptqn3Y2lAemdJfa8mzgfJB25QAWQ7cmu+kYKQzc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fj7x9atIOyCmGgM/ghe9Edn2KbtZW8qREh93SXjv9cfnWv1R9HlZ4/f7LpJyFsP8a1nEySlFvCCzFNnxBuibuiglndLUvHekke20f9dciNu/W/VXFEOJm+p4Nb47BUzHMQg06NSuIHlO3bXqF8dVEWwZ+hchZZSOgvGNQdIYBtM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fjs4n93V; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fjs4n93V" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ef9864e006so7283050a91.2 for ; Fri, 31 Jan 2025 18:17:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376263; x=1738981063; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=T4DEhCWG0SuSvz0a+mPR7euxpU0fSN+tQ/CkCS9MqCY=; b=fjs4n93VSxxTFtsT1CdCYXluuJhttoGgBXKCZxmAtD44Oii4XE++21RCHxMhwV5DVe /IUDAyyan5YgKx9IdDMnyPS70Zdwr1PYcaXE0T8os/NPnwHAGOEExT4VF0Rv4sON2BhD oXUggauFBGR85HieWhZ7xikuKMnZM3I71vUcSFNGE/jY0EMNGWb/5OH0E9rnTx3eogyd Yw9Y9R7Op1WuZVqqrPjIDBuGt2ZzT8l7XUtE6y8X1ChJcl0kRhrkJoOEeDreFeFKm5rl 1FH0VtCmvvUPVjFlvjKSaQ/j7tvhmQdjdCLKVtYjHcfM82wtfp7NPqJJHcTCssoYIg+G 4fcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376263; x=1738981063; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=T4DEhCWG0SuSvz0a+mPR7euxpU0fSN+tQ/CkCS9MqCY=; b=wlkB8fOgl+QjPmH27e3oq0X1LeahIQ6qX0P9fCNZ0E7FIYgh3otknWOgy0dM5T2XR2 ifRXLVw1P5bdQwSOzOFJbm6HIkfYu/8yuilnQzM3rmVMfu4lUWcCyBioCShvz2ERF/18 Cr31OaE2yXBrY+UpB5ddWabjCQS+4sEzQVXn4jCL/lQra4MkP7P8SusjbtsNW9F3Op9A eYg8ayWMnU96pzYvJ9lKrvBF2XapJgG7i3+iR/4fbQHfZsmSeanGdqGiJ33WMCSXLsv9 5Q3Ft4mUx3kDr5V4+wH0clbQvtyhS7hv9smqx8ihaq6o3blJtbha1mK4R6NSMF9PmW62 XzLw== X-Forwarded-Encrypted: i=1; AJvYcCWTT/svA8gYc37yzZ4DnFYrb0M113P9Wbk8g6YOLNPN3N7em3Gad7aIkAZn6a1Mw4/KT/Q=@vger.kernel.org X-Gm-Message-State: AOJu0Yx/pOld6N/TOeP4A/aaGSy1eJSKngZAp6n+yZaUVneHC3rntWLJ kla/KS3hUJlsB6SOVwVdNSR0OevZr7VxRjuDyB0t10oG73YGYDmDh/fSB0OxcTttwNOVYh9VoMD VfQ== X-Google-Smtp-Source: AGHT+IHof1w34LjQ30XVM2gwsfD2n69eJy7Rgvm5USApfq7itvhXgigdwO4D30qo87G/8bS8xHYNlWrIUTM= X-Received: from pjd6.prod.google.com ([2002:a17:90b:54c6:b0:2e2:9f67:1ca3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:51c1:b0:2ea:7cd5:4ad6 with SMTP id 98e67ed59e1d1-2f83ac86a44mr17727640a91.32.1738376263046; Fri, 31 Jan 2025 18:17:43 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:11 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-10-seanjc@google.com> Subject: [PATCH 09/16] x86/tsc: Rejects attempts to override TSC calibration with lesser routine From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky When registering a TSC frequency calibration routine, sanity check that the incoming routine is as robust as the outgoing routine, and reject the incoming routine if the sanity check fails. Because native calibration routines only mark the TSC frequency as known and reliable when they actually run, the effective progression of capabilities is: None (native) => Known and maybe Reliable (PV) => Known and Reliable (CoCo). Violating that progression for a PV override is relatively benign, but messing up the progression when CoCo is involved is more problematic, as it likely means a trusted source of information (hardware/firmware) is being discarded in favor of a less trusted source (hypervisor). Signed-off-by: Sean Christopherson --- arch/x86/kernel/tsc.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 47776f450720..d7096323c2c4 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1260,8 +1260,13 @@ void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), if (properties & TSC_FREQUENCY_KNOWN) setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + else if (WARN_ON(boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ))) + return; + if (properties & TSC_RELIABLE) setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + else if (WARN_ON(boot_cpu_has(X86_FEATURE_TSC_RELIABLE))) + return; x86_platform.calibrate_tsc = calibrate_tsc; if (calibrate_cpu) From patchwork Sat Feb 1 02:17:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956075 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B35861A4F0A for ; Sat, 1 Feb 2025 02:17:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376267; cv=none; b=LIocQH/6Zbv9dcd07cH4tzLcolfCooEzYhS2CmMajwYyPMPdnDICG7fAeXAo01FeYKSZvdP6AQK3nFRWr7BirFhoxfcWKYP7J/Bi2iDjyoS0u6T2wKDnxkLlaKHjJfE5gdUwkwpJxVuYHXKiLHOBNSSiOHpiNT1vys4KxYPfKF8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376267; c=relaxed/simple; bh=kx1WuhibcuTx6WiHPmvUxDfP1htj6y2KJdDmu/FjmBY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GuuuoKEjEmKyiuq5NBSAKmSsS5+aMuHEvKgoTbZGaA91vfhlvAYFT51mUo3QqrZqPyi92VYxgKkvOYkbBy05pclDCQB9TbUkGR2wJRL2n+JowA9U6J/CdDpsE5OwBT/0Jb+jRnSGQ5kIo4y4DpF3SB7hrvflEakbTsoGH/3392I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=wVK9N+FI; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wVK9N+FI" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2163a2a1ec2so84584305ad.1 for ; Fri, 31 Jan 2025 18:17:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376265; x=1738981065; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=W1XV2ZKxBzpt6fRRKseHNiWV51nuIILy1d39LBlT9SE=; b=wVK9N+FI4a2ceQOs7i0AkqT31sCQJmoaHqMMyptLTslSjOcDBE8Ilnx3viQKaXVXDH q2cIO9ilRm10t+7wHx6WV0reM/uqdKvGwoeB164bSV+wzhiH/IdkdJWDx6PhfsBV29Q9 YR5MTjav3JjL7QB+qv0Sb4LJ6n0lRbr7ozI6QRfmuW6deEEmgVu2cxyNwwxdBNhiV0Bt bipYle1qf4jcKQjyHx2CDEH4uvOFD+LIp979llQaI6PgZ+bO+Zdfws2Vyi+C+P+jM9NS KrGnDAGI2gdzkz8Ws6TMx9SdyPsZOGGV+EZdw2+XMdnI3eukjkR/XMLmcmm10GNiIxsJ 6XLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376265; x=1738981065; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=W1XV2ZKxBzpt6fRRKseHNiWV51nuIILy1d39LBlT9SE=; b=RmxXDsfVxfMgVJ+2jTCCGQdnCd0kXtifywaXmRHh2vfuuXW67gmKhFc/O/23klXQ2B A3LMhGz7HVCNM1z44ruiQQ197wZboqXxAl0Rba7RAjXk1tz3vD2W0Ig2nDFJa0ATolgB VLrM9bUkHIm2b3Xz+FAbwmYq0BJZ2w3H1XfYSi3czQk0cwpfQ9+EvIPa1DUmsJm91L+l 5JoGy9FY4TiCojNkO6SKPS+iG03XFDwBjWQRL/SpGrQ+qmS9JALowc/bTHh+p6icJVLr OC+TiZPCz8RNgqcm0MeMrIGZjGOfVtlPboMKegPXKGRh6CKGCo4cER/WjGQL/G4uzJQs 4O/w== X-Forwarded-Encrypted: i=1; AJvYcCUw2aafx+XSuymQIRDkyC7gclIuR7duj3hAQ3RcbhenYAQ7lR2Ws3aM25Y8taj/99Acjwc=@vger.kernel.org X-Gm-Message-State: AOJu0YwzOQPtaZDZ0xzX2hC7fwlzS8l4EsMM951lz3qG1ux/Ca7wnjcH AXjjAEWztazKjn8VqL+fmL2/tiFeo4MtuEUPUd8LGIVrTcTWWPVVwQnDJic1A9R4zYzA3ohxBKG kLA== X-Google-Smtp-Source: AGHT+IGSSeREbj02hPb3cDa39T9c0oKEk7xpncELnx14V1Vrs2HnSpCp6+OTk/Z1j4CkHarX3h8UUIGOLcI= X-Received: from pjbsb14.prod.google.com ([2002:a17:90b:50ce:b0:2f4:465d:5c61]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:244c:b0:215:f1c2:fcc4 with SMTP id d9443c01a7336-21dd7deeff4mr227157665ad.41.1738376264849; Fri, 31 Jan 2025 18:17:44 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:12 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-11-seanjc@google.com> Subject: [PATCH 10/16] x86/paravirt: Move handling of unstable PV clocks into paravirt_set_sched_clock() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Move the handling of unstable PV clocks, of which kvmclock is the only example, into paravirt_set_sched_clock(). This will allow modifying paravirt_set_sched_clock() to keep using the TSC for sched_clock in certain scenarios without unintentionally marking the TSC-based clock as unstable. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 7 ++++++- arch/x86/kernel/kvmclock.c | 5 +---- arch/x86/kernel/paravirt.c | 6 +++++- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index 041aff51eb50..cfceabd5f7e1 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -28,7 +28,12 @@ u64 dummy_sched_clock(void); DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); -void paravirt_set_sched_clock(u64 (*func)(void)); +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable); + +static inline void paravirt_set_sched_clock(u64 (*func)(void)) +{ + __paravirt_set_sched_clock(func, true); +} static __always_inline u64 paravirt_sched_clock(void) { diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index b41ac7f27b9f..890535ddc059 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -12,7 +12,6 @@ #include #include #include -#include #include #include #include @@ -93,10 +92,8 @@ static noinstr u64 kvm_sched_clock_read(void) static inline void kvm_sched_clock_init(bool stable) { - if (!stable) - clear_sched_clock_stable(); kvm_sched_clock_offset = kvm_clock_read(); - paravirt_set_sched_clock(kvm_sched_clock_read); + __paravirt_set_sched_clock(kvm_sched_clock_read, stable); pr_info("kvm-clock: using sched offset of %llu cycles", kvm_sched_clock_offset); diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 1ccaa3397a67..55c819673a9d 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -85,8 +86,11 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); -void paravirt_set_sched_clock(u64 (*func)(void)) +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable) { + if (!stable) + clear_sched_clock_stable(); + static_call_update(pv_sched_clock, func); } From patchwork Sat Feb 1 02:17:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956076 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E00C41ADC6B for ; Sat, 1 Feb 2025 02:17:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376268; cv=none; b=amYkH+LsihTDkTV3nYnuvYUdiNtN4Mm8jRdlc9hr5dErw3K4rybw6egsuv8Bi0/IeU/L8EJgBEz+/Q5NbfoM4gixK+E97xM8iv++UHbNx7xpodRIQSi87bekt7erLHogAxzc84ngAXXcAizZpuNwxTed6t7/8+0+x2ZdYGWs46M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376268; c=relaxed/simple; bh=I7ICP95Ld9V+B6MPp3tXCsSnHLq4xQG7ZATAfWkmSSA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DwFiHGj/3SozFIiXzUvyRF5oV4Vj3JBUYZayX3poI2XXawJ0Qqtu7e3cs6cA9absqECQghK4uFr+1Jk2Uuw3xG96ICF3FY/AOdIOa3fBIAyE6Whkhvdhb+JGPLvHP2HFa7/tlGEyOmno5HXAtvMnW4S0J7L1cfBsntTw0nVpzys= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=I+zdv+FC; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="I+zdv+FC" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2161d5b3eb5so50868885ad.3 for ; Fri, 31 Jan 2025 18:17:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376266; x=1738981066; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=AK/mWgvwJvlM2VErSRU3T5I3ox4v68bEfhNUVA1UAEc=; b=I+zdv+FCyeDWqU0kLUOvk/nmJt/mmLOaIPkGd9fKk3eqlgFdOen7wrBVa+Jl7uqGv8 W0WS1cGB+qD/QjfZYzHw+yYVfPPereyHMsADL+cI5XM5jCbyI4g+DVRZFYDfPz9mtosg DNxkdLmGqlduIuV9X4RBLGfjuBYCq0ENTuOZ0AoyfPM09VwLnl1TDeaE8d3jLrrZAfSN HJHFD1vm4eCbocB9LmehkDWaUa+BZIZsEVdJJ09AvtkGgdmB6iXWtBRv4pSq+MhRW+V5 P1iA3ZThC+0U0drg5FOYMsmn+g001KkAlvMm1MgdWNitNNc/cyMnQvYZZIgVRUOhnhic S68A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376266; x=1738981066; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AK/mWgvwJvlM2VErSRU3T5I3ox4v68bEfhNUVA1UAEc=; b=TsVUHM3mhVn1miH/BG/MAC/G6g7oKVT8eP53MF5N3CWLmVOYNHiYG9IY+DUcfpxOIs yb/1SFcUWGyiQwGDDWLctSw/rjPLzw1lCs5kJgD70+vK/zFxBmKaWCDFp7Ko5HpSHGdA MVD7mBwYDAkAd59ZWIFRhdrIea85nvYzVTOGhiM1s/pgGY8VeUsFdnGK2PMd38hz2dfr dhmMGOYE43uX7VI/4qtsA7GTgeV0/NzTwNWtu960ilMBzZaqgtEv4j/B8B2KQtPvxnoi ICctzG88DX/Xkz9Z4YOLiuSbHBThCze42Lgzgay2rv8q7XbiuchB8ObuWScPTDdEiY4t hK4g== X-Forwarded-Encrypted: i=1; AJvYcCXuM05wlV/pNDzW6fz9eJlAvzjAjPOMFkeWAUCgxA7NFHfVQkOAUy06f9XFs1aWI7nJ2tU=@vger.kernel.org X-Gm-Message-State: AOJu0Yw9F9lPW8tGVOYz39MWQYMzCl+Bu7SpRbfMjH9M5OADD3QkZy3T jFA7yqJh2g35W0o+TQujbRXg0mSBqQPLLUMQGoPZJZg3jNw8S/TywjJXKCqZxkCeH7HbgX5nvME vWw== X-Google-Smtp-Source: AGHT+IGIBUdgj+cCgBZpvoXPxY6FOVmtNbk7sDeQo6dqnIl0GeJKfhw5O39y3pDqzJDExNb4JeazGQgwjQ8= X-Received: from pjbdy6.prod.google.com ([2002:a17:90b:6c6:b0:2ef:7483:e770]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:11c3:b0:215:b1a3:4701 with SMTP id d9443c01a7336-21dd7d64c1emr194518565ad.13.1738376266294; Fri, 31 Jan 2025 18:17:46 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:13 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-12-seanjc@google.com> Subject: [PATCH 11/16] x86/paravirt: Don't use a PV sched_clock in CoCo guests with trusted TSC From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Silently ignore attempts to switch to a paravirt sched_clock when running as a CoCo guest with trusted TSC. In hand-wavy theory, a misbehaving hypervisor could attack the guest by manipulating the PV clock to affect guest scheduling in some weird and/or predictable way. More importantly, reading TSC on such platforms is faster than any PV clock, and sched_clock is all about speed. Signed-off-by: Sean Christopherson --- arch/x86/kernel/paravirt.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 55c819673a9d..980440d34997 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -88,6 +88,15 @@ DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); void __paravirt_set_sched_clock(u64 (*func)(void), bool stable) { + /* + * Don't replace TSC with a PV clock when running as a CoCo guest and + * the TSC is secure/trusted; PV clocks are emulated by the hypervisor, + * which isn't in the guest's TCB. + */ + if (cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC) || + boot_cpu_has(X86_FEATURE_TDX_GUEST)) + return; + if (!stable) clear_sched_clock_stable(); From patchwork Sat Feb 1 02:17:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956077 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC2AF1B392E for ; Sat, 1 Feb 2025 02:17:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376270; cv=none; b=Lm1673WmE4vmOeaYdqdbG1aJzScD/+Hil5F7F7zGYHTsAbd2g743FVuSbeE9fOCMFW46/w0cELa4gPvtJfXfHhuCGpjob1/5zfuopMCwjWMj7FleTwe8/nQ9Z0h7xivZFngfz0gmqfmBfrnX3zGlvdTrFOuwpz0yygLQNmZCY8o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376270; c=relaxed/simple; bh=xkVUUxGUDTn4R+vL3GICv7gTh6ZLEkeNiGFIyJAu9Bg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GAwGTTqXHjFqWM4TkhQCUN+qSFn6lwjalTV/NDFT8nqyCYGMbKnnxK7aQKo16p5sZE7I2ZCAKi31kbDJbj2gr6NpKQL2kCyAxOF8n8sUPpgmEYcR+zpECwYcUJftyr5WLp11Mu04AGdu7ND/ZROeGhARfmp3E1MqHSGye4ahL/o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=r+aBF04b; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="r+aBF04b" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2efc3292021so7281472a91.1 for ; Fri, 31 Jan 2025 18:17:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376268; x=1738981068; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=IrGW5KyAKjJvdPU9ageDhgEv89aW3VoCTAEai48xeak=; b=r+aBF04bMXteYNEUdpZ/KADsthpRfht39q2wEsPQeBT+khZhBofTSYdnvQvsxEOJ1i NchDdLKDgLbHiwN6DRHxmlg9KPbE4YzBHFed4CzWRAErW4DkLFP71gOQnF3f8SiwiM+S +wDyW0gVkO7DFExXLUWwEhGRZnGK5ondcEQ7uOYnFpqfn9rnxi/3yIuEIHRlOa5zFWER hA6EFvctVRaUvbUz+X0PfZmnyrjEXXGmqR68UssIQ9zhrtQ5qGefLxl6kIDWHuZeD1ZO 4Jznl/ksONtVfVYvLV4RcpLRIzEE9J0Ubl45xbYFHyLyQUPE/Kf/qq19sTEotMffu89H mFig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376268; x=1738981068; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=IrGW5KyAKjJvdPU9ageDhgEv89aW3VoCTAEai48xeak=; b=ioo3/1HdgCG1bUC7FotYk+h6OXhg2x2D7cGpdpOkBPBoTA/DKv2TTD5y9ja3bpcm5s 11pH+S1zWvrD08jYamMNYDb89JyKp+cwErNV08rqUZfjRFRnboaZBynNVJfg+P77pa3V iBQADSUX/nnOeB1U7meubbS8iikFiPhAe5oSc3Kb6gYg5tB8digUpsBGS0ZvQ3i4aYtp 2muLfnn70LmlQyB98lZVFc4kocsulQeaEXrrYhW+qwkdBmCcuviIXCjB2N13S4jA9yAN L/c2IUP5wVv79fKfRHgtfBvzSbz4hrNRLzU7m/NqUMKb1WJD4krhiCL32uUf+GYQmoWn 46dg== X-Forwarded-Encrypted: i=1; AJvYcCXa3yk/dyadsXPOGQ4E5WP2zVnCce379dOyIQSy+XBYoISmxMCLN+RA8Rm6uI89KMalQb4=@vger.kernel.org X-Gm-Message-State: AOJu0YzkQo6K+xEn38TbA9KyTtWCmShYFOnhc1WD0LlFS8R/z1Kj9gJl vma5H6ALx7IoU7+v5gMeAYlcPMHvZSGJ5fli4WWOh8NDkd/7q5VxTHauBIldHIOvWdLVMre9pRO ZnA== X-Google-Smtp-Source: AGHT+IFnviFY5P8yLFi8Q8khLJ9/h4GGB0jx1XaxjiOcD/F0E8KD3icYKrUfW55PtjtMBfI0KqMQJtaVac0= X-Received: from pgbci10.prod.google.com ([2002:a05:6a02:200a:b0:a97:3102:ea5d]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:9102:b0:1e1:faa:d8cf with SMTP id adf61e73a8af0-1ed7a6e1efcmr25085282637.40.1738376268141; Fri, 31 Jan 2025 18:17:48 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:14 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-13-seanjc@google.com> Subject: [PATCH 12/16] x86/kvmclock: Mark TSC as reliable when it's constant and nonstop From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Mark the TSC as reliable if the hypervisor (KVM) has enumerated the TSC as constant and nonstop, and the admin hasn't explicitly marked the TSC as unstable. Like most (all?) virtualization setups, any secondary clocksource that's used as a watchdog is guaranteed to be less reliable than a constant, nonstop TSC, as all clocksources the kernel uses as a watchdog are all but guaranteed to be emulated when running as a KVM guest. I.e. any observed discrepancies between the TSC and watchdog will be due to jitter in the watchdog. This is especially true for KVM, as the watchdog clocksource is usually emulated in host userspace, i.e. reading the clock incurs a roundtrip cost of thousands of cycles. Marking the TSC reliable addresses a flaw where the TSC will occasionally be marked unstable if the host is under moderate/heavy load. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 890535ddc059..a7c4ae7f92e2 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -283,6 +283,7 @@ static int kvmclock_setup_percpu(unsigned int cpu) void __init kvmclock_init(void) { + enum tsc_properties tsc_properties = TSC_FREQUENCY_KNOWN; u8 flags; if (!kvm_para_available() || !kvmclock) @@ -313,11 +314,26 @@ void __init kvmclock_init(void) if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT)) pvclock_set_flags(PVCLOCK_TSC_STABLE_BIT); + /* + * X86_FEATURE_NONSTOP_TSC is TSC runs at constant rate + * with P/T states and does not stop in deep C-states. + * + * Invariant TSC exposed by host means kvmclock is not necessary: + * can use TSC as clocksource. + * + */ + if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && + boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && + !check_tsc_unstable()) { + kvm_clock.rating = 299; + tsc_properties = TSC_FREQ_KNOWN_AND_RELIABLE; + } + flags = pvclock_read_flags(&hv_clock_boot[0].pvti); kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, - TSC_FREQUENCY_KNOWN); + tsc_properties); x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; @@ -328,19 +344,6 @@ void __init kvmclock_init(void) x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; kvm_get_preset_lpj(); - /* - * X86_FEATURE_NONSTOP_TSC is TSC runs at constant rate - * with P/T states and does not stop in deep C-states. - * - * Invariant TSC exposed by host means kvmclock is not necessary: - * can use TSC as clocksource. - * - */ - if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && - boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && - !check_tsc_unstable()) - kvm_clock.rating = 299; - clocksource_register_hz(&kvm_clock, NSEC_PER_SEC); pv_info.name = "KVM"; } From patchwork Sat Feb 1 02:17:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956078 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 919251BBBD4 for ; Sat, 1 Feb 2025 02:17:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376272; cv=none; b=EUQkfTmuAFgeqBVYFKBd96uc3WPtULQDhATjbpCj00C5jpwm6oxSY1xJGn17g46Sf/jmFd/4+pT1Vna6AF8wRIL0xj0R9BA+OA+Cb790Pf1Vgo6eE6MMakMAArDGMn4D29Z8sepIJ4QbXTcQmS8RbG2ckIDPwP4vw4nfyZ2GRTY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376272; c=relaxed/simple; bh=azNw66jnMSVEiPAB+2l0m6Z/DQ0p90Ig+X/Tx6OROQQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Lz/9lyRSKH7LA6PWDNoLdmf7thyR1Jx+JWz6b5I+aSJPD0B1qHZSmBdsnoT8XGIkNPnGAGDxGACZSlXJ6nHjptcHjlztoTkmEWC+cNfkDSgBm17mCW7cZ6MoKuDCGNLg9e3v6I0t7IedFRPT2E516e3x0qeEdsHg9uoOUVH74jw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0pxQ6kb0; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0pxQ6kb0" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ef9dbeb848so4914356a91.0 for ; Fri, 31 Jan 2025 18:17:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376270; x=1738981070; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9ZKnHj9nGtIPNyoOdVCsq53ZVe6onbdoIhwKKzEiQlk=; b=0pxQ6kb0eBNyesUtr0hnPt4rA1lRx/bFzkRAAlmSZUgmotcW5wDSMPEDk9OI1DyEu1 KMzsVbatfHzwpsVT4/Ru+q4viu8nVOfwzl+4AwDHSkkfmIshJpeVJ5DWQsSbniIODazk rmtaXGa3b9ak3K74udhMCZpBVCC/AUqQBfwcVGEh0jGia41pmuaWX66YFgt6r0grxZZN sI/5HC8QBABRHNUryNiff7Cf5V9xraNL0fqzBpzHjHs/mI6bp2SOkHFcQvXIUomQInMz KdL2kpBSU/xC0TZkGRydfZtVUafLBS79cKg+4Vo0p55BDXqSYDzC66FBOZDMIm8MALBX OOfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376270; x=1738981070; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9ZKnHj9nGtIPNyoOdVCsq53ZVe6onbdoIhwKKzEiQlk=; b=K7eaFzFqahKF0LprsIKxPfIAn8V39ZZ9s22k4IAQe28NvxZR4TUXuaucs+oXe7a+yC Jztg56s59ZxDVe5HfQc3jd1iU5PO+kA/Ms+15idozaPXPeQ8s2FqIlX+B4pVWKw3DDXx cjSlyjfC1OwnQgFb4Qnkv2dmIutVcUjnK4AbUD5sg9evt5eL86E6Yr9JnVcqeTTUOXZF GC3kxx2i/McOcp/kfIOt49kaG0c/H9Pp9uLqL0RuQ7s8qR4Cd8mfVKUb91zm6ef8864n XOUTJchHhw+9DAJQLR864U/8ukeiDBkTvMgxlWVEjOF64+nn+IxTvHD6S5iFkxKn/O2w gQuQ== X-Forwarded-Encrypted: i=1; AJvYcCWysAEr7n0DHAKHtuDTSF4/7TZXYkuoCyZEFLVCMGdRG7AtoKcaXpJ7HKVmpL7he0W84S4=@vger.kernel.org X-Gm-Message-State: AOJu0Ywc7GTJ6Mq2UtN+dW6KWYDs/8aCYWRG/NZPG0/xLe3rnjTbFKtz mTLn1KK2tPreLd+fpTFJZuhEkoa5NsXK3znmo++LwoB5A/aZomEtx9HSRJXh6xOozH5AuKnbiGf 72Q== X-Google-Smtp-Source: AGHT+IFgXEe7/w9yeWgyg6qQLihx/5RXWr7kMK6/niF7ETqn4L9GsYCh+B6w5nh2NPJ849LxusX87Gxqnac= X-Received: from pfiy14.prod.google.com ([2002:a05:6a00:190e:b0:724:eefc:69ef]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:9a6:b0:725:e1de:c0bf with SMTP id d2e1a72fcca58-72fd0bf47cdmr18371303b3a.9.1738376270050; Fri, 31 Jan 2025 18:17:50 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:15 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-14-seanjc@google.com> Subject: [PATCH 13/16] x86/kvmclock: Get CPU base frequency from CPUID when it's available From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky If CPUID.0x16 is present and valid, use the CPU frequency provided by CPUID instead of assuming that the virtual CPU runs at the same frequency as TSC and/or kvmclock. Back before constant TSCs were a thing, treating the TSC and CPU frequencies as one and the same was somewhat reasonable, but now it's nonsensical, especially if the hypervisor explicitly enumerates the CPU frequency. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index a7c4ae7f92e2..66e53b15dd1d 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -102,6 +102,20 @@ static inline void kvm_sched_clock_init(bool stable) sizeof(((struct pvclock_vcpu_time_info *)NULL)->system_time)); } +static unsigned long kvm_get_cpu_khz(void) +{ + unsigned int cpu_khz; + + /* + * Prefer CPUID over kvmclock when possible, as the base CPU frequency + * isn't necessary the same as the kvmlock "TSC" frequency. + */ + if (!cpuid_get_cpu_freq(&cpu_khz)) + return cpu_khz; + + return pvclock_tsc_khz(this_cpu_pvti()); +} + /* * If we don't do that, there is the possibility that the guest * will calibrate under heavy load - thus, getting a lower lpj - @@ -332,7 +346,7 @@ void __init kvmclock_init(void) flags = pvclock_read_flags(&hv_clock_boot[0].pvti); kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_cpu_khz, tsc_properties); x86_platform.get_wallclock = kvm_get_wallclock; From patchwork Sat Feb 1 02:17:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956079 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84DAE1C0DED for ; Sat, 1 Feb 2025 02:17:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376274; cv=none; b=IHW+dIRLDGLOfcbsy0sL45RLf4t+cZ5aLlBiaxSVKRCtUjrd47DB+dv2IUbCyaIqkdgKAuPoKmkZEmK/KtcmPrLpSlGTkZl0HioV+SrJnOVp/SLUXjCQl8EVU925/fKRA/MydaxmYVpnwzGUEV0bO4AKkcazgI+P/6R7PvMzXFs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376274; c=relaxed/simple; bh=OViTJZ+XBu7MgZ7inPoS/83nOvRpwHQF3rmoy13X/Q4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GtdY4OKrNaHpV+567BIi3PWLg7qwauLFxn5HE7MC8LdF7Pwjq39Vpp6hVZFz1U3zqNIWFkgAwvimePrQfeZ57kPZEncD8w2RVpFdBMylY9Pta2mCInrRVNRV+oXEaVyTbDxXEgNlx3xYvThl9vzqIjx1xJPhZHkBrhO490yHTmU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xLZck8sR; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xLZck8sR" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2efa74481fdso5200765a91.1 for ; Fri, 31 Jan 2025 18:17:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376272; x=1738981072; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=H8aoO3CAJ0dqDpElLUQaiYDlNJUSBbklsVcflcMYifM=; b=xLZck8sRCsh6ZzrmWOmjuk8Hv78zyuo0OOxOs2Hfusk5JnezTviP4BJBhZanyHrIzT nce3DJP5stu0uOgPy+pqp9EA/4AKr92j0GWQBnhmX3C1l+KwwgDNqTD9LaPyTpeYARpN A9c6pqT28q9cV2tBuqa9s6y4ytSkWGfwsPYCiqZyEk/+O5TdwZuPqzBioJqXHIv5CGAv 8bURPC99UeSpwcj61g51IVOp5GNBGWizao0RoVSeB03/qy5WwNZDT9TXMKZGDlr2qAvV 67QJ+8pa6A1RXgCJO3beZ9wnS10FqNApSBj+G5WP0ismTgeDYKXNbgmvFloaX2D74Jju p47w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376272; x=1738981072; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H8aoO3CAJ0dqDpElLUQaiYDlNJUSBbklsVcflcMYifM=; b=RLhPTyDQ29KmKj7OBQTnoJofwWyRbBxCcZgUoDkStU0IeMOguwgU3RdaxPozSanNLc eyrrv5aNSPqraANuQNnFtC7UZFhx5gLLZt4KRbbhRQpS05+u7GV48vDQttjjYkizpy8x /dqzp1YezolLPlsZHG1nbusZmP0wmbj2iVTDIDewHHmeVtshWnge/XZgD8j9y3D1/okf wdgRtfsZ1TylkjILMAzvr/veqJta2lTMENbvpzd8yp5dx8WUKHzwqjBDUkJ0V4G3haoS B4RpKaSbOb0W2U6ZD/CIFVUBjEMqFgsY3S3zDa3GIjy1tOkpq0CDhEbVXryO/+QixWqV od3A== X-Forwarded-Encrypted: i=1; AJvYcCV8lo9oMwiQW6gfjk1bqM8ZuS5Dy3ODcGONnEZ50HPtMq0X2iPFJLcK+Mqz4Mb+llMQAwk=@vger.kernel.org X-Gm-Message-State: AOJu0Yx7AfEZyodsk9SVJAUzsM7bEi3kuCEJqEg6svYdCaYjJXVKvccn pj/Moq4N/z9ZXtRxQZdnD1KcPKAXzvFBSqczFnIeGfPXGdAqbHULMT60r3TzHtMGPuKgJi9d3aY xEg== X-Google-Smtp-Source: AGHT+IFlM5iy+/YhDLhDvbxJ2kgMrd1qTg/6Z4sP06sibC0sgnf3CqMto9NWjotO4O8YEuu2B+OhUgsrUhU= X-Received: from pjbsw14.prod.google.com ([2002:a17:90b:2c8e:b0:2f7:d453:e587]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3d45:b0:2ee:8358:385 with SMTP id 98e67ed59e1d1-2f83abb34bfmr19091952a91.4.1738376271854; Fri, 31 Jan 2025 18:17:51 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:16 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-15-seanjc@google.com> Subject: [PATCH 14/16] x86/kvmclock: Get TSC frequency from CPUID when its available From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky When kvmclock and CPUID.0x15 are both present, use the TSC frequency from CPUID.0x15 instead of kvmclock's frequency. Barring a misconfigured setup, both sources should provide the same frequency, CPUID.0x15 is arguably a better source when using the TSC over kvmclock, and most importantly, using CPUID.0x15 will allow stuffing the local APIC timer frequency based on the core crystal frequency, i.e. will allow skipping APIC timer calibration. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 66e53b15dd1d..0ec867807b84 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -102,6 +102,16 @@ static inline void kvm_sched_clock_init(bool stable) sizeof(((struct pvclock_vcpu_time_info *)NULL)->system_time)); } +static unsigned long kvm_get_tsc_khz(void) +{ + unsigned int __tsc_khz, crystal_khz; + + if (!cpuid_get_tsc_freq(&__tsc_khz, &crystal_khz)) + return __tsc_khz; + + return pvclock_tsc_khz(this_cpu_pvti()); +} + static unsigned long kvm_get_cpu_khz(void) { unsigned int cpu_khz; @@ -125,11 +135,6 @@ static unsigned long kvm_get_cpu_khz(void) * poll of guests can be running and trouble each other. So we preset * lpj here */ -static unsigned long kvm_get_tsc_khz(void) -{ - return pvclock_tsc_khz(this_cpu_pvti()); -} - static void __init kvm_get_preset_lpj(void) { unsigned long khz; From patchwork Sat Feb 1 02:17:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956080 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24E441C3BFC for ; Sat, 1 Feb 2025 02:17:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376275; cv=none; b=NBbhM3yP/NyYALI4nJgU0bYe2EuQTpFaOrYryOlPAQ+osSXS5KEmp4uxEFi5h/zChA15U4q4GVpLKdhgEdop5rnK2e7G6BDieXfAltfExjvS0tJz5bQoCAf2VV2pgxjFlL/XaZ+vVBud7R+4+AC7IRoQeP+t7+GReydroJjOBjE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376275; c=relaxed/simple; bh=MHFjgcDixq+MEkE5s8mB+CTWIGx9I1EGKDIGAXqaNs4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=EIbc//LFHeUEvcKk4xaLli8mpCHRUhWT2J4dsgPqm63EcN6ppqw+nPZq3hfFE9eB6KYDQJIO0zYb0/mX2N1DLfdLaUwmjLWviQN1v2miNZCZ/y0SEzf7yXUQ6vU0zWCMtGYehFtt7lHF8MCGAklZjORaaMra0OzX2E9DqxZ1hOU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RJUUCxcX; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RJUUCxcX" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2efc3292021so7281617a91.1 for ; Fri, 31 Jan 2025 18:17:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376274; x=1738981074; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=MLts4lXwUIqNNu2uIA1UT+VV0O42NNK5wDATllLGuXo=; b=RJUUCxcXVPVK376KPUS32yvH34C3xwr2s+nsiLKJPwXWXKIGpFlRIcLKcd1isn41Xc sTjXeli6Oi3KaEAP6xxY/jBToyEPNgBJ8bCtYu6hevil03oL+zGTHR+q0naPIU4teNjQ QY5wBjD3riUWSotk+i4P7+ZwIRyl42zS7ILs2Ejn11OphsB+oDZp92BlHcMt8p9XB29y fUu5Ba9tcafiv4dfcc022FOrDyJJyrt2hI0HRtqK3KRVbSof6yhdctw8KWlvVBxwwy5M 1PcQOFgOZFysgFzUpiP8+ihOE5XmTgOLo1VOAnPzwiVrsLGwM4Qgi1o7IoD6Qx+RioMw q3bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376274; x=1738981074; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=MLts4lXwUIqNNu2uIA1UT+VV0O42NNK5wDATllLGuXo=; b=XMquweG0MN45A0G7p1EdMtXtMh+UpNhSqYMC1Rqqj6GsrBHyP8gbn6O6YOHomAW6xi +P93A57AgTAeJdn6v9sSAFJSikDjg31iODHZs3oTUA6pKuankhGU6Ng1+CQAobSjF7t2 jy25ILDAZHUzE1uHK8ASjRb/cYdpO6XX0REoFK8Xnzwg5m1FftywWkvrd1CS2gyMGgbS dpTnBFbrKxpBjd5jQWASL2IWqucUnHZcg6NRdcTFKgUHDYg0RpObVlLxs3UIu2l1WbaN VDZLbeuzmezVRSO5r+SWWiAK/3QGaOwUpHgVOPzEyvGjaKjbOAycmtVLBZQpi8pxvfwJ YTpA== X-Forwarded-Encrypted: i=1; AJvYcCUXjkLztuUG06u0hLrXRjACQ8tOg7aqqHmeeJ5xczddsSfMV/TzZF+75E2TOfM156iffmE=@vger.kernel.org X-Gm-Message-State: AOJu0YyPKChA2kIvCWv9YXDTzFbOtandvKEM9dr6+eDr3In+qwxFkpzC rhe5eqWJYVqEdYoiwAmNw371U8kL7AaBWshQQn70QtXAow9S0EBP33ThQH02fto2FNEmlyOrEro CWg== X-Google-Smtp-Source: AGHT+IFZkSSDE4sQNSYE1P+oxZtfHJOA6RLwDBSraCmTI0kfGCTz9b39MqhkWv/tuyQdm2uggkOJw5pWH7U= X-Received: from pjbfr16.prod.google.com ([2002:a17:90a:e2d0:b0:2ea:5c73:542c]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:53c4:b0:2ee:d63f:d77 with SMTP id 98e67ed59e1d1-2f83abd7ccfmr20755900a91.9.1738376273601; Fri, 31 Jan 2025 18:17:53 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:17 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-16-seanjc@google.com> Subject: [PATCH 15/16] x86/kvmclock: Stuff local APIC bus period when core crystal freq comes from CPUID From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky When running as a KVM guest with kvmclock support enabled, stuff the APIC timer period/frequency with the core crystal frequency from CPUID.0x15 (if CPUID.0x15 is provided). KVM's ABI adheres to Intel's SDM, which states that the APIC timer runs at the core crystal frequency when said frequency is enumerated via CPUID.0x15. The APIC timer frequency will be the processor’s bus clock or core crystal clock frequency (when TSC/core crystal clock ratio is enumerated in CPUID leaf 0x15). Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 0ec867807b84..9d05d070fe25 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -106,8 +106,18 @@ static unsigned long kvm_get_tsc_khz(void) { unsigned int __tsc_khz, crystal_khz; - if (!cpuid_get_tsc_freq(&__tsc_khz, &crystal_khz)) + /* + * Prefer CPUID over kvmclock when possible, as CPUID also includes the + * core crystal frequency, i.e. the APIC timer frequency. When the core + * crystal frequency is enumerated in CPUID.0x15, KVM's ABI is that the + * (virtual) APIC BUS runs at the same frequency. + */ + if (!cpuid_get_tsc_freq(&__tsc_khz, &crystal_khz)) { +#ifdef CONFIG_X86_LOCAL_APIC + lapic_timer_period = crystal_khz * 1000 / HZ; +#endif return __tsc_khz; + } return pvclock_tsc_khz(this_cpu_pvti()); } From patchwork Sat Feb 1 02:17:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13956081 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27EEF1C54A2 for ; Sat, 1 Feb 2025 02:17:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376277; cv=none; b=Ve4y2XccBZOl6aOUa3kn2pnAnIc0pmTgUFTBvp8eWC+17B7DcHTtd4/WOKwipNUdNgWF+YIFX73wS9IXN8V1mz/U0NK4ODH1gEpkB28XnPX/QnNNjKrlhpTI0ksjRu9x9GucP0PHX7x/Bq6YAsUBNoGIM118BTlls0vtVntLsn0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738376277; c=relaxed/simple; bh=p/Vg9iE4sgVLr3/9VKFQQjtH8mkL0jM6rjwkOfa6jHY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QTwU3nly7PZA2UxNr7P/hZpfLGyuIg1/pxDhIOCSjA4I0lbOcs0ANAxQMO31wB+wHqz4wdeSCVSNV+svvAUi4cXL+4Dh9VBYJlIIO7JRDktlDqPw74ysAYilOI82dNholGnS0t4PfDpnf9wa1UbK3TNmooOpWCKvv7wOvYLBcvw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TTdJFo6H; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TTdJFo6H" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ef79d9c692so7142949a91.0 for ; Fri, 31 Jan 2025 18:17:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738376275; x=1738981075; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SJJEXGT/CGEg26HmUNCtCBi/lOkkZBhLk4mR9XODbn0=; b=TTdJFo6HyULXojAv3H0Yn6VjSgoycBc39w0m4rwblQ1P0OYlAecEzDKp8Qu4VJwfqY 3PeVvp7Snr0vKO3Zy1BKPUnShOW8GroZ284B0DbnjYCDBeA/GWfIrzbt02HbLK5IjARG ofQgfWll6AFlBScGymjB5TKvVgrgOpVYSi6L6pWVlbmNPS0JkhSmTA8AJolYxp/tgUDg GSzUJ7YitOEqxAwuSr72r1TrMky1PThqO95dfvPhbV14hcwpUualVKomF9IsY6zv9ezl 51srobwLT18I6geG2e4VcrDD5Q6/3X58lk3ygUZJ6xHb9uLB1GaH/4yOfGNwk7g66ums zIOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738376275; x=1738981075; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SJJEXGT/CGEg26HmUNCtCBi/lOkkZBhLk4mR9XODbn0=; b=iAkrnx9HdKWvMOcCHYdjNsqBWXEz6z8YUnGdnvrr7dmr+FOCMSEocrYLkq6spyxlYb esBMT53Y8+w8ESnflEUT3Xq3Bt67z/dWTh2PbV2RxV1T12/2POPcQBWBOB0Xzod8bACG nGv9q4Eu4avMxFD8XDfTnlaoFkvdVR7JKIzK8Z57ku+9S7RxI+gA7M6n41TK8lGBNRh2 AlJJtyP97ETfcxRrJiKCfW1znAhh+nMGbu3QXQqD4DQa+jIdAAnVZm8Z+IXdOH39zmGU LfqVDnDIKeGsoyfSZhjt7wX5HZ2KJnX0XVXGt6jndzMCqHtxwohNjDvg+HvTQ8zSf3fH 8g2g== X-Forwarded-Encrypted: i=1; AJvYcCVyMI3i3+jmLN/WhgrwCrAp1QHNWduH2UTNOFcdCm1xCtYF43JFa49FUs9XzoxuxI2jNpo=@vger.kernel.org X-Gm-Message-State: AOJu0YzaNmi+M4okWbVg17HOXTHmR+I9Iuelt2edswNGDNlDt+DZSos8 Vfrn6jLQ69O3ViZ5GcgYyh3AgIBffii5AmAeWUTB/w8uscn8EOvnho/4vrzfsKHwPoad8h4Vmbh qZQ== X-Google-Smtp-Source: AGHT+IGR6HwuD9w+8Kze6jNqduo1hcKEvlqzGV2yi3bR0PK9E6ixYJtwZg6BVPpJ2g9E6rQRruKKBVyC7NI= X-Received: from pjtu8.prod.google.com ([2002:a17:90a:c888:b0:2f7:f660:cfe7]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2702:b0:2f4:4003:f3d4 with SMTP id 98e67ed59e1d1-2f83ac83632mr19746549a91.30.1738376275470; Fri, 31 Jan 2025 18:17:55 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 31 Jan 2025 18:17:18 -0800 In-Reply-To: <20250201021718.699411-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250201021718.699411-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201021718.699411-17-seanjc@google.com> Subject: [PATCH 16/16] x86/kvmclock: Use TSC for sched_clock if it's constant and non-stop From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Paolo Bonzini , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, jailhouse-dev@googlegroups.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Sean Christopherson , Nikunj A Dadhania , Tom Lendacky Prefer the TSC over kvmclock for sched_clock if the TSC is constant, nonstop, and not marked unstable via command line. I.e. use the same criteria as tweaking the clocksource rating so that TSC is preferred over kvmclock. Per the below comment from native_sched_clock(), sched_clock is more tolerant of slop than clocksource; using TSC for clocksource but not sched_clock makes little to no sense, especially now that KVM CoCo guests with a trusted TSC use TSC, not kvmclock. /* * Fall back to jiffies if there's no TSC available: * ( But note that we still use it if the TSC is marked * unstable. We do this because unlike Time Of Day, * the scheduler clock tolerates small errors and it's * very important for it to be as fast as the platform * can achieve it. ) */ The only advantage of using kvmclock is that doing so allows for early and common detection of PVCLOCK_GUEST_STOPPED, but that code has been broken for nearly two years with nary a complaint, i.e. it can't be _that_ valuable. And as above, certain types of KVM guests are losing the functionality regardless, i.e. acknowledging PVCLOCK_GUEST_STOPPED needs to be decoupled from sched_clock() no matter what. Link: https://lore.kernel.org/all/Z4hDK27OV7wK572A@google.com Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 9d05d070fe25..fb8cd8313d18 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -344,23 +344,23 @@ void __init kvmclock_init(void) pvclock_set_flags(PVCLOCK_TSC_STABLE_BIT); /* - * X86_FEATURE_NONSTOP_TSC is TSC runs at constant rate - * with P/T states and does not stop in deep C-states. - * - * Invariant TSC exposed by host means kvmclock is not necessary: - * can use TSC as clocksource. - * + * If the TSC counts at a constant frequency across P/T states, counts + * in deep C-states, and the TSC hasn't been marked unstable, prefer + * the TSC over kvmclock for sched_clock and drop kvmclock's rating so + * that TSC is chosen as the clocksource. Note, the TSC unstable check + * exists purely to honor the TSC being marked unstable via command + * line, any runtime detection of an unstable will happen after this. */ if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && !check_tsc_unstable()) { kvm_clock.rating = 299; tsc_properties = TSC_FREQ_KNOWN_AND_RELIABLE; + } else { + flags = pvclock_read_flags(&hv_clock_boot[0].pvti); + kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); } - flags = pvclock_read_flags(&hv_clock_boot[0].pvti); - kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_cpu_khz, tsc_properties); @@ -369,6 +369,11 @@ void __init kvmclock_init(void) #ifdef CONFIG_X86_LOCAL_APIC x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; #endif + /* + * Save/restore "sched" clock state even if kvmclock isn't being used + * for sched_clock, as kvmclock is still used for wallclock and relies + * on these hooks to re-enable kvmclock after suspend+resume. + */ x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; kvm_get_preset_lpj();