From patchwork Thu Feb 27 02:18:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0DCEAC021B8 for ; Thu, 27 Feb 2025 02:19:32 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897365.1306019 (Exim 4.92) (envelope-from ) id 1tnTUS-0005qh-EM; Thu, 27 Feb 2025 02:19:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897365.1306019; Thu, 27 Feb 2025 02:19:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUS-0005qa-BD; Thu, 27 Feb 2025 02:19:08 +0000 Received: by outflank-mailman (input) for mailman id 897365; Thu, 27 Feb 2025 02:19:07 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUR-0005qU-4S for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:07 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 379a80a3-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:06 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc1e7efdffso1555127a91.0 for ; Wed, 26 Feb 2025 18:19:05 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 379a80a3-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622744; x=1741227544; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KtBuHiug8RyBl7zdSfnpeCyQMsoAneIAJhrbW8v3FfQ=; b=sDnw7K4bSJVwxcxE1AGL4Cq8/I8xw+JtpPE872WF3UwovWowaXDkJbFwLYX1Ph9h/d PKlcONKNynffqUen+AFSLhVY+pG/m/0rfmHEZx2U2oPKkbmFx8Bz0XHdpPvcShLsQtS4 vOQ8JFRrzGKeO1evwjfk8dcBdvOt5xcbuieCLfyR1hx+MLDcS5+aep0N1A3mJy+5DEnb hOijyzNgrd68T/Y7g91fVfJecrbOgQK2lCas9CuaOoSUjhlw6LvK00dvpQV4KSO8V1Ft yqH2JIfig8nBw9xNOTdsxu+CPa5DBfYM4q8zHCbotJZBBHlDLh5DOTUTDstdGTqJi6Rv rmgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622744; x=1741227544; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KtBuHiug8RyBl7zdSfnpeCyQMsoAneIAJhrbW8v3FfQ=; b=F1CbQoZqGEoSSniSzPUV5pXvCq0oSJ22jJF3EG+ppgbNrNStJaNZi6V8Eq2XLk7EP/ jIMwvVX7PFG3kVaUo+pFOmcI3wSEdFMqSGfjIAOWIJEQ2Mwgp722hXQBB1kc+DiGZUpy 0bT/hSwxpkIe02uCnZ/xG6Yqd8oUMCnqZxJn8a/hlM/C45Aez/hUVa2NyFiq8x0lK2vq ACcVmLMbBs7PKlvhoAOfUWS2IzahFAqefHUsu4JL3ZdalgUfM+K7xFt/RH1Ns2oDDO0B rB7EHQqowy0YUNP3WYArUn9PkfRkgbFbm1akQ2jv7qTGjFy0FA7lEcrFKUiZmX0kqZdk 4/MQ== X-Forwarded-Encrypted: i=1; AJvYcCVGM5K+mHScQJykqy0PaMOwKRwL+UUczh/+Lw29ifvOYMxFmLtTquaBAV/AEFTw52NVyrE1SONn4GE=@lists.xenproject.org X-Gm-Message-State: AOJu0Yze6hd7tRDVBiH9sjASw7h7wN+aQFkpEUiUFZ88Dzh3dpq/C2Gi G+q0VPOYBDWsy6txLE+BBOs5GLJeyIyN8yWflLgtdi9xIeHtjxZrUHtETW2PfeCbDioiS1qw1Fk K5g== X-Google-Smtp-Source: AGHT+IEZuLrSoCpVWZXJcy9TmAp9qIs68IJxs5ZGUPBqjbzVn9jX2mg4vMG9kljqKgFFkZhvGqcDyGJTu1M= X-Received: from pjbsn14.prod.google.com ([2002:a17:90b:2e8e:b0:2fc:15bf:92f6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1f8e:b0:2ee:b8ac:73b0 with SMTP id 98e67ed59e1d1-2fe68acd43fmr15933009a91.2.1740622744544; Wed, 26 Feb 2025 18:19:04 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:17 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-2-seanjc@google.com> Subject: [PATCH v2 01/38] x86/tsc: Add a standalone helpers for getting TSC info from CPUID.0x15 From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Extract retrieval of TSC frequency information from CPUID into standalone helpers so that TDX guest support and kvmlock can reuse the logic. Provide a version that includes the multiplier math as TDX in particular does NOT want to use native_calibrate_tsc()'s fallback logic that derives the TSC frequency based on CPUID.0x16 when the core crystal frequency isn't known. Opportunsitically drop native_calibrate_tsc()'s "== 0" and "!= 0" check in favor of the kernel's preferred style. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/tsc.h | 9 +++++ arch/x86/kernel/tsc.c | 67 +++++++++++++++++++++++++------------- 2 files changed, 53 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index 94408a784c8e..a4d84f721775 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -28,6 +28,15 @@ static inline cycles_t get_cycles(void) } #define get_cycles get_cycles +struct cpuid_tsc_info { + unsigned int denominator; + unsigned int numerator; + unsigned int crystal_khz; + unsigned int tsc_khz; +}; +extern int cpuid_get_tsc_info(struct cpuid_tsc_info *info); +extern int cpuid_get_tsc_freq(struct cpuid_tsc_info *info); + extern void tsc_early_init(void); extern void tsc_init(void); extern void mark_tsc_unstable(char *reason); diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 34dec0b72ea8..93713eb81f52 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -655,46 +655,67 @@ static unsigned long quick_pit_calibrate(void) return delta; } +int cpuid_get_tsc_info(struct cpuid_tsc_info *info) +{ + unsigned int ecx_hz, edx; + + memset(info, 0, sizeof(*info)); + + if (boot_cpu_data.cpuid_level < CPUID_LEAF_TSC) + return -ENOENT; + + /* CPUID 15H TSC/Crystal ratio, plus optionally Crystal Hz */ + cpuid(CPUID_LEAF_TSC, &info->denominator, &info->numerator, &ecx_hz, &edx); + + if (!info->denominator || !info->numerator) + return -ENOENT; + + /* + * Note, some CPUs provide the multiplier information, but not the core + * crystal frequency. The multiplier information is still useful for + * such CPUs, as the crystal frequency can be gleaned from CPUID.0x16. + */ + info->crystal_khz = ecx_hz / 1000; + return 0; +} + +int cpuid_get_tsc_freq(struct cpuid_tsc_info *info) +{ + if (cpuid_get_tsc_info(info) || !info->crystal_khz) + return -ENOENT; + + info->tsc_khz = info->crystal_khz * info->numerator / info->denominator; + return 0; +} + /** * native_calibrate_tsc - determine TSC frequency * Determine TSC frequency via CPUID, else return 0. */ unsigned long native_calibrate_tsc(void) { - unsigned int eax_denominator, ebx_numerator, ecx_hz, edx; - unsigned int crystal_khz; + struct cpuid_tsc_info info; if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) return 0; - if (boot_cpu_data.cpuid_level < CPUID_LEAF_TSC) + if (cpuid_get_tsc_info(&info)) return 0; - eax_denominator = ebx_numerator = ecx_hz = edx = 0; - - /* CPUID 15H TSC/Crystal ratio, plus optionally Crystal Hz */ - cpuid(CPUID_LEAF_TSC, &eax_denominator, &ebx_numerator, &ecx_hz, &edx); - - if (ebx_numerator == 0 || eax_denominator == 0) - return 0; - - crystal_khz = ecx_hz / 1000; - /* * Denverton SoCs don't report crystal clock, and also don't support * CPUID_LEAF_FREQ for the calculation below, so hardcode the 25MHz * crystal clock. */ - if (crystal_khz == 0 && - boot_cpu_data.x86_vfm == INTEL_ATOM_GOLDMONT_D) - crystal_khz = 25000; + if (!info.crystal_khz && boot_cpu_data.x86_vfm == INTEL_ATOM_GOLDMONT_D) + info.crystal_khz = 25000; /* * TSC frequency reported directly by CPUID is a "hardware reported" * frequency and is the most accurate one so far we have. This * is considered a known frequency. */ - if (crystal_khz != 0) + if (info.crystal_khz) setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); /* @@ -702,15 +723,15 @@ unsigned long native_calibrate_tsc(void) * clock, but we can easily calculate it to a high degree of accuracy * by considering the crystal ratio and the CPU speed. */ - if (crystal_khz == 0 && boot_cpu_data.cpuid_level >= CPUID_LEAF_FREQ) { + if (!info.crystal_khz && boot_cpu_data.cpuid_level >= CPUID_LEAF_FREQ) { unsigned int eax_base_mhz, ebx, ecx, edx; cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx, &ecx, &edx); - crystal_khz = eax_base_mhz * 1000 * - eax_denominator / ebx_numerator; + info.crystal_khz = eax_base_mhz * 1000 * + info.denominator / info.numerator; } - if (crystal_khz == 0) + if (!info.crystal_khz) return 0; /* @@ -727,10 +748,10 @@ unsigned long native_calibrate_tsc(void) * lapic_timer_period here to avoid having to calibrate the APIC * timer later. */ - lapic_timer_period = crystal_khz * 1000 / HZ; + lapic_timer_period = info.crystal_khz * 1000 / HZ; #endif - return crystal_khz * ebx_numerator / eax_denominator; + return info.crystal_khz * info.numerator / info.denominator; } static unsigned long cpu_khz_from_cpuid(void) From patchwork Thu Feb 27 02:18:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993540 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E4C6FC19776 for ; Thu, 27 Feb 2025 02:20:00 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897367.1306040 (Exim 4.92) (envelope-from ) id 1tnTUV-0006JE-WF; Thu, 27 Feb 2025 02:19:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897367.1306040; Thu, 27 Feb 2025 02:19:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUV-0006J7-Sf; Thu, 27 Feb 2025 02:19:11 +0000 Received: by outflank-mailman (input) for mailman id 897367; Thu, 27 Feb 2025 02:19:10 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUU-00063X-99 for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:10 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 38b008da-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:07 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc4dc34291so1060409a91.3 for ; Wed, 26 Feb 2025 18:19:07 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 38b008da-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622746; x=1741227546; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Dwfv1w4tsOCmi3NCgULhIMD4IZcsOFfau2ULDYaMRLg=; b=N1qfhRChPdcad4Avh2DERSe+cjhEEl7TUjETVBdCtu8HuwYGn+OGofPczp6jyG0XG5 qWONUxxJOcY/ZM8xSR7cXuAb+fKheA4B53nwqWZKe9pgoFc7z7dOSknTVrIUz9NaUDlf NQSFhl49AxbqZgsI2DbjbCpr8+3JJnDRf1z+3ay9bMPFrTaLJpAkKFSLiw9w1aeF4q2V Uc+N6H0E1gK7KNiRwrzCQ/j/M3SE8/84GgI3jWnrpdZY6XvyCOICovIUbkqFolxH/OH7 vvFEKngbFmGw2x+vk2fqjyt0Mo/DRGHPczUOfQps9Ahvh0N8RUR+qC+VMoKkzublphFI pJCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622746; x=1741227546; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Dwfv1w4tsOCmi3NCgULhIMD4IZcsOFfau2ULDYaMRLg=; b=tTy7M2VTH35fWjiEgUatkdGZwu1i4qjm2//1MjNHKMal/Akjq/9g60qRfdZ267q5LF ib1hMFxKOW9tXGwehO3zno8nGf/ewVXQCKJbnwx6CMUMfGxXBGDKoecD92fiBwPrfb3C egW5rohMLZ0ewKItjmQkrx35xIrEPrbyYmV1eRfyjpt5QGBVz4Bx5vqjfG0KHNksAjQL auO5X9KbRMg+24R6PZGT2V/dTFuNQw35PVryt6biaRFC1f1DZe6rfjsL/ni0sOJNx088 CjLTaNmLrK+98z3mHkOz6iZSabZkkZz6qrJPPsKL3bUbBW0jZtm0bPfVGof+AbSyFOHy uWFQ== X-Forwarded-Encrypted: i=1; AJvYcCX11zNIkKgYzwpthKLwbWC3s4W82QYAkNTe5OLdVvrDJ9zCgW7LEPlDkbnbeLQnZAPwZVPxPmeZKiQ=@lists.xenproject.org X-Gm-Message-State: AOJu0YzdKN4I2h7lV4Xtve0komaWpLzeVMSV376cw3LziQo0+OmsoZYB C/6YGrbjsvvQFRWA9OvRiy3gjEflUkwI2W7Vls9b2z5+PCyQuhRDTpoz0sHVuE4WPYXTMXm39wf z8w== X-Google-Smtp-Source: AGHT+IH6wZ/mGewjHTQw2W+siQpDLDG/G2QvagXg40e21ebxNRSjKEBR8DQxe4UjV6eqsE0kkd3QFpygjvI= X-Received: from pjbsn14.prod.google.com ([2002:a17:90b:2e8e:b0:2fc:15bf:92f6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1d83:b0:2fc:3264:3666 with SMTP id 98e67ed59e1d1-2fce7b221c3mr36215820a91.30.1740622746375; Wed, 26 Feb 2025 18:19:06 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:18 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-3-seanjc@google.com> Subject: [PATCH v2 02/38] x86/tsc: Add standalone helper for getting CPU frequency from CPUID From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Extract the guts of cpu_khz_from_cpuid() to a standalone helper that doesn't restrict the usage to Intel CPUs. This will allow sharing the core logic with kvmclock, as (a) CPUID.0x16 may be enumerated alongside kvmclock, and (b) KVM generally doesn't restrict CPUID based on vendor. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/tsc.h | 1 + arch/x86/kernel/tsc.c | 37 +++++++++++++++++++++++-------------- 2 files changed, 24 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index a4d84f721775..c3a14df46327 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -36,6 +36,7 @@ struct cpuid_tsc_info { }; extern int cpuid_get_tsc_info(struct cpuid_tsc_info *info); extern int cpuid_get_tsc_freq(struct cpuid_tsc_info *info); +extern int cpuid_get_cpu_freq(unsigned int *cpu_khz); extern void tsc_early_init(void); extern void tsc_init(void); diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 93713eb81f52..bb4619148161 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -688,6 +688,24 @@ int cpuid_get_tsc_freq(struct cpuid_tsc_info *info) return 0; } +int cpuid_get_cpu_freq(unsigned int *cpu_khz) +{ + unsigned int eax_base_mhz, ebx, ecx, edx; + + *cpu_khz = 0; + + if (boot_cpu_data.cpuid_level < CPUID_LEAF_FREQ) + return -ENOENT; + + cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx, &ecx, &edx); + + if (!eax_base_mhz) + return -ENOENT; + + *cpu_khz = eax_base_mhz * 1000; + return 0; +} + /** * native_calibrate_tsc - determine TSC frequency * Determine TSC frequency via CPUID, else return 0. @@ -723,13 +741,8 @@ unsigned long native_calibrate_tsc(void) * clock, but we can easily calculate it to a high degree of accuracy * by considering the crystal ratio and the CPU speed. */ - if (!info.crystal_khz && boot_cpu_data.cpuid_level >= CPUID_LEAF_FREQ) { - unsigned int eax_base_mhz, ebx, ecx, edx; - - cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx, &ecx, &edx); - info.crystal_khz = eax_base_mhz * 1000 * - info.denominator / info.numerator; - } + if (!info.crystal_khz && !cpuid_get_cpu_freq(&cpu_khz)) + info.crystal_khz = cpu_khz * info.denominator / info.numerator; if (!info.crystal_khz) return 0; @@ -756,19 +769,15 @@ unsigned long native_calibrate_tsc(void) static unsigned long cpu_khz_from_cpuid(void) { - unsigned int eax_base_mhz, ebx_max_mhz, ecx_bus_mhz, edx; + unsigned int cpu_khz; if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) return 0; - if (boot_cpu_data.cpuid_level < CPUID_LEAF_FREQ) + if (cpuid_get_cpu_freq(&cpu_khz)) return 0; - eax_base_mhz = ebx_max_mhz = ecx_bus_mhz = edx = 0; - - cpuid(CPUID_LEAF_FREQ, &eax_base_mhz, &ebx_max_mhz, &ecx_bus_mhz, &edx); - - return eax_base_mhz * 1000; + return cpu_khz; } /* From patchwork Thu Feb 27 02:18:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 15A12C19776 for ; Thu, 27 Feb 2025 02:19:35 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897368.1306045 (Exim 4.92) (envelope-from ) id 1tnTUW-0006MZ-9X; Thu, 27 Feb 2025 02:19:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897368.1306045; Thu, 27 Feb 2025 02:19:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUW-0006Lf-3p; Thu, 27 Feb 2025 02:19:12 +0000 Received: by outflank-mailman (input) for mailman id 897368; Thu, 27 Feb 2025 02:19:11 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUV-00063X-9T for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:11 +0000 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [2607:f8b0:4864:20::104a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 39bd186e-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:09 +0100 (CET) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2fc1c7c8396so1082722a91.0 for ; Wed, 26 Feb 2025 18:19:09 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 39bd186e-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622748; x=1741227548; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=VvDX2+gOZciJHv0b670CNF4YevTYL2569oCfzMxNGMY=; b=VE7xjkVyM37vonGgRzpU5rDoiywPKkNotpsaiU6G2QV3N9fGkR5oFNeLPwont3l5EO BJ65reI8KazLtFjvlCGPSoPr5vRaw8mpePiHFVY1Hf3GGdcQzqp5ICGl5lcuPvf6A/Lu FQqKdbSiJ5vQWwV8H6+rG1iozkTDWxnGYv5YB62rBQY1+dCXNaQCZcnXQIoVhciWs2ug MhtagnMjQLQWYSBkrjyzoOTDt/4Gqeatiy8+c/KhfzUvRTjFqB78YzepMjZg3HPoaasY o93X3yP1i6MNyX2b/1fGEYxwM4xnI/yX6o1smAQyZa50kALL8Xk15Y5r6xgDZ1B6ppTY 9wAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622748; x=1741227548; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VvDX2+gOZciJHv0b670CNF4YevTYL2569oCfzMxNGMY=; b=Ki8kMeF6fvLpEITeoCKKe+j3JHTKiw6M4gDcD7s43XOHRhLSdVW0TZqIsSVjEFWUO9 3v2HXhCJIV3yvy52tPKX788ebEfOKOo6HpkJ3KD6gVFOUDK/ALz6cbO8hpiEUen5Czxg ngccMCTB3KhaWBCFHyKY0NBH8jbk+USQpmzzZHmhQfVUm7A6UXwt0EcggpJPcsz97bBK n+SlkGMLrjy+8/RxalrB9VULLNdxnxqtRitd5qfvr7nIs5I6KRbZ/jMK56u7jq3UwB9n 46JOgM18oIIcucSngY1eqb/G5KwgNuvyHGpoOjAos0lQWjIKR0x9umN4jHza+nnJPy3k 7VOg== X-Forwarded-Encrypted: i=1; AJvYcCXMKZ+zcNlK/OEFEseyI5eAJuXev98jvPGr97CsK7ka59JSgHyvZE0uids4JZx27oHnJ3aNn6kcWIQ=@lists.xenproject.org X-Gm-Message-State: AOJu0YxDm4gOhs7j3PiIpflYXYD1ND2nf/OtLgwCOENvCedmjZRolZ6Z KAJgEYYG+G1aw2PokDqXqsnCdv0clq7CyjwP6C4W0MpdQA+Gw7DO88QR5SeUolU7wZYvLEcXz3t ItQ== X-Google-Smtp-Source: AGHT+IFmK8jVkmb1xHfpVWU6EZoqjhev83fKG44NRMRJnNlywOI5c93DmBzKNvUnwRZS9xQkuc4Cdr3SCGs= X-Received: from pjbsz5.prod.google.com ([2002:a17:90b:2d45:b0:2ea:5be5:da6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5148:b0:2ee:c9b6:4c42 with SMTP id 98e67ed59e1d1-2fce86cf0ebmr41779532a91.16.1740622748050; Wed, 26 Feb 2025 18:19:08 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:19 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-4-seanjc@google.com> Subject: [PATCH v2 03/38] x86/tsc: Add helper to register CPU and TSC freq calibration routines From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Add a helper to register non-native, i.e. PV and CoCo, CPU and TSC frequency calibration routines. This will allow consolidating handling of common TSC properties that are forced by hypervisor (PV routines), and will also allow adding sanity checks to guard against overriding a TSC calibration routine with a routine that is less robust/trusted. Make the CPU calibration routine optional, as Xen (very sanely) doesn't assume the CPU runs as the same frequency as the TSC. Wrap the helper in an #ifdef to document that the kernel overrides the native routines when running as a VM, and to guard against unwanted usage. Add a TODO to call out that AMD_MEM_ENCRYPT is a mess and doesn't depend on HYPERVISOR_GUEST because it gates both guest and host code. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 4 ++-- arch/x86/include/asm/tsc.h | 4 ++++ arch/x86/kernel/cpu/acrn.c | 4 ++-- arch/x86/kernel/cpu/mshyperv.c | 3 +-- arch/x86/kernel/cpu/vmware.c | 4 ++-- arch/x86/kernel/jailhouse.c | 4 ++-- arch/x86/kernel/kvmclock.c | 4 ++-- arch/x86/kernel/tsc.c | 17 +++++++++++++++++ arch/x86/xen/time.c | 2 +- 9 files changed, 33 insertions(+), 13 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index 82492efc5d94..684cef70edc1 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3291,6 +3291,6 @@ void __init snp_secure_tsc_init(void) rdmsrl(MSR_AMD64_GUEST_TSC_FREQ, tsc_freq_mhz); snp_tsc_freq_khz = (unsigned long)(tsc_freq_mhz * 1000); - x86_platform.calibrate_cpu = securetsc_get_tsc_khz; - x86_platform.calibrate_tsc = securetsc_get_tsc_khz; + tsc_register_calibration_routines(securetsc_get_tsc_khz, + securetsc_get_tsc_khz); } diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index c3a14df46327..9318c74e8d13 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -40,6 +40,10 @@ extern int cpuid_get_cpu_freq(unsigned int *cpu_khz); extern void tsc_early_init(void); extern void tsc_init(void); +#if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) +extern void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), + unsigned long (*calibrate_cpu)(void)); +#endif extern void mark_tsc_unstable(char *reason); extern int unsynchronized_tsc(void); extern int check_tsc_unstable(void); diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index 2c5b51aad91a..c1506cb87d8c 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -29,8 +29,8 @@ static void __init acrn_init_platform(void) /* Install system interrupt handler for ACRN hypervisor callback */ sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); - x86_platform.calibrate_tsc = acrn_get_tsc_khz; - x86_platform.calibrate_cpu = acrn_get_tsc_khz; + tsc_register_calibration_routines(acrn_get_tsc_khz, + acrn_get_tsc_khz); } static bool acrn_x2apic_available(void) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index f285757618fc..aa60491bf738 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -478,8 +478,7 @@ static void __init ms_hyperv_init_platform(void) if (ms_hyperv.features & HV_ACCESS_FREQUENCY_MSRS && ms_hyperv.misc_features & HV_FEATURE_FREQUENCY_MSRS_AVAILABLE) { - x86_platform.calibrate_tsc = hv_get_tsc_khz; - x86_platform.calibrate_cpu = hv_get_tsc_khz; + tsc_register_calibration_routines(hv_get_tsc_khz, hv_get_tsc_khz); setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); } diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index 00189cdeb775..d6f079a75f05 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -416,8 +416,8 @@ static void __init vmware_platform_setup(void) } vmware_tsc_khz = tsc_khz; - x86_platform.calibrate_tsc = vmware_get_tsc_khz; - x86_platform.calibrate_cpu = vmware_get_tsc_khz; + tsc_register_calibration_routines(vmware_get_tsc_khz, + vmware_get_tsc_khz); #ifdef CONFIG_X86_LOCAL_APIC /* Skip lapic calibration since we know the bus frequency. */ diff --git a/arch/x86/kernel/jailhouse.c b/arch/x86/kernel/jailhouse.c index cd8ed1edbf9e..b0a053692161 100644 --- a/arch/x86/kernel/jailhouse.c +++ b/arch/x86/kernel/jailhouse.c @@ -209,8 +209,6 @@ static void __init jailhouse_init_platform(void) x86_init.mpparse.parse_smp_cfg = jailhouse_parse_smp_config; x86_init.pci.arch_init = jailhouse_pci_arch_init; - x86_platform.calibrate_cpu = jailhouse_get_tsc; - x86_platform.calibrate_tsc = jailhouse_get_tsc; x86_platform.get_wallclock = jailhouse_get_wallclock; x86_platform.legacy.rtc = 0; x86_platform.legacy.warm_reset = 0; @@ -220,6 +218,8 @@ static void __init jailhouse_init_platform(void) machine_ops.emergency_restart = jailhouse_no_restart; + tsc_register_calibration_routines(jailhouse_get_tsc, jailhouse_get_tsc); + while (pa_data) { mapping = early_memremap(pa_data, sizeof(header)); memcpy(&header, mapping, sizeof(header)); diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 5b2c15214a6b..b898b95a7d50 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -320,8 +320,8 @@ void __init kvmclock_init(void) flags = pvclock_read_flags(&hv_clock_boot[0].pvti); kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); - x86_platform.calibrate_tsc = kvm_get_tsc_khz; - x86_platform.calibrate_cpu = kvm_get_tsc_khz; + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz); + x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; #ifdef CONFIG_X86_LOCAL_APIC diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index bb4619148161..d65e85929d3e 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1294,6 +1294,23 @@ static void __init check_system_tsc_reliable(void) tsc_disable_clocksource_watchdog(); } +/* + * TODO: Disentangle AMD_MEM_ENCRYPT and make SEV guest support depend on + * HYPERVISOR_GUEST. + */ +#if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) +void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), + unsigned long (*calibrate_cpu)(void)) +{ + if (WARN_ON_ONCE(!calibrate_tsc)) + return; + + x86_platform.calibrate_tsc = calibrate_tsc; + if (calibrate_cpu) + x86_platform.calibrate_cpu = calibrate_cpu; +} +#endif + /* * Make an educated guess if the TSC is trustworthy and synchronized * over all CPUs. diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 96521b1874ac..9e2e900dc0c7 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -566,7 +566,7 @@ static void __init xen_init_time_common(void) static_call_update(pv_steal_clock, xen_steal_clock); paravirt_set_sched_clock(xen_sched_clock); - x86_platform.calibrate_tsc = xen_tsc_khz; + tsc_register_calibration_routines(xen_tsc_khz, NULL); x86_platform.get_wallclock = xen_get_wallclock; } From patchwork Thu Feb 27 02:18:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993536 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 57F7CC021B8 for ; Thu, 27 Feb 2025 02:19:43 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897369.1306060 (Exim 4.92) (envelope-from ) id 1tnTUX-0006mF-EU; Thu, 27 Feb 2025 02:19:13 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897369.1306060; Thu, 27 Feb 2025 02:19:13 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUX-0006l2-AQ; Thu, 27 Feb 2025 02:19:13 +0000 Received: by outflank-mailman (input) for mailman id 897369; Thu, 27 Feb 2025 02:19:11 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUV-0005qU-Ph for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:11 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 3ac0d216-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:11 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fe8c5dbdb0so1061021a91.3 for ; Wed, 26 Feb 2025 18:19:11 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3ac0d216-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622750; x=1741227550; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=HbJ/1jLoPDWkVBe7cDWaMugTDenciqtu5nrTgQ9Zvm0=; b=I7QL8ZPwWO1oqFZxwYCoc80EI0hRHQqQbAlsZ79RHZP5h7pHY2rgFVbPQbgWEr5n2r VBHeP2JmEdCC1wP2RlLyeUNUmG2ZWz0kIEQF0Jrozm3YRPMN6ugck3l1JAbT5Nrk65j5 lbMkvC+eBphKxw4kW74TJZVinvGV12zcXK1ZlmCaXZ2sNP5EKPkmVHY0irhQaZ6+O1Pi bdxYUekTWNQYZuS6H+em1PmOYf3+G99dxPinYNYerXNRCBsCsZA8glWKkA/wF4hI5Q5M EH9cutti+f40rGyoABf9BbIeB8uZdOExG6JmI6C+09ubVTb2MobHlPKVf1N+4I9ka87w FeXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622750; x=1741227550; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HbJ/1jLoPDWkVBe7cDWaMugTDenciqtu5nrTgQ9Zvm0=; b=shn+Fk10sgtbEI6KnY45MQOD3np4lq7E6vwlHRqEonToNQpZiaWzBn39U08aMLbZ+P QkbCa6QvAE0Mfld7TPqmv9mvfUsiH3B0fHlSY0tIVqarZ0L7QLqWj+USsHOXHkrHqXng H2AIyMM1oJFfRkbRLCPOVEjHUmQRtTj+IUxK8+sE1GVm/hltjHjKsI1AyBDCtZzfwT4e /g91B/H9wMR8ck33bpJ+z1+3q2gwTIwfnSSHQerzggKVFMhRRapMYQW4soEmh+8TyC3p uT5wL/02FtUmB7fOmf+YUOQD5Cu3JxPPSgdWm6nsuz0E5TYaYEkxTcgttSu7MHgZUTCd 5tAw== X-Forwarded-Encrypted: i=1; AJvYcCWEQ5lMELySa5Ls5j4RSozENyYOmOpiOKZNk9OLrgq0RjgRgArkEQv0xxpKJeGWu5hfgaiALDzWP+M=@lists.xenproject.org X-Gm-Message-State: AOJu0YztioBU/Tw3TsPVdvvVBwT9AVEMPdNobpkn6gKou74gh9whY0SU XGQKDpr3/R+U/oRCnnvGDlrk2wGwMnb0MRxSJhVUhduekj3zftVpXSqXui9sgpv2eKHVECE+1lp G2w== X-Google-Smtp-Source: AGHT+IFmgEsUJJ1z3bRk5Nf8mtm9mdRWiQdO+4qbvCwBd/hyjI3jP5eNnlt+3d6bfd4x/QDsZrTqSfxiFQY= X-Received: from pjbsw3.prod.google.com ([2002:a17:90b:2c83:b0:2fa:15aa:4d2b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:56cc:b0:2f1:2fa5:1924 with SMTP id 98e67ed59e1d1-2fe7e39f2afmr7133368a91.26.1740622749803; Wed, 26 Feb 2025 18:19:09 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:20 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-5-seanjc@google.com> Subject: [PATCH v2 04/38] x86/sev: Mark TSC as reliable when configuring Secure TSC From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move the code to mark the TSC as reliable from sme_early_init() to snp_secure_tsc_init(). The only reader of TSC_RELIABLE is the aptly named check_system_tsc_reliable(), which runs in tsc_init(), i.e. after snp_secure_tsc_init(). This will allow consolidating the handling of TSC_KNOWN_FREQ and TSC_RELIABLE when overriding the TSC calibration routine. Cc: Tom Lendacky Reviewed-by: Nikunj A Dadhania Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 2 ++ arch/x86/mm/mem_encrypt_amd.c | 3 --- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index 684cef70edc1..e6ce4ca72465 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3288,6 +3288,8 @@ void __init snp_secure_tsc_init(void) return; setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + rdmsrl(MSR_AMD64_GUEST_TSC_FREQ, tsc_freq_mhz); snp_tsc_freq_khz = (unsigned long)(tsc_freq_mhz * 1000); diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c index b56c5c073003..774f9677458f 100644 --- a/arch/x86/mm/mem_encrypt_amd.c +++ b/arch/x86/mm/mem_encrypt_amd.c @@ -541,9 +541,6 @@ void __init sme_early_init(void) * kernel mapped. */ snp_update_svsm_ca(); - - if (sev_status & MSR_AMD64_SNP_SECURE_TSC) - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); } void __init mem_encrypt_free_decrypted_mem(void) From patchwork Thu Feb 27 02:18:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 85A9FC19776 for ; Thu, 27 Feb 2025 02:19:57 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897370.1306070 (Exim 4.92) (envelope-from ) id 1tnTUZ-00073L-MR; Thu, 27 Feb 2025 02:19:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897370.1306070; Thu, 27 Feb 2025 02:19:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUZ-000738-JK; Thu, 27 Feb 2025 02:19:15 +0000 Received: by outflank-mailman (input) for mailman id 897370; Thu, 27 Feb 2025 02:19:14 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUY-00063X-Lx for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:14 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 3bdf564a-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:13 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fe86c01f4aso1065320a91.2 for ; Wed, 26 Feb 2025 18:19:13 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3bdf564a-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622752; x=1741227552; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=JV0vL5ay0SjXSCwSvRRqMw5uYQ06F8yr+AS2fSIuB98=; b=pWLcT2rKPGHuSwHWKRkTx2avZ+yfI2qcUh53fODALnArzXGoBp2/gmbd1urtjXQd7i TeHStqrHY4xxn0RC+AZUrXI+dGErIt+pIym/XjWy+XT9TVn+3kV38bjeFykHwd2lt0ff fCuiILxwOtlVbSfzhD+/G2TQsP25fiAAUTVE14+8vDsD4bxIhyXIWSwWL/QfH/zEaM3j mobB1K+4B3N3ggameCADMSbRKPGaNYnufliuSvjX3fQ5PUiThE3996Wak+MUdqh9kO4j +LQ6xnEiWwmi9FjR0HB0JwDM0t66xUkHBHHM+i7q6S1AG4+EfXHk1FUg4d/6SAJURn8/ RXFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622752; x=1741227552; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=JV0vL5ay0SjXSCwSvRRqMw5uYQ06F8yr+AS2fSIuB98=; b=nZQ/MS2/hy04fPYk3ONX0Msty4s3+f5DZN+S/jo+2wA99tTJzGze7yyHSAbhF1ak/i 0njdgxscZkOqKxkpbOyVG/UGJJsqOrFak+/6EVsAoWavehXRfJ539nEjuFIu+XNJWVq3 LF7a9LpUV/y9jPnAZ8RUKOA+S09cEz3zmW79UL6E0mbswla9T75nAJgvCd7iApVs4jiu kmDpycyYnzoqJugZ/07R2s8r7+0TjZz34eEnjDR3vRD72MDIWDNEu5inTcYaW70Pq6JA XspSAeJv/QqgW5YTLcF6iVLlH0d7tN0D1l3nTuiyxai26JAVRXOBNNTk8pY4bCP+Alr9 gF4Q== X-Forwarded-Encrypted: i=1; AJvYcCXDnanQS/k7nhziVQ460TpZrTEn6qfY1BxJI4zroFyllPN6jkO7NbnpfW9y+1MNH4wgsPUITd0Exhk=@lists.xenproject.org X-Gm-Message-State: AOJu0YybD9IFfG+F/1q8bFPBPX/Z7Z4pPXBbIRHskvRnHgqPnZbQI5I+ Y+PDWlXBqeugGYdPPoHdJV93CvtPsQFKTt4TgHiDdeiAmoGHpNaDUU30DOrfVXeoOZ+XE78f6YB o3Q== X-Google-Smtp-Source: AGHT+IEvNph7CiMpObG1u+jVz8JxqMtO0a8+Wef5S19XtPgcpuEdHq4SZt8ZBvsmjyd64opTzInKcYcJX7E= X-Received: from pjbsw14.prod.google.com ([2002:a17:90b:2c8e:b0:2fc:11a0:c546]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3b48:b0:2fc:9967:acd8 with SMTP id 98e67ed59e1d1-2fe7e3b327fmr9422757a91.33.1740622751672; Wed, 26 Feb 2025 18:19:11 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:21 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-6-seanjc@google.com> Subject: [PATCH v2 05/38] x86/sev: Move check for SNP Secure TSC support to tsc_early_init() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move the check on having a Secure TSC to the common tsc_early_init() so that it's obvious that having a Secure TSC is conditional, and to prepare for adding TDX to the mix (blindly initializing *both* SNP and TDX TSC logic looks especially weird). No functional change intended. Cc: Tom Lendacky Reviewed-by: Nikunj A Dadhania Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 3 --- arch/x86/kernel/tsc.c | 3 ++- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index e6ce4ca72465..dab386f782ce 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3284,9 +3284,6 @@ void __init snp_secure_tsc_init(void) { unsigned long long tsc_freq_mhz; - if (!cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)) - return; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index d65e85929d3e..6a011cd1ff94 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1563,7 +1563,8 @@ void __init tsc_early_init(void) if (is_early_uv_system()) return; - snp_secure_tsc_init(); + if (cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)) + snp_secure_tsc_init(); if (!determine_cpu_tsc_frequencies(true)) return; From patchwork Thu Feb 27 02:18:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 230BAC19776 for ; Thu, 27 Feb 2025 02:20:06 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897371.1306081 (Exim 4.92) (envelope-from ) id 1tnTUc-0007MX-6f; Thu, 27 Feb 2025 02:19:18 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897371.1306081; Thu, 27 Feb 2025 02:19:18 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUc-0007MQ-0d; Thu, 27 Feb 2025 02:19:18 +0000 Received: by outflank-mailman (input) for mailman id 897371; Thu, 27 Feb 2025 02:19:16 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUZ-0005qU-VV for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:15 +0000 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [2607:f8b0:4864:20::649]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 3d0b7bca-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:15 +0100 (CET) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-220e04e67e2so10180855ad.2 for ; Wed, 26 Feb 2025 18:19:15 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3d0b7bca-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622753; x=1741227553; darn=lists.xenproject.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=yramOYZYSvDmgSz8M8zEvByRhd31043Cze2FYqM0HuQ=; b=yblITf8yRGWXe63wTFOCFb9Dsu0I7Tt4DfGmxd5B+W/jefrV+XhwBI91lTrM28Oy6S g2RXqDSuRSPxy6mhvOAduy9must7/L9/QInYFypbxB8XqNvF+URdSuycgS6LRBzYVB1D P9faqfDYYIFCca5tCwvxdkkWUERPmXtBliR4e+/Dsfr3ILLqKXUj5oFWI0Px8jUzRj1A dK2M9YEr8En9DgCHy7mCYObuHfBcG1sEekkTPQcJ2UCyGBy3JLpD0w4s7GT73Hr2eHKf xDbPfiWMitG9fTmgi4d33NsC9x9tPoM6Uwfo0zch5JrZdDgLt5TSt2/sIaRQHcecmot9 PLvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622753; x=1741227553; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=yramOYZYSvDmgSz8M8zEvByRhd31043Cze2FYqM0HuQ=; b=voiL6hXyITG8QDTV3Q44BqViCwH4GqbERmHNtREKjWlUHtefVhCy35SzNRhXjUozEH h4EHoG7SCqbo2rKLgWoSWPIvwjKTCPyouZoaGNnTHRDqWOcOmpUNaxEOBIKFasJzddik 9VehfdvErm+quJ9gLoncwdkbE4jOGZ2GFG1IU0Sq4/EC5GAT4P5WRWxhvSQtdQ4WYuJB +JioAicLrAsN/VaomXQsgiz56tMs770IJeTM7z4GyaATHQndIWqP9TPmx0dhzu+ptwfJ IRKRpP3UdGtd92tZhJkE1D61nQBKFd0bmXUaz/AV4XKNY6MAnamD/CsoSlLf80KNjcwa cHnA== X-Forwarded-Encrypted: i=1; AJvYcCVkZxaHpfB9mD7I8Pb8uZi+AndweijEWVCvOlPkzAkyvDoWrgOkML7jI/Ki61g55e6ma0jFyi77h8s=@lists.xenproject.org X-Gm-Message-State: AOJu0YwcxmTCN6GRavLlw49kwAk4NiIIMlPrG6BoJUwC4M4AEiTJGhxs SPd8R5Oj5e2uUoVhok2UjRA8YRsGRqC07SIcY+e+cQnJO3yjNMtVtghamH/BvOnAiNZpfNtkXKF +ZA== X-Google-Smtp-Source: AGHT+IEj/1REcONxv6g8r36LvNpCxOj7lRlC5Gm8UGTSETJJJpMqJ2Q+yEo6DBGNGx6TMjORq5yBRC/aLmE= X-Received: from pfbf4.prod.google.com ([2002:a05:6a00:ad84:b0:730:94db:d304]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:dacc:b0:220:d6ac:b5a3 with SMTP id d9443c01a7336-22320214b94mr83920905ad.51.1740622753535; Wed, 26 Feb 2025 18:19:13 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:22 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-7-seanjc@google.com> Subject: [PATCH v2 06/38] x86/tdx: Override PV calibration routines with CPUID-based calibration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When running as a TDX guest, explicitly override the TSC frequency calibration routine with CPUID-based calibration instead of potentially relying on a hypervisor-controlled PV routine. For TDX guests, CPUID.0x15 is always emulated by the TDX-Module, i.e. the information from CPUID is more trustworthy than the information provided by the hypervisor. To maintain backwards compatibility with TDX guest kernels that use native calibration, and because it's the least awful option, retain native_calibrate_tsc()'s stuffing of the local APIC bus period using the core crystal frequency. While it's entirely possible for the hypervisor to emulate the APIC timer at a different frequency than the core crystal frequency, the commonly accepted interpretation of Intel's SDM is that APIC timer runs at the core crystal frequency when that latter is enumerated via CPUID: The APIC timer frequency will be the processor’s bus clock or core crystal clock frequency (when TSC/core crystal clock ratio is enumerated in CPUID leaf 0x15). If the hypervisor is malicious and deliberately runs the APIC timer at the wrong frequency, nothing would stop the hypervisor from modifying the frequency at any time, i.e. attempting to manually calibrate the frequency out of paranoia would be futile. Deliberately leave the CPU frequency calibration routine as is, since the TDX-Module doesn't provide any guarantees with respect to CPUID.0x16. Opportunistically add a comment explaining that CoCo TSC initialization needs to come after hypervisor specific initialization. Cc: Kirill A. Shutemov Signed-off-by: Sean Christopherson --- arch/x86/coco/tdx/tdx.c | 30 +++++++++++++++++++++++++++--- arch/x86/include/asm/tdx.h | 2 ++ arch/x86/kernel/tsc.c | 8 ++++++++ 3 files changed, 37 insertions(+), 3 deletions(-) diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index 32809a06dab4..42cdaa98dc5e 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -1063,9 +1064,6 @@ void __init tdx_early_init(void) setup_force_cpu_cap(X86_FEATURE_TDX_GUEST); - /* TSC is the only reliable clock in TDX guest */ - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - cc_vendor = CC_VENDOR_INTEL; /* Configure the TD */ @@ -1122,3 +1120,29 @@ void __init tdx_early_init(void) tdx_announce(); } + +static unsigned long tdx_get_tsc_khz(void) +{ + struct cpuid_tsc_info info; + + if (WARN_ON_ONCE(cpuid_get_tsc_freq(&info))) + return 0; + + lapic_timer_period = info.crystal_khz * 1000 / HZ; + + return info.tsc_khz; +} + +void __init tdx_tsc_init(void) +{ + /* TSC is the only reliable clock in TDX guest */ + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + + /* + * Override the PV calibration routines (if set) with more trustworthy + * CPUID-based calibration. The TDX module emulates CPUID, whereas any + * PV information is provided by the hypervisor. + */ + tsc_register_calibration_routines(tdx_get_tsc_khz, NULL); +} diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index b4b16dafd55e..621fbdd101e2 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -53,6 +53,7 @@ struct ve_info { #ifdef CONFIG_INTEL_TDX_GUEST void __init tdx_early_init(void); +void __init tdx_tsc_init(void); void tdx_get_ve_info(struct ve_info *ve); @@ -72,6 +73,7 @@ void __init tdx_dump_td_ctls(u64 td_ctls); #else static inline void tdx_early_init(void) { }; +static inline void tdx_tsc_init(void) { } static inline void tdx_safe_halt(void) { }; static inline bool tdx_early_handle_ve(struct pt_regs *regs) { return false; } diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 6a011cd1ff94..472d6c71d3a5 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -32,6 +32,7 @@ #include #include #include +#include unsigned int __read_mostly cpu_khz; /* TSC clocks / usec, not used here */ EXPORT_SYMBOL(cpu_khz); @@ -1563,8 +1564,15 @@ void __init tsc_early_init(void) if (is_early_uv_system()) return; + /* + * Do CoCo specific "secure" TSC initialization *after* hypervisor + * platform initialization so that the secure variant can override the + * hypervisor's PV calibration routine with a more trusted method. + */ if (cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)) snp_secure_tsc_init(); + else if (boot_cpu_has(X86_FEATURE_TDX_GUEST)) + tdx_tsc_init(); if (!determine_cpu_tsc_frequencies(true)) return; From patchwork Thu Feb 27 02:18:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9E951C021B8 for ; Thu, 27 Feb 2025 02:19:38 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897373.1306084 (Exim 4.92) (envelope-from ) id 1tnTUc-0007PU-IA; Thu, 27 Feb 2025 02:19:18 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897373.1306084; Thu, 27 Feb 2025 02:19:18 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUc-0007OY-9e; Thu, 27 Feb 2025 02:19:18 +0000 Received: by outflank-mailman (input) for mailman id 897373; Thu, 27 Feb 2025 02:19:17 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUb-0005qU-9Z for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:17 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 3e0f5d37-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:16 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc0bc05c00so1507791a91.2 for ; Wed, 26 Feb 2025 18:19:16 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3e0f5d37-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622755; x=1741227555; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YtBqucRQyVVjFtgUlGbh/U8C9xbSv2L/3t5kwpG/0fs=; b=KNAtg8tWo9X0AQhlg9SQSFD7Smjra3pgMsgkpywJh/syOPi4T9+huMzm15Qo6yDnhH 3bTNGPwswFMm4WA25KlTnQ6nTW18mDBAcE/ViZkKmdntJRMSfgqfZXvFYlQk6PVY6kQz sgQy4xp4oa8scPieu0OCUs9zUzhEG3mMTtFEEleI8earuM0vgL4WyXTa29oydcc/1bON c7rSIvYbb1JlC1m6HZoiUVTDH5fR3ywA8TYzOf+B5j/elscpDzA0agxc++3gZAQYSz8v IxceItMwRzrEIUHDjzYOIugNsDg9U/SnAiwbsOCBUfPCbrh+fOvjkrGO3W+wuId0nvQV e9yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622755; x=1741227555; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YtBqucRQyVVjFtgUlGbh/U8C9xbSv2L/3t5kwpG/0fs=; b=b3mm7snRTqbO+ZJO0pndztJTJEyM8H2aq47sOfJqFzZQsLNRvL4wliUQDR9eDs/HZn Ca9TuP1W4CXthI7+OXoP5xIVdRoYToHUs49XpkcFqY4h2pGl6aZq3m6s5NotKhlmGIf3 FEip2oSMQHN70Hd3z2RTYCjyruPHtFhznDP+st3Zwk2uE2TlYc7T5pr6csREz3tWiZVD O/s7H0BV34bXkOcxg9yvO2PYZdL5RXrYXjW784qcb+r1Xn3frdrEe8pk78I3UWDUlxX7 rsEOuyaDs/lALyVhdLGismdlmiTb5qfiS/TLCML6d3/LbyncGiJ/LRjIlofSs+Y/bhz3 RPwA== X-Forwarded-Encrypted: i=1; AJvYcCWSddeo1xTdiZzmF8K2MGuhusacUqxHWVss+i13ZoyOxCND/xPAxxKwAJmURtuCveSB24glFszolv0=@lists.xenproject.org X-Gm-Message-State: AOJu0YzKcbGmPcpzX/KDEu9i14Tm8Z6ocCSJKs1UvRMkrGvKhXvueoQV tiJVHlaXY31tJn0+GJflYy1YzR3nuwoJdOqeFLzcTjnN6jPio9lHFS75ktIukZO3lm3UwvvuLbK 5oA== X-Google-Smtp-Source: AGHT+IH9Y0SwBcAS02xHNq+a4IakpnWSLKqVw4P68iCQM2m6waYT1AwLAOPZPnrjzYdSpJiao+kBYkSTr34= X-Received: from pjb4.prod.google.com ([2002:a17:90b:2f04:b0:2ef:78ff:bc3b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2dc8:b0:2fa:ba3:5457 with SMTP id 98e67ed59e1d1-2fe68ae6c4fmr17520974a91.17.1740622755379; Wed, 26 Feb 2025 18:19:15 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:23 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-8-seanjc@google.com> Subject: [PATCH v2 07/38] x86/acrn: Mark TSC frequency as known when using ACRN for calibration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Mark the TSC frequency as known when using ACRN's PV CPUID information. Per commit 81a71f51b89e ("x86/acrn: Set up timekeeping") and common sense, the TSC freq is explicitly provided by the hypervisor. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/acrn.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index c1506cb87d8c..2da3de4d470e 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -29,6 +29,7 @@ static void __init acrn_init_platform(void) /* Install system interrupt handler for ACRN hypervisor callback */ sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); tsc_register_calibration_routines(acrn_get_tsc_khz, acrn_get_tsc_khz); } From patchwork Thu Feb 27 02:18:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993538 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4E16DC19776 for ; Thu, 27 Feb 2025 02:19:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897375.1306100 (Exim 4.92) (envelope-from ) id 1tnTUe-0007xO-SN; Thu, 27 Feb 2025 02:19:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897375.1306100; Thu, 27 Feb 2025 02:19:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUe-0007wr-MH; Thu, 27 Feb 2025 02:19:20 +0000 Received: by outflank-mailman (input) for mailman id 897375; Thu, 27 Feb 2025 02:19:19 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUd-0005qU-A1 for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:19 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 3f2266a6-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:18 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fe8fdfdd94so1107722a91.0 for ; Wed, 26 Feb 2025 18:19:18 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3f2266a6-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622757; x=1741227557; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=xfyweVUhHjn5qLEM1jkmIOD/E67k0zFXQwB6BCCQHG8=; b=vH6LX4pc8jDNnYlm0y6WFbo5ENE3fg7aY4WY3XnGsbseK8A8a7Ki/9jVqGPL2NVGil mzTm9mcTazopeT1lUDrgR00ynR7FtMGm9Ohxf0VPSDr2HLXYVagvipqESwPO3QWc7n26 fIllSZ73iRDc4sFXFrsfWDEsWVssI+iEPJ8EaCRk6Zz+uhM0iIhl+91IuJMnh5PCBGk0 O8p7YCSpX0eupP8TIp0HPbBiNlGQ9dNnAQq0viE3mp7YQLzJ9dhwoSuaMXtowRFczsN7 YNY/ad7Kx/8DjYhBxzYD48n5I7jcB8/fqGLGyOgA+vG9kn51oFw+Dx0tukiNloTwaci4 2LJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622757; x=1741227557; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xfyweVUhHjn5qLEM1jkmIOD/E67k0zFXQwB6BCCQHG8=; b=maiteFCWC33pDdQ+TkHhG2d7nrDVPRlPt4+OoGD2+ykX9FaO6divq7cimmMBiTo5G/ Z4MyIHcI5OvpIVg82Caw/w5oZ8Zk88D+F3FkBf/byx7Tkb6NJEIwE+mF9dsykQwvHUhA QTIA/GssSHNrNOHpLSyavurigHKUQlyPS/CoBDaaem2F34GslwualWd+p9X3eXhJ6Voz 6RnfrOLcoZBenT+7tQ7lVrCO6lD0JkH7tWK4q5Z1nsCcouEbnKsOCUMAae9hz7fSEnLA ax3x7mQe7vVcjfBRvyV4X0OOA8kX5lciuGJfjz1UeBGaZ67InrNVpPEraV/FEN+oWUa7 pNKg== X-Forwarded-Encrypted: i=1; AJvYcCVyg6WrBMC+zHchBmFc8qFz3OvdUd/pcejIquQaZZs9g8xDyZyViVms1RJqD6PIqvJ69un0RkJxeb8=@lists.xenproject.org X-Gm-Message-State: AOJu0YzPfWdsyJMOgeeqwKi0h17aqgOHOMzCy2uOa2jVuR72z6wUf0bg 82EBDlVgSRTKdKOrESvUKc9J1M3qV/ra84/jMum6UHduy2M8WEERFqg6LEGaVYEfBijzNP5r3Xr PBA== X-Google-Smtp-Source: AGHT+IGK7T2uVtHCkzgg9idpaqI5CgjOHQ8lJUTz55dy8O8u0q+xXKgUvujnuayeBIrgMdHIgLk5Hrx+hJs= X-Received: from pjbta8.prod.google.com ([2002:a17:90b:4ec8:b0:2fa:1771:e276]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2d48:b0:2ee:d7d3:3019 with SMTP id 98e67ed59e1d1-2fe7e31f7c1mr10280973a91.12.1740622757149; Wed, 26 Feb 2025 18:19:17 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:24 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-9-seanjc@google.com> Subject: [PATCH v2 08/38] clocksource: hyper-v: Register sched_clock save/restore iff it's necessary From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Register the Hyper-V timer callbacks or saving/restoring its PV sched_clock if and only if the timer is actually being used for sched_clock. Currently, Hyper-V overrides the save/restore hooks if the reference TSC available, whereas the Hyper-V timer code only overrides sched_clock if the reference TSC is available *and* it's not invariant. The flaw is effectively papered over by invoking the "old" save/restore callbacks as part of save/restore, but that's unnecessary and fragile. To avoid introducing more complexity, and to allow for additional cleanups of the PV sched_clock code, move the save/restore hooks and logic into hyperv_timer.c and simply wire up the hooks when overriding sched_clock itself. Note, while the Hyper-V timer code is intended to be architecture neutral, CONFIG_PARAVIRT is firmly x86-only, i.e. adding a small amount of x86 specific code (which will be reduced in future cleanups) doesn't meaningfully pollute generic code. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/mshyperv.c | 58 ------------------------------ drivers/clocksource/hyperv_timer.c | 50 ++++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 58 deletions(-) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index aa60491bf738..174f6a71c899 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -223,63 +223,6 @@ static void hv_machine_crash_shutdown(struct pt_regs *regs) hyperv_cleanup(); } #endif /* CONFIG_CRASH_DUMP */ - -static u64 hv_ref_counter_at_suspend; -static void (*old_save_sched_clock_state)(void); -static void (*old_restore_sched_clock_state)(void); - -/* - * Hyper-V clock counter resets during hibernation. Save and restore clock - * offset during suspend/resume, while also considering the time passed - * before suspend. This is to make sure that sched_clock using hv tsc page - * based clocksource, proceeds from where it left off during suspend and - * it shows correct time for the timestamps of kernel messages after resume. - */ -static void save_hv_clock_tsc_state(void) -{ - hv_ref_counter_at_suspend = hv_read_reference_counter(); -} - -static void restore_hv_clock_tsc_state(void) -{ - /* - * Adjust the offsets used by hv tsc clocksource to - * account for the time spent before hibernation. - * adjusted value = reference counter (time) at suspend - * - reference counter (time) now. - */ - hv_adj_sched_clock_offset(hv_ref_counter_at_suspend - hv_read_reference_counter()); -} - -/* - * Functions to override save_sched_clock_state and restore_sched_clock_state - * functions of x86_platform. The Hyper-V clock counter is reset during - * suspend-resume and the offset used to measure time needs to be - * corrected, post resume. - */ -static void hv_save_sched_clock_state(void) -{ - old_save_sched_clock_state(); - save_hv_clock_tsc_state(); -} - -static void hv_restore_sched_clock_state(void) -{ - restore_hv_clock_tsc_state(); - old_restore_sched_clock_state(); -} - -static void __init x86_setup_ops_for_tsc_pg_clock(void) -{ - if (!(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE)) - return; - - old_save_sched_clock_state = x86_platform.save_sched_clock_state; - x86_platform.save_sched_clock_state = hv_save_sched_clock_state; - - old_restore_sched_clock_state = x86_platform.restore_sched_clock_state; - x86_platform.restore_sched_clock_state = hv_restore_sched_clock_state; -} #endif /* CONFIG_HYPERV */ static uint32_t __init ms_hyperv_platform(void) @@ -635,7 +578,6 @@ static void __init ms_hyperv_init_platform(void) /* Register Hyper-V specific clocksource */ hv_init_clocksource(); - x86_setup_ops_for_tsc_pg_clock(); hv_vtl_init_platform(); #endif /* diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index f00019b078a7..86a55167bf5d 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -534,10 +534,60 @@ static __always_inline void hv_setup_sched_clock(void *sched_clock) sched_clock_register(sched_clock, 64, NSEC_PER_SEC); } #elif defined CONFIG_PARAVIRT +static u64 hv_ref_counter_at_suspend; +static void (*old_save_sched_clock_state)(void); +static void (*old_restore_sched_clock_state)(void); + +/* + * Hyper-V clock counter resets during hibernation. Save and restore clock + * offset during suspend/resume, while also considering the time passed + * before suspend. This is to make sure that sched_clock using hv tsc page + * based clocksource, proceeds from where it left off during suspend and + * it shows correct time for the timestamps of kernel messages after resume. + */ +static void save_hv_clock_tsc_state(void) +{ + hv_ref_counter_at_suspend = hv_read_reference_counter(); +} + +static void restore_hv_clock_tsc_state(void) +{ + /* + * Adjust the offsets used by hv tsc clocksource to + * account for the time spent before hibernation. + * adjusted value = reference counter (time) at suspend + * - reference counter (time) now. + */ + hv_adj_sched_clock_offset(hv_ref_counter_at_suspend - hv_read_reference_counter()); +} +/* + * Functions to override save_sched_clock_state and restore_sched_clock_state + * functions of x86_platform. The Hyper-V clock counter is reset during + * suspend-resume and the offset used to measure time needs to be + * corrected, post resume. + */ +static void hv_save_sched_clock_state(void) +{ + old_save_sched_clock_state(); + save_hv_clock_tsc_state(); +} + +static void hv_restore_sched_clock_state(void) +{ + restore_hv_clock_tsc_state(); + old_restore_sched_clock_state(); +} + static __always_inline void hv_setup_sched_clock(void *sched_clock) { /* We're on x86/x64 *and* using PV ops */ paravirt_set_sched_clock(sched_clock); + + old_save_sched_clock_state = x86_platform.save_sched_clock_state; + x86_platform.save_sched_clock_state = hv_save_sched_clock_state; + + old_restore_sched_clock_state = x86_platform.restore_sched_clock_state; + x86_platform.restore_sched_clock_state = hv_restore_sched_clock_state; } #else /* !CONFIG_GENERIC_SCHED_CLOCK && !CONFIG_PARAVIRT */ static __always_inline void hv_setup_sched_clock(void *sched_clock) {} From patchwork Thu Feb 27 02:18:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993542 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 365FBC19776 for ; Thu, 27 Feb 2025 02:20:10 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897377.1306110 (Exim 4.92) (envelope-from ) id 1tnTUh-0008Jm-4x; Thu, 27 Feb 2025 02:19:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897377.1306110; Thu, 27 Feb 2025 02:19:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUh-0008JF-0s; Thu, 27 Feb 2025 02:19:23 +0000 Received: by outflank-mailman (input) for mailman id 897377; Thu, 27 Feb 2025 02:19:21 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUf-00063X-KE for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:21 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 400dc887-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:20 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc404aaed5so1539241a91.3 for ; Wed, 26 Feb 2025 18:19:20 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 400dc887-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622759; x=1741227559; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=M9J4vMOsFh+KFESDavQmkJXQ4sHD+OspEOtA7Gz9lOo=; b=kBMCtQTlzwT17dQYBNJWc9bKWsopHxZmCAeRB7bx9TAq5bLcxA7wtVi7ssglwJ52Bw lISBR+xA+YYBVd2hC5R0cMqKhbahYj3bxiQJyeqjNtBUnxv13eBH5vr4K9WsLf4q3qhF 1LmRAFd3dilGyJ6cj2cm76QiTHB89WJBXX/Wem0lST/iOzBOP1wMWQ9YM7j1KTI018VQ O032z3q4aEmnO7AB1ERh1uJbe/vB4ulQUZ1ao2d1FEH48mtoE62Eeui7+75ymDZ7srMY r24Hnqef5v35XzYjJscjLk8Z95EdCOEV1SnVlvH+DZtoouIvhoZKGjGFuhKmdUKjM317 rBCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622759; x=1741227559; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=M9J4vMOsFh+KFESDavQmkJXQ4sHD+OspEOtA7Gz9lOo=; b=TKaMTdRPzMN0W+N0l8OXoJpAPtMSSW81vPaRRLpUcW1St1B7RQPsmeB9Tc9AT6pDXh 8+EnwFba5I74ogBkBoyqBao9O9qvCS5Hqv+MpBt0o73lUgvUn4fXay8sfR9Gzk7j82x6 uS3QZXRtZxHqXWuX3Vaeiy9uU5L5xOWwMn2H04Cs2EmWAXRiADZX0assqQfwUQT2or5a 4Srzb9uEKpLlLXuJAFniw8mWMIsABLqOA0m8szI6libUprJ8lJwB4GWpCH5+D9Q7Dx4o L1+c8NWzmxkMaxb6GXTmyDdxGo2xZ6RqYLagTMhBXHvEY0tzt3oRhPOJz/AEhd6S3Ags YieQ== X-Forwarded-Encrypted: i=1; AJvYcCVVJKek9IGTpH/A+oW2F/N0RDCkZ1nZtGbF2DAxNe8Rq3cz2CaEDx4hKM6wSpO2irn3ko9RhJH9Lvw=@lists.xenproject.org X-Gm-Message-State: AOJu0YwzIUvkNIyVTov3L3KYHKlmetuRQdGqhMwnUKVfSRC/foNDvKl2 Ek+OsaLrTBTic8cZ/QYqLy+15Z4lJ3AQYOWFfhOzDDSK8MlOLU1hKcDUVA5QLyd2gkjzykS5xe0 Y5Q== X-Google-Smtp-Source: AGHT+IHGg2YNnpdN9W11waKGKjVdoMsbSztU341tlOE27g2wibRX2aXDt93sxEvHQRpF0p5gOf3qjCfl60c= X-Received: from pjbsn14.prod.google.com ([2002:a17:90b:2e8e:b0:2fc:15bf:92f6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2dc5:b0:2f6:be57:49d2 with SMTP id 98e67ed59e1d1-2fe7e32b7c8mr10195616a91.17.1740622758747; Wed, 26 Feb 2025 18:19:18 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:25 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-10-seanjc@google.com> Subject: [PATCH v2 09/38] clocksource: hyper-v: Drop wrappers to sched_clock save/restore helpers From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Now that all of the Hyper-V timer sched_clock code is located in a single file, drop the superfluous wrappers for the save/restore flows. No functional change intended. Signed-off-by: Sean Christopherson --- drivers/clocksource/hyperv_timer.c | 34 +++++------------------------- include/clocksource/hyperv_timer.h | 2 -- 2 files changed, 5 insertions(+), 31 deletions(-) diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index 86a55167bf5d..5a52d0acf31f 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -471,17 +471,6 @@ static void resume_hv_clock_tsc(struct clocksource *arg) hv_set_msr(HV_MSR_REFERENCE_TSC, tsc_msr.as_uint64); } -/* - * Called during resume from hibernation, from overridden - * x86_platform.restore_sched_clock_state routine. This is to adjust offsets - * used to calculate time for hv tsc page based sched_clock, to account for - * time spent before hibernation. - */ -void hv_adj_sched_clock_offset(u64 offset) -{ - hv_sched_clock_offset -= offset; -} - #ifdef HAVE_VDSO_CLOCKMODE_HVCLOCK static int hv_cs_enable(struct clocksource *cs) { @@ -545,12 +534,14 @@ static void (*old_restore_sched_clock_state)(void); * based clocksource, proceeds from where it left off during suspend and * it shows correct time for the timestamps of kernel messages after resume. */ -static void save_hv_clock_tsc_state(void) +static void hv_save_sched_clock_state(void) { + old_save_sched_clock_state(); + hv_ref_counter_at_suspend = hv_read_reference_counter(); } -static void restore_hv_clock_tsc_state(void) +static void hv_restore_sched_clock_state(void) { /* * Adjust the offsets used by hv tsc clocksource to @@ -558,23 +549,8 @@ static void restore_hv_clock_tsc_state(void) * adjusted value = reference counter (time) at suspend * - reference counter (time) now. */ - hv_adj_sched_clock_offset(hv_ref_counter_at_suspend - hv_read_reference_counter()); -} -/* - * Functions to override save_sched_clock_state and restore_sched_clock_state - * functions of x86_platform. The Hyper-V clock counter is reset during - * suspend-resume and the offset used to measure time needs to be - * corrected, post resume. - */ -static void hv_save_sched_clock_state(void) -{ - old_save_sched_clock_state(); - save_hv_clock_tsc_state(); -} + hv_sched_clock_offset -= (hv_ref_counter_at_suspend - hv_read_reference_counter()); -static void hv_restore_sched_clock_state(void) -{ - restore_hv_clock_tsc_state(); old_restore_sched_clock_state(); } diff --git a/include/clocksource/hyperv_timer.h b/include/clocksource/hyperv_timer.h index d48dd4176fd3..a4c81a60f53d 100644 --- a/include/clocksource/hyperv_timer.h +++ b/include/clocksource/hyperv_timer.h @@ -38,8 +38,6 @@ extern void hv_remap_tsc_clocksource(void); extern unsigned long hv_get_tsc_pfn(void); extern struct ms_hyperv_tsc_page *hv_get_tsc_page(void); -extern void hv_adj_sched_clock_offset(u64 offset); - static __always_inline bool hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg, u64 *cur_tsc, u64 *time) From patchwork Thu Feb 27 02:18:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 58968C19776 for ; Thu, 27 Feb 2025 02:20:14 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897379.1306116 (Exim 4.92) (envelope-from ) id 1tnTUi-0008SR-4i; Thu, 27 Feb 2025 02:19:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897379.1306116; Thu, 27 Feb 2025 02:19:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUh-0008Po-P7; Thu, 27 Feb 2025 02:19:23 +0000 Received: by outflank-mailman (input) for mailman id 897379; Thu, 27 Feb 2025 02:19:22 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUg-0005qU-G3 for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:22 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 411b7768-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:21 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc1c7c8396so1083061a91.0 for ; Wed, 26 Feb 2025 18:19:21 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 411b7768-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622760; x=1741227560; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=yCluOSGlRhOy+WE1mTtSxuryf3jXdB/g/GtrHZOBhNw=; b=jhj5R6Pbyj7HPRfMhXii6TDUEoQxKTkTia213p42enH2TU6SrbxrvsI1Kr0luv1aOO kfmdiLXWkzBlKmm+7hITQoX+kaahp0OnTJZLZKOxQ3b7o4E9FpuHSuU2/wzHDdYcgFbs MNnBemY9lHGhjCx4r9NAWLcpB5pzXMSvSnZ09+cKXpMHkul6p6RulEJ+wwejnGad6dVs VFkp+SgKCteIapK/MGhc01tNIQuj0s/jIWxUKoZg+s/k6Gx8JhM3lEWhhPq1jqxg1P1p FqAjE7KIYY+kOFGWmt1skLbuLh9o50Z0UD3D4AvNLqwxFm4kHTTlsp6WpQrVovenr6RU ZmWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622760; x=1741227560; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yCluOSGlRhOy+WE1mTtSxuryf3jXdB/g/GtrHZOBhNw=; b=bmDIic+sqSc4Wgsvrtvhhy157ar6IHejzrPkN0+9eR/o/dBhT6TEQymVCFbeKnD4N9 kj5rYsOzBLwq64j1tcRK9bSIjpBziqbdFEf/xWUAZpq4yrthqHwo/mnpka3E3WdYmEEn 4ZqCitvpFFtd8QK4gJ+wpYdm2Vhh9NaFo3kffu0I4JcbzW8qyfkqIfLMbHRmYllAaZfY Y8BQ7dhTP7AgFkSvtUxDkNifkzu/bk6YQ3iPqyvrJ2b8K9KWTqsWwsdd6lh1IKOGWurK qjaJB+KO2Y5bEBq1XZADs5Q81GR476ty2fOABYa6Y1sQzIevu6EzZmZQmPlbuLnzdW85 eYAg== X-Forwarded-Encrypted: i=1; AJvYcCUy1PnpSoyd1Jufdyq3E/diP5vusjGJ9iRBuoCa9Gwx0ublWYUqGL9smJsbgDSqR5Z2VD7BFZgdeqs=@lists.xenproject.org X-Gm-Message-State: AOJu0YwtGWnjhHDCGZmImwI7LWp4lG+EPkIRFfpltC9ZYdCzNRgyavsV G5cn27gHlgztwXAz3RgJ/sa6/ccS47z9884ZioFuHhbxq+5xbrJaq4Kj1D1i+wbbdxqmozIsp5z lHQ== X-Google-Smtp-Source: AGHT+IEg2qVczQMppIuK2WVUrcQpfIoWQRuX4zIj2kwQnn94y9fLsBQqATNYRMsqTAxp9OZvbzuNSvbgNPw= X-Received: from pgte20.prod.google.com ([2002:a65:6894:0:b0:ae1:49ef:10d4]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6300:8004:b0:1f0:e368:4a48 with SMTP id adf61e73a8af0-1f0e3684a6fmr22807696637.8.1740622760519; Wed, 26 Feb 2025 18:19:20 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:26 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-11-seanjc@google.com> Subject: [PATCH v2 10/38] clocksource: hyper-v: Don't save/restore TSC offset when using HV sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Now that Hyper-V overrides the sched_clock save/restore hooks if and only sched_clock itself is set to the Hyper-V timer, drop the invocation of the "old" save/restore callbacks. When the registration of the PV sched_clock was done separate from overriding the save/restore hooks, it was possible for Hyper-V to clobber the TSC save/restore callbacks without actually switching to the Hyper-V timer. Enabling a PV sched_clock is a one-way street, i.e. the kernel will never revert to using TSC for sched_clock, and so there is no need to invoke the TSC save/restore hooks (and if there was, it belongs in common PV code). Signed-off-by: Sean Christopherson --- drivers/clocksource/hyperv_timer.c | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index 5a52d0acf31f..4a21874e91b9 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -524,9 +524,6 @@ static __always_inline void hv_setup_sched_clock(void *sched_clock) } #elif defined CONFIG_PARAVIRT static u64 hv_ref_counter_at_suspend; -static void (*old_save_sched_clock_state)(void); -static void (*old_restore_sched_clock_state)(void); - /* * Hyper-V clock counter resets during hibernation. Save and restore clock * offset during suspend/resume, while also considering the time passed @@ -536,8 +533,6 @@ static void (*old_restore_sched_clock_state)(void); */ static void hv_save_sched_clock_state(void) { - old_save_sched_clock_state(); - hv_ref_counter_at_suspend = hv_read_reference_counter(); } @@ -550,8 +545,6 @@ static void hv_restore_sched_clock_state(void) * - reference counter (time) now. */ hv_sched_clock_offset -= (hv_ref_counter_at_suspend - hv_read_reference_counter()); - - old_restore_sched_clock_state(); } static __always_inline void hv_setup_sched_clock(void *sched_clock) @@ -559,10 +552,7 @@ static __always_inline void hv_setup_sched_clock(void *sched_clock) /* We're on x86/x64 *and* using PV ops */ paravirt_set_sched_clock(sched_clock); - old_save_sched_clock_state = x86_platform.save_sched_clock_state; x86_platform.save_sched_clock_state = hv_save_sched_clock_state; - - old_restore_sched_clock_state = x86_platform.restore_sched_clock_state; x86_platform.restore_sched_clock_state = hv_restore_sched_clock_state; } #else /* !CONFIG_GENERIC_SCHED_CLOCK && !CONFIG_PARAVIRT */ From patchwork Thu Feb 27 02:18:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993544 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A979CC19776 for ; Thu, 27 Feb 2025 02:20:17 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897383.1306131 (Exim 4.92) (envelope-from ) id 1tnTUk-0000bs-Gd; Thu, 27 Feb 2025 02:19:26 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897383.1306131; Thu, 27 Feb 2025 02:19:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUk-0000bB-6F; Thu, 27 Feb 2025 02:19:26 +0000 Received: by outflank-mailman (input) for mailman id 897383; Thu, 27 Feb 2025 02:19:24 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUi-0005qU-GS for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:24 +0000 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [2607:f8b0:4864:20::104a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 422f8eb7-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:23 +0100 (CET) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2fce2954a10so1535179a91.1 for ; Wed, 26 Feb 2025 18:19:23 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 422f8eb7-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622762; x=1741227562; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=yPGNLLFa/vXB2a0UM8EIYJpWd3IhBx8wcQJSj3RCAHc=; b=B/dey7IAOjVMKUiMDXw/wfJVZDWrZfZQUbhf/yuGxaTxUrlK9I56kfgXiWXt47VMGV ZAvKpaAQZOcABMA91ame9QQ6OJJA/XuzZjsbCI1bB2ABAzFYQaPAi7sjGvQ5mr9tIHQ6 5+5Y1LD3ayWjKF6nrpUMV3iPn75qh3bcDUGP4i0dSQtD6/IyTrZ8HGOaLVcU5oDMA8Z/ naJhyQEbHi/7m2WyEW6Hxx5Mro0EavAlcE97zYmuus5M9NJkHC0kdgw0gy+spkW1+cPv FA1VeI/WGMtt4Lnx5hQEFMhiCtxv6wNhZpX2P+bxgv7Vs5hddFXfti5Pr081zp1114pZ Zm6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622762; x=1741227562; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yPGNLLFa/vXB2a0UM8EIYJpWd3IhBx8wcQJSj3RCAHc=; b=AM/+8yvv1KMX+81tY5t8KeroiRBCr67YnyyrPL/pkR09NY4T9EIZh1uZLXuyqvf9JJ kXjxEpcKC+XWODsAaaswI+3Zdm8Etuk2XIfgmSGXi5qlHMLWohM2MLaisdC8upvmuus9 mIBTeUKnJ//nXjlfHYom2L5lXITIb8cFo/Py+lE2zQSlYKx01CHhRTyuPgouxsOcOkWX vTxQb3sEOHpB+RoUpGzgYtzGHyJY206V906rIXE6qCzOfgJnBlN5ziC/rkptkVyIsfUa 7qONIBsYh2YaH6hrbJF+TsUpULzGiHJHwZ2DtZG1xbZewsIsmfdr0JKY9qwBqyWLpRRc BbWg== X-Forwarded-Encrypted: i=1; AJvYcCXC+piLYOW4u2C6h9cPGnT998aSpfqIpBtUymA/ewptwtPJbu3t05GROrveDFLbwbzaQfhjhtX0l88=@lists.xenproject.org X-Gm-Message-State: AOJu0YwdUOUq00pNyBGf2FHSsrSgpT6RmYebG9DHRa34T0OsdcRJeR7t bXRCQUnUyfYgv3KG1Y0B73v4QgpSyvMjGZ42FqZH+Dgb9dQiKPUbbFkRwVXnhKTSU4eIbHKmnoR 3LQ== X-Google-Smtp-Source: AGHT+IGTG4bhtwk6xF9NkwSwRr4MZ0JMu1OW/uXbFyNTNSUBj4feLwfxPugt+I59qi4711mZsSVJuOvx1D0= X-Received: from pjbqn3.prod.google.com ([2002:a17:90b:3d43:b0:2ea:5469:76c2]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:fa14:b0:2fe:84d6:cdfc with SMTP id 98e67ed59e1d1-2fe84d6cf88mr6952534a91.35.1740622762299; Wed, 26 Feb 2025 18:19:22 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:27 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-12-seanjc@google.com> Subject: [PATCH v2 11/38] x86/kvmclock: Setup kvmclock for secondary CPUs iff CONFIG_SMP=y From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Gate kvmclock's secondary CPU code on CONFIG_SMP, not CONFIG_X86_LOCAL_APIC. Originally, kvmclock piggybacked PV APIC ops to setup secondary CPUs. When that wart was fixed by commit df156f90a0f9 ("x86: Introduce x86_cpuinit.early_percpu_clock_init hook"), the dependency on a local APIC got carried forward unnecessarily. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index b898b95a7d50..80d1a06609c8 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -186,7 +186,7 @@ static void kvm_restore_sched_clock_state(void) kvm_register_clock("primary cpu clock, resume"); } -#ifdef CONFIG_X86_LOCAL_APIC +#ifdef CONFIG_SMP static void kvm_setup_secondary_clock(void) { kvm_register_clock("secondary cpu clock"); @@ -324,7 +324,7 @@ void __init kvmclock_init(void) x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; -#ifdef CONFIG_X86_LOCAL_APIC +#ifdef CONFIG_SMP x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; #endif x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; From patchwork Thu Feb 27 02:18:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993618 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E437AC19F32 for ; Thu, 27 Feb 2025 02:26:21 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897536.1306314 (Exim 4.92) (envelope-from ) id 1tnTbL-0003yb-68; Thu, 27 Feb 2025 02:26:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897536.1306314; Thu, 27 Feb 2025 02:26:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbK-0003vF-HR; Thu, 27 Feb 2025 02:26:14 +0000 Received: by outflank-mailman (input) for mailman id 897536; Thu, 27 Feb 2025 02:26:12 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUl-00063X-3L for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:27 +0000 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [2607:f8b0:4864:20::649]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4332d8c7-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:25 +0100 (CET) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-220f0382404so8343695ad.1 for ; Wed, 26 Feb 2025 18:19:25 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4332d8c7-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622764; x=1741227564; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=pWgviwpmhKNHN5XYXpAWmYugB+1r1OlVvpsKGRVLlVs=; b=1cgV3pK4eenyNCaGDAOC/Stcw0hfyELiiY3mAK85SF5KPap1ePmkS7Av7MMEj1971C 0/Lo1LptvSlzBCJooYgahaZyXKO+d8l1G41C99nxvWlejyG5aOPVzSuOx1/Zne8Z8+hv F7SNrX2ueq9yux0daD3zPEB18DMCK7LfLshzY8LEp/fkfZrlG7j+12DJM5Nj1MFBUhGK JcorCzgTlePw9FMJnhzDKxNG2c2ugKjnF/25zQd5nAWLEeiooeYKOiEcfm68LD89ZOYT 70lrK5Du9ysZWvskw8/zdVBCLMt14QomZxJpmsGBi0WpC+zsYN4/BuNuUjVoi494CJxr iYMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622764; x=1741227564; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pWgviwpmhKNHN5XYXpAWmYugB+1r1OlVvpsKGRVLlVs=; b=ozw4cXFRXocNqE4iYXTGO3V8ZRTdjd8rac0iDa+osg7HI4eDByPdJJvaUgiTTP4PNF NvdLSZ3qwvMO29XDHpaANI45SLgf1OkYaP6lO1TY9sQl1F4jfN/zblw51bf9h/2PGqPx z4bgNBkYnuowcKm12AYLcEqv58DOLXljH0FO5EcXxDJhE17NvoBzYQO3Iau6w0dcO2zp 2BVTza1hIADN9SDPWKLiVcNEiywUkvbjPDAZMoU1RmN9EUEroy1fgZbs4PinrFT6/JF4 sfuiuV0hySxGzODGHRGoRUwhquCNPHjBjgHfJIKJFjvqJSmxH5HCXjSeC6w/Qp8365+6 9qkg== X-Forwarded-Encrypted: i=1; AJvYcCUP5pG15656jlsjP3AqUSlCUeBrgM6b9dBYXV5JybSHUf3J6DRpZ+mEreTXxbpV9qyc7gUdcwl1gTg=@lists.xenproject.org X-Gm-Message-State: AOJu0YwpnZ8DE/Cp+B4V3WmmdW1NzDCtKrTacqRORVpUYmkoo9JLZbQs XZOYadgXioX2WNJ4NpbYpUTnz64UY/748Y1ZNgnlY68+dwQhUtZWusDUk81jJ/L66n8eVhz4EUT Tvw== X-Google-Smtp-Source: AGHT+IGQazOm0zxmui59w45JCda9qTDjxRSvZp9ylt3tyNT0rDqeQIE65vrdsSLJOsVjaLh1NupfPRbt/dc= X-Received: from plblo7.prod.google.com ([2002:a17:903:4347:b0:21f:429a:36ae]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ecca:b0:220:f06b:318 with SMTP id d9443c01a7336-22320080b32mr91766195ad.14.1740622763961; Wed, 26 Feb 2025 18:19:23 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:28 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-13-seanjc@google.com> Subject: [PATCH v2 12/38] x86/kvm: Don't disable kvmclock on BSP in syscore_suspend() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Don't disable kvmclock on the BSP during syscore_suspend(), as the BSP's clock is NOT restored during syscore_resume(), but is instead restored earlier via the sched_clock restore callback. If suspend is aborted, e.g. due to a late wakeup, the BSP will run without its clock enabled, which "works" only because KVM-the-hypervisor is kind enough to not clobber the shared memory when the clock is disabled. But over time, the BSP's view of time will drift from APs. Plumb in an "action" to KVM-as-a-guest and kvmclock code in preparation for additional cleanups to kvmclock's suspend/resume logic. Fixes: c02027b5742b ("x86/kvm: Disable kvmclock on all CPUs on shutdown") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_para.h | 8 +++++++- arch/x86/kernel/kvm.c | 15 ++++++++------- arch/x86/kernel/kvmclock.c | 31 +++++++++++++++++++++++++------ 3 files changed, 40 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 57bc74e112f2..8708598f5b8e 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -118,8 +118,14 @@ static inline long kvm_sev_hypercall3(unsigned int nr, unsigned long p1, } #ifdef CONFIG_KVM_GUEST +enum kvm_guest_cpu_action { + KVM_GUEST_BSP_SUSPEND, + KVM_GUEST_AP_OFFLINE, + KVM_GUEST_SHUTDOWN, +}; + void kvmclock_init(void); -void kvmclock_disable(void); +void kvmclock_cpu_action(enum kvm_guest_cpu_action action); bool kvm_para_available(void); unsigned int kvm_arch_para_features(void); unsigned int kvm_arch_para_hints(void); diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 7a422a6c5983..866b061ee0d9 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -447,7 +447,7 @@ static void __init sev_map_percpu_data(void) } } -static void kvm_guest_cpu_offline(bool shutdown) +static void kvm_guest_cpu_offline(enum kvm_guest_cpu_action action) { kvm_disable_steal_time(); if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) @@ -455,9 +455,10 @@ static void kvm_guest_cpu_offline(bool shutdown) if (kvm_para_has_feature(KVM_FEATURE_MIGRATION_CONTROL)) wrmsrl(MSR_KVM_MIGRATION_CONTROL, 0); kvm_pv_disable_apf(); - if (!shutdown) + if (action != KVM_GUEST_SHUTDOWN) apf_task_wake_all(); - kvmclock_disable(); + + kvmclock_cpu_action(action); } static int kvm_cpu_online(unsigned int cpu) @@ -713,7 +714,7 @@ static int kvm_cpu_down_prepare(unsigned int cpu) unsigned long flags; local_irq_save(flags); - kvm_guest_cpu_offline(false); + kvm_guest_cpu_offline(KVM_GUEST_AP_OFFLINE); local_irq_restore(flags); return 0; } @@ -724,7 +725,7 @@ static int kvm_suspend(void) { u64 val = 0; - kvm_guest_cpu_offline(false); + kvm_guest_cpu_offline(KVM_GUEST_BSP_SUSPEND); #ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL if (kvm_para_has_feature(KVM_FEATURE_POLL_CONTROL)) @@ -751,7 +752,7 @@ static struct syscore_ops kvm_syscore_ops = { static void kvm_pv_guest_cpu_reboot(void *unused) { - kvm_guest_cpu_offline(true); + kvm_guest_cpu_offline(KVM_GUEST_SHUTDOWN); } static int kvm_pv_reboot_notify(struct notifier_block *nb, @@ -775,7 +776,7 @@ static struct notifier_block kvm_pv_reboot_nb = { #ifdef CONFIG_CRASH_DUMP static void kvm_crash_shutdown(struct pt_regs *regs) { - kvm_guest_cpu_offline(true); + kvm_guest_cpu_offline(KVM_GUEST_SHUTDOWN); native_machine_crash_shutdown(regs); } #endif diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 80d1a06609c8..223e5297f5ee 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -177,8 +177,22 @@ static void kvm_register_clock(char *txt) pr_debug("kvm-clock: cpu %d, msr %llx, %s", smp_processor_id(), pa, txt); } +static void kvmclock_disable(void) +{ + if (msr_kvm_system_time) + native_write_msr(msr_kvm_system_time, 0, 0); +} + static void kvm_save_sched_clock_state(void) { + /* + * Stop host writes to kvmclock immediately prior to suspend/hibernate. + * If the system is hibernating, then kvmclock will likely reside at a + * different physical address when the system awakens, and host writes + * to the old address prior to reconfiguring kvmclock would clobber + * random memory. + */ + kvmclock_disable(); } static void kvm_restore_sched_clock_state(void) @@ -186,6 +200,17 @@ static void kvm_restore_sched_clock_state(void) kvm_register_clock("primary cpu clock, resume"); } +void kvmclock_cpu_action(enum kvm_guest_cpu_action action) +{ + /* + * Don't disable kvmclock on the BSP during suspend. If kvmclock is + * being used for sched_clock, then it needs to be kept alive until the + * last minute, and restored as quickly as possible after resume. + */ + if (action != KVM_GUEST_BSP_SUSPEND) + kvmclock_disable(); +} + #ifdef CONFIG_SMP static void kvm_setup_secondary_clock(void) { @@ -193,12 +218,6 @@ static void kvm_setup_secondary_clock(void) } #endif -void kvmclock_disable(void) -{ - if (msr_kvm_system_time) - native_write_msr(msr_kvm_system_time, 0, 0); -} - static void __init kvmclock_init_mem(void) { unsigned long ncpus; From patchwork Thu Feb 27 02:18:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D9BC4C021B8 for ; Thu, 27 Feb 2025 02:25:59 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897477.1306204 (Exim 4.92) (envelope-from ) id 1tnTay-0007Ky-A2; Thu, 27 Feb 2025 02:25:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897477.1306204; Thu, 27 Feb 2025 02:25:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTay-0007KI-1U; Thu, 27 Feb 2025 02:25:52 +0000 Received: by outflank-mailman (input) for mailman id 897477; Thu, 27 Feb 2025 02:25:50 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUl-0005qU-SQ for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:27 +0000 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [2607:f8b0:4864:20::64a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 4446e037-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:27 +0100 (CET) Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-22112e86501so7658465ad.0 for ; Wed, 26 Feb 2025 18:19:27 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4446e037-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622766; x=1741227566; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=UfxMQF/rYWh5wJy03aYqaPrcV7f4BOaa6HmKd5c/EfY=; b=EFMtJS4FYa7sIRalJDUuePo0Imt2KKiYnomvPyDuhLo3o+a8JQUg665986TocuBNmd Xd8wFAaXyLujDLDnHGbBMav6mjooMvDq1Gz8ADTWKCy33RCxztTp/7lfQrrEpvU5VdR+ H/QGfXbxNnuVU87qns3fZt7fISZiiIztnnDUnNMUaVSPrVDrej30qoJy0IUY7vupKOA9 yibf23VW55iU1QJbDTOy0PT+GZYGnXcgEDd59Ofe01Tb7aQNbZSAXigstG8CK2IYnGzM aBm2wcXKvX6j72Y7SN0DthAlq+P+ex69KRg4JRzLa6PjNOGz1sslS4fOpcj33r5LMaJ3 TCXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622766; x=1741227566; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UfxMQF/rYWh5wJy03aYqaPrcV7f4BOaa6HmKd5c/EfY=; b=LcAeien9mQe3Hr/M1n1FAnnhj94kt6YlnJ28j3R/TWU0ngv2Kn2dtcvX1RT7bYiQcu womvNbSLwPJyM/HIDHY/7PV0lEJ2EjMMcUVRZJ2t3sj5ZyJgxlIvL2oENLk0dtjuK497 AVeEywGn0mbA00KLrj8UG1a28WpOF6CmoNZEKPcjYbQYbOwloYpABHbnHjj9LOvjIkqL bKdtYw0BNqirU4Vo4UUGpcaIx7OFzpi+9uKLPGAUy4HJF8nhFhjft9MroL7vEanDgmvC u3M6db0wNYmJj0wRyOlVXfigu/dWLogdip3v6hgWH7NZ38q8qqUH5ggkiJcFZ+L40hvR qb1A== X-Forwarded-Encrypted: i=1; AJvYcCVKs91PEu47u6OVUoxFOSgrsbKDz12SBoxuTv0SgKPF+dlI80Wdw3nGgp1juvIeQmFX4h9+Vu+N5h8=@lists.xenproject.org X-Gm-Message-State: AOJu0YxxvirgjXl6y1d4oGIfxnNBfv6PtXsZoFhsViOz32MgSaF1oGUE W5DaJq0ITiBdJjSqShh7/jWIIMrZecyqpiWcsBb61lgHK3AOWghyq9mJsq2NW5ytZhEzUv2/m4a PkA== X-Google-Smtp-Source: AGHT+IE7Xj/t7ND33DY7WpQVYCStd1wMVi/xBc5OTLhEqC2e0MCFeJOTyP9t5m80ApVOgGC3SmRws/tVvjE= X-Received: from pjuw3.prod.google.com ([2002:a17:90a:d603:b0:2fc:2b27:9d35]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ec8c:b0:21f:3e2d:7d2e with SMTP id d9443c01a7336-2219ffb8b65mr337766155ad.27.1740622765716; Wed, 26 Feb 2025 18:19:25 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:29 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-14-seanjc@google.com> Subject: [PATCH v2 13/38] x86/paravirt: Move handling of unstable PV clocks into paravirt_set_sched_clock() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move the handling of unstable PV clocks, of which kvmclock is the only example, into paravirt_set_sched_clock(). This will allow modifying paravirt_set_sched_clock() to keep using the TSC for sched_clock in certain scenarios without unintentionally marking the TSC-based clock as unstable. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 7 ++++++- arch/x86/kernel/kvmclock.c | 5 +---- arch/x86/kernel/paravirt.c | 6 +++++- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index 041aff51eb50..cfceabd5f7e1 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -28,7 +28,12 @@ u64 dummy_sched_clock(void); DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); -void paravirt_set_sched_clock(u64 (*func)(void)); +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable); + +static inline void paravirt_set_sched_clock(u64 (*func)(void)) +{ + __paravirt_set_sched_clock(func, true); +} static __always_inline u64 paravirt_sched_clock(void) { diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 223e5297f5ee..aae6fba21331 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -12,7 +12,6 @@ #include #include #include -#include #include #include #include @@ -93,10 +92,8 @@ static noinstr u64 kvm_sched_clock_read(void) static inline void kvm_sched_clock_init(bool stable) { - if (!stable) - clear_sched_clock_stable(); kvm_sched_clock_offset = kvm_clock_read(); - paravirt_set_sched_clock(kvm_sched_clock_read); + __paravirt_set_sched_clock(kvm_sched_clock_read, stable); pr_info("kvm-clock: using sched offset of %llu cycles", kvm_sched_clock_offset); diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 1ccaa3397a67..55c819673a9d 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -85,8 +86,11 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); -void paravirt_set_sched_clock(u64 (*func)(void)) +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable) { + if (!stable) + clear_sched_clock_stable(); + static_call_update(pv_sched_clock, func); } From patchwork Thu Feb 27 02:18:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A2FFC021B8 for ; Thu, 27 Feb 2025 02:26:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897578.1306330 (Exim 4.92) (envelope-from ) id 1tnTbW-0005uC-9o; Thu, 27 Feb 2025 02:26:26 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897578.1306330; Thu, 27 Feb 2025 02:26:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbW-0005u1-5b; Thu, 27 Feb 2025 02:26:26 +0000 Received: by outflank-mailman (input) for mailman id 897578; Thu, 27 Feb 2025 02:26:24 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUo-0005qU-Db for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:30 +0000 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [2607:f8b0:4864:20::104a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 45571ecb-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:29 +0100 (CET) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2fc43be27f8so1570831a91.1 for ; Wed, 26 Feb 2025 18:19:29 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 45571ecb-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622767; x=1741227567; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=cUIL6SYLHU+yTwrAtTegZ21noDNMwlKuhwHKs3VPMSg=; b=bpkIl56Hgd9ZObF8Ur3HqS1ANUSqe2kP2SO9AipsYonh45flBgG0KX0UAw11OxVeFY 4bG+UeElb7DtRs3fzp7UYCazs5af0x89OpwsQug7+V6/dQ0w6pBdFGpAbr1/ft5S8ooE wjUtSIqLCm5xVp07b2nUqlBdxa3lSRdCXx0/aEbj45U2SUUEzZLhoJUcF6zGAzCGViIg lPmoloxmwHXIav28bK9TwynRfAd0sCgBTxvsHd1WChlZZJwIC/zquHnJHUKdKzN3Tmn7 7jKtg5AseKvziOviX5HjXVAYHOh2bEnfNrFM+J8oEbNoH6QO2C2qVMGvRzfaa2c1YXX1 eNlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622767; x=1741227567; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cUIL6SYLHU+yTwrAtTegZ21noDNMwlKuhwHKs3VPMSg=; b=FytxsfMaWf6QA/eYpw3/bWRa2ghs9PPMz2807uEi5H7wsfh4+ac5Bl3SifYPOI8edq l+RGzmcdHS9DXgN7ThWNlGoQbgfOExjF3jwiPeDvzFaU8E/vujfCf8321CkdWOsq83ST 7m7bMvBz+rg8gLLXixvIPnPaJbc+6oxtalePCYMUrprUulMX6b6Rhnna4nxE5jp3zSGY QwEHOrS3S05uvRY85f8up32ImdRw2lrKZCYkv643/myQ4tavWXZvmQo3xpxHgbNOAtBL eeNgC0UnnTf16rOO55j+HCG0gHnYUhKfXZIGYfQd/zJy56h4j4pN+vKcGhATIb065Z5C +xEA== X-Forwarded-Encrypted: i=1; AJvYcCUxqThM6v+XYntyhT5y4xfY9JIX7EcBnLzCpBqZXIHeWAn8ghBhpVqVpfS8JvfcmKzc7aMZOfs3c/A=@lists.xenproject.org X-Gm-Message-State: AOJu0Ywu5ZrYr2p0S5gC3VLXXjGq9UjcdOB6UtobpN9ZMuBVsc0q/ryL e6ykpkrzgL61efVdSBCSHjZ/kN2XTe0hjJGW2TeNYgNJikaDOyXH3dQLj0w/ArBARSvU4zRCaD0 GBg== X-Google-Smtp-Source: AGHT+IHyh9LqedkCKW+Gv/tYRFdHEhi/V70ldbMXUa2l8chtlUNDSfd68q5Rf4dBrz6sq443DVBJXoiGfdY= X-Received: from pja6.prod.google.com ([2002:a17:90b:5486:b0:2ef:95f4:4619]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1d44:b0:2f6:f32e:90ac with SMTP id 98e67ed59e1d1-2fe7e2fe521mr9393579a91.11.1740622767576; Wed, 26 Feb 2025 18:19:27 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:30 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-15-seanjc@google.com> Subject: [PATCH v2 14/38] x86/kvmclock: Move sched_clock save/restore helpers up in kvmclock.c From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move kvmclock's sched_clock save/restore helper "up" so that they can (eventually) be referenced by kvm_sched_clock_init(). No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 108 ++++++++++++++++++------------------- 1 file changed, 54 insertions(+), 54 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index aae6fba21331..c78db52ae399 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -70,6 +70,25 @@ static int kvm_set_wallclock(const struct timespec64 *now) return -ENODEV; } +static void kvm_register_clock(char *txt) +{ + struct pvclock_vsyscall_time_info *src = this_cpu_hvclock(); + u64 pa; + + if (!src) + return; + + pa = slow_virt_to_phys(&src->pvti) | 0x01ULL; + wrmsrl(msr_kvm_system_time, pa); + pr_debug("kvm-clock: cpu %d, msr %llx, %s", smp_processor_id(), pa, txt); +} + +static void kvmclock_disable(void) +{ + if (msr_kvm_system_time) + native_write_msr(msr_kvm_system_time, 0, 0); +} + static u64 kvm_clock_read(void) { u64 ret; @@ -90,6 +109,30 @@ static noinstr u64 kvm_sched_clock_read(void) return pvclock_clocksource_read_nowd(this_cpu_pvti()) - kvm_sched_clock_offset; } +static void kvm_save_sched_clock_state(void) +{ + /* + * Stop host writes to kvmclock immediately prior to suspend/hibernate. + * If the system is hibernating, then kvmclock will likely reside at a + * different physical address when the system awakens, and host writes + * to the old address prior to reconfiguring kvmclock would clobber + * random memory. + */ + kvmclock_disable(); +} + +#ifdef CONFIG_SMP +static void kvm_setup_secondary_clock(void) +{ + kvm_register_clock("secondary cpu clock"); +} +#endif + +static void kvm_restore_sched_clock_state(void) +{ + kvm_register_clock("primary cpu clock, resume"); +} + static inline void kvm_sched_clock_init(bool stable) { kvm_sched_clock_offset = kvm_clock_read(); @@ -102,6 +145,17 @@ static inline void kvm_sched_clock_init(bool stable) sizeof(((struct pvclock_vcpu_time_info *)NULL)->system_time)); } +void kvmclock_cpu_action(enum kvm_guest_cpu_action action) +{ + /* + * Don't disable kvmclock on the BSP during suspend. If kvmclock is + * being used for sched_clock, then it needs to be kept alive until the + * last minute, and restored as quickly as possible after resume. + */ + if (action != KVM_GUEST_BSP_SUSPEND) + kvmclock_disable(); +} + /* * If we don't do that, there is the possibility that the guest * will calibrate under heavy load - thus, getting a lower lpj - @@ -161,60 +215,6 @@ static struct clocksource kvm_clock = { .enable = kvm_cs_enable, }; -static void kvm_register_clock(char *txt) -{ - struct pvclock_vsyscall_time_info *src = this_cpu_hvclock(); - u64 pa; - - if (!src) - return; - - pa = slow_virt_to_phys(&src->pvti) | 0x01ULL; - wrmsrl(msr_kvm_system_time, pa); - pr_debug("kvm-clock: cpu %d, msr %llx, %s", smp_processor_id(), pa, txt); -} - -static void kvmclock_disable(void) -{ - if (msr_kvm_system_time) - native_write_msr(msr_kvm_system_time, 0, 0); -} - -static void kvm_save_sched_clock_state(void) -{ - /* - * Stop host writes to kvmclock immediately prior to suspend/hibernate. - * If the system is hibernating, then kvmclock will likely reside at a - * different physical address when the system awakens, and host writes - * to the old address prior to reconfiguring kvmclock would clobber - * random memory. - */ - kvmclock_disable(); -} - -static void kvm_restore_sched_clock_state(void) -{ - kvm_register_clock("primary cpu clock, resume"); -} - -void kvmclock_cpu_action(enum kvm_guest_cpu_action action) -{ - /* - * Don't disable kvmclock on the BSP during suspend. If kvmclock is - * being used for sched_clock, then it needs to be kept alive until the - * last minute, and restored as quickly as possible after resume. - */ - if (action != KVM_GUEST_BSP_SUSPEND) - kvmclock_disable(); -} - -#ifdef CONFIG_SMP -static void kvm_setup_secondary_clock(void) -{ - kvm_register_clock("secondary cpu clock"); -} -#endif - static void __init kvmclock_init_mem(void) { unsigned long ncpus; From patchwork Thu Feb 27 02:18:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993608 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 353C5C1B087 for ; Thu, 27 Feb 2025 02:26:00 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897480.1306218 (Exim 4.92) (envelope-from ) id 1tnTaz-0007bq-FH; Thu, 27 Feb 2025 02:25:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897480.1306218; Thu, 27 Feb 2025 02:25:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTaz-0007aR-2l; Thu, 27 Feb 2025 02:25:53 +0000 Received: by outflank-mailman (input) for mailman id 897480; Thu, 27 Feb 2025 02:25:51 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUq-00063X-PA for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:32 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 465fdc50-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:30 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fe86c01f5cso1041713a91.1 for ; Wed, 26 Feb 2025 18:19:30 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 465fdc50-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622769; x=1741227569; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=4/yyaam3CnBmuwJ2ykq85CXFFo+ZsEjkVRvpD/uHncQ=; b=YcwkwIIleAKRlntExHb9vQgLgnTyzuYYEhkAugBFro7K6ktnQHomqCtjoqq339OEf/ xXktGCrfDmlg99Z79gNb6wRs7HsAPppONnMiM5e4CJLc70zeZqBLObjAQRKCgSRHa1nV DhDZA8NwP2xpqn4ogBTtpYguqs4eiCAc2L5qB1y4z6j3o8w7q9JMhYFX5djnNHtEau98 o02ezOULqbNmsoTAD2BmJ/4UN4+oLoNhJBm0vPi3gYRyV02lC3Lmk9tJGkWT00EW1MfS OSftaM63qHWzC876At4YLYtU6W33N4/v8xUNv2UJNEhn8veTHVrZFOI8tJqD+1oBjICa Jg5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622769; x=1741227569; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4/yyaam3CnBmuwJ2ykq85CXFFo+ZsEjkVRvpD/uHncQ=; b=SoMTCGwN/VCQ7BxaXOcFnGi5C3XNxQvZUNOFYM0nJEIYvVg9na7TItHhtmQqcjmcz9 /+a1u4b16oalLGZuH5KFDAM875yGLWEhKAUyct30hj+58KyqJuIzaK0/803PjafkrwcD Xs+JgSuy2oMbFLk8eoeZ2XdpC2wZsq+D+cYwWKIlGsJaTYPUhAYL6L4Tt7ANiRU4UmMf roQO2zlFS8wpWCZsOrvQk42YbdoVGsOowt3cm9l8zl7dtoa8ZgxXYwFqe0bmuPikO366 QvewKXfQQUZKPdW2bJfA9TaAJ7r4LYWlk5QDgjjSM8EixZjWYX0ANsSlVFEV5xaytwKQ isrw== X-Forwarded-Encrypted: i=1; AJvYcCWlqKTSWnpvZre4HSViseA9HmK3w/G40uffa1Kg/8RK+YEaUIArS2SJAaK14lMrD5W8zpBGLwdBqd4=@lists.xenproject.org X-Gm-Message-State: AOJu0Yw1FzlpHpTFJU7+7y7hzPeu5oIoPKu7tJCmOotN25pTy+sZ7F/V KDhEO6ayNKMLA/dUlgwJrcQ19UoKRD4h+xNcf+V3yN6fbfmSW6ms0ht+rkv+vfpyily4Du1nvZ8 Uig== X-Google-Smtp-Source: AGHT+IHB6t2yNhuRRLSjnQcCsiQss9r1P7c1nIVM64h4xrkl38FU8N4jisiF+hTwqGE4B/+REyTZDinhF2Y= X-Received: from pjbqb10.prod.google.com ([2002:a17:90b:280a:b0:2e0:915d:d594]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:280c:b0:2ee:5bc9:75c7 with SMTP id 98e67ed59e1d1-2fe68ac9330mr14041142a91.5.1740622769336; Wed, 26 Feb 2025 18:19:29 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:31 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-16-seanjc@google.com> Subject: [PATCH v2 15/38] x86/xen/time: Nullify x86_platform's sched_clock save/restore hooks From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Nullify the x86_platform sched_clock save/restore hooks when setting up Xen's PV clock to make it somewhat obvious the hooks aren't used when running as a Xen guest (Xen uses a paravirtualized suspend/resume flow). Signed-off-by: Sean Christopherson --- arch/x86/xen/time.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 9e2e900dc0c7..51eba986cd18 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -565,6 +565,12 @@ static void __init xen_init_time_common(void) xen_sched_clock_offset = xen_clocksource_read(); static_call_update(pv_steal_clock, xen_steal_clock); paravirt_set_sched_clock(xen_sched_clock); + /* + * Xen has paravirtualized suspend/resume and so doesn't use the common + * x86 sched_clock save/restore hooks. + */ + x86_platform.save_sched_clock_state = NULL; + x86_platform.restore_sched_clock_state = NULL; tsc_register_calibration_routines(xen_tsc_khz, NULL); x86_platform.get_wallclock = xen_get_wallclock; From patchwork Thu Feb 27 02:18:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5476C19776 for ; Thu, 27 Feb 2025 02:25:59 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897479.1306210 (Exim 4.92) (envelope-from ) id 1tnTay-0007Rf-Rw; Thu, 27 Feb 2025 02:25:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897479.1306210; Thu, 27 Feb 2025 02:25:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTay-0007O3-HM; Thu, 27 Feb 2025 02:25:52 +0000 Received: by outflank-mailman (input) for mailman id 897479; Thu, 27 Feb 2025 02:25:51 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUv-0005qU-Ea for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:37 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 47695b87-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:32 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2f83e54432dso1545883a91.2 for ; Wed, 26 Feb 2025 18:19:32 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 47695b87-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622771; x=1741227571; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Cdk6AVEMbYJ/6Avsp+2Egdk5E71fbrI/hFT7BN4L+X8=; b=G8u70gZaPECovz6o4nyxJ1rz3iX3M5lp0QvQrtQC0eQ/14Zp9tuACIv3X5H56crdha OyHTOHhzCXOFvy6Wc4/O3k9sbqmAPn5GoeuIPKZp0HVdSnWhk8nyVjxfGokW2Jis142P FMNZMqqm/WAsVTALccrz4+QAnr/t8PGm3uN65itcyu4WPNv7NfOfIWf2iEdAnfrEBN/+ 2eo/AjDrwFvjfunxC3djxb0nhKPpZKkxabmmHdf9UBpbumXuggVmqM5X9iuwkvHwTRGL 3SCnJuIQVx1eWKvtd/Nks4pxu2UNOGkZds/oB+yDGnmIrMP4ZHcH7sFEYt2pe6FDm2gM S55Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622771; x=1741227571; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Cdk6AVEMbYJ/6Avsp+2Egdk5E71fbrI/hFT7BN4L+X8=; b=lDQ1ShDdngC2b7vajFdls7gpkzEfV19yyyEYNko0hGLJTSOy28F5gupNBuOXQqTWL4 5eQ1tM9RujYsQf8Fc5a2rG2opAPvknoDTe4OBfDNT5z/8499ux6AwMfadw4PWdDCecQA 4RXbtqx0vyt/ON4Sglkw8im/Gxw3BeZ/K97dKlHwOUQwWWRPZLjuRqR4jl1K0lzLFCcu lxAZTxYI9cguB8ZTqW1YRXbpfS8xA8U1BnK5mRfpYZeNXQOKiyxw6GqBL2IkzjSkCQXs JqMB4P1l0G/+RdocKKCJAUG75JGq8welI/zE/jRlsGia7H+UQeT0QexOZINsu3KCqYG2 sT2w== X-Forwarded-Encrypted: i=1; AJvYcCUCU1FbuqPt7I8u+K0dHWBfp/ia4jxPUJhiXY4NlfVbhlwGWOou+iEBoZEgJHXoyoiu1DKOqwR8/+U=@lists.xenproject.org X-Gm-Message-State: AOJu0YzoYLrIN/4AcZ+rTlyjnj2YGyrvJm9OsioMI0JClRr8Z4X0b8RQ gvohLLfcGEArITKlsAQ8JCjIBqHFh2LTBpE1qN+A6r3OrGxboDfyCNh8GFPTS+/gx4+H952CaKt d8w== X-Google-Smtp-Source: AGHT+IGFILkiLdlR4RAAXuJxcaVfe/0bxh03RUlKL7+1zFXEwiB1qD3NnSdVhpPB8u60HxzWl62A5c233Wg= X-Received: from pjbdb4.prod.google.com ([2002:a17:90a:d644:b0:2fc:2ee0:d385]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5688:b0:2ee:a4f2:b307 with SMTP id 98e67ed59e1d1-2fe7e2eaca6mr8295061a91.4.1740622771119; Wed, 26 Feb 2025 18:19:31 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:32 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-17-seanjc@google.com> Subject: [PATCH v2 16/38] x86/vmware: Nullify save/restore hooks when using VMware's sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Nullify the sched_clock save/restore hooks when using VMware's version of sched_clock. This will allow extending paravirt_set_sched_clock() to set the save/restore hooks, without having to simultaneously change the behavior of VMware guests. Note, it's not at all obvious that it's safe/correct for VMware guests to do nothing on suspend/resume, but that's a pre-existing problem. Leave it for a VMware expert to sort out. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/vmware.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index d6f079a75f05..d6eadb5b37fd 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -344,8 +344,11 @@ static void __init vmware_paravirt_ops_setup(void) vmware_cyc2ns_setup(); - if (vmw_sched_clock) + if (vmw_sched_clock) { paravirt_set_sched_clock(vmware_sched_clock); + x86_platform.save_sched_clock_state = NULL; + x86_platform.restore_sched_clock_state = NULL; + } if (vmware_is_stealclock_available()) { has_steal_clock = true; From patchwork Thu Feb 27 02:18:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9CDF7C19776 for ; Thu, 27 Feb 2025 02:26:52 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897612.1306345 (Exim 4.92) (envelope-from ) id 1tnTbn-0007XD-VM; Thu, 27 Feb 2025 02:26:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897612.1306345; Thu, 27 Feb 2025 02:26:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbn-0007WI-Rh; Thu, 27 Feb 2025 02:26:43 +0000 Received: by outflank-mailman (input) for mailman id 897612; Thu, 27 Feb 2025 02:26:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUt-00063X-PE for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:35 +0000 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [2607:f8b0:4864:20::649]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4875acc8-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:34 +0100 (CET) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-2210305535bso12713345ad.1 for ; Wed, 26 Feb 2025 18:19:34 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4875acc8-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622773; x=1741227573; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=iUPGjlm6EABKAwlimnT9Yj0uy9Y0kc3vikfZk6zMyUM=; b=wGmFhVUxZ15qHzYCAZmewBLmyp/Eu581w701AgHZPUQ9+Ww1W8Z9G5ML8wo102ZXt4 jY6UeZtB9By2V0udvxnTJhLohBxBLUhFZdcxagixbd8Jd9/0jw+1BZvZEFnyYf5AbeXq 80Lu5n0NyQzRv5eMKejbaiAFY2mP1zCD2COhtnWXxPA67bBSQlMNtvqsPVatp/vARKZ6 PjJ50bDM1FlEsQSjdQ4t+/ZWzUPFjzg7g/wGHHF6WheIRK67G+xq04qkiYaxOVe7cxn/ tYz1Embx8Vp0XSd5DOeLkGHu+EWCT7LCBU3ctZ6+Gb2FRqqvZWXLLOLaKlWFI799Ocfn 8TSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622773; x=1741227573; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=iUPGjlm6EABKAwlimnT9Yj0uy9Y0kc3vikfZk6zMyUM=; b=Q7cCoO7F6WTb00wuAf2jGkIC9BAceERHmXMzprKSgwE4c60sBxP0S5GxGbYo91Rh3f 1/a1jRWWxS7YVe3r6L0WVy/PpC0ouO89Ck9YFLJk4PvRIviQJiH1n7mEFL6/E9J6AZFo 6GqUHm2h0h8rgAiz3hK3zexD6bR5YSeLEwNz4bpRGuj9bKTLKs6XZ5eUxJU8XXF3IsOZ 5DjID9oZObsm5Ngqn43qeWcuhc4ngxOfSirStSLyh10OYaavz9MbISphZp0iPpfI+8CA pYj5DJV0VsRvOf569JaPWRwf/x5VhkqTI0ZxhvKAI3xgVpkDBHGO1S0vB/+KwKC1TScI DE5w== X-Forwarded-Encrypted: i=1; AJvYcCVGzvlAiuvkUUxyqY3HXwMVfpWIcs0ClEnE0m25PQddAETeseaIm9N7S+wopFMH6NOcdhqv0pksSj0=@lists.xenproject.org X-Gm-Message-State: AOJu0Yyor9GNoGTolafLCs/DYYq/iFOAbOZ+um7A96khEvaPgoO/cae3 XT1gM8dR7ZLQmDUBvqDXcnRZWXbd3rs0AFLhyXO+0HDq4LNpOa2WEkHKsvMarReywBmRLY1s2OG 85A== X-Google-Smtp-Source: AGHT+IESbVD77xvo7C+ciGzHtrGqGL0Rj+DzLIK2/TaODo5QawXajXsWSJTUc7OSugj7U0H5GgKBaGLNEhI= X-Received: from pfld12.prod.google.com ([2002:a05:6a00:198c:b0:732:7e28:8f7d]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1813:b0:732:6221:7180 with SMTP id d2e1a72fcca58-73426c943c4mr38161657b3a.5.1740622772828; Wed, 26 Feb 2025 18:19:32 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:33 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-18-seanjc@google.com> Subject: [PATCH v2 17/38] x86/tsc: WARN if TSC sched_clock save/restore used with PV sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Now that all PV clocksources override the sched_clock save/restore hooks when overriding sched_clock, WARN if the "default" TSC hooks are invoked when using a PV sched_clock, e.g. to guard against regressions. Signed-off-by: Sean Christopherson --- arch/x86/kernel/tsc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 472d6c71d3a5..5501d76243c8 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -990,7 +990,7 @@ static unsigned long long cyc2ns_suspend; void tsc_save_sched_clock_state(void) { - if (!sched_clock_stable()) + if (!sched_clock_stable() || WARN_ON_ONCE(!using_native_sched_clock())) return; cyc2ns_suspend = sched_clock(); @@ -1010,7 +1010,7 @@ void tsc_restore_sched_clock_state(void) unsigned long flags; int cpu; - if (!sched_clock_stable()) + if (!sched_clock_stable() || WARN_ON_ONCE(!using_native_sched_clock())) return; local_irq_save(flags); From patchwork Thu Feb 27 02:18:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6AE7CC021B8 for ; Thu, 27 Feb 2025 02:26:58 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897625.1306380 (Exim 4.92) (envelope-from ) id 1tnTbv-0000To-QT; Thu, 27 Feb 2025 02:26:51 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897625.1306380; Thu, 27 Feb 2025 02:26:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbv-0000Te-M5; Thu, 27 Feb 2025 02:26:51 +0000 Received: by outflank-mailman (input) for mailman id 897625; Thu, 27 Feb 2025 02:26:50 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTUv-00063X-EP for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:37 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 496bb26a-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:35 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2f2a9f056a8so981926a91.2 for ; Wed, 26 Feb 2025 18:19:35 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 496bb26a-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622774; x=1741227574; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=+aPHxZSdvUlWxMuV2uSxypL8fcuaMUyvD+y52Dh+Bv8=; b=kjDYok3pJb1MNFl2cwY+zlq7YQhE6taTODPI0KST9ttvSlTFVbeSk/QHQ4HjD8GhaE IOQy/V1yo2q9atYR3iLyI3AcWU+OechjrgC7mHdQZdNML5idoY3tHqhrUgdnIm88/MBx cd/EvKQnLwYLol3PpyDHIegq3x3rITBg+Vx9K79XcoqhoLRxNWt2Tr+O+wn9TIQFtT1J /AuNz/0wNA0gEVLd3BfRKwcHthtXW99MkzErDBsRni4vF6wDTmvnbuVoMx8H1hIeYmqU FZNmDi3wqaWxVUbDS445hFyE6wbAxP27UqirLTNlJXTd/0BtiD+L5tN1Op8uE0k+A4jD uoug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622774; x=1741227574; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+aPHxZSdvUlWxMuV2uSxypL8fcuaMUyvD+y52Dh+Bv8=; b=j+t22CqbERUtWdFGH6suI8J8uuftmrKAN3VEGOvgb3wCpMEB3+8Qc4PoekiSb8BjUo +nRrBEW9Ioxu9aEyfBetOzWxT7ZPzJFlCztGYpqS9M/Y7NsDbrS9CxsQSaussWXDKpgd JpOVr3TWf5btrRTMX5kPGrw0BTCH6faBD/TxLRzd+eja08lFdOmZveBnci6Ko9R1k1z8 LwoGCYsGVj/bMMGQrcYfTC6LiUc4QGlUXrSHiP+8ES9PHerADQa+IobJUdkjwqenNDBf 1Us6Wu/lB5T8gTMQIVCM0Zc6bfMMEj3jA0eznpyCBYXgasKH3QhA3++5DLG9FrindFwr B38g== X-Forwarded-Encrypted: i=1; AJvYcCVKk/+HucoMj8d9+ZHeYLDyYCi/d/KQDtKhbD6S1LPaxBWeD3uVDSYl9VbzHmOSf1K+KBjc7Py7MRU=@lists.xenproject.org X-Gm-Message-State: AOJu0Yw+cvfA+sHCLanlYTlJhawoJElxpc0gSSCL5Ypg4zKJwv+XVGF7 kJwzyFttjiXnupXYpkAK3sWbzX0QZ8aaUmaL/1FZ2g6dsBGkt8rBcKmskNZ5CuqBagWckr8cCUv 8bA== X-Google-Smtp-Source: AGHT+IGtJhSaBP2AlM0Njvk3g8X/xMhIC1fqVJrfdIV8pKqmQJT6ad+58HkgFke1VLmpWg41h8WiiOdBO04= X-Received: from pjbta3.prod.google.com ([2002:a17:90b:4ec3:b0:2ea:448a:8cd1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5483:b0:2ee:f076:20f1 with SMTP id 98e67ed59e1d1-2fe7e218ab9mr10711632a91.0.1740622774461; Wed, 26 Feb 2025 18:19:34 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:34 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-19-seanjc@google.com> Subject: [PATCH v2 18/38] x86/paravirt: Pass sched_clock save/restore helpers during registration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Pass in a PV clock's save/restore helpers when configuring sched_clock instead of relying on each PV clock to manually set the save/restore hooks. In addition to bringing sanity to the code, this will allow gracefully "rejecting" a PV sched_clock, e.g. when running as a CoCo guest that has access to a "secure" TSC. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 8 +++++--- arch/x86/kernel/cpu/vmware.c | 7 ++----- arch/x86/kernel/kvmclock.c | 5 ++--- arch/x86/kernel/paravirt.c | 5 ++++- arch/x86/xen/time.c | 5 ++--- drivers/clocksource/hyperv_timer.c | 6 ++---- 6 files changed, 17 insertions(+), 19 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index cfceabd5f7e1..dc26a3c26527 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -28,11 +28,13 @@ u64 dummy_sched_clock(void); DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); -void __paravirt_set_sched_clock(u64 (*func)(void), bool stable); +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)); -static inline void paravirt_set_sched_clock(u64 (*func)(void)) +static inline void paravirt_set_sched_clock(u64 (*func)(void), + void (*save)(void), void (*restore)(void)) { - __paravirt_set_sched_clock(func, true); + __paravirt_set_sched_clock(func, true, save, restore); } static __always_inline u64 paravirt_sched_clock(void) diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index d6eadb5b37fd..399cf3286a60 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -344,11 +344,8 @@ static void __init vmware_paravirt_ops_setup(void) vmware_cyc2ns_setup(); - if (vmw_sched_clock) { - paravirt_set_sched_clock(vmware_sched_clock); - x86_platform.save_sched_clock_state = NULL; - x86_platform.restore_sched_clock_state = NULL; - } + if (vmw_sched_clock) + paravirt_set_sched_clock(vmware_sched_clock, NULL, NULL); if (vmware_is_stealclock_available()) { has_steal_clock = true; diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index c78db52ae399..1ad3878cc1d9 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -136,7 +136,8 @@ static void kvm_restore_sched_clock_state(void) static inline void kvm_sched_clock_init(bool stable) { kvm_sched_clock_offset = kvm_clock_read(); - __paravirt_set_sched_clock(kvm_sched_clock_read, stable); + __paravirt_set_sched_clock(kvm_sched_clock_read, stable, + kvm_save_sched_clock_state, kvm_restore_sched_clock_state); pr_info("kvm-clock: using sched offset of %llu cycles", kvm_sched_clock_offset); @@ -343,8 +344,6 @@ void __init kvmclock_init(void) #ifdef CONFIG_SMP x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; #endif - x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; - x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; kvm_get_preset_lpj(); /* diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 55c819673a9d..9673cd3a3f0a 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -86,12 +86,15 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); -void __paravirt_set_sched_clock(u64 (*func)(void), bool stable) +void __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)) { if (!stable) clear_sched_clock_stable(); static_call_update(pv_sched_clock, func); + x86_platform.save_sched_clock_state = save; + x86_platform.restore_sched_clock_state = restore; } /* These are in entry.S */ diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 51eba986cd18..3179f850352d 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -564,13 +564,12 @@ static void __init xen_init_time_common(void) { xen_sched_clock_offset = xen_clocksource_read(); static_call_update(pv_steal_clock, xen_steal_clock); - paravirt_set_sched_clock(xen_sched_clock); + /* * Xen has paravirtualized suspend/resume and so doesn't use the common * x86 sched_clock save/restore hooks. */ - x86_platform.save_sched_clock_state = NULL; - x86_platform.restore_sched_clock_state = NULL; + paravirt_set_sched_clock(xen_sched_clock, NULL, NULL); tsc_register_calibration_routines(xen_tsc_khz, NULL); x86_platform.get_wallclock = xen_get_wallclock; diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index 4a21874e91b9..1c4ed9995cb2 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -550,10 +550,8 @@ static void hv_restore_sched_clock_state(void) static __always_inline void hv_setup_sched_clock(void *sched_clock) { /* We're on x86/x64 *and* using PV ops */ - paravirt_set_sched_clock(sched_clock); - - x86_platform.save_sched_clock_state = hv_save_sched_clock_state; - x86_platform.restore_sched_clock_state = hv_restore_sched_clock_state; + paravirt_set_sched_clock(sched_clock, hv_save_sched_clock_state, + hv_restore_sched_clock_state); } #else /* !CONFIG_GENERIC_SCHED_CLOCK && !CONFIG_PARAVIRT */ static __always_inline void hv_setup_sched_clock(void *sched_clock) {} From patchwork Thu Feb 27 02:18:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63370C19F32 for ; Thu, 27 Feb 2025 02:26:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897617.1306360 (Exim 4.92) (envelope-from ) id 1tnTbr-00088g-70; Thu, 27 Feb 2025 02:26:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897617.1306360; Thu, 27 Feb 2025 02:26:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbr-00088D-33; Thu, 27 Feb 2025 02:26:47 +0000 Received: by outflank-mailman (input) for mailman id 897617; Thu, 27 Feb 2025 02:26:46 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTV0-0005qU-FF for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:42 +0000 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [2607:f8b0:4864:20::649]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 4a7ff58a-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:37 +0100 (CET) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-22126a488d7so8276585ad.2 for ; Wed, 26 Feb 2025 18:19:37 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4a7ff58a-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622776; x=1741227576; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DqrQU03sQXOQDWLkbiIYCQHYscRa6LsfmxFd/y9lSQI=; b=MOrzuNjphMzLzWSAxt0FRr6Sn3t+yVH7K4wk8nTVo7tWGA01Hm6lhfbx4wWibVW66p eqJRe3e0cJ+G6yxSQDIJzvy+zrETBYefayFgLcqdf4hOLXe9cO6OMmb0e6fFKQ5fJa5c AzfqdIkSi4OvsEMVhn0wNZY54+BLFP5yuHLIbI0HJ3Kyj4mM4xFRJYeNQvOji5mHnO/i i5K8QviHztECCMU7MVZvH/LcHsfRzgq2VtmuOhIpHVvwtRG5HE4SQdH07lcDfB50KHoC cse6wFvjMKUAho0jQHljr5H/JyzyjDdvn4eCMI0SdjvqnV8Ue1obK45ooOLLwfmSJB+G 3y9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622776; x=1741227576; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DqrQU03sQXOQDWLkbiIYCQHYscRa6LsfmxFd/y9lSQI=; b=Zf7v//qKeQ6WEJ431ZQWhNYYxZY7p1GfEi1Ff3JvVbw2gRAOmzq2AKYuBvwse2ZSHu Xm65/caY+/xG+BCHMaFpzQNab0gmBm9U97qjQJuyu8un0sCAF7C4XVh2MsPMa8fP5/dY jmVOLcJsqoi600A5Z08+9shmHxS2oRUZV33erseE/FTGFbgwHvw1wdOUL269JSuIQqf3 /nmk+Cbpinafepj+O8KirIs/uXJmguS56USlwjqVwMNOfYn9ogGQY29aaLl1NNyqzOGZ lESKB2oITBRvo9UTy8xauZaqGqav7YmGGKcq//zbx7UQ3f3462NyhVvGp22NrFgR7bkV 39vA== X-Forwarded-Encrypted: i=1; AJvYcCXHWuKMkmYDSd5qyKKobjUfM0Hjbo59hwudRXBu9Q71rH4oj+Pvy95UhCSHS6+HNx46d7PDFgqC6iE=@lists.xenproject.org X-Gm-Message-State: AOJu0YwtLtmMTnaGVa4xUTi+17oQKmo2fRETsXg2QtwtSi5fsEE7vj2I WTLzbprDV1JA40UftzV6ITX0PbgaxhcNQPXx+Mq45ScekvfvPBn+H1Ub7gsDdTPvfSOJ6JpiRo3 Vwg== X-Google-Smtp-Source: AGHT+IEOb0ihEjgZTJgsK50piH41vKgvY42fSlJ0VND1RngWC+UvT9KhzxC1r0flylu86Z/N4Su58D1alpU= X-Received: from pllg7.prod.google.com ([2002:a17:902:7407:b0:223:2747:3d22]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:d58e:b0:21f:140e:2929 with SMTP id d9443c01a7336-22307b52eb5mr158736135ad.15.1740622776273; Wed, 26 Feb 2025 18:19:36 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:35 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-20-seanjc@google.com> Subject: [PATCH v2 19/38] x86/kvmclock: Move kvm_sched_clock_init() down in kvmclock.c From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Move kvm_sched_clock_init() "down" so that it can reference the global kvm_clock structure without needing a forward declaration. Opportunistically mark the helper as "__init" instead of "inline" to make its usage more obvious; modern compilers don't need a hint to inline a single-use function, and an extra CALL+RET pair during boot is a complete non-issue. And, if the compiler ignores the hint and does NOT inline the function, the resulting code may not get discarded after boot due lack of an __init annotation. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 1ad3878cc1d9..934ee4a4c6d4 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -133,19 +133,6 @@ static void kvm_restore_sched_clock_state(void) kvm_register_clock("primary cpu clock, resume"); } -static inline void kvm_sched_clock_init(bool stable) -{ - kvm_sched_clock_offset = kvm_clock_read(); - __paravirt_set_sched_clock(kvm_sched_clock_read, stable, - kvm_save_sched_clock_state, kvm_restore_sched_clock_state); - - pr_info("kvm-clock: using sched offset of %llu cycles", - kvm_sched_clock_offset); - - BUILD_BUG_ON(sizeof(kvm_sched_clock_offset) > - sizeof(((struct pvclock_vcpu_time_info *)NULL)->system_time)); -} - void kvmclock_cpu_action(enum kvm_guest_cpu_action action) { /* @@ -302,6 +289,19 @@ static int kvmclock_setup_percpu(unsigned int cpu) return p ? 0 : -ENOMEM; } +static void __init kvm_sched_clock_init(bool stable) +{ + kvm_sched_clock_offset = kvm_clock_read(); + __paravirt_set_sched_clock(kvm_sched_clock_read, stable, + kvm_save_sched_clock_state, kvm_restore_sched_clock_state); + + pr_info("kvm-clock: using sched offset of %llu cycles", + kvm_sched_clock_offset); + + BUILD_BUG_ON(sizeof(kvm_sched_clock_offset) > + sizeof(((struct pvclock_vcpu_time_info *)NULL)->system_time)); +} + void __init kvmclock_init(void) { u8 flags; From patchwork Thu Feb 27 02:18:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993604 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 528DFC19F32 for ; Thu, 27 Feb 2025 02:25:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897460.1306174 (Exim 4.92) (envelope-from ) id 1tnTaq-0005xn-75; Thu, 27 Feb 2025 02:25:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897460.1306174; Thu, 27 Feb 2025 02:25:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTaq-0005xQ-2F; Thu, 27 Feb 2025 02:25:44 +0000 Received: by outflank-mailman (input) for mailman id 897460; Thu, 27 Feb 2025 02:25:42 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTV2-0005qU-FT for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:44 +0000 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [2607:f8b0:4864:20::64a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 4b84a399-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:39 +0100 (CET) Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-2217a4bfcc7so6991145ad.3 for ; Wed, 26 Feb 2025 18:19:39 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4b84a399-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622778; x=1741227578; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Fan7xWhbYPUjoe6Gj73DFLAXBBtIUty3w4hEBXyAFt4=; b=uN1liozK+r1vGE8azzH9vy6u2FEILmjHx3OBBoSxV9e0UnzrgzYrTI6vSQf9glFgO8 A/EURcOmx+94RDs3lDvDnN6KsVJkEu80deNqWvEUxBGE8ZFVi7XQdYgWHOr7vF313LMO za3D08j/hyJEVu18Jes8duXg019uPbI464JqU445eJ/FhDW12NN6FR7J2RbFoR3bAszY V9ppLjv6VzAHnbu7d9duLOO+CXk0BE4xP4E0xq4a/2xVcOhPhuNPGGK2cb2LzBppYaKX JNGvHnnYgEpRWvqkDBTVfqQ5yaM7VDFQuKjswcwhzytpoeiIa0/DsB79eWgWLS5BGMz+ e+Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622778; x=1741227578; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Fan7xWhbYPUjoe6Gj73DFLAXBBtIUty3w4hEBXyAFt4=; b=JSoODSdzGou92nXSt5S1/e2WtiNhjdboL2nlSZpni6eLTpApF9plVZWUU+ZlS5/G1Q P9q1yx7bd0qPNPe8C4V5ebNCRXBiVpsIDFfbqiVLMXTjt/SeihPCp9q/nQVoDxskfu5D BSfusUYZJTHwwtsn3hS9wWjYGPEVsF/NDzQesIoXbN25v/IddeD2yBg89gndvv/GmWPP p5UPi1jdRmRTrgFsGKuL6d1VN6JpmsyfYK7MH/zP1mGBE50CYkTnbqUcNkhNb8/jOaA2 eWOWDb9b8HdSQYtoMgQaIJIA6uUi3JbCpsyz8QazXfkPuT3GG4Eo8eOFkvh1GeA9F+uc M2Og== X-Forwarded-Encrypted: i=1; AJvYcCVzoa1ix3Um0RKLufwkfTErtiuedd6FGHhr1TZ2q1LBF7hFspiMrc5Ryx/HR2mHPwJe7bc0j3C4gqA=@lists.xenproject.org X-Gm-Message-State: AOJu0Yz8xjbvYmZtC3qLbIsSwdM4vwScX4ntSTchnzjLzUBkRg0gptz2 RIrpnPHeShCheBXNM08AgQwanacDJnlT9w19nQU/a4LVc0T52iU05Ucwk8bigTjT0UH+t2QEz8P E4g== X-Google-Smtp-Source: AGHT+IEIDOZ4TbuvBdNqbq0BRf/iBYYL2tXyd/xxECXyr6rznFcwd1mvvnFfD4EHjsDge5bCz6C8+YGYPK0= X-Received: from plbje3.prod.google.com ([2002:a17:903:2643:b0:223:4e55:d29a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e80d:b0:220:f5d7:6405 with SMTP id d9443c01a7336-221a0edbfc3mr358957175ad.16.1740622777972; Wed, 26 Feb 2025 18:19:37 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:36 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-21-seanjc@google.com> Subject: [PATCH v2 20/38] x86/xen/time: Mark xen_setup_vsyscall_time_info() as __init From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Annotate xen_setup_vsyscall_time_info() as being used only during kernel initialization; it's called only by xen_time_init(), which is already tagged __init. Signed-off-by: Sean Christopherson --- arch/x86/xen/time.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 3179f850352d..13e5888c4501 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -441,7 +441,7 @@ void xen_restore_time_memory_area(void) xen_sched_clock_offset = xen_clocksource_read() - xen_clock_value_saved; } -static void xen_setup_vsyscall_time_info(void) +static void __init xen_setup_vsyscall_time_info(void) { struct vcpu_register_time_memory_area t; struct pvclock_vsyscall_time_info *ti; From patchwork Thu Feb 27 02:18:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3A8AEC19776 for ; Thu, 27 Feb 2025 02:27:01 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897630.1306400 (Exim 4.92) (envelope-from ) id 1tnTby-00019E-Rt; Thu, 27 Feb 2025 02:26:54 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897630.1306400; Thu, 27 Feb 2025 02:26:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTby-00018t-Lo; Thu, 27 Feb 2025 02:26:54 +0000 Received: by outflank-mailman (input) for mailman id 897630; Thu, 27 Feb 2025 02:26:52 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTV1-00063X-86 for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:43 +0000 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [2607:f8b0:4864:20::104a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4c9efc0d-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:41 +0100 (CET) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2fbff6426f5so1023400a91.3 for ; Wed, 26 Feb 2025 18:19:41 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4c9efc0d-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622780; x=1741227580; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=kU2lFcUz0RS/25JB9tsyegi1crsFJmJvQFST0gISHao=; b=dj7MugigeW8T3upUlj7yLWUbOANaFMSVYZGWnYDqHHmXlMB9oLtXeaNdxdUq9/N5Us D9I6oIhP4lL+/1b+q0DyBWOtWo8qi50baq9cummqnLEcHzHEoH1e1C9GQWEm9iePvii6 A4lxBVw2RxXuPae9XuETkkhtax9bVjf/DYudUnaK9dfJ7tfr4PweRt9vc45QtkzMdmr8 43ZZX1EraJFHxOuGDCVA4tVlqIGXt0nHU1n55Vk1h5znzBUAqSRG8ilfQlzaBb+HFPrv dczM88G6K1+xAUpIkWa/0b5/eGNh4PI2UblixrP/h43wDJv4eraXKlfnuft3n10+57aw L+Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622780; x=1741227580; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kU2lFcUz0RS/25JB9tsyegi1crsFJmJvQFST0gISHao=; b=capsnjW+kgpjPjwHUAeUKbg+0WHkm6UOrPi6h0wpbroF5ntoYAB8OYBr+Yinpllp7x G8AnoPtDLsmn2K6sF8F8SSMccy6WPoJKa4JAOY+lSbLVFIQEbPGcD4Vhvbfsh8nlsD+D rmW2YOizQdmMsRJ1f2fK8BXPG1JhQfoajEdrqqZpv0whNK/t866HC2/o9HMnFnYFNJSS iByOOXFLpil1TIKrDuFH2jtMSNMw/RJXPWbSmKh2tiFTNSJa3BhJTy8LQa0VYoTh0rk3 ymVBsnDgwl3FN6VoT0ahgQdJuJGkrmTn6+crUWJ0yFdyOhfuxTRhxbdwZFrOT43hh3xw yI4A== X-Forwarded-Encrypted: i=1; AJvYcCUJg28p6fD5Q03sUSV2rtp+XnzDy32mX4Ap27RNFvYFPbVedjmOtIDcwNUn4KXR7eyKv3CRrMuGAQ4=@lists.xenproject.org X-Gm-Message-State: AOJu0Ywk8VApYdehjTp5zfpbrAJPuhXnMOAeC2c4YXgttkV3P9G2tYmw TC2DkiS6NvIQ5bQOHwZqeYgIFuSn5/binKKqxRHFL/5XeEpA7aQ/27CH6v198C2sfNHo4OIfbqd Brw== X-Google-Smtp-Source: AGHT+IGPqE+K3f21q2ak8FFNHYsAgDpuaP0E98T4gncRvFB5lVR5BPc4qf0azJ1lc37DAZgmd7FgPnilX0Q= X-Received: from pjbsh7.prod.google.com ([2002:a17:90b:5247:b0:2f2:e97a:e77f]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:fc44:b0:2f6:f107:fae6 with SMTP id 98e67ed59e1d1-2fe68cf3f5fmr12813327a91.23.1740622779809; Wed, 26 Feb 2025 18:19:39 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:37 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-22-seanjc@google.com> Subject: [PATCH v2 21/38] x86/pvclock: Mark setup helpers and related various as __init/__ro_after_init From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Now that Xen PV clock and kvmclock explicitly do setup only during init, tag the common PV clock flags/vsyscall variables and their mutators with __init. Signed-off-by: Sean Christopherson --- arch/x86/kernel/pvclock.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c index b3f81379c2fc..a51adce67f92 100644 --- a/arch/x86/kernel/pvclock.c +++ b/arch/x86/kernel/pvclock.c @@ -16,10 +16,10 @@ #include #include -static u8 valid_flags __read_mostly = 0; -static struct pvclock_vsyscall_time_info *pvti_cpu0_va __read_mostly; +static u8 valid_flags __ro_after_init = 0; +static struct pvclock_vsyscall_time_info *pvti_cpu0_va __ro_after_init; -void pvclock_set_flags(u8 flags) +void __init pvclock_set_flags(u8 flags) { valid_flags = flags; } @@ -153,7 +153,7 @@ void pvclock_read_wallclock(struct pvclock_wall_clock *wall_clock, set_normalized_timespec64(ts, now.tv_sec, now.tv_nsec); } -void pvclock_set_pvti_cpu0_va(struct pvclock_vsyscall_time_info *pvti) +void __init pvclock_set_pvti_cpu0_va(struct pvclock_vsyscall_time_info *pvti) { WARN_ON(vclock_was_used(VDSO_CLOCKMODE_PVCLOCK)); pvti_cpu0_va = pvti; From patchwork Thu Feb 27 02:18:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993606 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 98DE9C19F32 for ; Thu, 27 Feb 2025 02:25:58 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897476.1306200 (Exim 4.92) (envelope-from ) id 1tnTax-0007J6-ST; Thu, 27 Feb 2025 02:25:51 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897476.1306200; Thu, 27 Feb 2025 02:25:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTax-0007IT-MB; Thu, 27 Feb 2025 02:25:51 +0000 Received: by outflank-mailman (input) for mailman id 897476; Thu, 27 Feb 2025 02:25:50 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTV3-00063X-8h for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:45 +0000 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [2607:f8b0:4864:20::104a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4dae1e81-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:43 +0100 (CET) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2fe98fad333so1065916a91.2 for ; Wed, 26 Feb 2025 18:19:43 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4dae1e81-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622781; x=1741227581; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=idfxGEE7w4MZK5vL28VhSjtN1o6ADW/NzBB0c7HM3Jg=; b=0uN8j7/8XHXkiThOcGwluLYKvE9J2OpxVVA/q6EdLLJBPvuYfj/mxx1JYHhjjFFs4s nFbuQxovjFDJ7wsHcLbGtdPk0Yx0qCJX77FRwCwHFkV92CjkoiIVuwnXZLiKXX+3Suqu 3V46th+LQeKG081Oz0QwhkKB+0lJnGKnecw0bkrUdJifAMpD4iVGCTB0VvzQU4fT2IBp /8KpefDvpCsEFkMW2qHFvzlwEWlTrZiexomlX2M1BaFgbcSeTyyU73tobps2wlRKXDZ3 6fZb9QYUvsmUOLC22YCKkKEiESW5oqhVj5VLeKiGE+9SFHu4Ov+0wLeT+0pE9vuo5aeK PcMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622781; x=1741227581; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=idfxGEE7w4MZK5vL28VhSjtN1o6ADW/NzBB0c7HM3Jg=; b=HGbzE7DL3LXmIyCkgypIRgWyj+OH73BFWWTwF+frpPo8zyDBQjbJcqDFIh6cSv23cO kT1TUjJX10+7BmvAOKmFLyJ5AoVOd7u2rcm94Xko1eGaKMwSTCZZ1vZ6TT/xvFIBeTdV 2X/aX84ct0bk8FyU++OZ4PUvTIwEe1ze296QLgRtPYlcZcUyZgl4JkZU1tN8DuYwwGM6 kV6otSdPCGhZW9MKYWgoi75VaOIi1ymvmNOqhLNxVTjFQNe2QnWubX196neXRGAv8tgb r4VNam83JCv75AyRTTvBv8SoLFpA6kyoC+avz5MJszPD3lpx3WM/Vx9/n2iZmwQibka5 9EhA== X-Forwarded-Encrypted: i=1; AJvYcCVMn1GEE4MvEL4aeYBCdLv5ntOVvbMWF3vu+bL5yzZFqNNN2gaMakw142avberX2F2mAHy8eM5m+EM=@lists.xenproject.org X-Gm-Message-State: AOJu0Yx5UQ6HxVOpJGPri/5UTISsuDfRgDsgQ9JERy1zOzFmk7iBHrzS UvqU5oaxXWBOIDgT7HzZwi3CgF5Mk6LKMybczdXsPwQBYIKBK6eX9dJyPU7Yld202+xZ/r4jHXo VaA== X-Google-Smtp-Source: AGHT+IHCzPrRKHA3wP3HVgIKrQRSiWCMkFG4NMbhGFfjlCu1Pi9S6BKyZnSxtjm5Zx+28kwNIdNRFoi5Jdo= X-Received: from pjbqi7.prod.google.com ([2002:a17:90b:2747:b0:2f5:4762:e778]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4c04:b0:2fa:20f4:d27f with SMTP id 98e67ed59e1d1-2fce7b23bb5mr32633734a91.32.1740622781602; Wed, 26 Feb 2025 18:19:41 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:38 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-23-seanjc@google.com> Subject: [PATCH v2 22/38] x86/pvclock: WARN if pvclock's valid_flags are overwritten From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania WARN if the common PV clock valid_flags are overwritten; all PV clocks expect that they are the one and only PV clock, i.e. don't guard against another PV clock having modified the flags. Signed-off-by: Sean Christopherson --- arch/x86/kernel/pvclock.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c index a51adce67f92..8d098841a225 100644 --- a/arch/x86/kernel/pvclock.c +++ b/arch/x86/kernel/pvclock.c @@ -21,6 +21,7 @@ static struct pvclock_vsyscall_time_info *pvti_cpu0_va __ro_after_init; void __init pvclock_set_flags(u8 flags) { + WARN_ON(valid_flags); valid_flags = flags; } From patchwork Thu Feb 27 02:18:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993610 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C8944C19F32 for ; Thu, 27 Feb 2025 02:26:01 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897484.1306228 (Exim 4.92) (envelope-from ) id 1tnTb0-0007vm-QS; Thu, 27 Feb 2025 02:25:54 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897484.1306228; Thu, 27 Feb 2025 02:25:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTb0-0007qm-3u; Thu, 27 Feb 2025 02:25:54 +0000 Received: by outflank-mailman (input) for mailman id 897484; Thu, 27 Feb 2025 02:25:51 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTV7-0005qU-Fn for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:49 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 4ec514aa-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:44 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2f81a0d0a18so1091531a91.3 for ; Wed, 26 Feb 2025 18:19:44 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4ec514aa-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622783; x=1741227583; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Za0nk5QeJovZveNmDgTJSzh3R0hb/7p0/gaKYZ4C/Vs=; b=Q5q2J/VjzoU+MMZ2merUpsvt4BFtgiTRf0sDvkFD4U6xwqPnu1LSys7fgsAf5yR9yF SpoJ53WspMLmuOZtGPm3xrFxJJZ8kieBtnrVM6qYjcYhx0FdozfXu9j9rKxfy2BwnnAi f/jDtWh5RFXyKfUqxuNjXdmDQUNupJhGvbWePrzSWnmc9ZaFfGqrSFI6m66dp8a8I+Ya HcxceJ8PLFygCaMItvADDlFnPSclPKU74xYspK/yAeYDS49bHXl/YJs7ehzCzKDXoclt zM5vmeejTDfPKadKC5Cd8CmEEig7avwdjc6zymHn59xz4iZcwr0LHms8+EPjpRsTQ9Jl jYXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622783; x=1741227583; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Za0nk5QeJovZveNmDgTJSzh3R0hb/7p0/gaKYZ4C/Vs=; b=nHk3UCOJjswsWO03CILbmS4cfSzyTo9RuIO33fM94MdzwR0tdLZE5y8YOvbFI0hiWo JATZoNPys0SX7MpR313VRZcuvZ8jfgnV5Ka0MwZOqP1/v0hpcXgQezRFaYP1fud1jKST 48HbNb8UKENAbzAjMxTPy5kxBYuAKEMs78FgVtcIgcYfzdKtrceIqPkXVQBoJVh2SdfI cHu/XXlhSOcRRBr8FdlN4xclEpki49mALkPbviiVjRpWbxgszhpqMF7BpYUuxFCKlbTy in8j/Zgi4buiEaU98npoV1auDEAq3fg0NfTfJJOoHF6zpkdFWqUaz9uAMULnOX0LwPb2 IC3A== X-Forwarded-Encrypted: i=1; AJvYcCWMHLYiQxxbkhqGy7LK7udu/zWNHrjCOtVPTxcF3zoFl7PHC1VX2K4q+Cl2xxboeSCE+7jkjmhvhYg=@lists.xenproject.org X-Gm-Message-State: AOJu0Yy5wS1vQsgsMn3bShGAyoz1pi6l3UtGIn+0UPLL9Y+WDnFrGen5 JGDevl6vL+nTPJQI/zDvmLmotnidur3aGxbeDH/u9lZz23IntuTN0+uoDLYeLlE4z9g1io/5q+E QqQ== X-Google-Smtp-Source: AGHT+IHPEy5y9WPVcI/hX8J+CDDBk9ezIawKhLRmwSNwFlW/vj5Sb4uKTsOn6iseYydXRMp+Q/+8QjZLtQQ= X-Received: from pjbsm1.prod.google.com ([2002:a17:90b:2e41:b0:2f8:4024:b59a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:d2d0:b0:2ee:b6c5:1def with SMTP id 98e67ed59e1d1-2fe68ad9ef3mr15165448a91.8.1740622783395; Wed, 26 Feb 2025 18:19:43 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:39 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-24-seanjc@google.com> Subject: [PATCH v2 23/38] x86/kvmclock: Refactor handling of PVCLOCK_TSC_STABLE_BIT during kvmclock_init() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Clean up the setting of PVCLOCK_TSC_STABLE_BIT during kvmclock init to make it somewhat obvious that pvclock_read_flags() must be called *after* pvclock_set_flags(). Note, in theory, a different PV clock could have set PVCLOCK_TSC_STABLE_BIT in the supported flags, i.e. reading flags only if KVM_FEATURE_CLOCKSOURCE_STABLE_BIT is set could very, very theoretically result in a change in behavior. In practice, the kernel only supports a single PV clock. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 934ee4a4c6d4..0580fe1aefa0 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -304,7 +304,7 @@ static void __init kvm_sched_clock_init(bool stable) void __init kvmclock_init(void) { - u8 flags; + bool stable = false; if (!kvm_para_available() || !kvmclock) return; @@ -331,11 +331,18 @@ void __init kvmclock_init(void) kvm_register_clock("primary cpu clock"); pvclock_set_pvti_cpu0_va(hv_clock_boot); - if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT)) + if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT)) { pvclock_set_flags(PVCLOCK_TSC_STABLE_BIT); - flags = pvclock_read_flags(&hv_clock_boot[0].pvti); - kvm_sched_clock_init(flags & PVCLOCK_TSC_STABLE_BIT); + /* + * Check if the clock is stable *after* marking TSC_STABLE as a + * valid flag. + */ + stable = pvclock_read_flags(&hv_clock_boot[0].pvti) & + PVCLOCK_TSC_STABLE_BIT; + } + + kvm_sched_clock_init(stable); tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz); From patchwork Thu Feb 27 02:18:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 48FDFC19776 for ; Thu, 27 Feb 2025 02:26:55 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897620.1306370 (Exim 4.92) (envelope-from ) id 1tnTbs-0008Sb-GR; Thu, 27 Feb 2025 02:26:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897620.1306370; Thu, 27 Feb 2025 02:26:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbs-0008RY-BC; Thu, 27 Feb 2025 02:26:48 +0000 Received: by outflank-mailman (input) for mailman id 897620; Thu, 27 Feb 2025 02:26:46 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTV5-00063X-SF for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:47 +0000 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [2607:f8b0:4864:20::104a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4fa8b661-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:46 +0100 (CET) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2fbff6426f5so1023516a91.3 for ; Wed, 26 Feb 2025 18:19:46 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4fa8b661-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622785; x=1741227585; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=qN8YI5wmqMG6yVEs+7cu8Sw/AfoU86Z3NaGBSlCYqu0=; b=VqxlZ8u3A9VuOK4nb63P+b/hscjZnqQeFj/mLqrrblGm+Nu0A+8RDqn6EexwcYGrWQ KdSJ2uI/6kuUPtG01edtsXm99kO4KLxKXtLAsdAHSIFoGJHoONJgf2MV5LA/6CYIPzsS CEhjN03WP1kDpkKSE/727M3uz9b9SfMOitP9tAV+6WqzOLm5u2miWrzuLXyTywfivBX7 9L0dSRG/6b3Yk90UpXafoy6+11CKte/ltszDrAg7ComHVg2LmJhoeCtEqpbntOgLY78V FNwXTFYQKXCOqfA/Eo0Ei1v1hTpph1xoyK5SSU2OOjbPuiVF08OCjF4of2s4uqQnvIa9 oAaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622785; x=1741227585; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qN8YI5wmqMG6yVEs+7cu8Sw/AfoU86Z3NaGBSlCYqu0=; b=xG0a7uD5m3SQEKt6ZZaBd67liBRUWu+DRgjotGBR5vL6VZg/xCLq/scMms63j+Bl9q H1DGC8QAcBcFhP1t3w0XmBTfN47h/bJreyEUnlxhIQ9l+9QLMK3oLE9eSK4TcGErylQ8 5bWfCkGfgMYB6AzQRWxjUedgQ0JqNlwAaZv2p8coMCZ+eBgwOLrZBz/UyyD5yn3gzUVN tbw3PquN6Tul+8zkcxC/jzEI7VU7HgqMa9l33Qi18H2vUZwPWZos9PNi1t2GHOBcJNvG J9Wx9xvLJzFbHp/b8h4ZNkyGWZBfxg40vfXDnTuLaG9OT+tcEg86a275DMXkh/Mclvi3 kPdg== X-Forwarded-Encrypted: i=1; AJvYcCWXOyWZqXJywfMNVUCNa0c/7ON+Y3i68YTKoMYEuZpsmG1qP60oUP3j2IMZkub4C6ONRtyrE+GTHxI=@lists.xenproject.org X-Gm-Message-State: AOJu0YwyATi1fubndOA0bwIAQFCbimuXQdAdnCks2R2/bpAi7lt1qE2j 3chqJKJ6/Bws1O2FL/R+uxa71JbdBc2pgVOsmMdpg7Okn1vAWjN6qtAKIMz0jiVydK5cAQrP9Df oqg== X-Google-Smtp-Source: AGHT+IEw7VZRln4iIrg1rWM/7IZnuYkNWSUUNxKKGfY9HmA7vrLySNMf4INY1k647TDU8xAK61s1yi1H724= X-Received: from pjbsw12.prod.google.com ([2002:a17:90b:2c8c:b0:2fa:284f:adae]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5243:b0:2f4:434d:c7f0 with SMTP id 98e67ed59e1d1-2fe68ada3e8mr17648463a91.12.1740622784911; Wed, 26 Feb 2025 18:19:44 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:40 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-25-seanjc@google.com> Subject: [PATCH v2 24/38] timekeeping: Resume clocksources before reading persistent clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When resuming timekeeping after suspend, restore clocksources prior to reading the persistent clock. Paravirt clocks, e.g. kvmclock, tie the validity of a PV persistent clock to a clocksource, i.e. reading the PV persistent clock will return garbage if the underlying PV clocksource hasn't been enabled. The flaw has gone unnoticed because kvmclock is a mess and uses its own suspend/resume hooks instead of the clocksource suspend/resume hooks, which happens to work by sheer dumb luck (the kvmclock resume hook runs before timekeeping_resume()). Note, there is no evidence that any clocksource supported by the kernel depends on a persistent clock. Signed-off-by: Sean Christopherson Reviewed-by: Thomas Gleixner --- kernel/time/timekeeping.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 1e67d076f195..332d053fa9ce 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1794,11 +1794,16 @@ void timekeeping_resume(void) u64 cycle_now, nsec; unsigned long flags; - read_persistent_clock64(&ts_new); - clockevents_resume(); clocksource_resume(); + /* + * Read persistent time after clocksources have been resumed. Paravirt + * clocks have a nasty habit of piggybacking a persistent clock on a + * system clock, and may return garbage if the system clock is suspended. + */ + read_persistent_clock64(&ts_new); + raw_spin_lock_irqsave(&tk_core.lock, flags); /* From patchwork Thu Feb 27 02:18:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ECF65C19F32 for ; Thu, 27 Feb 2025 02:26:58 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897627.1306386 (Exim 4.92) (envelope-from ) id 1tnTbw-0000Y4-Ab; Thu, 27 Feb 2025 02:26:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897627.1306386; Thu, 27 Feb 2025 02:26:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbw-0000XU-20; Thu, 27 Feb 2025 02:26:52 +0000 Received: by outflank-mailman (input) for mailman id 897627; Thu, 27 Feb 2025 02:26:50 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVA-0005qU-GQ for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:52 +0000 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [2607:f8b0:4864:20::64a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 5096aaac-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:47 +0100 (CET) Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-22107b29ac3so7797435ad.1 for ; Wed, 26 Feb 2025 18:19:47 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5096aaac-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622786; x=1741227586; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3WIlT5dxc8eqreSuMAUxshuXGBEM8C/NLPwcD1zy3Y0=; b=4Tn+HIhTEofezILu9LoLr1+pu4sBPUvjgk3YbPzy3M0AXjaXsUbZc4f6OR46LFkdHd xciMaZqRjYs9i8IlJcCFsXIKUhq357GYjhw/8k2kwgHHXHLhdRLEjrmJXqkb/BAm4Blo nlMlJftWNE+2aydySXvYI1v8NY0/b3vk0j4P+EreGa30VYXHy41E4Tpxsxj9qASbqZrF pz6zd1luV3gjXWWMi8+kLvZxHZuWuEHmd05S62mi1r17x9bOlsRtSLOTNUl30lUmHV1L Dgqulh3zfVGUM0FQ4UTFDhyfKqhNwiqDSlw0+OslEpFXCdDVslX3qA8EppCYsSoGhrc9 PfoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622786; x=1741227586; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3WIlT5dxc8eqreSuMAUxshuXGBEM8C/NLPwcD1zy3Y0=; b=pBfn8g52sZb2u7+SrPjtdiu2x28t9qsilOhu9eXbSR3C4gfmpgQ3Mi+vQMJmJV++1R PrN3KqdEfUdFL98LsA5iMLWcC/V5ykzjyp6Vzi87aR9Q2DNg8BnxAnr42G+4yOW0XHgM /UuL7rdOhm5QsMPT0JM4/LgdoT0dwK+5Fm+kKQusFsK+pcGyhZ59hLgWXTfNUhe2iWZj pYoeKeEPzypdUofgR3Jm22Lhk28etdcwJSjjw/qGtPpT5QQsHu/+0skrhuTpYweDa9kp 2JFyVCo3cHr4jh0CfyADMbGttjPnMoCyrS9VeNdl/5Tl3xJJ4xPGTwJL3iS84Hc/5eBd j1Eg== X-Forwarded-Encrypted: i=1; AJvYcCX/5Q0fgb7TMow7A6XNTNRB0mGDgxw0JCUtdWLrETFN/QOzUOW4TzC1BsBUgfIpIxCZQwKZ1dH45Iw=@lists.xenproject.org X-Gm-Message-State: AOJu0YxTC2fNGBINAsPX2TBLLe3WxWgTTA7zrgV5m5+TcLSCr3rg6DU8 z575FgCnVes/xzDGGsyR3OlNEQTKMTvfh3LIFt5TAp1rgsssv7yvvxXgQ+rGs/tqcW74z2Npb4h aqw== X-Google-Smtp-Source: AGHT+IEKa+U3XmhHVLGnMc9mY3OxJMjJ1pR3+E2erpOOt4xnyyZwND9G3ZoTVgYNZ7eny0ZZLwJ5fe0HKa8= X-Received: from pjvf4.prod.google.com ([2002:a17:90a:da84:b0:2ea:3a1b:f493]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e80e:b0:21f:3d0d:2408 with SMTP id d9443c01a7336-2234a28af91mr27386885ad.10.1740622786458; Wed, 26 Feb 2025 18:19:46 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:41 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-26-seanjc@google.com> Subject: [PATCH v2 25/38] x86/kvmclock: Hook clocksource.suspend/resume when kvmclock isn't sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Save/restore kvmclock across suspend/resume via clocksource hooks when kvmclock isn't being used for sched_clock. This will allow using kvmclock as a clocksource (or for wallclock!) without also using it for sched_clock. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 0580fe1aefa0..319f8b2d0702 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -130,7 +130,17 @@ static void kvm_setup_secondary_clock(void) static void kvm_restore_sched_clock_state(void) { - kvm_register_clock("primary cpu clock, resume"); + kvm_register_clock("primary cpu, sched_clock resume"); +} + +static void kvmclock_suspend(struct clocksource *cs) +{ + kvmclock_disable(); +} + +static void kvmclock_resume(struct clocksource *cs) +{ + kvm_register_clock("primary cpu, clocksource resume"); } void kvmclock_cpu_action(enum kvm_guest_cpu_action action) @@ -201,6 +211,8 @@ static struct clocksource kvm_clock = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .id = CSID_X86_KVM_CLK, .enable = kvm_cs_enable, + .suspend = kvmclock_suspend, + .resume = kvmclock_resume, }; static void __init kvmclock_init_mem(void) @@ -295,6 +307,15 @@ static void __init kvm_sched_clock_init(bool stable) __paravirt_set_sched_clock(kvm_sched_clock_read, stable, kvm_save_sched_clock_state, kvm_restore_sched_clock_state); + /* + * The BSP's clock is managed via dedicated sched_clock save/restore + * hooks when kvmclock is used as sched_clock, as sched_clock needs to + * be kept alive until the very end of suspend entry, and restored as + * quickly as possible after resume. + */ + kvm_clock.suspend = NULL; + kvm_clock.resume = NULL; + pr_info("kvm-clock: using sched offset of %llu cycles", kvm_sched_clock_offset); From patchwork Thu Feb 27 02:18:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E43D3C19F32 for ; Thu, 27 Feb 2025 02:26:08 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897509.1306271 (Exim 4.92) (envelope-from ) id 1tnTb8-0001VO-HV; Thu, 27 Feb 2025 02:26:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897509.1306271; Thu, 27 Feb 2025 02:26:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTb8-0001Uo-7o; Thu, 27 Feb 2025 02:26:02 +0000 Received: by outflank-mailman (input) for mailman id 897509; Thu, 27 Feb 2025 02:26:01 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTV9-00063X-Rv for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:51 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 519e24e8-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:49 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fe8de1297eso1024013a91.0 for ; Wed, 26 Feb 2025 18:19:49 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 519e24e8-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622788; x=1741227588; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=8l5U+hO6xQEeVCf+kI9O/UrrnIVeF1W04fkwqmR0RAk=; b=hBRjHQwroWuQe0D6w5S5WeOvvhd4qUq+NIo/KC+TXKSjwnbJOKEW36HuHBziksEpBN NAeNYXcFcg1gtbOrI3J6RsHDlKdLjjZVIAG6Etn42AKUtI7WGngGrqgK2oL7+zPx08oz xxidpZBHamUqNM37vQxPfLUTeBCWdofYf/roGa/yu6B+4JYMfQYipF3ZHsTa09izwIHf HWJM2ToQKC31sGOCyegpg9ZuAuGNAAle5AlFYW1tFb6mbEDg7j47Cu2WTwP/jHjS+OJu XfYKOZ5wdh2TCMpwGP5dpsX99KZ5fN3DQ1b1zENZytuer3XJY3cXtmEJ5887jkjg9Ruo HTaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622788; x=1741227588; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8l5U+hO6xQEeVCf+kI9O/UrrnIVeF1W04fkwqmR0RAk=; b=q5h9h6rwVZc90wj5okuqf5nj7PtTiptKNIZkHIhLCxWjbCrieEgSVRuLteAZaHiYJO kA7e1x5yIgnfXtXXiVyVpoSvAtpicZPAPh+L20sL2rn5FX8+ekNeFfF5w6BcBlhWL9gI lqAIeLRTs8cixrT1KRm2DwtoLfLrzzqXrQv0NYTDgdcI1iljp1Dt3iWCaBdNZ5mtycza hokLC3HpiA/V9OHOLW1ZeyY2NOILQH8e+8CQhbkZHaa4yI8bid/0N3kei8m4cI0Gej9i v6VMmvBrq1tZBP0Nw3vy74KujJiVn0SX2Wk+YkqFktwyMf2PxiQKpZAzcIQSIpX8bNs4 rm+A== X-Forwarded-Encrypted: i=1; AJvYcCW7csTlozPxSO/6empjNhgha0bsSccOQ+InpZAUFM2TSHJzCEZyfsHgYq0TkvtURZI9rCP1m4gxUb8=@lists.xenproject.org X-Gm-Message-State: AOJu0YxFoKMmFOV3oBnDYMnHOG3psvAPB/lqEr17rIn+6leG/Fu7t/Pt D86j2tYQ0MfzdK2W8SM1Qe8YQdDiu3yf5JQrzV+UpaianoSqCKuYt3EemmQ4Bw4RapYSZoRmRTb 6aQ== X-Google-Smtp-Source: AGHT+IGsGVabM6lqtL4epZ9oGORwajTYA23ku4RspaNu+4NGjN39FOibtOjpJfF6SNul6q4RPaHxkAISArM= X-Received: from pjbph15.prod.google.com ([2002:a17:90b:3bcf:b0:2ef:d136:17fc]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3901:b0:2fa:2268:1af4 with SMTP id 98e67ed59e1d1-2fea1299b06mr2509653a91.7.1740622788182; Wed, 26 Feb 2025 18:19:48 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:42 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-27-seanjc@google.com> Subject: [PATCH v2 26/38] x86/kvmclock: WARN if wall clock is read while kvmclock is suspended From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania WARN if kvmclock is still suspended when its wallclock is read, i.e. when the kernel reads its persistent clock. The wallclock subtly depends on the BSP's kvmclock being enabled, and returns garbage if kvmclock is disabled. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 319f8b2d0702..0ce23f862cbd 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -52,6 +52,8 @@ static struct pvclock_vsyscall_time_info *hvclock_mem; DEFINE_PER_CPU(struct pvclock_vsyscall_time_info *, hv_clock_per_cpu); EXPORT_PER_CPU_SYMBOL_GPL(hv_clock_per_cpu); +static bool kvmclock_suspended; + /* * The wallclock is the time of day when we booted. Since then, some time may * have elapsed since the hypervisor wrote the data. So we try to account for @@ -59,6 +61,7 @@ EXPORT_PER_CPU_SYMBOL_GPL(hv_clock_per_cpu); */ static void kvm_get_wallclock(struct timespec64 *now) { + WARN_ON_ONCE(kvmclock_suspended); wrmsrl(msr_kvm_wall_clock, slow_virt_to_phys(&wall_clock)); preempt_disable(); pvclock_read_wallclock(&wall_clock, this_cpu_pvti(), now); @@ -118,6 +121,7 @@ static void kvm_save_sched_clock_state(void) * to the old address prior to reconfiguring kvmclock would clobber * random memory. */ + kvmclock_suspended = true; kvmclock_disable(); } @@ -130,16 +134,19 @@ static void kvm_setup_secondary_clock(void) static void kvm_restore_sched_clock_state(void) { + kvmclock_suspended = false; kvm_register_clock("primary cpu, sched_clock resume"); } static void kvmclock_suspend(struct clocksource *cs) { + kvmclock_suspended = true; kvmclock_disable(); } static void kvmclock_resume(struct clocksource *cs) { + kvmclock_suspended = false; kvm_register_clock("primary cpu, clocksource resume"); } From patchwork Thu Feb 27 02:18:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993620 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 92825C021B8 for ; Thu, 27 Feb 2025 02:26:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897611.1306340 (Exim 4.92) (envelope-from ) id 1tnTbn-0007Qh-I5; Thu, 27 Feb 2025 02:26:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897611.1306340; Thu, 27 Feb 2025 02:26:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbn-0007QW-EA; Thu, 27 Feb 2025 02:26:43 +0000 Received: by outflank-mailman (input) for mailman id 897611; Thu, 27 Feb 2025 02:26:42 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVB-0005qU-GT for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:53 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 529c7166-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:51 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc0bc05afdso1130504a91.0 for ; Wed, 26 Feb 2025 18:19:51 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 529c7166-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622790; x=1741227590; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=QzXAit3+eiuwe2E434PEIcPVDj3bfcCBW02+u03qap4=; b=jOs9cCCVzGV/8vYkXZYZYFtS7XhUN5G9gvf0y2CpkxdjepZ+8zvrKbymut8LypWtaU WMAOvlpysiX2wiUraFjYLAEqcZYlbunKQ2bPBv483AZdPHJKdoTwiCoM8hfK6eCDY0KC w4+d4pVT29XYoEe/L9B+nCORBfZSERsqDnPNk/lnhx8O7aAy68MC1FxFuVFlRtYsMvFo B0mVe2Zzmsn3OK5XPIeADmWco6CwppLoG56VNUgRt0LC2ZLRAOVfLvh0aW72nRSFvomV oUG3er57sTQkaL0vJScxfBNh9ZTu7Vb9+ioEP/lWFb9SIFdKm1jpkeVlL/2g5GOhLCYe whSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622790; x=1741227590; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QzXAit3+eiuwe2E434PEIcPVDj3bfcCBW02+u03qap4=; b=EwMqwS7tUbCo/A4A58kiqF5+yId9prBY3sEZk0BoLM95sVlYmraYIaCKtXWQ6wxbxi GgNwETnb3TIaZdREC1GvvcmUUDUHStTb5fDQVTO+KJKctC39m18f0xlZ7+IdWHkuQk5t 5SSA0dEWeGzkSe0xx7KRo0qcp/KD8iQHKKFOC+YWkWWQIeUkIAbDpQXKvBjqScFkejV1 HqO3nlu/p/AyySNylfyztxuUBSvtOBYPsQAijsv4sK6V+lV2V3BKdzGpkKKXuuzWdlJ9 g0iaPO3ucx20EgTVu1eR22pbhH7h73+gHtAc/GwWc5eNRq1D01pprkDQjJbBTyaagPWG n1CA== X-Forwarded-Encrypted: i=1; AJvYcCXjiwdj8EQxFMjGCQ9F/mf/eNO+PWFhNlyNLUEMwHFfU94yNagywog+NdT+RoV0XuVB8JD+ozy5YDw=@lists.xenproject.org X-Gm-Message-State: AOJu0YxPkgf+2+SRhFU1KcrULSdFrccPLyqoJQOadrG7aCdbrXVZNg9R TFUiY4Z7wws0X/lfxHewpnx7kALO0QFgZWvi9k7BNtqOa7g7I30BLD1oYT9x9ZtW9gwBb+KF4j7 plA== X-Google-Smtp-Source: AGHT+IEy19UXAGhpZ6BooyEkiFGbgvHpy57ZvxRHzoGzPaLhW9PSe6CovWEb42HIAZDNRdCcii7axkfnXfc= X-Received: from pjbsh13.prod.google.com ([2002:a17:90b:524d:b0:2ee:53fe:d0fc]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5385:b0:2f5:63a:449c with SMTP id 98e67ed59e1d1-2fe68cf4000mr14888818a91.28.1740622789852; Wed, 26 Feb 2025 18:19:49 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:43 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-28-seanjc@google.com> Subject: [PATCH v2 27/38] x86/kvmclock: Enable kvmclock on APs during onlining if kvmclock isn't sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania In anticipation of making x86_cpuinit.early_percpu_clock_init(), i.e. kvm_setup_secondary_clock(), a dedicated sched_clock hook that will be invoked if and only if kvmclock is set as sched_clock, ensure APs enable their kvmclock during CPU online. While a redundant write to the MSR is technically ok, skip the registration when kvmclock is sched_clock so that it's somewhat obvious that kvmclock *needs* to be enabled during early bringup when it's being used as sched_clock. Plumb in the BSP's resume path purely for documentation purposes. Both KVM (as-a-guest) and timekeeping/clocksource hook syscore_ops, and it's not super obvious that using KVM's hooks would be flawed. E.g. it would work today, because KVM's hooks happen to run after/before timekeeping's hooks during suspend/resume, but that's sheer dumb luck as the order in which syscore_ops are invoked depends entirely on when a subsystem is initialized and thus registers its hooks. Opportunsitically make the registration messages more precise to help debug issues where kvmclock is enabled too late. Opportunstically WARN in kvmclock_{suspend,resume}() to harden against future bugs. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_para.h | 2 ++ arch/x86/kernel/kvm.c | 24 +++++++++++------- arch/x86/kernel/kvmclock.c | 44 +++++++++++++++++++++++++++------ 3 files changed, 53 insertions(+), 17 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 8708598f5b8e..42d90bf8fde9 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -120,6 +120,8 @@ static inline long kvm_sev_hypercall3(unsigned int nr, unsigned long p1, #ifdef CONFIG_KVM_GUEST enum kvm_guest_cpu_action { KVM_GUEST_BSP_SUSPEND, + KVM_GUEST_BSP_RESUME, + KVM_GUEST_AP_ONLINE, KVM_GUEST_AP_OFFLINE, KVM_GUEST_SHUTDOWN, }; diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 866b061ee0d9..5f093190df17 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -461,18 +461,24 @@ static void kvm_guest_cpu_offline(enum kvm_guest_cpu_action action) kvmclock_cpu_action(action); } +static int __kvm_cpu_online(unsigned int cpu, enum kvm_guest_cpu_action action) +{ + unsigned long flags; + + local_irq_save(flags); + kvmclock_cpu_action(action); + kvm_guest_cpu_init(); + local_irq_restore(flags); + return 0; +} + +#ifdef CONFIG_SMP + static int kvm_cpu_online(unsigned int cpu) { - unsigned long flags; - - local_irq_save(flags); - kvm_guest_cpu_init(); - local_irq_restore(flags); - return 0; + return __kvm_cpu_online(cpu, KVM_GUEST_AP_ONLINE); } -#ifdef CONFIG_SMP - static DEFINE_PER_CPU(cpumask_var_t, __pv_cpu_mask); static bool pv_tlb_flush_supported(void) @@ -737,7 +743,7 @@ static int kvm_suspend(void) static void kvm_resume(void) { - kvm_cpu_online(raw_smp_processor_id()); + __kvm_cpu_online(raw_smp_processor_id(), KVM_GUEST_BSP_RESUME); #ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL if (kvm_para_has_feature(KVM_FEATURE_POLL_CONTROL) && has_guest_poll) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 0ce23f862cbd..76884dfc77f4 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -52,6 +52,7 @@ static struct pvclock_vsyscall_time_info *hvclock_mem; DEFINE_PER_CPU(struct pvclock_vsyscall_time_info *, hv_clock_per_cpu); EXPORT_PER_CPU_SYMBOL_GPL(hv_clock_per_cpu); +static bool kvmclock_is_sched_clock; static bool kvmclock_suspended; /* @@ -128,7 +129,7 @@ static void kvm_save_sched_clock_state(void) #ifdef CONFIG_SMP static void kvm_setup_secondary_clock(void) { - kvm_register_clock("secondary cpu clock"); + kvm_register_clock("secondary cpu, sched_clock setup"); } #endif @@ -140,25 +141,51 @@ static void kvm_restore_sched_clock_state(void) static void kvmclock_suspend(struct clocksource *cs) { + if (WARN_ON_ONCE(kvmclock_is_sched_clock)) + return; + kvmclock_suspended = true; kvmclock_disable(); } static void kvmclock_resume(struct clocksource *cs) { + if (WARN_ON_ONCE(kvmclock_is_sched_clock)) + return; + kvmclock_suspended = false; kvm_register_clock("primary cpu, clocksource resume"); } void kvmclock_cpu_action(enum kvm_guest_cpu_action action) { - /* - * Don't disable kvmclock on the BSP during suspend. If kvmclock is - * being used for sched_clock, then it needs to be kept alive until the - * last minute, and restored as quickly as possible after resume. - */ - if (action != KVM_GUEST_BSP_SUSPEND) + switch (action) { + /* + * The BSP's clock is managed via clocksource suspend/resume, + * to ensure it's enabled/disabled when timekeeping needs it + * to be, e.g. before reading wallclock (which uses kvmclock). + */ + case KVM_GUEST_BSP_SUSPEND: + case KVM_GUEST_BSP_RESUME: + break; + case KVM_GUEST_AP_ONLINE: + /* + * Secondary CPUs use dedicated sched_clock hooks to enable + * kvmclock early during bringup, there's nothing to be done + * when during CPU online. + */ + if (kvmclock_is_sched_clock) + break; + kvm_register_clock("secondary cpu, online"); + break; + case KVM_GUEST_AP_OFFLINE: + case KVM_GUEST_SHUTDOWN: kvmclock_disable(); + break; + default: + WARN_ON_ONCE(1); + break; + } } /* @@ -313,6 +340,7 @@ static void __init kvm_sched_clock_init(bool stable) kvm_sched_clock_offset = kvm_clock_read(); __paravirt_set_sched_clock(kvm_sched_clock_read, stable, kvm_save_sched_clock_state, kvm_restore_sched_clock_state); + kvmclock_is_sched_clock = true; /* * The BSP's clock is managed via dedicated sched_clock save/restore @@ -356,7 +384,7 @@ void __init kvmclock_init(void) msr_kvm_system_time, msr_kvm_wall_clock); this_cpu_write(hv_clock_per_cpu, &hv_clock_boot[0]); - kvm_register_clock("primary cpu clock"); + kvm_register_clock("primary cpu, online"); pvclock_set_pvti_cpu0_va(hv_clock_boot); if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT)) { From patchwork Thu Feb 27 02:18:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993614 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 856D6C1B087 for ; Thu, 27 Feb 2025 02:26:11 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897512.1306280 (Exim 4.92) (envelope-from ) id 1tnTbA-0001yq-UT; Thu, 27 Feb 2025 02:26:04 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897512.1306280; Thu, 27 Feb 2025 02:26:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbA-0001yP-ND; Thu, 27 Feb 2025 02:26:04 +0000 Received: by outflank-mailman (input) for mailman id 897512; Thu, 27 Feb 2025 02:26:03 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVC-0005qU-GY for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:54 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 53945838-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:52 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fea1685337so792101a91.0 for ; Wed, 26 Feb 2025 18:19:52 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 53945838-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622791; x=1741227591; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=w772IaQ5YRbZvSP+H3ufucBFIq7F/07nn6vrGLJgcY8=; b=M0wfsNdgYDrakqB6sVRnOUEpd5awj+N4zJY4GUa5TyA/FH7/6o3DsUecgp9hvKyLFc AzMnrPcYERuf759wstpiN0yqaHJz51nXzc1zFAwEpx6CX1QYNIyXtsoYcqn5xWoQ/k86 jJp9/J5ILo7H+hq9Ne1vqSU1pK0WtVMFaU8qomY252NxvC2F+uGR/vzYE6Ou1TOJBZLf Q7Qz3cEDKH3fU93ZHsHMvRxjrTdjuqA2hGj2L26MA8JB3YFudC0f9GgjdiDy1Ob4z9nQ DtHYwuYKhC+JtyPaaN8kjxhzWPOU0DkuHs7xz5EcC42Ztechfh7KDAAC3Lr+W8n8qf4p nHfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622791; x=1741227591; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=w772IaQ5YRbZvSP+H3ufucBFIq7F/07nn6vrGLJgcY8=; b=E+64/F8NOAbRx/PV3j46NTY+zdx25ml9MnGjCuuHuTjdyD49jUfFEgkeHdmtNgPTR+ yIRu4ANdJEH6nPklMyUN4vHYDhWHco4SdTG2DdYju8BrT56bt4gWZnAyaTcxF0DraBVV nvSe/BCP1K6SFnox2Vx6g78Vf/Q8/VozKJ3w0HxercJ72dgCNRZXD2MFpqn2jQmwbaWH dJNpcY8RFdqdjP2qeOLqfm82DcN6nBXcSshGXIo63Vg2oxAcEXqg9MDrK6ZXx8KjWRhw KGIG2x9T+Le0YWYxFwXZZnjDHmS+V6pvP/iV6adWP/hyu39IKqnM7J6DC9pSel/mVu3G 99bw== X-Forwarded-Encrypted: i=1; AJvYcCWkPESRIktiugc7t1lPHpcW7wqm4BrI/TSQHnMPZQ77NASMQCB8PeNrTXaFn2qlxTrAgJodhe0PK24=@lists.xenproject.org X-Gm-Message-State: AOJu0Yy+ZU5S+kmJR83og8NCMQ1qZY06I3fKc2YovMWNb937ivC4UgXk fI6kMzkMyyns7WntspcOs/r5Ae1sRHj1VpDd+s6W0dBnuKbdxKHasci5xpVC5iJJGDrz4+8v449 mYg== X-Google-Smtp-Source: AGHT+IFwaP3Gmshvc7uvq62wN2IryjydNwW4vN0WeuDaKx0968TrkTSYTEdNA9orJIr9gY0PmdxDWYBBhxM= X-Received: from pjbsm1.prod.google.com ([2002:a17:90b:2e41:b0:2f8:4024:b59a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2688:b0:2f9:d0cd:3403 with SMTP id 98e67ed59e1d1-2fea12fcb22mr2427211a91.16.1740622791492; Wed, 26 Feb 2025 18:19:51 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:44 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-29-seanjc@google.com> Subject: [PATCH v2 28/38] x86/paravirt: Mark __paravirt_set_sched_clock() as __init From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Annotate __paravirt_set_sched_clock() as __init, and make its wrapper __always_inline to ensure sanitizers don't result in a non-inline version hanging around. All callers run during __init, and changing sched_clock after boot would be all kinds of crazy. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 9 +++++---- arch/x86/kernel/paravirt.c | 4 ++-- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index dc26a3c26527..e6d5e77753c4 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -28,11 +28,12 @@ u64 dummy_sched_clock(void); DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); -void __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)); +void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)); -static inline void paravirt_set_sched_clock(u64 (*func)(void), - void (*save)(void), void (*restore)(void)) +static __always_inline void paravirt_set_sched_clock(u64 (*func)(void), + void (*save)(void), + void (*restore)(void)) { __paravirt_set_sched_clock(func, true, save, restore); } diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 9673cd3a3f0a..92bf831a63b1 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -86,8 +86,8 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); -void __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)) +void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)) { if (!stable) clear_sched_clock_stable(); From patchwork Thu Feb 27 02:18:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C5FEC021B8 for ; Thu, 27 Feb 2025 02:26:21 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897531.1306306 (Exim 4.92) (envelope-from ) id 1tnTbJ-0003Xj-9o; Thu, 27 Feb 2025 02:26:13 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897531.1306306; Thu, 27 Feb 2025 02:26:13 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbI-0003Uz-Lb; Thu, 27 Feb 2025 02:26:12 +0000 Received: by outflank-mailman (input) for mailman id 897531; Thu, 27 Feb 2025 02:26:11 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVD-0005qU-Gl for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:55 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 549ff9e5-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:19:54 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2f83e54432dso1546794a91.2 for ; Wed, 26 Feb 2025 18:19:54 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 549ff9e5-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622793; x=1741227593; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=P4tBmLUacP8DGxGQBI6X9mo8c2uhJ9G5VZle1I388Hk=; b=zAeu1pwzHmdkhNT40HUtzMwGG5lHKkXCah/BLKvHIVTqaFKFeb1Qtf0yz5eY7L9AXS yln5XE0Gs0PawpvSE3bdcthUsB1DEU3CsEIjSbvSQQJlalnCg34hWpdXLw5mm+sUxM9y vjoctYUNcim/37upo7LCSVsa8QHGsYwvE01EC3CcFoVIsg/rokgzQIW1aHKmRQsTDgoP WqQRSeEeIQOSn+kZoXuc9zAn7Z2vntBtE2rnncvKJypMVL12RJc9mqgzCUn0cA3p/wVt dyreOJDq41uSisnZ0RtoXlbrEhIM87ZuIXN9rx7dZCc2bJvVAtF/x8DgIBH86fHTF0zf UjVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622793; x=1741227593; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=P4tBmLUacP8DGxGQBI6X9mo8c2uhJ9G5VZle1I388Hk=; b=aQlYSuYJ3dxTPVHgD8xHCtdInHGXZ9LkczzhissbOWVO9IejwtEKJaVv5twvVUofvM sfeL5fSNjhf4bRXMkS+T/7zJK795twoaBlZdS06HD0trsybnNuM9yQo0PajLKN8OSnmW 4ggEwrmqYqAShvUb2idtmJeQXG01ujyXpWzBxxDtHqGfWJS+qajuquJgIQZCArTPX8r0 ZxgF8qGwx5Jn0/9JLUKw3tZrsaqG7RF+m3p8KKo5wK5fUCgB74SSWxR8+jYbWn+3vB1m y5lhKutvij4p3pfmGvVBlKI0FDajSi0rSc1VH29ePtmK4bagRRGOuPKNoJcTBkdt40eR EJ/w== X-Forwarded-Encrypted: i=1; AJvYcCXaJShNf+UE6FvSZDTv9dvazB9rNWTje6hEBOSBjKjs51bMU9USPHtXPUrvNa5/kcd95c8XxCnjHw8=@lists.xenproject.org X-Gm-Message-State: AOJu0YzxEwgDomfZMsg+1NfO20itHj6OgQs7FO5ed1lwuKEXsqA3jqIU ops0iIk59mcP2GacbjbzIlVR3pnK/uUDl7+mEgrZdMAn7o39IG92lBINAhnkuDR0OsIg0BR1Wxh UPw== X-Google-Smtp-Source: AGHT+IGdVjOlq0BY70ioN6i4xVGmLOeDmFSQNBlkaPaHsOsARCADALqyiFXLpG70t8VDnA7M1OSlhaskPoo= X-Received: from pjbsw3.prod.google.com ([2002:a17:90b:2c83:b0:2fa:15aa:4d2b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:54cd:b0:2f2:8bdd:cd8b with SMTP id 98e67ed59e1d1-2fe7e3b1756mr8832651a91.29.1740622793310; Wed, 26 Feb 2025 18:19:53 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:45 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-30-seanjc@google.com> Subject: [PATCH v2 29/38] x86/paravirt: Plumb a return code into __paravirt_set_sched_clock() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Add a return code to __paravirt_set_sched_clock() so that the kernel can reject attempts to use a PV sched_clock without breaking the caller. E.g. when running as a CoCo VM with a secure TSC, using a PV clock is generally undesirable. Note, kvmclock is the only PV clock that does anything "extra" beyond simply registering itself as sched_clock, i.e. is the only caller that needs to check the new return value. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 6 +++--- arch/x86/kernel/kvmclock.c | 7 +++++-- arch/x86/kernel/paravirt.c | 5 +++-- 3 files changed, 11 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index e6d5e77753c4..5de31b22aa5f 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -28,14 +28,14 @@ u64 dummy_sched_clock(void); DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); -void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)); +int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)); static __always_inline void paravirt_set_sched_clock(u64 (*func)(void), void (*save)(void), void (*restore)(void)) { - __paravirt_set_sched_clock(func, true, save, restore); + (void)__paravirt_set_sched_clock(func, true, save, restore); } static __always_inline u64 paravirt_sched_clock(void) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 76884dfc77f4..1dbe12ecb26e 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -337,9 +337,12 @@ static int kvmclock_setup_percpu(unsigned int cpu) static void __init kvm_sched_clock_init(bool stable) { + if (__paravirt_set_sched_clock(kvm_sched_clock_read, stable, + kvm_save_sched_clock_state, + kvm_restore_sched_clock_state)) + return; + kvm_sched_clock_offset = kvm_clock_read(); - __paravirt_set_sched_clock(kvm_sched_clock_read, stable, - kvm_save_sched_clock_state, kvm_restore_sched_clock_state); kvmclock_is_sched_clock = true; /* diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 92bf831a63b1..a3a1359cfc26 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -86,8 +86,8 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); -void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)) +int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, + void (*save)(void), void (*restore)(void)) { if (!stable) clear_sched_clock_stable(); @@ -95,6 +95,7 @@ void __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, static_call_update(pv_sched_clock, func); x86_platform.save_sched_clock_state = save; x86_platform.restore_sched_clock_state = restore; + return 0; } /* These are in entry.S */ From patchwork Thu Feb 27 02:18:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993616 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 65119C19776 for ; Thu, 27 Feb 2025 02:26:19 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897522.1306300 (Exim 4.92) (envelope-from ) id 1tnTbH-0003Da-5K; Thu, 27 Feb 2025 02:26:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897522.1306300; Thu, 27 Feb 2025 02:26:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbG-0003D7-TN; Thu, 27 Feb 2025 02:26:10 +0000 Received: by outflank-mailman (input) for mailman id 897522; Thu, 27 Feb 2025 02:26:09 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVG-00063X-0K for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:19:58 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 55b94b10-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:56 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc1a4c14d4so1064519a91.0 for ; Wed, 26 Feb 2025 18:19:56 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 55b94b10-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622795; x=1741227595; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3F6YIZD9bSIlkGuU2KG3Ao1xkSJhhKmqbmLiePXFSIM=; b=Lnlxozbi/yN1o8Viq02FIzcgGRLBgPRx8p6Zpm8qLLfWaZ1q16o82uHa+WMrfwJyR/ /OwNdsSWqBssiky9mrtndvrblBVFFahshAK2jagf+y7Gx6S58gBYqBdu85jno1GlwcvY xTivkPIjiKNzFH7eJRsyQzxijdm/AAU6uyQYuZV3A1zX6hm798SD0W3pz8CaZunBFIFQ 5pPjDqq4LZlxzIxQ4S/saRE1OjVoyg6FNyK/L2jOd/kbY0D9p7wE1dXG+51DNsz8sVT4 99cgkVovaYDzdR9h5VPEi3byJ2XxuWQ4FnQoS5hnzjEZUa+LHiiWs7P7JS2atmUbhvjv hCGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622795; x=1741227595; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3F6YIZD9bSIlkGuU2KG3Ao1xkSJhhKmqbmLiePXFSIM=; b=Pj0OMFL7ZlN/Zc2qcdkkARiQG56prihcokYRddXHpmGBfnCCabP/HwUWPbS7ZBB/2H 9sEGYWYNaigslvW4XlVeHR8Shh9xSpln4mG9BCxHHNZ1p2H8brfBe9GLVQ1Ewyd+K/AT 3R9yJbhVxbBl9NQvH9zdwbCWSuETtk6hOorsWvXSh54uKmA8OdmY3yL4H/O/eoWwy64D +sfGB6bm/4QEmg3i614oXIRtOh5uYTsYlaw3pO09gcsmJ54zIJN+QBZ5bqeGJizYfzVR 0EToreMppz89AzayUvyfBd1ZIrsIMRJka/1c+PkiI6ASjVpuHsqDREXsPXT7lhbIBwfO 5rjA== X-Forwarded-Encrypted: i=1; AJvYcCUZlJtigV5ng+ahUYUIpax0fiA37vQOj4Le/5S/SQlKTz+HYRjKIxlf7oY4iSG/5PwFnL6XogWSJs8=@lists.xenproject.org X-Gm-Message-State: AOJu0Yzz/weqbJc8Rw+korELYe8LaNRQm65aFCrXEB4d22W1SDubp0b0 ZQdgpYK5Lu7S5JTDSDFrFk2dStZ13D48IY5okNtSkp9Db2QrLVT7M61JGRG4JW7ILjhz6wBBGBU rHA== X-Google-Smtp-Source: AGHT+IGJ6z3saIzRbkZlSBZLnZw7mlbnCfDCjtlOA/kyHI3FVoB/Y5gfMuQ1H6WQ1/ZAhh6gzh0XRDRpGl4= X-Received: from pfbfb4.prod.google.com ([2002:a05:6a00:2d84:b0:732:1ead:f8ac]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:1509:b0:1f2:e2b0:dd91 with SMTP id adf61e73a8af0-1f2e2b0ddb7mr3698859637.21.1740622795028; Wed, 26 Feb 2025 18:19:55 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:46 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-31-seanjc@google.com> Subject: [PATCH v2 30/38] x86/paravirt: Don't use a PV sched_clock in CoCo guests with trusted TSC From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Silently ignore attempts to switch to a paravirt sched_clock when running as a CoCo guest with trusted TSC. In hand-wavy theory, a misbehaving hypervisor could attack the guest by manipulating the PV clock to affect guest scheduling in some weird and/or predictable way. More importantly, reading TSC on such platforms is faster than any PV clock, and sched_clock is all about speed. Signed-off-by: Sean Christopherson --- arch/x86/kernel/paravirt.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index a3a1359cfc26..c538c608d9fb 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -89,6 +89,15 @@ DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, void (*save)(void), void (*restore)(void)) { + /* + * Don't replace TSC with a PV clock when running as a CoCo guest and + * the TSC is secure/trusted; PV clocks are emulated by the hypervisor, + * which isn't in the guest's TCB. + */ + if (cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC) || + boot_cpu_has(X86_FEATURE_TDX_GUEST)) + return -EPERM; + if (!stable) clear_sched_clock_stable(); From patchwork Thu Feb 27 02:18:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A2C4FC021B8 for ; Thu, 27 Feb 2025 02:26:03 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897485.1306234 (Exim 4.92) (envelope-from ) id 1tnTb1-00086N-DN; Thu, 27 Feb 2025 02:25:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897485.1306234; Thu, 27 Feb 2025 02:25:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTb0-00082R-Tp; Thu, 27 Feb 2025 02:25:54 +0000 Received: by outflank-mailman (input) for mailman id 897485; Thu, 27 Feb 2025 02:25:52 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVI-00063X-21 for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:20:00 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 56d9a4ba-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:19:58 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc0bc05c00so1509346a91.2 for ; Wed, 26 Feb 2025 18:19:58 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 56d9a4ba-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622797; x=1741227597; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0SoCRyyhI3eHKNBGNMMOnftZcsartKHSJO+ab+uvDMM=; b=Vomz7gNiRPuxis3iVhjpgV4G2NGPHJTxgDnONGLw+4v1cN2JihxRI0+VjCCaI7kekC 7c1vQrAlYoPWoewAfmRLk1X5K9jiwn0dKfqSqtbIOMHsFKS5szawJxW9/Hc34cESfNXY kJnCuAoyxmZ/M4KG8bk6I8lWTP1W5PI0fs7UJ6XFC8YCIWpaMCgH4w+NhlQUUmXBRuM0 03hG4ohFX5+X23bWr022yGoWeu81+nwJf39MF77SjcFEhrmAVXRpG7YZzYgMLtPBG/Hp LJlhrUFsiRBSEu1cOaaJCPwuZDtqlJlPeRwOE5hejNpX8Qg8MOPGtHXctb24JlGOZFbz 66ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622797; x=1741227597; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0SoCRyyhI3eHKNBGNMMOnftZcsartKHSJO+ab+uvDMM=; b=C6JQUpiNstfmxRzeWWlqv5JWtIjnWZPPPnAICzi0Cs8fHc3JegEy3E4d9Ah/jY+ri/ T9hQox6mpo6/0KUlMXXtxkI3lT3Mv8huf4NykDyDTyLpesPw0oNJA/61QZdtyFyW9gpD 5KQumVo9KHWTfx+bntcghDNLY9RMYVd/vn108oiupY8UtxJhS47f7ab33z4J+GwdRlya 55YRvfoY3sXJd4MZ4OqbIEpLKdkzNdotEx0vLTEUQkgy/Oktf3Hx8aSqkHNWL4eVZXmB 5tC7hBOJXNiYTja+eS+yslTf2k7jiqqB9sp9w38Edr98KYFxVoapxYTvsOH9Ab4fj3AJ 2bew== X-Forwarded-Encrypted: i=1; AJvYcCXlT5vyaHrMRwOOcT+far1LiSQzQpPro1eFAZkeHZHEcetX0EJfFpOgBAEDT8YjT5qgjS20yh0LGwA=@lists.xenproject.org X-Gm-Message-State: AOJu0Yz6lmygvlPoVTM1fMWE7M3Po5lgxKS8trEVX2o2qpiG0hVSD+3X j0Hi/wx8ff1URetUljUs1FYskIPu88t10DHNS+BFmMjk6ltSMCrSFQlt1S8Dq18ivgcvGbfWs8N UYA== X-Google-Smtp-Source: AGHT+IGBOkpo7XivcprIZNXQBr8QlNDoqFx2IFaBrMW4afTfcx1rvnIfDlcWOduAP6/zUs4HYAwvxKFyq/w= X-Received: from pjboi12.prod.google.com ([2002:a17:90b:3a0c:b0:2fa:a101:755]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4ec6:b0:2ea:696d:732f with SMTP id 98e67ed59e1d1-2fe692c6ba8mr15211686a91.29.1740622796750; Wed, 26 Feb 2025 18:19:56 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:47 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-32-seanjc@google.com> Subject: [PATCH v2 31/38] x86/tsc: Pass KNOWN_FREQ and RELIABLE as params to registration From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Add a "tsc_properties" set of flags and use it to annotate whether the TSC operates at a known and/or reliable frequency when registering a paravirtual TSC calibration routine. Currently, each PV flow manually sets the associated feature flags, but often in haphazard fashion that makes it difficult for unfamiliar readers to see the properties of the TSC when running under a particular hypervisor. The other, bigger issue with manually setting the feature flags is that it decouples the flags from the calibration routine. E.g. in theory, PV code could mark the TSC as having a known frequency, but then have its PV calibration discarded in favor of a method that doesn't use that known frequency. Passing the TSC properties along with the calibration routine will allow adding sanity checks to guard against replacing a "better" calibration routine with a "worse" routine. As a bonus, the flags also give developers working on new PV code a heads up that they should at least mark the TSC as having a known frequency. Signed-off-by: Sean Christopherson --- arch/x86/coco/sev/core.c | 6 ++---- arch/x86/coco/tdx/tdx.c | 7 ++----- arch/x86/include/asm/tsc.h | 8 +++++++- arch/x86/kernel/cpu/acrn.c | 4 ++-- arch/x86/kernel/cpu/mshyperv.c | 10 +++++++--- arch/x86/kernel/cpu/vmware.c | 7 ++++--- arch/x86/kernel/jailhouse.c | 4 ++-- arch/x86/kernel/kvmclock.c | 4 ++-- arch/x86/kernel/tsc.c | 8 +++++++- arch/x86/xen/time.c | 4 ++-- 10 files changed, 37 insertions(+), 25 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index dab386f782ce..29dd50552715 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -3284,12 +3284,10 @@ void __init snp_secure_tsc_init(void) { unsigned long long tsc_freq_mhz; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - rdmsrl(MSR_AMD64_GUEST_TSC_FREQ, tsc_freq_mhz); snp_tsc_freq_khz = (unsigned long)(tsc_freq_mhz * 1000); tsc_register_calibration_routines(securetsc_get_tsc_khz, - securetsc_get_tsc_khz); + securetsc_get_tsc_khz, + TSC_FREQ_KNOWN_AND_RELIABLE); } diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index 42cdaa98dc5e..ca31560d0dd3 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -1135,14 +1135,11 @@ static unsigned long tdx_get_tsc_khz(void) void __init tdx_tsc_init(void) { - /* TSC is the only reliable clock in TDX guest */ - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); - /* * Override the PV calibration routines (if set) with more trustworthy * CPUID-based calibration. The TDX module emulates CPUID, whereas any * PV information is provided by the hypervisor. */ - tsc_register_calibration_routines(tdx_get_tsc_khz, NULL); + tsc_register_calibration_routines(tdx_get_tsc_khz, NULL, + TSC_FREQ_KNOWN_AND_RELIABLE); } diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h index 9318c74e8d13..360f47610258 100644 --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -41,8 +41,14 @@ extern int cpuid_get_cpu_freq(unsigned int *cpu_khz); extern void tsc_early_init(void); extern void tsc_init(void); #if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) +enum tsc_properties { + TSC_FREQUENCY_KNOWN = BIT(0), + TSC_RELIABLE = BIT(1), + TSC_FREQ_KNOWN_AND_RELIABLE = TSC_FREQUENCY_KNOWN | TSC_RELIABLE, +}; extern void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), - unsigned long (*calibrate_cpu)(void)); + unsigned long (*calibrate_cpu)(void), + enum tsc_properties properties); #endif extern void mark_tsc_unstable(char *reason); extern int unsynchronized_tsc(void); diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index 2da3de4d470e..4f2f4f7ec334 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -29,9 +29,9 @@ static void __init acrn_init_platform(void) /* Install system interrupt handler for ACRN hypervisor callback */ sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); tsc_register_calibration_routines(acrn_get_tsc_khz, - acrn_get_tsc_khz); + acrn_get_tsc_khz, + TSC_FREQUENCY_KNOWN); } static bool acrn_x2apic_available(void) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 174f6a71c899..445ac3adfebc 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -421,8 +421,13 @@ static void __init ms_hyperv_init_platform(void) if (ms_hyperv.features & HV_ACCESS_FREQUENCY_MSRS && ms_hyperv.misc_features & HV_FEATURE_FREQUENCY_MSRS_AVAILABLE) { - tsc_register_calibration_routines(hv_get_tsc_khz, hv_get_tsc_khz); - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + enum tsc_properties tsc_properties = TSC_FREQUENCY_KNOWN; + + if (ms_hyperv.features & HV_ACCESS_TSC_INVARIANT) + tsc_properties = TSC_FREQ_KNOWN_AND_RELIABLE; + + tsc_register_calibration_routines(hv_get_tsc_khz, hv_get_tsc_khz, + tsc_properties); } if (ms_hyperv.priv_high & HV_ISOLATION) { @@ -525,7 +530,6 @@ static void __init ms_hyperv_init_platform(void) * is called. */ wrmsrl(HV_X64_MSR_TSC_INVARIANT_CONTROL, HV_EXPOSE_INVARIANT_TSC); - setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); } /* diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c index 399cf3286a60..a3a71309214c 100644 --- a/arch/x86/kernel/cpu/vmware.c +++ b/arch/x86/kernel/cpu/vmware.c @@ -385,10 +385,10 @@ static void __init vmware_paravirt_ops_setup(void) */ static void __init vmware_set_capabilities(void) { + /* TSC is non-stop and reliable even if the frequency isn't known. */ setup_force_cpu_cap(X86_FEATURE_CONSTANT_TSC); setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); - if (vmware_tsc_khz) - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + if (vmware_hypercall_mode == CPUID_VMWARE_FEATURES_ECX_VMCALL) setup_force_cpu_cap(X86_FEATURE_VMCALL); else if (vmware_hypercall_mode == CPUID_VMWARE_FEATURES_ECX_VMMCALL) @@ -417,7 +417,8 @@ static void __init vmware_platform_setup(void) vmware_tsc_khz = tsc_khz; tsc_register_calibration_routines(vmware_get_tsc_khz, - vmware_get_tsc_khz); + vmware_get_tsc_khz, + TSC_FREQ_KNOWN_AND_RELIABLE); #ifdef CONFIG_X86_LOCAL_APIC /* Skip lapic calibration since we know the bus frequency. */ diff --git a/arch/x86/kernel/jailhouse.c b/arch/x86/kernel/jailhouse.c index b0a053692161..d73a4d0fb118 100644 --- a/arch/x86/kernel/jailhouse.c +++ b/arch/x86/kernel/jailhouse.c @@ -218,7 +218,8 @@ static void __init jailhouse_init_platform(void) machine_ops.emergency_restart = jailhouse_no_restart; - tsc_register_calibration_routines(jailhouse_get_tsc, jailhouse_get_tsc); + tsc_register_calibration_routines(jailhouse_get_tsc, jailhouse_get_tsc, + TSC_FREQUENCY_KNOWN); while (pa_data) { mapping = early_memremap(pa_data, sizeof(header)); @@ -256,7 +257,6 @@ static void __init jailhouse_init_platform(void) pr_debug("Jailhouse: PM-Timer IO Port: %#x\n", pmtmr_ioport); precalibrated_tsc_khz = setup_data.v1.tsc_khz; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); pci_probe = 0; diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 1dbe12ecb26e..ce676e735ced 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -199,7 +199,6 @@ void kvmclock_cpu_action(enum kvm_guest_cpu_action action) */ static unsigned long kvm_get_tsc_khz(void) { - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); return pvclock_tsc_khz(this_cpu_pvti()); } @@ -403,7 +402,8 @@ void __init kvmclock_init(void) kvm_sched_clock_init(stable); - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz); + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, + TSC_FREQUENCY_KNOWN); x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 5501d76243c8..be58df4fef66 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1301,11 +1301,17 @@ static void __init check_system_tsc_reliable(void) */ #if defined(CONFIG_HYPERVISOR_GUEST) || defined(CONFIG_AMD_MEM_ENCRYPT) void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), - unsigned long (*calibrate_cpu)(void)) + unsigned long (*calibrate_cpu)(void), + enum tsc_properties properties) { if (WARN_ON_ONCE(!calibrate_tsc)) return; + if (properties & TSC_FREQUENCY_KNOWN) + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + if (properties & TSC_RELIABLE) + setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + x86_platform.calibrate_tsc = calibrate_tsc; if (calibrate_cpu) x86_platform.calibrate_cpu = calibrate_cpu; diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 13e5888c4501..4de06ea55397 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -40,7 +40,6 @@ static unsigned long xen_tsc_khz(void) struct pvclock_vcpu_time_info *info = &HYPERVISOR_shared_info->vcpu_info[0].time; - setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); return pvclock_tsc_khz(info); } @@ -571,7 +570,8 @@ static void __init xen_init_time_common(void) */ paravirt_set_sched_clock(xen_sched_clock, NULL, NULL); - tsc_register_calibration_routines(xen_tsc_khz, NULL); + tsc_register_calibration_routines(xen_tsc_khz, NULL, + TSC_FREQUENCY_KNOWN); x86_platform.get_wallclock = xen_get_wallclock; } From patchwork Thu Feb 27 02:18:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993600 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20A66C021B8 for ; Thu, 27 Feb 2025 02:25:44 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897452.1306145 (Exim 4.92) (envelope-from ) id 1tnTaf-0004p5-BV; Thu, 27 Feb 2025 02:25:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897452.1306145; Thu, 27 Feb 2025 02:25:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTaf-0004oA-6e; Thu, 27 Feb 2025 02:25:33 +0000 Received: by outflank-mailman (input) for mailman id 897452; Thu, 27 Feb 2025 02:25:32 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVJ-00063X-Iv for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:20:01 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 57d499a2-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:20:00 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fe870bc003so993596a91.1 for ; Wed, 26 Feb 2025 18:20:00 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 57d499a2-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622798; x=1741227598; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=LZJJLySVHQgVgABAB7C0XZjDtBlLYwz9ES+Z1vP/8LE=; b=3YPN1FNhKSXnlzOxt6Xjm0bBGCTIyOASokGc5GWGO5cGMuroXWZo27cm1agedkNm0v 25bObTBqK3lsKKvPtcz2NbvvUcncBwOT/jEzrzi9ueVXUqzvqBsUoWbhTrkuGhzXHAg+ u6hyjXV2m3Nq3PkhnXp0fspzvEhkENptpCakt2k1JI6X8n7Oxr96oteggWp26POL6Yvb E76TxfhFDhOC+xZLvhY41UyH3D8OXdYDMQjxpdmzVEie3XAJV81lliE/TMQAqiMWS3z7 vWidnVoj5f30+4vSM+jhvqKXKtTzyWeu40Q0uRiYEGxRMIlc+Vo8RTUIeKUaS6fngx8S VX9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622798; x=1741227598; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=LZJJLySVHQgVgABAB7C0XZjDtBlLYwz9ES+Z1vP/8LE=; b=l1cDNYBZo4yJlJo9j9OlguNOrpc6Fmi1bwlYGfnNeOHPHEqj7GwpCY+1GYztpS8mxB YRFDGRsR57DVk1d+v8sUMfuje4QMp6UJtk8xgfEozNFPvnTR0moh0U/YXzWkNJMtdjlF o4wFc/LA5FQl/d7DkFHTcP7ZnzdshKm2vPCyq5aOul2tOtSzSQUU8bCh07ssJAVqIiji bohUC/Nexg3aY20jq73TFkvMVD2lJVxTgoBXmSfWb9OAWz3Y44vgdvMwAxTVXkR9J9kH 05Z1+hJxNMcjTpUE8Wth08Jmx1XlptFTVs9M9QNofDN41ErTJNZG8EwPnjemPhAJnB4x QQlw== X-Forwarded-Encrypted: i=1; AJvYcCVb9c0WAjt3aIiNi+N1lotTcGcSnAfiFr6cU6iHr2HhilUu4JT8cY2P/4cDqyvvG22jCTkiGaP9Ny0=@lists.xenproject.org X-Gm-Message-State: AOJu0YwEsB805hooLv1UWv1qbZiCt6jkOIlKKeRgvhuR3ZhKN59nCk+v 7+IiOMRC7lWRpjlHaUPRJ3NQnvYO9VZqNfnLxT+mLXeoYbbliE/KzNeowOUQlhub7Lr9hrbldE+ hAw== X-Google-Smtp-Source: AGHT+IGhmfOgzOy0w88MLax4aQJq7WSM6/RTmeD/HK48z8QB3SAzs8w4kTiAhr4wwBRMQkXZhflq/qsZ2GU= X-Received: from pjtq6.prod.google.com ([2002:a17:90a:c106:b0:2fc:11a0:c53f]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:570c:b0:2fa:2c61:3e5a with SMTP id 98e67ed59e1d1-2fea12c36b0mr2515446a91.10.1740622798574; Wed, 26 Feb 2025 18:19:58 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:48 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-33-seanjc@google.com> Subject: [PATCH v2 32/38] x86/tsc: Rejects attempts to override TSC calibration with lesser routine From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When registering a TSC frequency calibration routine, sanity check that the incoming routine is as robust as the outgoing routine, and reject the incoming routine if the sanity check fails. Because native calibration routines only mark the TSC frequency as known and reliable when they actually run, the effective progression of capabilities is: None (native) => Known and maybe Reliable (PV) => Known and Reliable (CoCo). Violating that progression for a PV override is relatively benign, but messing up the progression when CoCo is involved is more problematic, as it likely means a trusted source of information (hardware/firmware) is being discarded in favor of a less trusted source (hypervisor). Signed-off-by: Sean Christopherson --- arch/x86/kernel/tsc.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index be58df4fef66..ebcfaf7dcd38 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1309,8 +1309,13 @@ void tsc_register_calibration_routines(unsigned long (*calibrate_tsc)(void), if (properties & TSC_FREQUENCY_KNOWN) setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ); + else if (WARN_ON(boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ))) + return; + if (properties & TSC_RELIABLE) setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE); + else if (WARN_ON(boot_cpu_has(X86_FEATURE_TSC_RELIABLE))) + return; x86_platform.calibrate_tsc = calibrate_tsc; if (calibrate_cpu) From patchwork Thu Feb 27 02:18:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8392EC021B8 for ; Thu, 27 Feb 2025 02:25:56 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897470.1306190 (Exim 4.92) (envelope-from ) id 1tnTav-0006mE-Fk; Thu, 27 Feb 2025 02:25:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897470.1306190; Thu, 27 Feb 2025 02:25:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTav-0006lw-Bv; Thu, 27 Feb 2025 02:25:49 +0000 Received: by outflank-mailman (input) for mailman id 897470; Thu, 27 Feb 2025 02:25:48 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVK-0005qU-BD for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:20:02 +0000 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [2607:f8b0:4864:20::104a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 58d77496-f4b1-11ef-9aaf-95dc52dad729; Thu, 27 Feb 2025 03:20:01 +0100 (CET) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2fe98fad333so1066307a91.2 for ; Wed, 26 Feb 2025 18:20:01 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 58d77496-f4b1-11ef-9aaf-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622800; x=1741227600; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SUWURP6DS00BmWQuQkka/ewYMygd0uN/XlEQWE3xarE=; b=1SwvhmWWtXw3Y02jVM9OXi2DMHZlOTX4lVGk580BiDJ/dZX7fDCIwYBIVS3mdltBb+ g/62CiA95WoicGEGR5sAjmaunnDYtD0NDoS6W7LxbGW6m1l4FF8rZz2+hPtPL08F84xd psacq1srYs4V1EL+lT1/4wmpGvrkCBJTXsmeIx4AiftR0/Y/lype2h2D89nGZYzSINbJ L5XuKQpLxGrQb3GAV5WTOPXHHelAAro9bYe7wV9pYAzvo/ZDPusMGl2zjBe5k18WeLkf BLYhrXBQnckWYYgZDF8khDtvAGYoUc1XE18YgGvaaKJUmAcWRcjOOVtoow7h3MQhw7BS p1Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622800; x=1741227600; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SUWURP6DS00BmWQuQkka/ewYMygd0uN/XlEQWE3xarE=; b=JK3oLNRQGC7oFhiscrR/yBiz7DvT78cVv1YgtQPKwLQn6bB6pluUxY5ruLYJXdQAL+ 4f4TmHSaNzpeuzZnu88nvpQrxBke9txCB3ixFHfFMoxe8WXD719blpSgcSfjQ4CHrX+T YVx8JFpMb1crYay1cVOODmpNoXnxPF5Guc9Kt6OO54fLoSyE15gJpZDaVXjnDQmN25g9 dHEDVTYjhSI4GCoj2eH8YwEJrnM/FB9JSeMpeSL5G5/+yNUZNYMmraDwEBF/Zo1Avn46 cQJrhDUfbTGQ5UjfHl+wDj7z2F6vF7C0vmvC3AqTTszrxoJ5H/5OncesD3Z/9uC8MGOY y+IA== X-Forwarded-Encrypted: i=1; AJvYcCVJgNtI712FRPty8INGVHuEmIqiMuepl5jNzemvIR/lsrzmIp3oNPX1RXGzW2sEwPCa70DMRA9JSeM=@lists.xenproject.org X-Gm-Message-State: AOJu0YyKlkQ0mfD/G6iaI+wzYjd3u0HsuksRPP4fvdAqwA2ery3CvWaB SU+aS1wTda+64ZwbqRYBTr7vWECCag8Er8TYsJSzLBrCwDvOTRsUotTTZ500ig0q7uAaj+ludS5 rUA== X-Google-Smtp-Source: AGHT+IEaJegSwuz7gp17gUBIq0YV/a5p90hmpWu2RxMMtE/LMlRmTHKg61dycPOdBb/l8JsfSEdKtOPrkAY= X-Received: from pjboh8.prod.google.com ([2002:a17:90b:3a48:b0:2fc:2b96:2d4b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4fcf:b0:2f8:34df:5652 with SMTP id 98e67ed59e1d1-2fce78beb41mr33366155a91.21.1740622800327; Wed, 26 Feb 2025 18:20:00 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:49 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-34-seanjc@google.com> Subject: [PATCH v2 33/38] x86/kvmclock: Mark TSC as reliable when it's constant and nonstop From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Mark the TSC as reliable if the hypervisor (KVM) has enumerated the TSC as constant and nonstop, and the admin hasn't explicitly marked the TSC as unstable. Like most (all?) virtualization setups, any secondary clocksource that's used as a watchdog is guaranteed to be less reliable than a constant, nonstop TSC, as all clocksources the kernel uses as a watchdog are all but guaranteed to be emulated when running as a KVM guest. I.e. any observed discrepancies between the TSC and watchdog will be due to jitter in the watchdog. This is especially true for KVM, as the watchdog clocksource is usually emulated in host userspace, i.e. reading the clock incurs a roundtrip cost of thousands of cycles. Marking the TSC reliable addresses a flaw where the TSC will occasionally be marked unstable if the host is under moderate/heavy load. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index ce676e735ced..b924b19e8f0f 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -362,6 +362,7 @@ static void __init kvm_sched_clock_init(bool stable) void __init kvmclock_init(void) { + enum tsc_properties tsc_properties = TSC_FREQUENCY_KNOWN; bool stable = false; if (!kvm_para_available() || !kvmclock) @@ -400,18 +401,6 @@ void __init kvmclock_init(void) PVCLOCK_TSC_STABLE_BIT; } - kvm_sched_clock_init(stable); - - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, - TSC_FREQUENCY_KNOWN); - - x86_platform.get_wallclock = kvm_get_wallclock; - x86_platform.set_wallclock = kvm_set_wallclock; -#ifdef CONFIG_SMP - x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; -#endif - kvm_get_preset_lpj(); - /* * X86_FEATURE_NONSTOP_TSC is TSC runs at constant rate * with P/T states and does not stop in deep C-states. @@ -422,8 +411,22 @@ void __init kvmclock_init(void) */ if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && - !check_tsc_unstable()) + !check_tsc_unstable()) { kvm_clock.rating = 299; + tsc_properties = TSC_FREQ_KNOWN_AND_RELIABLE; + } + + kvm_sched_clock_init(stable); + + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, + tsc_properties); + + x86_platform.get_wallclock = kvm_get_wallclock; + x86_platform.set_wallclock = kvm_set_wallclock; +#ifdef CONFIG_SMP + x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; +#endif + kvm_get_preset_lpj(); clocksource_register_hz(&kvm_clock, NSEC_PER_SEC); pv_info.name = "KVM"; From patchwork Thu Feb 27 02:18:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993615 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 13BF3C021B8 for ; Thu, 27 Feb 2025 02:26:14 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897516.1306287 (Exim 4.92) (envelope-from ) id 1tnTbC-0002IE-Gs; Thu, 27 Feb 2025 02:26:06 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897516.1306287; Thu, 27 Feb 2025 02:26:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTbC-0002Gt-6b; Thu, 27 Feb 2025 02:26:06 +0000 Received: by outflank-mailman (input) for mailman id 897516; Thu, 27 Feb 2025 02:26:05 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVN-00063X-6C for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:20:05 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 59fa727e-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:20:03 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc45101191so1080596a91.1 for ; Wed, 26 Feb 2025 18:20:03 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 59fa727e-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622802; x=1741227602; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=evC/AJYAKH4bntjSNStZX5d0W0KQw/JlJZazMUftcQU=; b=uBuIornD6TSt8bcrWwRHfRoXTQplgjGgDYnnfh/DBHr0KMLKThKoE+UrGe9l4GC4eI gvYCQ8oezfU5TE/PTmk9d9th8oWTFDPQFeIDdldcJVCJovXK+hoa729mQwR5Q5M+ALW1 Jqw/yVzSRMkkJbPg/LJZPXwKJsdZoELc/YD38s16JTBzQuEAhDXEE1vaZ0e2Q3fTUDys EevG7oDVCB//+/RfLtRR914M7u4CkW9F6eEGeHUP/HkVyZxSGWgnZpyPx4HCNjZFwuXt dKM5MdD76LfE1EkuojbUx2YncnJcqYi1NufZxGBfiPFjxAQ9IxS1+XArMQfiYV0j8sd/ AEeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622802; x=1741227602; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=evC/AJYAKH4bntjSNStZX5d0W0KQw/JlJZazMUftcQU=; b=lbuuvmpfuQWNadWzgz2L5NFolNnwB6YefvAmyy2nU8qm8J3CTDCbE/mtfzNziCVyWl gBnVJRKqQybaQYWaYmxj+obPNqYICrkH7kh4tGIPZ3MDknz9JpFNhr4VwDfQXCLa5Rg4 iFn4yQLEYD4cpk6oPpLhXE6gwlAq6iIUblfPw3jDPL0eSrBYBozTYu3A8EO1Ek6+qEwg ftpXe6p4MvCwqK2c0z0B0viPP6BjvB+g7XusBugFSoEQ2HmG3M2Szi9nbxfjN05eakue ATI6ULSoIP9kDq8Oot2SikV09V1lf5PsoRrnAaRWC6okh1iKMeDvAZD8wEkk3Pt4Pkcb emhw== X-Forwarded-Encrypted: i=1; AJvYcCVSKLxdsJjbGn/dwAJDa8qrAO8O1HbgFX+EBa/DM9mTF+OKUbL131qwR5/VDwalgbQ2wr7cg+BCrvc=@lists.xenproject.org X-Gm-Message-State: AOJu0YwpbTOLLxe5OrnYYvyPkALP2tTv9xkiy8zJNbLrRsf9EwGi7lWV JfqQW4T4Og3e6GtOh8+Y/ke44/PphXNK5OtPbCkAoPFXD8IO6/o7p8jT41yrMZwyRd0c1+3+gTb D2A== X-Google-Smtp-Source: AGHT+IGv/zpuplWrcklhWH3aZ9H+bWJRwaJtozM+byhaXyymURPHFd4oTcq5U1mZsa54ViZesa4/C8fD/pk= X-Received: from pjvb12.prod.google.com ([2002:a17:90a:d88c:b0:2ea:aa56:49c]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5101:b0:2fe:955d:cdb1 with SMTP id 98e67ed59e1d1-2fe955dd224mr3596029a91.23.1740622802051; Wed, 26 Feb 2025 18:20:02 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:50 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-35-seanjc@google.com> Subject: [PATCH v2 34/38] x86/kvmclock: Get CPU base frequency from CPUID when it's available From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania If CPUID.0x16 is present and valid, use the CPU frequency provided by CPUID instead of assuming that the virtual CPU runs at the same frequency as TSC and/or kvmclock. Back before constant TSCs were a thing, treating the TSC and CPU frequencies as one and the same was somewhat reasonable, but now it's nonsensical, especially if the hypervisor explicitly enumerates the CPU frequency. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index b924b19e8f0f..c45b321533e5 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -188,6 +188,20 @@ void kvmclock_cpu_action(enum kvm_guest_cpu_action action) } } +static unsigned long kvm_get_cpu_khz(void) +{ + unsigned int cpu_khz; + + /* + * Prefer CPUID over kvmclock when possible, as the base CPU frequency + * isn't necessarily the same as the kvmlock "TSC" frequency. + */ + if (!cpuid_get_cpu_freq(&cpu_khz)) + return cpu_khz; + + return pvclock_tsc_khz(this_cpu_pvti()); +} + /* * If we don't do that, there is the possibility that the guest * will calibrate under heavy load - thus, getting a lower lpj - @@ -418,7 +432,7 @@ void __init kvmclock_init(void) kvm_sched_clock_init(stable); - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_tsc_khz, + tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_cpu_khz, tsc_properties); x86_platform.get_wallclock = kvm_get_wallclock; From patchwork Thu Feb 27 02:18:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993612 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E3A05C021B8 for ; Thu, 27 Feb 2025 02:26:07 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897501.1306255 (Exim 4.92) (envelope-from ) id 1tnTb5-0000dV-96; Thu, 27 Feb 2025 02:25:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897501.1306255; Thu, 27 Feb 2025 02:25:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTb4-0000bA-S1; Thu, 27 Feb 2025 02:25:58 +0000 Received: by outflank-mailman (input) for mailman id 897501; Thu, 27 Feb 2025 02:25:58 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVO-00063X-U9 for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:20:06 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 5b08f7d6-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:20:05 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fce2954a10so1537182a91.1 for ; Wed, 26 Feb 2025 18:20:05 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5b08f7d6-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622804; x=1741227604; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=tqdKNUZqbb8d2CJvLoLFEQfT4OY26dkczDpR+rA3UIo=; b=h+jDYWr79d7/qby+mpfQpw9mTI+8xzZ6SpHtXzcjKgGdrRVStaQiM7E/5YSZfpICoB 4/Dx7wUFR8RGR/gFdWD5YqioUEf3e+txRiRtX0zPyl7TEeNaVmOmVyUUga95J6OkpIbh Ej6a33+mHuAzJFYLw6dA5DK9sgDoDpqL6OFJdmuZdwgBEtSufWhZupZOVVsmcw4oHGEy 3RfhKJP/jnuQmndynztU/cKBPg9eqe6I78vr9lt8AzJ2Je6xMSJizUjFBEsBGac1yAIh PEplV9IzBnoK/MGaqzR+b1maDX35xWVt1ubYIxEDs3WxA4OGkIDhMbkmnKe29ShTWXzl qRmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622804; x=1741227604; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tqdKNUZqbb8d2CJvLoLFEQfT4OY26dkczDpR+rA3UIo=; b=A5O5t5idWdt2JpK9ceBRwAa5R++e6MdgCtsyyvPQCh3XoOocSZF2hJDPXQBVszSRcm tTnyw6rBhRArRVdQLylGDSBC83/dsO7/wZ3DpdruPs7YtPwcarMXj/37yWf+d8muvaDs FNkJ0xZtyRQRbmtZrbwcYORGTymxAuQ7OemDrHngPmPE77SidVX5vDTy+qE3rcVFw+2/ 3do3nEYpADUoqczMokwubF8rVKMePXQEz+OjxjA1sLRI26hiEa/GvkgMJrYk8GBNE4Sq Z7OhQvFqcu/RBTGLCvrdVcuOZLy7pKUByqpkra5TXlGZTg4xd1MhzWZ7mbfXiViad6jq lvdg== X-Forwarded-Encrypted: i=1; AJvYcCVPxK5vhjabS26h9x/PB0GZJO6Y0V7iIMjxs/d09eARA8QMNPWNtY4GzXbaLC5vlCuTwSyT1JxNW5g=@lists.xenproject.org X-Gm-Message-State: AOJu0YxNviiD+QkpHl4zcZT2AYBwJkifSXS28ISF6UEtZkIEHxeH03mT 5uZgylk+tiRnZ5Z2CHYob8b+ECq+1hkNLV4FhPHMFe+/cTbAhKODG9gXj9cr6/3ylpdpob3lQEb 06A== X-Google-Smtp-Source: AGHT+IGMz3bH4S0o1xgsITZANeNn+Z/tGuVwcYyVyvqnl2NMWrRx99tESpyiBKte9PgB4aGO5q0vwXh4uHk= X-Received: from pjuj3.prod.google.com ([2002:a17:90a:d003:b0:2fc:b544:749e]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3c84:b0:2fc:c262:ef4b with SMTP id 98e67ed59e1d1-2fce86cf0e0mr42953377a91.18.1740622803772; Wed, 26 Feb 2025 18:20:03 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:51 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-36-seanjc@google.com> Subject: [PATCH v2 35/38] x86/kvmclock: Get TSC frequency from CPUID when its available From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When kvmclock and CPUID.0x15 are both present, use the TSC frequency from CPUID.0x15 instead of kvmclock's frequency. Barring a misconfigured setup, both sources should provide the same frequency, CPUID.0x15 is arguably a better source when using the TSC over kvmclock, and most importantly, using CPUID.0x15 will allow stuffing the local APIC timer frequency based on the core crystal frequency, i.e. will allow skipping APIC timer calibration. Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index c45b321533e5..3efb837c7406 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -188,6 +188,16 @@ void kvmclock_cpu_action(enum kvm_guest_cpu_action action) } } +static unsigned long kvm_get_tsc_khz(void) +{ + struct cpuid_tsc_info info; + + if (!cpuid_get_tsc_freq(&info)) + return info.tsc_khz; + + return pvclock_tsc_khz(this_cpu_pvti()); +} + static unsigned long kvm_get_cpu_khz(void) { unsigned int cpu_khz; @@ -211,11 +221,6 @@ static unsigned long kvm_get_cpu_khz(void) * poll of guests can be running and trouble each other. So we preset * lpj here */ -static unsigned long kvm_get_tsc_khz(void) -{ - return pvclock_tsc_khz(this_cpu_pvti()); -} - static void __init kvm_get_preset_lpj(void) { unsigned long khz; From patchwork Thu Feb 27 02:18:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 64DF8C19776 for ; Thu, 27 Feb 2025 02:25:43 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897453.1306160 (Exim 4.92) (envelope-from ) id 1tnTah-0005Ix-M9; Thu, 27 Feb 2025 02:25:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897453.1306160; Thu, 27 Feb 2025 02:25:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTah-0005Iq-IO; Thu, 27 Feb 2025 02:25:35 +0000 Received: by outflank-mailman (input) for mailman id 897453; Thu, 27 Feb 2025 02:25:34 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVQ-00063X-Jk for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:20:08 +0000 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [2607:f8b0:4864:20::649]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 5c06baf9-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:20:07 +0100 (CET) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-220ee2e7746so7721785ad.2 for ; Wed, 26 Feb 2025 18:20:07 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5c06baf9-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622805; x=1741227605; darn=lists.xenproject.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=pIteN/a7AZFre1KWgiuGeMT9S5HjiwYxYNebWBQLyrA=; b=asI0nJNsczcNUCxSFHXPsDZW5yIxjZbWtUPYBB3Zyz57F0cPXyGvxhsO5Cu63Yn6Q9 j5faga0kCgcTSf/SdfWN5vQP55ui/cXocAH8N0OSfT2my1LipIYxDPr7OiyN2vAei0uj HE3XxCxTfZY4ZgMGQh4BWceHMBrApupJnUwR4yIfcLrE3+HnlUzlyXPG4RpAZ8PhP4Vs 5MTXSUyDnLYUB/l/95fp+Yys/PavTLTUTFfwWEZsEwoZyf/JpxuYlynEloUpYwJ4Ets1 kvc7ZbNi0rvL0Uvg7QyycC+8UYWKX9+eLGpJLgWtPETAgkECts/WqcwHcPrSXPiKAPaW N3dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622805; x=1741227605; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=pIteN/a7AZFre1KWgiuGeMT9S5HjiwYxYNebWBQLyrA=; b=drOLvk+9Uq7N5FZNh0+1m0+8YN8WOQH4MZNM98kS3pIaCWb02k0avevx6YleLyGJQN 2YpgLFubIQ2jWeobotP1e4qntoinVOvhGPm15kNWjhtuPgy+ToQOu0l4mHQyRqUbAnKh zTYoRoAsZ2EODqpDGAt24Z75Nb1BkQ4pvrs139kThp9lvCzDb6XuzNju/gvUQdu7ZCPT B9Ns2qpXNXLvKLYBJzSfZnLR4+C/eWzQejGEK8Q8C2QHAvxUy8xlVTZ+4eXI6xYMP79Q PE6c9bANUd9NNmK9DTH/wND43LmVBBGjpdKgIKryJbL5utmFmmSZLN9yenjPZgqFWefv BYyg== X-Forwarded-Encrypted: i=1; AJvYcCWnf9ISgiKlISTmCJiz3PQn5YeywOkdUjm901EFely+BLLY0OFddCP+tjM1VEU7CHQfyjhUXt1kPt4=@lists.xenproject.org X-Gm-Message-State: AOJu0Yw2BqPPB/sJrdi9ITTGJIxoAKfIewffcrB4lLb6nafNXKZuLeKP NyWA0AyCn3RNt4C9VD11355OYPuQ+CC8arma5NSKrBYTSe4gpKixVfEt99lD3Hxt0EUvQ+tG0qG DQg== X-Google-Smtp-Source: AGHT+IGP2cZX6cw4gCRMv2fLrFe/TXNI5NPzTOobUK44ADJodiEA9LQbSXwMRfyGc0rCQ4WOggdSpeUhPyc= X-Received: from pfhk13.prod.google.com ([2002:aa7:998d:0:b0:730:479d:3482]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2181:b0:731:ff1b:dd6a with SMTP id d2e1a72fcca58-7348be4c455mr8342588b3a.20.1740622805598; Wed, 26 Feb 2025 18:20:05 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:52 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-37-seanjc@google.com> Subject: [PATCH v2 36/38] x86/kvmclock: Stuff local APIC bus period when core crystal freq comes from CPUID From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania When running as a KVM guest with kvmclock support enabled, stuff the APIC timer period/frequency with the core crystal frequency from CPUID.0x15 (if CPUID.0x15 is provided). KVM's ABI adheres to Intel's SDM, which states that the APIC timer runs at the core crystal frequency when said frequency is enumerated via CPUID.0x15. The APIC timer frequency will be the processor’s bus clock or core crystal clock frequency (when TSC/core crystal clock ratio is enumerated in CPUID leaf 0x15). Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 3efb837c7406..80d9c86e0671 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -192,8 +192,18 @@ static unsigned long kvm_get_tsc_khz(void) { struct cpuid_tsc_info info; - if (!cpuid_get_tsc_freq(&info)) + /* + * Prefer CPUID over kvmclock when possible, as CPUID also includes the + * core crystal frequency, i.e. the APIC timer frequency. When the core + * crystal frequency is enumerated in CPUID.0x15, KVM's ABI is that the + * (virtual) APIC BUS runs at the same frequency. + */ + if (!cpuid_get_tsc_freq(&info)) { +#ifdef CONFIG_X86_LOCAL_APIC + lapic_timer_period = info.crystal_khz * 1000 / HZ; +#endif return info.tsc_khz; + } return pvclock_tsc_khz(this_cpu_pvti()); } From patchwork Thu Feb 27 02:18:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EA895C021B8 for ; Thu, 27 Feb 2025 02:25:50 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897459.1306170 (Exim 4.92) (envelope-from ) id 1tnTap-0005uq-TJ; Thu, 27 Feb 2025 02:25:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897459.1306170; Thu, 27 Feb 2025 02:25:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTap-0005uf-Pe; Thu, 27 Feb 2025 02:25:43 +0000 Received: by outflank-mailman (input) for mailman id 897459; Thu, 27 Feb 2025 02:25:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVS-00063X-GW for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:20:10 +0000 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [2607:f8b0:4864:20::1049]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 5d2b23a0-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:20:09 +0100 (CET) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2fc1cb0c2cbso1527740a91.1 for ; Wed, 26 Feb 2025 18:20:09 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5d2b23a0-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622807; x=1741227607; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KI0queLIxZnghHgRHs7xWlDt+M3CNsR2JqWjVaFF+mE=; b=uVuGAREo7eudcx3WRAiYRQAv2yvnGG+/676mWDEB8T3RRIQQ697WvO8vSXRdhYPPNX mwkwv00lv9So2O/vvxgNSxQU4RBmW4GQBplf3dUK8EAFqj6trThaTP1tsfQ+uDiHEUDB x/U8F/zJIBvXGPMu18yVA4eDKjt5KbrINKJL2o3VObPsi8hQb1MB34V1Q7WplbP7rYYJ 9Xc7+1TDH8V7eVWUVVu2vxuw1ILSxG6tXIs7BhGw7fu+I8ybmP1pWv84vRTUZGGNgV6w Mup2sLVY/Dmk6FoAr/sOWGYKkyG/Se0Q8XpmAtmiFF1vcfdurUQhOOPpUoZdS9XQQ2yR krtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622807; x=1741227607; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KI0queLIxZnghHgRHs7xWlDt+M3CNsR2JqWjVaFF+mE=; b=jpG/IGhqg+nU9lvOEOKn/HFNTg4KQPgi2/VkllZ2QSUWp3cVhvZlaMzmsZTdbjyINT zvwPiGcfgapkBsneAXtfCmBmISRF4MXx0MrqZaFFngwXgwr/lorcD/4UArjggOkgBRVu Nva92i95Wtmg0b2tsneEmMl/WQf2x48DWKL8XmS9JY1VQc0M5x+kSwvqoGy4Ekp716tH EUx54ohTB3psqFQR+0SWgdV84Q/7vIGIIUe+yH7H0TdkZCxx5fdZVwKOZT1l+n6zqwGH Mf5PYe6LoMkWd+cYakRGxtBTt+G/M9Serfb0CTbCDjcnKgitlb9fKpdwNiPpAf6Om/Mv 5aYw== X-Forwarded-Encrypted: i=1; AJvYcCUWCBm31Vem7+U+W5RCCpkifoyVh/iuzSDKxtHlHUF2PkHjumNYiY92avc22LFG00Ls+FJbru+1NNU=@lists.xenproject.org X-Gm-Message-State: AOJu0YxWVo0JLGOoMqqKrIlVpivsJsern6fhAPFukeF93JRVo63r7Yn2 8meH5O7Fg5qCjSOYHoLf6xhLOwx8Bil5NZPMqp+rGyIH1FZkWAWzv6vqy5SlXYFa6EN3eytkyv1 jCA== X-Google-Smtp-Source: AGHT+IFuRPRb2/rR+n2YIE9T1JgT09dE9pwcHYDYXuVdPUREy4VCpJNwziBTuQzC841m2IS5b8M/SH3vvo0= X-Received: from pjbsf13.prod.google.com ([2002:a17:90b:51cd:b0:2fc:e37d:85dc]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3d50:b0:2ee:9b2c:3253 with SMTP id 98e67ed59e1d1-2fe692c6c47mr14798005a91.30.1740622807422; Wed, 26 Feb 2025 18:20:07 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:53 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-38-seanjc@google.com> Subject: [PATCH v2 37/38] x86/kvmclock: Use TSC for sched_clock if it's constant and non-stop From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Prefer the TSC over kvmclock for sched_clock if the TSC is constant, nonstop, and not marked unstable via command line. I.e. use the same criteria as tweaking the clocksource rating so that TSC is preferred over kvmclock. Per the below comment from native_sched_clock(), sched_clock is more tolerant of slop than clocksource; using TSC for clocksource but not sched_clock makes little to no sense, especially now that KVM CoCo guests with a trusted TSC use TSC, not kvmclock. /* * Fall back to jiffies if there's no TSC available: * ( But note that we still use it if the TSC is marked * unstable. We do this because unlike Time Of Day, * the scheduler clock tolerates small errors and it's * very important for it to be as fast as the platform * can achieve it. ) */ The only advantage of using kvmclock is that doing so allows for early and common detection of PVCLOCK_GUEST_STOPPED, but that code has been broken for nearly two years with nary a complaint, i.e. it can't be _that_ valuable. And as above, certain types of KVM guests are losing the functionality regardless, i.e. acknowledging PVCLOCK_GUEST_STOPPED needs to be decoupled from sched_clock() no matter what. Link: https://lore.kernel.org/all/Z4hDK27OV7wK572A@google.com Signed-off-by: Sean Christopherson --- arch/x86/kernel/kvmclock.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 80d9c86e0671..280bb964f30a 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -431,22 +431,22 @@ void __init kvmclock_init(void) } /* - * X86_FEATURE_NONSTOP_TSC is TSC runs at constant rate - * with P/T states and does not stop in deep C-states. - * - * Invariant TSC exposed by host means kvmclock is not necessary: - * can use TSC as clocksource. - * + * If the TSC counts at a constant frequency across P/T states, counts + * in deep C-states, and the TSC hasn't been marked unstable, prefer + * the TSC over kvmclock for sched_clock and drop kvmclock's rating so + * that TSC is chosen as the clocksource. Note, the TSC unstable check + * exists purely to honor the TSC being marked unstable via command + * line, any runtime detection of an unstable will happen after this. */ if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && !check_tsc_unstable()) { kvm_clock.rating = 299; tsc_properties = TSC_FREQ_KNOWN_AND_RELIABLE; + } else { + kvm_sched_clock_init(stable); } - kvm_sched_clock_init(stable); - tsc_register_calibration_routines(kvm_get_tsc_khz, kvm_get_cpu_khz, tsc_properties); From patchwork Thu Feb 27 02:18:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13993602 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6FB3EC1B087 for ; Thu, 27 Feb 2025 02:25:45 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.897451.1306139 (Exim 4.92) (envelope-from ) id 1tnTaf-0004mE-2F; Thu, 27 Feb 2025 02:25:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 897451.1306139; Thu, 27 Feb 2025 02:25:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTae-0004m7-Vz; Thu, 27 Feb 2025 02:25:32 +0000 Received: by outflank-mailman (input) for mailman id 897451; Thu, 27 Feb 2025 02:25:32 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tnTVU-00063X-4P for xen-devel@lists.xenproject.org; Thu, 27 Feb 2025 02:20:12 +0000 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [2607:f8b0:4864:20::104a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 5e1655f0-f4b1-11ef-9898-31a8f345e629; Thu, 27 Feb 2025 03:20:10 +0100 (CET) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2fc0bc05c00so1509821a91.2 for ; Wed, 26 Feb 2025 18:20:10 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5e1655f0-f4b1-11ef-9898-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740622809; x=1741227609; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=7QXAixlOaLHnLrPcLZCTebtwCxVmNNMI9kZmebDzK/g=; b=RR1cZhmPh9YcgkXBlnEpoiWzI6uPs+ZcsjKABn8RSLVv+UnRzxoL9OFQdXusrkqbG0 zm2LNt1U2m5sec7EUyL4pqDz6WwN7slWdHnMM5IzQBdh5t4o0L4koJ2jGi2RDZDnXVlQ rO8YUY7qfJ9F5YcMeXBBLFgTsbRu12xpGkq0ildrIMio2rvt25ROHo2t+AhS+3CcFcaK Z8sy661HuGC/8jl76iA44E9UreA/0mVaR01QKy/AJADfyjq8C26nwXzI+D06r+D54NFv 4qHbOsaIAIwey3JcS68BBY0N5CRoGpNz94G+hSvb1BTqZDT2jND5PImRtKWWq2ZrwTnK ls3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740622809; x=1741227609; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7QXAixlOaLHnLrPcLZCTebtwCxVmNNMI9kZmebDzK/g=; b=VGnGZLF9sxIp3zyyH9qLEdKxeDjPNu1npNvOysR4VRU43i6DwxHfj9/TidCeXbZNq5 sPnioU8BVbKSa3GMNhtMv+k+I71nXxKKW8Ev5UGR2Ibd/mk5b/bXIJrWVTh/GAq02ErM OAVEwYdgwahz0flbOJDIMtpDoO7cM3RxNHUL6L2x3O1L1g8yKFeB025C+K/hcnzFm+xU eSpqyIuR56yVhnwty3aALezlh7HMFvgrPyI1uqFwX224MmZL0Oq5B1rN/w0G2nTfCXA5 rbGzzc+6kDmwjoVGvxPBCX/z2A2Zb4IIw/Zes600xkaWk1daL8NzHZIREca4TIbkskQz y6Fw== X-Forwarded-Encrypted: i=1; AJvYcCW/9JOsSN4Ce5TmShP1umvaUxPeSHvF2ZZC0nFNj6WXMfQmjsT8RcfkoabX/TvGxdWlJqEm7z9/rfc=@lists.xenproject.org X-Gm-Message-State: AOJu0YzBZAoOfPWprWa35s9JKaAz/lD/e/RhOSPU5EGFmNw2TiQxAqms j65d+p7wyktFxd/3sdR2G3HIBlSgrCEwK8WCU/T0c+lpCJav8wmhv9KJ/+56JgSocWZo0Vopj8Z HHw== X-Google-Smtp-Source: AGHT+IHi4d1QOUfXEPfDRjR4DxqWNosFyXVnkFi4SDlf4Oxmtkdi3La4kP/2IDxMG83GHnzWpK766unKU5E= X-Received: from pjz6.prod.google.com ([2002:a17:90b:56c6:b0:2fc:3022:36b8]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5688:b0:2ee:d63f:d77 with SMTP id 98e67ed59e1d1-2fe68adec61mr16362401a91.9.1740622809153; Wed, 26 Feb 2025 18:20:09 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 26 Feb 2025 18:18:54 -0800 In-Reply-To: <20250227021855.3257188-1-seanjc@google.com> Mime-Version: 1.0 References: <20250227021855.3257188-1-seanjc@google.com> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <20250227021855.3257188-39-seanjc@google.com> Subject: [PATCH v2 38/38] x86/paravirt: kvmclock: Setup kvmclock early iff it's sched_clock From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "Kirill A. Shutemov" , Paolo Bonzini , Sean Christopherson , Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Ajay Kaher , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Daniel Lezcano , John Stultz Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania Rework the seemingly generic x86_cpuinit_ops.early_percpu_clock_init hook into a dedicated PV sched_clock hook, as the only reason the hook exists is to allow kvmclock to enable its PV clock on secondary CPUs before the kernel tries to reference sched_clock, e.g. when grabbing a timestamp for printk. Rearranging the hook doesn't exactly reduce complexity; arguably it does the opposite. But as-is, it's practically impossible to understand *why* kvmclock needs to do early configuration. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/paravirt.h | 10 ++++++++-- arch/x86/include/asm/x86_init.h | 2 -- arch/x86/kernel/kvmclock.c | 13 ++++++------- arch/x86/kernel/paravirt.c | 18 +++++++++++++++++- arch/x86/kernel/smpboot.c | 2 +- arch/x86/kernel/x86_init.c | 1 - 6 files changed, 32 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index 5de31b22aa5f..8550262fc710 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -29,13 +29,14 @@ DECLARE_STATIC_CALL(pv_steal_clock, dummy_steal_clock); DECLARE_STATIC_CALL(pv_sched_clock, dummy_sched_clock); int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)); + void (*save)(void), void (*restore)(void), + void (*start_secondary)); static __always_inline void paravirt_set_sched_clock(u64 (*func)(void), void (*save)(void), void (*restore)(void)) { - (void)__paravirt_set_sched_clock(func, true, save, restore); + (void)__paravirt_set_sched_clock(func, true, save, restore, NULL); } static __always_inline u64 paravirt_sched_clock(void) @@ -43,6 +44,8 @@ static __always_inline u64 paravirt_sched_clock(void) return static_call(pv_sched_clock)(); } +void paravirt_sched_clock_start_secondary(void); + struct static_key; extern struct static_key paravirt_steal_enabled; extern struct static_key paravirt_steal_rq_enabled; @@ -756,6 +759,9 @@ void native_pv_lock_init(void) __init; static inline void native_pv_lock_init(void) { } +static inline void paravirt_sched_clock_start_secondary(void) +{ +} #endif #endif /* !CONFIG_PARAVIRT */ diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h index 213cf5379a5a..e3456def5aea 100644 --- a/arch/x86/include/asm/x86_init.h +++ b/arch/x86/include/asm/x86_init.h @@ -187,13 +187,11 @@ struct x86_init_ops { /** * struct x86_cpuinit_ops - platform specific cpu hotplug setups * @setup_percpu_clockev: set up the per cpu clock event device - * @early_percpu_clock_init: early init of the per cpu clock event device * @fixup_cpu_id: fixup function for cpuinfo_x86::topo.pkg_id * @parallel_bringup: Parallel bringup control */ struct x86_cpuinit_ops { void (*setup_percpu_clockev)(void); - void (*early_percpu_clock_init)(void); void (*fixup_cpu_id)(struct cpuinfo_x86 *c, int node); bool parallel_bringup; }; diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 280bb964f30a..11f078b91f22 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -126,12 +126,13 @@ static void kvm_save_sched_clock_state(void) kvmclock_disable(); } -#ifdef CONFIG_SMP -static void kvm_setup_secondary_clock(void) +static void kvm_setup_secondary_sched_clock(void) { + if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_SMP))) + return; + kvm_register_clock("secondary cpu, sched_clock setup"); } -#endif static void kvm_restore_sched_clock_state(void) { @@ -367,7 +368,8 @@ static void __init kvm_sched_clock_init(bool stable) { if (__paravirt_set_sched_clock(kvm_sched_clock_read, stable, kvm_save_sched_clock_state, - kvm_restore_sched_clock_state)) + kvm_restore_sched_clock_state, + kvm_setup_secondary_sched_clock)) return; kvm_sched_clock_offset = kvm_clock_read(); @@ -452,9 +454,6 @@ void __init kvmclock_init(void) x86_platform.get_wallclock = kvm_get_wallclock; x86_platform.set_wallclock = kvm_set_wallclock; -#ifdef CONFIG_SMP - x86_cpuinit.early_percpu_clock_init = kvm_setup_secondary_clock; -#endif kvm_get_preset_lpj(); clocksource_register_hz(&kvm_clock, NSEC_PER_SEC); diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index c538c608d9fb..f93278ddb1d2 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -86,8 +86,13 @@ static u64 native_steal_clock(int cpu) DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); +#ifdef CONFIG_SMP +static void (*pv_sched_clock_start_secondary)(void) __ro_after_init; +#endif + int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, - void (*save)(void), void (*restore)(void)) + void (*save)(void), void (*restore)(void), + void (*start_secondary)) { /* * Don't replace TSC with a PV clock when running as a CoCo guest and @@ -104,9 +109,20 @@ int __init __paravirt_set_sched_clock(u64 (*func)(void), bool stable, static_call_update(pv_sched_clock, func); x86_platform.save_sched_clock_state = save; x86_platform.restore_sched_clock_state = restore; +#ifdef CONFIG_SMP + pv_sched_clock_start_secondary = start_secondary; +#endif return 0; } +#ifdef CONFIG_SMP +void paravirt_sched_clock_start_secondary(void) +{ + if (pv_sched_clock_start_secondary) + pv_sched_clock_start_secondary(); +} +#endif + /* These are in entry.S */ static struct resource reserve_ioports = { .start = 0, diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index c10850ae6f09..e6fff67dd264 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -278,7 +278,7 @@ static void notrace start_secondary(void *unused) cpu_init(); fpu__init_cpu(); rcutree_report_cpu_starting(raw_smp_processor_id()); - x86_cpuinit.early_percpu_clock_init(); + paravirt_sched_clock_start_secondary(); ap_starting(); diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c index 0a2bbd674a6d..1d4cf071c74b 100644 --- a/arch/x86/kernel/x86_init.c +++ b/arch/x86/kernel/x86_init.c @@ -128,7 +128,6 @@ struct x86_init_ops x86_init __initdata = { }; struct x86_cpuinit_ops x86_cpuinit = { - .early_percpu_clock_init = x86_init_noop, .setup_percpu_clockev = setup_secondary_APIC_clock, .parallel_bringup = true, };