From patchwork Tue Jan 14 19:36:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Igor Druzhinin X-Patchwork-Id: 11332833 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6DFEF1398 for ; Tue, 14 Jan 2020 19:37:34 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4A03724655 for ; Tue, 14 Jan 2020 19:37:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="RnCZ5t/f" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A03724655 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1irRzJ-0004Vf-EP; Tue, 14 Jan 2020 19:36:29 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1irRzH-0004Va-94 for xen-devel@lists.xenproject.org; Tue, 14 Jan 2020 19:36:27 +0000 X-Inumbo-ID: 26b50114-3705-11ea-8440-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 26b50114-3705-11ea-8440-12813bfff9fa; Tue, 14 Jan 2020 19:36:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1579030585; h=from:to:cc:subject:date:message-id:mime-version; bh=AB0Pd8o5+d7R1L1sAc/etgHve8cNnPZU9HusSaXdKfk=; b=RnCZ5t/fYcu6GL8S1FWZu1X2zeFt2bUBs5On928enoSNsZBKaIgwBIvL Ue3pcGdrXUE7T9hIFtlEvPh5OI9LPEvuit+ZqWuhXKZOtlaXTr7s2r/22 eooBldyupB7QNAdj5PBZB2Rl7BwYSOgXdFG3MFX6DmqRJjp5Hp/Pb02sx w=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=igor.druzhinin@citrix.com; spf=Pass smtp.mailfrom=igor.druzhinin@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of igor.druzhinin@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of igor.druzhinin@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: lD+8Cm8wOuf2tc8jvmP2ouhk8vq2MFWUr32k5wd0hkZNEtEdd96l50UOAZwenOd3MVcLisIKo6 TawlJleG2+FbuOpbNe9Y7UGRpc31tz0VKPxI1WhH+cBN4pJ+jzhgTjU3jc4fVH/S1WhRzR18V7 Kg2quwyoKyfxgvfkEsBeqaDltL8MQk3LdNVce1Y01uTMsVIcGqx8WaYpGkxmz47LKEZ9NMbqmO RGF/9NMfsgKN4n0cyXMLevql4KeSoUMuSm4QfFP86nWqzxLAn2SHwLKlcW2ydl8LQ6z+5BR+Zj 9w4= X-SBRS: 2.7 X-MesageID: 11277150 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.70,320,1574139600"; d="scan'208";a="11277150" From: Igor Druzhinin To: Date: Tue, 14 Jan 2020 19:36:21 +0000 Message-ID: <1579030581-7929-1-git-send-email-igor.druzhinin@citrix.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH] x86/time: update TSC stamp on restore from deep C-state X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: andrew.cooper3@citrix.com, Igor Druzhinin , wl@xen.org, jbeulich@suse.com, roger.pau@citrix.com Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" If ITSC is not available on CPU (e.g if running nested as PV shim) then X86_FEATURE_NONSTOP_TSC is not advertised in certain cases, i.e. all AMD and some old Intel processors. In which case TSC would need to be restored on CPU from platform time by Xen upon exiting deep C-states. As platform time might be behind the last TSC stamp recorded for the current CPU, invariant of TSC stamp being always behind local TSC counter is violated. This has an effect of get_s_time() going negative resulting in eventual system hang or crash. Fix this issue by updating local TSC stamp along with TSC counter write. Signed-off-by: Igor Druzhinin Reviewed-by: Roger Pau Monné Acked-by: Jan Beulich --- This caused reliable hangs of shim domains with multiple vCPUs on all AMD systems. The problem got also reproduced on bare-metal by artifically masking ITSC feature bit. The proposed fix has been verified for both cases. --- xen/arch/x86/time.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c index e79cb4d..f6b26f8 100644 --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -955,10 +955,16 @@ u64 stime2tsc(s_time_t stime) void cstate_restore_tsc(void) { + struct cpu_time *t = &this_cpu(cpu_time); + if ( boot_cpu_has(X86_FEATURE_NONSTOP_TSC) ) return; - write_tsc(stime2tsc(read_platform_stime(NULL))); + t->stamp.master_stime = read_platform_stime(NULL); + t->stamp.local_tsc = stime2tsc(t->stamp.master_stime); + t->stamp.local_stime = t->stamp.master_stime; + + write_tsc(t->stamp.local_tsc); } /***************************************************************************