From patchwork Tue Feb 9 12:57:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12078055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75C84C433E0 for ; Tue, 9 Feb 2021 12:57:16 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1102D64E75 for ; Tue, 9 Feb 2021 12:57:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1102D64E75 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.83225.154355 (Exim 4.92) (envelope-from ) id 1l9SZo-00088G-1V; Tue, 09 Feb 2021 12:57:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 83225.154355; Tue, 09 Feb 2021 12:57:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l9SZn-000888-Ui; Tue, 09 Feb 2021 12:57:07 +0000 Received: by outflank-mailman (input) for mailman id 83225; Tue, 09 Feb 2021 12:57:06 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l9SZm-00087u-5T for xen-devel@lists.xenproject.org; Tue, 09 Feb 2021 12:57:06 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 279d4154-5031-449e-842b-7b4652fc68ce; Tue, 09 Feb 2021 12:57:05 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 78800AF57; Tue, 9 Feb 2021 12:57:04 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 279d4154-5031-449e-842b-7b4652fc68ce X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1612875424; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FDe1jUzRfbbh7+YuIc5yGiZBwFURovh6CEvEftGqGYw=; b=S3/TWOADLqQR2nKGj5HFEPFdKZxPTyk+Xw+ZRLaMAKmUwrIaPIGXaD12RG9NWrB/qjwijr 1A2IglhliX5W9ILvav41xKsxqTVOQK+Wv1k+GxMbTkF6ap5QpbfTum7eteNr8gYgaore8U rpN26Ghp/9Y896HOb3jD3i+5CPEBITQ= Subject: [PATCH RFC v3 4/4] x86/time: re-arrange struct calibration_rendezvous From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: Message-ID: <56d70757-a887-6824-18f4-93b1f244e44b@suse.com> Date: Tue, 9 Feb 2021 13:57:05 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US To reduce latency on time_calibration_tsc_rendezvous()'s last loop iteration, separate fields written on the last iteration enough from the crucial field read by all CPUs on the last iteration such that they end up in distinct cache lines. Prefetch this field on an earlier iteration. Signed-off-by: Jan Beulich --- v3: New. --- While readability would likely suffer, we may want to consider avoiding the prefetch on at least the first two iterations (where the field still gets / may get written to by CPU0). Could e.g. be switch ( i ) { case 0: write_tsc(r->master_tsc_stamp); break; case 1: prefetch(&r->master_tsc_stamp); break; } Of course it would also be nice to avoid the pretty likely branch misprediction on the last iteration. But with the static prediction hints having been rather short-lived in the architecture, I don't see any good means to do so. --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -1655,10 +1655,20 @@ static void tsc_check_reliability(void) * All CPUS snapshot their local TSC and extrapolation of system time. */ struct calibration_rendezvous { - cpumask_t cpu_calibration_map; atomic_t semaphore; s_time_t master_stime; - uint64_t master_tsc_stamp, max_tsc_stamp; + cpumask_t cpu_calibration_map; + /* + * All CPUs want to read master_tsc_stamp on the last iteration. If + * cpu_calibration_map isn't large enough to push the field into a cache + * line different from the one used by semaphore (written by all CPUs on + * every iteration) and master_stime (written by CPU0 on the last + * iteration), align the field suitably. + */ + uint64_t __aligned(BITS_TO_LONGS(NR_CPUS) * sizeof(long) + + sizeof(atomic_t) + sizeof(s_time_t) < SMP_CACHE_BYTES + ? SMP_CACHE_BYTES : 1) master_tsc_stamp; + uint64_t max_tsc_stamp; }; static void @@ -1709,6 +1719,8 @@ static void time_calibration_tsc_rendezv if ( i == 0 ) write_tsc(r->master_tsc_stamp); + else + prefetch(&r->master_tsc_stamp); while ( atomic_read(&r->semaphore) != (2*total_cpus - 1) ) cpu_relax(); @@ -1731,6 +1743,8 @@ static void time_calibration_tsc_rendezv if ( i == 0 ) write_tsc(r->master_tsc_stamp); + else + prefetch(&r->master_tsc_stamp); atomic_inc(&r->semaphore); while ( atomic_read(&r->semaphore) > total_cpus )