From patchwork Mon Jul 17 14:27:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 9845245 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5189460386 for ; Mon, 17 Jul 2017 14:27:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 447D028509 for ; Mon, 17 Jul 2017 14:27:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 392A728521; Mon, 17 Jul 2017 14:27:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7372028509 for ; Mon, 17 Jul 2017 14:27:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751382AbdGQO13 (ORCPT ); Mon, 17 Jul 2017 10:27:29 -0400 Received: from mail-wm0-f54.google.com ([74.125.82.54]:37359 "EHLO mail-wm0-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751336AbdGQO11 (ORCPT ); Mon, 17 Jul 2017 10:27:27 -0400 Received: by mail-wm0-f54.google.com with SMTP id b134so49973469wma.0 for ; Mon, 17 Jul 2017 07:27:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5Mva1gYEuB1izj8BE9bw16Y5Q9i9VKJ4428zXTbZ9+U=; b=KHQND72fpRWLvMRZyx2ciX3mysI/1Ge8MvRTFHYgR9YJygK0/TN99sAI8h4yuJLx5n MH0Ii/XQU8r+SVXGOHtp0Xdlzp2hD2aMxtAr6XtX9oYvHC7IH2Z/f4d9Ry0HFdbGMVsR 0NBxyzJSv6o49MsOOKaSW+GhIgT5N135vEXy0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5Mva1gYEuB1izj8BE9bw16Y5Q9i9VKJ4428zXTbZ9+U=; b=CWoXdCL9PjlpVE1BDv3V3lc2w9bXgKcr+KiQiy3r0ZeNPe6mzVGgTo0Oxs1DCaBJcv wkpxhYTS+q1bYSdTECtP7ZA1ODmVspxRQ9G52iHympmn2dXIOxvhPczSFxUnNgi5F127 xnEnaFzEPVSrBNTK3T+S9TXpXFu9zVzsmMBn07lRFKJli0IcvwUdnnxg1XmbxJbH+9ry C1mPGjmzCj3mZ+81EYl4VHCwUJD1TRfGSkt1Jpdnk4XQ+zFyFDa8i3uI4voApSfN2b/j ytFTh+CLPVHclVKTejm7DRPZMNmk+sRMN6vFxPmgrRiGu42kjMAlPJ0sOUH+mBiYIJfF oizw== X-Gm-Message-State: AIVw110J31rsEE63vKnxmxnaQ4mn3d38Nms/1dxKTsNl2iRV3XwrDU5a CGtWxV9RRnRBtFUe X-Received: by 10.80.186.93 with SMTP id 29mr17465218eds.47.1500301646204; Mon, 17 Jul 2017 07:27:26 -0700 (PDT) Received: from localhost.localdomain (xd93ddc2d.cust.hiper.dk. [217.61.220.45]) by smtp.gmail.com with ESMTPSA id b30sm9428952edd.6.2017.07.17.07.27.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 17 Jul 2017 07:27:25 -0700 (PDT) From: Christoffer Dall To: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Cc: kvm@vger.kernel.org, Marc Zyngier , Christoffer Dall , Catalin Marinas , Will Deacon , Mark Rutland Subject: [RFC PATCH v2 02/19] arm64: Use the physical counter when available for read_cycles Date: Mon, 17 Jul 2017 16:27:01 +0200 Message-Id: <20170717142718.13853-3-cdall@linaro.org> X-Mailer: git-send-email 2.9.0 In-Reply-To: <20170717142718.13853-1-cdall@linaro.org> References: <20170717142718.13853-1-cdall@linaro.org> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently get_cycles() is hardwired to arch_counter_get_cntvct() on arm64, but as we move to using the physical timer for the in-kernel time-keeping, we need to make that more flexible. First, we need to make sure the physical counter can be read on equal terms to the virtual counter, which includes adding physical counter read functions for timers that require errata. Second, we need to make a choice between reading the physical vs virtual counter, depending on which timer is used for time keeping in the kernel otherwise. We can do this using a static key to avoid a performance penalty during runtime when reading the counter. Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: Marc Zyngier Signed-off-by: Christoffer Dall --- arch/arm64/include/asm/arch_timer.h | 18 ++++++++++++------ arch/arm64/include/asm/timex.h | 2 +- drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++++++-- 3 files changed, 43 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h index ee5619b..48d522c 100644 --- a/arch/arm64/include/asm/arch_timer.h +++ b/arch/arm64/include/asm/arch_timer.h @@ -30,6 +30,8 @@ #include +extern struct static_key_false arch_timer_phys_counter_available; + #if IS_ENABLED(CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND) extern struct static_key_false arch_timer_read_ool_enabled; #define needs_unstable_timer_counter_workaround() \ @@ -52,6 +54,7 @@ struct arch_timer_erratum_workaround { const char *desc; u32 (*read_cntp_tval_el0)(void); u32 (*read_cntv_tval_el0)(void); + u64 (*read_cntpct_el0)(void); u64 (*read_cntvct_el0)(void); int (*set_next_event_phys)(unsigned long, struct clock_event_device *); int (*set_next_event_virt)(unsigned long, struct clock_event_device *); @@ -148,13 +151,8 @@ static inline void arch_timer_set_cntkctl(u32 cntkctl) static inline u64 arch_counter_get_cntpct(void) { - u64 cval; - /* - * AArch64 kernel and user space mandate the use of CNTVCT. - */ isb(); - asm volatile("mrs %0, cntpct_el0" : "=r" (cval)); - return cval; + return arch_timer_reg_read_stable(cntpct_el0); } static inline u64 arch_counter_get_cntvct(void) @@ -163,6 +161,14 @@ static inline u64 arch_counter_get_cntvct(void) return arch_timer_reg_read_stable(cntvct_el0); } +static inline u64 arch_counter_get_cycles(void) +{ + if (static_branch_unlikely(&arch_timer_phys_counter_available)) + return arch_counter_get_cntpct(); + else + return arch_counter_get_cntvct(); +} + static inline int arch_timer_arch_init(void) { return 0; diff --git a/arch/arm64/include/asm/timex.h b/arch/arm64/include/asm/timex.h index 81a076e..c0d214c 100644 --- a/arch/arm64/include/asm/timex.h +++ b/arch/arm64/include/asm/timex.h @@ -22,7 +22,7 @@ * Use the current timer as a cycle counter since this is what we use for * the delay loop. */ -#define get_cycles() arch_counter_get_cntvct() +#define get_cycles() arch_counter_get_cycles() #include diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index c24327c..f4e7261 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -77,6 +77,9 @@ static bool arch_timer_mem_use_virtual; static bool arch_counter_suspend_stop; static bool vdso_default = true; +DEFINE_STATIC_KEY_FALSE(arch_timer_phys_counter_available); +EXPORT_SYMBOL_GPL(arch_timer_phys_counter_available); + static bool evtstrm_enable = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM); static int __init early_evtstrm_cfg(char *buf) @@ -217,6 +220,11 @@ static u32 notrace fsl_a008585_read_cntv_tval_el0(void) return __fsl_a008585_read_reg(cntv_tval_el0); } +static u64 notrace fsl_a008585_read_cntpct_el0(void) +{ + return __fsl_a008585_read_reg(cntpct_el0); +} + static u64 notrace fsl_a008585_read_cntvct_el0(void) { return __fsl_a008585_read_reg(cntvct_el0); @@ -258,6 +266,11 @@ static u32 notrace hisi_161010101_read_cntv_tval_el0(void) return __hisi_161010101_read_reg(cntv_tval_el0); } +static u64 notrace hisi_161010101_read_cntpct_el0(void) +{ + return __hisi_161010101_read_reg(cntpct_el0); +} + static u64 notrace hisi_161010101_read_cntvct_el0(void) { return __hisi_161010101_read_reg(cntvct_el0); @@ -288,6 +301,15 @@ static struct ate_acpi_oem_info hisi_161010101_oem_info[] = { #endif #ifdef CONFIG_ARM64_ERRATUM_858921 +static u64 notrace arm64_858921_read_cntpct_el0(void) +{ + u64 old, new; + + old = read_sysreg(cntpct_el0); + new = read_sysreg(cntpct_el0); + return (((old ^ new) >> 32) & 1) ? old : new; +} + static u64 notrace arm64_858921_read_cntvct_el0(void) { u64 old, new; @@ -346,6 +368,7 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = { .desc = "Freescale erratum a005858", .read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0, .read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0, + .read_cntpct_el0 = fsl_a008585_read_cntpct_el0, .read_cntvct_el0 = fsl_a008585_read_cntvct_el0, .set_next_event_phys = erratum_set_next_event_tval_phys, .set_next_event_virt = erratum_set_next_event_tval_virt, @@ -358,6 +381,7 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = { .desc = "HiSilicon erratum 161010101", .read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0, .read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0, + .read_cntpct_el0 = hisi_161010101_read_cntpct_el0, .read_cntvct_el0 = hisi_161010101_read_cntvct_el0, .set_next_event_phys = erratum_set_next_event_tval_phys, .set_next_event_virt = erratum_set_next_event_tval_virt, @@ -368,6 +392,7 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = { .desc = "HiSilicon erratum 161010101", .read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0, .read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0, + .read_cntpct_el0 = hisi_161010101_read_cntpct_el0, .read_cntvct_el0 = hisi_161010101_read_cntvct_el0, .set_next_event_phys = erratum_set_next_event_tval_phys, .set_next_event_virt = erratum_set_next_event_tval_virt, @@ -378,6 +403,7 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = { .match_type = ate_match_local_cap_id, .id = (void *)ARM64_WORKAROUND_858921, .desc = "ARM erratum 858921", + .read_cntpct_el0 = arm64_858921_read_cntpct_el0, .read_cntvct_el0 = arm64_858921_read_cntvct_el0, }, #endif @@ -886,10 +912,12 @@ static void __init arch_counter_register(unsigned type) /* Register the CP15 based counter if we have one */ if (type & ARCH_TIMER_TYPE_CP15) { - if (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI) + if (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI) { arch_timer_read_counter = arch_counter_get_cntvct; - else + } else { arch_timer_read_counter = arch_counter_get_cntpct; + static_branch_enable(&arch_timer_phys_counter_available); + } clocksource_counter.archdata.vdso_direct = vdso_default; } else {