From patchwork Tue Jun 25 16:18:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 11015959 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F6996C5 for ; Tue, 25 Jun 2019 16:18:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0081C27F89 for ; Tue, 25 Jun 2019 16:18:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E84A027F93; Tue, 25 Jun 2019 16:18:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 864E527F89 for ; Tue, 25 Jun 2019 16:18:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726696AbfFYQSf (ORCPT ); Tue, 25 Jun 2019 12:18:35 -0400 Received: from foss.arm.com ([217.140.110.172]:44760 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726420AbfFYQSf (ORCPT ); Tue, 25 Jun 2019 12:18:35 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 916D1360; Tue, 25 Jun 2019 09:18:34 -0700 (PDT) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F12313F718; Tue, 25 Jun 2019 09:18:31 -0700 (PDT) From: Vincenzo Frascino To: linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, arnd@arndb.de, linux@armlinux.org.uk, ralf@linux-mips.org, paul.burton@mips.com, daniel.lezcano@linaro.org, tglx@linutronix.de, salyzyn@android.com, pcc@google.com, shuah@kernel.org, 0x7f454c46@gmail.com, linux@rasmusvillemoes.dk, huw@codeweavers.com, sthotton@marvell.com, andre.przywara@arm.com Subject: [PATCH 1/3] lib/vdso: Delay mask application in do_hres() Date: Tue, 25 Jun 2019 17:18:02 +0100 Message-Id: <20190625161804.38713-1-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190624133607.GI29497@fuggles.cambridge.arm.com> References: <20190624133607.GI29497@fuggles.cambridge.arm.com> MIME-Version: 1.0 Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP do_hres() in the vDSO generic library masks the hw counter value immediately after reading it. Postpone the mask application after checking if the syscall fallback is enabled, in order to be able to detect a possible fallback for the architectures that have masks smaller than ULLONG_MAX. Signed-off-by: Vincenzo Frascino --- lib/vdso/gettimeofday.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c index ef28cc5d7bff..ee1221ba1d32 100644 --- a/lib/vdso/gettimeofday.c +++ b/lib/vdso/gettimeofday.c @@ -35,12 +35,12 @@ static int do_hres(const struct vdso_data *vd, clockid_t clk, do { seq = vdso_read_begin(vd); - cycles = __arch_get_hw_counter(vd->clock_mode) & - vd->mask; + cycles = __arch_get_hw_counter(vd->clock_mode); ns = vdso_ts->nsec; last = vd->cycle_last; if (unlikely((s64)cycles < 0)) return clock_gettime_fallback(clk, ts); + cycles &= vd->mask; if (cycles > last) ns += (cycles - last) * vd->mult; ns >>= vd->shift; From patchwork Tue Jun 25 16:18:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 11015953 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E96FD1398 for ; Tue, 25 Jun 2019 16:18:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC8FC27F89 for ; Tue, 25 Jun 2019 16:18:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D002427F92; Tue, 25 Jun 2019 16:18:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B42E27F89 for ; Tue, 25 Jun 2019 16:18:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727793AbfFYQSi (ORCPT ); Tue, 25 Jun 2019 12:18:38 -0400 Received: from foss.arm.com ([217.140.110.172]:44794 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726420AbfFYQSh (ORCPT ); Tue, 25 Jun 2019 12:18:37 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5EEE8C0A; Tue, 25 Jun 2019 09:18:37 -0700 (PDT) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C5DB03F718; Tue, 25 Jun 2019 09:18:34 -0700 (PDT) From: Vincenzo Frascino To: linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, arnd@arndb.de, linux@armlinux.org.uk, ralf@linux-mips.org, paul.burton@mips.com, daniel.lezcano@linaro.org, tglx@linutronix.de, salyzyn@android.com, pcc@google.com, shuah@kernel.org, 0x7f454c46@gmail.com, linux@rasmusvillemoes.dk, huw@codeweavers.com, sthotton@marvell.com, andre.przywara@arm.com Subject: [PATCH 2/3] arm64: Fix __arch_get_hw_counter() implementation Date: Tue, 25 Jun 2019 17:18:03 +0100 Message-Id: <20190625161804.38713-2-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190625161804.38713-1-vincenzo.frascino@arm.com> References: <20190624133607.GI29497@fuggles.cambridge.arm.com> <20190625161804.38713-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Provide the following fixes for the __arch_get_hw_counter() implementation on arm64: - Fallback on syscall when an unstable counter is detected. - Introduce isb()s before and after the counter read to avoid speculation of the counter value and of the seq lock respectively. The second isb() is a temporary solution that will be revisited in 5.3-rc1. These fixes restore the semantics that __arch_counter_get_cntvct() had on arm64. Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/vdso/gettimeofday.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h index 447ef417de45..b08f476b72b4 100644 --- a/arch/arm64/include/asm/vdso/gettimeofday.h +++ b/arch/arm64/include/asm/vdso/gettimeofday.h @@ -10,6 +10,8 @@ #include #include +#define __VDSO_USE_SYSCALL ULLONG_MAX + #define VDSO_HAS_CLOCK_GETRES 1 static __always_inline @@ -68,7 +70,24 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode) { u64 res; + /* + * clock_mode == 0 implies that vDSO are enabled otherwise + * fallback on syscall. + */ + if (clock_mode) + return __VDSO_USE_SYSCALL; + + /* + * This isb() is required to prevent that the counter value + * is speculated. + */ + isb(); asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory"); + /* + * This isb() is required to prevent that the seq lock is + * speculated.# + */ + isb(); return res; } From patchwork Tue Jun 25 16:18:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 11015949 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE7601398 for ; Tue, 25 Jun 2019 16:18:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D0C8327F89 for ; Tue, 25 Jun 2019 16:18:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C4B6E27F92; Tue, 25 Jun 2019 16:18:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 662AF27F8C for ; Tue, 25 Jun 2019 16:18:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728360AbfFYQSl (ORCPT ); Tue, 25 Jun 2019 12:18:41 -0400 Received: from foss.arm.com ([217.140.110.172]:44832 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726420AbfFYQSk (ORCPT ); Tue, 25 Jun 2019 12:18:40 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2D8ACED1; Tue, 25 Jun 2019 09:18:40 -0700 (PDT) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 96EB73F718; Tue, 25 Jun 2019 09:18:37 -0700 (PDT) From: Vincenzo Frascino To: linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, arnd@arndb.de, linux@armlinux.org.uk, ralf@linux-mips.org, paul.burton@mips.com, daniel.lezcano@linaro.org, tglx@linutronix.de, salyzyn@android.com, pcc@google.com, shuah@kernel.org, 0x7f454c46@gmail.com, linux@rasmusvillemoes.dk, huw@codeweavers.com, sthotton@marvell.com, andre.przywara@arm.com Subject: [PATCH 3/3] arm64: compat: Fix __arch_get_hw_counter() implementation Date: Tue, 25 Jun 2019 17:18:04 +0100 Message-Id: <20190625161804.38713-3-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190625161804.38713-1-vincenzo.frascino@arm.com> References: <20190624133607.GI29497@fuggles.cambridge.arm.com> <20190625161804.38713-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Provide the following fixes for the __arch_get_hw_counter() implementation on arm64: - Fallback on syscall when an unstable counter is detected. - Introduce isb()s before and after the counter read to avoid speculation of the counter value and of the seq lock respectively. The second isb() is a temporary solution that will be revisited in 5.3-rc1. These fixes restore the semantics that __arch_counter_get_cntvct() had on arm64. Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Vincenzo Frascino --- .../include/asm/vdso/compat_gettimeofday.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h index 93dbd935b66d..f4812777f5c5 100644 --- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h +++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h @@ -12,6 +12,8 @@ #include +#define __VDSO_USE_SYSCALL ULLONG_MAX + #define VDSO_HAS_CLOCK_GETRES 1 static __always_inline @@ -74,8 +76,24 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode) { u64 res; + /* + * clock_mode == 0 implies that vDSO are enabled otherwise + * fallback on syscall. + */ + if (clock_mode) + return __VDSO_USE_SYSCALL; + + /* + * This isb() is required to prevent that the counter value + * is speculated. + */ isb(); asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res)); + /* + * This isb() is required to prevent that the seq lock is + * speculated. + */ + isb(); return res; }