From patchwork Fri Oct 27 22:25:10 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Salyzyn X-Patchwork-Id: 10030717 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 895276032C for ; Fri, 27 Oct 2017 22:26:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6029C28FE7 for ; Fri, 27 Oct 2017 22:26:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 54C2B28FEB; Fri, 27 Oct 2017 22:26:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BDDE728FE7 for ; Fri, 27 Oct 2017 22:26:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=HQ1kd1UXAGr+wwwycBnmoWwjUaaIwdYJCJBVjfd2FtI=; b=rjK PLu6X0PE6M9g2Qt1rRGVeUG3+TIN1HFzq9UrUs+xBapQHpKGa6kLn9sm20ZfU+RSiTYQQLqje7bKo uQJfVh/r7yaN72bzT+1VKZytqVmF8qmLKFoKZOnrnOTWhWZJ9zrsOo6MGey+J5K3Vzyd9bqcwyS7h 5E0yOrSkUUEtS2Ysosxml5xvDV6McPXIi1TmU68/vCaUP1KfyxPw2LmVM1Jx2EgJUDXLaQWstdevZ F+0MUScrQV8SlQYQsSkV9N/OIyT9+8fWs8zR/F1B3SIXjyITL5dkLF0P9IH+0irEshgbL5wJuPELt u1vRxuwjVBEa7gOP8ZfNX5rELsmb5uw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e8D4k-0002tw-77; Fri, 27 Oct 2017 22:26:02 +0000 Received: from mail-io0-x244.google.com ([2607:f8b0:4001:c06::244]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1e8D4S-0002fj-SS for linux-arm-kernel@lists.infradead.org; Fri, 27 Oct 2017 22:25:59 +0000 Received: by mail-io0-x244.google.com with SMTP id 101so15746637ioj.3 for ; Fri, 27 Oct 2017 15:25:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=android.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=v1tPVr3XrbxBpBAqewxxBsDBvNzt6Fu17wGoosb5O98=; b=ce3YD+qBQ94Hcp/9izlsujsAWFyJ+uaOl9ZFZnPIqam4iVgx50oC7iMV/+Tkna6bCt lyFMI9tG2I1Zo0AP/ljOSX5GasCeJmQWDVW1/1JxqwPNQBrC9TIQhL+w+yR56OAcbPj/ kimfjH1T/HeifW8C3ZEUM0YhBp4j00teHWwag138QwapGcJrBsT3/KCTMz1kpFK1uML0 LvugP+CwVQmvLmgLkijbN1Sd0hic2K+W3Fk3TP4pe6/9zQynAUX+hqKV2qc1CCg7amqP fibYGsd7u53Ta+/Ouu6EUPJlZfn0ec7xwTpRpwtRW988Mf5CnXYBu1D6dvlkABe53nkO DVKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=v1tPVr3XrbxBpBAqewxxBsDBvNzt6Fu17wGoosb5O98=; b=W9sXsEIXXpCQaELMLOUzPGQhlR/dLKISASzJ92zsQlaJF1P/g4APGcJF8kLAVsWCua YHDloEK8HS9WYmkY0EF8ZHr0Toz0W+O30u1ptZlYEA1xQdljGGViahxcOqnQ0YmITLQP WsUz5mGLqXeJviM3z8u/uM9ZZqo5Sow1Pj8SlRDsLtSRGrpGLTVBmpEaBvGkTW72bCWF WSTXtzo0vxtdSy/D8XgV4FGsPIVzC8b/Bh0FPVB/p1uDR5uTSDNSOBEQ35BnPBf2vp7I cEFCNIp8iYkMLSsGF9QwzKC8t3iTamCh6EPOJ635m0Ik8Ux9B79xJDkxX9WL9j865czL gmZg== X-Gm-Message-State: AMCzsaX77CxH31jSN3EWfMrpc2aqgF9mRdY8/Xe8oBi4EwAeq7B02yun pBEkXR9ideU7VSKqW7JHN8FJaA== X-Google-Smtp-Source: ABhQp+R4RkjMNcqzrEUHNzBfSm9PppuLLL7JbzzA3dL7EiSYZnibsouXSfFrlrbRq7Csy1enTE7SVw== X-Received: by 10.36.71.74 with SMTP id t71mr2525953itb.53.1509143123465; Fri, 27 Oct 2017 15:25:23 -0700 (PDT) Received: from nebulus.mtv.corp.google.com ([100.98.120.17]) by smtp.gmail.com with ESMTPSA id m67sm1387165itm.16.2017.10.27.15.25.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 27 Oct 2017 15:25:22 -0700 (PDT) From: Mark Salyzyn To: linux-kernel@vger.kernel.org Subject: [PATCH v3 03/12] arm: vdso: inline assembler operations to compiler.h Date: Fri, 27 Oct 2017 15:25:10 -0700 Message-Id: <20171027222518.57161-1-salyzyn@android.com> X-Mailer: git-send-email 2.15.0.rc2.357.g7e34df9404-goog X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171027_152545_124383_67B72C90 X-CRM114-Status: GOOD ( 20.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Laura Abbott , Kees Cook , Ard Biesheuvel , Catalin Marinas , Kevin Brodsky , Will Deacon , Russell King , Andy Lutomirski , Mark Salyzyn , James Morse , Andrew Pinski , Dmitry Safonov , Andy Gross , John Stultz , Mark Salyzyn , linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Move compiler-specific code to a local compiler.h file. CONFIG_AEABI dependency check. System call fallback functions standardized into a DEFINE_FALLBACK macro. Accept that __arch_counter_get_cntvct() is the API for aarch64. Deal with unresolved references emitted by GCC. Optimize handling of fallback calls. Most notably add unlikely nullptr checking to __vdso_gettimeofday, if tv null no need to proceed to fallback, as vdso is capable of filling in the tv values. For time functions that always return success, do not waste time checking return value for switch to fallback. Signed-off-by: Mark Salyzyn Cc: James Morse Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Andy Lutomirski Cc: Dmitry Safonov Cc: John Stultz Cc: Mark Rutland Cc: Laura Abbott Cc: Kees Cook Cc: Ard Biesheuvel Cc: Andy Gross Cc: Kevin Brodsky Cc: Andrew Pinski Cc: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org v2: - split first CL into 3 of 7 pieces v3: - rebase (unchanged) --- arch/arm/vdso/compiler.h | 69 ++++++++++++++++++++++++++ arch/arm/vdso/vgettimeofday.c | 109 +++++++++++------------------------------- 2 files changed, 96 insertions(+), 82 deletions(-) create mode 100644 arch/arm/vdso/compiler.h diff --git a/arch/arm/vdso/compiler.h b/arch/arm/vdso/compiler.h new file mode 100644 index 000000000000..ef1d664fb4fa --- /dev/null +++ b/arch/arm/vdso/compiler.h @@ -0,0 +1,69 @@ +/* + * Userspace implementations of fallback calls + * + * Copyright (C) 2017 Cavium, Inc. + * Copyright (C) 2012 ARM Limited + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * Author: Will Deacon + * Rewriten into C by: Andrew Pinski + */ + +#ifndef __VDSO_COMPILER_H +#define __VDSO_COMPILER_H + +#include +#include +#include +#include + +#ifndef CONFIG_AEABI +#error This code depends on AEABI system call conventions +#endif + +#define DEFINE_FALLBACK(name, type_arg1, name_arg1, type_arg2, name_arg2) \ +static notrace long name##_fallback(type_arg1 _##name_arg1, \ + type_arg2 _##name_arg2) \ +{ \ + register type_arg1 name_arg1 asm("r0") = _##name_arg1; \ + register type_arg2 name_arg2 asm("r1") = _##name_arg2; \ + register long ret asm ("r0"); \ + register long nr asm("r7") = __NR_##name; \ + \ + asm volatile( \ + " swi #0\n" \ + : "=r" (ret) \ + : "r" (name_arg1), "r" (name_arg2), "r" (nr) \ + : "memory"); \ + \ + return ret; \ +} + +#define __arch_counter_get_cntvct() arch_counter_get_cntvct() + +/* Avoid unresolved references emitted by GCC */ + +void __aeabi_unwind_cpp_pr0(void) +{ +} + +void __aeabi_unwind_cpp_pr1(void) +{ +} + +void __aeabi_unwind_cpp_pr2(void) +{ +} + +#endif /* __VDSO_COMPILER_H */ diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c index 2474c17dc356..5f596911bd53 100644 --- a/arch/arm/vdso/vgettimeofday.c +++ b/arch/arm/vdso/vgettimeofday.c @@ -22,21 +22,16 @@ * Reworked and rebased over arm version by: Mark Salyzyn */ -#include -#include -#include -#include #include -#include -#include -#include - -#ifndef CONFIG_AEABI -#error This code depends on AEABI system call conventions -#endif +#include +#include +#include "compiler.h" #include "datapage.h" +DEFINE_FALLBACK(gettimeofday, struct timeval *, tv, struct timezone *, tz) +DEFINE_FALLBACK(clock_gettime, clockid_t, clock, struct timespec *, ts) + static notrace u32 vdso_read_begin(const struct vdso_data *vd) { u32 seq; @@ -63,23 +58,6 @@ static notrace int vdso_read_retry(const struct vdso_data *vd, u32 start) return seq != start; } -static notrace long clock_gettime_fallback(clockid_t _clkid, - struct timespec *_ts) -{ - register struct timespec *ts asm("r1") = _ts; - register clockid_t clkid asm("r0") = _clkid; - register long ret asm ("r0"); - register long nr asm("r7") = __NR_clock_gettime; - - asm volatile( - " swi #0\n" - : "=r" (ret) - : "r" (clkid), "r" (ts), "r" (nr) - : "memory"); - - return ret; -} - static notrace int do_realtime_coarse(const struct vdso_data *vd, struct timespec *ts) { @@ -127,7 +105,7 @@ static notrace u64 get_ns(const struct vdso_data *vd) u64 cycle_now; u64 nsec; - cycle_now = arch_counter_get_cntvct(); + cycle_now = __arch_counter_get_cntvct(); cycle_delta = (cycle_now - vd->cs_cycle_last) & vd->cs_mask; @@ -200,85 +178,52 @@ static notrace int do_monotonic(const struct vdso_data *vd, struct timespec *ts) #endif /* CONFIG_ARM_ARCH_TIMER */ -notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts) +notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts) { - int ret = -1; - const struct vdso_data *vd = __get_datapage(); - switch (clkid) { + switch (clock) { case CLOCK_REALTIME_COARSE: - ret = do_realtime_coarse(vd, ts); + do_realtime_coarse(vd, ts); break; case CLOCK_MONOTONIC_COARSE: - ret = do_monotonic_coarse(vd, ts); + do_monotonic_coarse(vd, ts); break; case CLOCK_REALTIME: - ret = do_realtime(vd, ts); + if (do_realtime(vd, ts)) + goto fallback; break; case CLOCK_MONOTONIC: - ret = do_monotonic(vd, ts); + if (do_monotonic(vd, ts)) + goto fallback; break; default: - break; + goto fallback; } - if (ret) - ret = clock_gettime_fallback(clkid, ts); - - return ret; -} - -static notrace long gettimeofday_fallback(struct timeval *_tv, - struct timezone *_tz) -{ - register struct timezone *tz asm("r1") = _tz; - register struct timeval *tv asm("r0") = _tv; - register long ret asm ("r0"); - register long nr asm("r7") = __NR_gettimeofday; - - asm volatile( - " swi #0\n" - : "=r" (ret) - : "r" (tv), "r" (tz), "r" (nr) - : "memory"); - - return ret; + return 0; +fallback: + return clock_gettime_fallback(clock, ts); } notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz) { - struct timespec ts; - int ret; - const struct vdso_data *vd = __get_datapage(); - ret = do_realtime(vd, &ts); - if (ret) - return gettimeofday_fallback(tv, tz); + if (likely(tv != NULL)) { + struct timespec ts; + + if (do_realtime(vd, &ts)) + return gettimeofday_fallback(tv, tz); - if (tv) { tv->tv_sec = ts.tv_sec; tv->tv_usec = ts.tv_nsec / 1000; } - if (tz) { + + if (unlikely(tz != NULL)) { tz->tz_minuteswest = vd->tz_minuteswest; tz->tz_dsttime = vd->tz_dsttime; } - return ret; -} - -/* Avoid unresolved references emitted by GCC */ - -void __aeabi_unwind_cpp_pr0(void) -{ -} - -void __aeabi_unwind_cpp_pr1(void) -{ -} - -void __aeabi_unwind_cpp_pr2(void) -{ + return 0; }