From patchwork Mon Oct 1 17:58:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Salyzyn X-Patchwork-Id: 10622665 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B22AE6CB for ; Mon, 1 Oct 2018 19:30:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9991F28524 for ; Mon, 1 Oct 2018 19:30:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8AB6D2852A; Mon, 1 Oct 2018 19:30:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9E0E228524 for ; Mon, 1 Oct 2018 19:30:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tprwz1cWlUqre0t0Gvnnzq3KvwDHX1xcMJwhCqEwaE0=; b=tE6wRnOtQypvB5 FpURUOXYGPTkFVKCSOEg0tLYoopWCTZdfbsR4yKb1sLyz48Lj/Zh3IYjJSXxsWXkB91f2LiFvAGbl 0tm4TrOdEYoOLu22DjJHkfwoBJ/acDncrvFk3hWGDtf09sz4fKrPqiaDwMYOXEriPxzQyS9Jl4qo8 BJ0f17/31uiUOlUeE3TwKzBtoWU7CxCA3TMh9AaiTeAtOpCKTE7mmCRE69z46MCZZRaJz9RKlwzdS +FpMHd6H3Aq0EW7ud6G98U/fmnTJRu69nU9ppQxrRAQ7Dg6Cifv8hFiTciLrYwY3NLRrg9am1f7sq b8c60ICL5cAkNLrNftUA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g73tL-0006fY-EZ; Mon, 01 Oct 2018 19:30:03 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g73pr-0005Fa-Nn for linux-arm-kernel@bombadil.infradead.org; Mon, 01 Oct 2018 19:26:27 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=q4MDyK/xpZ2rJeVO6MpkOTduxc1guqAC/giITauVW44=; b=VUC6ZEo8yJl91iG9qLeQ04V4AP 8wZKFd43cpycsD52g6bcFaBNlE6NE+KWvwaV9D0C8oWigmDiVq8Y/GXnTL4gkAg5FqqAqaCn8n06r 43Wqir2NrnsRKIrT7TA/qCLpf48idEh7JDdpZ5ERb0YKVOqwsw6a+RJNKHMgsHoxxVEB8jwFXaCa3 t2YGfr7+isZ8G0ihnN1b90YESvVvbAa7wAgLbSqTKBNojeuIFXyRfTDBcMDsSmeBXpDZDLfmjD78h 24IiM8zcj676ALkupk8zxJzalfvRJORxV6yjU9A5N6YxR9ApGtCuzPAQ4RSKsidPa+XvdjEbB5YXs XCmbzeOA==; Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]) by merlin.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g72V0-0008Ax-Dv for linux-arm-kernel@lists.infradead.org; Mon, 01 Oct 2018 18:00:52 +0000 Received: by mail-pg1-x544.google.com with SMTP id g12-v6so4331303pgs.1 for ; Mon, 01 Oct 2018 11:00:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=android.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q4MDyK/xpZ2rJeVO6MpkOTduxc1guqAC/giITauVW44=; b=tj30xD9pcDHGyPcFmZM2qpxBuDA4ap8T+j6Li2TC8PGMb6nBPmALsBxlKJW8/8183Z JscOrrTI3FJTBgywd8A6fYuLkjefaiTTrdAJRscO2KI+0WdZH6rN2svfDSiDSujx89we uaZpGSkkyW8RpdTpj5Puwhrshy1CUgTwOnu3f1BByZ75MXoYwHhKJViKihXiuYQYYaLs 1conwfxVgZU9EX6f73v7Xz5LWZ2ayIdf1VMr1sHR0o17VRcRHxVKMgyhKqjOhiBy97Ae +QBmaVOlA5Kw1fGUICKGRrfNZDeUHMT7artoWjhcxiyciy4kGY3DKxEv/8vnshe1DKMw OetQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q4MDyK/xpZ2rJeVO6MpkOTduxc1guqAC/giITauVW44=; b=paGYYIz6CvNil1qZ2FPzL8CuaY/rQ9qf7MuRToAlzP+3oaM6OZ14zhWnrnKCuGvIv6 W+10LQsU+QBF8Sj+rN+cVogt0xHRNCcXAhK0n4qcsg04a/QozAwSo/FRVfse9smVM0i4 Rej53rYlIyzivUEYpbd87PEc1Dujha96xRFlSaZF8b5qFZbw8rKJcwPytyQJNTs7JtJO og9umV3zeSe0BCaXQhT1g/VLYuwuLDeAv2BzBC8+6NCGeGLAbh+ptBx7L+OBf/nhsj0U tYNhkjW7JgOQj/Xjopw6Zur/RkRkinhMpZ6Zi31oxk35lN3r9IVd+RhMWxWZ+a7U8nhy 5q9g== X-Gm-Message-State: ABuFfogX5jP48+TzzK/3tv2ighoJ5dj5EEgkZ8aI9cUn0QfhimErZ8Ng XWZVAVV7zoPEN9fu1J2CffNf8w== X-Google-Smtp-Source: ACcGV63p6PiO4kRynN1SImA1bh+u7lfmvLd67LFZhHAhNZG1ru/jmBYEsyYKLf3kbZ2G2G45MLBwsw== X-Received: by 2002:a62:6547:: with SMTP id z68-v6mr12539030pfb.20.1538416833339; Mon, 01 Oct 2018 11:00:33 -0700 (PDT) Received: from nebulus.mtv.corp.google.com ([2620:0:1000:1612:b4fb:6752:f21f:3502]) by smtp.gmail.com with ESMTPSA id d186-v6sm24469261pfg.173.2018.10.01.11.00.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Oct 2018 11:00:32 -0700 (PDT) From: Mark Salyzyn To: linux-kernel@vger.kernel.org Subject: RESEND [PATCH v5 03/12] arm: vdso: inline assembler operations to compiler.h Date: Mon, 1 Oct 2018 10:58:23 -0700 Message-Id: <20181001175845.168430-4-salyzyn@android.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog In-Reply-To: <20181001175845.168430-1-salyzyn@android.com> References: <20181001175845.168430-1-salyzyn@android.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181001_140050_493262_FE096CAE X-CRM114-Status: GOOD ( 32.47 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Laura Abbott , Kees Cook , Ard Biesheuvel , "Paul E. McKenney" , Catalin Marinas , Kevin Brodsky , Will Deacon , Russell King , Jeremy Linton , Andy Lutomirski , android-kernel@android.com, James Morse , Andrew Pinski , Dmitry Safonov , Andy Gross , John Stultz , Thomas Gleixner , Mark Salyzyn , Ingo Molnar , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Take an effort to recode the arm64 vdso code from assembler to C previously submitted by Andrew Pinski , rework it for use in both arm and arm64, overlapping any optimizations for each architecture. But instead of landing it in arm64, land the result into lib/vdso and unify both implementations to simplify future maintenance. Move compiler-specific code to a local compiler.h file: - CONFIG_AEABI dependency check. - System call fallback functions standardized into a DEFINE_FALLBACK macro. - Replace arch_counter_get_cntvct() with arch_vdso_read_counter. - Deal with architecture specific unresolved references emitted by GCC. - Optimize handling of fallback calls in callers. - For time functions that always return success, do not waste time checking return value for switch to fallback. - Optimize unlikely nullptr checking in __vdso_gettimeofday, if tv null no need to proceed to fallback, as vdso is still capable of filling in the tv values. Signed-off-by: Mark Salyzyn Tested-by: Mark Salyzyn Cc: James Morse Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Andy Lutomirski Cc: Dmitry Safonov Cc: John Stultz Cc: Mark Rutland Cc: Laura Abbott Cc: Kees Cook Cc: Ard Biesheuvel Cc: Andy Gross Cc: Kevin Brodsky Cc: Andrew Pinski Cc: Thomas Gleixner Cc: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: Jeremy Linton Cc: android-kernel@android.com v2: - split first CL into 3 of 7 pieces v4: - switch to arch_vdso_read_counter as common API - update commit message to reflect overall reasoning v5: - comment about asm/arch_timer.h and asm/processor.h regarding why they have been added in compiler.h for porting clarity. - add linux/compiler.h in vgettimeofday.c because of notrace. - remove unnecessary dependency on linux/hrtimer.h --- arch/arm/vdso/compiler.h | 69 +++++++++++++++++++++ arch/arm/vdso/vgettimeofday.c | 109 +++++++++------------------------- 2 files changed, 96 insertions(+), 82 deletions(-) create mode 100644 arch/arm/vdso/compiler.h diff --git a/arch/arm/vdso/compiler.h b/arch/arm/vdso/compiler.h new file mode 100644 index 000000000000..af24502797e8 --- /dev/null +++ b/arch/arm/vdso/compiler.h @@ -0,0 +1,69 @@ +/* + * Userspace implementations of fallback calls + * + * Copyright (C) 2017 Cavium, Inc. + * Copyright (C) 2012 ARM Limited + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * Author: Will Deacon + * Rewriten into C by: Andrew Pinski + */ + +#ifndef __VDSO_COMPILER_H +#define __VDSO_COMPILER_H + +#include /* for arch_counter_get_cntvct() */ +#include /* for cpu_relax() */ +#include +#include + +#ifndef CONFIG_AEABI +#error This code depends on AEABI system call conventions +#endif + +#define DEFINE_FALLBACK(name, type_arg1, name_arg1, type_arg2, name_arg2) \ +static notrace long name##_fallback(type_arg1 _##name_arg1, \ + type_arg2 _##name_arg2) \ +{ \ + register type_arg1 name_arg1 asm("r0") = _##name_arg1; \ + register type_arg2 name_arg2 asm("r1") = _##name_arg2; \ + register long ret asm ("r0"); \ + register long nr asm("r7") = __NR_##name; \ + \ + asm volatile( \ + " swi #0\n" \ + : "=r" (ret) \ + : "r" (name_arg1), "r" (name_arg2), "r" (nr) \ + : "memory"); \ + \ + return ret; \ +} + +#define arch_vdso_read_counter() arch_counter_get_cntvct() + +/* Avoid unresolved references emitted by GCC */ + +void __aeabi_unwind_cpp_pr0(void) +{ +} + +void __aeabi_unwind_cpp_pr1(void) +{ +} + +void __aeabi_unwind_cpp_pr2(void) +{ +} + +#endif /* __VDSO_COMPILER_H */ diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c index 2474c17dc356..522094b147a2 100644 --- a/arch/arm/vdso/vgettimeofday.c +++ b/arch/arm/vdso/vgettimeofday.c @@ -22,21 +22,16 @@ * Reworked and rebased over arm version by: Mark Salyzyn */ -#include -#include -#include -#include #include -#include -#include -#include - -#ifndef CONFIG_AEABI -#error This code depends on AEABI system call conventions -#endif +#include /* for notrace */ +#include +#include "compiler.h" #include "datapage.h" +DEFINE_FALLBACK(gettimeofday, struct timeval *, tv, struct timezone *, tz) +DEFINE_FALLBACK(clock_gettime, clockid_t, clock, struct timespec *, ts) + static notrace u32 vdso_read_begin(const struct vdso_data *vd) { u32 seq; @@ -63,23 +58,6 @@ static notrace int vdso_read_retry(const struct vdso_data *vd, u32 start) return seq != start; } -static notrace long clock_gettime_fallback(clockid_t _clkid, - struct timespec *_ts) -{ - register struct timespec *ts asm("r1") = _ts; - register clockid_t clkid asm("r0") = _clkid; - register long ret asm ("r0"); - register long nr asm("r7") = __NR_clock_gettime; - - asm volatile( - " swi #0\n" - : "=r" (ret) - : "r" (clkid), "r" (ts), "r" (nr) - : "memory"); - - return ret; -} - static notrace int do_realtime_coarse(const struct vdso_data *vd, struct timespec *ts) { @@ -127,7 +105,7 @@ static notrace u64 get_ns(const struct vdso_data *vd) u64 cycle_now; u64 nsec; - cycle_now = arch_counter_get_cntvct(); + cycle_now = arch_vdso_read_counter(); cycle_delta = (cycle_now - vd->cs_cycle_last) & vd->cs_mask; @@ -200,85 +178,52 @@ static notrace int do_monotonic(const struct vdso_data *vd, struct timespec *ts) #endif /* CONFIG_ARM_ARCH_TIMER */ -notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts) +notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts) { - int ret = -1; - const struct vdso_data *vd = __get_datapage(); - switch (clkid) { + switch (clock) { case CLOCK_REALTIME_COARSE: - ret = do_realtime_coarse(vd, ts); + do_realtime_coarse(vd, ts); break; case CLOCK_MONOTONIC_COARSE: - ret = do_monotonic_coarse(vd, ts); + do_monotonic_coarse(vd, ts); break; case CLOCK_REALTIME: - ret = do_realtime(vd, ts); + if (do_realtime(vd, ts)) + goto fallback; break; case CLOCK_MONOTONIC: - ret = do_monotonic(vd, ts); + if (do_monotonic(vd, ts)) + goto fallback; break; default: - break; + goto fallback; } - if (ret) - ret = clock_gettime_fallback(clkid, ts); - - return ret; -} - -static notrace long gettimeofday_fallback(struct timeval *_tv, - struct timezone *_tz) -{ - register struct timezone *tz asm("r1") = _tz; - register struct timeval *tv asm("r0") = _tv; - register long ret asm ("r0"); - register long nr asm("r7") = __NR_gettimeofday; - - asm volatile( - " swi #0\n" - : "=r" (ret) - : "r" (tv), "r" (tz), "r" (nr) - : "memory"); - - return ret; + return 0; +fallback: + return clock_gettime_fallback(clock, ts); } notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz) { - struct timespec ts; - int ret; - const struct vdso_data *vd = __get_datapage(); - ret = do_realtime(vd, &ts); - if (ret) - return gettimeofday_fallback(tv, tz); + if (likely(tv != NULL)) { + struct timespec ts; + + if (do_realtime(vd, &ts)) + return gettimeofday_fallback(tv, tz); - if (tv) { tv->tv_sec = ts.tv_sec; tv->tv_usec = ts.tv_nsec / 1000; } - if (tz) { + + if (unlikely(tz != NULL)) { tz->tz_minuteswest = vd->tz_minuteswest; tz->tz_dsttime = vd->tz_dsttime; } - return ret; -} - -/* Avoid unresolved references emitted by GCC */ - -void __aeabi_unwind_cpp_pr0(void) -{ -} - -void __aeabi_unwind_cpp_pr1(void) -{ -} - -void __aeabi_unwind_cpp_pr2(void) -{ + return 0; }