From patchwork Thu Oct 12 22:58:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Salyzyn X-Patchwork-Id: 10003133 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7D215602BF for ; Thu, 12 Oct 2017 22:59:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6BD3528997 for ; Thu, 12 Oct 2017 22:59:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5FA6D28B2D; Thu, 12 Oct 2017 22:59:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BEB8128B2D for ; Thu, 12 Oct 2017 22:59:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=7hPdJr4JJRrkVnGC1BWeS8k8SfM7jC8MOH5eXW5ib78=; b=D+n /qAbH4aU3ndd9kKmNbg8Q5Y+1EiHsFUaYdzkKshzY0oJfV+guB32LMTG0R0P8Ht4aWO8Ped/vecJ3 XaZv0KctW6/Yb8JhzFjFPjaPybBXMBGKuOnWprZwcPD6saF++g8nwDHhVZTP6mLtOwIUdo6wACaCy PYuNuLHb18F4afyf3ltYB22EkCr/V5xbfKpmj1rqtALSekhhdQhiMAZqBpMhhNVt6jvTGlZUTfQwY Ewgq87tNBqeyJjAQLTx3clQjcvGWr0WV8cxDuk1H9zq3dDoslK3zniuVtv10XmggpNbK6obMfouwI j98RslP3HDf6PVwYL1Dned7d2pz8zvA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e2mS2-0003cK-Bu; Thu, 12 Oct 2017 22:59:38 +0000 Received: from mail-pf0-x233.google.com ([2607:f8b0:400e:c00::233]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1e2mRt-0003TT-IY for linux-arm-kernel@lists.infradead.org; Thu, 12 Oct 2017 22:59:35 +0000 Received: by mail-pf0-x233.google.com with SMTP id l188so7036118pfc.6 for ; Thu, 12 Oct 2017 15:59:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=android.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=PGywV3lqT6ErdJsqhCVcTQxTDR0zkP/bxtXDtsXhhzA=; b=NnW4mLK88DzRKWhcKndourLXct/p5aFpIW66YOCxO9x0il89ceZyxiVNfs1PJi+Aw3 lDumbqPnsZFWys8w3fmfKuZs7js9pk1VgOzaE98pf9Tn4D3gi7J+aDrJVtjo5/+x0SfG 5w/dbfDV+BP7XqqECUtdQYFNabTbh2vzoXHSqcgidfvDVUf6TPmMJOVh6qkQ8rBqUWCL hcArvYHd5WbWCFZpSelUGQzAkdjmnNwbzX7IUWYOgZv9PRguXPBs8weZp6yCswig7/Np ZUOrNvWTWM7D4JW703LlCO82K4FFVDl7yZsAFBTjAjM0ieQYa+mVsVHvrwH2DTQ94TzZ FpRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=PGywV3lqT6ErdJsqhCVcTQxTDR0zkP/bxtXDtsXhhzA=; b=s87dzHdbI7QRoV1X5gxrXmd+ROjcQYrzBtOn0texDV0KH6a799/NuRW8nYuT9NbFPF EmJCEETRcCWYvsJE0jMos1Beula1nACQ4KITOiubbqbg3RYD3md5Z6g3Lu27ctjSEDqw rrHV5x6rthLsdYcOUcryhJ0lSArg3GJ5/wi6mIIgxvGt++Z6zhD5AxLHiczyHeTP0ygF g/+0g07Zh/BQI07nhQ5Pw7+iKOxU2Xt2pIJM+GNVnMGHIF+VBmguUtsxxuvRkEicogx8 yPOKRXvoYwHz5Aybyuqo6g5W6mfbgelAfXl3kiwjv/puw/RCfbIld/m7qac95Rx9Blhd KrEA== X-Gm-Message-State: AMCzsaUiyHEm2Kf2pkucJpd7XaSlIBWFtZX8OZ1pUR6bKtjZVbzqHlKR 57Tw8qL8nGQftaygjE8Actxn7Q== X-Google-Smtp-Source: AOwi7QAPVKkNzKZBczBQ+UYS3nWf522FhSJvPBpJendi8xZ90t5ruNZMWgcWU25iJdE1+66YswaSZw== X-Received: by 10.99.168.76 with SMTP id i12mr1373293pgp.427.1507849147682; Thu, 12 Oct 2017 15:59:07 -0700 (PDT) Received: from nebulus.mtv.corp.google.com ([100.98.120.17]) by smtp.gmail.com with ESMTPSA id h186sm5357180pfg.47.2017.10.12.15.59.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 12 Oct 2017 15:59:07 -0700 (PDT) From: Mark Salyzyn To: linux-kernel@vger.kernel.org Subject: [PATCH v2 1/10] arm: vdso: rename vdso_datapage variables Date: Thu, 12 Oct 2017 15:58:46 -0700 Message-Id: <20171012225853.86413-1-salyzyn@android.com> X-Mailer: git-send-email 2.15.0.rc0.271.g36b669edcc-goog X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171012_155929_738014_BA7D12E0 X-CRM114-Status: GOOD ( 21.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dmitry Safonov , Russell King , Andy Lutomirski , james.morse@arm.com, linux-arm-kernel@lists.infradead.org, Mark Salyzyn , Mark Salyzyn MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Rename seq_count to tb_seq_count. Rename tk_is_cntvct to use_syscall. Rename cs_mult to cs_mono_mult. All to align with the variables in the arm64 vdso datapage. Rework vdso_read_begin() and vdso_read_retry() functions to reflect modern access patterns for tb_seq_count field. Update copyright message to reflect the start of the contributions in this series. Signed-off-by: Mark Salyzyn v2: split first CL into 1 of 7 pieces --- arch/arm/include/asm/vdso_datapage.h | 6 ++-- arch/arm/kernel/vdso.c | 17 +++++----- arch/arm/vdso/vgettimeofday.c | 61 +++++++++++++++++++----------------- 3 files changed, 45 insertions(+), 39 deletions(-) diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h index 9be259442fca..fa3e1856244d 100644 --- a/arch/arm/include/asm/vdso_datapage.h +++ b/arch/arm/include/asm/vdso_datapage.h @@ -29,8 +29,8 @@ * 32 bytes. */ struct vdso_data { - u32 seq_count; /* sequence count - odd during updates */ - u16 tk_is_cntvct; /* fall back to syscall if false */ + u32 tb_seq_count; /* sequence count - odd during updates */ + u16 use_syscall; /* fall back to syscall if true */ u16 cs_shift; /* clocksource shift */ u32 xtime_coarse_sec; /* coarse time */ u32 xtime_coarse_nsec; @@ -38,7 +38,7 @@ struct vdso_data { u32 wtm_clock_sec; /* wall to monotonic offset */ u32 wtm_clock_nsec; u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */ - u32 cs_mult; /* clocksource multiplier */ + u32 cs_mono_mult; /* clocksource multiplier */ u64 cs_cycle_last; /* last cycle value */ u64 cs_mask; /* clocksource mask */ diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c index a4d6dc0f2427..aed84b659cc5 100644 --- a/arch/arm/kernel/vdso.c +++ b/arch/arm/kernel/vdso.c @@ -274,14 +274,14 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr) static void vdso_write_begin(struct vdso_data *vdata) { - ++vdso_data->seq_count; + ++vdso_data->tb_seq_count; smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */ } static void vdso_write_end(struct vdso_data *vdata) { smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */ - ++vdso_data->seq_count; + ++vdso_data->tb_seq_count; } static bool tk_is_cntvct(const struct timekeeper *tk) @@ -305,10 +305,10 @@ static bool tk_is_cntvct(const struct timekeeper *tk) * counter again, making it even, indicating to userspace that the * update is finished. * - * Userspace is expected to sample seq_count before reading any other - * fields from the data page. If seq_count is odd, userspace is + * Userspace is expected to sample tb_seq_count before reading any other + * fields from the data page. If tb_seq_count is odd, userspace is * expected to wait until it becomes even. After copying data from - * the page, userspace must sample seq_count again; if it has changed + * the page, userspace must sample tb_seq_count again; if it has changed * from its previous value, userspace must retry the whole sequence. * * Calls to update_vsyscall are serialized by the timekeeping core. @@ -326,18 +326,19 @@ void update_vsyscall(struct timekeeper *tk) vdso_write_begin(vdso_data); - vdso_data->tk_is_cntvct = tk_is_cntvct(tk); + vdso_data->use_syscall = !tk_is_cntvct(tk); vdso_data->xtime_coarse_sec = tk->xtime_sec; vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift); vdso_data->wtm_clock_sec = wtm->tv_sec; vdso_data->wtm_clock_nsec = wtm->tv_nsec; - if (vdso_data->tk_is_cntvct) { + if (!vdso_data->use_syscall) { vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last; vdso_data->xtime_clock_sec = tk->xtime_sec; vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec; - vdso_data->cs_mult = tk->tkr_mono.mult; + vdso_data->cs_mono_mult = tk->tkr_mono.mult; + /* tkr_mono.shift == tkr_raw.shift */ vdso_data->cs_shift = tk->tkr_mono.shift; vdso_data->cs_mask = tk->tkr_mono.mask; } diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c index 79214d5ff097..8cf13af1323c 100644 --- a/arch/arm/vdso/vgettimeofday.c +++ b/arch/arm/vdso/vgettimeofday.c @@ -1,18 +1,25 @@ /* - * Copyright 2015 Mentor Graphics Corporation. + * Userspace implementations of gettimeofday() and friends. * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License - * as published by the Free Software Foundation; version 2 of the - * License. + * Copyright (C) 2017 Cavium, Inc. + * Copyright (C) 2015 Mentor Graphics Corporation + * Copyright (C) 2012 ARM Limited * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License for more details. + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program. If not, see . + * + * Author: Will Deacon + * Rewriten from arch64 version into C by: Andrew Pinski + * Reworked and rebased over arm version by: Mark Salyzyn */ #include @@ -31,32 +38,30 @@ extern struct vdso_data *__get_datapage(void); -static notrace u32 __vdso_read_begin(const struct vdso_data *vdata) -{ - u32 seq; -repeat: - seq = ACCESS_ONCE(vdata->seq_count); - if (seq & 1) { - cpu_relax(); - goto repeat; - } - return seq; -} - static notrace u32 vdso_read_begin(const struct vdso_data *vdata) { u32 seq; - seq = __vdso_read_begin(vdata); + do { + seq = READ_ONCE(vdata->tb_seq_count); + + if ((seq & 1) == 0) + break; - smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */ + cpu_relax(); + } while (true); + + smp_rmb(); /* Pairs with second smp_wmb in update_vsyscall */ return seq; } static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start) { - smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */ - return vdata->seq_count != start; + u32 seq; + + smp_rmb(); /* Pairs with first smp_wmb in update_vsyscall */ + seq = READ_ONCE(vdata->tb_seq_count); + return seq != start; } static notrace long clock_gettime_fallback(clockid_t _clkid, @@ -127,7 +132,7 @@ static notrace u64 get_ns(struct vdso_data *vdata) cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask; - nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec; + nsec = (cycle_delta * vdata->cs_mono_mult) + vdata->xtime_clock_snsec; nsec >>= vdata->cs_shift; return nsec; @@ -141,7 +146,7 @@ static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata) do { seq = vdso_read_begin(vdata); - if (!vdata->tk_is_cntvct) + if (vdata->use_syscall) return -1; ts->tv_sec = vdata->xtime_clock_sec; @@ -164,7 +169,7 @@ static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata) do { seq = vdso_read_begin(vdata); - if (!vdata->tk_is_cntvct) + if (vdata->use_syscall) return -1; ts->tv_sec = vdata->xtime_clock_sec;