From patchwork Sun Nov 11 03:20:40 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Herring X-Patchwork-Id: 1724771 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by patchwork2.kernel.org (Postfix) with ESMTP id 3AB85DFE7E for ; Sun, 11 Nov 2012 03:23:22 +0000 (UTC) Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.76 #1 (Red Hat Linux)) id 1TXO6c-0006Gp-0Z; Sun, 11 Nov 2012 03:21:06 +0000 Received: from mail-ob0-f177.google.com ([209.85.214.177]) by merlin.infradead.org with esmtps (Exim 4.76 #1 (Red Hat Linux)) id 1TXO6Y-0006GW-B6 for linux-arm-kernel@lists.infradead.org; Sun, 11 Nov 2012 03:21:03 +0000 Received: by mail-ob0-f177.google.com with SMTP id ta17so556463obb.36 for ; Sat, 10 Nov 2012 19:21:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:x-mailer; bh=pmXF6kIixuAOh9tBa15raEupIM1QIGeQoAJvLPlTSBc=; b=OhbCDVa8Mx/hSQz4nUxJj0E56ae8vaYN1YrnMl3xq9K1wWU0b7c/LwtCeSwFTO+3Fc FX54wFiWLvsNYu5WDSWN6wU0+0TgqdfsGgXeH17/33b1SIMYs4mC9E2Spl92xLug6dC9 jUcGea1mRStGLcJs7PXywygtTcL+wKcMYgx2lyvuXadLP44kLT7PyF/5QN44EmnJ78NS Y9ivStVQflfOdekePw/4sj8DE86uncG1BIu6Lt71PA/JUCJcSD/OM0VH2cM3ih3uCvAP 4FsUElrIhv7EQFNt51BIIxAKRBcgcIOf2wJTKzTWqt6mHyXkj8vgE//7FR1OwG/dhn03 +8tw== Received: by 10.182.156.99 with SMTP id wd3mr12408349obb.15.1352604061434; Sat, 10 Nov 2012 19:21:01 -0800 (PST) Received: from rob-laptop.grandenetworks.net (65-36-73-129.dyn.grandenetworks.net. [65.36.73.129]) by mx.google.com with ESMTPS id h13sm3226681obp.2.2012.11.10.19.20.59 (version=SSLv3 cipher=OTHER); Sat, 10 Nov 2012 19:21:00 -0800 (PST) From: Rob Herring To: linux-arm-kernel@lists.infradead.org Subject: [PATCH] ARM: implement optimized percpu variable access Date: Sat, 10 Nov 2012 21:20:40 -0600 Message-Id: <1352604040-10014-1-git-send-email-robherring2@gmail.com> X-Mailer: git-send-email 1.7.10.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20121110_222102_533267_53FDD0E9 X-CRM114-Status: GOOD ( 17.97 ) X-Spam-Score: -2.5 (--) X-Spam-Report: SpamAssassin version 3.3.2 on merlin.infradead.org summary: Content analysis details: (-2.5 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [209.85.214.177 listed in list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (robherring2[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in digit (robherring2[at]gmail.com) -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature Cc: will.deacon@arm.com, Russell King , Rob Herring , Nicolas Pitre X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: linux-arm-kernel-bounces@lists.infradead.org Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org From: Rob Herring Use the previously unused TPIDRPRW register to store percpu offsets. TPIDRPRW is only accessible in PL1, so it can only be used in the kernel. This saves 2 loads for each percpu variable access which should yield improved performance, but the improvement has not been quantified. Signed-off-by: Rob Herring --- arch/arm/include/asm/Kbuild | 1 - arch/arm/include/asm/percpu.h | 44 +++++++++++++++++++++++++++++++++++++++++ arch/arm/kernel/smp.c | 3 +++ 3 files changed, 47 insertions(+), 1 deletion(-) create mode 100644 arch/arm/include/asm/percpu.h diff --git a/arch/arm/include/asm/Kbuild b/arch/arm/include/asm/Kbuild index f70ae17..2ffdaac 100644 --- a/arch/arm/include/asm/Kbuild +++ b/arch/arm/include/asm/Kbuild @@ -16,7 +16,6 @@ generic-y += local64.h generic-y += msgbuf.h generic-y += param.h generic-y += parport.h -generic-y += percpu.h generic-y += poll.h generic-y += resource.h generic-y += sections.h diff --git a/arch/arm/include/asm/percpu.h b/arch/arm/include/asm/percpu.h new file mode 100644 index 0000000..9eb7372 --- /dev/null +++ b/arch/arm/include/asm/percpu.h @@ -0,0 +1,44 @@ +/* + * Copyright 2012 Calxeda, Inc. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ +#ifndef _ASM_ARM_PERCPU_H_ +#define _ASM_ARM_PERCPU_H_ + +/* + * Same as asm-generic/percpu.h, except that we store the per cpu offset + * in the TPIDRPRW. + */ +#if defined(CONFIG_SMP) && (__LINUX_ARM_ARCH__ >= 6) + +static inline void set_my_cpu_offset(unsigned long off) +{ + asm volatile("mcr p15, 0, %0, c13, c0, 4 @ set TPIDRPRW" : : "r" (off) : "cc" ); +} + +static inline unsigned long __my_cpu_offset(void) +{ + unsigned long off; + asm("mrc p15, 0, %0, c13, c0, 4 @ get TPIDRPRW" : "=r" (off) : ); + return off; +} +#define __my_cpu_offset __my_cpu_offset() +#else +#define set_my_cpu_offset(x) do {} while(0) + +#endif /* CONFIG_SMP */ + +#include + +#endif /* _ASM_ARM_PERCPU_H_ */ diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index fbc8b26..897ef60 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -313,6 +313,8 @@ asmlinkage void __cpuinit secondary_start_kernel(void) current->active_mm = mm; cpumask_set_cpu(cpu, mm_cpumask(mm)); + set_my_cpu_offset(per_cpu_offset(cpu)); + printk("CPU%u: Booted secondary processor\n", cpu); cpu_init(); @@ -371,6 +373,7 @@ void __init smp_cpus_done(unsigned int max_cpus) void __init smp_prepare_boot_cpu(void) { + set_my_cpu_offset(per_cpu_offset(smp_processor_id())); } void __init smp_prepare_cpus(unsigned int max_cpus)