From patchwork Fri May 3 17:18:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13653157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 53F7CC4345F for ; Fri, 3 May 2024 17:19:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=7vceydTsefWTvIuDTyJndYc72zuxCBeDmaQsLyDb4Mg=; b=kSoVYQM11mxgs0 21YvmeAawEkta9MM7rgzxbZWgd2EKvwPKEDh6Q8I6GREjttpXqdk5Tv+R7BUegWG2H20EqDTglSn4 JS9SJCy+mzRapr8Xqlmd60FggcxuXMg5Y1SCb4gMDCGVBAH52TS9h4G4rG60oiR6BrPJ2De0nnz1D ENR87vbZiCWQggsSRCjhLOC2EjKp56vMUg1KkdezZpw9X6ZFR0rqEncTKM40PNrC4LZXx+5ljR7WZ jAxoDrNds9OEETnLvR/fZjLo4wjC7ooT2jbq8NyaL93kD8Y/EZulIIyEkz/079MbDJ6IkdqghEO7C DL6z240Tct9T0An0Yadw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s2wYt-0000000HMSc-1RLZ; Fri, 03 May 2024 17:19:07 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s2wYq-0000000HMQC-1qev for linux-arm-kernel@lists.infradead.org; Fri, 03 May 2024 17:19:05 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 37378CE19D2; Fri, 3 May 2024 17:19:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 257BCC116B1; Fri, 3 May 2024 17:19:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1714756741; bh=sxFjeHo/U9d6yJ4gu/S8oCYHW6M/QMHpy5fuSlTg3o8=; h=From:To:Cc:Subject:Date:From; b=hIRN+DUFCywNEszWU9lwdZU2kY1GdPQ/vI695iOCohOzJou/FEySpToU/05dvz1P0 ZGNOaxMi/LHxI8UwG6/irrNE79FoDmsE6N81uZG/60U+ISuXKU4vUNVodtvYFA+4i0 dT1+Y3XUIr/2WwBj57H5tUUL9bgXm3QkzWI17B7W6w9mrZjvSPoXA4BnaXNZOMJiIQ NDrAUyHfKGMaCnvapDfx5CuyznN2IiuyDyL2YFAwhaVzhlCKXXh3prw7StOm4ApR6A n9LAHQYNU74Hb1IfUBy6AZlV7JSGaMh8FctgeyyunHhpfzdXJU7/TNdWUR/5yceNDd DyXx3tDu99nJQ== From: Puranjay Mohan To: Catalin Marinas , Will Deacon , Sumit Garg , Stephen Boyd , Douglas Anderson , "Peter Zijlstra (Intel)" , Thomas Gleixner , Mark Rutland , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: puranjay12@gmail.com Subject: [PATCH v3 1/2] arm64/arch_timer: include Date: Fri, 3 May 2024 17:18:46 +0000 Message-Id: <20240503171847.68267-1-puranjay@kernel.org> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240503_101904_686325_05A8CA7F X-CRM114-Status: GOOD ( 12.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org arch_timer.h includes linux/smp.h since the commit: 6acc71ccac7187fc ("arm64: arch_timer: Allows a CPU-specific erratum to only affect a subset of CPUs") It was included to use DEFINE_PER_CPU(), etc. But It should have included rather than . It worked because smp.h includes percpu.h. The next commit will remove percpu.h from smp.h and it will break this usage. Explicitly include percpu.h and remove smp.h Signed-off-by: Puranjay Mohan Acked-by: Mark Rutland Reviewed-by: Stephen Boyd --- arch/arm64/include/asm/arch_timer.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h index 934c658ee947..f5794d50f51d 100644 --- a/arch/arm64/include/asm/arch_timer.h +++ b/arch/arm64/include/asm/arch_timer.h @@ -15,7 +15,7 @@ #include #include #include -#include +#include #include #include From patchwork Fri May 3 17:18:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13653158 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A7F0AC4345F for ; Fri, 3 May 2024 17:19:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+AuECGvrKd2EYdcOy/AM8VUJhwIr0C6RCeCxMWrst8Q=; b=NFZ9NpoqfFvfCW o5cmr1JEWAFJeXSV1INB9pzAcaPWHIyrkm1yteH6DgU4l10y1x1MbwaoAzNo6g8/p79NA3tk3iPZx vqy1ARomXIKn+fXxM0Vt3ctgPhrmhTh02vHOGGHhVLmGk/0VwHkiCWgx9ChDxnGyxqpgWJ1HLRBb4 mWEG1PnZb/BhefOgKf964nwjnBI+hVdRhxdhGeidwEhBFV0N1y52bOEW0vmLVeWJmazCmBApqmYhE 5UiAMRSlM/W9NjNj4ZheGWamabIiL59dX9RMD0yajlG5UWxOtMmtcp5dsvzKJ/xXytI4mdBJxt3LA pRYeQJPJCVR8gnXmGLjA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s2wYw-0000000HMU6-09o4; Fri, 03 May 2024 17:19:10 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s2wYs-0000000HMS8-2Nma for linux-arm-kernel@lists.infradead.org; Fri, 03 May 2024 17:19:08 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id E9A6161DD3; Fri, 3 May 2024 17:19:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79A30C116B1; Fri, 3 May 2024 17:19:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1714756745; bh=SKPGGFh3wFauCRM9bosYEwSGbWHR5tDTuYSe2kSfmNQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qI3izxXwFqNX4HpMhaKsfqCMLJdgy+IySCBeCY29ifhaPt902Hca20463lWJ3VH+/ Tqj338XIPGTpolG8s+ut9LJWxjNZKPNtbZ7qfwPvVyDqp9Tm9l5CC+TGRSeLs0r/QA ewaJpUaDScg/uFwSOxlI+GFyhHYZwV4o7542q9DdMg/vBiQ9SZB0CpY4vWp1IZJht2 4niqwjcIgnYWZcD6HeGFMplquUnrQfLLUXW9bFHPtKBSqQ0rYQX9UzsLwYiksBYGUI 0SPKxpx7bWmyoPsuMjCLj3VLJYC9MmZnIzb3GXAktmb+3T1ESmuPd/PmgEjakJ4E2E 8n4c87SVmbp6g== From: Puranjay Mohan To: Catalin Marinas , Will Deacon , Sumit Garg , Stephen Boyd , Douglas Anderson , "Peter Zijlstra (Intel)" , Thomas Gleixner , Mark Rutland , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: puranjay12@gmail.com Subject: [PATCH v3 2/2] arm64: implement raw_smp_processor_id() using thread_info Date: Fri, 3 May 2024 17:18:47 +0000 Message-Id: <20240503171847.68267-2-puranjay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240503171847.68267-1-puranjay@kernel.org> References: <20240503171847.68267-1-puranjay@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240503_101906_717605_8D574747 X-CRM114-Status: GOOD ( 18.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Historically, arm64 implemented raw_smp_processor_id() as a read of current_thread_info()->cpu. This changed when arm64 moved thread_info into task struct, as at the time CONFIG_THREAD_INFO_IN_TASK made core code use thread_struct::cpu for the cpu number, and due to header dependencies prevented using this in raw_smp_processor_id(). As a workaround, we moved to using a percpu variable in commit: 57c82954e77fa12c ("arm64: make cpu number a percpu variable") Since then, thread_info::cpu was reintroduced, and core code was made to use this in commits: 001430c1910df65a ("arm64: add CPU field to struct thread_info") bcf9033e5449bdca ("sched: move CPU field back into thread_info if THREAD_INFO_IN_TASK=y") Consequently it is possible to use current_thread_info()->cpu again. This decreases the number of emitted instructions like in the following example: Dump of assembler code for function bpf_get_smp_processor_id: 0xffff8000802cd608 <+0>: nop 0xffff8000802cd60c <+4>: nop 0xffff8000802cd610 <+8>: adrp x0, 0xffff800082138000 0xffff8000802cd614 <+12>: mrs x1, tpidr_el1 0xffff8000802cd618 <+16>: add x0, x0, #0x8 0xffff8000802cd61c <+20>: ldrsw x0, [x0, x1] 0xffff8000802cd620 <+24>: ret After this patch: Dump of assembler code for function bpf_get_smp_processor_id: 0xffff8000802c9130 <+0>: nop 0xffff8000802c9134 <+4>: nop 0xffff8000802c9138 <+8>: mrs x0, sp_el0 0xffff8000802c913c <+12>: ldr w0, [x0, #24] 0xffff8000802c9140 <+16>: ret A microbenchmark[1] was built to measure the performance improvement provided by this change. It calls the following function given number of times and finds the runtime overhead: static noinline int get_cpu_id(void) { return smp_processor_id(); } Run the benchmark like: modprobe smp_processor_id nr_function_calls=1000000000 +--------------------------+------------------------+ | | Number of Calls | Time taken | +--------+-----------------+------------------------+ | Before | 1000000000 | 1602888401ns | +--------+-----------------+------------------------+ | After | 1000000000 | 1206212658ns | +--------+-----------------+------------------------+ | Difference (decrease) | 396675743ns (24.74%) | +---------------------------------------------------+ Remove the percpu variable cpu_number as it is used only in set_smp_ipi_range() as a dummy variable to be passed to ipi_handler(). Use irq_stat in place of cpu_number here like arm32. [1] https://github.com/puranjaymohan/linux/commit/77d3fdd Signed-off-by: Puranjay Mohan Acked-by: Mark Rutland Reviewed-by: Stephen Boyd --- arch/arm64/include/asm/smp.h | 13 +------------ arch/arm64/kernel/smp.c | 9 ++------- 2 files changed, 3 insertions(+), 19 deletions(-) diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index efb13112b408..2510eec026f7 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -25,22 +25,11 @@ #ifndef __ASSEMBLY__ -#include - #include #include #include -DECLARE_PER_CPU_READ_MOSTLY(int, cpu_number); - -/* - * We don't use this_cpu_read(cpu_number) as that has implicit writes to - * preempt_count, and associated (compiler) barriers, that we'd like to avoid - * the expense of. If we're preemptible, the value can be stale at use anyway. - * And we can't use this_cpu_ptr() either, as that winds up recursing back - * here under CONFIG_DEBUG_PREEMPT=y. - */ -#define raw_smp_processor_id() (*raw_cpu_ptr(&cpu_number)) +#define raw_smp_processor_id() (current_thread_info()->cpu) /* * Logical CPU mapping. diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 4ced34f62dab..98d4e352c3d0 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -55,9 +55,6 @@ #include -DEFINE_PER_CPU_READ_MOSTLY(int, cpu_number); -EXPORT_PER_CPU_SYMBOL(cpu_number); - /* * as from 2.5, kernels no longer have an init_tasks structure * so we need some other way of telling a new secondary core @@ -742,8 +739,6 @@ void __init smp_prepare_cpus(unsigned int max_cpus) */ for_each_possible_cpu(cpu) { - per_cpu(cpu_number, cpu) = cpu; - if (cpu == smp_processor_id()) continue; @@ -1021,12 +1016,12 @@ void __init set_smp_ipi_range(int ipi_base, int n) if (ipi_should_be_nmi(i)) { err = request_percpu_nmi(ipi_base + i, ipi_handler, - "IPI", &cpu_number); + "IPI", &irq_stat); WARN(err, "Could not request IPI %d as NMI, err=%d\n", i, err); } else { err = request_percpu_irq(ipi_base + i, ipi_handler, - "IPI", &cpu_number); + "IPI", &irq_stat); WARN(err, "Could not request IPI %d as IRQ, err=%d\n", i, err); }