From patchwork Wed May 1 15:42:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13650969 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 22275C10F1A for ; Wed, 1 May 2024 15:43:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=KMTbI8r9bVCvfvYw3Ooid087x+ykqIHbx0aUkPaRlpw=; b=Nad8B4CW/3pbVk aOdi0kMMMb9SL34D/0IfuTSFmvr9eFthpE/xkxMu+CvHODRrXQg5Ck78vRk8dVHcEp2u1tAgdmzk9 MYE9is7fMb39LsrKIL2eKqkmbjAE5Vjxsil4cFwcHK5Dkz5oJDgHk3DmlUjPb59/YdzjJGVr5j0s5 WX3po+bsxy7vP+Dx4As4px7FirLuQdHGhP5QA0MdehRojnu+nEuBf00cyqt3jPxKdKyBfxmYv0DIq M1OTGMzFgjaRyrGea4g2Mkl3K4kpXVTBASdJBqZqd75iAqRCt/XKg4xZipbjmmkxpJ8NXajfzmZIp 6fvO7mH0uWPgp5dIF78g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s2C7E-0000000A1jF-2lIH; Wed, 01 May 2024 15:43:28 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s2C7B-0000000A1hM-1xpH for linux-arm-kernel@lists.infradead.org; Wed, 01 May 2024 15:43:27 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 8CFE26198A; Wed, 1 May 2024 15:43:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6A5EFC072AA; Wed, 1 May 2024 15:43:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1714578203; bh=KALlXTVUTaG7KIY4xfSeGKWZAwSJ5/dytvx09z4PAuA=; h=From:To:Cc:Subject:Date:From; b=th6c1+mm5tEOQDCQZKB6/0pXmDxMTXaBzvR1lpXEFQJPkq9GJSEKcwTW2jyNHYjr7 TfGDpwsO2mcdGdRbuwZigx9Qvocz20fn2W+ubLTx30PXku5RilOYMB2Pm2QSC6NIK8 W/P0Cd93k5yfwdAzOyNsRifq291WpbE2ZPbL7m+Fp44F+Q4gcMs+qhHNtAzUdnSsN+ WBTDTW0CerAW/OQ1YH4uUi43tiRvR/8jNPJ8S3A6q84o/5GXHwi6/sliWmj/b1HgT2 TlLIyU76GEGF6bQNrV9tHqkhDRtg6xiA7qsYh2XZynGE1eOyxQe0xEgGqwOzMhFrR2 +y4cw2FaaZ90A== From: Puranjay Mohan To: Catalin Marinas , Will Deacon , Sumit Garg , Stephen Boyd , Douglas Anderson , "Peter Zijlstra (Intel)" , Thomas Gleixner , Mark Rutland , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: puranjay12@gmail.com Subject: [PATCH] arm64: implement raw_smp_processor_id() using thread_info Date: Wed, 1 May 2024 15:42:36 +0000 Message-Id: <20240501154236.10236-1-puranjay@kernel.org> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240501_084325_640149_E284490A X-CRM114-Status: GOOD ( 15.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org ARM64 defines THREAD_INFO_IN_TASK which means the cpu id can be found from current_thread_info()->cpu. Implement raw_smp_processor_id() using the above. This decreases the number of emitted instructions like in the following example: Dump of assembler code for function bpf_get_smp_processor_id: 0xffff8000802cd608 <+0>: nop 0xffff8000802cd60c <+4>: nop 0xffff8000802cd610 <+8>: adrp x0, 0xffff800082138000 0xffff8000802cd614 <+12>: mrs x1, tpidr_el1 0xffff8000802cd618 <+16>: add x0, x0, #0x8 0xffff8000802cd61c <+20>: ldrsw x0, [x0, x1] 0xffff8000802cd620 <+24>: ret After this patch: Dump of assembler code for function bpf_get_smp_processor_id: 0xffff8000802c9130 <+0>: nop 0xffff8000802c9134 <+4>: nop 0xffff8000802c9138 <+8>: mrs x0, sp_el0 0xffff8000802c913c <+12>: ldr w0, [x0, #24] 0xffff8000802c9140 <+16>: ret A microbenchmark[1] was built to measure the performance improvement provided by this change. It calls the following function given number of times and finds the runtime overhead: static noinline int get_cpu_id(void) { return smp_processor_id(); } Run the benchmark like: modprobe smp_processor_id nr_function_calls=1000000000 +--------------------------+------------------------+ | | Number of Calls | Time taken | +--------+-----------------+------------------------+ | Before | 1000000000 | 1602888401ns | +--------+-----------------+------------------------+ | After | 1000000000 | 1206212658ns | +--------+-----------------+------------------------+ | Difference (decrease) | 396675743ns (24.74%) | +---------------------------------------------------+ This improvement is in this very specific microbenchmark but it proves the point. The percpu variable cpu_number is left as it is because it is used in set_smp_ipi_range() [1] https://github.com/puranjaymohan/linux/commit/77d3fdd Signed-off-by: Puranjay Mohan --- arch/arm64/include/asm/smp.h | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index efb13112b408..88fd2ab805ec 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -34,13 +34,9 @@ DECLARE_PER_CPU_READ_MOSTLY(int, cpu_number); /* - * We don't use this_cpu_read(cpu_number) as that has implicit writes to - * preempt_count, and associated (compiler) barriers, that we'd like to avoid - * the expense of. If we're preemptible, the value can be stale at use anyway. - * And we can't use this_cpu_ptr() either, as that winds up recursing back - * here under CONFIG_DEBUG_PREEMPT=y. + * This relies on THREAD_INFO_IN_TASK, but arm64 defines that unconditionally. */ -#define raw_smp_processor_id() (*raw_cpu_ptr(&cpu_number)) +#define raw_smp_processor_id() (current_thread_info()->cpu) /* * Logical CPU mapping.