From patchwork Fri Sep 3 12:19:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Leizhen (ThunderTown)" X-Patchwork-Id: 12474325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E43BDC433F5 for ; Fri, 3 Sep 2021 12:24:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A903E60E8B for ; Fri, 3 Sep 2021 12:24:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A903E60E8B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:CC :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=SwkMgEPNfCM9QXpjeG8E1GB4WHl8kPqMhc99srKvSuE=; b=sjxBMBvkAm9Y3r Ipcsn45dyJvM8B6jb+iVHeU5g2T3qY4YdFidogq1huOP2JM5lAFjHFzXuN4HCEzZKU2hlmMc+o8Qw OvVuH2UZZGjlF5x6ADiZssW0ZjEsZbhpl+vQCOxIrtEUWlXNZcT5M9Ql3AMMkg3OUsXhFGP3vtqxb +B7Eczo4R1LH+66lsy0NrrlaFToAQJoBuYuVUOYSndLzsIoZPGfuuOlGl6FqG0eOjwHr3BtwXogVc +VMjWO9ES3xjkQmz+25lqRDHcJGbZGalL+Zs4kB9iW3zIp93BCdMBWqvet8xUi0W0ESUslDPvGJV5 y64fEBXwXlH9mZ1Ddmlg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mM8Di-00Bu5F-1J; Fri, 03 Sep 2021 12:22:58 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mM8Dd-00Bu3n-Nt for linux-arm-kernel@lists.infradead.org; Fri, 03 Sep 2021 12:22:55 +0000 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4H1Gzp6SJqzbh1N; Fri, 3 Sep 2021 20:18:46 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 3 Sep 2021 20:22:40 +0800 Received: from thunder-town.china.huawei.com (10.174.178.242) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.8; Fri, 3 Sep 2021 20:22:39 +0800 From: Zhen Lei To: Catalin Marinas , Will Deacon , linux-arm-kernel , CC: Zhen Lei , Mark Rutland Subject: [PATCH] arm64: entry: Improve the performance of system calls Date: Fri, 3 Sep 2021 20:19:50 +0800 Message-ID: <20210903121950.2284-1-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.26.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.178.242] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210903_052254_171534_83AA2361 X-CRM114-Status: GOOD ( 10.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Commit 582f95835a8f ("arm64: entry: convert el0_sync to C") converted lots of functions from assembly to C, this greatly improves readability. But el0_svc()/el0_svc_compat() is in response to system call requests from user mode and may be in the hot path. Although the SVC is in the first case of the switch statement in C, the compiler optimizes the switch statement as a whole, and does not give SVC a small boost. Use "likely()" to help SVC directly invoke its handler after a simple judgment to avoid entering the switch table lookup process. After: 0000000000000ff0 : ff0: d503245f bti c ff4: d503233f paciasp ff8: a9bf7bfd stp x29, x30, [sp, #-16]! ffc: 910003fd mov x29, sp 1000: d5385201 mrs x1, esr_el1 1004: 531a7c22 lsr w2, w1, #26 1008: f100545f cmp x2, #0x15 100c: 540000a1 b.ne 1020 1010: 97fffe14 bl 860 1014: a8c17bfd ldp x29, x30, [sp], #16 1018: d50323bf autiasp 101c: d65f03c0 ret 1020: f100705f cmp x2, #0x1c Execute "./lat_syscall null" on my board (BogoMIPS : 200.00), it can save about 10ns. Before: Simple syscall: 0.2365 microseconds Simple syscall: 0.2354 microseconds Simple syscall: 0.2339 microseconds After: Simple syscall: 0.2255 microseconds Simple syscall: 0.2254 microseconds Simple syscall: 0.2256 microseconds Signed-off-by: Zhen Lei --- arch/arm64/kernel/entry-common.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 32f9796c4ffe77b..062eb5a895ec6f3 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -607,11 +607,14 @@ static void noinstr el0_fpac(struct pt_regs *regs, unsigned long esr) asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + unsigned long ec = ESR_ELx_EC(esr); - switch (ESR_ELx_EC(esr)) { - case ESR_ELx_EC_SVC64: + if (likely(ec == ESR_ELx_EC_SVC64)) { el0_svc(regs); - break; + return; + } + + switch (ec) { case ESR_ELx_EC_DABT_LOW: el0_da(regs, esr); break; @@ -730,11 +733,14 @@ static void noinstr el0_svc_compat(struct pt_regs *regs) asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + unsigned long ec = ESR_ELx_EC(esr); - switch (ESR_ELx_EC(esr)) { - case ESR_ELx_EC_SVC32: + if (likely(ec == ESR_ELx_EC_SVC32)) { el0_svc_compat(regs); - break; + return; + } + + switch (ec) { case ESR_ELx_EC_DABT_LOW: el0_da(regs, esr); break;