From patchwork Thu Sep 30 13:17:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12528289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24EDDC433F5 for ; Thu, 30 Sep 2021 13:23:23 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E65576187A for ; Thu, 30 Sep 2021 13:23:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E65576187A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qrVZBqXmowc8J7C+/eak1GJ0T68hy1VZ7c1wPCjPPe0=; b=zt6HMZNz/xNvr1 vU+AYNBQScuw7/j6AakFxGlpHIQ4HUfVcllX7ryfAsz+MWxUDF9Sih2pvb/FYg7pjGxMM0EvatnRU 2Ddc/FzDbI+iNYI/YQwFqgUJgCTao0pKV9XM0MeWRvYu36NbJ17LdDyVFSKH4jY5RDWCtbDIOZCS7 LJi6T9tAQp1CQI9gYJ/e7kiGau2YPCShX87MGPvidLXEqUkI9wYHvVIPakR1xKVecEnNCEwug0dlS tVkVAJAKxsi5KS5vyg8yxbip7E0sSKAT3LNM4qoO+NUWFmltC1/rQX8vDLT+58a2O6x0bCf0B9c3Y Le+PCGVvb0big6l43jkw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVvzu-00EQpX-HY; Thu, 30 Sep 2021 13:21:15 +0000 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVvwZ-00EPdx-52 for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 13:17:48 +0000 Received: by mail-pg1-x535.google.com with SMTP id k24so6181759pgh.8 for ; Thu, 30 Sep 2021 06:17:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HXE3iOAOss3Z9oh0TZW0hIqzqp3k7bsAeW3PnBZvSKc=; b=JWSPYOzy0aJQVjxIgwrovYZ5SbArKquJMnCLOFV3OdxA5WB+aWwPjq4uGtnidzZKnj RLLYiU6FIz387VbJM4gLgGfe6KBe+BEVeI19+cSwLZVZUnrBa/TqMSbMG6msisG4Nss/ GyoONpN8RHnpN9AmHRtV/lTfmU1fCymgs3viwO9W2rXWkE/dHAyvCSJuBoAl5ctoSPZc 7KmWfSi6Dhx+4qKAoJtVEFeiXtTYijNtA8gbA1y6uUVrPEE+VqYyLApw/o0hdzlTJc+z l5dlHyZQcfb1/CtfBoODQDHWDjYBLy/DWpsDVxxC3xc8FOWBSG0HpXyUg7tlSqKHb9CA 0cpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HXE3iOAOss3Z9oh0TZW0hIqzqp3k7bsAeW3PnBZvSKc=; b=DN43FOZJwBbjSz86c8sdndvqavm+QVXeyfyBSo2OnvC3OmTy31yf5+hOPlTOLZ5sRS UNdY1iJ7HlBP4eOTKkuZRsa6QgmvnQ0RAFdIaOALAYWZmdHa+ZUv7IsMbV87fz6GOW2e UuAy+0RolYU+OqwSsLZ9pBAFzBFFMBeckGC/diOic3oGhT5CNDq7ZHbNAUkMEv9z4Ziw 1u7I/Iw/N1zlx2kl1YRrQQNRGgAPSRGnZf/hsXTn7Rdf6gVZgSh34EihOKZJ2kQgw0fF cHvm3ByPhfE588n1P9KOqmqqg/hL8HyCoMuh6fOwfJhrJgr093PGiGgRnvAdW1lAql6X xltA== X-Gm-Message-State: AOAM531oPP4nmvShzztNY2pjVyzzeVK4jhbIet6zImatezhVo2WMPbuI NkXM1+7Hvw4ZzdYb0rAexlPfdtKJAA== X-Google-Smtp-Source: ABdhPJwQBFLoAbju9IJNC8x90MSsuAFWCJ5Ag/k6yu0oTeZAR8k8qjBWlkrJ+GzpNqmBoEEMIZRBeQ== X-Received: by 2002:a63:c101:: with SMTP id w1mr4856301pgf.53.1633007865645; Thu, 30 Sep 2021 06:17:45 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id q3sm3083167pgf.18.2021.09.30.06.17.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 06:17:45 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Mark Rutland , "Paul E. McKenney" , Catalin Marinas , Will Deacon , Marc Zyngier , Joey Gouly , Sami Tolvanen , Julien Thierry , Thomas Gleixner , Yuichi Ito , Pingfan Liu , linux-kernel@vger.kernel.org Subject: [PATCHv3 2/3] arm64: entry: refactor EL1 interrupt entry logic Date: Thu, 30 Sep 2021 21:17:07 +0800 Message-Id: <20210930131708.35328-3-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210930131708.35328-1-kernelfans@gmail.com> References: <20210930131708.35328-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_061747_274556_712F676E X-CRM114-Status: GOOD ( 15.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Rutland Currently we distinguish IRQ and definitely-PNMI at entry/exit time via the enter_el1_irq_or_nmi() and enter_el1_irq_or_nmi() helpers. In subsequent patches we'll need to handle the two cases more distinctly in the body of the exception handler. To make this possible, this patch refactors el1_interrupt to be a top-level dispatcher to separate handlers for the IRQ and PNMI cases, removing the need for the enter_el1_irq_or_nmi() and exit_el1_irq_or_nmi() helpers. Note that since arm64_enter_nmi() calls __nmi_enter(), which increments the preemt_count, we could never preempt when handling a PNMI. We now only check for preemption in the IRQ case, which makes this clearer. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: "Paul E. McKenney" Cc: Catalin Marinas Cc: Will Deacon Cc: Marc Zyngier Cc: Joey Gouly Cc: Sami Tolvanen Cc: Julien Thierry Cc: Thomas Gleixner Cc: Yuichi Ito Cc: Pingfan Liu Cc: linux-kernel@vger.kernel.org To: linux-arm-kernel@lists.infradead.org Reviewed-by: Marc Zyngier --- arch/arm64/kernel/entry-common.c | 44 ++++++++++++++++---------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 32f9796c4ffe..5f1473319fb0 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -219,22 +219,6 @@ static void noinstr arm64_exit_el1_dbg(struct pt_regs *regs) lockdep_hardirqs_on(CALLER_ADDR0); } -static void noinstr enter_el1_irq_or_nmi(struct pt_regs *regs) -{ - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) - arm64_enter_nmi(regs); - else - enter_from_kernel_mode(regs); -} - -static void noinstr exit_el1_irq_or_nmi(struct pt_regs *regs) -{ - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) - arm64_exit_nmi(regs); - else - exit_to_kernel_mode(regs); -} - static void __sched arm64_preempt_schedule_irq(void) { lockdep_assert_irqs_disabled(); @@ -432,14 +416,19 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) } } -static void noinstr el1_interrupt(struct pt_regs *regs, - void (*handler)(struct pt_regs *)) +static __always_inline void +__el1_pnmi(struct pt_regs *regs, void (*handler)(struct pt_regs *)) { - write_sysreg(DAIF_PROCCTX_NOIRQ, daif); - - enter_el1_irq_or_nmi(regs); + arm64_enter_nmi(regs); do_interrupt_handler(regs, handler); + arm64_exit_nmi(regs); +} +static __always_inline void +__el1_interrupt(struct pt_regs *regs, void (*handler)(struct pt_regs *)) +{ + enter_from_kernel_mode(regs); + do_interrupt_handler(regs, handler); /* * Note: thread_info::preempt_count includes both thread_info::count * and thread_info::need_resched, and is not equivalent to @@ -448,8 +437,19 @@ static void noinstr el1_interrupt(struct pt_regs *regs, if (IS_ENABLED(CONFIG_PREEMPTION) && READ_ONCE(current_thread_info()->preempt_count) == 0) arm64_preempt_schedule_irq(); + exit_to_kernel_mode(regs); +} + +static void noinstr el1_interrupt(struct pt_regs *regs, + void (*handler)(struct pt_regs *)) +{ + write_sysreg(DAIF_PROCCTX_NOIRQ, daif); + + if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) + __el1_pnmi(regs, handler); + else + __el1_interrupt(regs, handler); - exit_el1_irq_or_nmi(regs); } asmlinkage void noinstr el1h_64_irq_handler(struct pt_regs *regs)