From patchwork Tue Apr 16 14:22:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10903209 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1910613B5 for ; Tue, 16 Apr 2019 14:23:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ECD1C28662 for ; Tue, 16 Apr 2019 14:23:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E0DB728A74; Tue, 16 Apr 2019 14:23:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8CEA628A64 for ; Tue, 16 Apr 2019 14:23:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=0JJLATO5auJGH0qxJAvBeZBiRcCN0Q3mOrl3nDy1J8M=; b=Z6m1pUQ4HN5WcAcEb54TygmL9A 6oOANt7ed+b3w02/JKHUio2X2nD/lb7qZmggo5V2JuBaUng+We8gPLm1s5T3/ScElTRq9f64EmMR6 Ot+alYsuDom77ZRSol7RNOsb94nkc+nudg0rOBzUaYkypDB00QBN760oK7wLNakr7Skm1IhGcMDav ilMHq/9v/itaFYln0KHewSa8cmhxsct7RZss/7v3LcuEgp6isDhKYpFONI7zA8+dRj14I/jndVWdY XVm8uuut32Wv5QlCIyMjfmsXe64kfq4GQSlG7vhr62IHbHHLIm8/acDuv9RG5wkcnxWDQkEL6teJa CNcw+Qrw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hGOzC-00057y-WD; Tue, 16 Apr 2019 14:22:59 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hGOz3-0004zy-F9 for linux-arm-kernel@lists.infradead.org; Tue, 16 Apr 2019 14:22:51 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7CD5F15A2; Tue, 16 Apr 2019 07:22:47 -0700 (PDT) Received: from e112298-lin.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8D18E3F59C; Tue, 16 Apr 2019 07:22:45 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 1/4] arm64: Do not enable IRQs for ct_user_exit Date: Tue, 16 Apr 2019 15:22:33 +0100 Message-Id: <1555424556-46023-2-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1555424556-46023-1-git-send-email-julien.thierry@arm.com> References: <1555424556-46023-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190416_072249_558004_56B36BB5 X-CRM114-Status: GOOD ( 10.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, rostedt@goodmis.org, james.morse@arm.com, yuzenghui@huawei.com, wanghaibin.wang@huawei.com, liwei391@huawei.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP For el0_dbg and el0_error, DAIF bits get explicitly cleared before calling ct_user_exit. When context tracking is disabled, DAIF gets set (almost) immediately after. When context tracking is enabled, among the first things done is disabling IRQs. What is actually needed is: - PSR.D = 0 so the system can be debugged (should be already the case) - PSR.A = 0 so async error can be handled during context tracking Do not clear PSR.I in those two locations. Signed-off-by: Julien Thierry Cc:Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: Marc Zyngier --- arch/arm64/kernel/entry.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 1.9.1 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index c50a7a7..6a38903 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -855,7 +855,7 @@ el0_dbg: mov x1, x25 mov x2, sp bl do_debug_exception - enable_daif + enable_da_f ct_user_exit b ret_to_user el0_inv: @@ -907,7 +907,7 @@ el0_error_naked: enable_dbg mov x0, sp bl do_serror - enable_daif + enable_da_f ct_user_exit b ret_to_user ENDPROC(el0_error) From patchwork Tue Apr 16 14:22:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10903213 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98DBE13B5 for ; Tue, 16 Apr 2019 14:23:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78A3328A3A for ; Tue, 16 Apr 2019 14:23:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6C49828A52; Tue, 16 Apr 2019 14:23:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BDCE22884F for ; Tue, 16 Apr 2019 14:23:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=5CHMCHKzR0eQJXU+0bkWuo/SVv8zvIk9+R9TcGG2VOE=; b=dVisKv3/Hc5V2B2kmiXGqa6oxk YSODcq107/1NED6uw3Wjxke+8d9kxiwWhG8iNxhTpXLGsbuDMBN+lVl4fcU5osmOi8IjA526Pjqug VA7QnV1z2/cwvm1GRL7wu/3wa5eXyDc2A5Pmo5c112kcHaQB8Z9ij0ihj0IV0MQnY9jgoeA6scJZ/ /gVkKPvMjfOerX2qQjKWtagKY7cUXnhRMWpUfWBQyczXCnx7FwPg1P/J1PvErONCDwLcG5aYTykyL +ouH3NP0YafjKCFVw8PfwsZrzLpTchYaAoysX/pIlXwgGJzSzDEsYJ4LTvLfcIGngngV9pgqJlZ7p wMLsFPiw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hGOzW-0005Ug-QA; Tue, 16 Apr 2019 14:23:18 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hGOz4-00050q-0F for linux-arm-kernel@lists.infradead.org; Tue, 16 Apr 2019 14:22:51 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id ABECF15AB; Tue, 16 Apr 2019 07:22:49 -0700 (PDT) Received: from e112298-lin.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BB5303F59C; Tue, 16 Apr 2019 07:22:47 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 2/4] arm64: Always trace_hardirq_off when taking an IRQ Date: Tue, 16 Apr 2019 15:22:34 +0100 Message-Id: <1555424556-46023-3-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1555424556-46023-1-git-send-email-julien.thierry@arm.com> References: <1555424556-46023-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190416_072250_140785_90A04CCB X-CRM114-Status: GOOD ( 11.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, rostedt@goodmis.org, james.morse@arm.com, yuzenghui@huawei.com, wanghaibin.wang@huawei.com, liwei391@huawei.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP It is not an issue to have redundant calls to trace_hardirqs_off(). Simplify the EL1 IRQ vector handler, checking the PMR value only when doing trace_hardirqs_on(). Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: Marc Zyngier --- arch/arm64/kernel/entry.S | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) -- 1.9.1 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 6a38903..b0467a1 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -617,18 +617,7 @@ el1_irq: kernel_entry 1 enable_da_f #ifdef CONFIG_TRACE_IRQFLAGS -#ifdef CONFIG_ARM64_PSEUDO_NMI -alternative_if ARM64_HAS_IRQ_PRIO_MASKING - ldr x20, [sp, #S_PMR_SAVE] -alternative_else - mov x20, #GIC_PRIO_IRQON -alternative_endif - cmp x20, #GIC_PRIO_IRQOFF - /* Irqs were disabled, don't trace */ - b.ls 1f -#endif bl trace_hardirqs_off -1: #endif irq_handler @@ -649,6 +638,11 @@ alternative_else_nop_endif #endif #ifdef CONFIG_TRACE_IRQFLAGS #ifdef CONFIG_ARM64_PSEUDO_NMI +alternative_if ARM64_HAS_IRQ_PRIO_MASKING + ldr x20, [sp, #S_PMR_SAVE] +alternative_else + mov x20, #GIC_PRIO_IRQON +alternative_endif /* * if IRQs were disabled when we received the interrupt, we have an NMI * and we are not re-enabling interrupt upon eret. Skip tracing. From patchwork Tue Apr 16 14:22:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10903215 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 193141515 for ; Tue, 16 Apr 2019 14:23:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E8BA62888C for ; Tue, 16 Apr 2019 14:23:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DA5FE288CE; Tue, 16 Apr 2019 14:23:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CDBE828720 for ; Tue, 16 Apr 2019 14:23:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2kVvp28yCKMkro/VyuVD2/0ss5r7x8V6dBuM/E75R/w=; b=MKFYNcoeW/iG2TSL5/IcKvJSPK 1HjDtwMDI7sEVhxPK1bQ9XRR0FNepuyVDhfTeoF4Fu1cmYnBPTvRxDWwCKV6hea37iutNfCFJyYIT cQ1R0raePiNr5k/CevENw12GMX3IJACHzkAw2T0dwUGEHa12qkS2HoWudzwbcyRdh7owATtujL/GE HB2HaQtMP9Wj0yRb2XVVnUOVH8xj9/HxKXRGDYPT4CZKaSB9ReJnD5dMAmEgHLrZnfJl8t9iZRPwn Y8JUB2BOuQ32MR44PZ7cvgBKDXFGCO0FSiuJ8qlze70tvpwaImex2JBZbZ8Cv/LowjE8N9TIe8M8P uuqVUQ1g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hGOzd-0005dY-DL; Tue, 16 Apr 2019 14:23:25 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hGOz6-00051o-KI for linux-arm-kernel@lists.infradead.org; Tue, 16 Apr 2019 14:22:59 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5165715AD; Tue, 16 Apr 2019 07:22:52 -0700 (PDT) Received: from e112298-lin.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id EA9CA3F59C; Tue, 16 Apr 2019 07:22:49 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 3/4] arm64: Fix incorrect irqflag restore for priority masking Date: Tue, 16 Apr 2019 15:22:35 +0100 Message-Id: <1555424556-46023-4-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1555424556-46023-1-git-send-email-julien.thierry@arm.com> References: <1555424556-46023-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190416_072252_800503_488C649D X-CRM114-Status: GOOD ( 20.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, Suzuki K Pouloze , will.deacon@arm.com, Christoffer Dall , rostedt@goodmis.org, Oleg Nesterov , james.morse@arm.com, yuzenghui@huawei.com, wanghaibin.wang@huawei.com, liwei391@huawei.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP When using IRQ priority masking to disable interrupts, in order to deal with the PSR.I state, local_irq_save() would convert the I bit into a PMR value (GIC_PRIO_IRQOFF). This resulted in local_irq_restore() potentially modifying the value of PMR in undesired location due to the state of PSR.I upon flag saving [1]. In an attempt to solve this issue in a less hackish manner, introduce new PMR values to represent possible states where interrupts are disabled: - GIC_PRIO_SUSPEND_MASKING represents sections of code where interrupt disabling relies on PSR.I instead of PMR. The value masks less IRQs than GIC_PRIO_IRQON in order to make sure IRQs are signaled to the CPU (but not taken). - GIC_PRIO_WAIT_GIC_BIT is used to indicate an action from the GIC is pending before interrupt status can be toggled. While this bit is set, PSR.I should also be set. [1] https://www.spinics.net/lists/arm-kernel/msg716956.html Fixes: commit 4a503217ce37 ("arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking") Signed-off-by: Julien Thierry Reported-by: Zenghui Yu Cc: Steven Rostedt Cc: Wei Li Cc: Catalin Marinas Cc: Will Deacon Cc: Christoffer Dall Cc: Marc Zyngier Cc: James Morse Cc: Suzuki K Pouloze Cc: Oleg Nesterov --- arch/arm64/include/asm/arch_gicv3.h | 4 +++- arch/arm64/include/asm/assembler.h | 18 +++++++++++++++++ arch/arm64/include/asm/daifflags.h | 16 +++++++++++---- arch/arm64/include/asm/irqflags.h | 39 +++++++++++-------------------------- arch/arm64/include/asm/kvm_host.h | 4 +++- arch/arm64/include/asm/ptrace.h | 24 +++++++++++++++++++++-- arch/arm64/kernel/entry.S | 22 +++++++++++++++++---- arch/arm64/kernel/process.c | 2 +- arch/arm64/kernel/smp.c | 8 +++++--- arch/arm64/kvm/hyp/switch.c | 2 +- 10 files changed, 94 insertions(+), 45 deletions(-) -- 1.9.1 diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index 14b41dd..10deb86 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -163,7 +163,9 @@ static inline bool gic_prio_masking_enabled(void) static inline void gic_pmr_mask_irqs(void) { - BUILD_BUG_ON(GICD_INT_DEF_PRI <= GIC_PRIO_IRQOFF); + BUILD_BUG_ON(GICD_INT_DEF_PRI < (GIC_PRIO_IRQOFF | + GIC_PRIO_WAIT_GIC_BIT)); + BUILD_BUG_ON(GICD_INT_DEF_PRI >= GIC_PRIO_IRQON); gic_write_pmr(GIC_PRIO_IRQOFF); } diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index c5308d0..90bd3ef 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -62,6 +62,24 @@ msr daifclr, #(8 | 4 | 1) .endm + .macro suspend_irq_prio_masking, tmp:req +#ifdef CONFIG_ARM64_PSEUDO_NMI + alternative_if ARM64_HAS_IRQ_PRIO_MASKING + mov \tmp, #GIC_PRIO_SUSPEND_MASKING + msr_s SYS_ICC_PMR_EL1, \tmp + alternative_else_nop_endif +#endif + .endm + + .macro set_prio_wait_gic, curr_pmr:req, tmp:req +#ifdef CONFIG_ARM64_PSEUDO_NMI + alternative_if ARM64_HAS_IRQ_PRIO_MASKING + orr \tmp, \curr_pmr, #GIC_PRIO_WAIT_GIC_BIT + msr_s SYS_ICC_PMR_EL1, \tmp + alternative_else_nop_endif +#endif + .endm + /* * Save/restore interrupts. */ diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h index db452aa..e575fdb 100644 --- a/arch/arm64/include/asm/daifflags.h +++ b/arch/arm64/include/asm/daifflags.h @@ -18,6 +18,7 @@ #include +#include #include #define DAIF_PROCCTX 0 @@ -32,6 +33,11 @@ static inline void local_daif_mask(void) : : : "memory"); + + /* Don't really care for a dsb here, we don't intend to enable IRQs */ + if (system_uses_irq_prio_masking()) + gic_write_pmr(GIC_PRIO_SUSPEND_MASKING); + trace_hardirqs_off(); } @@ -43,7 +49,7 @@ static inline unsigned long local_daif_save(void) if (system_uses_irq_prio_masking()) { /* If IRQs are masked with PMR, reflect it in the flags */ - if (read_sysreg_s(SYS_ICC_PMR_EL1) <= GIC_PRIO_IRQOFF) + if (read_sysreg_s(SYS_ICC_PMR_EL1) != GIC_PRIO_IRQON) flags |= PSR_I_BIT; } @@ -59,8 +65,10 @@ static inline void local_daif_restore(unsigned long flags) if (!irq_disabled) { trace_hardirqs_on(); - if (system_uses_irq_prio_masking()) - arch_local_irq_enable(); + if (system_uses_irq_prio_masking()) { + gic_write_pmr(GIC_PRIO_IRQON); + dsb(sy); + } } else if (!(flags & PSR_A_BIT)) { /* * If interrupts are disabled but we can take @@ -87,7 +95,7 @@ static inline void local_daif_restore(unsigned long flags) * * So we don't need additional synchronization here. */ - arch_local_irq_disable(); + gic_write_pmr(GIC_PRIO_IRQOFF); } } diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h index 43d8366..4eeaa65 100644 --- a/arch/arm64/include/asm/irqflags.h +++ b/arch/arm64/include/asm/irqflags.h @@ -67,31 +67,14 @@ static inline void arch_local_irq_disable(void) */ static inline unsigned long arch_local_save_flags(void) { - unsigned long daif_bits; unsigned long flags; - daif_bits = read_sysreg(daif); - - /* - * The asm is logically equivalent to: - * - * if (system_uses_irq_prio_masking()) - * flags = (daif_bits & PSR_I_BIT) ? - * GIC_PRIO_IRQOFF : - * read_sysreg_s(SYS_ICC_PMR_EL1); - * else - * flags = daif_bits; - */ asm volatile(ALTERNATIVE( - "mov %0, %1\n" - "nop\n" - "nop", - "mrs_s %0, " __stringify(SYS_ICC_PMR_EL1) "\n" - "ands %1, %1, " __stringify(PSR_I_BIT) "\n" - "csel %0, %0, %2, eq", - ARM64_HAS_IRQ_PRIO_MASKING) - : "=&r" (flags), "+r" (daif_bits) - : "r" ((unsigned long) GIC_PRIO_IRQOFF) + "mrs %0, daif", + "mrs_s %0, " __stringify(SYS_ICC_PMR_EL1) "\n", + ARM64_HAS_IRQ_PRIO_MASKING) + : "=&r" (flags) + : : "memory"); return flags; @@ -119,8 +102,8 @@ static inline void arch_local_irq_restore(unsigned long flags) "msr_s " __stringify(SYS_ICC_PMR_EL1) ", %0\n" "dsb sy", ARM64_HAS_IRQ_PRIO_MASKING) - : "+r" (flags) : + : "r" (flags) : "memory"); } @@ -129,11 +112,11 @@ static inline int arch_irqs_disabled_flags(unsigned long flags) int res; asm volatile(ALTERNATIVE( - "and %w0, %w1, #" __stringify(PSR_I_BIT) "\n" - "nop", - "cmp %w1, #" __stringify(GIC_PRIO_IRQOFF) "\n" - "cset %w0, ls", - ARM64_HAS_IRQ_PRIO_MASKING) + "and %w0, %w1, #" __stringify(PSR_I_BIT) "\n" + "nop", + "cmp %w1, #" __stringify(GIC_PRIO_IRQON) "\n" + "cset %w0, ne", + ARM64_HAS_IRQ_PRIO_MASKING) : "=&r" (res) : "r" ((int) flags) : "memory"); diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index a01fe087..1abf77e 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -532,9 +532,11 @@ static inline void kvm_arm_vhe_guest_enter(void) * will not signal the CPU of interrupts of lower priority, and the * only way to get out will be via guest exceptions. * Naturally, we want to avoid this. + * + * local_daif_mask() already sets PMR to SUSPEND_MASKING, we just need a + * dsb to ensure the redistributor is forwards EL2 IRQs to the CPU. */ if (system_uses_irq_prio_masking()) { - gic_write_pmr(GIC_PRIO_IRQON); dsb(sy); } } diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index ec60174..751a928 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -35,9 +35,29 @@ * means masking more IRQs (or at least that the same IRQs remain masked). * * To mask interrupts, we clear the most significant bit of PMR. + * + * Some code sections either automatically switch back to PSR.I or explicitly + * require to not use priority masking. GIC_PRIO_SUSPEND_MASKING is the PMR + * value intended for such a section, where interrupts are disabled, but isn't + * done via priority masking. + * + * Another special case is the path from taking an interrupt to acknowledging + * it. Interrupt needs to be unmasked in order to be acknowledged, but other + * interrupts musn't get unmasked at that point, as the signaled interrupt + * could have been spurious. + * + * Overall, the layout used for PMR is: + * - State[7:5]: + * 0x7 => Suspended, 0x6 => IRQs on, 0x4 => IRQs off, other => Undefined + * + * - Wait GIC bit[4]: + * 0x1 => Action from GIC driver is needed before modifying PMR, + * 0x0 => no action required */ -#define GIC_PRIO_IRQON 0xf0 -#define GIC_PRIO_IRQOFF (GIC_PRIO_IRQON & ~0x80) +#define GIC_PRIO_IRQON 0xc0 +#define GIC_PRIO_IRQOFF (GIC_PRIO_IRQON & ~0x80) +#define GIC_PRIO_SUSPEND_MASKING (GIC_PRIO_IRQON | 0x20) +#define GIC_PRIO_WAIT_GIC_BIT (1 << 4) /* Additional SPSR bits not exposed in the UABI */ #define PSR_IL_BIT (1 << 20) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index b0467a1..9db06e4 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -258,6 +258,7 @@ alternative_else_nop_endif /* * Registers that may be useful after this macro is invoked: * + * x20 - ICC_PMR_EL1 * x21 - aborted SP * x22 - aborted PC * x23 - aborted PSTATE @@ -598,6 +599,7 @@ el1_dbg: cmp x24, #ESR_ELx_EC_BRK64 // if BRK64 cinc x24, x24, eq // set bit '0' tbz x24, #0, el1_inv // EL1 only + suspend_irq_prio_masking tmp=x21 mrs x0, far_el1 mov x2, sp // struct pt_regs bl do_debug_exception @@ -615,7 +617,9 @@ ENDPROC(el1_sync) .align 6 el1_irq: kernel_entry 1 + set_prio_wait_gic curr_pmr=x20, tmp=x0 enable_da_f + #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off #endif @@ -644,11 +648,12 @@ alternative_else mov x20, #GIC_PRIO_IRQON alternative_endif /* - * if IRQs were disabled when we received the interrupt, we have an NMI - * and we are not re-enabling interrupt upon eret. Skip tracing. + * When using IRQ priority masking, we can get spurious interrupts while + * PMR is set to GIC_PRIO_IRQOFF. An NMI might also have occurred in a + * section with interrupts disabled. Skip tracing in those cases. */ - cmp x20, #GIC_PRIO_IRQOFF - b.ls 1f + cmp x20, #GIC_PRIO_IRQON + b.ne 1f #endif bl trace_hardirqs_on 1: @@ -766,6 +771,7 @@ el0_ia: * Instruction abort handling */ mrs x26, far_el1 + suspend_irq_prio_masking tmp=x0 enable_da_f #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off @@ -811,6 +817,7 @@ el0_sp_pc: * Stack or PC alignment exception handling */ mrs x26, far_el1 + suspend_irq_prio_masking tmp=x0 enable_da_f #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off @@ -845,6 +852,7 @@ el0_dbg: * Debug exception handling */ tbnz x24, #0, el0_inv // EL0 only + suspend_irq_prio_masking tmp=x0 mrs x0, far_el1 mov x1, x25 mov x2, sp @@ -866,7 +874,9 @@ ENDPROC(el0_sync) el0_irq: kernel_entry 0 el0_irq_naked: + set_prio_wait_gic curr_pmr=x20, tmp=x0 enable_da_f + #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off #endif @@ -888,6 +898,7 @@ ENDPROC(el0_irq) el1_error: kernel_entry 1 mrs x1, esr_el1 + suspend_irq_prio_masking tmp=x0 enable_dbg mov x0, sp bl do_serror @@ -898,6 +909,7 @@ el0_error: kernel_entry 0 el0_error_naked: mrs x1, esr_el1 + suspend_irq_prio_masking tmp=x0 enable_dbg mov x0, sp bl do_serror @@ -922,6 +934,7 @@ work_pending: */ ret_to_user: disable_daif + suspend_irq_prio_masking tmp=x0 ldr x1, [tsk, #TSK_TI_FLAGS] and x2, x1, #_TIF_WORK_MASK cbnz x2, work_pending @@ -938,6 +951,7 @@ ENDPROC(ret_to_user) */ .align 6 el0_svc: + suspend_irq_prio_masking tmp=x1 mov x0, sp bl el0_svc_handler b ret_to_user diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 3767fb2..10a660e 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -94,7 +94,7 @@ static void __cpu_do_idle_irqprio(void) * be raised. */ pmr = gic_read_pmr(); - gic_write_pmr(GIC_PRIO_IRQON); + gic_write_pmr(GIC_PRIO_SUSPEND_MASKING); __cpu_do_idle(); diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 824de70..f772cb5 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -192,11 +192,13 @@ static void init_gic_priority_masking(void) WARN_ON(!(cpuflags & PSR_I_BIT)); - gic_write_pmr(GIC_PRIO_IRQOFF); - /* We can only unmask PSR.I if we can take aborts */ - if (!(cpuflags & PSR_A_BIT)) + if (!(cpuflags & PSR_A_BIT)) { + gic_write_pmr(GIC_PRIO_IRQOFF); write_sysreg(cpuflags & ~PSR_I_BIT, daif); + } else { + gic_write_pmr(GIC_PRIO_SUSPEND_MASKING); + } } /* diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 3563fe6..9694ad4 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -533,7 +533,7 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) * Naturally, we want to avoid this. */ if (system_uses_irq_prio_masking()) { - gic_write_pmr(GIC_PRIO_IRQON); + gic_write_pmr(GIC_PRIO_SUSPEND_MASKING); dsb(sy); } From patchwork Tue Apr 16 14:22:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10903217 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6805B1515 for ; Tue, 16 Apr 2019 14:23:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49BC028720 for ; Tue, 16 Apr 2019 14:23:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3CDFB2875F; Tue, 16 Apr 2019 14:23:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C6D1928720 for ; Tue, 16 Apr 2019 14:23:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=TscRiThnu+fdyjFOaZXDW6FU9cgG1frepsl+vboAVWo=; b=XY7yyUwfAWl5mtFKuIchv4OZBn TZptXpeK/D2lq4LHYfeFJXffRepy7Nb0t5h2j70QVsIPQcxSJHXT4174459uQIdYgmkByIOYne1UF qna9iA6ZpBtFD7d90dQ5BBb9JuyfVzbRtqA8BTjijZmYGkdr1IrZftEfPwgQ4jX/TbmvA7M6M0qNW K00s9VV4vmq8BGVisrWTGG0aGnKVuod4vDC5Sn8B+UgVIX8rYZRkM8ZXFgPq+tzMLDOkB752ISY2u Der2Lg4Pv7Cx16Dop1GUqrcvcjskQNTxKlFU9mNVlFDVGEG3NF0IqNqCJULjSp0M5LVXhzd9Y47Ys NrCSW6xw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hGOzo-0005q9-06; Tue, 16 Apr 2019 14:23:36 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hGOz9-00054e-Ep for linux-arm-kernel@lists.infradead.org; Tue, 16 Apr 2019 14:22:59 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B1ECB1682; Tue, 16 Apr 2019 07:22:54 -0700 (PDT) Received: from e112298-lin.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8FE713F59C; Tue, 16 Apr 2019 07:22:52 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 4/4] arm64: irqflags: Introduce explicit debugging for IRQ priorities Date: Tue, 16 Apr 2019 15:22:36 +0100 Message-Id: <1555424556-46023-5-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1555424556-46023-1-git-send-email-julien.thierry@arm.com> References: <1555424556-46023-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190416_072255_894998_E2565D81 X-CRM114-Status: GOOD ( 14.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, rostedt@goodmis.org, james.morse@arm.com, yuzenghui@huawei.com, wanghaibin.wang@huawei.com, liwei391@huawei.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Using IRQ priority masking to enable/disable interrupts is a bit sensitive as it requires to deal with both ICC_PMR_EL1 and PSR.I. Introduce some validity checks to both highlight the states in which functions dealing with IRQ enabling/disabling can (not) be called, and bark a warning when called in an unexpected state. Since these checks are done on hotpaths, introduce a build option to choose whether to do the checking. Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon --- arch/arm64/Kconfig | 11 +++++++++++ arch/arm64/include/asm/daifflags.h | 9 +++++++++ arch/arm64/include/asm/irqflags.h | 23 +++++++++++++++++++++-- 3 files changed, 41 insertions(+), 2 deletions(-) -- 1.9.1 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7e34b9e..3fb38f3 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1359,6 +1359,17 @@ config ARM64_PSEUDO_NMI If unsure, say N +if ARM64_PSEUDO_NMI +config ARM64_DEBUG_PRIORITY_MASKING + bool "Debug interrupt priority masking" + help + This adds runtime checks to functions enabling/disabling + interrupts when using priority masking. The additional checks verify + the validity of ICC_PMR_EL1 when calling concerned functions. + + If unsure, say N +endif + config RELOCATABLE bool help diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h index e575fdb..0a1add9 100644 --- a/arch/arm64/include/asm/daifflags.h +++ b/arch/arm64/include/asm/daifflags.h @@ -28,6 +28,10 @@ /* mask/save/unmask/restore all exceptions, including interrupts. */ static inline void local_daif_mask(void) { + WARN_ON(IS_ENABLED(CONFIG_ARM64_DEBUG_PRIORITY_MASKING) && + system_uses_irq_prio_masking() && + (read_sysreg_s(SYS_ICC_PMR_EL1) & GIC_PRIO_WAIT_GIC_BIT)); + asm volatile( "msr daifset, #0xf // local_daif_mask\n" : @@ -62,6 +66,11 @@ static inline void local_daif_restore(unsigned long flags) { bool irq_disabled = flags & PSR_I_BIT; + WARN_ON(IS_ENABLED(CONFIG_ARM64_DEBUG_PRIORITY_MASKING) + && system_uses_irq_prio_masking() + && (!(read_sysreg(daif) & PSR_I_BIT) || + read_sysreg_s(SYS_ICC_PMR_EL1) & GIC_PRIO_WAIT_GIC_BIT)); + if (!irq_disabled) { trace_hardirqs_on(); diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h index 4eeaa65..2cb13a7 100644 --- a/arch/arm64/include/asm/irqflags.h +++ b/arch/arm64/include/asm/irqflags.h @@ -40,6 +40,13 @@ */ static inline void arch_local_irq_enable(void) { + if (IS_ENABLED(CONFIG_ARM64_DEBUG_PRIORITY_MASKING) && + system_uses_irq_prio_masking()) { + u32 pmr = read_sysreg_s(SYS_ICC_PMR_EL1); + + WARN_ON(pmr != GIC_PRIO_IRQON && pmr != GIC_PRIO_IRQOFF); + } + asm volatile(ALTERNATIVE( "msr daifclr, #2 // arch_local_irq_enable\n" "nop", @@ -51,7 +58,7 @@ static inline void arch_local_irq_enable(void) : "memory"); } -static inline void arch_local_irq_disable(void) +static inline void _arch_local_irq_disable_nocheck(void) { asm volatile(ALTERNATIVE( "msr daifset, #2 // arch_local_irq_disable", @@ -62,6 +69,18 @@ static inline void arch_local_irq_disable(void) : "memory"); } +static inline void arch_local_irq_disable(void) +{ + if (IS_ENABLED(CONFIG_ARM64_DEBUG_PRIORITY_MASKING) && + system_uses_irq_prio_masking()) { + u32 pmr = read_sysreg_s(SYS_ICC_PMR_EL1); + + WARN_ON(pmr != GIC_PRIO_IRQON && pmr != GIC_PRIO_IRQOFF); + } + + _arch_local_irq_disable_nocheck(); +} + /* * Save the current interrupt enable state. */ @@ -86,7 +105,7 @@ static inline unsigned long arch_local_irq_save(void) flags = arch_local_save_flags(); - arch_local_irq_disable(); + _arch_local_irq_disable_nocheck(); return flags; }