From patchwork Fri May 6 05:52:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 12840585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BC485C433FE for ; Fri, 6 May 2022 05:52:41 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322571.543948 (Exim 4.92) (envelope-from ) id 1nmqt5-0001wc-7x; Fri, 06 May 2022 05:52:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322571.543948; Fri, 06 May 2022 05:52:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmqt5-0001wT-5A; Fri, 06 May 2022 05:52:23 +0000 Received: by outflank-mailman (input) for mailman id 322571; Fri, 06 May 2022 05:52:21 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmqt3-0001gH-Pw for xen-devel@lists.xenproject.org; Fri, 06 May 2022 05:52:21 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id b1c5d8ee-cd00-11ec-a406-831a346695d4; Fri, 06 May 2022 07:52:21 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 67A3814BF; Thu, 5 May 2022 22:52:20 -0700 (PDT) Received: from a015966.shanghai.arm.com (a015966.shanghai.arm.com [10.169.190.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E58803FA27; Thu, 5 May 2022 22:52:15 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b1c5d8ee-cd00-11ec-a406-831a346695d4 From: Henry Wang To: xen-devel@lists.xenproject.org Cc: Henry Wang , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu , Wei Chen , Henry Wang , Wei Chen , Julien Grall Subject: [PATCH v2 1/2] xen/arm: Defer request_irq on secondary CPUs after local_irq_enable Date: Fri, 6 May 2022 13:52:00 +0800 Message-Id: <20220506055201.999277-2-Henry.Wang@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506055201.999277-1-Henry.Wang@arm.com> References: <20220506055201.999277-1-Henry.Wang@arm.com> MIME-Version: 1.0 With the enhanced ASSERT_ALLOC_CONTEXT, calling request_irq before local_irq_enable on secondary cores will lead to (XEN) Xen call trace: (XEN) [<000000000021d86c>] alloc_xenheap_pages+0x74/0x194 (PC) (XEN) [<000000000021d864>] alloc_xenheap_pages+0x6c/0x194 (LR) (XEN) [<0000000000229e90>] xmalloc_tlsf.c#xmalloc_pool_get+0x1c/0x28 (XEN) [<000000000022a270>] xmem_pool_alloc+0x21c/0x448 (XEN) [<000000000022a8dc>] _xmalloc+0x8c/0x290 (XEN) [<000000000026b57c>] request_irq+0x40/0xb8 (XEN) [<0000000000272780>] init_timer_interrupt+0x74/0xcc (XEN) [<000000000027212c>] start_secondary+0x1b4/0x238 (XEN) [<0000000084000200>] 0000000084000200 (XEN) (XEN) (XEN) **************************************** (XEN) Panic on CPU 4: (XEN) Assertion '!in_irq() && (local_irq_is_enabled() || num_online_cpus() <= 1)' failed at common/page_alloc.c:2212 (XEN) **************************************** on systems without a big enough pool for xmalloc() to cater the requested size. Moving the call of request_irq() past local_irq_enable() on secondary cores will make sure the assertion condition in alloc_xenheap_pages(), i.e. !in_irq && local_irq_enabled() is satisfied. It is also safe because the timer and GIC maintenance interrupt will not be used until the CPU is fully online. Reported-by: Wei Chen Suggested-by: Julien Grall Signed-off-by: Henry Wang --- v1 -> v2: - Explain why the moving of code is safe in the commit message and add comments. --- xen/arch/arm/smpboot.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index 7bfd0a73a7..9bb32a301a 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -361,9 +361,6 @@ void start_secondary(void) init_secondary_IRQ(); - init_maintenance_interrupt(); - init_timer_interrupt(); - set_current(idle_vcpu[cpuid]); setup_cpu_sibling_map(cpuid); @@ -380,6 +377,15 @@ void start_secondary(void) cpumask_set_cpu(cpuid, &cpu_online_map); local_irq_enable(); + + /* + * Calling request_irq() after local_irq_enable() on secondary cores + * will make sure the assertion condition in alloc_xenheap_pages(), + * i.e. !in_irq && local_irq_enabled() is satisfied. + */ + init_maintenance_interrupt(); + init_timer_interrupt(); + local_abort_enable(); check_local_cpu_errata();