From patchwork Sun Nov 30 12:37:43 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Borislav Petkov X-Patchwork-Id: 5407921 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 07904BEEA8 for ; Sun, 30 Nov 2014 12:41:30 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id DC9FE2015E for ; Sun, 30 Nov 2014 12:41:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F036520120 for ; Sun, 30 Nov 2014 12:41:19 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1Xv3la-0006zh-LD; Sun, 30 Nov 2014 12:38:18 +0000 Received: from mail.skyhub.de ([2a01:4f8:120:8448::d00d]) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1Xv3lV-0006wL-IT for linux-arm-kernel@lists.infradead.org; Sun, 30 Nov 2014 12:38:16 +0000 X-Virus-Scanned: Nedap ESD1 at mail.skyhub.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=alien8.de; s=alien8; t=1417351066; bh=5fGr/tjksYDMzgE3LaFOKjTqtdtpjK62R6K2+acImfQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=qHOG X0VIDs/PaGSsQvvzRPwYC2Wtc6AMB6gFId+UyOByCdZahdHXqYyqVW6vNNis627Sj+A 28Nr+o0oG2Wvoo/rtZWLmM16kunQXNF4lBN/8qbmy6eMzMZ5LtAfHL/rMLtdPHtpYps TEkeBTNPfkBuWx7TIKc5pVdxe1WbBrmFs= Received: from mail.skyhub.de ([127.0.0.1]) by localhost (door.skyhub.de [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id PH+1bEqohvvJ; Sun, 30 Nov 2014 13:37:45 +0100 (CET) Received: from pd.tnic (p5DDC47A2.dip0.t-ipconnect.de [93.220.71.162]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.skyhub.de (SuperMail on ZX Spectrum 128k) with ESMTPSA id 6C0FB244943; Sun, 30 Nov 2014 13:37:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=alien8.de; s=alien8; t=1417351065; bh=5fGr/tjksYDMzgE3LaFOKjTqtdtpjK62R6K2+acImfQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=dj9K DGyS43Q5Vzwpy2qum7hTI5d96NEviSxj1/AMbXn9x6BBuVjBUFbaxKWxaRAqv2s540G I2uhxSEeqQCeFXQUvmKxcksz/Ap0lmeiQrRDjtL7DozBUeW8yBvqP6JQfkjWtnDYo7p 0utgRaIWoXvudZbEE63UyRsFE4FNaqoSs= Received: by pd.tnic (Postfix, from userid 1000) id 68292100AF4; Sun, 30 Nov 2014 13:37:43 +0100 (CET) From: Borislav Petkov To: LKML Subject: [PATCH] irqdomain: Correct early allocation of irq domains with IRQs off Date: Sun, 30 Nov 2014 13:37:43 +0100 Message-Id: <1417351063-19039-1-git-send-email-bp@alien8.de> X-Mailer: git-send-email 2.0.0 In-Reply-To: References: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20141130_043813_974778_F7E7C160 X-CRM114-Status: GOOD ( 18.53 ) X-Spam-Score: -0.1 (/) Cc: Tony Luck , Marc Zyngier , Benjamin Herrenschmidt , Joerg Roedel , Grant Likely , Yijing Wang , Matthias Brugger , Bjorn Helgaas , Yingjoe Chen , Thomas Gleixner , Jiang Liu , linux-arm-kernel@lists.infradead.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_LOW, T_DKIM_INVALID, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Borislav Petkov I'm seeing the following lockdep splat on an x2APIC machine with interrupts remapping. The problem is that enable_IR_x2apic() disables interrupts before doing any further initialization. However, after having moved to irq domains, domain allocation cannot happen with interrupts disabled (GFP_KERNEL). A proper fix would be to move that initialization to the early irq setup path, which is something for another day. It is more involved work too. So do a temporary fix which should not encourage the behaviour of even assuming the irq domains code should be called with interrupts disabled. Make it x86-only too. ... dmar: Host address width 36 dmar: DRHD base: 0x000000fed90000 flags: 0x0 dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020e60262 ecap f0101a dmar: DRHD base: 0x000000fed91000 flags: 0x1 dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap c9008020660262 ecap f0105a dmar: RMRR base: 0x000000da2ba000 end: 0x000000da2d0fff dmar: RMRR base: 0x000000db800000 end: 0x000000df9fffff IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1 HPET id 0 under DRHD base 0xfed91000 Queued invalidation will be enabled to support x2apic and Intr-remapping. ------------[ cut here ]------------ WARNING: CPU: 0 PID: 1 at kernel/locking/lockdep.c:2744 lockdep_trace_alloc+0xd4/0xe0() DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)) Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.18.0-rc6+ #1 Hardware name: LENOVO 2320CTO/2320CTO, BIOS G2ET86WW (2.06 ) 11/13/2012 0000000000000009 ffff880213d07b58 ffffffff816502d7 0000000000000000 ffff880213d07ba8 ffff880213d07b98 ffffffff81059790 0000000000000001 0000000000000092 0000000000000000 00000000000080d0 0000000000000000 Call Trace: dump_stack warn_slowpath_common warn_slowpath_fmt lockdep_trace_alloc __alloc_pages_nodemask ? get_page_from_freelist ? trace_hardirqs_off_caller alloc_kmem_pages_node kmalloc_large_node __kmalloc_node ? __dmar_enable_qi __irq_domain_add irq_domain_add_hierarchy intel_setup_irq_remapping.part.4 intel_enable_irq_remapping irq_remapping_enable enable_IR enable_IR_x2apic default_setup_apic_routing native_smp_prepare_cpus kernel_init_freeable ? ret_from_fork ? rest_init kernel_init ret_from_fork ? rest_init ---[ end trace fac50e785fc22942 ]--- Enabled IRQ remapping in x2apic mode Enabling x2apic Enabled x2apic Switched APIC routing to cluster x2apic. ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 ... Cc: Thomas Gleixner Cc: Jiang Liu Cc: Tony Luck Cc: linux-arm-kernel@lists.infradead.org Cc: Bjorn Helgaas Cc: Grant Likely Cc: Marc Zyngier Cc: Yijing Wang Cc: Yingjoe Chen Cc: Benjamin Herrenschmidt Cc: Matthias Brugger Cc: Joerg Roedel Link: Link: http://lkml.kernel.org/r/20141129125319.GA6491@pd.tnic Signed-off-by: Borislav Petkov --- kernel/irq/irqdomain.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c index 7fac311057b8..3395d8923f96 100644 --- a/kernel/irq/irqdomain.c +++ b/kernel/irq/irqdomain.c @@ -46,14 +46,31 @@ struct irq_domain *__irq_domain_add(struct device_node *of_node, int size, void *host_data) { struct irq_domain *domain; + gfp_t gfp_flags = GFP_KERNEL; + +#ifdef CONFIG_X86 + /* + * BIG FAT COMMENT: Early initialization paths like enable_IR_x2apic(), + * for example, call into here with interrupts disabled but then we do + * allocate memory and can sleep so no-no. A proper fix would be to do + * x2APIC IR setup in the early irq setup path but it is too late for + * fixing it this way now, shortly before the merge window. + * + * So we do this little brown paper bag, which is temporary! Do not even + * think of calling irq domain setup code with IRQs disabled. You will + * get frozen-sharked! + */ + if (irqs_disabled()) + gfp_flags = GFP_NOFS; +#endif domain = kzalloc_node(sizeof(*domain) + (sizeof(unsigned int) * size), - GFP_KERNEL, of_node_to_nid(of_node)); + gfp_flags, of_node_to_nid(of_node)); if (WARN_ON(!domain)) return NULL; /* Fill structure */ - INIT_RADIX_TREE(&domain->revmap_tree, GFP_KERNEL); + INIT_RADIX_TREE(&domain->revmap_tree, gfp_flags); domain->ops = ops; domain->host_data = host_data; domain->of_node = of_node_get(of_node);