From patchwork Thu May 6 15:31:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pali_Roh=C3=A1r?= X-Patchwork-Id: 12242595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23580C433ED for ; Thu, 6 May 2021 15:42:23 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7FEAE6101A for ; Thu, 6 May 2021 15:42:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7FEAE6101A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=h61LW2N4Ohb5BqMspQTMFToWKI98Cm33tQqwh4wxMZ8=; b=SMNWfpfV7bdDHIS/H984+S0ZD 5HR4bDIXo5Kipc/yYtBxqGZ4eNK1pzyFRaU/CXgabw3dWagUW4GX28dD4VuYX2hpcsmgVbCAsgKe4 S2IMCn1967qzYgqGRd02Kp21tK6yxNPONXpFDcZWMcbjGiAbe3/SrQ6W11E1J5QhRlWsOLCm4XbCA Fhmd145Is4cRvHB6kDCo12HMI2qGI58S64olxgPbu7PmzQi/p25Ca7twcFOyX1Mdl9LxkHOd8G//l rjyidr7CH4JKraR2H25nyjKGJFlHEAuN7zoNKMeNl2xnYIGtsMlTRaPT5hIbvz4Anx+OxTbgoJveU 9aMS+lZSA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1leg77-004cid-46; Thu, 06 May 2021 15:40:34 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lefzn-004a5x-8M for linux-arm-kernel@desiato.infradead.org; Thu, 06 May 2021 15:32:59 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc: To:From:Sender:Reply-To:Content-ID:Content-Description; bh=WCXCSMlLju+RVMt1dqMxOrA+GhhtjeZ2xOqd4EhD97U=; b=mZ5OExcgnrfzJVBI2es8H/TVOO MFEKwO1nXv8hg6v1PO8DCbnyhjh3plAbsuZQemVls5slmcfuOrQtDcjg5YWSxoZEHbQDp8vBOj0ug 1mdmeU0PRYp0Hwljqa+hXIYE9i7UZfgHljrjM0u2O46bbZVuOmm2zjJZMxqbVHUj+wKDX0Pn/ETtN P/sQwGiRP36EsEkOhzc3f+bYtxaxrFmofJl0U1O00poApI+/miDcKAjv1UCcfM4KnpA1PMk6lOqLz e7fs1ScQWtEM0fTXyQyOp7cbiqUU1CRl7B9xLW0nctfnmWcweoiaixSr6vxfGoPm8FNLLAj0fSXTw UoUl+B6Q==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lefzh-0069jp-1F for linux-arm-kernel@lists.infradead.org; Thu, 06 May 2021 15:32:57 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5AEF5613F6; Thu, 6 May 2021 15:32:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1620315169; bh=yN1GVDsVClOczJBW/hcMrhTyF36UkgV7hMJV2qJoxrY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pxrXFrmgOLLG5cl31a3HZDuTj2XGD0N78K9/16eUsDSmLqb6edR911/rRpLPEtCKA NUdpDhPM0L4hVdUuTHkelRXUtS0myrC7wECtiYkKeKmZZwLJLGKQYMK/5qOIisBNKh 49hoh9odMpOUA6GnACS1dKV4kx50V4eDtoBo8wpSb1rxpT/fvESvS/zXQYlgdZpo9j AdVS62Rug9qBYXPZ2YzDcKcyuE7+7UHSmF2wJLBrjRE6KHOCU+Ul7VNl+SBby66YQP ellsp8wawVYmHN1+oySrX4HBSZox+iQ8y0NbOYFUaZ3AUEkg+T1efLtbRehf/zbSC9 m8tpKEYfSeKrg== Received: by pali.im (Postfix) id 120E5BF9; Thu, 6 May 2021 17:32:49 +0200 (CEST) From: =?utf-8?q?Pali_Roh=C3=A1r?= To: Lorenzo Pieralisi , Thomas Petazzoni , Rob Herring , Bjorn Helgaas Cc: Russell King , =?utf-8?q?Marek_Beh=C3=BAn?= , Remi Pommarel , Xogium , Tomasz Maciej Nowak , Marc Zyngier , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 17/42] PCI: aardvark: Fix support for MSI interrupts Date: Thu, 6 May 2021 17:31:28 +0200 Message-Id: <20210506153153.30454-18-pali@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210506153153.30454-1-pali@kernel.org> References: <20210506153153.30454-1-pali@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210506_083253_220800_C5D7229F X-CRM114-Status: GOOD ( 19.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org MSI domain callback .alloc (implemented by advk_msi_irq_domain_alloc() function) should return zero on success. Returning non-zero value indicates failure. Fix return value of this function as in many cases it now returns failure while allocating IRQs. Aardvark hardware supports Multi-MSI and MSI_FLAG_MULTI_PCI_MSI is already set. But when allocating MSI interrupt numbers for Multi-MSI, they need to be properly aligned, otherwise endpoint devices send MSI interrupt with incorrect numbers. Fix this issue by using function bitmap_find_free_region() instead of bitmap_find_next_zero_area(). To ensure that aligned MSI interrupt numbers are used by endpoint devices, we cannot use Linux virtual irq numbers (as they are random and not properly aligned). So use hwirq numbers allocated by the function bitmap_find_free_region(), which are aligned. This needs an update in advk_msi_irq_compose_msi_msg() and advk_pcie_handle_msi() functions to do proper mapping between Linux virtual irq numbers and hwirq MSI inner domain numbers. Also the whole 16-bit MSI number is stored in the PCIE_MSI_PAYLOAD_REG register, not only lower 8 bits. Fix reading content of this register. This change fixes receiving MSI interrupts on Armada 3720 boards and allows using NVMe disks which use Multi-MSI feature with 3 interrupts. Without this change, NVMe disks just freeze booting Linux on Armada 3720 boards as linux nvme-core.c driver is waiting 60s for an interrupt. Signed-off-by: Pali Rohár Reviewed-by: Marek Behún Cc: stable@vger.kernel.org # f21a8b1b6837 ("PCI: aardvark: Move to MSI handling using generic MSI support") --- drivers/pci/controller/pci-aardvark.c | 32 ++++++++++++++++----------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c index 366d7480bc1b..498810c00b6d 100644 --- a/drivers/pci/controller/pci-aardvark.c +++ b/drivers/pci/controller/pci-aardvark.c @@ -118,6 +118,7 @@ #define PCIE_MSI_STATUS_REG (CONTROL_BASE_ADDR + 0x58) #define PCIE_MSI_MASK_REG (CONTROL_BASE_ADDR + 0x5C) #define PCIE_MSI_PAYLOAD_REG (CONTROL_BASE_ADDR + 0x9C) +#define PCIE_MSI_DATA_MASK GENMASK(15, 0) /* LMI registers base address and register offsets */ #define LMI_BASE_ADDR 0x6000 @@ -861,7 +862,7 @@ static void advk_msi_irq_compose_msi_msg(struct irq_data *data, msg->address_lo = lower_32_bits(msi_msg); msg->address_hi = upper_32_bits(msi_msg); - msg->data = data->irq; + msg->data = data->hwirq; } static int advk_msi_set_affinity(struct irq_data *irq_data, @@ -878,15 +879,11 @@ static int advk_msi_irq_domain_alloc(struct irq_domain *domain, int hwirq, i; mutex_lock(&pcie->msi_used_lock); - hwirq = bitmap_find_next_zero_area(pcie->msi_used, MSI_IRQ_NUM, - 0, nr_irqs, 0); - if (hwirq >= MSI_IRQ_NUM) { - mutex_unlock(&pcie->msi_used_lock); - return -ENOSPC; - } - - bitmap_set(pcie->msi_used, hwirq, nr_irqs); + hwirq = bitmap_find_free_region(pcie->msi_used, MSI_IRQ_NUM, + order_base_2(nr_irqs)); mutex_unlock(&pcie->msi_used_lock); + if (hwirq < 0) + return -ENOSPC; for (i = 0; i < nr_irqs; i++) irq_domain_set_info(domain, virq + i, hwirq + i, @@ -894,7 +891,7 @@ static int advk_msi_irq_domain_alloc(struct irq_domain *domain, domain->host_data, handle_simple_irq, NULL, NULL); - return hwirq; + return 0; } static void advk_msi_irq_domain_free(struct irq_domain *domain, @@ -904,7 +901,7 @@ static void advk_msi_irq_domain_free(struct irq_domain *domain, struct advk_pcie *pcie = domain->host_data; mutex_lock(&pcie->msi_used_lock); - bitmap_clear(pcie->msi_used, d->hwirq, nr_irqs); + bitmap_release_region(pcie->msi_used, d->hwirq, order_base_2(nr_irqs)); mutex_unlock(&pcie->msi_used_lock); } @@ -1048,6 +1045,7 @@ static void advk_pcie_handle_msi(struct advk_pcie *pcie) { u32 msi_val, msi_mask, msi_status, msi_idx; u16 msi_data; + int virq; msi_mask = advk_readl(pcie, PCIE_MSI_MASK_REG); msi_val = advk_readl(pcie, PCIE_MSI_STATUS_REG); @@ -1057,9 +1055,17 @@ static void advk_pcie_handle_msi(struct advk_pcie *pcie) if (!(BIT(msi_idx) & msi_status)) continue; + /* + * msi_idx contains bits [4:0] of the msi_data and msi_data + * contains 16bit MSI interrupt number from MSI inner domain + */ advk_writel(pcie, BIT(msi_idx), PCIE_MSI_STATUS_REG); - msi_data = advk_readl(pcie, PCIE_MSI_PAYLOAD_REG) & 0xFF; - generic_handle_irq(msi_data); + msi_data = advk_readl(pcie, PCIE_MSI_PAYLOAD_REG) & PCIE_MSI_DATA_MASK; + virq = irq_find_mapping(pcie->msi_inner_domain, msi_data); + if (virq) + generic_handle_irq(virq); + else + dev_err(&pcie->pdev->dev, "unexpected MSI 0x%04hx\n", msi_data); } advk_writel(pcie, PCIE_ISR0_MSI_INT_PENDING,