From patchwork Tue Feb 23 11:30:41 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 8391061 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 60765C0553 for ; Tue, 23 Feb 2016 11:33:27 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 07EAF202F8 for ; Tue, 23 Feb 2016 11:33:26 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 95BC320274 for ; Tue, 23 Feb 2016 11:33:24 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aYBB3-00033v-Ct; Tue, 23 Feb 2016 11:30:49 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aYBB1-00033J-Hh for xen-devel@lists.xenproject.org; Tue, 23 Feb 2016 11:30:47 +0000 Received: from [85.158.137.68] by server-17.bemta-3.messagelabs.com id C8/E5-03109-6E24CC65; Tue, 23 Feb 2016 11:30:46 +0000 X-Env-Sender: JBeulich@suse.com X-Msg-Ref: server-16.tower-31.messagelabs.com!1456227043!16885455!1 X-Originating-IP: [137.65.248.74] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 22768 invoked from network); 23 Feb 2016 11:30:45 -0000 Received: from prv-mh.provo.novell.com (HELO prv-mh.provo.novell.com) (137.65.248.74) by server-16.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 23 Feb 2016 11:30:45 -0000 Received: from INET-PRV-MTA by prv-mh.provo.novell.com with Novell_GroupWise; Tue, 23 Feb 2016 04:30:43 -0700 Message-Id: <56CC50F102000078000D52DB@prv-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 14.2.0 Date: Tue, 23 Feb 2016 04:30:41 -0700 From: "Jan Beulich" To: "xen-devel" References: <56CC4F0B02000078000D5290@prv-mh.provo.novell.com> In-Reply-To: <56CC4F0B02000078000D5290@prv-mh.provo.novell.com> Mime-Version: 1.0 Cc: Ian Campbell , Andrew Cooper , Keir Fraser , Ian Jackson , Tim Deegan Subject: [Xen-devel] [PATCH RFC 4/4] ns16550: enable use of PCI MSI X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Which, on x86, requires fiddling with the INTx bit in PCI config space, since for internally used MSI we can't delegate this to Dom0. ns16550_init_postirq() also needs (benign) re-ordering of its operations. Signed-off-by: Jan Beulich --- RFC reason: No interrupts occur for an unknown reason (presumably broken hardware/firmware on the device I have). ns16550: enable use of PCI MSI Which, on x86, requires fiddling with the INTx bit in PCI config space, since for internally used MSI we can't delegate this to Dom0. ns16550_init_postirq() also needs (benign) re-ordering of its operations. Signed-off-by: Jan Beulich --- RFC reason: No interrupts occur for an unknown reason (presumably broken hardware/firmware on the device I have). --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -738,6 +738,16 @@ static int msi_capability_init(struct pc *desc = entry; /* Restore the original MSI enabled bits */ + if ( !hardware_domain ) + { + /* + * Except for internal requests (before Dom0 starts), in which case + * we rather need to behave "normally", i.e. not follow the split + * brain model where Dom0 actually enables MSI (and disables INTx). + */ + pci_intx(dev, 0); + control |= PCI_MSI_FLAGS_ENABLE; + } pci_conf_write16(seg, bus, slot, func, msi_control_reg(pos), control); return 0; @@ -1071,6 +1081,8 @@ static void __pci_disable_msi(struct msi dev = entry->dev; msi_set_enable(dev, 0); + if ( entry->irq > 0 && !(irq_to_desc(entry->irq)->status & IRQ_GUEST) ) + pci_intx(dev, 1); BUG_ON(list_empty(&dev->msi_list)); } --- a/xen/drivers/char/ns16550.c +++ b/xen/drivers/char/ns16550.c @@ -74,6 +74,7 @@ static struct ns16550 { u32 bar64; u16 cr; u8 bar_idx; + bool_t msi; const struct ns16550_config_param *param; /* Points into .init.*! */ #endif } ns16550_com[2] = { { 0 } }; @@ -644,6 +645,16 @@ static void __init ns16550_init_preirq(s uart->fifo_size = 16; } +static void __init ns16550_init_irq(struct serial_port *port) +{ +#ifdef CONFIG_HAS_PCI + struct ns16550 *uart = port->uart; + + if ( uart->msi ) + uart->irq = create_irq(0); +#endif +} + static void ns16550_setup_postirq(struct ns16550 *uart) { if ( uart->irq > 0 ) @@ -678,17 +689,6 @@ static void __init ns16550_init_postirq( uart->timeout_ms = max_t( unsigned int, 1, (bits * uart->fifo_size * 1000) / uart->baud); - if ( uart->irq > 0 ) - { - uart->irqaction.handler = ns16550_interrupt; - uart->irqaction.name = "ns16550"; - uart->irqaction.dev_id = port; - if ( (rc = setup_irq(uart->irq, 0, &uart->irqaction)) != 0 ) - printk("ERROR: Failed to allocate ns16550 IRQ %d\n", uart->irq); - } - - ns16550_setup_postirq(uart); - #ifdef CONFIG_HAS_PCI if ( uart->bar || uart->ps_bdf_enable ) { @@ -709,8 +709,65 @@ static void __init ns16550_init_postirq( uart->ps_bdf[0], uart->ps_bdf[1], uart->ps_bdf[2]); } + + if ( uart->msi ) + { + struct msi_info msi = { + .bus = uart->ps_bdf[0], + .devfn = PCI_DEVFN(uart->ps_bdf[1], uart->ps_bdf[2]), + .irq = rc = uart->irq, + .entry_nr = 1 + }; + + if ( rc > 0 ) + { + struct msi_desc *msi_desc = NULL; + + spin_lock(&pcidevs_lock); + + rc = pci_enable_msi(&msi, &msi_desc); + if ( !rc ) + { + struct irq_desc *desc = irq_to_desc(msi.irq); + unsigned long flags; + + spin_lock_irqsave(&desc->lock, flags); + rc = setup_msi_irq(desc, msi_desc); + spin_unlock_irqrestore(&desc->lock, flags); + if ( rc ) + pci_disable_msi(msi_desc); + } + + spin_unlock(&pcidevs_lock); + + if ( rc ) + { + uart->irq = 0; + if ( msi_desc ) + msi_free_irq(msi_desc); + else + destroy_irq(msi.irq); + } + } + + if ( rc ) + printk(XENLOG_WARNING + "MSI setup failed (%d) for %02x:%02x.%o\n", + rc, uart->ps_bdf[0], uart->ps_bdf[1], uart->ps_bdf[2]); + } } #endif + + if ( uart->irq > 0 ) + { + uart->irqaction.handler = ns16550_interrupt; + uart->irqaction.name = "ns16550"; + uart->irqaction.dev_id = port; + if ( (rc = setup_irq(uart->irq, 0, &uart->irqaction)) != 0 ) + printk("ERROR: Failed to allocate ns16550 IRQ %d\n", uart->irq); + } + + ns16550_setup_postirq(uart); } static void ns16550_suspend(struct serial_port *port) @@ -820,6 +877,7 @@ static const struct vuart_info *ns16550_ static struct uart_driver __read_mostly ns16550_driver = { .init_preirq = ns16550_init_preirq, + .init_irq = ns16550_init_irq, .init_postirq = ns16550_init_postirq, .endboot = ns16550_endboot, .suspend = ns16550_suspend, @@ -1123,7 +1181,18 @@ static void __init ns16550_parse_port_co } if ( *conf == ',' && *++conf != ',' ) - uart->irq = simple_strtol(conf, &conf, 10); + { +#ifdef CONFIG_HAS_PCI + if ( strncmp(conf, "msi", 3) == 0 ) + { + conf += 3; + uart->msi = 1; + uart->irq = 0; + } + else +#endif + uart->irq = simple_strtol(conf, &conf, 10); + } #ifdef CONFIG_HAS_PCI if ( *conf == ',' && *++conf != ',' ) --- a/xen/drivers/pci/pci.c +++ b/xen/drivers/pci/pci.c @@ -115,6 +115,21 @@ int pci_find_next_ext_capability(int seg return 0; } +void pci_intx(const struct pci_dev *pdev, bool_t enable) +{ + uint16_t seg = pdev->seg; + uint8_t bus = pdev->bus; + uint8_t slot = PCI_SLOT(pdev->devfn); + uint8_t func = PCI_FUNC(pdev->devfn); + uint16_t cmd = pci_conf_read16(seg, bus, slot, func, PCI_COMMAND); + + if ( enable ) + cmd &= ~PCI_COMMAND_INTX_DISABLE; + else + cmd |= PCI_COMMAND_INTX_DISABLE; + pci_conf_write16(seg, bus, slot, func, PCI_COMMAND, cmd); +} + const char *__init parse_pci(const char *s, unsigned int *seg_p, unsigned int *bus_p, unsigned int *dev_p, unsigned int *func_p) --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -152,6 +152,7 @@ int pci_find_next_ext_capability(int seg const char *parse_pci(const char *, unsigned int *seg, unsigned int *bus, unsigned int *dev, unsigned int *func); +void pci_intx(const struct pci_dev *, bool_t enable); bool_t pcie_aer_get_firmware_first(const struct pci_dev *); struct pirq; --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -738,6 +738,16 @@ static int msi_capability_init(struct pc *desc = entry; /* Restore the original MSI enabled bits */ + if ( !hardware_domain ) + { + /* + * Except for internal requests (before Dom0 starts), in which case + * we rather need to behave "normally", i.e. not follow the split + * brain model where Dom0 actually enables MSI (and disables INTx). + */ + pci_intx(dev, 0); + control |= PCI_MSI_FLAGS_ENABLE; + } pci_conf_write16(seg, bus, slot, func, msi_control_reg(pos), control); return 0; @@ -1071,6 +1081,8 @@ static void __pci_disable_msi(struct msi dev = entry->dev; msi_set_enable(dev, 0); + if ( entry->irq > 0 && !(irq_to_desc(entry->irq)->status & IRQ_GUEST) ) + pci_intx(dev, 1); BUG_ON(list_empty(&dev->msi_list)); } --- a/xen/drivers/char/ns16550.c +++ b/xen/drivers/char/ns16550.c @@ -74,6 +74,7 @@ static struct ns16550 { u32 bar64; u16 cr; u8 bar_idx; + bool_t msi; const struct ns16550_config_param *param; /* Points into .init.*! */ #endif } ns16550_com[2] = { { 0 } }; @@ -644,6 +645,16 @@ static void __init ns16550_init_preirq(s uart->fifo_size = 16; } +static void __init ns16550_init_irq(struct serial_port *port) +{ +#ifdef CONFIG_HAS_PCI + struct ns16550 *uart = port->uart; + + if ( uart->msi ) + uart->irq = create_irq(0); +#endif +} + static void ns16550_setup_postirq(struct ns16550 *uart) { if ( uart->irq > 0 ) @@ -678,17 +689,6 @@ static void __init ns16550_init_postirq( uart->timeout_ms = max_t( unsigned int, 1, (bits * uart->fifo_size * 1000) / uart->baud); - if ( uart->irq > 0 ) - { - uart->irqaction.handler = ns16550_interrupt; - uart->irqaction.name = "ns16550"; - uart->irqaction.dev_id = port; - if ( (rc = setup_irq(uart->irq, 0, &uart->irqaction)) != 0 ) - printk("ERROR: Failed to allocate ns16550 IRQ %d\n", uart->irq); - } - - ns16550_setup_postirq(uart); - #ifdef CONFIG_HAS_PCI if ( uart->bar || uart->ps_bdf_enable ) { @@ -709,8 +709,65 @@ static void __init ns16550_init_postirq( uart->ps_bdf[0], uart->ps_bdf[1], uart->ps_bdf[2]); } + + if ( uart->msi ) + { + struct msi_info msi = { + .bus = uart->ps_bdf[0], + .devfn = PCI_DEVFN(uart->ps_bdf[1], uart->ps_bdf[2]), + .irq = rc = uart->irq, + .entry_nr = 1 + }; + + if ( rc > 0 ) + { + struct msi_desc *msi_desc = NULL; + + spin_lock(&pcidevs_lock); + + rc = pci_enable_msi(&msi, &msi_desc); + if ( !rc ) + { + struct irq_desc *desc = irq_to_desc(msi.irq); + unsigned long flags; + + spin_lock_irqsave(&desc->lock, flags); + rc = setup_msi_irq(desc, msi_desc); + spin_unlock_irqrestore(&desc->lock, flags); + if ( rc ) + pci_disable_msi(msi_desc); + } + + spin_unlock(&pcidevs_lock); + + if ( rc ) + { + uart->irq = 0; + if ( msi_desc ) + msi_free_irq(msi_desc); + else + destroy_irq(msi.irq); + } + } + + if ( rc ) + printk(XENLOG_WARNING + "MSI setup failed (%d) for %02x:%02x.%o\n", + rc, uart->ps_bdf[0], uart->ps_bdf[1], uart->ps_bdf[2]); + } } #endif + + if ( uart->irq > 0 ) + { + uart->irqaction.handler = ns16550_interrupt; + uart->irqaction.name = "ns16550"; + uart->irqaction.dev_id = port; + if ( (rc = setup_irq(uart->irq, 0, &uart->irqaction)) != 0 ) + printk("ERROR: Failed to allocate ns16550 IRQ %d\n", uart->irq); + } + + ns16550_setup_postirq(uart); } static void ns16550_suspend(struct serial_port *port) @@ -820,6 +877,7 @@ static const struct vuart_info *ns16550_ static struct uart_driver __read_mostly ns16550_driver = { .init_preirq = ns16550_init_preirq, + .init_irq = ns16550_init_irq, .init_postirq = ns16550_init_postirq, .endboot = ns16550_endboot, .suspend = ns16550_suspend, @@ -1123,7 +1181,18 @@ static void __init ns16550_parse_port_co } if ( *conf == ',' && *++conf != ',' ) - uart->irq = simple_strtol(conf, &conf, 10); + { +#ifdef CONFIG_HAS_PCI + if ( strncmp(conf, "msi", 3) == 0 ) + { + conf += 3; + uart->msi = 1; + uart->irq = 0; + } + else +#endif + uart->irq = simple_strtol(conf, &conf, 10); + } #ifdef CONFIG_HAS_PCI if ( *conf == ',' && *++conf != ',' ) --- a/xen/drivers/pci/pci.c +++ b/xen/drivers/pci/pci.c @@ -115,6 +115,21 @@ int pci_find_next_ext_capability(int seg return 0; } +void pci_intx(const struct pci_dev *pdev, bool_t enable) +{ + uint16_t seg = pdev->seg; + uint8_t bus = pdev->bus; + uint8_t slot = PCI_SLOT(pdev->devfn); + uint8_t func = PCI_FUNC(pdev->devfn); + uint16_t cmd = pci_conf_read16(seg, bus, slot, func, PCI_COMMAND); + + if ( enable ) + cmd &= ~PCI_COMMAND_INTX_DISABLE; + else + cmd |= PCI_COMMAND_INTX_DISABLE; + pci_conf_write16(seg, bus, slot, func, PCI_COMMAND, cmd); +} + const char *__init parse_pci(const char *s, unsigned int *seg_p, unsigned int *bus_p, unsigned int *dev_p, unsigned int *func_p) --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -152,6 +152,7 @@ int pci_find_next_ext_capability(int seg const char *parse_pci(const char *, unsigned int *seg, unsigned int *bus, unsigned int *dev, unsigned int *func); +void pci_intx(const struct pci_dev *, bool_t enable); bool_t pcie_aer_get_firmware_first(const struct pci_dev *); struct pirq;