From patchwork Thu Sep 19 13:22:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11152377 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9DA9814ED for ; Thu, 19 Sep 2019 13:24:00 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 83B08217D6 for ; Thu, 19 Sep 2019 13:24:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 83B08217D6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iAwOW-0007rl-T9; Thu, 19 Sep 2019 13:22:48 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iAwOV-0007rU-PS for xen-devel@lists.xenproject.org; Thu, 19 Sep 2019 13:22:47 +0000 X-Inumbo-ID: 91c8eb40-dae0-11e9-b299-bc764e2007e4 Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 91c8eb40-dae0-11e9-b299-bc764e2007e4; Thu, 19 Sep 2019 13:22:47 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 33B46AE40; Thu, 19 Sep 2019 13:22:46 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: Message-ID: <14624609-019f-2993-261c-d4f45ce78cbe@suse.com> Date: Thu, 19 Sep 2019 15:22:54 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: [Xen-devel] [PATCH v6 3/8] x86/PCI: read maximum MSI vector count early X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Suravee Suthikulpanit , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Rather than doing this every time we set up interrupts for a device anew (and then in several places) fill this invariant field right after allocating struct pci_dev. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné Reviewed-by: Paul Durrant Reviewed-by: Andrew Cooper --- v6: New. --- xen/arch/x86/msi.c | 13 +++++-------- xen/drivers/passthrough/pci.c | 10 ++++++++++ xen/drivers/vpci/msi.c | 9 ++++----- xen/include/xen/pci.h | 3 ++- xen/include/xen/vpci.h | 6 ++---- 5 files changed, 23 insertions(+), 18 deletions(-) --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -664,7 +664,7 @@ static int msi_capability_init(struct pc { struct msi_desc *entry; int pos; - unsigned int i, maxvec, mpos; + unsigned int i, mpos; u16 control, seg = dev->seg; u8 bus = dev->bus; u8 slot = PCI_SLOT(dev->devfn); @@ -675,9 +675,8 @@ static int msi_capability_init(struct pc if ( !pos ) return -ENODEV; control = pci_conf_read16(dev->sbdf, msi_control_reg(pos)); - maxvec = multi_msi_capable(control); - if ( nvec > maxvec ) - return maxvec; + if ( nvec > dev->msi_maxvec ) + return dev->msi_maxvec; control &= ~PCI_MSI_FLAGS_QSIZE; multi_msi_enable(control, nvec); @@ -711,7 +710,7 @@ static int msi_capability_init(struct pc /* All MSIs are unmasked by default, Mask them all */ maskbits = pci_conf_read32(dev->sbdf, mpos); - maskbits |= ~(u32)0 >> (32 - maxvec); + maskbits |= ~(u32)0 >> (32 - dev->msi_maxvec); pci_conf_write32(dev->sbdf, mpos, maskbits); } list_add_tail(&entry->list, &dev->msi_list); @@ -1284,7 +1283,6 @@ int pci_msi_conf_write_intercept(struct entry = find_msi_entry(pdev, -1, PCI_CAP_ID_MSI); if ( entry && entry->msi_attrib.maskbit ) { - uint16_t cntl; uint32_t unused; unsigned int nvec = entry->msi.nvec; @@ -1297,8 +1295,7 @@ int pci_msi_conf_write_intercept(struct if ( reg < entry->msi.mpos || reg >= entry->msi.mpos + 4 || size != 4 ) return -EACCES; - cntl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos)); - unused = ~(uint32_t)0 >> (32 - multi_msi_capable(cntl)); + unused = ~(uint32_t)0 >> (32 - pdev->msi_maxvec); for ( pos = 0; pos < nvec; ++pos, ++entry ) { entry->msi_attrib.guest_masked = --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -340,6 +340,16 @@ static struct pci_dev *alloc_pdev(struct pdev->domain = NULL; INIT_LIST_HEAD(&pdev->msi_list); + + pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn), + PCI_CAP_ID_MSI); + if ( pos ) + { + uint16_t ctrl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos)); + + pdev->msi_maxvec = multi_msi_capable(ctrl); + } + pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn), PCI_CAP_ID_MSIX); if ( pos ) --- a/xen/drivers/vpci/msi.c +++ b/xen/drivers/vpci/msi.c @@ -27,7 +27,7 @@ static uint32_t control_read(const struc { const struct vpci_msi *msi = data; - return MASK_INSR(fls(msi->max_vectors) - 1, PCI_MSI_FLAGS_QMASK) | + return MASK_INSR(fls(pdev->msi_maxvec) - 1, PCI_MSI_FLAGS_QMASK) | MASK_INSR(fls(msi->vectors) - 1, PCI_MSI_FLAGS_QSIZE) | (msi->enabled ? PCI_MSI_FLAGS_ENABLE : 0) | (msi->masking ? PCI_MSI_FLAGS_MASKBIT : 0) | @@ -40,7 +40,7 @@ static void control_write(const struct p struct vpci_msi *msi = data; unsigned int vectors = min_t(uint8_t, 1u << MASK_EXTR(val, PCI_MSI_FLAGS_QSIZE), - msi->max_vectors); + pdev->msi_maxvec); bool new_enabled = val & PCI_MSI_FLAGS_ENABLE; /* @@ -215,8 +215,7 @@ static int init_msi(struct pci_dev *pdev * FIXME: I've only been able to test this code with devices using a single * MSI interrupt and no mask register. */ - pdev->vpci->msi->max_vectors = multi_msi_capable(control); - ASSERT(pdev->vpci->msi->max_vectors <= 32); + ASSERT(pdev->msi_maxvec <= 32); /* The multiple message enable is 0 after reset (1 message enabled). */ pdev->vpci->msi->vectors = 1; @@ -298,7 +297,7 @@ void vpci_dump_msi(void) if ( msi->masking ) printk(" mask=%08x", msi->mask); printk(" vectors max: %u enabled: %u\n", - msi->max_vectors, msi->vectors); + pdev->msi_maxvec, msi->vectors); vpci_msi_arch_print(msi); } --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -94,7 +94,8 @@ struct pci_dev { pci_sbdf_t sbdf; }; - u8 phantom_stride; + uint8_t msi_maxvec; + uint8_t phantom_stride; nodeid_t node; /* NUMA node */ --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -99,14 +99,12 @@ struct vpci { uint32_t mask; /* Data. */ uint16_t data; - /* Maximum number of vectors supported by the device. */ - uint8_t max_vectors : 6; + /* Number of vectors configured. */ + uint8_t vectors : 6; /* Supports per-vector masking? */ bool masking : 1; /* 64-bit address capable? */ bool address64 : 1; - /* Number of vectors configured. */ - uint8_t vectors : 6; /* Enabled? */ bool enabled : 1; /* Arch-specific data. */