From patchwork Fri Jan 20 09:15:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cornelia Huck X-Patchwork-Id: 9527783 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A5B4A6020B for ; Fri, 20 Jan 2017 09:16:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 97D0228626 for ; Fri, 20 Jan 2017 09:16:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B0AF28683; Fri, 20 Jan 2017 09:16:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EBD6028626 for ; Fri, 20 Jan 2017 09:16:52 +0000 (UTC) Received: from localhost ([::1]:53257 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cUVJT-0004X1-S4 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 20 Jan 2017 04:16:51 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33834) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cUVIp-0004US-Vp for qemu-devel@nongnu.org; Fri, 20 Jan 2017 04:16:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cUVIl-0003cj-8C for qemu-devel@nongnu.org; Fri, 20 Jan 2017 04:16:11 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:48729 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cUVIl-0003cX-1q for qemu-devel@nongnu.org; Fri, 20 Jan 2017 04:16:07 -0500 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v0K9De5s131536 for ; Fri, 20 Jan 2017 04:16:06 -0500 Received: from e06smtp10.uk.ibm.com (e06smtp10.uk.ibm.com [195.75.94.106]) by mx0a-001b2d01.pphosted.com with ESMTP id 283dbydjd0-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Fri, 20 Jan 2017 04:16:06 -0500 Received: from localhost by e06smtp10.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 20 Jan 2017 09:16:04 -0000 Received: from d06dlp02.portsmouth.uk.ibm.com (9.149.20.14) by e06smtp10.uk.ibm.com (192.168.101.140) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 20 Jan 2017 09:16:02 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by d06dlp02.portsmouth.uk.ibm.com (Postfix) with ESMTP id 5F40E219005F; Fri, 20 Jan 2017 09:15:08 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v0K9G2Ke13697302; Fri, 20 Jan 2017 09:16:02 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 226C211C069; Fri, 20 Jan 2017 08:13:56 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DB0B211C064; Fri, 20 Jan 2017 08:13:55 +0000 (GMT) Received: from gondolin.boeblingen.de.ibm.com (unknown [9.152.224.55]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Fri, 20 Jan 2017 08:13:55 +0000 (GMT) From: Cornelia Huck To: peter.maydell@linaro.org Date: Fri, 20 Jan 2017 10:15:46 +0100 X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170120091553.11013-1-cornelia.huck@de.ibm.com> References: <20170120091553.11013-1-cornelia.huck@de.ibm.com> X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 17012009-0040-0000-0000-0000031A7336 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17012009-0041-0000-0000-000023E13D63 Message-Id: <20170120091553.11013-6-cornelia.huck@de.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-01-20_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=3 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1612050000 definitions=main-1701200135 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PULL v2 05/12] s390x/pci: dynamically allocate iommu X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yi Min Zhao , agraf@suse.de, qemu-devel@nongnu.org, borntraeger@de.ibm.com, jfrei@linux.vnet.ibm.com, Cornelia Huck Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Yi Min Zhao When initializing a PCI device, an address space is required during PCI core initialization and before the call to the embedding object hotplug callback. To provide this AS, we allocate a S390PCIIOMMU object containing this AS. Initialization of S390PCIIOMMU object is done before the PCI device is completely created. So that we cannot associate the IOMMU with the device at the moment. To track the IOMMU object, we use g_hash functions with the PCI device's bus address as a key to provide an array of pointers indexed by the PCI device's devfn to the allocated IOMMU. Signed-off-by: Yi Min Zhao Reviewed-by: Pierre Morel Signed-off-by: Cornelia Huck --- hw/s390x/s390-pci-bus.c | 72 +++++++++++++++++++++++++++++++++++++++---------- hw/s390x/s390-pci-bus.h | 7 ++++- 2 files changed, 64 insertions(+), 15 deletions(-) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index 11244fc93c..f3afb17f6a 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -432,11 +432,48 @@ static const MemoryRegionIOMMUOps s390_iommu_ops = { .translate = s390_translate_iommu, }; +static S390PCIIOMMU *s390_pci_get_iommu(S390pciState *s, PCIBus *bus, + int devfn) +{ + uint64_t key = (uintptr_t)bus; + S390PCIIOMMUTable *table = g_hash_table_lookup(s->iommu_table, &key); + S390PCIIOMMU *iommu; + + if (!table) { + table = g_malloc0(sizeof(S390PCIIOMMUTable)); + table->key = key; + g_hash_table_insert(s->iommu_table, &table->key, table); + } + + iommu = table->iommu[PCI_SLOT(devfn)]; + if (!iommu) { + iommu = S390_PCI_IOMMU(object_new(TYPE_S390_PCI_IOMMU)); + + char *mr_name = g_strdup_printf("iommu-root-%02x:%02x.%01x", + pci_bus_num(bus), + PCI_SLOT(devfn), + PCI_FUNC(devfn)); + char *as_name = g_strdup_printf("iommu-pci-%02x:%02x.%01x", + pci_bus_num(bus), + PCI_SLOT(devfn), + PCI_FUNC(devfn)); + memory_region_init(&iommu->mr, OBJECT(iommu), mr_name, UINT64_MAX); + address_space_init(&iommu->as, &iommu->mr, as_name); + table->iommu[PCI_SLOT(devfn)] = iommu; + + g_free(mr_name); + g_free(as_name); + } + + return iommu; +} + static AddressSpace *s390_pci_dma_iommu(PCIBus *bus, void *opaque, int devfn) { S390pciState *s = opaque; + S390PCIIOMMU *iommu = s390_pci_get_iommu(s, bus, devfn); - return &s->iommu[PCI_SLOT(devfn)]->as; + return &iommu->as; } static uint8_t set_ind_atomic(uint64_t ind_loc, uint8_t to_be_set) @@ -520,19 +557,21 @@ void s390_pci_iommu_disable(S390PCIIOMMU *iommu) object_unparent(OBJECT(&iommu->iommu_mr)); } -static void s390_pcihost_init_as(S390pciState *s) +static void s390_pci_iommu_free(PCIBus *bus, int32_t devfn) { - int i; - S390PCIIOMMU *iommu; + uint64_t key = (uintptr_t)bus; + S390PCIIOMMUTable *table = g_hash_table_lookup(s->iommu_table, &key); + S390PCIIOMMU *iommu = table ? table->iommu[PCI_SLOT(devfn)] : NULL; - for (i = 0; i < PCI_SLOT_MAX; i++) { - iommu = g_malloc0(sizeof(S390PCIIOMMU)); - memory_region_init(&iommu->mr, OBJECT(s), - "iommu-root-s390", UINT64_MAX); - address_space_init(&iommu->as, &iommu->mr, "iommu-pci"); - - s->iommu[i] = iommu; + if (!table || !iommu) { + return; } + + table->iommu[PCI_SLOT(devfn)] = NULL; + address_space_destroy(&iommu->as); + object_unparent(OBJECT(&iommu->mr)); + object_unparent(OBJECT(iommu)); + object_unref(OBJECT(iommu)); } static int s390_pcihost_init(SysBusDevice *dev) @@ -548,7 +587,6 @@ static int s390_pcihost_init(SysBusDevice *dev) s390_pci_set_irq, s390_pci_map_irq, NULL, get_system_memory(), get_system_io(), 0, 64, TYPE_PCI_BUS); - s390_pcihost_init_as(s); pci_setup_iommu(b, s390_pci_dma_iommu, s); bus = BUS(b); @@ -558,6 +596,8 @@ static int s390_pcihost_init(SysBusDevice *dev) s->bus = S390_PCI_BUS(qbus_create(TYPE_S390_PCI_BUS, DEVICE(s), NULL)); qbus_set_hotplug_handler(BUS(s->bus), DEVICE(s), NULL); + s->iommu_table = g_hash_table_new_full(g_int64_hash, g_int64_equal, + NULL, g_free); QTAILQ_INIT(&s->pending_sei); return 0; } @@ -661,7 +701,7 @@ static void s390_pcihost_hot_plug(HotplugHandler *hotplug_dev, } pbdev->pdev = pdev; - pbdev->iommu = s->iommu[PCI_SLOT(pdev->devfn)]; + pbdev->iommu = s390_pci_get_iommu(s, pdev->bus, pdev->devfn); pbdev->iommu->pbdev = pbdev; pbdev->state = ZPCI_FS_STANDBY; @@ -708,8 +748,9 @@ static void s390_pcihost_timer_cb(void *opaque) static void s390_pcihost_hot_unplug(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { - int i; PCIDevice *pci_dev = NULL; + PCIBus *bus; + int32_t devfn, i; S390PCIBusDevice *pbdev = NULL; S390pciState *s = s390_get_phb(); @@ -752,8 +793,11 @@ static void s390_pcihost_hot_unplug(HotplugHandler *hotplug_dev, s390_pci_generate_plug_event(HP_EVENT_STANDBY_TO_RESERVED, pbdev->fh, pbdev->fid); + bus = pci_dev->bus; + devfn = pci_dev->devfn; object_unparent(OBJECT(pci_dev)); s390_pci_msix_free(pbdev); + s390_pci_iommu_free(bus, devfn); pbdev->pdev = NULL; pbdev->state = ZPCI_FS_RESERVED; out: diff --git a/hw/s390x/s390-pci-bus.h b/hw/s390x/s390-pci-bus.h index 1d05c1a92e..fe108e9330 100644 --- a/hw/s390x/s390-pci-bus.h +++ b/hw/s390x/s390-pci-bus.h @@ -274,6 +274,11 @@ typedef struct S390PCIIOMMU { uint64_t pal; } S390PCIIOMMU; +typedef struct S390PCIIOMMUTable { + uint64_t key; + S390PCIIOMMU *iommu[PCI_SLOT_MAX]; +} S390PCIIOMMUTable; + typedef struct S390PCIBusDevice { DeviceState qdev; PCIDevice *pdev; @@ -304,7 +309,7 @@ typedef struct S390pciState { PCIHostState parent_obj; S390PCIBus *bus; S390PCIBusDevice *pbdev[PCI_SLOT_MAX]; - S390PCIIOMMU *iommu[PCI_SLOT_MAX]; + GHashTable *iommu_table; QTAILQ_HEAD(, SeiContainer) pending_sei; } S390pciState;