From patchwork Wed May 17 00:48:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dong Jia Shi X-Patchwork-Id: 9729841 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8516760386 for ; Wed, 17 May 2017 00:48:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 76D62285EE for ; Wed, 17 May 2017 00:48:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B78428618; Wed, 17 May 2017 00:48:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C55FA285EE for ; Wed, 17 May 2017 00:48:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752529AbdEQAsx (ORCPT ); Tue, 16 May 2017 20:48:53 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:41993 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751394AbdEQAsv (ORCPT ); Tue, 16 May 2017 20:48:51 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v4H0mpoP088456 for ; Tue, 16 May 2017 20:48:51 -0400 Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.149]) by mx0b-001b2d01.pphosted.com with ESMTP id 2ag7efkcky-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 16 May 2017 20:48:50 -0400 Received: from localhost by e31.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 16 May 2017 18:48:49 -0600 Received: from b03cxnp08026.gho.boulder.ibm.com (9.17.130.18) by e31.co.us.ibm.com (192.168.1.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 16 May 2017 18:48:46 -0600 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v4H0mf8W11141620; Tue, 16 May 2017 17:48:45 -0700 Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9EBA3C603E; Tue, 16 May 2017 18:48:44 -0600 (MDT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP id E4571C6047; Tue, 16 May 2017 18:48:42 -0600 (MDT) From: Dong Jia Shi To: kvm@vger.kernel.org, linux-s390@vger.kernel.org, qemu-devel@nongnu.org Cc: bjsdjshi@linux.vnet.ibm.com, cornelia.huck@de.ibm.com, borntraeger@de.ibm.com, agraf@suse.com, alex.williamson@redhat.com, eric.auger@redhat.com, Xiao Feng Ren Subject: [PATCH v8 07/13] vfio/ccw: vfio based subchannel passthrough driver Date: Wed, 17 May 2017 02:48:07 +0200 X-Mailer: git-send-email 2.11.2 In-Reply-To: <20170517004813.58227-1-bjsdjshi@linux.vnet.ibm.com> References: <20170517004813.58227-1-bjsdjshi@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 17051700-8235-0000-0000-00000B82A1AF X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007074; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000212; SDB=6.00861499; UDB=6.00427296; IPR=6.00641159; BA=6.00005352; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015485; XFM=3.00000015; UTC=2017-05-17 00:48:49 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17051700-8236-0000-0000-00003BD6923A Message-Id: <20170517004813.58227-8-bjsdjshi@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-16_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705170006 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Xiao Feng Ren We use the IOMMU_TYPE1 of VFIO to realize the subchannels passthrough, implement a vfio based subchannels passthrough driver called "vfio-ccw". Support qemu parameters in the style of: "-device vfio-ccw,sysfsdev=$mdev_file_path,devno=xx.x.xxxx' Reviewed-by: Eric Auger Acked-by: Alex Williamson Signed-off-by: Xiao Feng Ren Signed-off-by: Dong Jia Shi --- default-configs/s390x-softmmu.mak | 1 + hw/vfio/Makefile.objs | 1 + hw/vfio/ccw.c | 187 ++++++++++++++++++++++++++++++++++++++ include/hw/vfio/vfio-common.h | 1 + 4 files changed, 190 insertions(+) create mode 100644 hw/vfio/ccw.c diff --git a/default-configs/s390x-softmmu.mak b/default-configs/s390x-softmmu.mak index 9615a48f80..18aed56fc0 100644 --- a/default-configs/s390x-softmmu.mak +++ b/default-configs/s390x-softmmu.mak @@ -5,4 +5,5 @@ CONFIG_SCLPCONSOLE=y CONFIG_TERMINAL3270=y CONFIG_S390_FLIC=y CONFIG_S390_FLIC_KVM=$(CONFIG_KVM) +CONFIG_VFIO_CCW=$(CONFIG_LINUX) CONFIG_WDT_DIAG288=y diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs index 05e7fbb93f..c3ab9097f1 100644 --- a/hw/vfio/Makefile.objs +++ b/hw/vfio/Makefile.objs @@ -1,6 +1,7 @@ ifeq ($(CONFIG_LINUX), y) obj-$(CONFIG_SOFTMMU) += common.o obj-$(CONFIG_PCI) += pci.o pci-quirks.o +obj-$(CONFIG_VFIO_CCW) += ccw.o obj-$(CONFIG_SOFTMMU) += platform.o obj-$(CONFIG_VFIO_XGMAC) += calxeda-xgmac.o obj-$(CONFIG_VFIO_AMD_XGBE) += amd-xgbe.o diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c new file mode 100644 index 0000000000..7d2497cee6 --- /dev/null +++ b/hw/vfio/ccw.c @@ -0,0 +1,187 @@ +/* + * vfio based subchannel assignment support + * + * Copyright 2017 IBM Corp. + * Author(s): Dong Jia Shi + * Xiao Feng Ren + * Pierre Morel + * + * This work is licensed under the terms of the GNU GPL, version 2 or(at + * your option) any version. See the COPYING file in the top-level + * directory. + */ + +#include +#include + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "hw/sysbus.h" +#include "hw/vfio/vfio.h" +#include "hw/vfio/vfio-common.h" +#include "hw/s390x/s390-ccw.h" +#include "hw/s390x/ccw-device.h" + +#define TYPE_VFIO_CCW "vfio-ccw" +typedef struct VFIOCCWDevice { + S390CCWDevice cdev; + VFIODevice vdev; +} VFIOCCWDevice; + +static void vfio_ccw_compute_needs_reset(VFIODevice *vdev) +{ + vdev->needs_reset = false; +} + +/* + * We don't need vfio_hot_reset_multi and vfio_eoi operations for + * vfio_ccw device now. + */ +struct VFIODeviceOps vfio_ccw_ops = { + .vfio_compute_needs_reset = vfio_ccw_compute_needs_reset, +}; + +static void vfio_ccw_reset(DeviceState *dev) +{ + CcwDevice *ccw_dev = DO_UPCAST(CcwDevice, parent_obj, dev); + S390CCWDevice *cdev = DO_UPCAST(S390CCWDevice, parent_obj, ccw_dev); + VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev); + + ioctl(vcdev->vdev.fd, VFIO_DEVICE_RESET); +} + +static void vfio_put_device(VFIOCCWDevice *vcdev) +{ + g_free(vcdev->vdev.name); + vfio_put_base_device(&vcdev->vdev); +} + +static VFIOGroup *vfio_ccw_get_group(S390CCWDevice *cdev, Error **errp) +{ + char *tmp, group_path[PATH_MAX]; + ssize_t len; + int groupid; + + tmp = g_strdup_printf("/sys/bus/css/devices/%x.%x.%04x/%s/iommu_group", + cdev->hostid.cssid, cdev->hostid.ssid, + cdev->hostid.devid, cdev->mdevid); + len = readlink(tmp, group_path, sizeof(group_path)); + g_free(tmp); + + if (len <= 0 || len >= sizeof(group_path)) { + error_setg(errp, "vfio: no iommu_group found"); + return NULL; + } + + group_path[len] = 0; + + if (sscanf(basename(group_path), "%d", &groupid) != 1) { + error_setg(errp, "vfio: failed to read %s", group_path); + return NULL; + } + + return vfio_get_group(groupid, &address_space_memory, errp); +} + +static void vfio_ccw_realize(DeviceState *dev, Error **errp) +{ + VFIODevice *vbasedev; + VFIOGroup *group; + CcwDevice *ccw_dev = DO_UPCAST(CcwDevice, parent_obj, dev); + S390CCWDevice *cdev = DO_UPCAST(S390CCWDevice, parent_obj, ccw_dev); + VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev); + S390CCWDeviceClass *cdc = S390_CCW_DEVICE_GET_CLASS(cdev); + Error *err = NULL; + + /* Call the class init function for subchannel. */ + if (cdc->realize) { + cdc->realize(cdev, vcdev->vdev.sysfsdev, &err); + if (err) { + goto out_err_propagate; + } + } + + group = vfio_ccw_get_group(cdev, &err); + if (!group) { + goto out_group_err; + } + + vcdev->vdev.ops = &vfio_ccw_ops; + vcdev->vdev.type = VFIO_DEVICE_TYPE_CCW; + vcdev->vdev.name = g_strdup_printf("%x.%x.%04x", cdev->hostid.cssid, + cdev->hostid.ssid, cdev->hostid.devid); + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (strcmp(vbasedev->name, vcdev->vdev.name) == 0) { + error_setg(&err, "vfio: subchannel %s has already been attached", + vcdev->vdev.name); + goto out_device_err; + } + } + + if (vfio_get_device(group, cdev->mdevid, &vcdev->vdev, &err)) { + goto out_device_err; + } + + return; + +out_device_err: + vfio_put_group(group); +out_group_err: + if (cdc->unrealize) { + cdc->unrealize(cdev, NULL); + } +out_err_propagate: + error_propagate(errp, err); +} + +static void vfio_ccw_unrealize(DeviceState *dev, Error **errp) +{ + CcwDevice *ccw_dev = DO_UPCAST(CcwDevice, parent_obj, dev); + S390CCWDevice *cdev = DO_UPCAST(S390CCWDevice, parent_obj, ccw_dev); + VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev); + S390CCWDeviceClass *cdc = S390_CCW_DEVICE_GET_CLASS(cdev); + VFIOGroup *group = vcdev->vdev.group; + + vfio_put_device(vcdev); + vfio_put_group(group); + + if (cdc->unrealize) { + cdc->unrealize(cdev, errp); + } +} + +static Property vfio_ccw_properties[] = { + DEFINE_PROP_STRING("sysfsdev", VFIOCCWDevice, vdev.sysfsdev), + DEFINE_PROP_END_OF_LIST(), +}; + +static const VMStateDescription vfio_ccw_vmstate = { + .name = TYPE_VFIO_CCW, + .unmigratable = 1, +}; + +static void vfio_ccw_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + dc->props = vfio_ccw_properties; + dc->vmsd = &vfio_ccw_vmstate; + dc->desc = "VFIO-based subchannel assignment"; + dc->realize = vfio_ccw_realize; + dc->unrealize = vfio_ccw_unrealize; + dc->reset = vfio_ccw_reset; +} + +static const TypeInfo vfio_ccw_info = { + .name = TYPE_VFIO_CCW, + .parent = TYPE_S390_CCW, + .instance_size = sizeof(VFIOCCWDevice), + .class_init = vfio_ccw_class_init, +}; + +static void register_vfio_ccw_type(void) +{ + type_register_static(&vfio_ccw_info); +} + +type_init(register_vfio_ccw_type) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index c582de18c9..9521013d52 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -45,6 +45,7 @@ enum { VFIO_DEVICE_TYPE_PCI = 0, VFIO_DEVICE_TYPE_PLATFORM = 1, + VFIO_DEVICE_TYPE_CCW = 2, }; typedef struct VFIOMmap {