From patchwork Mon Feb 18 19:49:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Krowiak X-Patchwork-Id: 10818737 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22E34139A for ; Mon, 18 Feb 2019 19:51:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C6F92BAD5 for ; Mon, 18 Feb 2019 19:51:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 000C32BAD7; Mon, 18 Feb 2019 19:51:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6CDBA2BAD5 for ; Mon, 18 Feb 2019 19:51:40 +0000 (UTC) Received: from localhost ([127.0.0.1]:35874 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gvox1-0004XO-Q5 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 18 Feb 2019 14:51:39 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33393) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gvovn-0003hR-34 for qemu-devel@nongnu.org; Mon, 18 Feb 2019 14:50:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gvovk-0007to-Lz for qemu-devel@nongnu.org; Mon, 18 Feb 2019 14:50:23 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:40814) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gvovk-0007rM-34 for qemu-devel@nongnu.org; Mon, 18 Feb 2019 14:50:20 -0500 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x1IJhoCD096305 for ; Mon, 18 Feb 2019 14:50:18 -0500 Received: from e32.co.us.ibm.com (e32.co.us.ibm.com [32.97.110.150]) by mx0a-001b2d01.pphosted.com with ESMTP id 2qr151dybt-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 18 Feb 2019 14:50:18 -0500 Received: from localhost by e32.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 18 Feb 2019 19:50:17 -0000 Received: from b03cxnp07028.gho.boulder.ibm.com (9.17.130.15) by e32.co.us.ibm.com (192.168.1.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 18 Feb 2019 19:50:14 -0000 Received: from b03ledav003.gho.boulder.ibm.com (b03ledav003.gho.boulder.ibm.com [9.17.130.234]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x1IJoBBJ10027070 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 18 Feb 2019 19:50:11 GMT Received: from b03ledav003.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 23B556A04D; Mon, 18 Feb 2019 19:50:11 +0000 (GMT) Received: from b03ledav003.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7060E6A05A; Mon, 18 Feb 2019 19:50:09 +0000 (GMT) Received: from akrowiak-ThinkPad-P50.ibm.com (unknown [9.85.134.111]) by b03ledav003.gho.boulder.ibm.com (Postfix) with ESMTPS; Mon, 18 Feb 2019 19:50:09 +0000 (GMT) From: Tony Krowiak To: qemu-devel@nongnu.org Date: Mon, 18 Feb 2019 14:49:57 -0500 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1550519397-25359-1-git-send-email-akrowiak@linux.ibm.com> References: <1550519397-25359-1-git-send-email-akrowiak@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 19021819-0004-0000-0000-000014E3C801 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00010621; HX=3.00000242; KW=3.00000007; PH=3.00000004; SC=3.00000281; SDB=6.01162968; UDB=6.00607192; IPR=6.00943544; MB=3.00025641; MTD=3.00000008; XFM=3.00000015; UTC=2019-02-18 19:50:16 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19021819-0005-0000-0000-00008AA4D7EB Message-Id: <1550519397-25359-3-git-send-email-akrowiak@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-02-18_14:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=830 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902180147 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v3 2/2] s390x/vfio-ap: document hot plug/unplug of vfio-ap device X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: akrowiak@linux.ibm.com, cohuck@redhat.com, mjrosato@linux.ibm.com, pmorel@linux.ibm.com, david@redhat.com, alifm@linux.ibm.com, alex.williamson@redhat.com, pasic@linux.ibm.com, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, fiuczy@linux.ibm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Let's update the vfio-ap.txt document to include the hot plug/unplug support introduced in this patch set. Signed-off-by: Tony Krowiak --- docs/vfio-ap.txt | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 56 insertions(+), 5 deletions(-) diff --git a/docs/vfio-ap.txt b/docs/vfio-ap.txt index 12339684cd52..8cd060a01e10 100644 --- a/docs/vfio-ap.txt +++ b/docs/vfio-ap.txt @@ -440,8 +440,7 @@ unassign_control_domain 'unassign_domain' file. This may be done multiple times to unassign more than one control domain. -Notes: Hot plug/unplug is not currently supported for mediated AP matrix -devices, so no changes to the AP matrix will be allowed while a guest using +Notes: No changes to the AP matrix will be allowed while a guest using the mediated matrix device is running. Attempts to assign an adapter, domain or control domain will be rejected and an error (EBUSY) returned. @@ -562,6 +561,54 @@ facilities: for guest usage, no AP devices can be made accessible to a guest started without APFT installed. +Hot plug a vfio-ap device into a running guest: +============================================== +Only one vfio-ap device can be attached to the virtual machine's ap-bus, so a +vfio-ap device can be hot plugged if and only if no vfio-ap device is attached +to the bus already, whether via the QEMU command line or a prior hot plug +action. + +To hot plug a vfio-ap device, use the QEMU device_add command: + + (qemu) device_add vfio-ap,sysfsdev="$path-to-mdev" + + Where the '$path-to-mdev' value specifies the absolute path to a mediated + device to which AP resources to be used by the guest have been assigned. + +Note that on Linux guests, the AP devices will be created in the +/sys/bus/ap/devices directory when the AP bus subsequently performs its periodic +scan, so there may be a short delay before the AP devices are accessible on the +guest. + +The command will fail if: + +* A vfio-ap device has already been attached to the virtual machine's ap-bus. + +* The CPU model features for controlling guest access to AP facilities are not + enabled (see 'CPU model features' subsection in the previous section). + +Hot unplug a vfio-ap device from a running guest: +================================================ +A vfio-ap device can be unplugged from a running KVM guest if a vfio-ap device +has been attached to the virtual machine's ap-bus via the QEMU command line +or a prior hot plug action. + +To hot unplug a vfio-ap device, use the QEMU device_del command: + + (qemu) device_del vfio-ap,sysfsdev="$path-to-mdev" + + Where $path-to-mdev is the same as the path specified when the vfio-ap + device was attached to the virtual machine's ap-bus. + +On a Linux guest, the AP devices will be removed from the /sys/bus/ap/devices +directory on the guest when the AP bus subsequently performs its periodic scan, +so there may be a short delay before the AP devices are no longer accessible by +the guest. + +The command will fail if the $path-to-mdev specified on the device_del command +does not match the value specified when the vfio-ap device was attached to +the virtual machine's ap-bus. + Example: Configure AP Matrixes for Three Linux Guests: ===================================================== Let's now provide an example to illustrate how KVM guests may be given @@ -819,7 +866,11 @@ Limitations assigned lest the host be given access to the private data of the AP queue device, such as a private key configured specifically for the guest. -* Dynamically modifying the AP matrix for a running guest (which would amount to - hot(un)plug of AP devices for the guest) is currently not supported +* Dynamically assigning AP resources to or unassigning AP resources from a + mediated matrix device - see 'Configuring an AP matrix for a linux guest' + section above - while a running guest is using it is currently not supported. -* Live guest migration is not supported for guests using AP devices. +* Live guest migration is not supported for guests using AP devices. If a guest + is using AP devices, the vfio-ap device configured for the guest must be + unplugged before migrating the guest (see 'Hot unplug a vfio-ap device from a + running guest' section above.