From patchwork Wed Jul 31 22:41:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Krowiak X-Patchwork-Id: 11069745 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4F8F746 for ; Wed, 31 Jul 2019 22:42:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A5CBD204FA for ; Wed, 31 Jul 2019 22:42:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 99E3428068; Wed, 31 Jul 2019 22:42:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 06691204FA for ; Wed, 31 Jul 2019 22:42:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729127AbfGaWl2 (ORCPT ); Wed, 31 Jul 2019 18:41:28 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:39022 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727348AbfGaWl2 (ORCPT ); Wed, 31 Jul 2019 18:41:28 -0400 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x6VMX66x003995; Wed, 31 Jul 2019 18:41:26 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2u3j3n434k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Jul 2019 18:41:26 -0400 Received: from m0098394.ppops.net (m0098394.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x6VMXY72005387; Wed, 31 Jul 2019 18:41:26 -0400 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 2u3j3n4348-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Jul 2019 18:41:26 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x6VMenfT025332; Wed, 31 Jul 2019 22:41:25 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma02dal.us.ibm.com with ESMTP id 2u0e85vwrx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Jul 2019 22:41:25 +0000 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x6VMfMXK53215670 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 Jul 2019 22:41:22 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8AEB52805A; Wed, 31 Jul 2019 22:41:22 +0000 (GMT) Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B613D28059; Wed, 31 Jul 2019 22:41:21 +0000 (GMT) Received: from akrowiak-ThinkPad-P50.ibm.com (unknown [9.85.130.145]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTPS; Wed, 31 Jul 2019 22:41:21 +0000 (GMT) From: Tony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: freude@linux.ibm.com, borntraeger@de.ibm.com, cohuck@redhat.com, frankja@linux.ibm.com, david@redhat.com, mjrosato@linux.ibm.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, pmorel@linux.ibm.com, pasic@linux.ibm.com, alex.williamson@redhat.com, kwankhede@nvidia.com, Tony Krowiak Subject: [PATCH v5 1/7] s390: vfio-ap: Refactor vfio_ap driver probe and remove callbacks Date: Wed, 31 Jul 2019 18:41:11 -0400 Message-Id: <1564612877-7598-2-git-send-email-akrowiak@linux.ibm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1564612877-7598-1-git-send-email-akrowiak@linux.ibm.com> References: <1564612877-7598-1-git-send-email-akrowiak@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-07-31_11:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907310225 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In order to limit the number of private mdev functions called from the vfio_ap device driver as well as to provide a landing spot for dynamic configuration code related to binding/unbinding AP queue devices to/from the vfio_ap driver, the following changes are being introduced: * Move code from the vfio_ap driver's probe callback into a function defined in the mdev private operations file. * Move code from the vfio_ap driver's remove callback into a function defined in the mdev private operations file. Signed-off-by: Tony Krowiak --- drivers/s390/crypto/vfio_ap_drv.c | 23 +++-------- drivers/s390/crypto/vfio_ap_ops.c | 78 ++++++++++++++++++++++++++++------- drivers/s390/crypto/vfio_ap_private.h | 6 +-- 3 files changed, 71 insertions(+), 36 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_drv.c b/drivers/s390/crypto/vfio_ap_drv.c index 003662aa8060..9e61d4c6e6b5 100644 --- a/drivers/s390/crypto/vfio_ap_drv.c +++ b/drivers/s390/crypto/vfio_ap_drv.c @@ -49,15 +49,9 @@ MODULE_DEVICE_TABLE(vfio_ap, ap_queue_ids); */ static int vfio_ap_queue_dev_probe(struct ap_device *apdev) { - struct vfio_ap_queue *q; - - q = kzalloc(sizeof(*q), GFP_KERNEL); - if (!q) - return -ENOMEM; - dev_set_drvdata(&apdev->device, q); - q->apqn = to_ap_queue(&apdev->device)->qid; - q->saved_isc = VFIO_AP_ISC_INVALID; - return 0; + struct ap_queue *queue = to_ap_queue(&apdev->device); + + return vfio_ap_mdev_probe_queue(queue); } /** @@ -68,17 +62,10 @@ static int vfio_ap_queue_dev_probe(struct ap_device *apdev) */ static void vfio_ap_queue_dev_remove(struct ap_device *apdev) { - struct vfio_ap_queue *q; - int apid, apqi; + struct ap_queue *queue = to_ap_queue(&apdev->device); mutex_lock(&matrix_dev->lock); - q = dev_get_drvdata(&apdev->device); - dev_set_drvdata(&apdev->device, NULL); - apid = AP_QID_CARD(q->apqn); - apqi = AP_QID_QUEUE(q->apqn); - vfio_ap_mdev_reset_queue(apid, apqi, 1); - vfio_ap_irq_disable(q); - kfree(q); + vfio_ap_mdev_remove_queue(queue); mutex_unlock(&matrix_dev->lock); } diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index 0604b49a4d32..e9a6a7d8395d 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -144,7 +144,7 @@ static void vfio_ap_free_aqic_resources(struct vfio_ap_queue *q) * Returns if ap_aqic function failed with invalid, deconfigured or * checkstopped AP. */ -struct ap_queue_status vfio_ap_irq_disable(struct vfio_ap_queue *q) +static struct ap_queue_status vfio_ap_irq_disable(struct vfio_ap_queue *q) { struct ap_qirq_ctrl aqic_gisa = {}; struct ap_queue_status status; @@ -1128,23 +1128,47 @@ static void vfio_ap_irq_disable_apqn(int apqn) } } -int vfio_ap_mdev_reset_queue(unsigned int apid, unsigned int apqi, - unsigned int retry) +static void vfio_ap_mdev_wait_for_qempty(ap_qid_t qid) +{ + struct ap_queue_status status; + int retry = 5; + + do { + status = ap_tapq(qid, NULL); + switch (status.response_code) { + case AP_RESPONSE_NORMAL: + if (status.queue_empty) + return; + break; + case AP_RESPONSE_RESET_IN_PROGRESS: + case AP_RESPONSE_BUSY: + msleep(20); + break; + default: + pr_warn("%s: tapq response %02x waiting for queue %04x.%02x empty\n", + __func__, status.response_code, + AP_QID_CARD(qid), AP_QID_QUEUE(qid)); + return; + } + } while (--retry); + + WARN_ON_ONCE(retry <= 0); +} + +int vfio_ap_mdev_reset_queue(unsigned int apid, unsigned int apqi) { struct ap_queue_status status; - int retry2 = 2; int apqn = AP_MKQID(apid, apqi); + int retry = 5; do { status = ap_zapq(apqn); switch (status.response_code) { case AP_RESPONSE_NORMAL: - while (!status.queue_empty && retry2--) { - msleep(20); - status = ap_tapq(apqn, NULL); - } - WARN_ON_ONCE(retry <= 0); + vfio_ap_mdev_wait_for_qempty(AP_MKQID(apid, apqi)); return 0; + case AP_RESPONSE_DECONFIGURED: + return -ENODEV; case AP_RESPONSE_RESET_IN_PROGRESS: case AP_RESPONSE_BUSY: msleep(20); @@ -1169,14 +1193,10 @@ static int vfio_ap_mdev_reset_queues(struct mdev_device *mdev) matrix_mdev->matrix.apm_max + 1) { for_each_set_bit_inv(apqi, matrix_mdev->matrix.aqm, matrix_mdev->matrix.aqm_max + 1) { - ret = vfio_ap_mdev_reset_queue(apid, apqi, 1); - /* - * Regardless whether a queue turns out to be busy, or - * is not operational, we need to continue resetting - * the remaining queues. - */ + ret = vfio_ap_mdev_reset_queue(apid, apqi); if (ret) rc = ret; + vfio_ap_irq_disable_apqn(AP_MKQID(apid, apqi)); } } @@ -1302,3 +1322,31 @@ void vfio_ap_mdev_unregister(void) { mdev_unregister_device(&matrix_dev->device); } + +int vfio_ap_mdev_probe_queue(struct ap_queue *queue) +{ + struct vfio_ap_queue *q; + + q = kzalloc(sizeof(*q), GFP_KERNEL); + if (!q) + return -ENOMEM; + dev_set_drvdata(&queue->ap_dev.device, q); + q->apqn = queue->qid; + q->saved_isc = VFIO_AP_ISC_INVALID; + + return 0; +} + +void vfio_ap_mdev_remove_queue(struct ap_queue *queue) +{ + struct vfio_ap_queue *q; + int apid, apqi; + + q = dev_get_drvdata(&queue->ap_dev.device); + dev_set_drvdata(&queue->ap_dev.device, NULL); + apid = AP_QID_CARD(q->apqn); + apqi = AP_QID_QUEUE(q->apqn); + vfio_ap_mdev_reset_queue(apid, apqi); + vfio_ap_irq_disable(q); + kfree(q); +} diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h index f46dde56b464..5cc3c2ebf151 100644 --- a/drivers/s390/crypto/vfio_ap_private.h +++ b/drivers/s390/crypto/vfio_ap_private.h @@ -90,8 +90,6 @@ struct ap_matrix_mdev { extern int vfio_ap_mdev_register(void); extern void vfio_ap_mdev_unregister(void); -int vfio_ap_mdev_reset_queue(unsigned int apid, unsigned int apqi, - unsigned int retry); struct vfio_ap_queue { struct ap_matrix_mdev *matrix_mdev; @@ -100,5 +98,7 @@ struct vfio_ap_queue { #define VFIO_AP_ISC_INVALID 0xff unsigned char saved_isc; }; -struct ap_queue_status vfio_ap_irq_disable(struct vfio_ap_queue *q); +int vfio_ap_mdev_probe_queue(struct ap_queue *queue); +void vfio_ap_mdev_remove_queue(struct ap_queue *queue); + #endif /* _VFIO_AP_PRIVATE_H_ */