From patchwork Wed Dec 23 01:20:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Krowiak X-Patchwork-Id: 11987467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBC79C433E0 for ; Wed, 23 Dec 2020 01:21:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AA3EE22287 for ; Wed, 23 Dec 2020 01:21:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727211AbgLWBVI (ORCPT ); Tue, 22 Dec 2020 20:21:08 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:3644 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726514AbgLWBVH (ORCPT ); Tue, 22 Dec 2020 20:21:07 -0500 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0BN13OVN186260; Tue, 22 Dec 2020 20:20:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding; s=pp1; bh=D+EFG/gc3p78ZE8Oc9NsUtkZSWv2wrbRfIoTzSXMAds=; b=oXmZhCspfvG0uyTwNozePbIlXP/eYeeyQrX5veLysnTffU4pL4EXA510xaRgpdEWTaR0 yL8Lcu0TQH+a/X1xoirgn73PO4Vii40Ll15usmWjb0KXL/DUdmdkrnA0+ODqzNPVASid I24kdVovMP3/p6T8bhNYKRR8bDpFXVChuFCuYIc3QLNSkP60bmzUNYP6pp3N6uIIWGPd kprGitm1GTvW06UxNv6pJAlDGhBnF0GoDSrvzaLXhHAi0znS65+r8vu1i/fqIxI4288d aBkfYYhjAhhrDDDdXCce7y4JyiUizbwGtTBTPQ/aP9e2Dq0zVK16LL676pCwAXBzp1mt EQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 35ku7ts4qf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 22 Dec 2020 20:20:24 -0500 Received: from m0098404.ppops.net (m0098404.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 0BN1KEM4088148; Tue, 22 Dec 2020 20:20:24 -0500 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 35ku7ts4q8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 22 Dec 2020 20:20:24 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0BN1DETt000491; Wed, 23 Dec 2020 01:20:23 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma03dal.us.ibm.com with ESMTP id 35k02ev7r0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 23 Dec 2020 01:20:23 +0000 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0BN1KLkC22020606 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Dec 2020 01:20:21 GMT Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 96F7E13604F; Wed, 23 Dec 2020 01:20:21 +0000 (GMT) Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 15424136051; Wed, 23 Dec 2020 01:20:19 +0000 (GMT) Received: from cpe-66-24-58-13.stny.res.rr.com.com (unknown [9.85.193.150]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTP; Wed, 23 Dec 2020 01:20:19 +0000 (GMT) From: Tony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: stable@vger.kernel.org, borntraeger@de.ibm.com, cohuck@redhat.com, kwankhede@nvidia.com, pbonzini@redhat.com, alex.williamson@redhat.com, pasic@linux.vnet.ibm.com, Tony Krowiak , Halil Pasic Subject: [PATCH v5] s390/vfio-ap: clean up vfio_ap resources when KVM pointer invalidated Date: Tue, 22 Dec 2020 20:20:13 -0500 Message-Id: <20201223012013.5418-1-akrowiak@linux.ibm.com> X-Mailer: git-send-email 2.21.1 MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343,18.0.737 definitions=2020-12-22_13:2020-12-21,2020-12-22 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 bulkscore=0 impostorscore=0 priorityscore=1501 mlxscore=0 phishscore=0 spamscore=0 lowpriorityscore=0 suspectscore=0 clxscore=1015 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012230007 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The vfio_ap device driver registers a group notifier with VFIO when the file descriptor for a VFIO mediated device for a KVM guest is opened to receive notification that the KVM pointer is set (VFIO_GROUP_NOTIFY_SET_KVM event). When the KVM pointer is set, the vfio_ap driver takes the following actions: 1. Stashes the KVM pointer in the vfio_ap_mdev struct that holds the state of the mediated device. 2. Calls the kvm_get_kvm() function to increment its reference counter. 3. Sets the function pointer to the function that handles interception of the instruction that enables/disables interrupt processing. 4. Sets the masks in the KVM guest's CRYCB to pass AP resources through to the guest. In order to avoid memory leaks, when the notifier is called to receive notification that the KVM pointer has been set to NULL, the vfio_ap device driver should reverse the actions taken when the KVM pointer was set. Fixes: 258287c994de ("s390: vfio-ap: implement mediated device open callback") Cc: stable@vger.kernel.org Signed-off-by: Tony Krowiak Reviewed-by: Halil Pasic Reviewed-by: Cornelia Huck --- drivers/s390/crypto/vfio_ap_ops.c | 49 ++++++++++++++++++------------- 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index e0bde8518745..7339043906cf 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -1037,19 +1037,14 @@ static int vfio_ap_mdev_set_kvm(struct ap_matrix_mdev *matrix_mdev, { struct ap_matrix_mdev *m; - mutex_lock(&matrix_dev->lock); - list_for_each_entry(m, &matrix_dev->mdev_list, node) { - if ((m != matrix_mdev) && (m->kvm == kvm)) { - mutex_unlock(&matrix_dev->lock); + if ((m != matrix_mdev) && (m->kvm == kvm)) return -EPERM; - } } matrix_mdev->kvm = kvm; kvm_get_kvm(kvm); kvm->arch.crypto.pqap_hook = &matrix_mdev->pqap_hook; - mutex_unlock(&matrix_dev->lock); return 0; } @@ -1083,35 +1078,52 @@ static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb, return NOTIFY_DONE; } +static void vfio_ap_mdev_unset_kvm(struct ap_matrix_mdev *matrix_mdev) +{ + kvm_arch_crypto_clear_masks(matrix_mdev->kvm); + matrix_mdev->kvm->arch.crypto.pqap_hook = NULL; + vfio_ap_mdev_reset_queues(matrix_mdev->mdev); + kvm_put_kvm(matrix_mdev->kvm); + matrix_mdev->kvm = NULL; +} + static int vfio_ap_mdev_group_notifier(struct notifier_block *nb, unsigned long action, void *data) { - int ret; + int ret, notify_rc = NOTIFY_OK; struct ap_matrix_mdev *matrix_mdev; if (action != VFIO_GROUP_NOTIFY_SET_KVM) return NOTIFY_OK; matrix_mdev = container_of(nb, struct ap_matrix_mdev, group_notifier); + mutex_lock(&matrix_dev->lock); if (!data) { - matrix_mdev->kvm = NULL; - return NOTIFY_OK; + if (matrix_mdev->kvm) + vfio_ap_mdev_unset_kvm(matrix_mdev); + goto notify_done; } ret = vfio_ap_mdev_set_kvm(matrix_mdev, data); - if (ret) - return NOTIFY_DONE; + if (ret) { + notify_rc = NOTIFY_DONE; + goto notify_done; + } /* If there is no CRYCB pointer, then we can't copy the masks */ - if (!matrix_mdev->kvm->arch.crypto.crycbd) - return NOTIFY_DONE; + if (!matrix_mdev->kvm->arch.crypto.crycbd) { + notify_rc = NOTIFY_DONE; + goto notify_done; + } kvm_arch_crypto_set_masks(matrix_mdev->kvm, matrix_mdev->matrix.apm, matrix_mdev->matrix.aqm, matrix_mdev->matrix.adm); - return NOTIFY_OK; +notify_done: + mutex_unlock(&matrix_dev->lock); + return notify_rc; } static void vfio_ap_irq_disable_apqn(int apqn) @@ -1222,13 +1234,8 @@ static void vfio_ap_mdev_release(struct mdev_device *mdev) struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); mutex_lock(&matrix_dev->lock); - if (matrix_mdev->kvm) { - kvm_arch_crypto_clear_masks(matrix_mdev->kvm); - matrix_mdev->kvm->arch.crypto.pqap_hook = NULL; - vfio_ap_mdev_reset_queues(mdev); - kvm_put_kvm(matrix_mdev->kvm); - matrix_mdev->kvm = NULL; - } + if (matrix_mdev->kvm) + vfio_ap_mdev_unset_kvm(matrix_mdev); mutex_unlock(&matrix_dev->lock); vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,