From patchwork Wed Aug 31 14:10:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3E488C54EE9 for ; Wed, 31 Aug 2022 14:11:33 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395689.635563 (Exim 4.92) (envelope-from ) id 1oTOR9-0004kv-JV; Wed, 31 Aug 2022 14:11:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395689.635563; Wed, 31 Aug 2022 14:11:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR9-0004kb-7k; Wed, 31 Aug 2022 14:11:23 +0000 Received: by outflank-mailman (input) for mailman id 395689; Wed, 31 Aug 2022 14:11:22 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR7-0002bv-S2 for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:22 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c8dfbfd1-2936-11ed-934f-f50d60e1c1bd; Wed, 31 Aug 2022 16:11:20 +0200 (CEST) Received: from pps.filterd (m0174677.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBo4G8032537; Wed, 31 Aug 2022 14:11:03 GMT Received: from eur04-vi1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3ja427h4rd-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:03 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:10:59 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:10:59 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c8dfbfd1-2936-11ed-934f-f50d60e1c1bd ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kGiWyStxJ96eYQkm55zDqFJw0GRO0iQYLfvQE6OqmcZTWWmKiil0WB5ZIdgG9ubUeOCj9U8/ESnCpVYZm3iseR+75r+E9RoxqqVgJjNSnojNxEWHhXCNS+m5NgWAqbMA15DkFHVlgJ3gVwb802PhQXYL0T5vEMdE6pfiYWTR2kpUhxcu8o0+u2iOlEbGSw8tMNKSZQwYMhAZGy9eY+lOCZoSJrBkIhTjBChB/eAN6NFBRQlbCLklX00+slewnNg24htSvKEAx+Dt+MbiK78nRbf/T0t+Dw8KlXePqzEV1Tb3c6NuN4F7POvGG5bczMECO/Zkh/14TSxwkLliJmJU+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=25h/2px3CnbvvwOgU10bnc4euxiVjUY+0mEFpTjlpHY=; b=W+EQP9Q4xZBTySrx+LKwNp26AU7ZRjgnyW/v6k4phljBqbPVVBNA9HVJLEUWzN6WvxfN9rYOqQbVHoEMyE6fgc0dbfr0dAFrjM4mXum54IQ5rIgz5xI8VsGS7gwnClF3ZijWdKnUYlXaTvVMV++AdxCT7Brp9PbwCw4BIooqscnMBsnf0ucRtBp9VJBgSyXO2akrhs1d9naBFBOnNNKS0h3jOTHPme/u/hYTPjfJLdieFptsEW6+zgLz2d1mXGRLYltXtWwMWMZKXVrTg3hn8TOnZDESmEJFCGhzQ5UTd4Z2SvEPQ9oExArCLpjosW4fwrt97SGbptt2i59GGtevFw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=25h/2px3CnbvvwOgU10bnc4euxiVjUY+0mEFpTjlpHY=; b=XZ022wzN7S6uILbrCuuwT00g3JAKildyTTZ52yq4C0b2oKEvEiZy5F7TWlJOXm4rPnYD2yAiw6UCtvZFUl52picBNi8XHzeZach8jWZk91XyxClBloZSiM+yJzw+Q7yC94VvEiIp+yI2TY+/lOrBxnA/9KOR0bVihZHQn1r9knKDZiiAdn8zR3FMPfZE4Qy84bbBGYjdMAPg+6K38RjrVi9irgIcgGk2sZxAKqYto3BmiFtlUeQlBfcJDkBxQ4DmzOBFSalviB0Zp4WVJiysvf2S8CdyMooPEst6iMhEsV6HOOpbmRm3xeL/4oLyrKBCvaUiar4K81VgfzExKrHQdA== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu , Paul Durrant , =?iso-8859-1?q?Roger_Pau?= =?iso-8859-1?q?_Monn=E9?= , Kevin Tian Subject: [RFC PATCH 01/10] xen: pci: add per-domain pci list lock Thread-Topic: [RFC PATCH 01/10] xen: pci: add per-domain pci list lock Thread-Index: AQHYvUN+sHWq0w//vE+eH46Zd/aTJw== Date: Wed, 31 Aug 2022 14:10:58 +0000 Message-ID: <20220831141040.13231-2-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 11e52c0a-185d-407a-fde0-08da8b5aa0ee x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: O6cVO4Wn0KOBsr+G4queNNIxca/x5e3lDSpNf8YZI/CxvhGzF3Ep5hTJKWp0c+zULKaIiLkfQv7TU8LqZP7XKkkfC3YhNCpxxwdOnpkEc1+XsF/IhkU9iQ15LGJXLOOqqRzEtCEw/n8fL7oy2G3YzoDdW5CS6HQsixUGnX+UyXEpG/4Cv/LQBqVJsho8OUSR/7W1VRwvWVNM6wADuN8kW3zoEAN+zz2N93gEP9pme3rtlYwmF17QHxSZKA43gWboRzXrlN4MCxDv2PCmL9asa5jzFJSn+XIlNF5lYS5Ke8+PPdW7LUv6wPYnb24869Nw6GWP6ekNoD5+kfIvGcu5c+x6LKK0bBTblxDlZ/+x3FjNUmz91OcIobTslJcROI01DsJNAdnIw0wALp6PdQtJxOowkRPwQLYnmw7nt5001DbEyjpx6Ss7zDJxycRnbYvoohi4Im0K0zWKJWogiVqbUD3SXdJAcgfzCG9KFOix+IrSBse9DW6lLl2NHEmKH9hPJm7vkwLIKW9HNlJH9M8Q4KFB/B+OuTFA6pQxDUQIuzn7ALebCcndZGWSINBgvYZ0U/CcgMa/lyZTGpeVjCTwBR38Un0J6/xcF6qzLSzYsXupapq9mP8OZ6shOGnG5jsOGbVaf/5a5kMwiiqICh+kATx8xrvdaMhdFgwc5UjwcExOj9PQx5JBL8l6rPQZMDaINdBDFMysO4B2VEpUfMTPlgSzStnoIoMoECoc8ANbVM2jeoncVXetmLgwzekSm8aK/f59i7Ip9MOeQv0ps5rs7g== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(7416002)(8936002)(316002)(36756003)(6916009)(30864003)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?j8B963qYf7prYz6OAn7DZGH?= =?iso-8859-1?q?oUFazNkkji6ZFWreDgWiJL80elKDkJVUkCHd3FE3Z+slL6SGOlzpqAmKfOa+?= =?iso-8859-1?q?cOCVVBZh0EIIY/5HcN9gmOB1ZTz+bd36rgt2Y0NZJuoNWxDgLDo36sWUz7iz?= =?iso-8859-1?q?pvNo3HTUZl1rBJt9YbvVfzIqfUbmn8qsSe5CNkQ6XHfPpi+nYMCA9rqgG5rE?= =?iso-8859-1?q?UerNdZzI+9CKidz9hxKsAJMPJUkE7R3h3iE/UAS2sYyE/RDg8RC0bo8XWfTT?= =?iso-8859-1?q?Beh7papA4Hkjhs8J+yrZ3AgHT9BvIV6cFVoq6BGlBQHCwjmq9CAw5F2LkYJ9?= =?iso-8859-1?q?MeL+UILtrVcZ2mIS00X5tTfYKsmtT1iPql9GM0EqNea5cuNkqTcriqgN6Y+R?= =?iso-8859-1?q?Shw1pZk0QClEFEdA3oeryMWZDWK3mII18e0r6/1C8fOCri+uDEHxLAgejeXM?= =?iso-8859-1?q?goB+++roemiq6goyuJCkKasAWOYG0f/fLL1PKMADiv4+clAvY1ouruNAELEG?= =?iso-8859-1?q?Ku09DndnwEFM3/6f1qIVusia1SAD1qBRengakT8jqY2dppAgNstrxFa7BpZl?= =?iso-8859-1?q?wkULJ7exqFH2a2jvZ9e70h99nKqmOBDvCSwu3OCRvt33zbGdclNHkOFLpLHp?= =?iso-8859-1?q?ulLWe4cdHm8cdO3XR7vASo9TcKikshUbiXY7pbctzobYJ7OmolHnSebT0cu2?= =?iso-8859-1?q?+5KeG+K9xSj/FW6uNO4+Zbbx/k+nMB2JFkWvu95ZqjjFsISYbsmNJayJ23pc?= =?iso-8859-1?q?sVlEs5Km4sLi/UiB/lrzbpWJQ3tMgHQkqiHQz56LvTX2gdxtjw9eoc9BsVEs?= =?iso-8859-1?q?pSSuuCfkKrkiI64y8vxAahK2fVgc/Yt3TdF3GxIiCCSwCPhX9iIhbKb8p+no?= =?iso-8859-1?q?dFFnLlOroEi7vKWekuPcr33huF0KGzOyrq9RE6cp7ibGF19zZsHWOJ6hoicR?= =?iso-8859-1?q?KB7LSdj2yS+AaRQPgqj0PC4p0HSbTQuMZrGSfGcJ3bwN7nH3uWBlpfELXpS/?= =?iso-8859-1?q?gZOKgmKNwwaavsDR2JRt992pxpiSwe85h1cf/roEbubKVf+oYXjAD8I5oivj?= =?iso-8859-1?q?YIigkuQO5WugV/w54J9SOSHnLe9rgD2Uf/AdhlcM9nXm3KvUvFj8Trv6TM0P?= =?iso-8859-1?q?OFiZYO6MrrmeyHwba+4qVxb5Wo0XNAG2d7RjBG15hFWdliQh0fm5QGuoeWpe?= =?iso-8859-1?q?MQDt5jW1zlB5BdwSzzq1YOwbKbsY0xxIljMavDlYJmaC35mMB2yidYRt3xbT?= =?iso-8859-1?q?wGrHbL2GOGzORZiVIbWN09A8pId2v418yhMUVDWkPXGSFhy/FbH6v8uh0G5P?= =?iso-8859-1?q?gWbLiV+d0CaVFDtrSsKi41FUFBsJgllDSrH+eAzv+z1DqqDNETAuHsKKTouO?= =?iso-8859-1?q?dONB2f41h/S2UxaE1V4YmeIJJxByjoodJzWHOL/d+4KO108YfxPhzElSyWcM?= =?iso-8859-1?q?9rXQblaj2XkyeezvPUHgHi3Fwc7SPkyvLpLJ0P8aTaF2yQc3zBN4ZAZBG6vq?= =?iso-8859-1?q?6z4OcniIbKmZfFdI1URMEsqUlO/pmQDdGOqdczkMsjFMTHwt1+oKlAJnpP0I?= =?iso-8859-1?q?NqqxDcbAskVPu/4GIJSK4otPPps3ax/aaZO6kQJT2eyzqoLRz2qQbhCPZTLf?= =?iso-8859-1?q?RGoQ1MP/TP2tTSyPWHM3gk4WK94Mgaj8+XSwwFA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 11e52c0a-185d-407a-fde0-08da8b5aa0ee X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:10:58.9056 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Fd8Z4dAKIksYfLRLWQ4jPRXzhMktFgigUaWYBlilYF70wg0rxCj6n2GPUDtGB+vF8GJm9LdeMjXjVW9cxilT9I+4pfb7wVsMBTSdOfmTnkM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: yiKzlP3Rj-7UJUd3WzViieXMPjjVUZ3K X-Proofpoint-ORIG-GUID: yiKzlP3Rj-7UJUd3WzViieXMPjjVUZ3K X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=999 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 domain->pdevs_lock protects access to domain->pdev_list. As this, it should be used when we are adding, removing on enumerating PCI devices assigned to a domain. This enables more granular locking instead of one huge pcidevs_lock that locks entire PCI subsystem. Please note that pcidevs_lock() is still used, we are going to remove it in subsequent patches. Signed-off-by: Volodymyr Babchuk --- xen/common/domain.c | 1 + xen/drivers/passthrough/amd/iommu_cmd.c | 4 ++- xen/drivers/passthrough/amd/pci_amd_iommu.c | 7 ++++- xen/drivers/passthrough/pci.c | 29 ++++++++++++++++++++- xen/drivers/passthrough/vtd/iommu.c | 9 +++++-- xen/drivers/vpci/header.c | 3 +++ xen/drivers/vpci/msi.c | 7 ++++- xen/drivers/vpci/vpci.c | 4 +-- xen/include/xen/pci.h | 2 +- xen/include/xen/sched.h | 1 + 10 files changed, 58 insertions(+), 9 deletions(-) diff --git a/xen/common/domain.c b/xen/common/domain.c index 7062393e37..4611141b87 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -618,6 +618,7 @@ struct domain *domain_create(domid_t domid, #ifdef CONFIG_HAS_PCI INIT_LIST_HEAD(&d->pdev_list); + spin_lock_init(&d->pdevs_lock); #endif /* All error paths can depend on the above setup. */ diff --git a/xen/drivers/passthrough/amd/iommu_cmd.c b/xen/drivers/passthrough/amd/iommu_cmd.c index 40ddf366bb..47c45398d4 100644 --- a/xen/drivers/passthrough/amd/iommu_cmd.c +++ b/xen/drivers/passthrough/amd/iommu_cmd.c @@ -308,11 +308,12 @@ void amd_iommu_flush_iotlb(u8 devfn, const struct pci_dev *pdev, flush_command_buffer(iommu, iommu_dev_iotlb_timeout); } -static void amd_iommu_flush_all_iotlbs(const struct domain *d, daddr_t daddr, +static void amd_iommu_flush_all_iotlbs(struct domain *d, daddr_t daddr, unsigned int order) { struct pci_dev *pdev; + spin_lock(&d->pdevs_lock); for_each_pdev( d, pdev ) { u8 devfn = pdev->devfn; @@ -323,6 +324,7 @@ static void amd_iommu_flush_all_iotlbs(const struct domain *d, daddr_t daddr, } while ( devfn != pdev->devfn && PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) ); } + spin_unlock(&d->pdevs_lock); } /* Flush iommu cache after p2m changes. */ diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c index 4ba8e764b2..64c016491d 100644 --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -96,20 +96,25 @@ static int __must_check allocate_domain_resources(struct domain *d) return rc; } -static bool any_pdev_behind_iommu(const struct domain *d, +static bool any_pdev_behind_iommu(struct domain *d, const struct pci_dev *exclude, const struct amd_iommu *iommu) { const struct pci_dev *pdev; + spin_lock(&d->pdevs_lock); for_each_pdev ( d, pdev ) { if ( pdev == exclude ) continue; if ( find_iommu_for_device(pdev->seg, pdev->sbdf.bdf) == iommu ) + { + spin_unlock(&d->pdevs_lock); return true; + } } + spin_unlock(&d->pdevs_lock); return false; } diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index cdaf5c247f..4366f8f965 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -523,7 +523,9 @@ static void __init _pci_hide_device(struct pci_dev *pdev) if ( pdev->domain ) return; pdev->domain = dom_xen; + spin_lock(&dom_xen->pdevs_lock); list_add(&pdev->domain_list, &dom_xen->pdev_list); + spin_unlock(&dom_xen->pdevs_lock); } int __init pci_hide_device(unsigned int seg, unsigned int bus, @@ -595,7 +597,7 @@ struct pci_dev *pci_get_real_pdev(pci_sbdf_t sbdf) return pdev; } -struct pci_dev *pci_get_pdev(const struct domain *d, pci_sbdf_t sbdf) +struct pci_dev *pci_get_pdev(struct domain *d, pci_sbdf_t sbdf) { struct pci_dev *pdev; @@ -620,9 +622,16 @@ struct pci_dev *pci_get_pdev(const struct domain *d, pci_sbdf_t sbdf) return pdev; } else + { + spin_lock(&d->pdevs_lock); list_for_each_entry ( pdev, &d->pdev_list, domain_list ) if ( pdev->sbdf.bdf == sbdf.bdf ) + { + spin_unlock(&d->pdevs_lock); return pdev; + } + spin_unlock(&d->pdevs_lock); + } return NULL; } @@ -817,7 +826,9 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, if ( !pdev->domain ) { pdev->domain = hardware_domain; + spin_lock(&hardware_domain->pdevs_lock); list_add(&pdev->domain_list, &hardware_domain->pdev_list); + spin_unlock(&hardware_domain->pdevs_lock); /* * For devices not discovered by Xen during boot, add vPCI handlers @@ -827,7 +838,9 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, if ( ret ) { printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret); + spin_lock(&pdev->domain->pdevs_lock); list_del(&pdev->domain_list); + spin_unlock(&pdev->domain->pdevs_lock); pdev->domain = NULL; goto out; } @@ -835,7 +848,9 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, if ( ret ) { vpci_remove_device(pdev); + spin_lock(&pdev->domain->pdevs_lock); list_del(&pdev->domain_list); + spin_unlock(&pdev->domain->pdevs_lock); pdev->domain = NULL; goto out; } @@ -885,7 +900,11 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) pci_cleanup_msi(pdev); ret = iommu_remove_device(pdev); if ( pdev->domain ) + { + spin_lock(&pdev->domain->pdevs_lock); list_del(&pdev->domain_list); + spin_unlock(&pdev->domain->pdevs_lock); + } printk(XENLOG_DEBUG "PCI remove device %pp\n", &pdev->sbdf); free_pdev(pseg, pdev); break; @@ -967,12 +986,14 @@ int pci_release_devices(struct domain *d) pcidevs_unlock(); return ret; } + spin_lock(&d->pdevs_lock); list_for_each_entry_safe ( pdev, tmp, &d->pdev_list, domain_list ) { bus = pdev->bus; devfn = pdev->devfn; ret = deassign_device(d, pdev->seg, bus, devfn) ?: ret; } + spin_unlock(&d->pdevs_lock); pcidevs_unlock(); return ret; @@ -1194,7 +1215,9 @@ static int __hwdom_init cf_check _setup_hwdom_pci_devices( if ( !pdev->domain ) { pdev->domain = ctxt->d; + spin_lock(&pdev->domain->pdevs_lock); list_add(&pdev->domain_list, &ctxt->d->pdev_list); + spin_unlock(&pdev->domain->pdevs_lock); setup_one_hwdom_device(ctxt, pdev); } else if ( pdev->domain == dom_xen ) @@ -1556,6 +1579,7 @@ static int iommu_get_device_group( return group_id; pcidevs_lock(); + spin_lock(&d->pdevs_lock); for_each_pdev( d, pdev ) { unsigned int b = pdev->bus; @@ -1571,6 +1595,7 @@ static int iommu_get_device_group( if ( sdev_id < 0 ) { pcidevs_unlock(); + spin_unlock(&d->pdevs_lock); return sdev_id; } @@ -1581,6 +1606,7 @@ static int iommu_get_device_group( if ( unlikely(copy_to_guest_offset(buf, i, &bdf, 1)) ) { pcidevs_unlock(); + spin_unlock(&d->pdevs_lock); return -EFAULT; } i++; @@ -1588,6 +1614,7 @@ static int iommu_get_device_group( } pcidevs_unlock(); + spin_unlock(&d->pdevs_lock); return i; } diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index 62e143125d..fff1442265 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -183,12 +183,13 @@ static void cleanup_domid_map(domid_t domid, struct vtd_iommu *iommu) } } -static bool any_pdev_behind_iommu(const struct domain *d, +static bool any_pdev_behind_iommu(struct domain *d, const struct pci_dev *exclude, const struct vtd_iommu *iommu) { const struct pci_dev *pdev; + spin_lock(&d->pdevs_lock); for_each_pdev ( d, pdev ) { const struct acpi_drhd_unit *drhd; @@ -198,8 +199,12 @@ static bool any_pdev_behind_iommu(const struct domain *d, drhd = acpi_find_matched_drhd_unit(pdev); if ( drhd && drhd->iommu == iommu ) + { + spin_unlock(&d->pdevs_lock); return true; + } } + spin_unlock(&d->pdevs_lock); return false; } @@ -208,7 +213,7 @@ static bool any_pdev_behind_iommu(const struct domain *d, * If no other devices under the same iommu owned by this domain, * clear iommu in iommu_bitmap and clear domain_id in domid_bitmap. */ -static void check_cleanup_domid_map(const struct domain *d, +static void check_cleanup_domid_map(struct domain *d, const struct pci_dev *exclude, struct vtd_iommu *iommu) { diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index a1c928a0d2..a59aa7ad0b 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -267,6 +267,7 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) * Check for overlaps with other BARs. Note that only BARs that are * currently mapped (enabled) are checked for overlaps. */ + spin_lock(&pdev->domain->pdevs_lock); for_each_pdev ( pdev->domain, tmp ) { if ( tmp == pdev ) @@ -306,11 +307,13 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) printk(XENLOG_G_WARNING "Failed to remove [%lx, %lx]: %d\n", start, end, rc); rangeset_destroy(mem); + spin_unlock( &pdev->domain->pdevs_lock); return rc; } } } + spin_unlock( &pdev->domain->pdevs_lock); ASSERT(dev); if ( system_state < SYS_STATE_active ) diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c index 8f2b59e61a..8969c335b0 100644 --- a/xen/drivers/vpci/msi.c +++ b/xen/drivers/vpci/msi.c @@ -265,7 +265,7 @@ REGISTER_VPCI_INIT(init_msi, VPCI_PRIORITY_LOW); void vpci_dump_msi(void) { - const struct domain *d; + struct domain *d; rcu_read_lock(&domlist_read_lock); for_each_domain ( d ) @@ -277,6 +277,9 @@ void vpci_dump_msi(void) printk("vPCI MSI/MSI-X d%d\n", d->domain_id); + if ( !spin_trylock(&d->pdevs_lock) ) + continue; + for_each_pdev ( d, pdev ) { const struct vpci_msi *msi; @@ -326,6 +329,8 @@ void vpci_dump_msi(void) spin_unlock(&pdev->vpci->lock); process_pending_softirqs(); } + spin_unlock(&d->pdevs_lock); + } rcu_read_unlock(&domlist_read_lock); } diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 3467c0de86..7d1f9fd438 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -312,7 +312,7 @@ static uint32_t merge_result(uint32_t data, uint32_t new, unsigned int size, uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) { - const struct domain *d = current->domain; + struct domain *d = current->domain; const struct pci_dev *pdev; const struct vpci_register *r; unsigned int data_offset = 0; @@ -415,7 +415,7 @@ static void vpci_write_helper(const struct pci_dev *pdev, void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, uint32_t data) { - const struct domain *d = current->domain; + struct domain *d = current->domain; const struct pci_dev *pdev; const struct vpci_register *r; unsigned int data_offset = 0; diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h index 5975ca2f30..19047b4b20 100644 --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -177,7 +177,7 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, int pci_remove_device(u16 seg, u8 bus, u8 devfn); int pci_ro_device(int seg, int bus, int devfn); int pci_hide_device(unsigned int seg, unsigned int bus, unsigned int devfn); -struct pci_dev *pci_get_pdev(const struct domain *d, pci_sbdf_t sbdf); +struct pci_dev *pci_get_pdev(struct domain *d, pci_sbdf_t sbdf); struct pci_dev *pci_get_real_pdev(pci_sbdf_t sbdf); void pci_check_disable_device(u16 seg, u8 bus, u8 devfn); diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 1cf629e7ec..0775228ba9 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -457,6 +457,7 @@ struct domain #ifdef CONFIG_HAS_PCI struct list_head pdev_list; + spinlock_t pdevs_lock; #endif #ifdef CONFIG_HAS_PASSTHROUGH From patchwork Wed Aug 31 14:10:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 30B7DC67868 for ; Wed, 31 Aug 2022 14:11:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395683.635488 (Exim 4.92) (envelope-from ) id 1oTOR0-0002jf-TE; Wed, 31 Aug 2022 14:11:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395683.635488; Wed, 31 Aug 2022 14:11:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR0-0002j4-OD; Wed, 31 Aug 2022 14:11:14 +0000 Received: by outflank-mailman (input) for mailman id 395683; Wed, 31 Aug 2022 14:11:13 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOQz-0002bw-Hi for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:13 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c3b13193-2936-11ed-82f2-63bd783d45fa; Wed, 31 Aug 2022 16:11:11 +0200 (CEST) Received: from pps.filterd (m0174677.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBo4G9032537; Wed, 31 Aug 2022 14:11:04 GMT Received: from eur04-vi1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3ja427h4rd-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:04 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:10:59 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:10:59 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c3b13193-2936-11ed-82f2-63bd783d45fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VMYw3IIznt/gx2OITmQtAtPWVRAdhO0eSyiP3+uOfdNpk+/mHKGDQ/uXBH8uOn0sYl8urTCnyteFeszqzy2XW4prhRgWADebOThU57JnInEmTKUvMJnN9ptyaGt9lSS4moxisIombYJ1hMsF4gD30W1ZvO2PvW9XRGV+Bh9jL5G7lNp5rCA6TuXy0mPzTbxt+8CBhjm/m2NEV9PjV0PTFOjWnFBk3JErEVnEmzMo7bXSO6wF+uoy2fFu8xojSNEiTFc1d1CAx/Lpnt3xKSg/7/X7+UEIOGMQX4Wodh8MAVxqiGOjsineioHUrxWipADP2P2garJpODiUJ+ZsnXXvmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tlEJMtsRyS/9QiiQY8Z+TlGkWcyZ8F0YLDRgonTMQAk=; b=c81phSZI+3ZTniIBwxkEJIBQV8VTrFyqHLvRElANZhcCg8Tcj3ulnFyyXqzebRMBY+9RQWPrsA2CcVajaAfjYJe6grFPF/s+a1JcLvSIpyq8Lt7WF5nl/oYPb7OGQJ+EwnexPENPHfBP+f7Rnonnd49JJpkCLsS+aPsGv4ebFjfbZRiwhxDfmBhjCsbY/65nWfBsYKLHf8oVgo0VEYlI6IVnKKQ5J4ByTGu93XQZVLpxBsxvfqhs09NoXTnmlVfKNHce/S1nThJ/IBu8ce3vCNb5Sj2qwaosleWKsmCkDHs9Sl1189L+6HKSlhTk5uRG8RG9SLVNJU1z0N+RhRbfhQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tlEJMtsRyS/9QiiQY8Z+TlGkWcyZ8F0YLDRgonTMQAk=; b=MGe/X604Ln4dwQKTc0y31FjSv/L5dOOIBq8/YXZUYNROiYke+RGtKYV458BsJ0FEfvEsKHwZTo5QW7vHXl9JhHEHAMVkSfsauxfRHHNH4BU1EM/8G10w77eSkVjHNPqz1JXSpRioVm2FRID8Wpy0quCYOnMhDJ0M66DGHggjA/B017+760RJct754rkZElrjr0kMx5dU9PcX0p49nkYE79tfMID+5LEGN/RIthGRmOh3LHz9p6OPGFcDEEHj8IV8cApqWOFz6knmw2uqJFpkRX6J+ozj/01MbKTu3i8/Ex45aeE2uP5I0hDA74U/5G2UL2k71Hj6VsmQ/czKYCrCLQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Jan Beulich , Paul Durrant , =?iso-8859-1?q?Roger_Pau_Monn=E9?= Subject: [RFC PATCH 02/10] xen: pci: add pci_seg->alldevs_lock Thread-Topic: [RFC PATCH 02/10] xen: pci: add pci_seg->alldevs_lock Thread-Index: AQHYvUN+nWaGteAQq0y+sD+4Hw5CPQ== Date: Wed, 31 Aug 2022 14:10:59 +0000 Message-ID: <20220831141040.13231-3-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8c1a3ce0-2dfb-4acd-22e3-08da8b5aa115 x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: wQE/F5Zthkaam/Hk//iqdxkAUlZ34owr0ijOV276cWROiKEqHjHsdxJCEC4QQLeFJVnPUPS1SOk4+llxKXucH76Rn2vWXkxDVlabDfnW0SkBgAOFGd1K3g2ogjxMMfTkUIw0xnG9RcyvN+gtgNvl8fTozXVYjAzk05spzzxN/HGkqC50b3AZ/5cvqdtedvk2W4OAMjrCo+cPVJU2HW1b7K70PfMW6eaRxAnc0lVV4j5Cu+t+PITrRhsl0KgTx2B449weg5Gjq0QkzBmsmh6+7YgE575uVYRbVBfe4HscS0zxDwKdw8Myfi6IEyhz0hACTbFICeqh4Dwn63GhsbSvq0ntw4xxWHYeV3r6rmMjbSJxwI5LCUyWA3TWlflHaeW8IEDLb+QW3lW64SzFmXbkzAORJFz3ICH7JzXQpgLXbo4Qb1E86Ki0nSrIb5yZHImI1Q/fgzRDho5mx2tp259L+ZfI/iFw/w4fecPlQPZRWf7ktWTBXjxyigLEVwnJUPq81P8knfOmwb1SaxylZKd+Bjr4jx9n/ifHWXT4SHdqdeqX08cCC032tMTZACGKXFYxIGF2LMrzixED31zJR+thgWTZLF0FvyQ+JGpIMJsvBYgn5HKMs6EG0MIGGTdHeXL8PjEUQwxbN+B4YGkpEDn9wzFca4vFmBn9Ii7qddfjEjKuCnNjSHRbU0ygQn6hF1XSaabGKtzDcjFXQjlyEbpQX/tT85DLOfEtvPErC5iTdA8Go1YHx3vX+bFH75FCAkew+zeNYe2HRNON/r6NfBVy3g== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(8936002)(316002)(36756003)(6916009)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?L0PxtJCO8By9b+awW+CDmH0?= =?iso-8859-1?q?pH+KUd8cce+QZDSHXqFLk2lqg0lXU5hXi22AimbTpTvHF0gRhheD1U4jHpke?= =?iso-8859-1?q?Ao8zf2uqKYJBryf8lrvqyi30Izi5dGDv04QtuHzDfkk5pSSx5lxbVmEbF3OV?= =?iso-8859-1?q?QrkHjDpq+VZKSmb9eKbpS571S/VVd0pDYeqcx2EPgQ3sibNQOb43/nnfNwmh?= =?iso-8859-1?q?36LJuSICWjKWH6hKbuxgz2usse859y2MqDY3NssxEFDzDXkHLxK6kkvtHBY0?= =?iso-8859-1?q?twaRReFr6/b9gW3rkMEziX9YsqQMHBX+hwFXKoa3HO6sK1ne7ZMa8bpHPxtK?= =?iso-8859-1?q?PTkJ+jy8vfC9cBU6fNIXPGG8DbfVzwE40/10Z8MuMeNI+pxN6r4vdiNU2uTs?= =?iso-8859-1?q?5XUdLFDIQaTIXHYYtH+AeiQf/aLD9/KVqTCmO5xP/RQFFjpFbcX4b0PoJM51?= =?iso-8859-1?q?ojpvacR3E6itg546bWO2GBgaoR0UXDNZGcLTw15Ulv5ML0AXpbHMRlognmON?= =?iso-8859-1?q?VFwzt5DoDDk821eI3FEnBv+7hM6/xnh0n1ICe+0SPet8jiR9DDAtXZlMoEFq?= =?iso-8859-1?q?ba1ysc1tuQgJpKn/wBct4PAPFIPxnQjYsGIHGjf7tI+UTktzLVvEnOX/IV9C?= =?iso-8859-1?q?Ljrwzz4oA/HOO+hB6ZsaAlbMxw9wTPopV3nb5CJz+5Qbkq4A5Waz4L65wSKy?= =?iso-8859-1?q?iYaeSIEtV5MGDgrEXEYt+a85J40KeyYB9Gg9QaDvDoBAmpvUn+OZLTn7sb3L?= =?iso-8859-1?q?GrFJca39ij2Iwn8cIErz+r1V8KJNl8W2eNqOwz+dsTb1K3gDA6OGu7JXypPz?= =?iso-8859-1?q?r+MuTEJdCWs88C+5AB9pf7twqSLwaS8LC/G6p9r7qjoK3U3Q1tOEbutMxrX6?= =?iso-8859-1?q?mE3gkaeWj3nce2SU+D2ZDYe6tTgO9b39gc+w8tFAtWTTucLrjhvy8QpOgJpP?= =?iso-8859-1?q?5tGBesJh5eMoyGmt3wW1BL5WsO5iQxBR5+6sfVLKJFI7UpewAnx7xr3mMK3M?= =?iso-8859-1?q?rhCnzxd0Il+qjoA9yKNMyjhwnhAkC5+FYYYxjYT008RtBiKCiWlhEPLeuXtK?= =?iso-8859-1?q?Y+eCcBPS1bpI9qG8FVKciFM9c5cbbGfW5s7LVQid4f9X54j7CFkNJeqAoGXh?= =?iso-8859-1?q?+/+l8Mu14wMUG+8Ftv+fqNMLAn+YEfWLuhhMYpQSzBnuo7JGLQ6ULr175+KM?= =?iso-8859-1?q?TyC+CdFF0qqdea+0GU2NrosFuKTnMvugITLpyHS2lDaulqFrP4n8r6z/GHfQ?= =?iso-8859-1?q?1Fei+thxTceOu9HDMocUU8iwGc9IZEaIuX/q3I4Y9DlTnjrlcJSRf3iEDwQ5?= =?iso-8859-1?q?aa4WWzzrhvbizTOsWws5MF32IMozu9fNDfC9wg2SjNEtKTuoLOAX3dVmYVhV?= =?iso-8859-1?q?PXZd65Z7Hp5fkna3uFP1IBgjKTJKVbcYdtB2MUGxlLMjBCM4yFmuqtg0iXVe?= =?iso-8859-1?q?hf59xTRMZWxN5MVJ8F0cJ0X64DLseggPSSZr3cUK8wXvc0N9ZlIvs1HUcZx2?= =?iso-8859-1?q?P1ydyhRQtb8SPMLRu8cBBywNCnMDahh/onV90oUojtftk/NChtQ1EICVU8IS?= =?iso-8859-1?q?VOla/onJJzzhe5uUXa+qeB2+76Y9Zq6I1dMmebd2WlVHdGI0cQc41lBAJxTs?= =?iso-8859-1?q?NyHekEZxFsO0VZLQkB4gIEOg2NFi1XL5S80fsGw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8c1a3ce0-2dfb-4acd-22e3-08da8b5aa115 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:10:59.1712 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: h3GxOaaNhxJ+5/TPxqV4erIKWpyv1ASaMGEMizrHv7YDgF4OvR2JVXiBBmnZFzCuKtstE8X9o83LnQy2liVyCeCs4Z4cToXuEz+V7K1c+Bs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: bptCeS2jufyr60FKTsavamPbe-yL51DF X-Proofpoint-ORIG-GUID: bptCeS2jufyr60FKTsavamPbe-yL51DF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=999 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 This lock protects alldevs_list of struct pci_seg. As this, it should be used when we are adding, removing on enumerating PCI devices assigned to a PCI segment. Radix tree that stores PCI segment has own locking mechanism, also pci_seg structures are only allocated and newer freed, so we need no additional locking to access pci_seg structures. But we need a lock that protects alldevs_list field. This enables more granular locking instead of one huge pcidevs_lock that locks entire PCI subsystem. Please note that pcidevs_lock() is still used, we are going to remove it in subsequent patches. Signed-off-by: Volodymyr Babchuk --- xen/drivers/passthrough/pci.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 4366f8f965..2dfa1c2875 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -38,6 +38,7 @@ struct pci_seg { struct list_head alldevs_list; + spinlock_t alldevs_lock; u16 nr; unsigned long *ro_map; /* bus2bridge_lock protects bus2bridge array */ @@ -93,6 +94,7 @@ static struct pci_seg *alloc_pseg(u16 seg) pseg->nr = seg; INIT_LIST_HEAD(&pseg->alldevs_list); spin_lock_init(&pseg->bus2bridge_lock); + spin_lock_init(&pseg->alldevs_lock); if ( radix_tree_insert(&pci_segments, seg, pseg) ) { @@ -385,9 +387,13 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn) unsigned int pos; int rc; + spin_lock(&pseg->alldevs_lock); list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) if ( pdev->bus == bus && pdev->devfn == devfn ) + { + spin_unlock(&pseg->alldevs_lock); return pdev; + } pdev = xzalloc(struct pci_dev); if ( !pdev ) @@ -404,10 +410,12 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn) if ( rc ) { xfree(pdev); + spin_unlock(&pseg->alldevs_lock); return NULL; } list_add(&pdev->alldevs_list, &pseg->alldevs_list); + spin_unlock(&pseg->alldevs_lock); /* update bus2bridge */ switch ( pdev->type = pdev_type(pseg->nr, bus, devfn) ) @@ -611,15 +619,20 @@ struct pci_dev *pci_get_pdev(struct domain *d, pci_sbdf_t sbdf) */ if ( !d || is_hardware_domain(d) ) { - const struct pci_seg *pseg = get_pseg(sbdf.seg); + struct pci_seg *pseg = get_pseg(sbdf.seg); if ( !pseg ) return NULL; + spin_lock(&pseg->alldevs_lock); list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) if ( pdev->sbdf.bdf == sbdf.bdf && (!d || pdev->domain == d) ) + { + spin_unlock(&pseg->alldevs_lock); return pdev; + } + spin_unlock(&pseg->alldevs_lock); } else { @@ -893,6 +906,7 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) return -ENODEV; pcidevs_lock(); + spin_lock(&pseg->alldevs_lock); list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) if ( pdev->bus == bus && pdev->devfn == devfn ) { @@ -907,10 +921,12 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) } printk(XENLOG_DEBUG "PCI remove device %pp\n", &pdev->sbdf); free_pdev(pseg, pdev); + list_del(&pdev->alldevs_list); break; } pcidevs_unlock(); + spin_unlock(&pseg->alldevs_lock); return ret; } @@ -1363,6 +1379,7 @@ static int cf_check _dump_pci_devices(struct pci_seg *pseg, void *arg) printk("==== segment %04x ====\n", pseg->nr); + spin_lock(&pseg->alldevs_lock); list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) { printk("%pp - ", &pdev->sbdf); @@ -1376,6 +1393,7 @@ static int cf_check _dump_pci_devices(struct pci_seg *pseg, void *arg) pdev_dump_msi(pdev); printk("\n"); } + spin_unlock(&pseg->alldevs_lock); return 0; } From patchwork Wed Aug 31 14:10:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960754 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D0C76ECAAD1 for ; Wed, 31 Aug 2022 14:11:32 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395684.635511 (Exim 4.92) (envelope-from ) id 1oTOR2-0003Mm-Ad; Wed, 31 Aug 2022 14:11:16 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395684.635511; Wed, 31 Aug 2022 14:11:16 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR2-0003Ls-79; Wed, 31 Aug 2022 14:11:16 +0000 Received: by outflank-mailman (input) for mailman id 395684; Wed, 31 Aug 2022 14:11:14 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR0-0002bv-22 for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:14 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c45f90ea-2936-11ed-934f-f50d60e1c1bd; Wed, 31 Aug 2022 16:11:12 +0200 (CEST) Received: from pps.filterd (m0174676.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBpHD9027047; Wed, 31 Aug 2022 14:11:06 GMT Received: from eur03-ve1-obe.outbound.protection.outlook.com (mail-ve1eur03lp2050.outbound.protection.outlook.com [104.47.9.50]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3j9huk4eny-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:06 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:10:59 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:10:59 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c45f90ea-2936-11ed-934f-f50d60e1c1bd ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bdBMyeS2bf4dH2whMLT+e6ha5bob7QDdy73srMvsV96EzFJY7BPQu3nmJHxRlu/KbyfMn0usSNSG5J0labn/nn1QvZiRLi1ZtaQCQvGE2ra7A0ASI+uoWiH90Fpz3D3lB0TTzxXbQlYBHSH2bq4z4WoEB3YslP9/SANgUB/U2i8xZh18g7PYX9o3VNIRFBbidYUeR2BZHrLcmhsYhvAU2GGahmMNCS+Nkd6QiPS1VhzfuKBYu7Vso+ga1reWlYaX00wJ6B9dXBbGv8nwmULaO6ulEhGYil02K1uCnmD+ISm1+CwUtjlQoHJbY9aTLWGHpLgO34eKr/tShh29m8FLIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HNHHS4bFB++CGi9rIFiohdrogbiukdzlPrd3LW3sk80=; b=nJaFxHWaZKU7a6IZsKviC84uLMAx8hm5IHwF8IKuoTDBpMYn2+43eqmtJtQ1KzHzqGrNt6DoP7FndkTS3brv4GlvH94Csbj0qH9O7QQ1cfauVuAvJrzMtqGZhnCmkZ2ZvjyXiwTmUD4Txtyudp2snEibnLmO65QupJ1g0avu2mHvHICEsNzVzNvx4SbzC2kx+0WB9eMc2i0u0a3+CEFxCho17qOBWul7eZNFbDVx0j4UX8LaD8eUodr4nxXUEce7wdE2eZv+NdvMcc/5qpswt3tuoh6DnmAuId1KhqvNZ2327woPgG/5b3Trl7TSRTKOUdpTsB6Z0Y3jGamNTlI3Cg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HNHHS4bFB++CGi9rIFiohdrogbiukdzlPrd3LW3sk80=; b=l465qaLkIisF9FEXx3dQZMv8eaar7UBwty0sr4KZKF0Ldab0x0/J9l2qGqLRxjPJIu1CUNA3Xum+HmOdfHb7gsPoonZ35vMO9dcKARf0Np74xnfdGShwQSwbMl0XJ0slFhGxDyyQLiIJLge/c9V/BbISUjmReTORMNOa4XgTqZSLP10dTs/oCqFc16xWTnVAm6J8gDGwLJcAEC0uC6HIne9v5/h9Ne3GNS047mrTHLcC6nI2UwrfxCEo9OuK3zoSrNOfAFrUJpJ5KnqXd9+kdD0QsOk8WfjfBo9S+LvTELkvUDU4C9PMpmQiecJs0C69yGSaOOZi6o5Yj7ZHq+wZ0A== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Jan Beulich , Andrew Cooper , Paul Durrant , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Kevin Tian Subject: [RFC PATCH 03/10] xen: pci: introduce ats_list_lock Thread-Topic: [RFC PATCH 03/10] xen: pci: introduce ats_list_lock Thread-Index: AQHYvUN+A/bxwqt+90CvUR3e9vnmcA== Date: Wed, 31 Aug 2022 14:10:59 +0000 Message-ID: <20220831141040.13231-4-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4470ba1a-1e34-4fc9-d11d-08da8b5aa144 x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: QrEARB3ZrAuRvKT4Hde50oCdoWy9MwJlcb2SEbvL/yTmub1XIoSCNkYC0jgibufZQlVgbHqei+cL+VWCOeJEmrsIJFqv21+HCrtWODfMEz8bFZWPmwLb8OHrOeFMLhNDPPq14wZMRiJ2PKGxHdwd10NmTS0kTRn0DZ584Po9hM1XHp+nqOrYmGQNghG5qgAZf+Q2WX6bUR6E+/k14RTwcJxsuH2pdylCadYDMXszVrc79IJAdSlgoGeqdFkormGl0rX30/fyxKulMkjsA2Aam6dgdSHMms4iKynb+mdZJXt/vKcqaerueg+8RvC2GYUKpTFzOv1Cce+YEk+aKp/hYp3JUF2PR1dfuIYUu8bXmfW7+t1/zCRBQs7bBeg+hRDNuYfcvzBimO2IkT8SIPvIP3jPw0s2BKSF4FyVCXmAZqjgry33JKIeGhzPN0dacrlJSvT8fNWCiTAuoUM6PGiEWi7qFXqzRZx/HCEq9ogPTnbGxH0IVdSTpBraQjiE+x5s8oeXY+onBrZsx+4rvhSTFfex9UigqbGvoiEFi8uVuZ9levsESpsnRPn2tBJkwtit0qP4ZOy1OVlEFJmfD58BQzr+Bc/1GEI7dVCHcDoXRAUjJUtc8AQjH0nrvc/E2apL8YVfVhPwbIZYdNZ8UuOF5HaYQnVp55zANexEmrcoF9jMlKtFPfKwPrQ9z972nCXF991txtcZWLDkW/wDo+fl8SWvjpGH+bv0bu+EdUJuTB8oznQggeRKAQ3ZKWAQEwR7Hfxz0phQMcMdGF8fUcOpng== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(8936002)(316002)(36756003)(6916009)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?0o33FUy0zVvgwCVvxJmkpR6?= =?iso-8859-1?q?rD/i4n0Tc5fXmR3kTQE/H7adxHwzTuifncaKJ8f0YLDTNNEhrc4J7TT6IbHz?= =?iso-8859-1?q?PwCURRoNVqyJfJwhqvp1+RtG6sZGbtM0ju6a+636G2eTiWNtWZOfjVURmp2g?= =?iso-8859-1?q?o/gAjsGfD/b7yg41g4dME6ZvTGEwjSNLPS2mHhPYUvNP6FQhKHyEp9rzMZFs?= =?iso-8859-1?q?Yli6a9VMZM2jjG+GSV62eWpAbbhq0EtV7gtxgde22yBHtjIuXXml2OimGzz3?= =?iso-8859-1?q?Fz59s+5h0+leSlOcx1CJWgt8IiRyMg39Zjz6IFvcyy9lnEiTBQGQcVLMdN3g?= =?iso-8859-1?q?r9xiU0fe7vnl1SC4ol/yj32oGsuN97ThSyMq5FrHV9gZh5MSQnSeHOUAPiXl?= =?iso-8859-1?q?ZQgH/et7BIUi8eikTJLMw8xOkA8E2r3RmDcXLJOSvK/6iiL5eVCXGmOqNJok?= =?iso-8859-1?q?z/iqrtV1GZVHqXhJFtxL9dRSd3ZMcHvaywyOvzUvjV34uG+FBt4jhx4vFWwQ?= =?iso-8859-1?q?hw6NWuptEyPdHpNtHfOVnV+AOcxmd63VAXeF9wQcJvCOg8vp4GPATRDDagbh?= =?iso-8859-1?q?kMNNZVlem8x2btp+mmJEcfxjk050fnnV5woBfjM71czPjkW4mzbmx9P/nJ3I?= =?iso-8859-1?q?Jzj3VagI9/X1lsKzLnM6Yjx7BuCc9rgAw+nuuZh17CVALLfgMMPKV79hLzA3?= =?iso-8859-1?q?we9Ty2OHjjHaPb5oZzTw/wiUEIRcHBkDhgp4UgwAPF6OslVzHbAmcb7yaEwY?= =?iso-8859-1?q?mRvZhmsEXdSMglIOA5ONwrIHj5t6aeDEP/bdrzjt+mk6N5Cdp6DmooRXuoxt?= =?iso-8859-1?q?LQ8fi3Hc3cwNY35MR4fKeAll4M7HnMlNF90aYmEXyxZe4AXgM3WImatspJCz?= =?iso-8859-1?q?Qm4gaZJt6xIUKxY5jcAjmejuvdB0Y85yJrFpnu6/zFFDILGsEBJzNlpbbyKw?= =?iso-8859-1?q?BrhadRdWBfs/dZW/ikQZbZNiYlmZ6O3BYeqBh/FRRCBwWfhSfElRvwU+mRnZ?= =?iso-8859-1?q?2MIe8kjzTMmayi8IR2YSDmPLG7azjHj4/Hnj20mKWumRF70g490Wv36OSoIg?= =?iso-8859-1?q?CEb+Xy9DHiG93X4wgj4RVoePSrcFuEXEyx38hzz+O8vb68Y/v8qG+/W48sg8?= =?iso-8859-1?q?05IR8PruIKHnUkzu4BT7LXhiJfZY7vR7Yx8qXXM+AulLEo+YL8OuHn3c/Cg7?= =?iso-8859-1?q?QbpUONMJ5L5ClJ9SAK3tRoH0o0/e6H659k+XHFbsDkIX53Cz5dilj95TenN2?= =?iso-8859-1?q?kDrVn4YZOmyatq0W4Dstrhp+2Faxqt7EfB/n4dhR7mo7+tWyNu2DyqHfy7mf?= =?iso-8859-1?q?izZQvhaS5e8ZJTKIaNDcgbjKP8rTk0xNe5Dim3DI/cN9EHDOc+fdvoQ5USBy?= =?iso-8859-1?q?T8O/hYuGbeF0UJkrn6vPxcHgG28jEyKPhEuk9RxmmOTgN21oRrgogStYsYwJ?= =?iso-8859-1?q?Aauahy82a4s0Sx+HuoyoBHsD2z4QSThbUL9Sg3erSbot758o8oY0fMu6LFxl?= =?iso-8859-1?q?wGiw/9iMekSlJ2M44vjDP4CKZ94z7F33QGCbWDLsMjFGWbjC1YH0DNi81ab6?= =?iso-8859-1?q?SQOE+HzH61oXQcndqIOsPN7WVDgxM3RgrLw2wjw9GXPBWAYwQQKZ1HS1D2iG?= =?iso-8859-1?q?SfLU3Nax35Fh+5zQlPqm60HbIFMQUPJVnMsUOBg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4470ba1a-1e34-4fc9-d11d-08da8b5aa144 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:10:59.4680 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8SQbDnJXzFcC+TGh7MPXAff3MbCA/JAYHRg/5PKRriJyNQJFTCo6aaFQH56wvOIWAkCZu+RQUCT/PskOy67qgW9xl/HloThi/qeWGt7Ti8Q= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: upecQ0VtuA7okT37Z-I5QTk2lr0btmTN X-Proofpoint-ORIG-GUID: upecQ0VtuA7okT37Z-I5QTk2lr0btmTN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 clxscore=1015 impostorscore=0 mlxlogscore=999 priorityscore=1501 lowpriorityscore=0 phishscore=0 suspectscore=0 mlxscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 ATS subsystem has own list of PCI devices. As we are going to remove global pcidevs_lock() in favor to more granular locking, we need to ensure that this list is protected somehow. To do this, we need to add additional lock for each IOMMU, as list to be protected is also part of IOMMU. Signed-off-by: Volodymyr Babchuk --- xen/drivers/passthrough/amd/iommu.h | 1 + xen/drivers/passthrough/amd/iommu_detect.c | 1 + xen/drivers/passthrough/amd/pci_amd_iommu.c | 8 ++++++++ xen/drivers/passthrough/pci.c | 1 + xen/drivers/passthrough/vtd/iommu.c | 11 +++++++++++ xen/drivers/passthrough/vtd/iommu.h | 1 + xen/drivers/passthrough/vtd/qinval.c | 3 +++ xen/drivers/passthrough/vtd/x86/ats.c | 3 +++ 8 files changed, 29 insertions(+) diff --git a/xen/drivers/passthrough/amd/iommu.h b/xen/drivers/passthrough/amd/iommu.h index 8bc3c35b1b..edd6eb52b3 100644 --- a/xen/drivers/passthrough/amd/iommu.h +++ b/xen/drivers/passthrough/amd/iommu.h @@ -106,6 +106,7 @@ struct amd_iommu { int enabled; struct list_head ats_devices; + spinlock_t ats_list_lock; }; struct ivrs_unity_map { diff --git a/xen/drivers/passthrough/amd/iommu_detect.c b/xen/drivers/passthrough/amd/iommu_detect.c index 2317fa6a7d..1d6f4f2168 100644 --- a/xen/drivers/passthrough/amd/iommu_detect.c +++ b/xen/drivers/passthrough/amd/iommu_detect.c @@ -160,6 +160,7 @@ int __init amd_iommu_detect_one_acpi( } spin_lock_init(&iommu->lock); + spin_lock_init(&iommu->ats_list_lock); INIT_LIST_HEAD(&iommu->ats_devices); iommu->seg = ivhd_block->pci_segment_group; diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c index 64c016491d..955f3af57a 100644 --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -276,7 +276,11 @@ static int __must_check amd_iommu_setup_domain_device( !pci_ats_enabled(iommu->seg, bus, pdev->devfn) ) { if ( devfn == pdev->devfn ) + { + spin_lock(&iommu->ats_list_lock); enable_ats_device(pdev, &iommu->ats_devices); + spin_unlock(&iommu->ats_list_lock); + } amd_iommu_flush_iotlb(devfn, pdev, INV_IOMMU_ALL_PAGES_ADDRESS, 0); } @@ -416,7 +420,11 @@ static void amd_iommu_disable_domain_device(const struct domain *domain, if ( pci_ats_device(iommu->seg, bus, pdev->devfn) && pci_ats_enabled(iommu->seg, bus, pdev->devfn) ) + { + spin_lock(&iommu->ats_list_lock); disable_ats_device(pdev); + spin_unlock(&iommu->ats_list_lock); + } BUG_ON ( iommu->dev_table.buffer == NULL ); req_id = get_dma_requestor_id(iommu->seg, PCI_BDF(bus, devfn)); diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 2dfa1c2875..b5db5498a1 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -1641,6 +1641,7 @@ void iommu_dev_iotlb_flush_timeout(struct domain *d, struct pci_dev *pdev) { pcidevs_lock(); + /* iommu->ats_list_lock is taken by the caller of this function */ disable_ats_device(pdev); ASSERT(pdev->domain); diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index fff1442265..42661f22f4 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -1281,6 +1281,7 @@ int __init iommu_alloc(struct acpi_drhd_unit *drhd) spin_lock_init(&iommu->lock); spin_lock_init(&iommu->register_lock); spin_lock_init(&iommu->intremap.lock); + spin_lock_init(&iommu->ats_list_lock); iommu->drhd = drhd; drhd->iommu = iommu; @@ -1769,7 +1770,11 @@ static int domain_context_mapping(struct domain *domain, u8 devfn, if ( ret > 0 ) ret = 0; if ( !ret && devfn == pdev->devfn && ats_device(pdev, drhd) > 0 ) + { + spin_lock(&drhd->iommu->ats_list_lock); enable_ats_device(pdev, &drhd->iommu->ats_devices); + spin_unlock(&drhd->iommu->ats_list_lock); + } break; @@ -1977,7 +1982,11 @@ static const struct acpi_drhd_unit *domain_context_unmap( domain, &PCI_SBDF(seg, bus, devfn)); ret = domain_context_unmap_one(domain, iommu, bus, devfn); if ( !ret && devfn == pdev->devfn && ats_device(pdev, drhd) > 0 ) + { + spin_lock(&iommu->ats_list_lock); disable_ats_device(pdev); + spin_unlock(&iommu->ats_list_lock); + } break; @@ -2374,7 +2383,9 @@ static int cf_check intel_iommu_enable_device(struct pci_dev *pdev) if ( ret <= 0 ) return ret; + spin_lock(&drhd->iommu->ats_list_lock); ret = enable_ats_device(pdev, &drhd->iommu->ats_devices); + spin_unlock(&drhd->iommu->ats_list_lock); return ret >= 0 ? 0 : ret; } diff --git a/xen/drivers/passthrough/vtd/iommu.h b/xen/drivers/passthrough/vtd/iommu.h index 78aa8a96f5..2a7a4c1b58 100644 --- a/xen/drivers/passthrough/vtd/iommu.h +++ b/xen/drivers/passthrough/vtd/iommu.h @@ -506,6 +506,7 @@ struct vtd_iommu { } flush; struct list_head ats_devices; + spinlock_t ats_list_lock; unsigned long *pseudo_domid_map; /* "pseudo" domain id bitmap */ unsigned long *domid_bitmap; /* domain id bitmap */ domid_t *domid_map; /* domain id mapping array */ diff --git a/xen/drivers/passthrough/vtd/qinval.c b/xen/drivers/passthrough/vtd/qinval.c index 4f9ad136b9..6e876348db 100644 --- a/xen/drivers/passthrough/vtd/qinval.c +++ b/xen/drivers/passthrough/vtd/qinval.c @@ -238,7 +238,10 @@ static int __must_check dev_invalidate_sync(struct vtd_iommu *iommu, if ( d == NULL ) return rc; + spin_lock(&iommu->ats_list_lock); iommu_dev_iotlb_flush_timeout(d, pdev); + spin_unlock(&iommu->ats_list_lock); + rcu_unlock_domain(d); } else if ( rc == -ETIMEDOUT ) diff --git a/xen/drivers/passthrough/vtd/x86/ats.c b/xen/drivers/passthrough/vtd/x86/ats.c index 04d702b1d6..55e991183b 100644 --- a/xen/drivers/passthrough/vtd/x86/ats.c +++ b/xen/drivers/passthrough/vtd/x86/ats.c @@ -117,6 +117,7 @@ int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did, if ( !ecap_dev_iotlb(iommu->ecap) ) return ret; + spin_lock(&iommu->ats_list_lock); list_for_each_entry_safe( pdev, temp, &iommu->ats_devices, ats.list ) { bool_t sbit; @@ -155,12 +156,14 @@ int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did, break; default: dprintk(XENLOG_WARNING VTDPREFIX, "invalid vt-d flush type\n"); + spin_unlock(&iommu->ats_list_lock); return -EOPNOTSUPP; } if ( !ret ) ret = rc; } + spin_unlock(&iommu->ats_list_lock); return ret; } From patchwork Wed Aug 31 14:10:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E9213C0502C for ; Wed, 31 Aug 2022 14:11:32 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395688.635550 (Exim 4.92) (envelope-from ) id 1oTOR8-0004MR-56; Wed, 31 Aug 2022 14:11:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395688.635550; Wed, 31 Aug 2022 14:11:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR7-0004L5-NS; Wed, 31 Aug 2022 14:11:21 +0000 Received: by outflank-mailman (input) for mailman id 395688; Wed, 31 Aug 2022 14:11:20 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR5-0002bv-Vb for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:20 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c7fdaa66-2936-11ed-934f-f50d60e1c1bd; Wed, 31 Aug 2022 16:11:18 +0200 (CEST) Received: from pps.filterd (m0174677.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBo4GA032537; Wed, 31 Aug 2022 14:11:05 GMT Received: from eur04-vi1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3ja427h4rd-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:05 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:10:59 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:10:59 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c7fdaa66-2936-11ed-934f-f50d60e1c1bd ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MJGXXIDHoHiJtLrVaEAuqTYW+gejAS7XKvxe4xb4XAqEVKvEji2TnqMCChDZ9CtjME+d7JYJd92U1Jj/RMvPnNgwwrNbYVFVXmGZMZtUPFBwTsuP1BN5DDMHHf3X18kU/5mkl7BINV31ycs+ZJp5Hq3LnTb28aMLi8ZO88R2Yoywa17in5+ik3FEJjM/h2iztgfnAWhMc8Yo/RypRVqreccNz02NwyYeB7aJHDREo5Kwvm/1ggYizVwRxLtLjd2zhjEhmAhnXjX8zrLbqFNUye9G1+YyshAsiMebw18Oa9cKXoaYK0diX9mIoWm/5hov2K7tRJ8lHCkkVrg2a35+ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8pxAUO1JX4YrzbhZ2dXAYA6LnDaQYu3aeZFrK4TB2Vg=; b=TICh7n6uPBfXv7fpBpV0XOmjOyeUtb78AvszL1B7DMr/jo2bKQ1hYUOIvOxn6MiCuBze9IvERmjual8fcP7EdAouF7emC8XeN2KWaxI9HF+57UxQi4nJiOICTDoiGyM3Bj6AaabikYVwSyT97wnIvFuibXxRJ8FZV6CMoVC0THpX+V5JL3A8qztM83NAPdQd2B5KcZpJPzCcMDUMDlvoxmU9Md5wQgwVKOyKLiTaMz4X5uNtDZKju8LJCWWkckvieYbR+gpVvqMCpc0fqfXS9BJw7SobfkMWq2UH+7waD5KZ6Il/OktsjyLvZkjbRXbG0NaO9i3zPdN/uzspxAMGnQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8pxAUO1JX4YrzbhZ2dXAYA6LnDaQYu3aeZFrK4TB2Vg=; b=qN9ThUOfAA+oiD/3+m1wXl9T8rEXWVcrzJ37eMsQ/7E0G1xA2/MwlRaqOCWlo38nbHqz+LGDdx6FQFTM3CtHKyhVYUh/5zWzWdqRfY0Fnw82fCVcKxiG/Ux67xIqYBL1gLDwBafvL4kpA3SdaE2dur9miCGh0NpgzG/ke3615RyjVORi5DKzlvE1eIdoFj81p0QDPpmUK058YpQ4YAkgXf/MMHhxrN/+IPvrVvlNxodh6xO2ccjmaLAqNJAK0xiPBpJboI7bI9lJbz3789W+Nj7TNe9sqGTaeJwaSRxetb8pWoDKzsc0AEPomc3gprAutI4gdb9O3ry0DbubY4a3Rw== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [RFC PATCH 04/10] xen: add reference counter support Thread-Topic: [RFC PATCH 04/10] xen: add reference counter support Thread-Index: AQHYvUN+XE3jlJIvEkq5uMVpvnNlQg== Date: Wed, 31 Aug 2022 14:10:59 +0000 Message-ID: <20220831141040.13231-5-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 207dc193-9511-4eee-7f4f-08da8b5aa174 x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: lZNtxyrqOPkLQ8r7eYqum6dQcE3DuT6ECw//MIdMjJj0uN3zDNicZbEBcQatX4xk4prXXRsYwbq3iwgigB9X4imYllMYCCON6aFUHKxzk25CViqO+aYSFJwW9AFih/eSIr6dy/aeAeZv4htd39pLEHQsGrap4g2NuNoycWO5sIfZPvCHyyoGhoxtXvuveeF87ccM2adKSk9o/67t7N4TsL3tSFZPqeWZwG0sLsycpl4VAi8R2B6nEuMIFyOxuawQYMoTZxTLF+CDKzSoHJPPC3M4VhVjekbI0uvBzW+jj7wviAq9VnuSExbPHoAq/Sh+jrt6+LtaiQ5FdwQMUwC94IP3tid8eJfmqzyDPsmv6Mn8L30Go7xEFlXQQR0cHWCh/O1AI7yDrAWP3RM1HFObtYo6r9zHvAgvv7kAfoRL/iDon8BOhMsGXnsMWBaJ8+IC1zqBQpxwBVqBmE/hcpTvPji/0vVUHc10KRxv0NClgXzw3AwrGLUo2nrkEbfpDP2W/JTAL8+2pNjEdlJFAwy6lISpNbiBzGRqKQMUr4t5NteDy83UogzZQ6DbShfzvl+FOJWsICnXIBLRZGe7X4OnVgUCzBAm9sfN8iDjzHwE0Rv5SLegcEytgtBmIpBtOmRvMGFfGj3XuQSsJ5tOePpnpwlR/o/TB3mViHaXeSTbtFH498gyenAiNrDaUVO30fUW32+RYcoQ/Y8IO3J2LQMj0vqCxpq9jHDp1UPJfDA2BBxfFbI//G227he46hWluez2s4ubTrQHTWAs0lteS3QKEA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(8936002)(316002)(36756003)(6916009)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?PZ8FHTdjg5HkxzBbuxp7GUT?= =?iso-8859-1?q?ZPF5vCVSgmE9ue9TGO+ePw2diWX5hG6WzvsSxNt71Ey5EZZOINxnwyo6EeXN?= =?iso-8859-1?q?wu2LMvQ+OBjhCpfIvfUQ5tG4+5g7y6bcoToF/VdtBHjMXqxRAJhs26YtqSls?= =?iso-8859-1?q?izK0iy32t7zNsiO0bsQPZMPwJWYpldtDvNOQpPQoXr2tP0nTWFlPiz10hq7d?= =?iso-8859-1?q?7NvwAOU6JYTwajwq67xdHcY49cShp+ErhWf0Bm8ful0hLw+MHJjzketLZkfZ?= =?iso-8859-1?q?rS6DM+p5E/rGO6whELz6PKhaYMDGyRXuPmXj+pZDX5dYXGmZ5NRod2ZxGQqW?= =?iso-8859-1?q?xyK0l3tVGWuB07kDMNd6e9U8/X7V3SeVbO404pyA1EsvGzm8/IkAxnB/iQYz?= =?iso-8859-1?q?JEQuT7xFEMbgu8NmKkruHynON8fI61yyKKRn9QSsNFL33Hsf37We/vp/A8Qo?= =?iso-8859-1?q?jp4pw6z/ZgiMi1ad4E4WrcADAJZ72PacQUSqMPt09sSRUUwa7HDxnMWhloYW?= =?iso-8859-1?q?uTgn2YvEDej7zymjQg6wMmlpIjRfk4l6EFXNcRfZALz/cLxB8R1IBj3Dzi13?= =?iso-8859-1?q?iFHkFjoAgNIy8oiANPW0RPrP+lIL1Srs9cvFP+Fr+2mf1Yr2K7UdiIOM3blU?= =?iso-8859-1?q?VBIkG0ad+WL5I6dkI3H/ya1PFwhRUNd8w8rE0cPbd6s57z3l0EbCv//057l9?= =?iso-8859-1?q?+4ns819RL1KE9ga2aZ8yv7hFRAdLY8WnSX5CEON4gh+XSWqfKp0XjiuANahH?= =?iso-8859-1?q?1NQHf+nhkq42hOKGbqG2xr46b+0KnSCPtku2Y6fJ94WB+6th4CsuINiQSzEf?= =?iso-8859-1?q?ZtuNNuKVDk9/4d0EEUjGWtSF/FP5E1PrIs6tCB27USO5VxKSvsmzhkPuI/gq?= =?iso-8859-1?q?isjobposxst/1AxSrdCGN4fh/syaWjSW8pslcF0BlRW+OL9s5Lkht767f0bq?= =?iso-8859-1?q?3lQBfa/H6APwFnOlOfrqS9lPspreFM4BQ0vF9mG0P0i/daQxjISlSW8dALSn?= =?iso-8859-1?q?84tK7ZW4+An2elfHhbQ7ULNoIZrKVzFL8FkJXF/zyfBIB45FM6PuD+7WXn+N?= =?iso-8859-1?q?moy4lYkmj7y6z2dIFpBbzbM17F1Ep48ESGTaauQt4EIEvawmsOdSpdFVB37P?= =?iso-8859-1?q?FWq3E25afkvjPQcgLQtkQo4frkwd25fDqQpjs2kiJautwGldt2y4rz0L6HW1?= =?iso-8859-1?q?52KCR5DxK+RcMpR1E6selkkXorbbao2suloCOFlqjC5hPOaCZjlkRFoiSuRP?= =?iso-8859-1?q?RqThXAdeyHfKJD/wcekmgokTRPzKT/k6iv4Eitl1fdTptERcaeiohDh6zkH7?= =?iso-8859-1?q?YDnsU18y3k8h6UkEkx4C2DdnyoBNL6QSBgFpAgAhVGmA1Heuj4IgfE0SGGwy?= =?iso-8859-1?q?kMlvPPHMupxQ+itx5z3bW0psub+cAly4qQnJd4MvnFusC0VI32csqfwUjXOE?= =?iso-8859-1?q?arO6PiaI3zOFKmh7kuJDAhX35JWDt5bjlTh8O5P78RT0d+aSnpDKT42uaGed?= =?iso-8859-1?q?DyXDHmhiUjIR9/j27i9galf3Ee7O+GB45da8xZx7w138BWBFKkvirYIWDZBG?= =?iso-8859-1?q?5wms5o2DBZTF25tH3NX4z0n/A0fGAZ2+JoxDXKwC5WfaIsQhEzp+c8HIMQ77?= =?iso-8859-1?q?xgRUtFM2VBc+j8k2noXHEPcWk0ccZAIhxBicRhg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 207dc193-9511-4eee-7f4f-08da8b5aa174 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:10:59.7649 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: j7H9tmwfTR/LVzvAnNCcTXa10GGBEDZEBW7CR3cj6pH1oJ+2LxVocBgJBiolrmb7Ef7YMOBwHVTvRTTT2XDb1vDIAIgZWnBv5Q735iXX/cY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: MH5kAkVFCaQRAHFgGeDEACgEtjou0-EB X-Proofpoint-ORIG-GUID: MH5kAkVFCaQRAHFgGeDEACgEtjou0-EB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=813 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 We can use reference counter to ease up object lifetime management. This patch adds very basic support for reference counters. refcnt should be used in the following way: 1. Protected structure should have refcnt_t field 2. This field should be initialized with refcnt_init() during object construction. 3. If code holds a valid pointer to a structure/object it can increase refcount with refcnt_get(). No additional locking is required. 4. Code should call refcnt_put() before dropping pointer to a protected structure. `destructor` is a call back function that should destruct object and free all resources, including structure protected itself. Destructor will be called if reference counter reaches zero. 5. If code does not hold a valid pointer to a protected structure it should use other locking mechanism to obtain a pointer. For example, it should lock a list that hold protected objects. Signed-off-by: Volodymyr Babchuk --- xen/include/xen/refcnt.h | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 xen/include/xen/refcnt.h diff --git a/xen/include/xen/refcnt.h b/xen/include/xen/refcnt.h new file mode 100644 index 0000000000..7f5395a21c --- /dev/null +++ b/xen/include/xen/refcnt.h @@ -0,0 +1,28 @@ +#ifndef __XEN_REFCNT_H__ +#define __XEN_REFCNT_H__ + +#include + +typedef atomic_t refcnt_t; + +static inline void refcnt_init(refcnt_t *refcnt) +{ + atomic_set(refcnt, 1); +} + +static inline void refcnt_get(refcnt_t *refcnt) +{ +#ifndef NDEBUG + ASSERT(atomic_add_unless(refcnt, 1, 0) > 0); +#else + atomic_add_unless(refcnt, 1, 0); +#endif +} + +static inline void refcnt_put(refcnt_t *refcnt, void (*destructor)(refcnt_t *refcnt)) +{ + if ( atomic_dec_and_test(refcnt) ) + destructor(refcnt); +} + +#endif From patchwork Wed Aug 31 14:11:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5BA2ECAAD1 for ; Wed, 31 Aug 2022 14:11:37 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395691.635576 (Exim 4.92) (envelope-from ) id 1oTORB-00056L-56; Wed, 31 Aug 2022 14:11:25 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395691.635576; Wed, 31 Aug 2022 14:11:25 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTORA-00053b-Nj; Wed, 31 Aug 2022 14:11:24 +0000 Received: by outflank-mailman (input) for mailman id 395691; Wed, 31 Aug 2022 14:11:23 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR9-0002bv-7B for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:23 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c95c3cde-2936-11ed-934f-f50d60e1c1bd; Wed, 31 Aug 2022 16:11:21 +0200 (CEST) Received: from pps.filterd (m0174677.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBo4GC032537; Wed, 31 Aug 2022 14:11:07 GMT Received: from eur04-vi1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3ja427h4rd-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:06 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:11:00 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:11:00 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c95c3cde-2936-11ed-934f-f50d60e1c1bd ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T4tz3hcRtDxJOimtE+Qo210yGmvik98SPc2LVgIHmtcKAaM2MW06k70S7YRJbuzAfTPwqU4XSuaASPO5Dqsj6TEye++tK7bNeHsgUUmt3pnbgA8HZ8K2W7ljImRudhnRU+Z1MZsHTpFa2PqTasBMr1aCmBLRhNKjmdf6WTUeXQwUHBK4+u6f5f1zOiWX7uIs0yGNrqZFTbS5s5En/lWXR5O5BcZvAlRFVxHR2ToRd3qLbsQUe6bzeYpSuq8jDbaxLX7Uw0fETRtZ6ZsUE7GX+Hvjc574h1dQCia9rXd8HtFdDqysAs6p1IVisFYg9KliucHgWHRD1plEKnC1EpB/fQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=v2KJogI0cZ3+/0Y09TtsZB2L93c8/aUqCi7d7m62KUI=; b=aAhPNgj7Kqq0n9284TLky1fCFsCTJibZsA+Hu5vF36vIBRQEjtLIuW3mieEruinhJv6MGxf/cLWtAWMZMRfzZC/ofCNJWcayOghGjr5WDWSfkxMt2ZSI4DDHrXNR2KXCMZ5aaiN66ZKIHszrSsSXgiiDCCpP7sNZqF9u12iyjqxOWUuq/s5E5uTVCwF+Q8IVziUBFtDNVqJaEwORPeuTRqZ6fP6Xl4UtoDaSVWVvFU69XLcDeqXdCKYpGuTNbND86SHZO04py+owCHRvaOo/+qYcXSrBCP5B4riGjTxN7LiG6hAJIyHCUCJR/M2QrQjMUVCd1lH2GaP3oSXaHUsWJA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v2KJogI0cZ3+/0Y09TtsZB2L93c8/aUqCi7d7m62KUI=; b=Nda++i4c2UHeS/sbPzAdVwxvIIxMERodtX5ElzDdjsSGna6CUTyNKz3X/I5nMbfqT83MDMUMq/YTFdv8kRtYWJAvUZmjh7ObBpSqpEephbWSVyNOvM9r99dHGpw96KPRyeUXAH//3bBCoSj6DK/cjVNeuFj1ZoNcCyPW2EECG0BK78RH+j8i3wqc27IYYVwYnH/zB6/wH6bWBw1VMShyFed0kckYLpl/VeIB0sFpc7WQyqZmrWTxKB5Fm9G8gxAGVgT4Ws1Cu/GwB4/kl/Nzif3AngQ4NfX+KbrXtLkyykbUiSj1LKNHVKcRubucE0oZBmDfUDo2wlLy3BwOl4PWgw== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Jan Beulich , Andrew Cooper , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Paul Durrant , Kevin Tian Subject: [RFC PATCH 05/10] xen: pci: introduce reference counting for pdev Thread-Topic: [RFC PATCH 05/10] xen: pci: introduce reference counting for pdev Thread-Index: AQHYvUN/KKgAm1RVgESUCJeBYUFHMA== Date: Wed, 31 Aug 2022 14:11:00 +0000 Message-ID: <20220831141040.13231-6-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 117792c6-6aaa-4f73-4008-08da8b5aa1b7 x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 6bkUaBK9ZjdMIoeTKG355OI2ylD8DOW1tK5qf5n4uWgSK/vJAsytkTNkfyfKiUzKHCdPIrP4ednZScopZn6InwHEOdtb5Zxs22MbW5qAr++zcPGQhZAbX6a52YzEWTDYZDuLJXBT79IdkKZ5WiEhTF6gYVjDhtCCpQIwdp8RiqR43zlhvQSk1i2awB3T+B25fcuY8Z6A5a9/e5qa2UgcGTWhu4PLaGNd6zO1QUAStQpaqls/qBs6mz8YyR2n/+eAJS9UueUnJNRoeUbKjz7DLW/Typ1D0U/ILeye0kzJEtaudiKE//p5Wo0Vl9QiIwe0Dba8LtS8nMy5fsBnAj03KlisAOLK3aP+QmnJZhdmXGMFRiaEirFHqtc6+Z+Og3GHpvGW5Xooabn2j3hX0RnsA/OY1WWe3DYYWB7LzqSGQviULUck8ju6YurTiMQQe+ukUB8xv1YeoXWoXjITZoHG97TieaxAcjAvt4QUmJgK5DG06xjURNvpYMrj1evvDqyMolRGEto/IlN4ySl0VuV6gL8y3hyl904fdl+Gszsz/8nQm87WUsF2x9On2+FFWlwriS9GINbaCPB42Zhx+cXf9MnrPprLlYdPbV7Llp0UhWFqE51eOXEXsty6i743w2CV7OVj1jotLHWyc+rfcQs2w16q28Ka7twzyzhiCLh8DzLezkLmLOXcAKcpxJHjrUzoCmmQ+q7VzhhACg8nDHl921vLftrjlltL/BOtyEhF3JF3iiKFKlGNdxJ/tvJmGJIhi+lJZnF7UtirR8m2pZHPD+jczNCHAMpBpfEo4SXQH1Q= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(7416002)(8936002)(316002)(36756003)(6916009)(30864003)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002)(21314003)(579004)(559001);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?TlgNITTR6PX7M2TRGPJMuMx?= =?iso-8859-1?q?ZQ4NaKi5kuQfy4lRRGppqyXQpLMTOQdXh5GjOeIp5n7zdR8AE1mS6eOzv1iB?= =?iso-8859-1?q?TmsdxYsOGjHoJz9i/K9GDRzSLQI4v0KioFyiMby4BMOcQTe+pLCLoEoZQzE2?= =?iso-8859-1?q?DARkPnE7ntGB4mVYGfOpm2B9En1GPw94ddg+bL3VC0LwQJW7ay99DcJvkp1z?= =?iso-8859-1?q?NBRbur2iU52q4k4WYQCSRgD5kihZWYCt/Dyk/Z8k2P5c3G2b9JigRHwAaFOw?= =?iso-8859-1?q?faN8ayUj5sClTrn2HJvPEfLb/Xx85RTlPurda55D/QMOcH9jY60IOn5Dilp2?= =?iso-8859-1?q?WbqUkf6MbaNmlJdt9RuLjde/mk7ott9WFDdvBEJQyC13Eh3jxhHKlwXzG8HG?= =?iso-8859-1?q?dryEkZ/4zxwE8vbL//47fWS4e7tw5IkJc92Hw+puOBXYVoUOL8/TrKEOro8q?= =?iso-8859-1?q?nZtgoiNvTBT/FFv4wPu5rFuj2hPXEHWhgJOO1Nayi/2Sf9bUCpYHSFBZHtJY?= =?iso-8859-1?q?6fPAQvs/cbegGidu/h96DDGe10e+6mGmAlc83CaKgmKHBZQXoKN3AdBEc1k9?= =?iso-8859-1?q?S1C842HLlbwMc3QXkYRciEcarq1sQVFmNdymzQx6uh5QpvzozZo2ZfcLwtCv?= =?iso-8859-1?q?zUvjGNjeKDvCcH7tIgq8r9UAKVWPg5YD+Jv3Hims91ELiQd6dMbRSlW/7U/o?= =?iso-8859-1?q?evDt6byZyg8NcdtSeB58kiGD49Qxnnpv4pkM5+0Qe//QXPWeGXytmeNhlrKV?= =?iso-8859-1?q?MPaAiXijtJ7EfsQZWs9qUid9KlyL8yExQZ3NKVtsgeLsfUSm79BUpmIsqqMr?= =?iso-8859-1?q?bI9WmtWlDI4cIMp52WhSTtrsLJuwHZYbvvA/00qZd0D8smtr4QrTF8hU4g+h?= =?iso-8859-1?q?oOBoNLHFucuEY8CHy5wj7HogcSCw98EDF3qITLJNIQU/0tPbJTgOJUVcwt5F?= =?iso-8859-1?q?kBSi3gdgp9Y2X5mXVcyRxIuIN1eewE147r9AaCWoWCdfQhUEnc1GiKY+u503?= =?iso-8859-1?q?qaAZzLpf1FwHDQ80oqdzQ717cw2wzeM31qR2NIqLOMZWcq1tL7axcUcfp2s0?= =?iso-8859-1?q?zzoQI0amlwMrNiY5e9v/AUctxcsLcdFbXb31ReGZzxSFjWSKSSJVRJvoAOEA?= =?iso-8859-1?q?J+3VpCLGJnOvkmvI92eL8w0w/tJSuvA5l4gJsuL0HakbMRufj3AXhcs+alvL?= =?iso-8859-1?q?WZbKQPj1JBmrkBmXeTwp7/xeCNQVCCy8KmlKjVzvXpD1VtpQOu33OhAhTrzM?= =?iso-8859-1?q?qmz79F+Ixn5ttghPIvdPuoN4H8icQC+l3ot37S78NBMUcfsXiHK3mZMIgdZw?= =?iso-8859-1?q?ly0g2/8fO38PYaVQe1bwn8KmlDLE799aycwaMOdm3w0MSWi5Xr0vnSMnwJOs?= =?iso-8859-1?q?WFau27dO8ewah6fRiwDJIomzNdoF/M6mnpG9R9oIehK5SQ71NhUDDsRsDAE0?= =?iso-8859-1?q?KTEsZluNTgWh3aRNiVpVEQJ44HT9mVMDjZtbJVux4CDc6e9I23c/TO0U40f5?= =?iso-8859-1?q?AI/HvHvDdpZQesWlB+1d0BjKGmupIMOeNxS3+nYAWBfwHWwVZutZ4DtB8mo+?= =?iso-8859-1?q?5t4SpSXuSqp0TxTAHhpwhJg+IBWb5reKGZaYTmUhvtpfIElQrstfkwZNf1Qb?= =?iso-8859-1?q?53OujKzW2Tos8+e2SvvltTWJk7CZryhXrFftK/Q=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 117792c6-6aaa-4f73-4008-08da8b5aa1b7 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:11:00.2492 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XPPZQpvKHY2H6MHWjCiEW4XEJIVtOcT4d7o8m6QQsHPZmhAOCUt8ABlhTGNz9DSL0fD2nLTuQDogcXOkufQkRRjAWQjsz2CRTGUogsm8DFo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: aVlkLkwcfazWl26gJqx2Ls0rL_uqJPWH X-Proofpoint-ORIG-GUID: aVlkLkwcfazWl26gJqx2Ls0rL_uqJPWH X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=999 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 Prior to this change, lifetime of pci_dev objects was protected by global pcidevs_lock(). We are going to get if of this lock, so we need some other mechanism to ensure that those objects will not disappear under feet of code that access them. Reference counting is a good choice as it provides easy to comprehend way to control object lifetime with better granularity than global super lock. This patch adds two new helper functions: pcidev_get() and pcidev_put(). pcidev_get() will increase reference counter, while pcidev_put() will decrease it, destroying object when counter reaches zero. pcidev_get() should be used only when you already have a valid pointer to the object or you are holding lock that protects one of the lists (domain, pseg or ats) that store pci_dev structs. pcidev_get() is rarely used directly, because there already are functions that will provide valid pointer to pci_dev struct: pci_get_pdev() and pci_get_real_pdev(). They will lock appropriate list, find needed object and increase its reference counter before returning to the caller. Naturally, pci_put() should be called after finishing working with a received object. This is the reason why this patch have so many pcidev_put()s and so little pcidev_get()s: existing calls to pci_get_*() functions now will increase reference counter automatically, we just need to decrease it back when we finished. This patch removes "const" qualifier from some pdev pointers because pcidev_put() technically alters the contents of pci_dev structure. Signed-off-by: Volodymyr Babchuk --- - Jan, can I add your Suggested-by tag? --- xen/arch/x86/hvm/vmsi.c | 2 +- xen/arch/x86/irq.c | 4 + xen/arch/x86/msi.c | 41 ++++++- xen/arch/x86/pci.c | 4 +- xen/arch/x86/physdev.c | 17 ++- xen/common/sysctl.c | 5 +- xen/drivers/passthrough/amd/iommu_init.c | 12 ++- xen/drivers/passthrough/amd/iommu_map.c | 6 +- xen/drivers/passthrough/pci.c | 131 +++++++++++++++-------- xen/drivers/passthrough/vtd/quirks.c | 2 + xen/drivers/video/vga.c | 10 +- xen/drivers/vpci/vpci.c | 6 +- xen/include/xen/pci.h | 18 ++++ 13 files changed, 201 insertions(+), 57 deletions(-) diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c index 75f92885dc..7fb1075673 100644 --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -912,7 +912,7 @@ int vpci_msix_arch_print(const struct vpci_msix *msix) spin_unlock(&msix->pdev->vpci->lock); process_pending_softirqs(); - /* NB: we assume that pdev cannot go away for an alive domain. */ + if ( !pdev->vpci || !spin_trylock(&pdev->vpci->lock) ) return -EBUSY; if ( pdev->vpci->msix != msix ) diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index cd0c8a30a8..d8672a03e1 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2174,6 +2174,7 @@ int map_domain_pirq( msi->entry_nr = ret; ret = -ENFILE; } + pcidev_put(pdev); goto done; } @@ -2188,6 +2189,7 @@ int map_domain_pirq( msi_desc->irq = -1; msi_free_irq(msi_desc); ret = -EBUSY; + pcidev_put(pdev); goto done; } @@ -2272,10 +2274,12 @@ int map_domain_pirq( } msi_desc->irq = -1; msi_free_irq(msi_desc); + pcidev_put(pdev); goto done; } set_domain_irq_pirq(d, irq, info); + pcidev_put(pdev); spin_unlock_irqrestore(&desc->lock, flags); } else diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c index d0bf63df1d..bccaccb98b 100644 --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -572,6 +572,10 @@ int msi_free_irq(struct msi_desc *entry) virt_to_fix((unsigned long)entry->mask_base)); list_del(&entry->list); + + /* Corresponds to pcidev_get() in msi[x]_capability_init() */ + pcidev_put(entry->dev); + xfree(entry); return 0; } @@ -644,6 +648,7 @@ static int msi_capability_init(struct pci_dev *dev, entry[i].msi.mpos = mpos; entry[i].msi.nvec = 0; entry[i].dev = dev; + pcidev_get(dev); } entry->msi.nvec = nvec; entry->irq = irq; @@ -703,22 +708,36 @@ static u64 read_pci_mem_bar(u16 seg, u8 bus, u8 slot, u8 func, u8 bir, int vf) !num_vf || !offset || (num_vf > 1 && !stride) || bir >= PCI_SRIOV_NUM_BARS || !pdev->vf_rlen[bir] ) + { + if ( pdev ) + pcidev_put(pdev); return 0; + } base = pos + PCI_SRIOV_BAR; vf -= PCI_BDF(bus, slot, func) + offset; if ( vf < 0 ) + { + pcidev_put(pdev); return 0; + } if ( stride ) { if ( vf % stride ) + { + pcidev_put(pdev); return 0; + } vf /= stride; } if ( vf >= num_vf ) + { + pcidev_put(pdev); return 0; + } BUILD_BUG_ON(ARRAY_SIZE(pdev->vf_rlen) != PCI_SRIOV_NUM_BARS); disp = vf * pdev->vf_rlen[bir]; limit = PCI_SRIOV_NUM_BARS; + pcidev_put(pdev); } else switch ( pci_conf_read8(PCI_SBDF(seg, bus, slot, func), PCI_HEADER_TYPE) & 0x7f ) @@ -925,6 +944,8 @@ static int msix_capability_init(struct pci_dev *dev, entry->dev = dev; entry->mask_base = base; + pcidev_get(dev); + list_add_tail(&entry->list, &dev->msi_list); *desc = entry; } @@ -999,6 +1020,7 @@ static int __pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) { struct pci_dev *pdev; struct msi_desc *old_desc; + int ret; ASSERT(pcidevs_locked()); pdev = pci_get_pdev(NULL, msi->sbdf); @@ -1010,6 +1032,7 @@ static int __pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) { printk(XENLOG_ERR "irq %d already mapped to MSI on %pp\n", msi->irq, &pdev->sbdf); + pcidev_put(pdev); return -EEXIST; } @@ -1020,7 +1043,10 @@ static int __pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) __pci_disable_msix(old_desc); } - return msi_capability_init(pdev, msi->irq, desc, msi->entry_nr); + ret = msi_capability_init(pdev, msi->irq, desc, msi->entry_nr); + pcidev_put(pdev); + + return ret; } static void __pci_disable_msi(struct msi_desc *entry) @@ -1054,6 +1080,7 @@ static int __pci_enable_msix(struct msi_info *msi, struct msi_desc **desc) { struct pci_dev *pdev; struct msi_desc *old_desc; + int ret; ASSERT(pcidevs_locked()); pdev = pci_get_pdev(NULL, msi->sbdf); @@ -1061,13 +1088,17 @@ static int __pci_enable_msix(struct msi_info *msi, struct msi_desc **desc) return -ENODEV; if ( msi->entry_nr >= pdev->msix->nr_entries ) + { + pcidev_put(pdev); return -EINVAL; + } old_desc = find_msi_entry(pdev, msi->irq, PCI_CAP_ID_MSIX); if ( old_desc ) { printk(XENLOG_ERR "irq %d already mapped to MSI-X on %pp\n", msi->irq, &pdev->sbdf); + pcidev_put(pdev); return -EEXIST; } @@ -1078,7 +1109,11 @@ static int __pci_enable_msix(struct msi_info *msi, struct msi_desc **desc) __pci_disable_msi(old_desc); } - return msix_capability_init(pdev, msi, desc); + ret = msix_capability_init(pdev, msi, desc); + + pcidev_put(pdev); + + return ret; } static void _pci_cleanup_msix(struct arch_msix *msix) @@ -1161,6 +1196,8 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off) rc = msix_capability_init(pdev, NULL, NULL); pcidevs_unlock(); + pcidev_put(pdev); + return rc; } diff --git a/xen/arch/x86/pci.c b/xen/arch/x86/pci.c index 97b792e578..1d38f0df7c 100644 --- a/xen/arch/x86/pci.c +++ b/xen/arch/x86/pci.c @@ -91,8 +91,10 @@ int pci_conf_write_intercept(unsigned int seg, unsigned int bdf, pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bdf)); - if ( pdev ) + if ( pdev ) { rc = pci_msi_conf_write_intercept(pdev, reg, size, data); + pcidev_put(pdev); + } pcidevs_unlock(); diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c index 2f1d955a96..96214a3d40 100644 --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -533,7 +533,14 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(0, restore_msi.bus, restore_msi.devfn)); - ret = pdev ? pci_restore_msi_state(pdev) : -ENODEV; + if ( pdev ) + { + ret = pci_restore_msi_state(pdev); + pcidev_put(pdev); + } + else + ret = -ENODEV; + pcidevs_unlock(); break; } @@ -548,7 +555,13 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(dev.seg, dev.bus, dev.devfn)); - ret = pdev ? pci_restore_msi_state(pdev) : -ENODEV; + if ( pdev ) + { + ret = pci_restore_msi_state(pdev); + pcidev_put(pdev); + } + else + ret = -ENODEV; pcidevs_unlock(); break; } diff --git a/xen/common/sysctl.c b/xen/common/sysctl.c index 02505ab044..0feef94cd2 100644 --- a/xen/common/sysctl.c +++ b/xen/common/sysctl.c @@ -438,7 +438,7 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl) { physdev_pci_device_t dev; uint32_t node; - const struct pci_dev *pdev; + struct pci_dev *pdev; if ( copy_from_guest_offset(&dev, ti->devs, i, 1) ) { @@ -456,6 +456,9 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl) node = pdev->node; pcidevs_unlock(); + if ( pdev ) + pcidev_put(pdev); + if ( copy_to_guest_offset(ti->nodes, i, &node, 1) ) { ret = -EFAULT; diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c index 1f14aaf49e..7c1713a602 100644 --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -644,6 +644,7 @@ static void cf_check parse_ppr_log_entry(struct amd_iommu *iommu, u32 entry[]) if ( pdev ) guest_iommu_add_ppr_log(pdev->domain, entry); + pcidev_put(pdev); } static void iommu_check_ppr_log(struct amd_iommu *iommu) @@ -747,6 +748,11 @@ static bool_t __init set_iommu_interrupt_handler(struct amd_iommu *iommu) } pcidevs_lock(); + /* + * XXX: it is unclear if this device can be removed. Right now + * there is no code that clears msi.dev, so no one will decrease + * refcount on it. + */ iommu->msi.dev = pci_get_pdev(NULL, PCI_SBDF(iommu->seg, iommu->bdf)); pcidevs_unlock(); if ( !iommu->msi.dev ) @@ -1272,7 +1278,7 @@ static int __init cf_check amd_iommu_setup_device_table( { if ( ivrs_mappings[bdf].valid ) { - const struct pci_dev *pdev = NULL; + struct pci_dev *pdev = NULL; /* add device table entry */ iommu_dte_add_device_entry(&dt[bdf], &ivrs_mappings[bdf]); @@ -1297,7 +1303,10 @@ static int __init cf_check amd_iommu_setup_device_table( pdev->msix ? pdev->msix->nr_entries : pdev->msi_maxvec); if ( !ivrs_mappings[bdf].intremap_table ) + { + pcidev_put(pdev); return -ENOMEM; + } if ( pdev->phantom_stride ) { @@ -1315,6 +1324,7 @@ static int __init cf_check amd_iommu_setup_device_table( ivrs_mappings[bdf].intremap_inuse; } } + pcidev_put(pdev); } amd_iommu_set_intremap_table( diff --git a/xen/drivers/passthrough/amd/iommu_map.c b/xen/drivers/passthrough/amd/iommu_map.c index 993bac6f88..9d621e3d36 100644 --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -724,14 +724,18 @@ int cf_check amd_iommu_get_reserved_device_memory( if ( !iommu ) { /* May need to trigger the workaround in find_iommu_for_device(). */ - const struct pci_dev *pdev; + struct pci_dev *pdev; pcidevs_lock(); pdev = pci_get_pdev(NULL, sbdf); pcidevs_unlock(); if ( pdev ) + { iommu = find_iommu_for_device(seg, bdf); + /* XXX: Should we hold pdev reference till end of the loop? */ + pcidev_put(pdev); + } if ( !iommu ) continue; } diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index b5db5498a1..a6c6368769 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -403,6 +403,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn) *((u8*) &pdev->bus) = bus; *((u8*) &pdev->devfn) = devfn; pdev->domain = NULL; + refcnt_init(&pdev->refcnt); arch_pci_init_pdev(pdev); @@ -499,33 +500,6 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn) return pdev; } -static void free_pdev(struct pci_seg *pseg, struct pci_dev *pdev) -{ - /* update bus2bridge */ - switch ( pdev->type ) - { - unsigned int sec_bus, sub_bus; - - case DEV_TYPE_PCIe2PCI_BRIDGE: - case DEV_TYPE_LEGACY_PCI_BRIDGE: - sec_bus = pci_conf_read8(pdev->sbdf, PCI_SECONDARY_BUS); - sub_bus = pci_conf_read8(pdev->sbdf, PCI_SUBORDINATE_BUS); - - spin_lock(&pseg->bus2bridge_lock); - for ( ; sec_bus <= sub_bus; sec_bus++ ) - pseg->bus2bridge[sec_bus] = pseg->bus2bridge[pdev->bus]; - spin_unlock(&pseg->bus2bridge_lock); - break; - - default: - break; - } - - list_del(&pdev->alldevs_list); - pdev_msi_deinit(pdev); - xfree(pdev); -} - static void __init _pci_hide_device(struct pci_dev *pdev) { if ( pdev->domain ) @@ -596,10 +570,15 @@ struct pci_dev *pci_get_real_pdev(pci_sbdf_t sbdf) { if ( !(sbdf.devfn & stride) ) continue; + sbdf.devfn &= ~stride; + pcidev_put(pdev); pdev = pci_get_pdev(NULL, sbdf); if ( pdev && stride != pdev->phantom_stride ) + { + pcidev_put(pdev); pdev = NULL; + } } return pdev; @@ -629,6 +608,7 @@ struct pci_dev *pci_get_pdev(struct domain *d, pci_sbdf_t sbdf) if ( pdev->sbdf.bdf == sbdf.bdf && (!d || pdev->domain == d) ) { + pcidev_get(pdev); spin_unlock(&pseg->alldevs_lock); return pdev; } @@ -640,6 +620,7 @@ struct pci_dev *pci_get_pdev(struct domain *d, pci_sbdf_t sbdf) list_for_each_entry ( pdev, &d->pdev_list, domain_list ) if ( pdev->sbdf.bdf == sbdf.bdf ) { + pcidev_get(pdev); spin_unlock(&d->pdevs_lock); return pdev; } @@ -754,7 +735,10 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, PCI_SBDF(seg, info->physfn.bus, info->physfn.devfn)); if ( pdev ) + { pf_is_extfn = pdev->info.is_extfn; + pcidev_put(pdev); + } pcidevs_unlock(); if ( !pdev ) pci_add_device(seg, info->physfn.bus, info->physfn.devfn, @@ -920,8 +904,9 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) spin_unlock(&pdev->domain->pdevs_lock); } printk(XENLOG_DEBUG "PCI remove device %pp\n", &pdev->sbdf); - free_pdev(pseg, pdev); list_del(&pdev->alldevs_list); + pdev_msi_deinit(pdev); + pcidev_put(pdev); break; } @@ -952,7 +937,7 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, { ret = iommu_quarantine_dev_init(pci_to_dev(pdev)); if ( ret ) - return ret; + goto out; target = dom_io; } @@ -982,6 +967,7 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, pdev->fault.count = 0; out: + pcidev_put(pdev); if ( ret ) printk(XENLOG_G_ERR "%pd: deassign (%pp) failed (%d)\n", d, &PCI_SBDF(seg, bus, devfn), ret); @@ -1117,7 +1103,10 @@ void pci_check_disable_device(u16 seg, u8 bus, u8 devfn) pdev->fault.count >>= 1; pdev->fault.time = now; if ( ++pdev->fault.count < PT_FAULT_THRESHOLD ) + { + pcidev_put(pdev); pdev = NULL; + } } pcidevs_unlock(); @@ -1128,6 +1117,8 @@ void pci_check_disable_device(u16 seg, u8 bus, u8 devfn) * control it for us. */ cword = pci_conf_read16(pdev->sbdf, PCI_COMMAND); pci_conf_write16(pdev->sbdf, PCI_COMMAND, cword & ~PCI_COMMAND_MASTER); + + pcidev_put(pdev); } /* @@ -1246,6 +1237,7 @@ static int __hwdom_init cf_check _setup_hwdom_pci_devices( printk(XENLOG_WARNING "Dom%d owning %pp?\n", pdev->domain->domain_id, &pdev->sbdf); + pcidev_put(pdev); if ( iommu_verbose ) { pcidevs_unlock(); @@ -1495,33 +1487,28 @@ static int iommu_remove_device(struct pci_dev *pdev) return iommu_call(hd->platform_ops, remove_device, devfn, pci_to_dev(pdev)); } -static int device_assigned(u16 seg, u8 bus, u8 devfn) +static int device_assigned(struct pci_dev *pdev) { - struct pci_dev *pdev; int rc = 0; ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); - - if ( !pdev ) - rc = -ENODEV; /* * If the device exists and it is not owned by either the hardware * domain or dom_io then it must be assigned to a guest, or be * hidden (owned by dom_xen). */ - else if ( pdev->domain != hardware_domain && - pdev->domain != dom_io ) + if ( pdev->domain != hardware_domain && + pdev->domain != dom_io ) rc = -EBUSY; return rc; } /* Caller should hold the pcidevs_lock */ -static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) +static int assign_device(struct domain *d, struct pci_dev *pdev, u32 flag) { const struct domain_iommu *hd = dom_iommu(d); - struct pci_dev *pdev; + uint8_t devfn; int rc = 0; if ( !is_iommu_enabled(d) ) @@ -1532,10 +1519,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) /* device_assigned() should already have cleared the device for assignment */ ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); ASSERT(pdev && (pdev->domain == hardware_domain || pdev->domain == dom_io)); + devfn = pdev->devfn; + /* Do not allow broken devices to be assigned to guests. */ rc = -EBADF; if ( pdev->broken && d != hardware_domain && d != dom_io ) @@ -1570,7 +1558,7 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) done: if ( rc ) printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n", - d, &PCI_SBDF(seg, bus, devfn), rc); + d, &PCI_SBDF(pdev->seg, pdev->bus, devfn), rc); /* The device is assigned to dom_io so mark it as quarantined */ else if ( d == dom_io ) pdev->quarantine = true; @@ -1710,6 +1698,9 @@ int iommu_do_pci_domctl( ASSERT(d); /* fall through */ case XEN_DOMCTL_test_assign_device: + { + struct pci_dev *pdev; + /* Don't support self-assignment of devices. */ if ( d == current->domain ) { @@ -1737,26 +1728,36 @@ int iommu_do_pci_domctl( seg = machine_sbdf >> 16; bus = PCI_BUS(machine_sbdf); devfn = PCI_DEVFN(machine_sbdf); + pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); + if ( !pdev ) + { + printk(XENLOG_G_INFO "%pp non-existent\n", + &PCI_SBDF(seg, bus, devfn)); + ret = -EINVAL; + break; + } pcidevs_lock(); - ret = device_assigned(seg, bus, devfn); + ret = device_assigned(pdev); if ( domctl->cmd == XEN_DOMCTL_test_assign_device ) { if ( ret ) { - printk(XENLOG_G_INFO "%pp already assigned, or non-existent\n", + printk(XENLOG_G_INFO "%pp already assigned\n", &PCI_SBDF(seg, bus, devfn)); ret = -EINVAL; } } else if ( !ret ) - ret = assign_device(d, seg, bus, devfn, flags); + ret = assign_device(d, pdev, flags); + + pcidev_put(pdev); pcidevs_unlock(); if ( ret == -ERESTART ) ret = hypercall_create_continuation(__HYPERVISOR_domctl, "h", u_domctl); break; - + } case XEN_DOMCTL_deassign_device: /* Don't support self-deassignment of devices. */ if ( d == current->domain ) @@ -1796,6 +1797,46 @@ int iommu_do_pci_domctl( return ret; } +static void release_pdev(refcnt_t *refcnt) +{ + struct pci_dev *pdev = container_of(refcnt, struct pci_dev, refcnt); + struct pci_seg *pseg = get_pseg(pdev->seg); + + printk(XENLOG_DEBUG "PCI release device %pp\n", &pdev->sbdf); + + /* update bus2bridge */ + switch ( pdev->type ) + { + unsigned int sec_bus, sub_bus; + + case DEV_TYPE_PCIe2PCI_BRIDGE: + case DEV_TYPE_LEGACY_PCI_BRIDGE: + sec_bus = pci_conf_read8(pdev->sbdf, PCI_SECONDARY_BUS); + sub_bus = pci_conf_read8(pdev->sbdf, PCI_SUBORDINATE_BUS); + + spin_lock(&pseg->bus2bridge_lock); + for ( ; sec_bus <= sub_bus; sec_bus++ ) + pseg->bus2bridge[sec_bus] = pseg->bus2bridge[pdev->bus]; + spin_unlock(&pseg->bus2bridge_lock); + break; + + default: + break; + } + + xfree(pdev); +} + +void pcidev_get(struct pci_dev *pdev) +{ + refcnt_get(&pdev->refcnt); +} + +void pcidev_put(struct pci_dev *pdev) +{ + refcnt_put(&pdev->refcnt, release_pdev); +} + /* * Local variables: * mode: C diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c index fcc8f73e8b..d240da0416 100644 --- a/xen/drivers/passthrough/vtd/quirks.c +++ b/xen/drivers/passthrough/vtd/quirks.c @@ -429,6 +429,8 @@ static int __must_check map_me_phantom_function(struct domain *domain, rc = domain_context_unmap_one(domain, drhd->iommu, 0, PCI_DEVFN(dev, 7)); + pcidev_put(pdev); + return rc; } diff --git a/xen/drivers/video/vga.c b/xen/drivers/video/vga.c index 29a88e8241..1298f3a7b6 100644 --- a/xen/drivers/video/vga.c +++ b/xen/drivers/video/vga.c @@ -114,7 +114,7 @@ void __init video_endboot(void) for ( bus = 0; bus < 256; ++bus ) for ( devfn = 0; devfn < 256; ++devfn ) { - const struct pci_dev *pdev; + struct pci_dev *pdev; u8 b = bus, df = devfn, sb; pcidevs_lock(); @@ -126,7 +126,11 @@ void __init video_endboot(void) PCI_CLASS_DEVICE) != 0x0300 || !(pci_conf_read16(PCI_SBDF(0, bus, devfn), PCI_COMMAND) & (PCI_COMMAND_IO | PCI_COMMAND_MEMORY)) ) + { + if (pdev) + pcidev_put(pdev); continue; + } while ( b ) { @@ -144,7 +148,10 @@ void __init video_endboot(void) if ( pci_conf_read16(PCI_SBDF(0, b, df), PCI_BRIDGE_CONTROL) & PCI_BRIDGE_CTL_VGA ) + { + pcidev_put(pdev); continue; + } break; } break; @@ -157,6 +164,7 @@ void __init video_endboot(void) bus, PCI_SLOT(devfn), PCI_FUNC(devfn)); pci_hide_device(0, bus, devfn); } + pcidev_put(pdev); } } diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 7d1f9fd438..59dc55f498 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -313,7 +313,7 @@ static uint32_t merge_result(uint32_t data, uint32_t new, unsigned int size, uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) { struct domain *d = current->domain; - const struct pci_dev *pdev; + struct pci_dev *pdev; const struct vpci_register *r; unsigned int data_offset = 0; uint32_t data = ~(uint32_t)0; @@ -373,6 +373,7 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) ASSERT(data_offset < size); } spin_unlock(&pdev->vpci->lock); + pcidev_put(pdev); if ( data_offset < size ) { @@ -416,7 +417,7 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, uint32_t data) { struct domain *d = current->domain; - const struct pci_dev *pdev; + struct pci_dev *pdev; const struct vpci_register *r; unsigned int data_offset = 0; const unsigned long *ro_map = pci_get_ro_map(sbdf.seg); @@ -478,6 +479,7 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, ASSERT(data_offset < size); } spin_unlock(&pdev->vpci->lock); + pcidev_put(pdev); if ( data_offset < size ) /* Tailing gap, write the remaining. */ diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h index 19047b4b20..e71a180ef3 100644 --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -116,6 +117,9 @@ struct pci_dev { /* Device misbehaving, prevent assigning it to guests. */ bool broken; + /* Reference counter */ + refcnt_t refcnt; + enum pdev_type { DEV_TYPE_PCI_UNKNOWN, DEV_TYPE_PCIe_ENDPOINT, @@ -160,6 +164,14 @@ void pcidevs_lock(void); void pcidevs_unlock(void); bool_t __must_check pcidevs_locked(void); +/* + * Acquire and release reference to the given device. Holding + * reference ensures that device will not disappear under feet, but + * does not guarantee that code has exclusive access to the device. + */ +void pcidev_get(struct pci_dev *pdev); +void pcidev_put(struct pci_dev *pdev); + bool_t pci_known_segment(u16 seg); bool_t pci_device_detect(u16 seg, u8 bus, u8 dev, u8 func); int scan_pci_devices(void); @@ -177,8 +189,14 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, int pci_remove_device(u16 seg, u8 bus, u8 devfn); int pci_ro_device(int seg, int bus, int devfn); int pci_hide_device(unsigned int seg, unsigned int bus, unsigned int devfn); + +/* + * Next two functions will find a requested device and acquire + * reference to it. Use pcidev_put() to release the reference. + */ struct pci_dev *pci_get_pdev(struct domain *d, pci_sbdf_t sbdf); struct pci_dev *pci_get_real_pdev(pci_sbdf_t sbdf); + void pci_check_disable_device(u16 seg, u8 bus, u8 devfn); uint8_t pci_conf_read8(pci_sbdf_t sbdf, unsigned int reg); From patchwork Wed Aug 31 14:11:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8388CC0502A for ; Wed, 31 Aug 2022 14:11:33 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395682.635483 (Exim 4.92) (envelope-from ) id 1oTOR0-0002ef-LO; Wed, 31 Aug 2022 14:11:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395682.635483; Wed, 31 Aug 2022 14:11:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR0-0002e9-HL; Wed, 31 Aug 2022 14:11:14 +0000 Received: by outflank-mailman (input) for mailman id 395682; Wed, 31 Aug 2022 14:11:13 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOQy-0002bw-S4 for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:13 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c0fd0947-2936-11ed-82f2-63bd783d45fa; Wed, 31 Aug 2022 16:11:07 +0200 (CEST) Received: from pps.filterd (m0174677.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBo4GB032537; Wed, 31 Aug 2022 14:11:05 GMT Received: from eur04-vi1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3ja427h4rd-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:05 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:11:00 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:11:00 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c0fd0947-2936-11ed-82f2-63bd783d45fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fXnQyhJf4DX1we0FISOu932jDiOgUD4D1DKvbFxJE7zOlshE4C/kSdnG6StzEu7eql6gFrj/fRz0tWcHD95FLGi7lAIARpP/zY5o4G8/6egZwc5LtYPrnJwyiDphFT4S/eLMgB9aCxEwoEs2VLFOFshITZlfVFLdGsdd1+dt0Y4fRwbC6Kmd42n1A3QRJQSomwnobyiG/DP8vlzVP6ZbIt7C+1KpONgk6l6COy1LHu/qMaXdX4BMwF4oQfSQKDKdtxmz5qMmkhC/gSPhiLvHn1m0al3cVf+qTRtIzv9KNP2FrmzgjDZLxjDCsua9T2R+G5iquQVCMWj2dYXbC+btnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8+xdObH6FMWVQ4XGWc2Z5mSrpHg5ctG6VX5xgOC+obg=; b=dM6AN/o/6xQ4fhL9r8HF3PvAdbmILG3TFgllrB7qsOc9l4lIe4YTHufHJ0pgGp0unisJjj/b6btjMH16PWXMEeV6+ya4sExpCn7cjlgfFpY9omUM4Z2MDNr3EMS5ZP1K/IAVdDo4u3oeH4dgkdsD1ci1Fgq5DixcFFSc//FIhjhVaNM6i2DAGohrjtCs9K6dUf53dP+TzU2ci8md0bXCHds9rh5pt6liDGIqbqXVT3EUsbbquOLpWMxZArVIK1eHwmnWBhw2b1/r9bpkZZoWY6TQNzzhQ/n8hRt/8QKdchaXhZEwLWEzIeO8C04FJ51/ZgtNem7J9xy9bvljqDlmjw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8+xdObH6FMWVQ4XGWc2Z5mSrpHg5ctG6VX5xgOC+obg=; b=jDavVxJctqIsQT5YdlAtDMK5ww4sz/CXfYEXoTESTanakIRPyLFS8tZbSGqNvNsjysgZIiQ6F0HwmjKwV4GGzU6nqbOZAclSltwsRLMdKVX7sJYw6OaKHGIcD3icpqy+cBevnd2CmwXdtaPaozn0WD+LyINYRiajTGenmTbj2oAcYoTamDafoinG53gsFs8yxcxnO7SyJyqUB/5wPSF7rbjmbybVBzhqyHUaTImo+4OMCtE+doqOGfN8wFkAVYzVYz0VqWvMT2PMoj1+DpPylfjRutFakhSmpAymmDQZ1ze5Xfng7zmDnPr8qd17CvixOc3JnzWOkyMgAVC92p3ALg== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Jan Beulich , Paul Durrant , =?iso-8859-1?q?Roger_Pau_Monn=E9?= Subject: [RFC PATCH 06/10] xen: pci: print reference counter when dumping pci_devs Thread-Topic: [RFC PATCH 06/10] xen: pci: print reference counter when dumping pci_devs Thread-Index: AQHYvUN/kOplNkP+6UuZAhHgLrcI5Q== Date: Wed, 31 Aug 2022 14:11:00 +0000 Message-ID: <20220831141040.13231-7-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: df64cb83-6dce-4dd5-6313-08da8b5aa1fc x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: PiksFeHyITKAHmW+l2DEzpUlu9y2y4rQS6axx/M5N5hxnvixBNjMo+KbWZ+2UjbcpXi6FsNxd4OfETyMk/JJN8OCvf3jK0g2BqdAgKPASbPavlALnUM2dLePsfk47WTyv0NfHbDzKrbWb2wFDEtD9+xpOBVP8jExKx3gKuZbmNjn/fh5hb7qo+YHNZNL9AuhTFMhrniNWS2ezy6RI4a6/hjWeiJQvtzASMEImaWdTvxoEg6kV2V6fNNNCNGh7FbI2yG9CHwTlsPVRcF7Y1fFthNBLLrij7j2TTjNGoVJfbGogRvMatzVcV8E0Lu9+zI46TjJGTPRp4kdPw3cJKbCXk1CBvI0YaNOSMatNguhhygEj+Xtj+JiI0LWMtiJRG16ezGQWU99O1BGI9Q8htDxZhIff5P0l06kj6iOacaJqoU5Mj6HBaMysrzoEBzS4UWdmurhfH8rrxXyZT9DOj4K2WIRvpHYGbsikWLru9jI0KKZSLCEfIjs93wy+L02MLFQoX3I5QPXR7aAl2jseNvA3EupWa28mNz9RBsGjM7yfb8tV66rr/D+mgBnQu/li35aL82Z6YDlDpBLlW7y/6k4jQ/q2EzZHkEs3+ZYXzzjha+Cy3QXH/mgwdZKuh629c0LjVYx3QcU8MvglsmQ8Rt0sEO4H2/U/zBNFDHQjPCkdHQd2Ae+aClyTMdiw43oG1+PTbjOOZJpxVUAziJiA5qKlbHLBIeE2vv8SX95XqeqY/4BPHrf26L+3jQn4eej8cJuUlPA6wmEadI+jJkUf64xmw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(8936002)(316002)(36756003)(6916009)(4744005)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?IuH/fWW3kxM8p1glEEhKw38?= =?iso-8859-1?q?931IhQclU+0Kj3G65X+qCBUisKwrfKQBYshRkL6584p0EYSOZ7TSAFomV3HN?= =?iso-8859-1?q?YxvC1OVGvqzcgT2aOmxrNY7VIujzSorfyuOWtNDB2WGwWz8YMtcsuVBexmte?= =?iso-8859-1?q?vTe0z2ih3iXxRvOOJXloDDqeQ5anZvgs3wFqI3EJqqMydBRTNoWGeqpRDwSN?= =?iso-8859-1?q?Sz8F6tJ4wnsqyvFlM4IMKDNSiIIwPincdBgUMstIozA65rHmsiHKARAIJuJC?= =?iso-8859-1?q?fCfVMIrj3saqKqiL4xMHSH6+oANj+uLvYpIm1/Ye+EP5nwbnp3anPXtBWLtm?= =?iso-8859-1?q?RC49o5LPY0dI3VhX5IQUDmPddu9ar3WelpdfPnS166EFgz/OmWBdn/vxfzIN?= =?iso-8859-1?q?ynr1STIJGBR88r/YbdjXRxkJ0v4gIwSznvIvWJEEZctC3iB2b6SB7nk6/p05?= =?iso-8859-1?q?pDdrUPz3VdWlAAbuXv2IckzVRn/WLYHa20C6yJ6xoh0KhVV/YMgVGridPiVf?= =?iso-8859-1?q?Zv9jdDAAfcGIbMuiRQmPrwk0iBm4gfZQVXStHZbqNX2Bn7qTKUPNHI+i3IkO?= =?iso-8859-1?q?6bbDJtsaPnmJ8duMRmglHK9+u9tvheV52lNFJEUJgo3jj0AVNaa/RVy7DesS?= =?iso-8859-1?q?oWkqd0DKbBZ2H4tDFnT4QBD8RYhs34VLnycgYdagW/YlfmPNqFGMxNr8goPL?= =?iso-8859-1?q?1AcePVnDTboV1e1V88pMIyEcrc5hwAbKyhwn86ZB7uYM4AGR9Fex5jHHepsW?= =?iso-8859-1?q?ILNiamaHfMIMkeR8qVVCnzd4EEFWI1vBhepB+qtV/6hls1Gr1MsGfMOaZLRb?= =?iso-8859-1?q?1XN6T2t2LiqPjaNhaJDVBOaRBHft91lehzQDsaMrfJbPN+s0TjgJUA6H7j0s?= =?iso-8859-1?q?JsuQGJknGDT9n4/0LwRba/yWeYRn7jWHyuMjmuROeVngKZ8Aq7geWL79qrc7?= =?iso-8859-1?q?PA4NbTZgFrGpfIR2JeX/l7XuLr6Psqie1ICrsWzH/HBSFfRis83NJsQlVnzS?= =?iso-8859-1?q?q28OaOyPOZK1FnxP6BaL9VOnFI01p0AiYoWuWDb35SCdXDckur6ZtmCtIhob?= =?iso-8859-1?q?3cdzT0hbpHTdOOtukNLLprnfYKjE+Uh4Gol9+bRmTOMRBmnV2jD4m5gfXtxs?= =?iso-8859-1?q?wq7qqJ40DxypkNL4Yxkqv+RtiD5UtMKpYNdIT2PcbdBPQC7pgMQYgo4d3caA?= =?iso-8859-1?q?0TibRZ8Yfk+cgku3272ujup4nGBrMfokanXHNN1vYDy0hJYF2b/t8Qi/ZJqs?= =?iso-8859-1?q?0/U3RIxA+TW7rnoQAPk12V7ZGBiZ+m+SmIpN/LNMFfUbWunTkfF/sYeP6hY9?= =?iso-8859-1?q?UJ23YZuKeXPUHi+ElWtSrHEU0MB4hYAObH3YGjQWRYiAmoApi84dcZSlziRd?= =?iso-8859-1?q?cZs+wJYMi3CzD9qd9OnYL3nR6cRLdV4a/SpzaddKh942WDbWi/aS3sjAMmaA?= =?iso-8859-1?q?l3ysb2mJv5i+jJ3g9yTYkmsDOYBIqOuVF0jg5qI+7nFm3RNr8yAid7IiermX?= =?iso-8859-1?q?sHzFw9a2I48Z4VYeSOliq4vYSQdl6rfAGfKkMyeG3h1km8HzcfqcY64kAmr3?= =?iso-8859-1?q?DHCgz07/3mpcU7cuel2GsvdSHJxXSNaHOYKGfMECtisT0urjCn8FeMQzDZ9B?= =?iso-8859-1?q?CQUZ6bkkIs8b2xJap5IZ/dBzKyO+IqJiUE13vvw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: df64cb83-6dce-4dd5-6313-08da8b5aa1fc X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:11:00.7023 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: dgLiXuwiyeoSdXDyP3Y4VB5oTqp7dnSN8ZQLqPW3CvnTRQ1Zp7/Jr3XBHPHRvg3u4FB9nRiUyf1GkgxbcaBT9RjCRS5m/2/tUXVlQRoGRRg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: O2iKEQIqcWqnkjvBNSQL79VFtHSRVH5y X-Proofpoint-ORIG-GUID: O2iKEQIqcWqnkjvBNSQL79VFtHSRVH5y X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=963 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 This can be handy during new reference counter approach evaluation. Signed-off-by: Volodymyr Babchuk --- xen/drivers/passthrough/pci.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index a6c6368769..c8da80b981 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -1381,7 +1381,8 @@ static int cf_check _dump_pci_devices(struct pci_seg *pseg, void *arg) else #endif printk("%pd", pdev->domain); - printk(" - node %-3d", (pdev->node != NUMA_NO_NODE) ? pdev->node : -1); + printk(" - node %-3d refcnt %d", (pdev->node != NUMA_NO_NODE) ? pdev->node : -1, + atomic_read(&pdev->refcnt)); pdev_dump_msi(pdev); printk("\n"); } From patchwork Wed Aug 31 14:11:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960758 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 30784C3DA6B for ; Wed, 31 Aug 2022 14:11:33 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395686.635533 (Exim 4.92) (envelope-from ) id 1oTOR5-0003v9-1Y; Wed, 31 Aug 2022 14:11:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395686.635533; Wed, 31 Aug 2022 14:11:18 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR4-0003uZ-RA; Wed, 31 Aug 2022 14:11:18 +0000 Received: by outflank-mailman (input) for mailman id 395686; Wed, 31 Aug 2022 14:11:17 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR3-0002bw-LK for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:17 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c6347058-2936-11ed-82f2-63bd783d45fa; Wed, 31 Aug 2022 16:11:15 +0200 (CEST) Received: from pps.filterd (m0174676.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBpHDA027047; Wed, 31 Aug 2022 14:11:07 GMT Received: from eur03-ve1-obe.outbound.protection.outlook.com (mail-ve1eur03lp2050.outbound.protection.outlook.com [104.47.9.50]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3j9huk4eny-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:07 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:11:01 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:11:01 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c6347058-2936-11ed-82f2-63bd783d45fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h4KV915eBKmPv6X9qhSpCZKtA7uIzlw5te+xYV8nGdVQB2hOLb9O36P43Bzr6P1xMn/os2hkQkVeFdtbvuWJmMvjeFQWh20gBKWZxPM1aiCc2VJe09OS2dqxjtk5SG7omJC3JWb37eIWR9qL+lwk2on+Ek1MhS+hgPSXFXplL7NUqra9E9GQNojd/zFuJnC7V6iWIK1B7WCZoE1Zskl2AtNfFcezUa22w2RPrCcXFXnFP0krbQ6EojsKgujDAKmgZBgIUIPR8736+ByAdxnHozmtGiRgKqkcBsfmBChPsFdVHUWWcJOHLewHDHNTB4wjOBd33M00oF+bQRQnItSGRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yrA6DJoAo0xAR2GhIIjbwMHD+Xks+V3rsePQNzQTzjY=; b=mI9Cdiz4OPqEhyMisWELZYRfBGE1HbNL4dYo1RK2heqyZkWwGhSupCxMYbkLs0tC50yvzpC1hfWg0wc5doOAZaIPhi/uuf7R9qktQrbSLWfFdqfnY9czIEkZL/MEbVd0qs/JBoFPegdgaxAxp3PLv7xI8Jq6Snf8siALI6Ck0YiuttRXZoMfXOPqLZiTqmMYTTX7NaEYVplBgdYookjTnF4lOklL5+2GMZND9KKKTipiuhxlQ0xj4mlvRdJKAbKmq5HnMJa2iZdo0RP7CLfVvuR4Vu9KWVEd6w4uVox+FnPXn5UkOC7/9Vm1jsQKbvH11UBDTQ6oChVPqai2LGMqTg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yrA6DJoAo0xAR2GhIIjbwMHD+Xks+V3rsePQNzQTzjY=; b=KEzOEEs/eB4HJf5mOcA6dF4hDdzacwbh4cA/8+EtU3Y1E+gHT5jViiwB+9BAaNxBfBI7KdDodhrH6ygpGzmSqWrYr1vA4qXDv7I6nuJZFrIjtvnPd1tReAKlLWY6bDoi+i7BVQkpvLg1HmqkUMiyr1HD5afO13tXFQQnew7HYBkJgL/Y6ZexLdcCy88hQSB6GFkBRd7KwK4DG9dip4m93EhoVL1OQB9cDFukKKHyFUa5xEfZ7q1t1yK6cWPHcz2AChqtXOu3z0cow9Cc08s/ULEmamY4Zdz/Zw4oR4IJpuu117J6biEz52IXAUmJL8DG5C4DrKQQGy5ECMNX4VXGSQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Jan Beulich , Andrew Cooper , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Paul Durrant Subject: [RFC PATCH 07/10] xen: pci: add per-device locking Thread-Topic: [RFC PATCH 07/10] xen: pci: add per-device locking Thread-Index: AQHYvUN/+u7dK30KdkO8lqgvo0jPyQ== Date: Wed, 31 Aug 2022 14:11:01 +0000 Message-ID: <20220831141040.13231-8-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 43a02f0f-3cb0-494a-9a8c-08da8b5aa235 x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ypmqomZ9oE1BH1RluJkJFv9w9ikf4j1BR/1mbksiztkZak2o2tlEtGtO//s2Rx3CNEgzgTEsGxBfHKP0D5noRZ/QjS2VybdxFMgsk0ViSQkx6Pc4B0thlTyq79gYMvObmvW6di3/CvMMd5d9hK3WLHAoXYDeboM/TTArKk5vmL+py++pJjSSGdtLIbFkK2UZS4TqIqPLStbpDjDsVYsoqd73v7N6p+PykaITyWrxMMH30eXe1fvkcVU1Zzz4SErSfwht9e2C8hBPR0yrgADrclu+zKIwBGttmELAJdwWlL+lBbuOZDU2sLd2IEDPsKHuE1opLwBIjwh4ArUqA0MtRerRug6Vq5WypgMk3C3tLTb37BG2y1Evv84ZwbyShySkpSVOQTrfc3czpEInP2SeFgjLJl5gCcd2/FlqiqIKFO2h8h5Gb75K2BLPv6G173qATPC/mNzd3XP6t5JeihAhqZPSNg5Y1h2wky831Pardk/aezyQO6pK2Lw9c3ehw+tIiS93LzNyqu+hiEesh4PEAS6qYE8G8piQHq1cNf0N75bi57gv1ElsgDGLmLBPRcqI+3/tjSaycvTvH77yqjp2+GYYqCmbsNXxubgDMSHK24pnHZXI49/KPdXNqeEhafHsZ0wjJI0eTNXKnz85DkytIOOQ0Jel9VQk9pR2JBKNF7/43J+vZbOVA4fFm2+bF7YvHTm2zT2VGUIBmlUxH8sOYBopmEOuUvcGtNRKPvRtOIrt4GeRYILlrewls4G6mo8Cej85PGtzuoi5/HQiFhlvkQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(8936002)(316002)(36756003)(6916009)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?LFDZxt1POjjo7aKVoH+jtGN?= =?iso-8859-1?q?IfduiQp1kN0XIqFKnBQQGCgfnuzwoMjoaVRsxtiPcmz/qHENmdoQnPV4Fsh9?= =?iso-8859-1?q?JXfhBrwobtZ9WqX/HCNEjw+4wM0L1p/Gf/UCpX+6UmbJxRcZgKRT59/ga4of?= =?iso-8859-1?q?yUWAPJzEdQf+Ldl8d83obNyOWtfPaRA/gvNXo9VgAmrUWsap0+iMVC9NHkIo?= =?iso-8859-1?q?vrCbW9sAlcjbjhOZGaz1WhE62FH9SPK61ZRlD8eIhaAI0l+AWJ8fq4wjpYAl?= =?iso-8859-1?q?sXYD/qBJUfzp/bUdRc8h6wp6VYUGU5nN/qC8MowlupQuH0SRemmiWOZlYxQg?= =?iso-8859-1?q?VAaSlxxDT6lSQVVGMcm6/pD9KyFPD99L/+p7T5aK3rqIF2nUsnU51BgipLzf?= =?iso-8859-1?q?fwzzZscE3MWYcH2khvNGMfHO9HfZCUO6oj6JnMpIy5afsCvXg2QgsabMMfS0?= =?iso-8859-1?q?/XisrgE+NX7ee8g1I97CdPDEVnfpWbMGDuwVtPqHVnM6ssfrZeSWYQfNR7YY?= =?iso-8859-1?q?Cnk9ek3zKKPaSceCB/jbTmGvvpKVqsU3ic9etlX+3WCUBzsCqcwG81PUmxlV?= =?iso-8859-1?q?vkHMuUHqFS/p8aQ1qQIEqC7jwqRoSJ/fOV3qNv3/7W+oe5z74W2zjq/C4imt?= =?iso-8859-1?q?nny7AyuJLgaDMWdhui5gh5zPTU6O6nTYdq8hI8s92yK/4YgJVQR1jj4gi0DD?= =?iso-8859-1?q?kOTplJxH92caogWECwN9X32VvC+A/LGECdtE8hUsr8qBshBMNBeiJ0c7i5M4?= =?iso-8859-1?q?OBhpoIOxd5GlBJMwbzaUhpKODFAuDW63N0+tilu8KK+Wxnl0soETncKIswyZ?= =?iso-8859-1?q?d29ezj03X/eSon9K1D9OGC4I9eNntl7GXyQTUhqpokC6wQ8rOaY31AvxxTGr?= =?iso-8859-1?q?TJfN6Txuis+SK3KkPuHbgRFg6W+RasVgLhjm5FcH4umFxRJE3kHvgl4MMarq?= =?iso-8859-1?q?yEIE1mtm+EZHZJdij37bbnYo98ppV8iwcQ9nAITQvhYovWFbh9gbx8PDDvw9?= =?iso-8859-1?q?9qeHOPCShKxP8nDiDZr6Ncj/CixDHRehl78V6SPhzzLDh89YuBNPqfN2asir?= =?iso-8859-1?q?NLu7qc84wbg4ciAesMq9Hx+DyaCkYzJOGPFr0/nXpGHGsRMU2I+XHkFhejAK?= =?iso-8859-1?q?vacB7GAYkKXTiW9lDIcZdYNfRUssEBpGyLLpm3unIU1RFWQZOr3kDRocfxYl?= =?iso-8859-1?q?XVbkYfDCRa7H7eRPSyxzntS1S8q5iF3nR+NHeXEWyv9JIk7JHAfMWK1QMYgS?= =?iso-8859-1?q?4mNFPvWEPbFhXwG0PJ4jvwu0V6e7CFu2sIob7llUGOEvTQf0/mNraroswj9P?= =?iso-8859-1?q?/NtgwnvyLnYLhM0HQ9mkVQitZBVBJwInsqfL1z3b7WkWyE33fW1J/dGxAqCX?= =?iso-8859-1?q?H0Ape70mqZ1HeeJfto9JHAcgr7IIduPmKjsD4la4wyolCuToHMS1MGvymTQ3?= =?iso-8859-1?q?BKYptdRIwXy44KQm90+HiAo0s3B5hMxDrdIUQ5affsXsdPrVKRpNakqNCv5q?= =?iso-8859-1?q?0Kj8b8xzUW+lblysrYu/vJ/qHV0Ou7wyGZIaUjwyaJS/ik0qdxT3ipkW2pxW?= =?iso-8859-1?q?IzMYOL4Y0YWFBExoYu0tQDSGfwuswcdOGZRi30A7Yo3QrFj3ic6wR7uJ1lro?= =?iso-8859-1?q?ins9vayrKv921deaGmO+Jc58VjrhBolq8wbCspg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 43a02f0f-3cb0-494a-9a8c-08da8b5aa235 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:11:01.0617 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: jr7+mXjxvCEO/VS+k98EWxW3W7Ab3qMbYSNjvNx1RcVM4f3pRKQHXMnJ4ykeEhSuLrApGLQ3ZORm6ApcbjTdAX2W393FW2RK5u8MhjL7AoY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: TOWzXLhJLb6IVSkGjodKjVQZGU-iHdEN X-Proofpoint-ORIG-GUID: TOWzXLhJLb6IVSkGjodKjVQZGU-iHdEN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 clxscore=1015 impostorscore=0 mlxlogscore=727 priorityscore=1501 lowpriorityscore=0 phishscore=0 suspectscore=0 mlxscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 Spinlock in struct pci_device will be used to protect access to device itself. Right now it is used mostly by MSI code. Signed-off-by: Volodymyr Babchuk --- xen/arch/x86/hvm/vmsi.c | 6 +++++- xen/arch/x86/msi.c | 16 ++++++++++++++++ xen/drivers/passthrough/msi.c | 8 +++++++- xen/drivers/passthrough/pci.c | 2 ++ xen/include/xen/pci.h | 12 ++++++++++++ 5 files changed, 42 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c index 7fb1075673..c9e5f279c5 100644 --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -203,10 +203,14 @@ static struct msi_desc *msixtbl_addr_to_desc( nr_entry = (addr - entry->gtable) / PCI_MSIX_ENTRY_SIZE; + pcidev_lock(entry->pdev); list_for_each_entry( desc, &entry->pdev->msi_list, list ) if ( desc->msi_attrib.type == PCI_CAP_ID_MSIX && - desc->msi_attrib.entry_nr == nr_entry ) + desc->msi_attrib.entry_nr == nr_entry ) { + pcidev_unlock(entry->pdev); return desc; + } + pcidev_unlock(entry->pdev); return NULL; } diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c index bccaccb98b..6b62c4f452 100644 --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -389,6 +389,7 @@ static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest) default: return 0; } + entry->msi_attrib.host_masked = host; entry->msi_attrib.guest_masked = guest; @@ -585,12 +586,17 @@ static struct msi_desc *find_msi_entry(struct pci_dev *dev, { struct msi_desc *entry; + pcidev_lock(dev); list_for_each_entry( entry, &dev->msi_list, list ) { if ( entry->msi_attrib.type == cap_id && (irq == -1 || entry->irq == irq) ) + { + pcidev_unlock(dev); return entry; + } } + pcidev_unlock(dev); return NULL; } @@ -661,7 +667,9 @@ static int msi_capability_init(struct pci_dev *dev, maskbits |= ~(uint32_t)0 >> (32 - dev->msi_maxvec); pci_conf_write32(dev->sbdf, mpos, maskbits); } + pcidev_lock(dev); list_add_tail(&entry->list, &dev->msi_list); + pcidev_unlock(dev); *desc = entry; /* Restore the original MSI enabled bits */ @@ -946,7 +954,9 @@ static int msix_capability_init(struct pci_dev *dev, pcidev_get(dev); + pcidev_lock(dev); list_add_tail(&entry->list, &dev->msi_list); + pcidev_unlock(dev); *desc = entry; } @@ -1231,11 +1241,13 @@ static void msi_free_irqs(struct pci_dev* dev) { struct msi_desc *entry, *tmp; + pcidev_lock(dev); list_for_each_entry_safe( entry, tmp, &dev->msi_list, list ) { pci_disable_msi(entry); msi_free_irq(entry); } + pcidev_unlock(dev); } void pci_cleanup_msi(struct pci_dev *pdev) @@ -1354,6 +1366,7 @@ int pci_restore_msi_state(struct pci_dev *pdev) if ( ret ) return ret; + pcidev_lock(pdev); list_for_each_entry_safe( entry, tmp, &pdev->msi_list, list ) { unsigned int i = 0, nr = 1; @@ -1371,6 +1384,7 @@ int pci_restore_msi_state(struct pci_dev *pdev) dprintk(XENLOG_ERR, "Restore MSI for %pp entry %u not set?\n", &pdev->sbdf, i); spin_unlock_irqrestore(&desc->lock, flags); + pcidev_unlock(pdev); if ( type == PCI_CAP_ID_MSIX ) pci_conf_write16(pdev->sbdf, msix_control_reg(pos), control & ~PCI_MSIX_FLAGS_ENABLE); @@ -1393,6 +1407,7 @@ int pci_restore_msi_state(struct pci_dev *pdev) if ( unlikely(!memory_decoded(pdev)) ) { spin_unlock_irqrestore(&desc->lock, flags); + pcidev_unlock(pdev); pci_conf_write16(pdev->sbdf, msix_control_reg(pos), control & ~PCI_MSIX_FLAGS_ENABLE); return -ENXIO; @@ -1438,6 +1453,7 @@ int pci_restore_msi_state(struct pci_dev *pdev) pci_conf_write16(pdev->sbdf, msix_control_reg(pos), control | PCI_MSIX_FLAGS_ENABLE); + pcidev_unlock(pdev); return 0; } diff --git a/xen/drivers/passthrough/msi.c b/xen/drivers/passthrough/msi.c index ce1a450f6f..98f4d2721a 100644 --- a/xen/drivers/passthrough/msi.c +++ b/xen/drivers/passthrough/msi.c @@ -22,6 +22,7 @@ int pdev_msi_init(struct pci_dev *pdev) { unsigned int pos; + pcidev_lock(pdev); INIT_LIST_HEAD(&pdev->msi_list); pos = pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->devfn), @@ -41,7 +42,10 @@ int pdev_msi_init(struct pci_dev *pdev) uint16_t ctrl; if ( !msix ) - return -ENOMEM; + { + pcidev_unlock(pdev); + return -ENOMEM; + } spin_lock_init(&msix->table_lock); @@ -51,6 +55,8 @@ int pdev_msi_init(struct pci_dev *pdev) pdev->msix = msix; } + pcidev_unlock(pdev); + return 0; } diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index c8da80b981..c83397211b 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -1383,7 +1383,9 @@ static int cf_check _dump_pci_devices(struct pci_seg *pseg, void *arg) printk("%pd", pdev->domain); printk(" - node %-3d refcnt %d", (pdev->node != NUMA_NO_NODE) ? pdev->node : -1, atomic_read(&pdev->refcnt)); + pcidev_lock(pdev); pdev_dump_msi(pdev); + pcidev_unlock(pdev); printk("\n"); } spin_unlock(&pseg->alldevs_lock); diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h index e71a180ef3..d0a7339d84 100644 --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -106,6 +106,8 @@ struct pci_dev { uint8_t msi_maxvec; uint8_t phantom_stride; + /* Device lock */ + spinlock_t lock; nodeid_t node; /* NUMA node */ /* Device to be quarantined, don't automatically re-assign to dom0 */ @@ -235,6 +237,16 @@ int msixtbl_pt_register(struct domain *, struct pirq *, uint64_t gtable); void msixtbl_pt_unregister(struct domain *, struct pirq *); void msixtbl_pt_cleanup(struct domain *d); +static inline void pcidev_lock(struct pci_dev *pdev) +{ + spin_lock(&pdev->lock); +} + +static inline void pcidev_unlock(struct pci_dev *pdev) +{ + spin_unlock(&pdev->lock); +} + #ifdef CONFIG_HVM int arch_pci_clean_pirqs(struct domain *d); #else From patchwork Wed Aug 31 14:11:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960762 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B5642ECAAD4 for ; Wed, 31 Aug 2022 14:11:35 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395690.635570 (Exim 4.92) (envelope-from ) id 1oTORA-0004wP-E6; Wed, 31 Aug 2022 14:11:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395690.635570; Wed, 31 Aug 2022 14:11:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTORA-0004u6-4T; Wed, 31 Aug 2022 14:11:24 +0000 Received: by outflank-mailman (input) for mailman id 395690; Wed, 31 Aug 2022 14:11:22 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR8-0002bv-HB for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:22 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c8a5b30f-2936-11ed-934f-f50d60e1c1bd; Wed, 31 Aug 2022 16:11:20 +0200 (CEST) Received: from pps.filterd (m0174677.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBo4GF032537; Wed, 31 Aug 2022 14:11:08 GMT Received: from eur04-vi1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3ja427h4rd-8 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:08 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:11:01 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:11:01 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c8a5b30f-2936-11ed-934f-f50d60e1c1bd ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BylhPzfMiZD/a3ix4uXkaPyXO2X3+A4M8LLD/FrCC5n455S/y5awC784zdtc4QDGC969dPhCvf9v6CsBUKVPphMBBUhBSZY/Lua4mr+L1/mbbt+R1LVWGfvnu987g9yJQY8F635tZBwXFBDSIPvYuU+ogo7TPQYJlCY7qI1niG5atLrUrJzJfSvFyArAtbmUj1PVMY46iKTdLe7mHoi2SfFQTKYj3UiuJlIfGnqN7JcMBK2yRwXxun5Rb4C9AJdBiwCkwoJk5XWMI0YdD9Qh2TqBP6bcPTqlBjlH9XcGwthXU7wuY+60FHLAm1xe2FlLWxsYRJIPmNN+JG0NL4Xqfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LiizKjLCyBEAFE8BknfyWCXpKCtSOHBlrRlnh2irPYc=; b=CJXfYLGxTiE7nZ/3XDUXjQZi10iID28k4aA3glU1mdyrg5ZBB5NslWilecEntzy8LCpBw7Oa5LcbFwqbWkpr2T+1wEfUJit4XPSj+10ZDIXqHjaAV7fdu65xwYDopo1GgUszPLpgOOXS3hfEmcY82ZpKS9kvPKiPSCdeF3uQGFOmpIXWnk/8ukxVwQN5xZkhNmIX3hsXsTBXjqpLfGvlgbdkO9+DwXZq1J0KEML+00xWoNRmvZorqhtt7CQtrDgSckK+f7JbWaKDFvOlJv8stZebpir2Mt9H3iH9/0nP+RfqmxOzuA4JuFWxojmnAQAz02SchXuMkgN2jpUA5ALI0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LiizKjLCyBEAFE8BknfyWCXpKCtSOHBlrRlnh2irPYc=; b=rnA3SGKVZ6CjI2yzG8Zo6giGzXYWPfCAGw1PGRDpErlajt6xXe1AbBVQq1AjJbV6/gTW5Q0im23kwxxQFxluVKX7U2jYr/HW+MJ8QRsBPjWxVaL3TDDDrHPrJNfU0VpygEYGT8P+A44V8ap/T4UiepZOECk5Qn3agas13lQKwN3D9YAMMt4sEdzpvJI7VzzYtx/UYMvDNw19rLmZU0x50KTalu+wcVacd0quKeeJ7IKPFYosnjSaQqa3x2jz3x1vfKhOvWa7golH8Az5cadabLKAPB+vab+ySdtpb/vQG3H0TXzPSH+qJgFUcPCNkJGluP2hPHszwXwTDrgpRr9PIA== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Jan Beulich , Andrew Cooper , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Paul Durrant , Kevin Tian Subject: [RFC PATCH 08/10] xen: pci: remove pcidev_[un]lock[ed] calls Thread-Topic: [RFC PATCH 08/10] xen: pci: remove pcidev_[un]lock[ed] calls Thread-Index: AQHYvUN/BRbyNewJG0OqhFjBUWUt7Q== Date: Wed, 31 Aug 2022 14:11:01 +0000 Message-ID: <20220831141040.13231-9-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 43e7eaa6-475c-4549-5b42-08da8b5aa278 x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: fY4dwPhqrYUvR6va25FXVt+QM+VZvdj/8671K3X8ygBNS8GyBFVd4Tnb0eYZ13rFfQ82xwWmtgEOKJDUOnW99KAkYnIiN0wz6F+ZLVFxQiZH4fGc8XYq6aTxo8DYjF6oMioMyXetZvfhhLGnSfTqYvMDKUQoxeTj8KpxsMtQHWkCtKfBOU6AimMOfzRnNFZqBHeo0GW1hrnNHDqKXujxBD0k6hz6/DE2xuKIyq3QiMXhrm46Bu/repkzaXY5rlwFB0ZPqQAqmhfiW8IXuII7u8e/PYbgYyB5e2X/WJ1v9SwNq1/JCkbBNock2S/J6XxGgKXoe7xSt043PGK/0RFPE8C2ScQYEq/GTvfCJJ/1pplRTGeU1FU3yImC0zqRfMM1rWcbW+lH4tWVdpRwsub7bArMoMwuNBx0w+LvE0S9aqLQSV96FW2qOf1YMK2BLB1XpdFosZR5YvPD4AqM9gQpM3GwJThWMKeKEXAOIK8K6SGt77poFnuxUOjge9RRUtXa4y6YgrIUdgRs6nCiBlLvQ9A36ahoVt96lhwOlGkNHgEaj0M0y+tMeSI/DDlQk9H6K/pynev29HI9Zl7Gc3N3pjurEoiOiFpYIBsbjj/aRhxXlzuuefnzpB8iG/3kgy2g9tcOR6ubl+P1NDInzdADLoALvADaxcFaQ8hElezNJW+J7pX+nDMH+etidJm6nwjgH9Vb5TzQXbnpb46Tc4p1r98jfkAj+AGBaRlxbqVE+tCBD5GsLNG7wexAH3VrclRzBPeyjO9O9/hTlSRDnv4otYKKXiTkTID2bl1qTCyOg7s= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(7416002)(8936002)(316002)(36756003)(6916009)(30864003)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002)(21314003)(579004)(559001);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?i4WCKy2gPNX2cRjthJVXDpy?= =?iso-8859-1?q?y1ku0SH2n0McEZn93mJlYxXtEwFeZ5ObNqm4Ixgl9gRE+mRGUC7vql0i+e7K?= =?iso-8859-1?q?li/FvFFQFieUSZiPd63uzwKM3GTzSLPcQS+bsbUCPbZzbZzymXEl6FqY5joW?= =?iso-8859-1?q?gyGfLGbP/f1TZOQOyL63L4ynjLAfx+EnEeZQX/tlrYGXTR/U580FxvZt6VSR?= =?iso-8859-1?q?QG7nTBtASmvINhpKYgKQmk6gqPh4p7szYh4WJJPLgvYCeYpUUrgFfF91EPXV?= =?iso-8859-1?q?9lDheIl5RikH+/hT8G9XwynH24Lsy+RqiMBbLpFEMr0qzFCF7LLejGocbQ/v?= =?iso-8859-1?q?2PrXyc5MxQsc/2MAiu7MrObvaWREpGLlMM6LkBq/U3b0P8iQI7ABglN3NahR?= =?iso-8859-1?q?MPNv5WzwO2x1VoD8W4c6neYk9bRfYomU2o9f8fLUzLpQ6JGnl/qaFgCFhenF?= =?iso-8859-1?q?0DCtTqTec68fmpyfp/doD1OqqJ1KfGuKzv3U6rYanmELLJgEh5weFSuVq6GD?= =?iso-8859-1?q?Cr2CSctS3lLTjQWxgr2PEk+vXyKXKapAAdL6Pz6O01AYjOUJCy5ekc2TfsOb?= =?iso-8859-1?q?EmgkVXoexTSOWL2Ygn92ZRcwb1ZEnvcExVAZAzjkulNoKhEzXGGyAKPCuRMt?= =?iso-8859-1?q?Q0KNj+4DyObgUaI4CBcQAodfFLsycqels3ymWQSnp7PfzpNlXd4D2KGyMX9h?= =?iso-8859-1?q?8C8G02wnkk7oxDbS/hq/4RgKdvCNH+Q7Z/Q3kaYcYxEVRKyCrQ97Ncr3/hjy?= =?iso-8859-1?q?JpmGiXMEDHRpNYQWMUuTyW5J6k8LSL1u5ZlrSNJmecCrymadyFDoJZqnryDr?= =?iso-8859-1?q?KE5X3mkzuZboSPUn5sllnELu5Fn7Kqy50sFq8tRHb9f/K7TWOwtkTfrhR0sA?= =?iso-8859-1?q?qk9OkmtVHv0HDhmjnOriteBqblujvHXMALW9RVok90wr0YJixOADqAE0/ExI?= =?iso-8859-1?q?b+DjeIpF6XmycKVjL62ah3CiWfVvLwUsmVY8660pweR9a/CJXLEPrI6N+O7G?= =?iso-8859-1?q?WKvmH0XkHcgGYi8Hv0WPDx9x0pN5QTZrjsbfPH+EtQdKKiBZbwQb26L9tQgC?= =?iso-8859-1?q?wjum8HzGvv4YBk58D721CLLYjXx5ukKSnt3sW4d/0vXH0YJW5Vsqm1fvHB/t?= =?iso-8859-1?q?uV6EFKmC9pHjVP25EpoIuOUvIih+fGsgh6+3wqgux2w2saxjtlZ8Ib9vtY9g?= =?iso-8859-1?q?i3ujmrp7qmBbS9ajV+4nTLD+1Rf7fIZWP74rXlYhV4MbOABfHu8siQgiUVW7?= =?iso-8859-1?q?ld0Qi+7KzzRIUTrMWNHDc21/wvmQ++RoXYP1PIe3vsGIgqBjDu3EOVQgDMpW?= =?iso-8859-1?q?zQ4M86Z4LwboFTRWrzggqYoihUSBxj01t92aclK6QaKZsyHWSxsypwuvI+1u?= =?iso-8859-1?q?8MVS8tk/Iie1FY2v/0DlzEU8Abmv2cmlwD3zlfUzwqNtIEpfTP51Xm978n+Q?= =?iso-8859-1?q?IUOUEMfYp5uos8VRK4Nfst/2WmDkbC/KDaXuETE39lnFk3ePLJYMZQ7TmGbf?= =?iso-8859-1?q?oKlAYVNBn9lgk0JJMxlT7LknL2gfQLhTUXwGi70pD7BFNrwkvbtnSj21ATDq?= =?iso-8859-1?q?wttNCkcj2ych5pXnqM5UZYuepFZMiMfC0Y7KfY+Kwro3kiiO79rAcIJigRBS?= =?iso-8859-1?q?TQlXLiYsTZb6jPKNYdRjh9xWOTcUy9GdeG7rTbg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 43e7eaa6-475c-4549-5b42-08da8b5aa278 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:11:01.4991 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Pz8Nsf2rqRneOLWVXiES3m1n/mKV12QukkHumg5mwgbOxelyIBIzLSQMr1mWLDwsWUGz0qXJOZweu9Tfw9mKDVTzwcTgCLcac4PLVocEi+I= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: ejVwXPRGnwer7_PrxgkE1CwtrtoWDyIO X-Proofpoint-ORIG-GUID: ejVwXPRGnwer7_PrxgkE1CwtrtoWDyIO X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=999 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 As pci devices are refcounted now and all list that store them are protected by separate locks, we can safely drop global pcidevs_lock. Signed-off-by: Volodymyr Babchuk --- xen/arch/x86/domctl.c | 8 --- xen/arch/x86/hvm/vioapic.c | 2 - xen/arch/x86/hvm/vmsi.c | 12 ---- xen/arch/x86/irq.c | 7 --- xen/arch/x86/msi.c | 11 ---- xen/arch/x86/pci.c | 4 -- xen/arch/x86/physdev.c | 7 +-- xen/common/sysctl.c | 2 - xen/drivers/char/ns16550.c | 4 -- xen/drivers/passthrough/amd/iommu_init.c | 7 --- xen/drivers/passthrough/amd/iommu_map.c | 5 -- xen/drivers/passthrough/amd/pci_amd_iommu.c | 4 -- xen/drivers/passthrough/pci.c | 63 +-------------------- xen/drivers/passthrough/vtd/iommu.c | 8 --- xen/drivers/video/vga.c | 2 - 15 files changed, 4 insertions(+), 142 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 020df615bd..9f4ca03385 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -537,11 +537,7 @@ long arch_do_domctl( ret = -ESRCH; if ( is_iommu_enabled(d) ) - { - pcidevs_lock(); ret = pt_irq_create_bind(d, bind); - pcidevs_unlock(); - } if ( ret < 0 ) printk(XENLOG_G_ERR "pt_irq_create_bind failed (%ld) for dom%d\n", ret, d->domain_id); @@ -566,11 +562,7 @@ long arch_do_domctl( break; if ( is_iommu_enabled(d) ) - { - pcidevs_lock(); ret = pt_irq_destroy_bind(d, bind); - pcidevs_unlock(); - } if ( ret < 0 ) printk(XENLOG_G_ERR "pt_irq_destroy_bind failed (%ld) for dom%d\n", ret, d->domain_id); diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c index cb7f440160..aa4e7766a3 100644 --- a/xen/arch/x86/hvm/vioapic.c +++ b/xen/arch/x86/hvm/vioapic.c @@ -197,7 +197,6 @@ static int vioapic_hwdom_map_gsi(unsigned int gsi, unsigned int trig, return ret; } - pcidevs_lock(); ret = pt_irq_create_bind(currd, &pt_irq_bind); if ( ret ) { @@ -207,7 +206,6 @@ static int vioapic_hwdom_map_gsi(unsigned int gsi, unsigned int trig, unmap_domain_pirq(currd, pirq); write_unlock(&currd->event_lock); } - pcidevs_unlock(); return ret; } diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c index c9e5f279c5..344bbd646c 100644 --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -470,7 +470,6 @@ int msixtbl_pt_register(struct domain *d, struct pirq *pirq, uint64_t gtable) struct msixtbl_entry *entry, *new_entry; int r = -EINVAL; - ASSERT(pcidevs_locked()); ASSERT(rw_is_write_locked(&d->event_lock)); if ( !msixtbl_initialised(d) ) @@ -540,7 +539,6 @@ void msixtbl_pt_unregister(struct domain *d, struct pirq *pirq) struct pci_dev *pdev; struct msixtbl_entry *entry; - ASSERT(pcidevs_locked()); ASSERT(rw_is_write_locked(&d->event_lock)); if ( !msixtbl_initialised(d) ) @@ -686,8 +684,6 @@ static int vpci_msi_update(const struct pci_dev *pdev, uint32_t data, { unsigned int i; - ASSERT(pcidevs_locked()); - if ( (address & MSI_ADDR_BASE_MASK) != MSI_ADDR_HEADER ) { gdprintk(XENLOG_ERR, "%pp: PIRQ %u: unsupported address %lx\n", @@ -728,7 +724,6 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev) ASSERT(msi->arch.pirq != INVALID_PIRQ); - pcidevs_lock(); for ( i = 0; i < msi->vectors && msi->arch.bound; i++ ) { struct xen_domctl_bind_pt_irq unbind = { @@ -747,7 +742,6 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev) msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address, msi->vectors, msi->arch.pirq, msi->mask); - pcidevs_unlock(); } static int vpci_msi_enable(const struct pci_dev *pdev, unsigned int nr, @@ -785,10 +779,8 @@ int vpci_msi_arch_enable(struct vpci_msi *msi, const struct pci_dev *pdev, return rc; msi->arch.pirq = rc; - pcidevs_lock(); msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address, vectors, msi->arch.pirq, msi->mask); - pcidevs_unlock(); return 0; } @@ -800,7 +792,6 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq, ASSERT(pirq != INVALID_PIRQ); - pcidevs_lock(); for ( i = 0; i < nr && bound; i++ ) { struct xen_domctl_bind_pt_irq bind = { @@ -816,7 +807,6 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq, write_lock(&pdev->domain->event_lock); unmap_domain_pirq(pdev->domain, pirq); write_unlock(&pdev->domain->event_lock); - pcidevs_unlock(); } void vpci_msi_arch_disable(struct vpci_msi *msi, const struct pci_dev *pdev) @@ -863,7 +853,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry, entry->arch.pirq = rc; - pcidevs_lock(); rc = vpci_msi_update(pdev, entry->data, entry->addr, 1, entry->arch.pirq, entry->masked); if ( rc ) @@ -871,7 +860,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry, vpci_msi_disable(pdev, entry->arch.pirq, 1, false); entry->arch.pirq = INVALID_PIRQ; } - pcidevs_unlock(); return rc; } diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index d8672a03e1..6a08830a55 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2156,8 +2156,6 @@ int map_domain_pirq( struct pci_dev *pdev; unsigned int nr = 0; - ASSERT(pcidevs_locked()); - ret = -ENODEV; if ( !cpu_has_apic ) goto done; @@ -2317,7 +2315,6 @@ int unmap_domain_pirq(struct domain *d, int pirq) if ( (pirq < 0) || (pirq >= d->nr_pirqs) ) return -EINVAL; - ASSERT(pcidevs_locked()); ASSERT(rw_is_write_locked(&d->event_lock)); info = pirq_info(d, pirq); @@ -2423,7 +2420,6 @@ void free_domain_pirqs(struct domain *d) { int i; - pcidevs_lock(); write_lock(&d->event_lock); for ( i = 0; i < d->nr_pirqs; i++ ) @@ -2431,7 +2427,6 @@ void free_domain_pirqs(struct domain *d) unmap_domain_pirq(d, i); write_unlock(&d->event_lock); - pcidevs_unlock(); } static void cf_check dump_irqs(unsigned char key) @@ -2911,7 +2906,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p, msi->irq = irq; - pcidevs_lock(); /* Verify or get pirq. */ write_lock(&d->event_lock); pirq = allocate_pirq(d, index, *pirq_p, irq, type, &msi->entry_nr); @@ -2927,7 +2921,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p, done: write_unlock(&d->event_lock); - pcidevs_unlock(); if ( ret ) { switch ( type ) diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c index 6b62c4f452..f04b90e235 100644 --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -623,7 +623,6 @@ static int msi_capability_init(struct pci_dev *dev, u8 slot = PCI_SLOT(dev->devfn); u8 func = PCI_FUNC(dev->devfn); - ASSERT(pcidevs_locked()); pos = pci_find_cap_offset(seg, bus, slot, func, PCI_CAP_ID_MSI); if ( !pos ) return -ENODEV; @@ -810,8 +809,6 @@ static int msix_capability_init(struct pci_dev *dev, if ( !pos ) return -ENODEV; - ASSERT(pcidevs_locked()); - control = pci_conf_read16(dev->sbdf, msix_control_reg(pos)); /* * Ensure MSI-X interrupts are masked during setup. Some devices require @@ -1032,7 +1029,6 @@ static int __pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) struct msi_desc *old_desc; int ret; - ASSERT(pcidevs_locked()); pdev = pci_get_pdev(NULL, msi->sbdf); if ( !pdev ) return -ENODEV; @@ -1092,7 +1088,6 @@ static int __pci_enable_msix(struct msi_info *msi, struct msi_desc **desc) struct msi_desc *old_desc; int ret; - ASSERT(pcidevs_locked()); pdev = pci_get_pdev(NULL, msi->sbdf); if ( !pdev || !pdev->msix ) return -ENODEV; @@ -1191,7 +1186,6 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off) if ( !use_msi ) return 0; - pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); if ( !pdev ) rc = -ENODEV; @@ -1204,7 +1198,6 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off) } else rc = msix_capability_init(pdev, NULL, NULL); - pcidevs_unlock(); pcidev_put(pdev); @@ -1217,8 +1210,6 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off) */ int pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) { - ASSERT(pcidevs_locked()); - if ( !use_msi ) return -EPERM; @@ -1355,8 +1346,6 @@ int pci_restore_msi_state(struct pci_dev *pdev) unsigned int type = 0, pos = 0; u16 control = 0; - ASSERT(pcidevs_locked()); - if ( !use_msi ) return -EOPNOTSUPP; diff --git a/xen/arch/x86/pci.c b/xen/arch/x86/pci.c index 1d38f0df7c..4dcd6d96f3 100644 --- a/xen/arch/x86/pci.c +++ b/xen/arch/x86/pci.c @@ -88,15 +88,11 @@ int pci_conf_write_intercept(unsigned int seg, unsigned int bdf, if ( reg < 64 || reg >= 256 ) return 0; - pcidevs_lock(); - pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bdf)); if ( pdev ) { rc = pci_msi_conf_write_intercept(pdev, reg, size, data); pcidev_put(pdev); } - pcidevs_unlock(); - return rc; } diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c index 96214a3d40..a41366b609 100644 --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -162,11 +162,9 @@ int physdev_unmap_pirq(domid_t domid, int pirq) goto free_domain; } - pcidevs_lock(); write_lock(&d->event_lock); ret = unmap_domain_pirq(d, pirq); write_unlock(&d->event_lock); - pcidevs_unlock(); free_domain: rcu_unlock_domain(d); @@ -530,7 +528,6 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) if ( copy_from_guest(&restore_msi, arg, 1) != 0 ) break; - pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(0, restore_msi.bus, restore_msi.devfn)); if ( pdev ) @@ -541,7 +538,6 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) else ret = -ENODEV; - pcidevs_unlock(); break; } @@ -553,7 +549,6 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) if ( copy_from_guest(&dev, arg, 1) != 0 ) break; - pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(dev.seg, dev.bus, dev.devfn)); if ( pdev ) { @@ -562,7 +557,7 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) } else ret = -ENODEV; - pcidevs_unlock(); + break; } diff --git a/xen/common/sysctl.c b/xen/common/sysctl.c index 0feef94cd2..6bb8c5c295 100644 --- a/xen/common/sysctl.c +++ b/xen/common/sysctl.c @@ -446,7 +446,6 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl) break; } - pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(dev.seg, dev.bus, dev.devfn)); if ( !pdev ) node = XEN_INVALID_DEV; @@ -454,7 +453,6 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl) node = XEN_INVALID_NODE_ID; else node = pdev->node; - pcidevs_unlock(); if ( pdev ) pcidev_put(pdev); diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c index 01a05c9aa8..66c10b18e5 100644 --- a/xen/drivers/char/ns16550.c +++ b/xen/drivers/char/ns16550.c @@ -445,8 +445,6 @@ static void __init cf_check ns16550_init_postirq(struct serial_port *port) { struct msi_desc *msi_desc = NULL; - pcidevs_lock(); - rc = pci_enable_msi(&msi, &msi_desc); if ( !rc ) { @@ -460,8 +458,6 @@ static void __init cf_check ns16550_init_postirq(struct serial_port *port) pci_disable_msi(msi_desc); } - pcidevs_unlock(); - if ( rc ) { uart->irq = 0; diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c index 7c1713a602..e42af65a40 100644 --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -638,10 +638,7 @@ static void cf_check parse_ppr_log_entry(struct amd_iommu *iommu, u32 entry[]) uint16_t device_id = iommu_get_devid_from_cmd(entry[0]); struct pci_dev *pdev; - pcidevs_lock(); pdev = pci_get_real_pdev(PCI_SBDF(iommu->seg, device_id)); - pcidevs_unlock(); - if ( pdev ) guest_iommu_add_ppr_log(pdev->domain, entry); pcidev_put(pdev); @@ -747,14 +744,12 @@ static bool_t __init set_iommu_interrupt_handler(struct amd_iommu *iommu) return 0; } - pcidevs_lock(); /* * XXX: it is unclear if this device can be removed. Right now * there is no code that clears msi.dev, so no one will decrease * refcount on it. */ iommu->msi.dev = pci_get_pdev(NULL, PCI_SBDF(iommu->seg, iommu->bdf)); - pcidevs_unlock(); if ( !iommu->msi.dev ) { AMD_IOMMU_WARN("no pdev for %pp\n", @@ -1289,9 +1284,7 @@ static int __init cf_check amd_iommu_setup_device_table( { if ( !pci_init ) continue; - pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bdf)); - pcidevs_unlock(); } if ( pdev && (pdev->msix || pdev->msi_maxvec) ) diff --git a/xen/drivers/passthrough/amd/iommu_map.c b/xen/drivers/passthrough/amd/iommu_map.c index 9d621e3d36..d04aa37538 100644 --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -726,9 +726,7 @@ int cf_check amd_iommu_get_reserved_device_memory( /* May need to trigger the workaround in find_iommu_for_device(). */ struct pci_dev *pdev; - pcidevs_lock(); pdev = pci_get_pdev(NULL, sbdf); - pcidevs_unlock(); if ( pdev ) { @@ -848,7 +846,6 @@ int cf_check amd_iommu_quarantine_init(struct pci_dev *pdev, bool scratch_page) const struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(pdev->seg); int rc; - ASSERT(pcidevs_locked()); ASSERT(!hd->arch.amd.root_table); ASSERT(page_list_empty(&hd->arch.pgtables.list)); @@ -903,8 +900,6 @@ void amd_iommu_quarantine_teardown(struct pci_dev *pdev) { struct domain_iommu *hd = dom_iommu(dom_io); - ASSERT(pcidevs_locked()); - if ( !pdev->arch.amd.root_table ) return; diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c index 955f3af57a..919e30129e 100644 --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -268,8 +268,6 @@ static int __must_check amd_iommu_setup_domain_device( req_id, pdev->type, page_to_maddr(root_pg), domid, hd->arch.amd.paging_mode); - ASSERT(pcidevs_locked()); - if ( pci_ats_device(iommu->seg, bus, pdev->devfn) && !ivrs_dev->block_ats && iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) && @@ -416,8 +414,6 @@ static void amd_iommu_disable_domain_device(const struct domain *domain, if ( QUARANTINE_SKIP(domain, pdev) ) return; - ASSERT(pcidevs_locked()); - if ( pci_ats_device(iommu->seg, bus, pdev->devfn) && pci_ats_enabled(iommu->seg, bus, pdev->devfn) ) { diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index c83397211b..cc62a5aec4 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -517,7 +517,6 @@ int __init pci_hide_device(unsigned int seg, unsigned int bus, struct pci_seg *pseg; int rc = -ENOMEM; - pcidevs_lock(); pseg = alloc_pseg(seg); if ( pseg ) { @@ -528,7 +527,6 @@ int __init pci_hide_device(unsigned int seg, unsigned int bus, rc = 0; } } - pcidevs_unlock(); return rc; } @@ -588,8 +586,6 @@ struct pci_dev *pci_get_pdev(struct domain *d, pci_sbdf_t sbdf) { struct pci_dev *pdev; - ASSERT(d || pcidevs_locked()); - /* * The hardware domain owns the majority of the devices in the system. * When there are multiple segments, traversing the per-segment list is @@ -730,7 +726,6 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, pdev_type = "device"; else if ( info->is_virtfn ) { - pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(seg, info->physfn.bus, info->physfn.devfn)); @@ -739,7 +734,6 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, pf_is_extfn = pdev->info.is_extfn; pcidev_put(pdev); } - pcidevs_unlock(); if ( !pdev ) pci_add_device(seg, info->physfn.bus, info->physfn.devfn, NULL, node); @@ -756,7 +750,6 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, ret = -ENOMEM; - pcidevs_lock(); pseg = alloc_pseg(seg); if ( !pseg ) goto out; @@ -858,7 +851,6 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, pci_enable_acs(pdev); out: - pcidevs_unlock(); if ( !ret ) { printk(XENLOG_DEBUG "PCI add %s %pp\n", pdev_type, &pdev->sbdf); @@ -889,7 +881,6 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) if ( !pseg ) return -ENODEV; - pcidevs_lock(); spin_lock(&pseg->alldevs_lock); list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) if ( pdev->bus == bus && pdev->devfn == devfn ) @@ -910,12 +901,10 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) break; } - pcidevs_unlock(); spin_unlock(&pseg->alldevs_lock); return ret; } -/* Caller should hold the pcidevs_lock */ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, uint8_t devfn) { @@ -927,7 +916,6 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, if ( !is_iommu_enabled(d) ) return -EINVAL; - ASSERT(pcidevs_locked()); pdev = pci_get_pdev(d, PCI_SBDF(seg, bus, devfn)); if ( !pdev ) return -ENODEV; @@ -981,13 +969,10 @@ int pci_release_devices(struct domain *d) u8 bus, devfn; int ret; - pcidevs_lock(); ret = arch_pci_clean_pirqs(d); if ( ret ) - { - pcidevs_unlock(); return ret; - } + spin_lock(&d->pdevs_lock); list_for_each_entry_safe ( pdev, tmp, &d->pdev_list, domain_list ) { @@ -996,7 +981,6 @@ int pci_release_devices(struct domain *d) ret = deassign_device(d, pdev->seg, bus, devfn) ?: ret; } spin_unlock(&d->pdevs_lock); - pcidevs_unlock(); return ret; } @@ -1094,7 +1078,6 @@ void pci_check_disable_device(u16 seg, u8 bus, u8 devfn) s_time_t now = NOW(); u16 cword; - pcidevs_lock(); pdev = pci_get_real_pdev(PCI_SBDF(seg, bus, devfn)); if ( pdev ) { @@ -1108,7 +1091,6 @@ void pci_check_disable_device(u16 seg, u8 bus, u8 devfn) pdev = NULL; } } - pcidevs_unlock(); if ( !pdev ) return; @@ -1164,13 +1146,7 @@ static int __init cf_check _scan_pci_devices(struct pci_seg *pseg, void *arg) int __init scan_pci_devices(void) { - int ret; - - pcidevs_lock(); - ret = pci_segments_iterate(_scan_pci_devices, NULL); - pcidevs_unlock(); - - return ret; + return pci_segments_iterate(_scan_pci_devices, NULL); } struct setup_hwdom { @@ -1239,19 +1215,11 @@ static int __hwdom_init cf_check _setup_hwdom_pci_devices( pcidev_put(pdev); if ( iommu_verbose ) - { - pcidevs_unlock(); process_pending_softirqs(); - pcidevs_lock(); - } } if ( !iommu_verbose ) - { - pcidevs_unlock(); process_pending_softirqs(); - pcidevs_lock(); - } } return 0; @@ -1262,9 +1230,7 @@ void __hwdom_init setup_hwdom_pci_devices( { struct setup_hwdom ctxt = { .d = d, .handler = handler }; - pcidevs_lock(); pci_segments_iterate(_setup_hwdom_pci_devices, &ctxt); - pcidevs_unlock(); } /* APEI not supported on ARM yet. */ @@ -1396,9 +1362,7 @@ static int cf_check _dump_pci_devices(struct pci_seg *pseg, void *arg) static void cf_check dump_pci_devices(unsigned char ch) { printk("==== PCI devices ====\n"); - pcidevs_lock(); pci_segments_iterate(_dump_pci_devices, NULL); - pcidevs_unlock(); } static int __init cf_check setup_dump_pcidevs(void) @@ -1417,8 +1381,6 @@ static int iommu_add_device(struct pci_dev *pdev) if ( !pdev->domain ) return -EINVAL; - ASSERT(pcidevs_locked()); - hd = dom_iommu(pdev->domain); if ( !is_iommu_enabled(pdev->domain) ) return 0; @@ -1446,8 +1408,6 @@ static int iommu_enable_device(struct pci_dev *pdev) if ( !pdev->domain ) return -EINVAL; - ASSERT(pcidevs_locked()); - hd = dom_iommu(pdev->domain); if ( !is_iommu_enabled(pdev->domain) || !hd->platform_ops->enable_device ) @@ -1494,7 +1454,6 @@ static int device_assigned(struct pci_dev *pdev) { int rc = 0; - ASSERT(pcidevs_locked()); /* * If the device exists and it is not owned by either the hardware * domain or dom_io then it must be assigned to a guest, or be @@ -1507,7 +1466,6 @@ static int device_assigned(struct pci_dev *pdev) return rc; } -/* Caller should hold the pcidevs_lock */ static int assign_device(struct domain *d, struct pci_dev *pdev, u32 flag) { const struct domain_iommu *hd = dom_iommu(d); @@ -1521,7 +1479,6 @@ static int assign_device(struct domain *d, struct pci_dev *pdev, u32 flag) return -EXDEV; /* device_assigned() should already have cleared the device for assignment */ - ASSERT(pcidevs_locked()); ASSERT(pdev && (pdev->domain == hardware_domain || pdev->domain == dom_io)); @@ -1587,7 +1544,6 @@ static int iommu_get_device_group( if ( group_id < 0 ) return group_id; - pcidevs_lock(); spin_lock(&d->pdevs_lock); for_each_pdev( d, pdev ) { @@ -1603,7 +1559,6 @@ static int iommu_get_device_group( sdev_id = iommu_call(ops, get_device_group_id, seg, b, df); if ( sdev_id < 0 ) { - pcidevs_unlock(); spin_unlock(&d->pdevs_lock); return sdev_id; } @@ -1614,7 +1569,6 @@ static int iommu_get_device_group( if ( unlikely(copy_to_guest_offset(buf, i, &bdf, 1)) ) { - pcidevs_unlock(); spin_unlock(&d->pdevs_lock); return -EFAULT; } @@ -1622,7 +1576,6 @@ static int iommu_get_device_group( } } - pcidevs_unlock(); spin_unlock(&d->pdevs_lock); return i; @@ -1630,17 +1583,12 @@ static int iommu_get_device_group( void iommu_dev_iotlb_flush_timeout(struct domain *d, struct pci_dev *pdev) { - pcidevs_lock(); - /* iommu->ats_list_lock is taken by the caller of this function */ disable_ats_device(pdev); ASSERT(pdev->domain); if ( d != pdev->domain ) - { - pcidevs_unlock(); return; - } pdev->broken = true; @@ -1649,8 +1597,6 @@ void iommu_dev_iotlb_flush_timeout(struct domain *d, struct pci_dev *pdev) d->domain_id, &pdev->sbdf); if ( !is_hardware_domain(d) ) domain_crash(d); - - pcidevs_unlock(); } int iommu_do_pci_domctl( @@ -1740,7 +1686,6 @@ int iommu_do_pci_domctl( break; } - pcidevs_lock(); ret = device_assigned(pdev); if ( domctl->cmd == XEN_DOMCTL_test_assign_device ) { @@ -1755,7 +1700,7 @@ int iommu_do_pci_domctl( ret = assign_device(d, pdev, flags); pcidev_put(pdev); - pcidevs_unlock(); + if ( ret == -ERESTART ) ret = hypercall_create_continuation(__HYPERVISOR_domctl, "h", u_domctl); @@ -1787,9 +1732,7 @@ int iommu_do_pci_domctl( bus = PCI_BUS(machine_sbdf); devfn = PCI_DEVFN(machine_sbdf); - pcidevs_lock(); ret = deassign_device(d, seg, bus, devfn); - pcidevs_unlock(); break; default: diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index 42661f22f4..87868188b7 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -1490,7 +1490,6 @@ int domain_context_mapping_one( if ( QUARANTINE_SKIP(domain, pgd_maddr) ) return 0; - ASSERT(pcidevs_locked()); spin_lock(&iommu->lock); maddr = bus_to_context_maddr(iommu, bus); context_entries = (struct context_entry *)map_vtd_domain_page(maddr); @@ -1711,8 +1710,6 @@ static int domain_context_mapping(struct domain *domain, u8 devfn, if ( drhd && drhd->iommu->node != NUMA_NO_NODE ) dom_iommu(domain)->node = drhd->iommu->node; - ASSERT(pcidevs_locked()); - for_each_rmrr_device( rmrr, bdf, i ) { if ( rmrr->segment != pdev->seg || bdf != pdev->sbdf.bdf ) @@ -2072,8 +2069,6 @@ static void quarantine_teardown(struct pci_dev *pdev, { struct domain_iommu *hd = dom_iommu(dom_io); - ASSERT(pcidevs_locked()); - if ( !pdev->arch.vtd.pgd_maddr ) return; @@ -2341,8 +2336,6 @@ static int cf_check intel_iommu_add_device(u8 devfn, struct pci_dev *pdev) u16 bdf; int ret, i; - ASSERT(pcidevs_locked()); - if ( !pdev->domain ) return -EINVAL; @@ -3176,7 +3169,6 @@ static int cf_check intel_iommu_quarantine_init(struct pci_dev *pdev, bool rmrr_found = false; int rc; - ASSERT(pcidevs_locked()); ASSERT(!hd->arch.vtd.pgd_maddr); ASSERT(page_list_empty(&hd->arch.pgtables.list)); diff --git a/xen/drivers/video/vga.c b/xen/drivers/video/vga.c index 1298f3a7b6..1f7c496114 100644 --- a/xen/drivers/video/vga.c +++ b/xen/drivers/video/vga.c @@ -117,9 +117,7 @@ void __init video_endboot(void) struct pci_dev *pdev; u8 b = bus, df = devfn, sb; - pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(0, bus, devfn)); - pcidevs_unlock(); if ( !pdev || pci_conf_read16(PCI_SBDF(0, bus, devfn), From patchwork Wed Aug 31 14:11:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 30B28C65C0D for ; Wed, 31 Aug 2022 14:11:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395681.635478 (Exim 4.92) (envelope-from ) id 1oTOR0-0002cN-DK; Wed, 31 Aug 2022 14:11:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395681.635478; Wed, 31 Aug 2022 14:11:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR0-0002cG-AP; Wed, 31 Aug 2022 14:11:14 +0000 Received: by outflank-mailman (input) for mailman id 395681; Wed, 31 Aug 2022 14:11:13 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOQy-0002bv-S9 for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:13 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c2df0da8-2936-11ed-934f-f50d60e1c1bd; Wed, 31 Aug 2022 16:11:10 +0200 (CEST) Received: from pps.filterd (m0174677.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBo4GE032537; Wed, 31 Aug 2022 14:11:07 GMT Received: from eur04-vi1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3ja427h4rd-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:07 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:11:02 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:11:01 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c2df0da8-2936-11ed-934f-f50d60e1c1bd ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KvArwocYUmWcgj1cBM8dAwNTv+HzTn8sMGFViPImWj5Q3F9llb6oaaTnAutlo5noBmcc2zKT8IJxaSOZUH7d1S7F772NoVqMbdqcYe64AndKM5cI0Z2Z2fMjIDMXWdIEXEw9mHivEY0hB72qpgZAwpqyf33C/aQy/Cf0OYNeYNOgd3X4NopGuzI6zP4Kn9oB8/WNZzFOUg73SIXqke0doOkDWtn1VXAY7iri2kxnSNb4AjYV6oY+KjySFIHhQwl01mQNCT+3yAVx7bLIO9cuw5SXREBsKXJfvaHtdbLB9ZFZGYY9F1Eu1R8vZdWiLOwkvJdiEJqYFpLkKisFTEBAmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pKNemWZV2+FfurTu7vJyid03vkZnEYejDqPafwLthYA=; b=cPyGpxRKr5BUYR1PXasS2s/HiSxFSR9e9dM1UMDTP1o4OgZ75E0fXwPc68OIxhViChvYjK3MlcZuMgraR+x/sik/L5nuY4rCjUGL/IrFaY5UifPzrk0y38oVUeMlOyrnEcWA2LTWgHK6ITBj/hrrSLGUL+MA/uN2oniAAMStU3xhBOrreQ1SW6/ZjPDhkzj6ZEcrpSpngkP3/Y/AfsllRmSO9TG5Gy9GbBiF4xwVEa6ZAqQsGlDmN1VGt9gOnEfCBcAojPH1gbLuIceFw1eyNtxAD2Ho6IFTPZFlU0mewN+PJg924XA6miKPOuz0yHWx6plrOx3tMAh66z3IgaIcog== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pKNemWZV2+FfurTu7vJyid03vkZnEYejDqPafwLthYA=; b=DSjUsgDxzTZHhvTRrd+IuueionF9zq4+bW+5mEv/U1mYDZ6zCO/hYNLCyn2CmOsXRGkQhIaqW72keWvWlfMpes74FY2jMQySY7GlKS6o6rm41h75GBUosWwXJ2NqRtAdKAMEkMF1hL87Rcw7Px5AYL/Qli1ltFNnA8eHOrd3vsonxEHCYPq2eEDfv9tF3h1zq98sUu6CevTUs+45ryGM/H1IbGdkWAJektUZ8q6b/108Osl0ExaGn66hUcImtoZfCQQfHeT8IY2xX7YdtlQp6DrUJW9kreEi6FdmXPnZ1m/VRHLUtDwQS9rEE9ecuUqVdt3NngsuHT6zMI1FrU8vfg== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Kevin Tian , Jan Beulich , Paul Durrant , =?iso-8859-1?q?Roger_Pau_Monn=E9?= Subject: [RFC PATCH 09/10] [RFC only] xen: iommu: remove last pcidevs_lock() calls in iommu Thread-Topic: [RFC PATCH 09/10] [RFC only] xen: iommu: remove last pcidevs_lock() calls in iommu Thread-Index: AQHYvUOAlPa0wkGiakOWSGDBM2amAQ== Date: Wed, 31 Aug 2022 14:11:01 +0000 Message-ID: <20220831141040.13231-10-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9c2ed1f2-79cc-492b-ce23-08da8b5aa2ac x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: QThC1htusAtQAt2B2v9A+fgyL3/E9jsD1gfKXVTHPKPnETix/ezkg2zyQKyCwyFzyhO7zRg1HkQClmhC6APAaxz+68k0DRE6YqhG7HlNnB9EYZOQKgGNoQHAE4jXnsJd9xV4wB8WfQ2w0M2dZrvzg+RCe6rkY1QpLHdo+be0s1CF1e5gSjLvyuArl3abxc5Io+4gLGXVY97OzaOzYNTza/CIn+4XVxNn/qrhlDPwthJaVIm4JGyr7+kl9eOJcGUsNLty0VWbE49nCLqXG4c6ek+x1lg8zKSOm0BuXCRDCWMIz4A8ikUWfYSJt8SoU+mzvvIJ3YXWXtjzGCnyz1B7roapQIOv9jDBRP983/lVOW99SkjOv5pAmK7qdjK1nxojrf7/zDOAWfTC99/hxrjxzGgGCbVvzw9p1fvMsgxgDOUmafz69rsZsriGBhiThKsIvyxs/6Ows6FwMJsd/86H2vFfA5L9nqiJXvpkjFAcRtTbCsk4SN3N6VcDy3wzDsduog/wkdLmXh4BSI+6T/4lbHfktO8t9uaL4m6drBSP5ubTR/tJ4TRxIbjURisNa9UhANcqEzSGQrlyOhFbxBjb9G4gzNn1h7qde0aSXzpG8RUnH46wZNGVXUZDEGcNShDjB0DQhDCvFRlX1B5SufD8MgtpWv5+cehnvsXuiK2WrKNK8WIOD79nsieNbSVs00ExTpuH4gEjXvamddHWsZfUMPUfeD67CcIJqWQguYFVPOCwjGPpsTnRXJTq2zlD6bG01nOSYCnX3PZo7Z+AYe/AsA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(8936002)(316002)(36756003)(6916009)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?AnJnovz+0KakAiFLK/BUDZ1?= =?iso-8859-1?q?9KFy3tu4nx8ikSrUggWBAkttwdoDYcEUKwfmRR9aMB9dgu834OlA/Op4mQ6D?= =?iso-8859-1?q?plVpEytw+SoBYj71MMiwktj0wJbMBhDdTBwmOOpPWCzEXAI8bTqWXjz1ADlN?= =?iso-8859-1?q?W166jzeyI1hky7AGafynp6XOTLercFfeBI3fKRkx82ZevpIHSFFbUuTGeyZO?= =?iso-8859-1?q?LpXIsmmOYVF1R5SIO+JLyIHldChy7CQQxzhm+TuDr/8AiRcZB6/SoTHq3XVC?= =?iso-8859-1?q?lg8TppBGKoRU5zvO8Cs5DIRk2T8xOzx2vgeAKiZXJ74JCtMgk1zwg8Oeobyj?= =?iso-8859-1?q?O/0ZWm9Tn+aIlz5KcNYihl3WUSsHDBQCXs9OS5TcwNRTJd16MLkXjtRTF/I0?= =?iso-8859-1?q?rhH9M4/tv8V6yqhmJ1qOGtMj36D6jdufKotRsfg6LYj4PswqWQiopbA9kk31?= =?iso-8859-1?q?IDaqeRbOmnAPffXjdgKCOLP1ff/Z9Q1PVACCT5HdJQ4pXo46ZF2JRRARhClx?= =?iso-8859-1?q?Ih80Gdm3OSHBx6brIi+UxscfCXBjvka4lt2UyJizYhPclAByEaBjnd93agtc?= =?iso-8859-1?q?87P9hD9bhhqSMgknV9CViVGxTfTqVm96Ln2OXkTbaZz5GChqiVX+OGBQ7VFa?= =?iso-8859-1?q?Wcjy5fyLJ5GBHl7WzEYcfnjWipEAb94qtIFU2oH6usSKktNAJhK29sm06cEo?= =?iso-8859-1?q?L88JrR+OJlTjHDmEItfz3pXyp/Wv4ektah63mJxmgMdoZPlaLangct/kTfCY?= =?iso-8859-1?q?5uAwwiHZ5Sbc8k015Vnc/B0+ZK/sD4ySZnfasuDOqGb+/YIHsxFQx6SlE3Jd?= =?iso-8859-1?q?Xa21ZhO/Xn1NnWCJK38ErKXzvR2T4GdtgQS7ZRFeK7f+B0qxM8W073LaOtcd?= =?iso-8859-1?q?VOAlekq+xWzHEdhppVMpEAn90IuEyd8j/HvjnCuMwkQ6FC7FcmAkvxQ2bcx1?= =?iso-8859-1?q?+HAOWfL9N0/8DD+u7OAkgER8XeHaNPZ7Cxyhyq1mAi6U6A6iPZ8ToXD2bL/8?= =?iso-8859-1?q?zwErhka4Frqu63X8kG9T9d9QQ0iZ0EV7GUg+BEsMfNHHUZaYr5LHjVu74N9D?= =?iso-8859-1?q?AvyuWkj0PS4l70EqBGWGuuapjsW4AjvP/DMrCxF0HhuQWqak2+nVIRVw4v5Y?= =?iso-8859-1?q?+R1PKoNdYwvkzrjw0EQEEuGXzbhpOy1SadIzPCd5UqsITRjeobRhjG4UX1Qn?= =?iso-8859-1?q?dwn0EmnXnxKMMCxa7BUilNhnFL/C6pb6UhI1NmEFQ72H1fWqxSTW3w79H9Wa?= =?iso-8859-1?q?zTVB2gNc+rHYkT88g4P3ML1L2MD38su72dB+AdvtH2HMm3DRr4rRRctPrbHF?= =?iso-8859-1?q?nBtlWwmH0eBi9/DifxQHX1pgStR8SpXljNK27fkBOamiD8BVQKg97r7wma/h?= =?iso-8859-1?q?PHQs7QH2P741jt2NCXRkc7UOozk5l3jSj3mxW2khVWR6/IJPVkAVPeDjAFRk?= =?iso-8859-1?q?uEszT5wzDzRZFkqrQC3o0z2Z2iEoqso52y4kC6/Z3plvDTeb37vM251IiPTU?= =?iso-8859-1?q?QTTDiyujGKWuREV+mNCvXcwzsd3t/ziV7w+m08XvCeBwRv5KCZp+hsC8DK8S?= =?iso-8859-1?q?jS+qhdgRcH6jSxGnPtyC3wGXyygbyw8dXGbkkWRCFCwdNKfS/+kp7i6sygxY?= =?iso-8859-1?q?YWB0Rtbo9+CQh13ky0eVR3pR8yoa2zZozghICsw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9c2ed1f2-79cc-492b-ce23-08da8b5aa2ac X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:11:01.8272 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: LhNZ1/FbZV+oN13CFD2tj1ETKjJHk48362a7nk3NfOr1ZSmW8GpCCJLaw81vLHhmlEsfJMilNiSIRv2CxAsYgSoR39STmeejK8hbtByA/Qo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: d0beUWY9CpniSbmboyjYlg4wxNcL61o4 X-Proofpoint-ORIG-GUID: d0beUWY9CpniSbmboyjYlg4wxNcL61o4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=999 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 There are number of cases where pcidevs_lock() is used to protect something that is not related to PCI devices per se. Probably pcidev_lock in these places should be replaced with some other lock. This patch is not intended to be merged and is present only to discuss this use of pcidevs_lock() Signed-off-by: Volodymyr Babchuk --- xen/drivers/passthrough/vtd/intremap.c | 2 -- xen/drivers/passthrough/vtd/iommu.c | 5 ----- xen/drivers/passthrough/x86/iommu.c | 5 ----- 3 files changed, 12 deletions(-) diff --git a/xen/drivers/passthrough/vtd/intremap.c b/xen/drivers/passthrough/vtd/intremap.c index 1512e4866b..44e3b72f91 100644 --- a/xen/drivers/passthrough/vtd/intremap.c +++ b/xen/drivers/passthrough/vtd/intremap.c @@ -893,8 +893,6 @@ int pi_update_irte(const struct pi_desc *pi_desc, const struct pirq *pirq, spin_unlock_irq(&desc->lock); - ASSERT(pcidevs_locked()); - return msi_msg_write_remap_rte(msi_desc, &msi_desc->msg); unlock_out: diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index 87868188b7..9d258d154d 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -127,8 +127,6 @@ static int context_set_domain_id(struct context_entry *context, { unsigned int i; - ASSERT(pcidevs_locked()); - if ( domid_mapping(iommu) ) { unsigned int nr_dom = cap_ndoms(iommu->cap); @@ -1882,7 +1880,6 @@ int domain_context_unmap_one( int iommu_domid, rc, ret; bool_t flush_dev_iotlb; - ASSERT(pcidevs_locked()); spin_lock(&iommu->lock); maddr = bus_to_context_maddr(iommu, bus); @@ -2601,7 +2598,6 @@ static void __hwdom_init setup_hwdom_rmrr(struct domain *d) u16 bdf; int ret, i; - pcidevs_lock(); for_each_rmrr_device ( rmrr, bdf, i ) { /* @@ -2616,7 +2612,6 @@ static void __hwdom_init setup_hwdom_rmrr(struct domain *d) dprintk(XENLOG_ERR VTDPREFIX, "IOMMU: mapping reserved region failed\n"); } - pcidevs_unlock(); } static struct iommu_state { diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c index f671b0f2bb..4e94ad15df 100644 --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -207,7 +207,6 @@ int iommu_identity_mapping(struct domain *d, p2m_access_t p2ma, struct identity_map *map; struct domain_iommu *hd = dom_iommu(d); - ASSERT(pcidevs_locked()); ASSERT(base < end); /* @@ -479,8 +478,6 @@ domid_t iommu_alloc_domid(unsigned long *map) static unsigned int start; unsigned int idx = find_next_zero_bit(map, UINT16_MAX - DOMID_MASK, start); - ASSERT(pcidevs_locked()); - if ( idx >= UINT16_MAX - DOMID_MASK ) idx = find_first_zero_bit(map, UINT16_MAX - DOMID_MASK); if ( idx >= UINT16_MAX - DOMID_MASK ) @@ -495,8 +492,6 @@ domid_t iommu_alloc_domid(unsigned long *map) void iommu_free_domid(domid_t domid, unsigned long *map) { - ASSERT(pcidevs_locked()); - if ( domid == DOMID_INVALID ) return; From patchwork Wed Aug 31 14:11:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12960760 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3A53FC6FA81 for ; Wed, 31 Aug 2022 14:11:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.395687.635544 (Exim 4.92) (envelope-from ) id 1oTOR7-0004HZ-H7; Wed, 31 Aug 2022 14:11:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 395687.635544; Wed, 31 Aug 2022 14:11:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR7-0004HM-CI; Wed, 31 Aug 2022 14:11:21 +0000 Received: by outflank-mailman (input) for mailman id 395687; Wed, 31 Aug 2022 14:11:18 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oTOR4-0002bw-LM for xen-devel@lists.xenproject.org; Wed, 31 Aug 2022 14:11:18 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c6d1c12a-2936-11ed-82f2-63bd783d45fa; Wed, 31 Aug 2022 16:11:17 +0200 (CEST) Received: from pps.filterd (m0174676.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VBpHDB027047; Wed, 31 Aug 2022 14:11:08 GMT Received: from eur03-ve1-obe.outbound.protection.outlook.com (mail-ve1eur03lp2050.outbound.protection.outlook.com [104.47.9.50]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3j9huk4eny-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Aug 2022 14:11:08 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by VI1PR03MB6301.eurprd03.prod.outlook.com (2603:10a6:800:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Wed, 31 Aug 2022 14:11:02 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::9190:c0a7:bd87:f01f%6]) with mapi id 15.20.5566.019; Wed, 31 Aug 2022 14:11:02 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c6d1c12a-2936-11ed-82f2-63bd783d45fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aZ23+Y3+9yUKXKjciHha9W710G9Cck3NmzkTJfuGUsFUiGNmdGYoifiDnnkZo34vKmR3IDzrvAiPyBZ7ZupQLktvOqebhAGYoQHn4gHQGPe7y0KEUQc16LzaIitciVeFIEhzxwR97jalhFDE6DXyBCyXeRig3n4hQr5xqQMjClHfX7o0T6JDUeJSBAA8yvhJp68KN4MO6KandTMcjrVVSk0GfjmmD4n6yMEMUY9LT9TefmXXmKBkLL4C3aMa8RIEO5Df8o+B/uED9frh0rZfEWblzpou9A+ySJwVmYGFBC2ST3w7d7xhDj5sSuToyl5QDjlV8Kfx9bDOvUrVXiZrdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UVkD+a9ezE4I8M7A/v2OpU0OQqIDPY5B20O9xlCtPAM=; b=YD/k845QfCnvzNUP/R9BmAR2ShinV6EF1v8LCjDZudBXmjNWxk2iWIeS/qM4RgigEWEXDOqfC8k3l47n8MacU+YgfaRAohTO5+ZDAsZ6LFJsGz/48LV1mhC0l2MRar92SbwqvdK6aLJnwYNqDbenBe0zGDTYm5XgF8zXvuUBePD/HzlqNpWaZ3JGvS1eri7y3lyLKHCcoyLJhrYlmw/W/goXmnIcw5JoWeXfAFmo40d5OeX6qCi0MN88Z94+yg+5cWJAxpNS1tau/VB5tkqtF7FfJw/pc4jj1xm8N7KIA1xpQFQQYj5KVK6Is0yJrcidnyF5Lvm7ObYaIS5j5YxMHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UVkD+a9ezE4I8M7A/v2OpU0OQqIDPY5B20O9xlCtPAM=; b=bde3LfDGyTj+yjD+/ahrX45K3/FrRyaTMcTIElomTWPIRuMQpR9Iorolq/hFIzXvP1I5kLNEAgYhpY64QmBDN90izq5WMPIaZXK4saeLywFlhCFE4QXWL4ot69JeyEo3nsSVRBd2A2fq2hlj8zw2/cNUaJidmE4VSZXlIpexqtGb5RYVGP29soJBQPEvZzC5zmvaT9bxhXtAKjG1MaGF0llPoNFdxdwibDb4hh+i1sw04E/bh2/Krm4UT4p66+pulmwB7LX82q4qNFPPLBvS8USfcjYZfVoESCKUV/RkAsLuCaQUyz5yOywPke0uOoghogTGXrih8mT+ifkmpk3qQQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Volodymyr Babchuk , Jan Beulich , Paul Durrant , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Andrew Cooper , George Dunlap , Julien Grall , Stefano Stabellini , Wei Liu Subject: [RFC PATCH 10/10] [RFC only] xen: pci: remove pcidev_lock() function Thread-Topic: [RFC PATCH 10/10] [RFC only] xen: pci: remove pcidev_lock() function Thread-Index: AQHYvUOA+LvBLzOTSESi46h0pXBIvw== Date: Wed, 31 Aug 2022 14:11:02 +0000 Message-ID: <20220831141040.13231-11-volodymyr_babchuk@epam.com> References: <20220831141040.13231-1-volodymyr_babchuk@epam.com> In-Reply-To: <20220831141040.13231-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.37.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 14a91930-f49f-4022-05a2-08da8b5aa2e6 x-ms-traffictypediagnostic: VI1PR03MB6301:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jhjKyxlvzlIeXEyso1XtwYukepwuNYGZA7897hwHzlBTkIYL6mTDCIcYDMRlZQDdI9xYXCBJonOCFMHkUo9EZ0bSlYvFf1MXsk+HHsCNhC7Yvrku4I6l+7D0PmsBi2Ch/T1hszmmHKHmtCgNQmcblKpRKTdv3jnXosZeB5cUhsUTReJgIP112qWqBQqtgW2hr0ZNSWXWuJ/Sxtx4pCrbDHsPEL56USrLFLit+EUTK8xlesQ+HhHx2JH81WsDz4ckWPxKomzCrfUgm0YfFa6ev4JOWxWaGpjxJHMq/6MWOQFDI+Nus+yrURMqukbWD7ZeqUyl3wZhzJsMlr9tm5UF0k5kHfu76bST1f5WpCeRMlw3K8RmNpOOcA4yZFknGbr2yejg7cW9aXFjdoK+2o5Q6e8ouVyp9F1rrkQSFNbdvpVoUDv5HRmR28o4+keyHn6dooPo5K1dmicBKexJq43pgIKJnrgxton9F31hZY5AvGdOgZvfO1nUaMOwECche7Fw1KYNk8GrWHID7AgMCTSs8YnUlaNqAntA2bQpxtZqGcAnxVEMtJPrgX3JDWGkQbsMQFB6WE13fKAEp7aJMIh7jroLv3bm0Cshnr5pf/a9G7kwcZ3O1Oip2H2Yi0xg34pvFuufN7nhGt/TDi93JbmgR/cV3u+xLkcBhcLCxEqMjvh5+1ZL7u+DZElUcuMtoYl6UU37exT1UJzdd0/fYkDV4kgPGcnQibcTBUkGfA9tTqwBaqE1a3hNxr0yd6y9fJsQVtXMH3HsrR9hU086QCwjkkcpbTnUMhRAeUOoADFDijs= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(366004)(376002)(396003)(136003)(4326008)(66556008)(76116006)(66946007)(38070700005)(66476007)(8676002)(2906002)(66446008)(64756008)(54906003)(1076003)(186003)(83380400001)(2616005)(5660300002)(91956017)(8936002)(316002)(36756003)(6916009)(6486002)(478600001)(71200400001)(86362001)(55236004)(26005)(41300700001)(6512007)(6506007)(122000001)(38100700002)(309714004);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?Uv3RgkJSx4uWVzGl0BAsX42?= =?iso-8859-1?q?Ja7HbqJuKEsxTnPbjiUD8FRFqj52gDtlJ1yExCvrH7qFP2AQv/MACHRDxoUO?= =?iso-8859-1?q?Mfp2k869IT4eDFjoZjHaYcu5MsJBVCVUfkgoBbI49ar0fZFWNlz9UJqRRJ1K?= =?iso-8859-1?q?x6xBGBwaQKq47qoHRVsd53aBkEtOHEUPeMfWyrja/MEyinfnBLkfL2j1sj79?= =?iso-8859-1?q?NRAm0s3KKgUVnDANwkssDZrgQC+bt0PKqefvCtJNiAGLTGt/P1eH4gkDl19t?= =?iso-8859-1?q?NGGCUO/pexuxyYjxtxC7uSfN0ItiZHD9oQYzHqg976+nCW/Q1kw8nAFei3HU?= =?iso-8859-1?q?bJiDh8D5+IEELFbPE1vjof8ytmRmJLPRXxuURVqW8aLG8oEcRrgOqSqJVrfX?= =?iso-8859-1?q?8Qx5B/ev8E/qKF9x89O4VO2cFyt2VW38ha/Nd/60HKvyojGtATYhQJSE6LVx?= =?iso-8859-1?q?yUkmXoo/q4F1t4bu/FTWWwzJ4+DsmHWZ71WgVKc5LqG4e+l/SXZAbP48YQEF?= =?iso-8859-1?q?g1Ffm+5W1k8CI2DSbWr/yHrldOzU3iIaQ34UyTx5MuE1P7IWbjmJZfHBdNm0?= =?iso-8859-1?q?VVBn09GBXUk30peT3Gu1gIgssKcc4w0DEPtGILLZv4xLQBMJUNkJjIFg8tFr?= =?iso-8859-1?q?awEb0wPXTWpEmdDaydK+S8nGN5qH1/dIfAPH79bJcirevOFmjBN0JE3KFYsQ?= =?iso-8859-1?q?IvU2AXb6RI/cTB+q3RWAR51VXcJk5rpdAZwtR9ChS2x2cIUYVeq8o4Sh3Cw3?= =?iso-8859-1?q?ZPEkD+6FIJCbb6Gq9ujmmxzmXhmeqWadevv7cEi832nEfULjQBW2hZrILIdS?= =?iso-8859-1?q?iKSX47/y/xixkyYrL8/nuZ9YLkpFouJuc41QV0kpXVnaYJipUydlAIziiHUv?= =?iso-8859-1?q?/WyhRIbEmnCd4ttpWbJF8Jrw+XLtyZEHCLFgtn09nXoPUhxfsgjTsmL0pHBi?= =?iso-8859-1?q?t3LDENrYF46AZvwM9BKgfCi5gegGn4yiYjaTuGBwPiAUlNrcMGV48sui6ps+?= =?iso-8859-1?q?5Bq0h/nV3BxAG4noFFc0vgOPlGiMXF2V0ehbXX0o6+dnxG5gX5DojnyfUTBO?= =?iso-8859-1?q?+YxKzDBsF8xJh5sogGY9J4oMttfxBmqgQwxvCOl8SKjayEZXV6hU8G6J9DfB?= =?iso-8859-1?q?rhLNv0a7ESme79VK2iMJIQomwCNV2o2UMcy0XPFqJf4wnjYO3xT4lvnJ/SJw?= =?iso-8859-1?q?n6cML4pKElWGFWapzuOgGzIWQz5D7Jm87pvdi8het6bfPdlYpSkBBjYcghk7?= =?iso-8859-1?q?fSlxDtrHzonAffqqf/AGx0eEkZqKFkt8gQdPXLAhn9P0gT6qL0gy9+vp4X/o?= =?iso-8859-1?q?H3ultucmZpcuyERquq7A/tzYm6aAJzdjON1cnG4gEvv5PCD0140rWT2NuXIW?= =?iso-8859-1?q?ms5mpeK1mznHZXnnegk6eSVji1cAe0u/LZx8hbGlD5TQF1ku+PNim+GcBCsZ?= =?iso-8859-1?q?bvvttmAvMnRjU6xuDDKj1GAXLBJJqvuJQjEUQ0MTYfOm+Bs8CyL3a0T7HSxu?= =?iso-8859-1?q?lVY2qKmd/s8B3THZ1V2QB9xajbqCExDeSDhUD8uHHen4lHdZyqF0JwHTI5DW?= =?iso-8859-1?q?1FuqH80i6Us/lnt8iCP2Gj+tQSpFmxjMIrr5/qcLIAUBg/AfeW9cTH2U/TKe?= =?iso-8859-1?q?Xlc21nWebgriQu+J7rApDIzUVZ3f5i37jno+JGQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 14a91930-f49f-4022-05a2-08da8b5aa2e6 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2022 14:11:02.1241 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ErxqhRI1nPm6WLenAr0H/D9H1vgTB4mTC0/mxwIMyZPPz/nptBq24fL4y0m+RJlo6tN3HrdExCzMW5csO1WMwYwW8ApNID3Fv4+5SWpxoJs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR03MB6301 X-Proofpoint-GUID: r4Y87cm6JEYOyOTEqSo4LcqRQvW-t6ib X-Proofpoint-ORIG-GUID: r4Y87cm6JEYOyOTEqSo4LcqRQvW-t6ib X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_08,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 clxscore=1015 impostorscore=0 mlxlogscore=999 priorityscore=1501 lowpriorityscore=0 phishscore=0 suspectscore=0 mlxscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310070 This path will be squashed into "xen: pci: remove pcidev_[un]lock[ed] calls" after we resolve "[RFC only] xen: iommu: remove last pcidevs_lock() calls in iommu". Signed-off-by: Volodymyr Babchuk --- xen/drivers/passthrough/pci.c | 18 ------------------ xen/include/xen/pci.h | 10 ---------- 2 files changed, 28 deletions(-) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index cc62a5aec4..381eba3018 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -50,24 +50,6 @@ struct pci_seg { u8 devfn; } bus2bridge[MAX_BUSES]; }; - -static spinlock_t _pcidevs_lock = SPIN_LOCK_UNLOCKED; - -void pcidevs_lock(void) -{ - spin_lock_recursive(&_pcidevs_lock); -} - -void pcidevs_unlock(void) -{ - spin_unlock_recursive(&_pcidevs_lock); -} - -bool_t pcidevs_locked(void) -{ - return !!spin_is_locked(&_pcidevs_lock); -} - static struct radix_tree_root pci_segments; static inline struct pci_seg *get_pseg(u16 seg) diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h index d0a7339d84..0abc54ea39 100644 --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -156,16 +156,6 @@ struct pci_dev { #define has_arch_pdevs(d) (!list_empty(&(d)->pdev_list)) -/* - * The pcidevs_lock protect alldevs_list, and the assignment for the - * devices, it also sync the access to the msi capability that is not - * interrupt handling related (the mask bit register). - */ - -void pcidevs_lock(void); -void pcidevs_unlock(void); -bool_t __must_check pcidevs_locked(void); - /* * Acquire and release reference to the given device. Holding * reference ensures that device will not disappear under feet, but