From patchwork Wed Dec 13 23:44:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13492130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2370CC4167D for ; Wed, 13 Dec 2023 23:45:35 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.654255.1020955 (Exim 4.92) (envelope-from ) id 1rDYul-0003tp-F5; Wed, 13 Dec 2023 23:45:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 654255.1020955; Wed, 13 Dec 2023 23:45:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rDYul-0003ti-Bj; Wed, 13 Dec 2023 23:45:19 +0000 Received: by outflank-mailman (input) for mailman id 654255; Wed, 13 Dec 2023 23:45:18 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rDYuj-0003et-Sz for xen-devel@lists.xenproject.org; Wed, 13 Dec 2023 23:45:18 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a91fc728-9a11-11ee-9b0f-b553b5be7939; Thu, 14 Dec 2023 00:45:15 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BDJ2iKX012660; Wed, 13 Dec 2023 23:45:03 GMT Received: from eur04-he1-obe.outbound.protection.outlook.com (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uy0y4vams-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Dec 2023 23:45:02 +0000 (GMT) Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by AS8PR03MB8666.eurprd03.prod.outlook.com (2603:10a6:20b:54c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.26; Wed, 13 Dec 2023 23:44:58 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7091.022; Wed, 13 Dec 2023 23:44:58 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a91fc728-9a11-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RjcCE34Q8cYAXZKgD7kJW0DB45vnF8CZxwIVuvrVjiEb8RAIuCOuBtuEWzBMAVfJ8R0tjmgMD5TVZ+yVgJb23II1xkTsaKWcq+jxTAU8/g+MN7NZKQEgzLvzI227tLLOz3SULA7Re3RsvOn873lnTaSPKxI7Lp1xTMETjVYv+Yfl/IMP33zKbSCFM9qXEaCem5pFZuLRh/11Wa9aR45a96Eck6agHlkQuGaSc0yCivJvT4N2QgktlPrm3/A01oAEBqFrz8KZl02YvgrstBO5tE6iMLF7F7Dvi+jkT0VY2QRmmQLPknrL8CjEvgtGYTUmWYUMBssxwLGCHbpuiBqEuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dJERCSIdBtv8QPcVo8Rezsd1dZHzxbvjwYiJ7vM4OF8=; b=c4srFKbusB0BKFODJhzutgrmdCs8st8VDehNfBqoeDyWHZxBfq8ajhcKcdCNRP9mMP+pHwcSXdfTWD7kQ4pJok4bQaBA56gz8HNz/uo83zF6+h6L8OiEWIy9Kpm9ASrMKEUgbXADp8XHkFE5lWbM99vTcS0rStV4ZJSSz6+xyKHCfnVuAGg3IlzbgbMx85Cz4Vfb6cz4Y454ixnV4ywnDZtn7kLHcXgMLEvsXNN4BxPu8rQAhJJVYA/KPZRWWHWhH7NzjJzJOp0p72GwgCOJ7ap6IjgE/AJSyrvplF8FbBXfqGvgcac2OVrhrGQp4/cEg+8ikFbURfjxZNnlsT+8qw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dJERCSIdBtv8QPcVo8Rezsd1dZHzxbvjwYiJ7vM4OF8=; b=qwwfMLE3VqcvX84HbGT3d+xGPHosMRrdv5RA50Mh5gTCkFUE9gEbcVXIgqxM3snls77g6Hi3+i+ZPkM/v6SH4czZ9TALJHkSrs2tX5/szu2tbqWSutPWKOJ40Y3D8DRIfeLJrlybkXAejXCf9aNR0z6FoABhR6gpYyDCVcuaoJG8Km0rP/TeNZQStjK0ulObhLnKPPTggzN0xN7xIsQOK+xz2iNvUVnUKGZMWUQtHBJ4JPamIrFNUHUUnozhf1sBcSBWD13ygOkcGW7n+jy5OVL5QDFbF9GSpSvE3SzwDM4MF+4oWui3fLMHKyk513qYuSd9vWHDh34bFZZBtpksng== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Wei Liu , Anthony PERARD , Juergen Gross , Andrew Cooper , George Dunlap , Jan Beulich , Paul Durrant , =?iso-8859-1?q?Roger_Pau_Monn=E9?= Subject: [RFC,FUTURE 1/3] domctl/pci: add ability to provide/request a virtual SBDF Thread-Topic: [RFC,FUTURE 1/3] domctl/pci: add ability to provide/request a virtual SBDF Thread-Index: AQHaLh5hbb+FwMFvSku5VoT+rvFZMg== Date: Wed, 13 Dec 2023 23:44:57 +0000 Message-ID: <20231213234345.779722-2-volodymyr_babchuk@epam.com> References: <20231213234345.779722-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231213234345.779722-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.43.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|AS8PR03MB8666:EE_ x-ms-office365-filtering-correlation-id: 4066ccfc-1076-4190-f13d-08dbfc358420 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: DxAP/ZOjUgYq/8JnPc2f2uWdw/bLsvAoCwnkqKbnfd35oGiEqC7fn9ijhwCBzJFxttbxuTxuGjM6FolM7v8y0CmzbbIVvjGAgBQfZpobkQQP2pIXXgs5SkaJ/qCiOCUn9rHJ9/ho0wpRPBZWPszptK5zEKVtT+EhM4diRXaBQbHRtPgia2RvamhjZolE5RfJZmtPXBkANrQOF4phVDyZ6zOL+vWOzKsbWSbc9Ojce6CmQu8xrpG/Iw93qOl/m6u3LeOytxv4GwrN5fLkPuhoaxdXIl9PmkHIdnmtQWTMtghIA104sURA1PeF2LrYJqPqVjJVNDaog73oKNn8eSQsGWajdomh6iEgWAIjeTidzb9cUfrsGi8iE0pbD1SsnADAskv1RJh56+0bY8roD5Mjp9IFBJ0pjzx6FEZFzBcJMGkrhr44WF4/FNkGcZUp9obU4m5eu+WnPYhQNHhJUSxqnOk+cMEwyu1JhMyXST+/mEPIya4tTKb7ni+5cF/URISWCJUN9q6CcVM+Ew5sVQm6O7KrbruO7/0pybflE5SPKhKjlBlr9jRlTYbgbftopZwyBsy8plZdu/Taj0mYvh54xPDTflshow5UaRaoCc1973M= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(366004)(376002)(396003)(39860400002)(136003)(230922051799003)(451199024)(1800799012)(186009)(64100799003)(1076003)(26005)(2616005)(71200400001)(6506007)(6512007)(55236004)(83380400001)(5660300002)(7416002)(8936002)(41300700001)(2906002)(4326008)(478600001)(6486002)(966005)(66556008)(8676002)(76116006)(66946007)(91956017)(316002)(6916009)(54906003)(64756008)(66446008)(66476007)(86362001)(122000001)(38100700002)(36756003)(38070700009);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?P0AisEOffWOw8RyHmh8GlbY?= =?iso-8859-1?q?3gp6QLZMeGsD8tDoF7VQcVFv4dV/FOyXwsAvsIWUYt0XSZoZFJopnOO1FEwd?= =?iso-8859-1?q?58SvSjMA7P5Urn1kLFmQOZUElT/gZjGnVL+fiEi3prOZsKwqsqdf+cngx0hl?= =?iso-8859-1?q?v6nlN1jNwgZgT9B2WbcQebNFtyVoevJZc1EYwMGwxJ3JOrZjeF4tInN51nal?= =?iso-8859-1?q?nO6h7Hcbad1CzqSvqNYNOgkWQA5K8LzROyPDML53pghrcI8gFP1N6rdH9QNH?= =?iso-8859-1?q?iLRwGyH0ekkr4fWBN0Tn1ph/SWV93ZfaD4vWccD1S0XJJqnAoVGk9RyzYey8?= =?iso-8859-1?q?R8BzE8EnWi3yYCHPlkbMFwr2EmPKJWaFT0v+N2/KG88fmUUqy+W/5ayYTUt0?= =?iso-8859-1?q?E6R65PhryarRVs50L8UtvJIiksgYjzE97YMddN9RBhWsvj4vA9CMxe55JQNo?= =?iso-8859-1?q?9CqqiQWEFY6ua9jh6OnUHWKbxtcyK3L1zL2vwYKlzLeBiQh0splvf8xiBdHd?= =?iso-8859-1?q?Z6tR1Y7ZmKldYZSuQAiKq0pfwA78l9RAl7PW3HrW0lTKWNIo4dxfxlphYjhO?= =?iso-8859-1?q?DndoZouSDzSeucXP8LftcnlR7OQIfNNMOBYvv/2PlD7FBhJWe42qxNhxf85d?= =?iso-8859-1?q?bfvex8TRmBdIi3uPlDYooBG6X/F78VaVf6YE3FER/0TaBV+6FQezSFJiCvo3?= =?iso-8859-1?q?PRi1BSRXQ3vvRePdQGi1GsHML1Xy61NXxFbcUgVfLZ3zCRtYXE7MgyWyhYjj?= =?iso-8859-1?q?UM2oA4t6LSAz2pu4AKfgBWdoZX7a5PdQiG968qvYRlXL7cbKSaf318r1neUJ?= =?iso-8859-1?q?lLCzh7AYPRvsJ6QGJN7kd6zrOITssd9fnG3DNAZtPXcLVWMfG56xRUuNg+Vj?= =?iso-8859-1?q?MmsOoTNYHwHNlyaSBQTBZ8TbVVPXbczB99fInBrylexdTStMOuC7cG7Cqw34?= =?iso-8859-1?q?TejFk/z4Wi4tXFpYNKWoZQ6pWo4w0uwcrmT6m+b/E/aKR5Bu2KGU8FVrQg/N?= =?iso-8859-1?q?HFeeQJi2AzMVzftygbiK889MC1r4m8sSfKT+Nk07AuPRubj0QBaDy2cUG4C/?= =?iso-8859-1?q?R0uMqU2agKgp7PAPyRAUQJWsN1o7BwXuHXTkIwrakM4yl/IbW4Tlg7gNISxY?= =?iso-8859-1?q?UgZFQKgoWPB6M2qtUGKoDn0qrtaDCLiH/JoOnxtcONvK7bFb+6pGJIucBXYW?= =?iso-8859-1?q?OiwxJkPun6r3PggXVRaazm02tTP2NZ3keQycxMZbAgbCWxm+j4iA5QItwyWu?= =?iso-8859-1?q?PmFCeGVsazcR20q9dzTfkmf6GnSzvx8Y4CJGJIkVY+g5Q+n5MC8Bp33TdXsK?= =?iso-8859-1?q?uPk7wr7XT/2FbvCg4GX+ng+oFCXqNoSePJv5XTo4v6A6lF1IRh1SDXIe0Hzl?= =?iso-8859-1?q?yheGYEa4RQGIyb5dTHJuJJJwoKu9kKUTJr6cc/p60jhtY7kNTIgbYluU/3fi?= =?iso-8859-1?q?kSFKqnMwqs/TkKeYZ+JF1cU2nJ+nBZsnJ0KMvaI6FsYGYkQO9FoL6fV3it5O?= =?iso-8859-1?q?SzfxL1tcB9ST7r/MknAuMwco2Su4D7vn1qEElHzvzVlQ4G6KfXYnFHjI7VlB?= =?iso-8859-1?q?KHzXI801VuZYV1gANbKGhOrDy0PZqqKY52dUihKJLuiV8sKaX7vRkwsQDTR9?= =?iso-8859-1?q?v6rrd5LIMyFsIACjNMFr6sk5AZh8pytri0tBUlg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4066ccfc-1076-4190-f13d-08dbfc358420 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2023 23:44:57.9566 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: H+SoYpuLw5Kt9ENP8djB6Tei82SR4pLyf+Pzj2HjountWUJkj6j1rek3Vkc+TeN2k4LJzL29odbgRjvZth2EeXX4H538D/Jqzka12EFz6kg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB8666 X-Proofpoint-GUID: AslKyKk_D3ly4HoVn7Drgv3B9ttsx1zY X-Proofpoint-ORIG-GUID: AslKyKk_D3ly4HoVn7Drgv3B9ttsx1zY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 impostorscore=0 lowpriorityscore=0 adultscore=0 clxscore=1015 spamscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312130168 With CONFIG_HAS_VPCI_GUEST_SUPPORT enabled, hypervisor will assign a passed-through PCI device to a guest using virtual/guest SBDF number. Right now hypervisor automatically allocates next free SBDF. But there are cases mentioned in [1] when user should be able to control SBDF assigned to the passed-through device. To enable this, extend assign_device domctl call with optional parameter virtual_sbdf. When this parameter is set to XEN_DOMCTL_DEV_SDBF_ANY, hypervisor will act as previously, but it will return allocated vSBDF back to the toolstack. Alternatively, toolstack might provide desired vSBDF and hypervisor will try to use it, if it is free and falls into permitted range. [1] https://lore.kernel.org/all/d6a58e73-da51-40f1-a2f7-576274945585@xen.org/ Signed-off-by: Volodymyr Babchuk --- tools/libs/ctrl/xc_domain.c | 1 + xen/drivers/passthrough/pci.c | 17 ++++++++++------ xen/drivers/vpci/vpci.c | 38 +++++++++++++++++++++++++++-------- xen/include/public/domctl.h | 9 ++++++++- xen/include/xen/vpci.h | 4 ++-- 5 files changed, 52 insertions(+), 17 deletions(-) diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c index f2d9d14b4d..2405e39517 100644 --- a/tools/libs/ctrl/xc_domain.c +++ b/tools/libs/ctrl/xc_domain.c @@ -1505,6 +1505,7 @@ int xc_assign_device( domctl.domain = domid; domctl.u.assign_device.dev = XEN_DOMCTL_DEV_PCI; domctl.u.assign_device.u.pci.machine_sbdf = machine_sbdf; + domctl.u.assign_device.u.pci.virtual_sbdf = XEN_DOMCTL_DEV_SDBF_ANY; domctl.u.assign_device.flags = flags; return do_domctl(xch, &domctl); diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index a3312fdab2..9ea18c39f4 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -755,7 +755,7 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, * For devices not discovered by Xen during boot, add vPCI handlers * when Dom0 first informs Xen about such devices. */ - ret = vpci_assign_device(pdev); + ret = vpci_assign_device(pdev, NULL); if ( ret ) { list_del(&pdev->domain_list); @@ -891,7 +891,7 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, pdev->fault.count = 0; write_lock(&target->pci_lock); - ret = vpci_assign_device(pdev); + ret = vpci_assign_device(pdev, NULL); write_unlock(&target->pci_lock); out: @@ -1154,7 +1154,7 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt, PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) ); write_lock(&ctxt->d->pci_lock); - err = vpci_assign_device(pdev); + err = vpci_assign_device(pdev, NULL); write_unlock(&ctxt->d->pci_lock); if ( err ) printk(XENLOG_ERR "setup of vPCI for d%d failed: %d\n", @@ -1461,7 +1461,8 @@ static int device_assigned(u16 seg, u8 bus, u8 devfn) } /* Caller should hold the pcidevs_lock */ -static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) +static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag, + pci_sbdf_t *vsbdf) { const struct domain_iommu *hd = dom_iommu(d); struct pci_dev *pdev; @@ -1515,7 +1516,7 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) } write_lock(&d->pci_lock); - rc = vpci_assign_device(pdev); + rc = vpci_assign_device(pdev, vsbdf); write_unlock(&d->pci_lock); done: @@ -1616,6 +1617,7 @@ int iommu_do_pci_domctl( u8 bus, devfn; int ret = 0; uint32_t machine_sbdf; + pci_sbdf_t virtual_sbdf; switch ( domctl->cmd ) { @@ -1675,6 +1677,7 @@ int iommu_do_pci_domctl( break; machine_sbdf = domctl->u.assign_device.u.pci.machine_sbdf; + virtual_sbdf.sbdf = domctl->u.assign_device.u.pci.virtual_sbdf; ret = xsm_assign_device(XSM_HOOK, d, machine_sbdf); if ( ret ) @@ -1696,11 +1699,13 @@ int iommu_do_pci_domctl( } } else if ( !ret ) - ret = assign_device(d, seg, bus, devfn, flags); + ret = assign_device(d, seg, bus, devfn, flags, &virtual_sbdf); pcidevs_unlock(); if ( ret == -ERESTART ) ret = hypercall_create_continuation(__HYPERVISOR_domctl, "h", u_domctl); + + domctl->u.assign_device.u.pci.virtual_sbdf = virtual_sbdf.sbdf; break; case XEN_DOMCTL_deassign_device: diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 7c0b610ccc..12963b77c3 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -37,7 +37,7 @@ extern vpci_register_init_t *const __end_vpci_array[]; #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array) #ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT -static int add_virtual_device(struct pci_dev *pdev) +static int add_virtual_device(struct pci_dev *pdev, pci_sbdf_t *vsbdf) { struct domain *d = pdev->domain; unsigned int new_dev_number; @@ -57,13 +57,35 @@ static int add_virtual_device(struct pci_dev *pdev) &pdev->sbdf); return -EOPNOTSUPP; } - new_dev_number = find_first_zero_bit(d->vpci_dev_assigned_map, - VPCI_MAX_VIRT_DEV); - if ( new_dev_number == VPCI_MAX_VIRT_DEV ) - return -ENOSPC; - __set_bit(new_dev_number, &d->vpci_dev_assigned_map); + if ( !vsbdf || vsbdf->sbdf == XEN_DOMCTL_DEV_SDBF_ANY ) + { + new_dev_number = find_first_zero_bit(d->vpci_dev_assigned_map, + VPCI_MAX_VIRT_DEV); + if ( new_dev_number == VPCI_MAX_VIRT_DEV ) + return -ENOSPC; + if ( vsbdf ) + *vsbdf = PCI_SBDF(0, 0, new_dev_number, 0); + } + else + { + if ( vsbdf->seg != 0 || vsbdf->bus != 0 || vsbdf->fn != 0 ) + { + gdprintk(XENLOG_ERR, + "vSBDF %pp: segment, bus and function should be 0\n", + vsbdf); + return -EOPNOTSUPP; + } + new_dev_number = vsbdf->dev; + if ( test_bit(new_dev_number, &d->vpci_dev_assigned_map) ) + { + gdprintk(XENLOG_ERR, "vSBDF %pp already assigned\n", vsbdf); + return -EOPNOTSUPP; + } + } + + __set_bit(new_dev_number, &d->vpci_dev_assigned_map); /* * Both segment and bus number are 0: * - we emulate a single host bridge for the guest, e.g. segment 0 @@ -148,7 +170,7 @@ void vpci_deassign_device(struct pci_dev *pdev) pdev->vpci = NULL; } -int vpci_assign_device(struct pci_dev *pdev) +int vpci_assign_device(struct pci_dev *pdev, pci_sbdf_t *vsbdf) { unsigned int i; const unsigned long *ro_map; @@ -176,7 +198,7 @@ int vpci_assign_device(struct pci_dev *pdev) #ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT pdev->vpci->guest_sbdf.sbdf = ~0; - rc = add_virtual_device(pdev); + rc = add_virtual_device(pdev, vsbdf); if ( rc ) goto out; #endif diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index a33f9ec32b..60a59ce378 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -504,7 +504,12 @@ struct xen_domctl_sendtrigger { /* Assign a device to a guest. Sets up IOMMU structures. */ -/* XEN_DOMCTL_assign_device */ +/* XEN_DOMCTL_assign_device + * when assigning a PCI device, it is possible to either request + * or provide a virtual SBDF. When virtual_sbdf equals to + * XEN_DOMCTL_DEV_SDBF_ANY, hypervisor will return allocated + * vSBDF back. + */ /* * XEN_DOMCTL_test_assign_device: Pass DOMID_INVALID to find out whether the * given device is assigned to any DomU at all. Pass a specific domain ID to @@ -528,6 +533,8 @@ struct xen_domctl_assign_device { union { struct { uint32_t machine_sbdf; /* machine PCI ID of assigned device */ + uint32_t virtual_sbdf; /* IN/OUT virtual SBDF of the device */ +#define XEN_DOMCTL_DEV_SDBF_ANY 0xFFFFFFFF /* request a free SBDF */ } pci; struct { uint32_t size; /* Length of the path */ diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index ec6598237b..f66a641e4f 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -33,7 +33,7 @@ typedef int vpci_register_init_t(struct pci_dev *dev); __used_section(".data.vpci." p) = x /* Assign vPCI to device by adding handlers to device. */ -int __must_check vpci_assign_device(struct pci_dev *pdev); +int __must_check vpci_assign_device(struct pci_dev *pdev, pci_sbdf_t *vsbdf); /* Remove all handlers and free vpci related structures. */ void vpci_deassign_device(struct pci_dev *pdev); @@ -265,7 +265,7 @@ bool vpci_ecam_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, #else /* !CONFIG_HAS_VPCI */ struct vpci_vcpu {}; -static inline int vpci_assign_device(struct pci_dev *pdev) +static inline int vpci_assign_device(struct pci_dev *pdev, pci_sbdf_t *vsbdf) { return 0; } From patchwork Wed Dec 13 23:44:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13492129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 21A73C4332F for ; Wed, 13 Dec 2023 23:45:35 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.654257.1020975 (Exim 4.92) (envelope-from ) id 1rDYun-0004Oe-83; Wed, 13 Dec 2023 23:45:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 654257.1020975; Wed, 13 Dec 2023 23:45:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rDYun-0004OV-4T; Wed, 13 Dec 2023 23:45:21 +0000 Received: by outflank-mailman (input) for mailman id 654257; Wed, 13 Dec 2023 23:45:19 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rDYul-0003et-IA for xen-devel@lists.xenproject.org; Wed, 13 Dec 2023 23:45:19 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id aa52f849-9a11-11ee-9b0f-b553b5be7939; Thu, 14 Dec 2023 00:45:16 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BDJ2iKZ012660; Wed, 13 Dec 2023 23:45:04 GMT Received: from eur04-he1-obe.outbound.protection.outlook.com (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uy0y4vams-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Dec 2023 23:45:03 +0000 (GMT) Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by AS8PR03MB8666.eurprd03.prod.outlook.com (2603:10a6:20b:54c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.26; Wed, 13 Dec 2023 23:44:58 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7091.022; Wed, 13 Dec 2023 23:44:58 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: aa52f849-9a11-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HEAl8RyPV6a3z3UxXq+cAewuXcasPSFXNuJNEOZ0H5fZcZJ7IlMXwCbaOlwnyi+pXqdul2qFAmnsE85Oy1vksVc7KDCkI3Blkao52Y9jCWmnKQJgfajk1iUxlR0dlC6BXAUyU1nBY3Q/8+fa9hVWsf4WcJBwu8WDXb0+PhAGO3SnV89bpOLBEHEsjor32Zh0CJIf98RfB+wGGSFX37fHZZ4Z/hXvTFRv6P5L+JChn+VkM0o/hmCYGN+hP4kQup0pjSFHeQ8r1LGFsr1wZM5l4td3mRihXvUexzk978eSWRZpqk+Gi4nfjuZliIGvlU4dpJwVAsGrwEvXRDCeIbIZ5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8N7z/YACwnG4WAFTZ48WWNHL71zQbWaz9C/HbJ/rSg4=; b=RlIlBFnmG5ezZ8OU0nGgaKxDDq5cXtsapX4KnWslb0f6WyJaKmCoKRKTlIWfp7J+DkkJx0uuokS+WX1DFJwrrpR4lv+OhnCTZCKhNo+aaLNHdqoZS+xOQZ+/Fa53GK6mzE+gsdadczv+DArR2lnXmhKZUgWckeyI9ZZrkE9PFZkOD71RYDsmDJxyg/pQFu8g9DgsB4Jw/R7XK9KjLOAdjUyt5/mZvmujT7KLl8yS3067MSqGEZ3y2XxZKjo7/QPJEYQCy/5w/CT9/gJ0q2/1sXfabWJDtF2Du8EG6gIvGizlCFr+k5+3JakyJJhMBuWrjhQs74hqWE/srzcMwVE6GQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8N7z/YACwnG4WAFTZ48WWNHL71zQbWaz9C/HbJ/rSg4=; b=HabljxAwPla2Idr/nyLTjxnuuWUaA56c9RQ1/LexmTuhYZY6ZJCyhtIkBMkgEumyQ+AdBKHkuj+YKKy5AG/bE3betzkWmppI13fiSrfs0lQ+RItBKtC+AoCloRHmNrir4Xlj4GC5/aKpttjzUBmNr/sUKpZvrBkZ9eToOyUgifQp9FUAdER/8U8/3KCg2Gpl2+sn9BLyo5zCXZvYTIyoYSLHRPh0McEp0FJHIPbaH12vT7/xyvH0NhVGfX8DuKVFe346J8CH+dg6cMkxi5ycrzHhSYsQJjT/u2p2c0Vnpu1yCVs/chIdZlZaYDAOVeeMKDMCnnQmhzEiNan/VcWY/A== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Wei Liu , Anthony PERARD , Juergen Gross , Christian Lindig , David Scott , =?iso-8859-1?q?Marek_Marczykowski-G=F3recki?= Subject: [RFC,FUTURE 2/3] tools: libxc: add virtual_sbdf parameter to xc_assign_device Thread-Topic: [RFC,FUTURE 2/3] tools: libxc: add virtual_sbdf parameter to xc_assign_device Thread-Index: AQHaLh5h1p+/0eroN0+FN9vNJqoD+w== Date: Wed, 13 Dec 2023 23:44:58 +0000 Message-ID: <20231213234345.779722-3-volodymyr_babchuk@epam.com> References: <20231213234345.779722-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231213234345.779722-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.43.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|AS8PR03MB8666:EE_ x-ms-office365-filtering-correlation-id: b531f16f-b7ab-4881-28f8-08dbfc358458 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: +TVcbus1t8Xq089Hl8dEJ8MTTAEjayDFwDo/ceGAhqk7sS+QpYTJj/rOAixog2is7vJTS8OQ9CJXcGWhNjRX31sTfheVNrpkiss64IBJ3pSlb/7BHN0f3vwWLBahQ0p+Vl69JSyPkKNKLMZXUyi5+JpORt0Y3nl1HD6uW+6rd7hNuCSHxIME/9GB+YKrJ99YpE3b8rrtf1MN8jZmXGbXSyTRjMkoOjspr+BO7U8/lbOXWfYeKpS7T0x1Ggb7eiLfybxMgBGh7ae8EDpAPx1ZJhzCSSCJEUFFY5gKLtHE7oyYFMO/fBG+misAqcbrEWdQndTSlOE+heZRfVEbq0ZikQk/9DFoLdLWWeZbEGhJ8w/d+YjXdz0H1XXT5UKva5yV0I/y1mx7V5A3cnyFIMDkKKuQwHod6W1NXdnC5hq/2cuc9SCj0v3W/utYE5gh2R/v8QfHVsWUkJeZgg8rgVIdGPmytdyAm7XdTW2k2wkziajFcvL0MkWoUNgKLiU48DSWfq9HnG7+CeUzLPWpu0enKBLOoJQ82xcFSARRA5fcieKC7NBCVys1u0VopUlgc9xtMMZtubaubtiWTfgxPyZO5S/OvNthjKe/N4K0q6znCj5Af3IcutMezg4ggnY5IN2f x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(366004)(376002)(396003)(39860400002)(136003)(230922051799003)(451199024)(1800799012)(186009)(64100799003)(1076003)(26005)(2616005)(71200400001)(6506007)(6512007)(55236004)(83380400001)(5660300002)(7416002)(8936002)(41300700001)(2906002)(4326008)(478600001)(6486002)(66556008)(8676002)(76116006)(66946007)(91956017)(316002)(6916009)(54906003)(64756008)(66446008)(66476007)(86362001)(122000001)(38100700002)(36756003)(38070700009);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?/JM9ee2v30wqwFP+KWXMttO?= =?iso-8859-1?q?DE3YQRUIjNoY0ZU/rXO7KtJYyriWCGdO1WWlN6G6oODVENv93DvrzV6EMQmf?= =?iso-8859-1?q?Z28ArZerTmuEpjlTajwI6y7EtmzbY8qcV9IkbPxnuKEicbYt+WA8HZmn7vR9?= =?iso-8859-1?q?pLUJ8ET5PZMGDh9Q6BGBqfMZNAqePQZjuocmEZRx2YEu5YEpyOBMjQ8h3Z98?= =?iso-8859-1?q?jgk3wUqeDjJFkNroR4qidlLTpYkeZohx4Gm5MjZY7LwXC9fnAQPc4TMPGOck?= =?iso-8859-1?q?ExPh+zDOpWql+n30v/GA0fzqI3XNGEg1tZcMJNNmubzX6uBOPeey5gVkhLui?= =?iso-8859-1?q?7VSGGNQiQhd0e9eZ+pIJIva9nUcaokZ5kmpr7fqaqRuTGSyQau6yvvuvhMWb?= =?iso-8859-1?q?BnUuNIYL724XJrnFl5N0gpgWt5PSoMeZKpTV5P+ePRHmGHsbuh4EeP0gClIX?= =?iso-8859-1?q?XpWFZjLmpCRbf2ySS2y1zaNuzfk+UX2Hs+E0FWdphxOaf95zs0Y3CVt69MjE?= =?iso-8859-1?q?bIzvP4aTkICSVEx0l0HQ4GAg3592u98q4Y4uYhm2bi4Q550NWqUOVW0wz/Qx?= =?iso-8859-1?q?aKcGYQf2vf/5pOC68IXaM2h47bDq19VlAxjCPSRAlXWTLF/It7bmfkotriKY?= =?iso-8859-1?q?0g6gVHQe50VRDLxEiBtVZzm7G7J8/iPqh/IN/U3Yv5zktisgy7GAWl44JaKP?= =?iso-8859-1?q?0LUs2wC6s8Or2QZeCXJwB/dLhUiNYvxASZKvY8IPTYL4K6SYMf42Lh3B0Kbk?= =?iso-8859-1?q?ITey7pdsyUQ9bfwe0XTv9uKGqMwnmDMrzYqNjT/VrYXzB6HMm+c+1S/9Trja?= =?iso-8859-1?q?gMcAyl0v+7nPYi67b1E3LZlEH+Ixe+w/RIywNvNAldrhz/oQBDoBp5vqw6MW?= =?iso-8859-1?q?CyUvTApu4J3yR+zPhqcL9AjK3BQaia40FKdviyuXGXdKw+n7CAjXCErGoz1X?= =?iso-8859-1?q?NF0Qo99L7DfQ0WRdS4GisRNxKPWXfMa7GIbIQqm8euJQrm2DvoDSQJhFhGtj?= =?iso-8859-1?q?np7gNKTtjdcH2PrRxJWkfLKK3R+LCBuYFgvlH18hVUCAJNHmfFPlM4HbkmTW?= =?iso-8859-1?q?uU5tIu2+GNt7X1GYi8ztXXFGdB80yVgNluBE5iUhgs1wa8NWpblmHs28drnF?= =?iso-8859-1?q?hcpPHOOyY+jpGlV7Xj4ydZIUcFY7uQNkKDt4t0FrzJ3xcGKsyYDUZs7TxGgu?= =?iso-8859-1?q?00EuKI/XgYk/tfhzSF3+D+Akg7RM5TxUObdTpOPxWLH0hVsd3czI4kzUKqG2?= =?iso-8859-1?q?/w3VnCd+0jX+s9Ir5PQXJW+8Byh/Vns/IgEBrmqH4S2vzdd4fkTyc1+9Sx/o?= =?iso-8859-1?q?jSsMfClqdlXgem3b7biTdsPL9396NVgqYdY2C5bZsfVOs1n617zXtE+d+bmP?= =?iso-8859-1?q?yOnZVowDIBBwDX4cKu9KiksSuJzi6liv7Af+HfAvUa66gMQ98w1reSSTt8nv?= =?iso-8859-1?q?Ru7SNMxoglgl0YHTniHfr+dQWdDYas8rSS0f8+jUBRDJ4VvFC+Rlhvy0Cyq2?= =?iso-8859-1?q?ScZpNL2uwSKT9d6ZJ0N0Te7mA6Hs1MMVgiidFFtOMZVdIMUpLg0sapkkiiZj?= =?iso-8859-1?q?U03HsyEcckMcixPjVGBMMr/kzYFF82E45J8UvyfAPRWWDlqDUS19HbxI+nhg?= =?iso-8859-1?q?ybQyOIhRWxol6TNbp+0yQa64IJ512iK7auOOMRQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b531f16f-b7ab-4881-28f8-08dbfc358458 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2023 23:44:58.2239 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: u/yOERxGxfwt0JZo9G71uQDdHuVFpu/Nzku2PDDpLfk3S2eAEdFq0i0gMy0oVeEPiJIpWPklL9jDvdm7usCgOLedO12/pd0D/3i8njv8Uh0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB8666 X-Proofpoint-GUID: N-KnCT21uJ-hmOONFHBqsIY3oIezIrNn X-Proofpoint-ORIG-GUID: N-KnCT21uJ-hmOONFHBqsIY3oIezIrNn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 impostorscore=0 lowpriorityscore=0 adultscore=0 clxscore=1015 spamscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312130168 Now, when "assign_device" domctl supports virtual_sbdf option, make it available to libxc users. This is an optional parameter, if it is not provided, xc_assign_device() will ask hypervisor to allocate a free vSBDF. Signed-off-by: Volodymyr Babchuk --- This clearly breaks libxc API. So I wanted to discuss a proper way of doing this. Should I introduce a new xc_assign_device() function with an extra parameter? --- tools/include/xenctrl.h | 1 + tools/libs/ctrl/xc_domain.c | 14 ++++++++++++-- tools/libs/light/libxl_pci.c | 4 ++-- tools/ocaml/libs/xc/xenctrl_stubs.c | 2 +- tools/python/xen/lowlevel/xc/xc.c | 2 +- 5 files changed, 17 insertions(+), 6 deletions(-) diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h index 2ef8b4e054..ccc0745c35 100644 --- a/tools/include/xenctrl.h +++ b/tools/include/xenctrl.h @@ -1689,6 +1689,7 @@ int xc_get_hvm_param(xc_interface *handle, uint32_t dom, int param, unsigned lon int xc_assign_device(xc_interface *xch, uint32_t domid, uint32_t machine_sbdf, + uint32_t *virtual_sbdf, uint32_t flag); int xc_get_device_group(xc_interface *xch, diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c index 2405e39517..ca60b13a89 100644 --- a/tools/libs/ctrl/xc_domain.c +++ b/tools/libs/ctrl/xc_domain.c @@ -1497,18 +1497,28 @@ int xc_assign_device( xc_interface *xch, uint32_t domid, uint32_t machine_sbdf, + uint32_t *virtual_sbdf, uint32_t flags) { + int rc; struct xen_domctl domctl = {}; domctl.cmd = XEN_DOMCTL_assign_device; domctl.domain = domid; domctl.u.assign_device.dev = XEN_DOMCTL_DEV_PCI; domctl.u.assign_device.u.pci.machine_sbdf = machine_sbdf; - domctl.u.assign_device.u.pci.virtual_sbdf = XEN_DOMCTL_DEV_SDBF_ANY; + if (!virtual_sbdf) + domctl.u.assign_device.u.pci.virtual_sbdf = XEN_DOMCTL_DEV_SDBF_ANY; + else + domctl.u.assign_device.u.pci.virtual_sbdf = *virtual_sbdf; domctl.u.assign_device.flags = flags; - return do_domctl(xch, &domctl); + rc = do_domctl(xch, &domctl); + + if (!rc && virtual_sbdf) + *virtual_sbdf = domctl.u.assign_device.u.pci.virtual_sbdf; + + return rc; } int xc_get_device_group( diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c index 96cb4da079..723e577ef5 100644 --- a/tools/libs/light/libxl_pci.c +++ b/tools/libs/light/libxl_pci.c @@ -842,7 +842,7 @@ name: * so always pass XEN_DOMCTL_DEV_RDM_RELAXED to avoid assignment being * unnecessarily denied. */ - rc = xc_assign_device(ctx->xch, DOMID_IO, pci_encode_bdf(pci), + rc = xc_assign_device(ctx->xch, DOMID_IO, pci_encode_bdf(pci), NULL, XEN_DOMCTL_DEV_RDM_RELAXED); if ( rc < 0 ) { LOG(ERROR, "failed to quarantine "PCI_BDF, dom, bus, dev, func); @@ -1524,7 +1524,7 @@ out_no_irq: rc = ERROR_FAIL; goto out; } - r = xc_assign_device(ctx->xch, domid, pci_encode_bdf(pci), flag); + r = xc_assign_device(ctx->xch, domid, pci_encode_bdf(pci), NULL, flag); if (r < 0 && (hvm || errno != ENOSYS)) { LOGED(ERROR, domainid, "xc_assign_device failed"); rc = ERROR_FAIL; diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c index 3703f48c74..55d257abbb 100644 --- a/tools/ocaml/libs/xc/xenctrl_stubs.c +++ b/tools/ocaml/libs/xc/xenctrl_stubs.c @@ -1323,7 +1323,7 @@ CAMLprim value stub_xc_domain_assign_device(value xch_val, value domid, value de func = Int_val(Field(desc, 3)); sbdf = encode_sbdf(domain, bus, dev, func); - ret = xc_assign_device(xch, Int_val(domid), sbdf, + ret = xc_assign_device(xch, Int_val(domid), sbdf, NULL, XEN_DOMCTL_DEV_RDM_RELAXED); if (ret < 0) diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c index d3ea350e07..1e210c78b7 100644 --- a/tools/python/xen/lowlevel/xc/xc.c +++ b/tools/python/xen/lowlevel/xc/xc.c @@ -587,7 +587,7 @@ static PyObject *pyxc_assign_device(XcObject *self, sbdf |= (dev & 0x1f) << 3; sbdf |= (func & 0x7); - if ( xc_assign_device(self->xc_handle, dom, sbdf, 0) != 0 ) + if ( xc_assign_device(self->xc_handle, dom, sbdf, NULL, 0) != 0 ) { if (errno == ENOSYS) sbdf = -1; From patchwork Wed Dec 13 23:44:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13492131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 38A73C41535 for ; Wed, 13 Dec 2023 23:45:35 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.654256.1020960 (Exim 4.92) (envelope-from ) id 1rDYul-0003xK-Pr; Wed, 13 Dec 2023 23:45:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 654256.1020960; Wed, 13 Dec 2023 23:45:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rDYul-0003x7-KK; Wed, 13 Dec 2023 23:45:19 +0000 Received: by outflank-mailman (input) for mailman id 654256; Wed, 13 Dec 2023 23:45:18 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rDYuk-0003et-I4 for xen-devel@lists.xenproject.org; Wed, 13 Dec 2023 23:45:18 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id aa0628d6-9a11-11ee-9b0f-b553b5be7939; Thu, 14 Dec 2023 00:45:16 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BDJ2iKa012660; Wed, 13 Dec 2023 23:45:05 GMT Received: from eur04-he1-obe.outbound.protection.outlook.com (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uy0y4vams-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Dec 2023 23:45:04 +0000 (GMT) Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by AS8PR03MB8666.eurprd03.prod.outlook.com (2603:10a6:20b:54c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.26; Wed, 13 Dec 2023 23:44:59 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7091.022; Wed, 13 Dec 2023 23:44:59 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: aa0628d6-9a11-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=itAaYzhZY9Wp/aQ+boY7W3UK35VF9yEAd1O3Yh+a5ht6XbJ1UcQGystEJmxWmt+EeLQifxECsIQNhFDXba8N0HSI2YOvL1EhGbgcMCW/SXocdyLnrmLi5p/+Ar+TsyoczQFYzDjaNLkY08iuqXH4fUwwkhQennoVtqu7xuHCciDJnnrBz+7MBNqReGFz8NrUge9jVPh8qlzsRG7Ba0sGFYkeCnggPyDdNGOVfqUoOw4tdQjKUEAqMdTYx+fKn5V4gAzyhZYZRHfFpGobDdy7wCsv7fpEBEc6mKBMXdaP7eBl/MihXjCD03FgyOeCjNkyESs+KoKpV7yMtVm8sLYQGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YpRsE4hzDvT+283KhRaNySn9/iNEhYA8xfPItLgPBVE=; b=VmtLGRjOOiKw1o/IzNQWy+yT2ygjytJaqQPk8ZT9hA7o5+cPM3vJxD3BtrDjA+06itBLMwKmcy15e5qW6RxxYlwVms/FTE1YX9BHAZ5+K1Zak5GM3DmBgo+4AQH3HY58ApVuVTCHcujxMiE0xPUPueekt5sJqwnslmqZ1hOiBuucXJS1OtFzCFf8lMeXePrfv8wB0tnGIZv9WedJta23p1y04494wGZqwrsJm10siyhTeyXFs/viCfLfV8qR48UVUi4+BAH43QRGqW1+nMXoLomaXfDQ4P6wmlG0JC5+FngztxPWjatVV4he4eIAVSM4GjcN6zq+Ux/CZqilr+M+aA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YpRsE4hzDvT+283KhRaNySn9/iNEhYA8xfPItLgPBVE=; b=ZiXlOtY3tY+OhOdjLJz4pYh/qD46uQ7jnrfMli/Byp3YPkKTWJGClY+MdAxH0LIo4kZBFhVLCsxyXcfSS0P2i/KteX5ynqARprFIxEkmcThDeze8T7SsU39ioKRZ16ETFn2+5bovuucZxe1fXvg8offyVnsrV8CIVi5gE7KlEMr0zT5aEzYfZ8zqFZYB14v+uEKiJnkEAirXN8UyIARpgzCrABPTS4ITIwqVLMYOO+0b4UfFB8//wmxmluLztL+4HWbY7LN/B/ko2WPysDAiktB+tshVF8fKCKhHYKr1Q/l/fkkMP0RfvyGpVBb7c425lwbmFKOPdYPlM5h1cLZQNA== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Wei Liu , Anthony PERARD , Juergen Gross Subject: [RFC,FUTURE 3/3] tools: libxl: use "vslot" parameter when assigning a PCI device Thread-Topic: [RFC,FUTURE 3/3] tools: libxl: use "vslot" parameter when assigning a PCI device Thread-Index: AQHaLh5hhUhc1caE+UyL8B+mLfWnJg== Date: Wed, 13 Dec 2023 23:44:58 +0000 Message-ID: <20231213234345.779722-4-volodymyr_babchuk@epam.com> References: <20231213234345.779722-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231213234345.779722-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.43.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|AS8PR03MB8666:EE_ x-ms-office365-filtering-correlation-id: 1ef081db-9247-460f-e608-08dbfc358489 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: hPQQZDrRDl3eLGjHCj3alzVLu3Gfff2aS6wxGU0SX7NlbZDiFpn1WxFzdVaCnYKJXTl1VXtTUDSXLpOrqgYEoXpCyO4ks5/tYsb06c0QN5Al4m0kkE6/II8JfuGeoXDDw7HeMBbN3y2u3lAvGP67hCYvYA9uPB7wwMKDMq91LS8krEJqn/cspPluht1mtzEjhBPeHxQqg//KWAvKFgeRcGf8OfgCmszsDMHnKwi/TCrWmTkM8p9Qz9POFWs5uga2nt9YRd0UsE3AoUyLgYyrA03Ss7e1x43qtA5tLX+izT3NVWAKEmgUnR+nlU4YB+aPsxg3Ruf0HPkKA2Nzbe4GsP+qPHTvP5FVHhi1eFS4AFXmLE06v38oylEh4/sSFu096qkoku8n9tnKGrGrJYGe80tZwY3RsclfA/ku++OiT55EQOMeoEBzuzVfyHQy8Z/gjbdGR2kF6zvmar/xbFC2Z73ioKt0Y+35mNxjGO76tMZcBppZtup8PdvQIUyj+VF90nOXhu1Nrj+DC49Jj1RpQFAd1jAV27uO+7gA+SU1My24r3zC1r7S/3+RE+jbb77AxyjoiM7BQsrk9InYW6t637batrEW765pV0OpH9P5W3XOs8JHRsdhRK1/HQyTBHlQ x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(366004)(376002)(396003)(39860400002)(136003)(230922051799003)(451199024)(1800799012)(186009)(64100799003)(1076003)(26005)(2616005)(71200400001)(6506007)(6512007)(55236004)(83380400001)(5660300002)(8936002)(41300700001)(2906002)(4326008)(478600001)(6486002)(66556008)(8676002)(76116006)(66946007)(91956017)(316002)(6916009)(54906003)(64756008)(66446008)(66476007)(86362001)(122000001)(38100700002)(36756003)(38070700009);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?ASbcbR7+SLkXhVLuTiwMCdS?= =?iso-8859-1?q?AHP85ravYLOYZXXt1fHi1bQ8dQd8MQa3Gas2USLgNSHPspBx5kIW1xik7RU5?= =?iso-8859-1?q?yFJjsmVVpk3vcTIf4duGJBSEGabEAXCAWj++DcmoWImJtkR0sEzQBPFhzi5a?= =?iso-8859-1?q?4rI4cjoq6RUMRjOhaNwFnz/TkTQaIGcs61ckoaHlf0RQWYwNc2/28yQfYB25?= =?iso-8859-1?q?MpJ8UF8lcyrAKQfYwNFFFHoTWkxg1Z1EjupkMrmZE4iVX51RDOaA3Uc40xFB?= =?iso-8859-1?q?SOPpPxJUjnSQ9/UhgvEHD7Z6ExyMrmWRfh0Qixy3RuQdFWNy5sE2eUim35Ar?= =?iso-8859-1?q?ayCzoZ0qBAh9gexcMqCVRMoFbs9j2uj0OXXDQnXf8buZpTCd7euxeMI5sfwN?= =?iso-8859-1?q?UnBGblSMtx4MT/H+fCe86ASDXyNaW8NeXye6ldwMkAQCkmXX/woGE3RVez1O?= =?iso-8859-1?q?cdFdvg+uwavXYMo4q/S5W3FrA53Io/kvjUcnkEmTpSqpC8YFvuV56v7Am+H0?= =?iso-8859-1?q?HgVwxwuCq7vzcp76uSgdBuAQiz2SYu356sBQIxYZMnmH5RoEb1GhM7EnYT/i?= =?iso-8859-1?q?BGiXuarAo4R5X3uxlJyz/AHaN7Lp3piNstUKer8vE0/KDRBvVw/nPF5PpYyW?= =?iso-8859-1?q?Hp8b1O2zfsI5f7TzTiE5EYMtFTxlUQGk02XtpFSVsv4dCpyqlXZCGl08osEF?= =?iso-8859-1?q?6AT7R191b1gMkzi9YMZFCoaHlYpCE9RFqQhYDSRWoLRlZ1bg12D/LguJ7Rnt?= =?iso-8859-1?q?ULIUta+2RC+/UTrmBprE2TuAD1S8WVrWIpMi57DCscG8o387532N3Qudh7W1?= =?iso-8859-1?q?qZAH7+bS7XjSQR/noi+ejVmGw6uYHAzuWXSIzNBSXysjMiT/IsreP+2zVqb5?= =?iso-8859-1?q?uNsrL6x6hWyYvPpxW91Jz0tgvs9ZfkO/jfWMzsBYaXUCvs6o3toRZt5Ju6I4?= =?iso-8859-1?q?kbtf6beYCRNPAAjcoaGJPo+oYPAvhsYtMi07i1iWS6Xpy+oDQF9wD/vJARDn?= =?iso-8859-1?q?/JOy6it6+9eCakW4Sp3mGiXl5sSgH+d5CUbCL4wUUweTIyIdGaUrm7dSLgZw?= =?iso-8859-1?q?+JCbqZgmgnWw9SxraWy5FWMD6/CJB06Kwf1wAgCcjuINWQWg2DHnf76bDbcK?= =?iso-8859-1?q?ZGDKbMwiG01ZLp2uX51nL8PdGvqqAprjEf0Iubemlk7IHPV1/ztvCSmWWS/W?= =?iso-8859-1?q?4HAbMsoAHxybVOJFPVVnShgVMMJz+a+n2WjphPu+Y3i/Qu8HsJ/oVz5NkB/b?= =?iso-8859-1?q?2pd2m3cEl1NMuTOY0MrHQlbrruNJ3AwSayqv1+yJ1YkqODHvekNC89ucbyTM?= =?iso-8859-1?q?0xvTphpZWj31B6L8Ydk+vUzQcF2vvfzTO1ObcbajemqR4CLkQEFsLTCV7uxz?= =?iso-8859-1?q?C87PnoFsiWmGqOO0/G986DVjxQ0n61ONDRYAAaoKpkwcm4e5YUBSNclOCqo5?= =?iso-8859-1?q?JaM69ImTs378Sg4FL/UH6mpX5XMwJFJ4R3Gpq4f16Xt8HJ2CG0PxF8M2NXpl?= =?iso-8859-1?q?gRbUBCPXLnol7kQ1BiSYE48VPW1TAWo+MZIjFd/n30bTsk12jh1Vbn0HnxND?= =?iso-8859-1?q?DKPVPopjgorSj1Yl8f8RWEvg8GZLpgVBJIUkP6TLRza+KcW4wuL8zS8+MVlz?= =?iso-8859-1?q?6Z7Dzx5xSg3miOg8x6ZZaLYlpUhK46pqOqsGc9w=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1ef081db-9247-460f-e608-08dbfc358489 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2023 23:44:58.5066 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: VgNq8jdL/4lbhXmekqRDzCZqyxNT/kLCet0DjsG9Ie2r30n3dlDLmJSh5WIzS8b6rTeaxQbzEcQ5P8T1foEgjxS+yonqaGWpysGNdb5zsDU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB8666 X-Proofpoint-GUID: aQJr0MHTZ6mVGRNP0WHuntxA_k-knlLQ X-Proofpoint-ORIG-GUID: aQJr0MHTZ6mVGRNP0WHuntxA_k-knlLQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 impostorscore=0 lowpriorityscore=0 adultscore=0 clxscore=1015 spamscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312130168 Now, when "assign_device" domctl supports providing a virtual SBDF, we can leverage "vslot" option from PCI_SPEC_STRING and pass user requested function number to the hypervisor. Signed-off-by: Volodymyr Babchuk --- tools/libs/light/libxl_pci.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c index 723e577ef5..13c7b12343 100644 --- a/tools/libs/light/libxl_pci.c +++ b/tools/libs/light/libxl_pci.c @@ -1421,6 +1421,7 @@ static void pci_add_dm_done(libxl__egc *egc, uint32_t flag = XEN_DOMCTL_DEV_RDM_RELAXED; uint32_t domainid = domid; bool isstubdom = libxl_is_stubdom(ctx, domid, &domainid); + uint32_t vsbdf = XEN_DOMCTL_DEV_SDBF_ANY; /* Convenience aliases */ bool starting = pas->starting; @@ -1524,12 +1525,16 @@ out_no_irq: rc = ERROR_FAIL; goto out; } - r = xc_assign_device(ctx->xch, domid, pci_encode_bdf(pci), NULL, flag); + if (pci->vdevfn) + vsbdf = pci->vdevfn; + + r = xc_assign_device(ctx->xch, domid, pci_encode_bdf(pci), &vsbdf, flag); if (r < 0 && (hvm || errno != ENOSYS)) { LOGED(ERROR, domainid, "xc_assign_device failed"); rc = ERROR_FAIL; goto out; } + LOGD(DEBUG, domainid, "Got vSBDF %x", vsbdf); } if (!libxl_get_stubdom_id(CTX, domid))