From patchwork Wed Jul 17 01:00:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 11046857 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 97C076C5 for ; Wed, 17 Jul 2019 01:03:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 87DAF286C6 for ; Wed, 17 Jul 2019 01:03:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 74F4728734; Wed, 17 Jul 2019 01:03:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 53537285ED for ; Wed, 17 Jul 2019 01:03:16 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnYK3-000554-Um; Wed, 17 Jul 2019 01:01:31 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnYK3-00054t-57 for xen-devel@lists.xenproject.org; Wed, 17 Jul 2019 01:01:31 +0000 X-Inumbo-ID: 68650566-a82e-11e9-ac55-63be4eb0056e Received: from new3-smtp.messagingengine.com (unknown [66.111.4.229]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 68650566-a82e-11e9-ac55-63be4eb0056e; Wed, 17 Jul 2019 01:01:28 +0000 (UTC) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailnew.nyi.internal (Postfix) with ESMTP id 678DD2818; Tue, 16 Jul 2019 21:01:28 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Tue, 16 Jul 2019 21:01:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=oXw5SYpgWONCsQMlX8Tt0b6R78vNuA4o5rHtiIe0f 38=; b=dzcZv2sh31pMfqrHX68hT8O0/s/4lkmZTBmgyf45IiQPtWLtfLr6qFzwk viO0bVYCX+T6hCB77CaO0dr+bK4TjvQq/pW1IiA+dgZHVEdxXZZ4i2BN1wmExxtv P6WVlxkB8xXnHWRjT/y4ovhenOUIFAo8pHuGsdJ4kAZuJdroiUOxMU9CJNysnMtg Ih6myif9KbTwybJb84E04kmkaj7AIRKgZ8QLow9JtdbqdDftCQDoka8Kc7VQh1VI Zz9HJX86hDcy/QkMovjYsQmTz3YwYbK1AjZ1Wo2ExXM+vckDflPYCX3mjMP+RjZS /eMiur+2BD/XShLIgLW9Y96ugmZKQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddriedugdegvdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghrvghk ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucfkphepledurdeihedrfeegrdef feenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslh gvthhhihhnghhslhgrsgdrtghomhenucevlhhushhtvghrufhiiigvpedu X-ME-Proxy: Received: from localhost.localdomain (ip5b412221.dynamic.kabel-deutschland.de [91.65.34.33]) by mail.messagingengine.com (Postfix) with ESMTPA id 923C638008A; Tue, 16 Jul 2019 21:01:26 -0400 (EDT) From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Date: Wed, 17 Jul 2019 03:00:43 +0200 Message-Id: <7280f16277aa9d7bdc9c2373277ef1b40459090b.1563325215.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v5 5/6] xen/x86: add PHYSDEVOP_msi_control X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , =?utf-8?q?Marek_Marczykowski-G?= =?utf-8?q?=C3=B3recki?= , Tim Deegan , Julien Grall , Jan Beulich , Daniel De Graaf , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Allow device model running in stubdomain to enable/disable MSI(-X), bypassing pciback. While pciback is still used to access config space from within stubdomain, it refuse to write to PCI_MSI_FLAGS_ENABLE/PCI_MSIX_FLAGS_ENABLE in non-permissive mode. Which is the right thing to do for PV domain (the main use case for pciback), as PV domain should use XEN_PCI_OP_* commands for that. Unfortunately those commands are not good for stubdomain use, as they configure MSI in dom0's kernel too, which should not happen for HVM domain. This new physdevop is allowed only for stubdomain controlling the domain which own the device. Signed-off-by: Marek Marczykowski-Górecki --- Changes in v3: - new patch Changes in v4: - adjust code style - s/msi_msix/msi/ - add msi_set_enable XSM hook - flatten struct physdev_msi_set_enable - add to include/xlat.lst Changes in v5: - rename to PHYSDEVOP_msi_control - combine "mode" and "enable" into "flags" - refuse to enable both MSI and MSI-X, and also to enable MSI(-X) on incapable device - disable/enable INTx when enabling/disabling MSI (?) - refuse if !use_msi - adjust flask hook to make more sense (require "setup" access on device, not on domain) - rebase on master I'm not sure if XSM part is correct, compile-tested only, as I'm not sure how to set the policy. --- xen/arch/x86/msi.c | 42 ++++++++++++++++++++++++++++++- xen/arch/x86/physdev.c | 25 ++++++++++++++++++- xen/arch/x86/x86_64/physdev.c | 4 +++- xen/include/asm-x86/msi.h | 1 +- xen/include/public/physdev.h | 16 +++++++++++- xen/include/xlat.lst | 1 +- xen/include/xsm/dummy.h | 7 +++++- xen/include/xsm/xsm.h | 6 ++++- xen/xsm/dummy.c | 1 +- xen/xsm/flask/hooks.c | 24 +++++++++++++++++- xen/xsm/flask/policy/access_vectors | 1 +- 11 files changed, 128 insertions(+) diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c index 89e6116..fca1d04 100644 --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -1475,6 +1475,48 @@ int pci_restore_msi_state(struct pci_dev *pdev) return 0; } +int msi_control(struct pci_dev *pdev, bool msix, bool enable) +{ + int ret; + struct msi_desc *old_desc; + + if ( !use_msi ) + return -EOPNOTSUPP; + + ret = xsm_msi_control(XSM_DM_PRIV, pdev->domain, pdev->sbdf.sbdf, msix, enable); + if ( ret ) + return ret; + + if ( msix ) + { + if ( !pdev->msix ) + return -ENODEV; + old_desc = find_msi_entry(pdev, -1, PCI_CAP_ID_MSI); + if ( old_desc ) + return -EBUSY; + if ( enable ) + pci_intx(pdev, false); + msix_set_enable(pdev, enable); + } + else + { + if ( !pci_find_cap_offset(pdev->seg, + pdev->bus, + PCI_SLOT(pdev->devfn), + PCI_FUNC(pdev->devfn), + PCI_CAP_ID_MSI) ) + return -ENODEV; + old_desc = find_msi_entry(pdev, -1, PCI_CAP_ID_MSIX); + if ( old_desc ) + return -EBUSY; + if ( enable ) + pci_intx(pdev, false); + msi_set_enable(pdev, enable); + } + + return 0; +} + void __init early_msi_init(void) { if ( use_msi < 0 ) diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c index 3a3c158..5000998 100644 --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -662,6 +662,31 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) break; } + case PHYSDEVOP_msi_control: { + struct physdev_msi_control op; + struct pci_dev *pdev; + + ret = -EFAULT; + if ( copy_from_guest(&op, arg, 1) ) + break; + + ret = -EINVAL; + if ( op.flags & ~(PHYSDEVOP_MSI_CONTROL_MSIX | PHYSDEVOP_MSI_CONTROL_ENABLE) ) + break; + + pcidevs_lock(); + pdev = pci_get_pdev(op.seg, op.bus, op.devfn); + if ( pdev ) + ret = msi_control(pdev, + op.flags & PHYSDEVOP_MSI_CONTROL_MSIX, + op.flags & PHYSDEVOP_MSI_CONTROL_ENABLE); + else + ret = -ENODEV; + pcidevs_unlock(); + break; + + } + default: ret = -ENOSYS; break; diff --git a/xen/arch/x86/x86_64/physdev.c b/xen/arch/x86/x86_64/physdev.c index c5a00ea..69b4ce3 100644 --- a/xen/arch/x86/x86_64/physdev.c +++ b/xen/arch/x86/x86_64/physdev.c @@ -76,6 +76,10 @@ CHECK_physdev_pci_device_add CHECK_physdev_pci_device #undef xen_physdev_pci_device +#define xen_physdev_msi_control physdev_msi_control +CHECK_physdev_msi_control +#undef xen_physdev_msi_control + #define COMPAT #undef guest_handle_okay #define guest_handle_okay compat_handle_okay diff --git a/xen/include/asm-x86/msi.h b/xen/include/asm-x86/msi.h index 10387dc..05296de 100644 --- a/xen/include/asm-x86/msi.h +++ b/xen/include/asm-x86/msi.h @@ -252,5 +252,6 @@ void guest_mask_msi_irq(struct irq_desc *, bool mask); void ack_nonmaskable_msi_irq(struct irq_desc *); void end_nonmaskable_msi_irq(struct irq_desc *, u8 vector); void set_msi_affinity(struct irq_desc *, const cpumask_t *); +int msi_control(struct pci_dev *pdev, bool msix, bool enable); #endif /* __ASM_MSI_H */ diff --git a/xen/include/public/physdev.h b/xen/include/public/physdev.h index b6faf83..f9b728f 100644 --- a/xen/include/public/physdev.h +++ b/xen/include/public/physdev.h @@ -344,6 +344,22 @@ struct physdev_dbgp_op { typedef struct physdev_dbgp_op physdev_dbgp_op_t; DEFINE_XEN_GUEST_HANDLE(physdev_dbgp_op_t); +/* when PHYSDEVOP_MSI_CONTROL_MSIX not set, control MSI */ +#define PHYSDEVOP_MSI_CONTROL_MSIX 1 +/* when PHYSDEVOP_MSI_CONTROL_ENABLE not set, disable */ +#define PHYSDEVOP_MSI_CONTROL_ENABLE 2 + +#define PHYSDEVOP_msi_control 32 +struct physdev_msi_control { + /* IN */ + uint16_t seg; + uint8_t bus; + uint8_t devfn; + uint8_t flags; +}; +typedef struct physdev_msi_control physdev_msi_control_t; +DEFINE_XEN_GUEST_HANDLE(physdev_msi_control_t); + /* * Notify that some PIRQ-bound event channels have been unmasked. * ** This command is obsolete since interface version 0x00030202 and is ** diff --git a/xen/include/xlat.lst b/xen/include/xlat.lst index 95f5e55..3082761 100644 --- a/xen/include/xlat.lst +++ b/xen/include/xlat.lst @@ -108,6 +108,7 @@ ? physdev_irq_status_query physdev.h ? physdev_manage_pci physdev.h ? physdev_manage_pci_ext physdev.h +? physdev_msi_control physdev.h ? physdev_pci_device physdev.h ? physdev_pci_device_add physdev.h ? physdev_pci_mmcfg_reserved physdev.h diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h index 01d2814..4801838 100644 --- a/xen/include/xsm/dummy.h +++ b/xen/include/xsm/dummy.h @@ -514,6 +514,13 @@ static XSM_INLINE int xsm_pci_config_permission(XSM_DEFAULT_ARG struct domain *d return xsm_default_action(action, current->domain, d); } +static XSM_INLINE int xsm_msi_control(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf, + uint8_t msix, uint8_t enable) +{ + XSM_ASSERT_ACTION(XSM_DM_PRIV); + return xsm_default_action(action, current->domain, d); +} + static XSM_INLINE int xsm_add_to_physmap(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2) { XSM_ASSERT_ACTION(XSM_TARGET); diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h index b6141f6..bf39dbd 100644 --- a/xen/include/xsm/xsm.h +++ b/xen/include/xsm/xsm.h @@ -106,6 +106,7 @@ struct xsm_operations { int (*iomem_permission) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow); int (*iomem_mapping) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow); int (*pci_config_permission) (struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end, uint8_t access); + int (*msi_control) (struct domain *d, uint32_t machine_bdf, uint8_t msix, uint8_t enable); #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI) int (*get_device_group) (uint32_t machine_bdf); @@ -464,6 +465,11 @@ static inline int xsm_pci_config_permission (xsm_default_t def, struct domain *d return xsm_ops->pci_config_permission(d, machine_bdf, start, end, access); } +static inline int xsm_msi_control (xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint8_t msix, uint8_t enable) +{ + return xsm_ops->msi_control(d, machine_bdf, msix, enable); +} + #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI) static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf) { diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c index c9a566f..878eefe 100644 --- a/xen/xsm/dummy.c +++ b/xen/xsm/dummy.c @@ -81,6 +81,7 @@ void __init xsm_fixup_ops (struct xsm_operations *ops) set_to_dummy_if_null(ops, iomem_permission); set_to_dummy_if_null(ops, iomem_mapping); set_to_dummy_if_null(ops, pci_config_permission); + set_to_dummy_if_null(ops, msi_control); set_to_dummy_if_null(ops, get_vnumainfo); #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI) diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index a7d690a..5fb755e 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -1083,6 +1083,29 @@ static int flask_pci_config_permission(struct domain *d, uint32_t machine_bdf, u } +static int flask_msi_control(struct domain *d, uint32_t machine_bdf, uint8_t msix, uint8_t enable) +{ + uint32_t dsid, rsid; + int rc = -EPERM; + struct avc_audit_data ad; + uint32_t perm; + + AVC_AUDIT_DATA_INIT(&ad, DEV); + ad.device = machine_bdf; + + rc = security_device_sid(machine_bdf, &rsid); + if ( rc ) + return rc; + + rc = avc_current_has_perm(rsid, SECCLASS_RESOURCE, RESOURCE__SETUP, &ad); + if ( rc ) + return rc; + + perm = flask_iommu_resource_use_perm(); + dsid = domain_sid(d); + return avc_has_perm(dsid, rsid, SECCLASS_RESOURCE, perm, &ad); +} + static int flask_resource_plug_core(void) { return avc_current_has_perm(SECINITSID_DOMXEN, SECCLASS_RESOURCE, RESOURCE__PLUG, NULL); @@ -1800,6 +1823,7 @@ static struct xsm_operations flask_ops = { .iomem_permission = flask_iomem_permission, .iomem_mapping = flask_iomem_mapping, .pci_config_permission = flask_pci_config_permission, + .msi_control = flask_msi_control, .resource_plug_core = flask_resource_plug_core, .resource_unplug_core = flask_resource_unplug_core, diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors index 194d743..0ddfc91 100644 --- a/xen/xsm/flask/policy/access_vectors +++ b/xen/xsm/flask/policy/access_vectors @@ -466,6 +466,7 @@ class resource # checked for PHYSDEVOP_restore_msi* (target PCI device) # checked for PHYSDEVOP_setup_gsi (target IRQ) # checked for PHYSDEVOP_pci_mmcfg_reserved (target xen_t) +# checked for PHYSDEVOP_msi_control (target PCI device) setup }