From patchwork Sat Sep 14 15:37:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 11145703 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9406414DB for ; Sat, 14 Sep 2019 15:41:02 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6438720644 for ; Sat, 14 Sep 2019 15:41:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Emu6SOM/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6438720644 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=invisiblethingslab.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i9A7k-0001M8-HT; Sat, 14 Sep 2019 15:38:08 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i9A7j-0001LU-1q for xen-devel@lists.xenproject.org; Sat, 14 Sep 2019 15:38:07 +0000 X-Inumbo-ID: 995f9702-d705-11e9-b299-bc764e2007e4 Received: from new1-smtp.messagingengine.com (unknown [66.111.4.221]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 995f9702-d705-11e9-b299-bc764e2007e4; Sat, 14 Sep 2019 15:37:46 +0000 (UTC) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailnew.nyi.internal (Postfix) with ESMTP id DDB98292F; Sat, 14 Sep 2019 11:37:45 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Sat, 14 Sep 2019 11:37:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=XziRV+SUFoq9HrmcsJFUw/GIdKBfs0bngnqPI9cD5 Tc=; b=Emu6SOM/XDfHsL/371zHPFA8tQE4WpPaqzPnb5JgqIAKshibPwzaUoQFa A8WsqjCGwiaTrv7PpcQeUjaem8p5k+OHjSSgDbIs4O8piimJVTPZnlJIX96S1FGP j3mufkcFOKqcTT5Mck7tuJ2Y4TELWg2qAa3VGVqyw6gDZwZT0mtRzmN68jV/c2OM tkjbq2oGRkNODCCkHlDevHIaqwguHZ3bctUckZehlNLYEsLQhFLVYO1FN+k1qwGu b5+Q+HXmfrZFoBW92B3nbT+rMr10YBwsIb7Ry9qnOhFvZhnx6uZPoOrWG1Jg/825 CRaNW2VotSV2ajIviiCjh+yCImJ8A== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrtdelgdelfecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghrvghk ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucfkphepledurdeihedrfeegrdef feenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslh gvthhhihhnghhslhgrsgdrtghomhenucevlhhushhtvghrufhiiigvpedt X-ME-Proxy: Received: from localhost.localdomain (ip5b412221.dynamic.kabel-deutschland.de [91.65.34.33]) by mail.messagingengine.com (Postfix) with ESMTPA id 2246ED6005F; Sat, 14 Sep 2019 11:37:44 -0400 (EDT) From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Date: Sat, 14 Sep 2019 17:37:20 +0200 Message-Id: <819398f808613a1109bc06440268b8746e7540d4.1568475323.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v6 5/6] xen/x86: add PHYSDEVOP_interrupt_control X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , =?utf-8?q?Marek_Marczykowski-G?= =?utf-8?q?=C3=B3recki?= , Tim Deegan , Julien Grall , Jan Beulich , Daniel De Graaf , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Allow device model running in stubdomain to enable/disable INTx/MSI(-X), bypassing pciback. While pciback is still used to access config space from within stubdomain, it refuse to write to PCI_MSI_FLAGS_ENABLE/PCI_MSIX_FLAGS_ENABLE/PCI_COMMAND_INTX_DISABLE in non-permissive mode. Which is the right thing to do for PV domain (the main use case for pciback), as PV domain should use XEN_PCI_OP_* commands for that. Unfortunately those commands are not good for stubdomain use, as they configure MSI in dom0's kernel too, which should not happen for HVM domain. This new physdevop is allowed only for stubdomain controlling the domain which own the device. Signed-off-by: Marek Marczykowski-Górecki --- Changes in v3: - new patch Changes in v4: - adjust code style - s/msi_msix/msi/ - add msi_set_enable XSM hook - flatten struct physdev_msi_set_enable - add to include/xlat.lst Changes in v5: - rename to PHYSDEVOP_msi_control - combine "mode" and "enable" into "flags" - refuse to enable both MSI and MSI-X, and also to enable MSI(-X) on incapable device - disable/enable INTx when enabling/disabling MSI (?) - refuse if !use_msi - adjust flask hook to make more sense (require "setup" access on device, not on domain) - rebase on master Changes in v6: - rename to PHYSDEVOP_interrupt_control - extend with INTx control - Ensure than MSI(-X) can't be enabled together with INTx and the other MSI(-X). - deduplicate code in msi_control - explicitly refuse to operate on hidden devices - expand flags to uint16_t to avoid implicit padding I'm not sure if XSM part is correct, compile-tested only, as I'm not sure how to set the policy. --- xen/arch/x86/msi.c | 45 +++++++++++++++++++++++++- xen/arch/x86/physdev.c | 53 ++++++++++++++++++++++++++++++- xen/arch/x86/x86_64/physdev.c | 4 ++- xen/include/asm-x86/msi.h | 2 +- xen/include/public/physdev.h | 23 +++++++++++++- xen/include/xlat.lst | 1 +- xen/include/xsm/dummy.h | 7 ++++- xen/include/xsm/xsm.h | 6 +++- xen/xsm/dummy.c | 1 +- xen/xsm/flask/hooks.c | 24 ++++++++++++++- xen/xsm/flask/policy/access_vectors | 1 +- 11 files changed, 167 insertions(+) diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c index d630600..ecea91a 100644 --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -1443,6 +1443,51 @@ int pci_restore_msi_state(struct pci_dev *pdev) return 0; } +int msi_control(struct pci_dev *pdev, bool msix, bool enable) +{ + unsigned int cap = msix ? PCI_CAP_ID_MSIX : PCI_CAP_ID_MSI; + unsigned int other_cap = msix ? PCI_CAP_ID_MSI : PCI_CAP_ID_MSIX; + uint16_t cmd; + + if ( !use_msi ) + return -EOPNOTSUPP; + + if ( !pci_find_cap_offset(pdev->seg, + pdev->bus, + PCI_SLOT(pdev->devfn), + PCI_FUNC(pdev->devfn), + cap) ) + return -ENODEV; + + cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND); + + /* don't allow enabling MSI(-X) and INTx at the same time */ + if ( enable && ! (cmd & PCI_COMMAND_INTX_DISABLE) ) + return -EBUSY; + + /* don't allow enabling both MSI and MSI-X at the same time */ + if ( enable && find_msi_entry(pdev, -1, other_cap) ) + return -EBUSY; + + if ( msix ) + msix_set_enable(pdev, enable); + else + msi_set_enable(pdev, enable); + + return 0; +} + +int intx_control(struct pci_dev *pdev, bool enable) +{ + /* don't allow enabling INTx if MSI(-X) is already enabled */ + if ( enable && find_msi_entry(pdev, -1, PCI_CAP_ID_MSI) ) + return -EBUSY; + if ( enable && find_msi_entry(pdev, -1, PCI_CAP_ID_MSIX) ) + return -EBUSY; + pci_intx(pdev, enable); + return 0; +} + void __init early_msi_init(void) { if ( use_msi < 0 ) diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c index 3a3c158..7b71039 100644 --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -662,6 +662,59 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) break; } + case PHYSDEVOP_interrupt_control: { + struct physdev_interrupt_control op; + struct pci_dev *pdev; + int intr_type; + bool enable; + + ret = -EFAULT; + if ( copy_from_guest(&op, arg, 1) ) + break; + + ret = -EINVAL; + if ( op.flags & ~(PHYSDEVOP_INTERRUPT_CONTROL_TYPE_MASK | + PHYSDEVOP_INTERRUPT_CONTROL_ENABLE) ) + break; + + intr_type = op.flags & PHYSDEVOP_INTERRUPT_CONTROL_TYPE_MASK; + enable = op.flags & PHYSDEVOP_INTERRUPT_CONTROL_ENABLE; + + pcidevs_lock(); + pdev = pci_get_pdev(op.seg, op.bus, op.devfn); + ret = -ENODEV; + /* explicitly exclude hidden devices */ + if ( !pdev || pdev->domain == dom_xen ) + goto pci_unlock; + + ret = xsm_interrupt_control(XSM_DM_PRIV, + pdev->domain, + pdev->sbdf.sbdf, + intr_type, + enable); + if ( ret ) + goto pci_unlock; + + switch ( intr_type ) + { + case PHYSDEVOP_INTERRUPT_CONTROL_INTX: + ret = intx_control(pdev, enable); + break; + case PHYSDEVOP_INTERRUPT_CONTROL_MSI: + ret = msi_control(pdev, false, enable); + break; + case PHYSDEVOP_INTERRUPT_CONTROL_MSIX: + ret = msi_control(pdev, true, enable); + break; + default: + ret = -EINVAL; + break; + } +pci_unlock: + pcidevs_unlock(); + break; + } + default: ret = -ENOSYS; break; diff --git a/xen/arch/x86/x86_64/physdev.c b/xen/arch/x86/x86_64/physdev.c index c5a00ea..6e0e488 100644 --- a/xen/arch/x86/x86_64/physdev.c +++ b/xen/arch/x86/x86_64/physdev.c @@ -76,6 +76,10 @@ CHECK_physdev_pci_device_add CHECK_physdev_pci_device #undef xen_physdev_pci_device +#define xen_physdev_interrupt_control physdev_interrupt_control +CHECK_physdev_interrupt_control +#undef xen_physdev_interrupt_control + #define COMPAT #undef guest_handle_okay #define guest_handle_okay compat_handle_okay diff --git a/xen/include/asm-x86/msi.h b/xen/include/asm-x86/msi.h index 10387dc..4c13e6b 100644 --- a/xen/include/asm-x86/msi.h +++ b/xen/include/asm-x86/msi.h @@ -252,5 +252,7 @@ void guest_mask_msi_irq(struct irq_desc *, bool mask); void ack_nonmaskable_msi_irq(struct irq_desc *); void end_nonmaskable_msi_irq(struct irq_desc *, u8 vector); void set_msi_affinity(struct irq_desc *, const cpumask_t *); +int msi_control(struct pci_dev *pdev, bool msix, bool enable); +int intx_control(struct pci_dev *pdev, bool enable); #endif /* __ASM_MSI_H */ diff --git a/xen/include/public/physdev.h b/xen/include/public/physdev.h index b6faf83..689c11e 100644 --- a/xen/include/public/physdev.h +++ b/xen/include/public/physdev.h @@ -345,6 +345,29 @@ typedef struct physdev_dbgp_op physdev_dbgp_op_t; DEFINE_XEN_GUEST_HANDLE(physdev_dbgp_op_t); /* + * Choose which interrupt type to control. If neither MSI nor MSI-X is chosen, + * will apply to INTx - for convenience define PHYSDEVOP_INTERRUPT_CONTROL_INTX + * and PHYSDEVOP_INTERRUPT_CONTROL_TYPE_MASK + */ +#define PHYSDEVOP_INTERRUPT_CONTROL_TYPE_MASK 3 +#define PHYSDEVOP_INTERRUPT_CONTROL_INTX 0 +#define PHYSDEVOP_INTERRUPT_CONTROL_MSI 1 +#define PHYSDEVOP_INTERRUPT_CONTROL_MSIX 2 +/* when PHYSDEVOP_INTERRUPT_CONTROL_ENABLE not set, disable */ +#define PHYSDEVOP_INTERRUPT_CONTROL_ENABLE 4 + +#define PHYSDEVOP_interrupt_control 32 +struct physdev_interrupt_control { + /* IN */ + uint16_t seg; + uint8_t bus; + uint8_t devfn; + uint16_t flags; +}; +typedef struct physdev_interrupt_control physdev_interrupt_control_t; +DEFINE_XEN_GUEST_HANDLE(physdev_interrupt_control_t); + +/* * Notify that some PIRQ-bound event channels have been unmasked. * ** This command is obsolete since interface version 0x00030202 and is ** * ** unsupported by newer versions of Xen. ** diff --git a/xen/include/xlat.lst b/xen/include/xlat.lst index 95f5e55..18af663 100644 --- a/xen/include/xlat.lst +++ b/xen/include/xlat.lst @@ -104,6 +104,7 @@ ! vnuma_topology_info memory.h ? physdev_eoi physdev.h ? physdev_get_free_pirq physdev.h +? physdev_interrupt_control physdev.h ? physdev_irq physdev.h ? physdev_irq_status_query physdev.h ? physdev_manage_pci physdev.h diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h index ef52bb1..5a758c5 100644 --- a/xen/include/xsm/dummy.h +++ b/xen/include/xsm/dummy.h @@ -514,6 +514,13 @@ static XSM_INLINE int xsm_pci_config_permission(XSM_DEFAULT_ARG struct domain *d return xsm_default_action(action, current->domain, d); } +static XSM_INLINE int xsm_interrupt_control(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf, + uint8_t intr_type, uint8_t enable) +{ + XSM_ASSERT_ACTION(XSM_DM_PRIV); + return xsm_default_action(action, current->domain, d); +} + static XSM_INLINE int xsm_add_to_physmap(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2) { XSM_ASSERT_ACTION(XSM_TARGET); diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h index e22d616..f080189 100644 --- a/xen/include/xsm/xsm.h +++ b/xen/include/xsm/xsm.h @@ -106,6 +106,7 @@ struct xsm_operations { int (*iomem_permission) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow); int (*iomem_mapping) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow); int (*pci_config_permission) (struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end, uint8_t access); + int (*interrupt_control) (struct domain *d, uint32_t machine_bdf, uint8_t intr_type, uint8_t enable); #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI) int (*get_device_group) (uint32_t machine_bdf); @@ -464,6 +465,11 @@ static inline int xsm_pci_config_permission (xsm_default_t def, struct domain *d return xsm_ops->pci_config_permission(d, machine_bdf, start, end, access); } +static inline int xsm_interrupt_control (xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint8_t msix, uint8_t enable) +{ + return xsm_ops->interrupt_control(d, machine_bdf, msix, enable); +} + #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI) static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf) { diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c index 5705e52..3080ae7 100644 --- a/xen/xsm/dummy.c +++ b/xen/xsm/dummy.c @@ -81,6 +81,7 @@ void __init xsm_fixup_ops (struct xsm_operations *ops) set_to_dummy_if_null(ops, iomem_permission); set_to_dummy_if_null(ops, iomem_mapping); set_to_dummy_if_null(ops, pci_config_permission); + set_to_dummy_if_null(ops, interrupt_control); set_to_dummy_if_null(ops, get_vnumainfo); #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI) diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index 791c1f6..ee2fc52 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -1083,6 +1083,29 @@ static int flask_pci_config_permission(struct domain *d, uint32_t machine_bdf, u } +static int flask_interrupt_control(struct domain *d, uint32_t machine_bdf, uint8_t type, uint8_t enable) +{ + uint32_t dsid, rsid; + int rc = -EPERM; + struct avc_audit_data ad; + uint32_t perm; + + AVC_AUDIT_DATA_INIT(&ad, DEV); + ad.device = machine_bdf; + + rc = security_device_sid(machine_bdf, &rsid); + if ( rc ) + return rc; + + rc = avc_current_has_perm(rsid, SECCLASS_RESOURCE, RESOURCE__SETUP, &ad); + if ( rc ) + return rc; + + perm = flask_iommu_resource_use_perm(); + dsid = domain_sid(d); + return avc_has_perm(dsid, rsid, SECCLASS_RESOURCE, perm, &ad); +} + static int flask_resource_plug_core(void) { return avc_current_has_perm(SECINITSID_DOMXEN, SECCLASS_RESOURCE, RESOURCE__PLUG, NULL); @@ -1800,6 +1823,7 @@ static struct xsm_operations flask_ops = { .iomem_permission = flask_iomem_permission, .iomem_mapping = flask_iomem_mapping, .pci_config_permission = flask_pci_config_permission, + .interrupt_control = flask_interrupt_control, .resource_plug_core = flask_resource_plug_core, .resource_unplug_core = flask_resource_unplug_core, diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors index 194d743..82eaeac 100644 --- a/xen/xsm/flask/policy/access_vectors +++ b/xen/xsm/flask/policy/access_vectors @@ -466,6 +466,7 @@ class resource # checked for PHYSDEVOP_restore_msi* (target PCI device) # checked for PHYSDEVOP_setup_gsi (target IRQ) # checked for PHYSDEVOP_pci_mmcfg_reserved (target xen_t) +# checked for PHYSDEVOP_interrupt_control (target PCI device) setup }