From patchwork Fri Nov 17 06:22:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chao Gao X-Patchwork-Id: 10062331 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 298A66023A for ; Fri, 17 Nov 2017 06:27:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C62C2A539 for ; Fri, 17 Nov 2017 06:27:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1148F2A990; Fri, 17 Nov 2017 06:27:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 92B7A2A539 for ; Fri, 17 Nov 2017 06:27:23 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFa5H-0000h9-64; Fri, 17 Nov 2017 06:25:03 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFa5F-0000cR-Pk for xen-devel@lists.xen.org; Fri, 17 Nov 2017 06:25:01 +0000 Received: from [85.158.143.35] by server-8.bemta-6.messagelabs.com id 77/4A-32587-DB08E0A5; Fri, 17 Nov 2017 06:25:01 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrPLMWRWlGSWpSXmKPExsVywNwkVndPA1+ UwdMuC4slHxezODB6HN39mymAMYo1My8pvyKBNeNo7zP2gh96Fb07XjA2MDYodzFycQgJTGeU 2L35MHMXIyeHhACvxJFlM1gh7ACJoz+mM4LYQgJVEi3b14LVsAkoS1z82ssGYosISEtc+3yZE WQQs8BzZokl6/6CNQgLeEus+vofbBCLgKrE31trmLoYOTh4BZwlWhdGQsxXkJjy8D3YTE6g8P 43B5kgdjlJHFw3jWkCI+8CRoZVjOrFqUVlqUW6lnpJRZnpGSW5iZk5uoYGZnq5qcXFiempOYl JxXrJ+bmbGIHBwAAEOxjvbgo4xCjJwaQkystjzhclxJeUn1KZkVicEV9UmpNafIhRhoNDSYLX qx4oJ1iUmp5akZaZAwxLmLQEB4+SCG8BSJq3uCAxtzgzHSJ1itGY49nM1w3MHNOutjYxC7Hk5 eelSonzFoOUCoCUZpTmwQ2CxcslRlkpYV5GoNOEeApSi3IzS1DlXzGKczAqCfNWgkzhycwrgd v3CugUJqBTbG5wg5xSkoiQkmpgrM1Ospvet1XhXtxEQ4XvjOffSDMf/qbiw+J1rup9HY/ZQ7u JZ05Wt894Kbd42r+bv03ZhSete/ZabV3wuyPC2kViqc+vHzyndjM+USl1l6ZYpUvgp8vGV94E S35XYsr0XBYxxb+xKNV15sEVuv/rFu60fVZ6Z7PqccOQGMbdO0OFeV8Gu0lfV2Ipzkg01GIuK k4EAOjIu8WSAgAA X-Env-Sender: chao.gao@intel.com X-Msg-Ref: server-12.tower-21.messagelabs.com!1510899867!82378134!11 X-Originating-IP: [192.55.52.93] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTkyLjU1LjUyLjkzID0+IDMyNDY2NQ==\n X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 57334 invoked from network); 17 Nov 2017 06:25:00 -0000 Received: from mga11.intel.com (HELO mga11.intel.com) (192.55.52.93) by server-12.tower-21.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 17 Nov 2017 06:25:00 -0000 Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Nov 2017 22:25:00 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.44,407,1505804400"; d="scan'208"; a="1245165082" Received: from skl-4s-chao.sh.intel.com ([10.239.48.9]) by fmsmga002.fm.intel.com with ESMTP; 16 Nov 2017 22:24:57 -0800 From: Chao Gao To: xen-devel@lists.xen.org Date: Fri, 17 Nov 2017 14:22:17 +0800 Message-Id: <1510899755-40237-11-git-send-email-chao.gao@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1510899755-40237-1-git-send-email-chao.gao@intel.com> References: <1510899755-40237-1-git-send-email-chao.gao@intel.com> Cc: Lan Tianyu , Kevin Tian , Stefano Stabellini , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Ian Jackson , Tim Deegan , Jan Beulich , Andrew Cooper , Chao Gao , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [Xen-devel] [PATCH v4 10/28] x86/vvtd: Enable Interrupt Remapping through GCMD X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Software writes this field to enable/disable interrupt reampping. This patch emulate IRES field of GCMD. Currently, Guest's whole IRT are mapped to Xen permanently for the latency of delivering interrupt. And the old mapping is undone if present when trying to set up a new one. Signed-off-by: Chao Gao Signed-off-by: Lan Tianyu --- v4: - map guest's interrupt reampping table to Xen permanently rather than mapping one specific page on demand. --- xen/drivers/passthrough/vtd/iommu.h | 3 +- xen/drivers/passthrough/vtd/vvtd.c | 98 +++++++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/vtd/iommu.h b/xen/drivers/passthrough/vtd/iommu.h index 8579843..9c59aeb 100644 --- a/xen/drivers/passthrough/vtd/iommu.h +++ b/xen/drivers/passthrough/vtd/iommu.h @@ -161,9 +161,10 @@ #define DMA_GSTS_AFLS (((u64)1) << 28) #define DMA_GSTS_WBFS (((u64)1) << 27) #define DMA_GSTS_QIES (((u64)1) <<26) +#define DMA_GSTS_IRES_SHIFT 25 +#define DMA_GSTS_IRES (((u64)1) << DMA_GSTS_IRES_SHIFT) #define DMA_GSTS_SIRTPS_SHIFT 24 #define DMA_GSTS_SIRTPS (((u64)1) << DMA_GSTS_SIRTPS_SHIFT) -#define DMA_GSTS_IRES (((u64)1) <<25) #define DMA_GSTS_CFIS (((u64)1) <<23) /* IRTA_REG */ diff --git a/xen/drivers/passthrough/vtd/vvtd.c b/xen/drivers/passthrough/vtd/vvtd.c index f0476fe..06e522a 100644 --- a/xen/drivers/passthrough/vtd/vvtd.c +++ b/xen/drivers/passthrough/vtd/vvtd.c @@ -24,6 +24,7 @@ #include #include #include +#include #include "iommu.h" @@ -37,6 +38,7 @@ struct hvm_hw_vvtd { bool eim_enabled; + bool intremap_enabled; /* Interrupt remapping table base gfn and the max of entries */ uint16_t irt_max_entry; @@ -52,6 +54,7 @@ struct vvtd { struct domain *domain; struct hvm_hw_vvtd hw; + void *irt_base; }; /* Setting viommu_verbose enables debugging messages of vIOMMU */ @@ -118,6 +121,77 @@ static void *domain_vvtd(const struct domain *d) return NULL; } +static void *map_guest_pages(struct domain *d, uint64_t gfn, uint32_t nr) +{ + mfn_t *mfn = xmalloc_array(mfn_t, nr); + void* ret; + int i; + + if ( !mfn ) + return NULL; + + for ( i = 0; i < nr; i++) + { + struct page_info *p = get_page_from_gfn(d, gfn + i, NULL, P2M_ALLOC); + + if ( !p || !get_page_type(p, PGT_writable_page) ) + { + if ( p ) + put_page(p); + goto undo; + } + + mfn[i] = _mfn(page_to_mfn(p)); + } + + ret = vmap(mfn, nr); + if ( ret == NULL ) + goto undo; + xfree(mfn); + + return ret; + + undo: + for ( ; --i >= 0; ) + put_page_and_type(mfn_to_page(mfn_x(mfn[i]))); + xfree(mfn); + gprintk(XENLOG_ERR, "Failed to map guest pages %lx nr %x\n", gfn, nr); + + return NULL; +} + +static void unmap_guest_pages(void *va, uint32_t nr) +{ + unsigned long *mfn = xmalloc_array(unsigned long, nr); + int i; + void *va_copy = va; + + if ( !mfn ) + { + printk("%s %d: No free memory\n", __FILE__, __LINE__); + return; + } + + for ( i = 0; i < nr; i++, va += PAGE_SIZE) + mfn[i] = domain_page_map_to_mfn(va); + + vunmap(va_copy); + + for ( i = 0; i < nr; i++) + put_page_and_type(mfn_to_page(mfn[i])); +} + +static void write_gcmd_ire(struct vvtd *vvtd, uint32_t val) +{ + bool set = val & DMA_GCMD_IRE; + + vvtd_info("%sable Interrupt Remapping\n", set ? "En" : "Dis"); + + vvtd->hw.intremap_enabled = set; + (set ? vvtd_set_bit : vvtd_clear_bit) + (vvtd, DMAR_GSTS_REG, DMA_GSTS_IRES_SHIFT); +} + static void write_gcmd_sirtp(struct vvtd *vvtd, uint32_t val) { uint64_t irta = vvtd_get_reg_quad(vvtd, DMAR_IRTA_REG); @@ -131,16 +205,29 @@ static void write_gcmd_sirtp(struct vvtd *vvtd, uint32_t val) * the 'Set Interrupt Remap Table Pointer' operation. */ vvtd_clear_bit(vvtd, DMAR_GSTS_REG, DMA_GSTS_SIRTPS_SHIFT); + if ( vvtd->hw.intremap_enabled ) + vvtd_info("Update Interrupt Remapping Table when active\n"); if ( gfn_x(vvtd->hw.irt) != PFN_DOWN(DMA_IRTA_ADDR(irta)) || vvtd->hw.irt_max_entry != DMA_IRTA_SIZE(irta) ) { + if ( vvtd->irt_base ) + { + unmap_guest_pages(vvtd->irt_base, + PFN_UP(vvtd->hw.irt_max_entry * + sizeof(struct iremap_entry))); + vvtd->irt_base = NULL; + } vvtd->hw.irt = _gfn(PFN_DOWN(DMA_IRTA_ADDR(irta))); vvtd->hw.irt_max_entry = DMA_IRTA_SIZE(irta); vvtd->hw.eim_enabled = !!(irta & IRTA_EIME); vvtd_info("Update IR info (addr=%lx eim=%d size=%d)\n", gfn_x(vvtd->hw.irt), vvtd->hw.eim_enabled, vvtd->hw.irt_max_entry); + + vvtd->irt_base = map_guest_pages(vvtd->domain, gfn_x(vvtd->hw.irt), + PFN_UP(vvtd->hw.irt_max_entry * + sizeof(struct iremap_entry))); } vvtd_set_bit(vvtd, DMAR_GSTS_REG, DMA_GSTS_SIRTPS_SHIFT); } @@ -162,6 +249,8 @@ static void vvtd_write_gcmd(struct vvtd *vvtd, uint32_t val) if ( changed & DMA_GCMD_SIRTP ) write_gcmd_sirtp(vvtd, val); + if ( changed & DMA_GCMD_IRE ) + write_gcmd_ire(vvtd, val); } static int vvtd_in_range(struct vcpu *v, unsigned long addr) @@ -278,7 +367,16 @@ static int vvtd_destroy(struct viommu *viommu) struct vvtd *vvtd = viommu->priv; if ( vvtd ) + { + if ( vvtd->irt_base ) + { + unmap_guest_pages(vvtd->irt_base, + PFN_UP(vvtd->hw.irt_max_entry * + sizeof(struct iremap_entry))); + vvtd->irt_base = NULL; + } xfree(vvtd); + } return 0; }