From patchwork Wed Aug 31 03:56:27 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Feng" X-Patchwork-Id: 9306309 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3C3AA607D2 for ; Wed, 31 Aug 2016 04:28:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2864828B87 for ; Wed, 31 Aug 2016 04:28:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D57428C14; Wed, 31 Aug 2016 04:28:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4796A28B9C for ; Wed, 31 Aug 2016 04:28:01 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bex5c-000179-N7; Wed, 31 Aug 2016 04:25:28 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bex5b-00016k-38 for xen-devel@lists.xen.org; Wed, 31 Aug 2016 04:25:27 +0000 Received: from [85.158.143.35] by server-3.bemta-6.messagelabs.com id 0D/54-05661-63C56C75; Wed, 31 Aug 2016 04:25:26 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRWlGSWpSXmKPExsXS1tYhomsWcyz c4PJqJoslHxezODB6HN39mymAMYo1My8pvyKBNWP/jkbWghWiFQ0Ns5gaGDcJdjFycggJVEqs mb2VFcSWEOCVOLJsBpTtK/Fo3idWiJp6iden9rKA2GwCihIHLx4Ci4sISEtc+3yZsYuRi4NZY AGjROPFo8xdjBwcwgKREt/7uUBqWARUJZ7evgtWzyvgIPHq8i82iPlyEht2/2cEsTkFHCV+HH /IBtIqBFSz457xBEbeBYwMqxjVi1OLylKLdC31kooy0zNKchMzc3QNDcz0clOLixPTU3MSk4r 1kvNzNzECA4EBCHYw3t0UcIhRkoNJSZQ3/O/RcCG+pPyUyozE4oz4otKc1OJDjDIcHEoSvHbR x8KFBItS01Mr0jJzgCEJk5bg4FES4ZUASfMWFyTmFmemQ6ROMSpKifO+iwJKCIAkMkrz4Npgc XCJUVZKmJcR6BAhnoLUotzMElT5V4ziHIxKwryHQKbwZOaVwE1/BbSYCWhxwZ3DIItLEhFSUg 2MaSvnf2yr8m3jnsKzUO1d3+fWP6XbNSK5s04e3Hj4VM+i6PWbft9+dUfb4Q6/268Yfj/93I2 t29MaQ5N2ygTni+qy3XJPWzG/YZLfe55g3u38KdOsl349m+evEvDc8KjFL8nI0zu+OSvllGi8 Ya97Fh6e/K/+gEZf9Joz3as/FP6aMnvGjZQ9SizFGYmGWsxFxYkAwo+YMH4CAAA= X-Env-Sender: feng.wu@intel.com X-Msg-Ref: server-10.tower-21.messagelabs.com!1472617522!30750590!3 X-Originating-IP: [134.134.136.20] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjAgPT4gMzU1MzU4\n X-StarScan-Received: X-StarScan-Version: 8.84; banners=-,-,- X-VirusChecked: Checked Received: (qmail 31671 invoked from network); 31 Aug 2016 04:25:25 -0000 Received: from mga02.intel.com (HELO mga02.intel.com) (134.134.136.20) by server-10.tower-21.messagelabs.com with DHE-RSA-CAMELLIA256-SHA encrypted SMTP; 31 Aug 2016 04:25:25 -0000 Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP; 30 Aug 2016 21:25:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.30,260,1470726000"; d="scan'208";a="2931014" Received: from feng-bdw-de-pi.bj.intel.com ([10.238.154.74]) by fmsmga005.fm.intel.com with ESMTP; 30 Aug 2016 21:25:23 -0700 From: Feng Wu To: xen-devel@lists.xen.org Date: Wed, 31 Aug 2016 11:56:27 +0800 Message-Id: <1472615791-8664-3-git-send-email-feng.wu@intel.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1472615791-8664-1-git-send-email-feng.wu@intel.com> References: <1472615791-8664-1-git-send-email-feng.wu@intel.com> Cc: kevin.tian@intel.com, Feng Wu , george.dunlap@eu.citrix.com, andrew.cooper3@citrix.com, dario.faggioli@citrix.com, jbeulich@suse.com Subject: [Xen-devel] [PATCH v3 2/6] VMX: Properly handle pi when all the assigned devices are removed X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch handles some concern cases when the last assigned device is removed from the domain. In this case we should carefully handle pi descriptor and the per-cpu blocking list, to make sure: - all the PI descriptor are in the right state when next time a devices is assigned to the domain again. - No remaining vcpus of the domain in the per-cpu blocking list. Basically, we pause the domain before zapping the PI hooks and removing the vCPU from the blocking list, then unpause it after that. Signed-off-by: Feng Wu --- xen/arch/x86/hvm/vmx/vmx.c | 34 +++++++++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index f5d2d3c..b869728 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -158,14 +158,12 @@ static void vmx_pi_switch_to(struct vcpu *v) pi_clear_sn(pi_desc); } -static void vmx_pi_do_resume(struct vcpu *v) +static void vmx_pi_remove_vcpu_from_blocking_list(struct vcpu *v) { unsigned long flags; spinlock_t *pi_blocking_list_lock; struct pi_desc *pi_desc = &v->arch.hvm_vmx.pi_desc; - ASSERT(!test_bit(_VPF_blocked, &v->pause_flags)); - /* * Set 'NV' field back to posted_intr_vector, so the * Posted-Interrupts can be delivered to the vCPU when @@ -198,6 +196,21 @@ static void vmx_pi_do_resume(struct vcpu *v) spin_unlock_irqrestore(pi_blocking_list_lock, flags); } +static void vmx_pi_do_resume(struct vcpu *v) +{ + ASSERT(!test_bit(_VPF_blocked, &v->pause_flags)); + + vmx_pi_remove_vcpu_from_blocking_list(v); +} + +static void vmx_pi_blocking_cleanup(struct vcpu *v) +{ + if ( !iommu_intpost ) + return; + + vmx_pi_remove_vcpu_from_blocking_list(v); +} + /* This function is called when pcidevs_lock is held */ void vmx_pi_hooks_assign(struct domain *d) { @@ -213,13 +226,28 @@ void vmx_pi_hooks_assign(struct domain *d) /* This function is called when pcidevs_lock is held */ void vmx_pi_hooks_deassign(struct domain *d) { + struct vcpu *v; + if ( !iommu_intpost || !has_hvm_container_domain(d) ) return; ASSERT(d->arch.hvm_domain.vmx.vcpu_block); + /* + * Pausing the domain can make sure the vCPU is not + * running and hence calling the hooks simultaneously + * when deassigning the PI hooks and removing the vCPU + * from the blocking list. + */ + domain_pause(d); + d->arch.hvm_domain.vmx.vcpu_block = NULL; d->arch.hvm_domain.vmx.pi_do_resume = NULL; + + for_each_vcpu ( d, v ) + vmx_pi_blocking_cleanup(v); + + domain_unpause(d); } static int vmx_domain_initialise(struct domain *d)