From patchwork Tue May 12 19:44:32 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Tosatti X-Patchwork-Id: 23326 Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by demeter.kernel.org (8.14.2/8.14.2) with ESMTP id n4CJjOZN018454 for ; Tue, 12 May 2009 19:45:24 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752078AbZELTpU (ORCPT ); Tue, 12 May 2009 15:45:20 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751991AbZELTpU (ORCPT ); Tue, 12 May 2009 15:45:20 -0400 Received: from mx2.redhat.com ([66.187.237.31]:42863 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751973AbZELTpT (ORCPT ); Tue, 12 May 2009 15:45:19 -0400 Received: from int-mx2.corp.redhat.com (int-mx2.corp.redhat.com [172.16.27.26]) by mx2.redhat.com (8.13.8/8.13.8) with ESMTP id n4CJjJaT013156; Tue, 12 May 2009 15:45:19 -0400 Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id n4CJjIEO001667; Tue, 12 May 2009 15:45:18 -0400 Received: from amt.cnet (vpn-10-108.str.redhat.com [10.32.10.108]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id n4CJjG3g001525; Tue, 12 May 2009 15:45:17 -0400 Received: from amt.cnet (amt.cnet [127.0.0.1]) by amt.cnet (Postfix) with ESMTP id 304066800A8; Tue, 12 May 2009 16:44:35 -0300 (BRT) Received: (from marcelo@localhost) by amt.cnet (8.14.3/8.14.3/Submit) id n4CJiW0h020102; Tue, 12 May 2009 16:44:32 -0300 Date: Tue, 12 May 2009 16:44:32 -0300 From: Marcelo Tosatti To: "Yang, Sheng" Cc: Avi Kivity , Alex Williamson , kvm@vger.kernel.org Subject: Re: [PATCH 1/1] KVM: Fix potentially recursively get kvm lock Message-ID: <20090512194432.GA19969@amt.cnet> References: <200905121705.53176.sheng.yang@intel.com> <1242120729-2280-1-git-send-email-sheng@linux.intel.com> <20090512115524.GB10901@amt.cnet> <200905122213.36833.sheng.yang@intel.com> <20090512143021.GB12888@amt.cnet> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20090512143021.GB12888@amt.cnet> User-Agent: Mutt/1.5.19 (2009-01-05) X-Scanned-By: MIMEDefang 2.58 on 172.16.27.26 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Tue, May 12, 2009 at 11:30:21AM -0300, Marcelo Tosatti wrote: > On Tue, May 12, 2009 at 10:13:36PM +0800, Yang, Sheng wrote: > > > > + mutex_unlock(&kvm->lock); > > > > > > assigned_dev list is protected by kvm->lock. So you could have another > > > ioctl adding to it at the same time you're searching. > > > > Oh, yes... My fault... > > > > > Could either have a separate kvm->assigned_devs_lock, to protect > > > kvm->arch.assigned_dev_head (users are ioctls that manipulate it), or > > > change the IRQ injection to use a separate spinlock, kill the workqueue > > > and call kvm_set_irq from the assigned device interrupt handler. > > > > Peferred the latter, though needs more work. But the only reason for put a > > workqueue here is because kvm->lock is a mutex? I can't believe... If so, I > > think we had made a big mistake - we have to fix all kinds of racy problem > > caused by this, but finally find it's unnecessary... > > One issue is that kvm_set_irq can take too long while interrupts are > blocked, and you'd have to disable interrupts in other contexes that > inject interrupts (say qemu->ioctl(SET_INTERRUPT)->...->), so all i can > see is a tradeoff. > > > > But the interrupt injection path seems to be pretty short and efficient > to happen in host interrupt context. > > > > Avi, Gleb? > > > Maybe another reason is kvm_kick_vcpu(), but have already fixed by you. > > Note you tested the spinlock_irq patch with GigE and there was no > significant performance regression right? > > > > > Continue to check the code... OK, it might take some time for bigger changes to happen. I've changed your patch to drop the lock only around cancel_work_sync. Can deadlock if someone else tries to mess with the assigned device at the same time, but the VM won't go away under it because of the vmfd reference. --- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 4d00942..ba067db 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -250,7 +250,15 @@ static void deassign_host_irq(struct kvm *kvm, disable_irq_nosync(assigned_dev-> host_msix_entries[i].vector); + /* + * FIXME: kvm_assigned_dev_interrupt_work_handler can deadlock + * with cancel_work_sync, since it requires kvm->lock for irq + * injection. This is a hack, the irq code must use + * a separate lock. + */ + mutex_unlock(&kvm->lock); cancel_work_sync(&assigned_dev->interrupt_work); + mutex_lock(&kvm->lock); for (i = 0; i < assigned_dev->entries_nr; i++) free_irq(assigned_dev->host_msix_entries[i].vector,