From patchwork Tue Aug 15 11:00:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 9901607 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B5D4B60230 for ; Tue, 15 Aug 2017 11:02:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A92E528817 for ; Tue, 15 Aug 2017 11:02:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9493428833; Tue, 15 Aug 2017 11:02:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3DD2128817 for ; Tue, 15 Aug 2017 11:02:37 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dhZaK-0008GY-Gg; Tue, 15 Aug 2017 11:00:32 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dhZaJ-0008GQ-06 for xen-devel@lists.xen.org; Tue, 15 Aug 2017 11:00:31 +0000 Received: from [85.158.143.35] by server-10.bemta-6.messagelabs.com id A8/80-18185-E44D2995; Tue, 15 Aug 2017 11:00:30 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrIIsWRWlGSWpSXmKPExsWyU9JRQtf3yqR IgwnntS2WfFzM4sDocXT3b6YAxijWzLyk/IoE1owzs1qYC1olK85t7GdvYNwv3MXIySEh4Cdx 4dFzJhCbRUBV4uLdHcxdjBwcbAL2EtO/VoCERQSqJNpWtbKA2MIC3hLnGm6zgdi8Ah4SB6fsZ QWxhQRSJSZ+X8sMEReUODnzCVg9s4CexI2pU9hARjILSEss/8cBEZaXaN46G6ycU8BTYnvLH7 AxogIqEidXrmGCGKko0T/vARvElekSW//+Yp7AyD8LyYZZSDbMQtgwC8mGBYwsqxjVi1OLylK LdC30kooy0zNKchMzc3QNDcz0clOLixPTU3MSk4r1kvNzNzECg5IBCHYwzr7sf4hRkoNJSZR3 0dlJkUJ8SfkplRmJxRnxRaU5qcWHGGU4OJQkeI9cAsoJFqWmp1akZeYA4wMmLcHBoyTCa34ZK M1bXJCYW5yZDpE6xagoJc67GaRPACSRUZoH1waLyUuMslLCvIxAhwjxFKQW5WaWoMq/YhTnYF QS5v0LMoUnM68EbvoroMVMQIuvtIMtLklESEk1MPa+3iz8+qIEawVTzK5eh2kX+r9ObKs2mr9 0hlbCLE29SQFuDl/+7Lo3md1oOUfZjI9VjDYuMXob+Z7xnGmYtlx68YaYxGQTJrdNFx5sZJVI 3v3hmPi9o0lcv9+Fp7/fKNX05MVM8/7ncvLePV8+Xj+wRfjS9Jg5ugtlL809mPOMqyN68pJzh TeVWIozEg21mIuKEwENiNVjxAIAAA== X-Env-Sender: prvs=393075e76=roger.pau@citrix.com X-Msg-Ref: server-15.tower-21.messagelabs.com!1502794829!77832719!1 X-Originating-IP: [185.25.65.24] X-SpamReason: No, hits=0.0 required=7.0 tests=received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 41458 invoked from network); 15 Aug 2017 11:00:29 -0000 Received: from smtp.eu.citrix.com (HELO SMTP.EU.CITRIX.COM) (185.25.65.24) by server-15.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 15 Aug 2017 11:00:29 -0000 X-IronPort-AV: E=Sophos;i="5.41,377,1498521600"; d="scan'208";a="50996045" Date: Tue, 15 Aug 2017 12:00:24 +0100 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Andreas Kinzler , Jan Beulich , "xen-devel@lists.xen.org" Message-ID: <20170815110024.7grrprg7fg6gg3z3@MacBook-Pro-de-Roger.local> References: <20170815095306.rpxanbs7kd5m2tne@MacBook-Pro-de-Roger.local> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20170815095306.rpxanbs7kd5m2tne@MacBook-Pro-de-Roger.local> User-Agent: NeoMutt/20170714 (1.8.3) X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) Subject: Re: [Xen-devel] Regression PCI passthrough from 4.5.5 to 4.6.0-rc1 X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP On Tue, Aug 15, 2017 at 10:55:10AM +0100, Roger Pau Monné wrote: > On Mon, Aug 14, 2017 at 02:08:56PM +0200, Andreas Kinzler wrote: > > On Mon, 14 Aug 2017 13:56:58 +0200, Roger Pau Monné > > wrote: > > > > > I defined XEN_PT_LOGGING_ENABLED in xen_pt.h as requested without the > > > > > "hack" patch. Log is attached. Does it help? > > > > It tells me that there's nothing unexpected on that side. As I think I > > > > had indicated before, we really need to see both sides (qemu and > > > > hypervisor), as part of the MSI-X handling lives in Xen. And for the > > > > hypervisor side it is unlikely that we'll be able to get away without > > > > a debugging patch. I am intending to make such available to you in > > > > case you can't do so yourself, but I can't currently predict when I'll > > > > get to it. > > > I think the problem is that pci_msi_conf_write_intercept is failing to > > > unmask the entries when MSI-X is enabled with entries already > > > configured, but this will require some debugging patch as Jan said. > > > Following the MSI-X code is quite complicated, this split brain > > > between Xen and QEMU makes it quite hard. I can try to come up with a > > > patch later. > > > > I can try some debug patches although my workload is very high at the > > moment. It would help me quite a bit if the debug patches were suitable for > > the stable 4.8 tree. > > Hello, > > Could you please try the patch below and paste the output you get on > the Xen console? > > Jan, AFAICT (but I have to admit it's not easy to follow the code at > all), the following series of events will cause the MSIX entries to > not be unmasked: > > 1. Guest configures the MSIX table entries and unmasks each of them. > 2. Guest enables MSIX. > > This will cause the entries to remain masked, because QEMU will only > register the PIRQs and bind them when the MSI-X enable bit is set, > instead of doing it for each write to the MSIX table. > > I guess one way to solve this would be to force QEMU to call > xen_pt_msix_update_one in pci_msix_write once the entry is unmasked, > even if MSIX is not enabled. I can prepare a patch for that. So here is the patch for QEMU, if you don't mind giving it a try. I'm not really sure this is correct, since it will force Xen to enable MSIX in order to configure the entries, while the guest will still have MSIX disabled, but might we worth giving it a try. ---8<--- diff --git a/hw/xen/xen_pt_msi.c b/hw/xen/xen_pt_msi.c index ff9a79f5d2..dfb8d64654 100644 --- a/hw/xen/xen_pt_msi.c +++ b/hw/xen/xen_pt_msi.c @@ -451,8 +451,12 @@ static void pci_msix_write(void *opaque, hwaddr addr, } entry->updated = true; - } else if (msix->enabled && entry->updated && - !(val & PCI_MSIX_ENTRY_CTRL_MASKBIT)) { + } else if (entry->updated && !(val & PCI_MSIX_ENTRY_CTRL_MASKBIT)) { + /* + * NB: always register the entries with Xen when the write to the MSIX + * table happens, or else Xen won't be able to correctly snoop the + * entry control register write, and will fails to unmask the vector. + */ const volatile uint32_t *vec_ctrl; /*