From patchwork Tue Nov 1 09:04:41 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Razvan Cojocaru X-Patchwork-Id: 9407081 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3FF7760234 for ; Tue, 1 Nov 2016 09:07:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A88528947 for ; Tue, 1 Nov 2016 09:07:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1EB1C28A08; Tue, 1 Nov 2016 09:07:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7F85228BCD for ; Tue, 1 Nov 2016 09:07:04 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c1Uzk-0002C9-4i; Tue, 01 Nov 2016 09:04:36 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c1Uzj-0002C1-CD for xen-devel@lists.xenproject.org; Tue, 01 Nov 2016 09:04:35 +0000 Received: from [85.158.137.68] by server-17.bemta-3.messagelabs.com id 7B/58-31715-2AA58185; Tue, 01 Nov 2016 09:04:34 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrFIsWRWlGSWpSXmKPExsUSfTxjoe7CKIk Ig53bLS2+b5nM5MDocfjDFZYAxijWzLyk/IoE1ow/D6cyFhwWrfjx/wJjA+MtwS5GTg4hATeJ 4/cOMncxcgHZaxgl5izuZIFw7jBKrNqykg2masvCs1CJhYwSvxYfYQFJiAjoSRw+8RXMZhYok 1izYS4jiM0mYCixemMLUDMHh7CAssTJbSogYV4BJ4n3nZfBylkEVCQWfzwBNl9UIFzi/ayFbB A1ghInZz6BGqku8WfeJWYIW15i+9s5YLaEQI5E1/8FTCDjJQSkJP63KoGcJiHQxyKxu62NBaJ GRuLRxJtsExiFZyEZOwvJ2FlIxi5gZF7FqF6cWlSWWqRrqpdUlJmeUZKbmJmja2hgrJebWlyc mJ6ak5hUrJecn7uJERjm9QwMjDsYL391OsQoycGkJMr7NUw8QogvKT+lMiOxOCO+qDQntfgQo wwHh5IEb0ukRISQYFFqempFWmYOMOJg0hIcPEoivG0gad7igsTc4sx0iNQpRkUpcV4bkIQASC KjNA+uDRbllxhlpYR5GRkYGIR4ClKLcjNLUOVfMYpzMCoJQ2znycwrgZv+CmgxE9DitEIRkMU liQgpqQZGIfUJPzLXXRbaaLD/yZzmNwoPXEu2v+dZUuB854Qat+B1J9/OZJ2Fagfm7P2zonty RLPg1zvPG6eFRkrb+T6qmdNbuO5tQc3RLdFd8pH/Uv0c/VaJPcrnaeH/xFf5qFlE/Uz/QWfTh QeL9Nxq3oTMrW7kfbZDUZFPqOPrE0/v9cpsuRMn1fxRYinOSDTUYi4qTgQAuIOnIO0CAAA= X-Env-Sender: rcojocaru@bitdefender.com X-Msg-Ref: server-14.tower-31.messagelabs.com!1477991073!68454304!1 X-Originating-IP: [91.199.104.161] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.0.13; banners=-,-,- X-VirusChecked: Checked Received: (qmail 3925 invoked from network); 1 Nov 2016 09:04:33 -0000 Received: from mx01.bbu.dsd.mx.bitdefender.com (HELO mx01.bbu.dsd.mx.bitdefender.com) (91.199.104.161) by server-14.tower-31.messagelabs.com with DHE-RSA-AES128-GCM-SHA256 encrypted SMTP; 1 Nov 2016 09:04:33 -0000 Received: (qmail 4655 invoked from network); 1 Nov 2016 11:04:32 +0200 Received: from unknown (HELO mx-sr.buh.bitdefender.com) (10.17.80.103) by mx01.bbu.dsd.mx.bitdefender.com with AES256-GCM-SHA384 encrypted SMTP; 1 Nov 2016 11:04:32 +0200 Received: from smtp03.buh.bitdefender.org (smtp.bitdefender.biz [10.17.80.77]) by mx-sr.buh.bitdefender.com (Postfix) with ESMTP id 4DE797FC3B for ; Tue, 1 Nov 2016 11:04:32 +0200 (EET) Received: (qmail 6248 invoked from network); 1 Nov 2016 11:04:32 +0200 Received: from rcojocaru.dsd.ro (HELO ?10.10.14.59?) (rcojocaru@bitdefender.com@10.10.14.59) by smtp03.buh.bitdefender.org with SMTP; 1 Nov 2016 11:04:32 +0200 To: xen-devel From: Razvan Cojocaru Message-ID: <56bd0939-976b-ef45-4182-60e65c2f3ddc@bitdefender.com> Date: Tue, 1 Nov 2016 11:04:41 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 X-BitDefender-Scanner: Clean, Agent: BitDefender qmail 3.1.6 on smtp03.buh.bitdefender.org, sigver: 7.67867 X-BitDefender-Spam: No (0) X-BitDefender-SpamStamp: Build: [Engines: 2.15.6.911, Dats: 434535, Stamp: 3], Multi: [Enabled, t: (0.000010, 0.003917)], BW: [Enabled, t: (0.000009,0.000001)], RBL DNSBL: [Disabled], APM: [Enabled, Score: 500, t: (0.004001), Flags: 85D2ED72; NN_FORGED_THUNDERBIRD_2; NN_LEGIT_SUMM_400_WORDS; NN_NO_LINK_NMD; NN_LEGIT_BITDEFENDER; NN_LEGIT_MAILING_LIST_TO], SGN: [Enabled, t: (0.007269)], URL: [Enabled, t: (0.000005)], RTDA: [Enabled, t: (0.012427), Hit: No, Details: v2.4.0; Id: 2m1ghah.1avjonim7.5n2k5], total: 0(775) X-BitDefender-CF-Stamp: none Cc: Andrew Cooper , Tamas K Lengyel , Jan Beulich Subject: [Xen-devel] xc_hvm_inject_trap() races X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Hello, We've stumbled across the following scenario: we're injecting a #PF to try to bring a swapped page back, but Xen already have a pending interrupt, and the two collide. I've logged what happens in hvm_do_resume() at the point of injection, and stumbled across this: (XEN) [ 252.878389] vector: 14, type: 3, error_code: 0, VM_ENTRY_INTR_INFO: 0x00000000800000e1 VM_ENTRY_INTR_INFO does have INTR_INFO_VALID_MASK set here. This obviously leads to all sorts of unpleasantness with the guest. Ideally, the injected #PF should take precedence, and the other interrupt should not be lost as well. Suggestions on how best to solve this are welcome and appreciated. This problem would also occur with reinjected breakpoint events (please see xen-access.c), which also use xc_hvm_inject_trap(). The easiest approach would be to simply ignore the injected trap while VM_ENTRY_INTR_INFO & INTR_INFO_VALID_MASK, like this: However, this does not even guarantee delayed delivery of the trap, since by the time the next hvm_do_resume() call happens where there's no other pending interrupt we could have called xc_hvm_inject_trap() with different values, and have overwritten v->arch.hvm_vcpu.inject_trap. More importantly, the context might be different then and the injection would fail (or mess things up) because of that. So we'd be better off simply discarding the trap in this case (as has been suggested by Andrew), but here again the xc_hvm_inject_trap() caller has no clue whether the call succeeded or failed. This is inefficient (and thus comes with a performance impact) since the guest would be put in a loop until injection finally succeeds, and finally makes client applications significantly more cumbersome to implement, because they would need to guess when the injection failed (since xc_hvm_inject_trap() won't tell us), and handle duplicates for these corner cases. Is there an elegant way to fix this that keeps both interrupts but makes the injected one higher priority? Thanks, Razvan diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 2d78e45..cfe04b4 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -532,8 +532,16 @@ void hvm_do_resume(struct vcpu *v) /* Inject pending hw/sw trap */ if ( v->arch.hvm_vcpu.inject_trap.vector != -1 ) { - hvm_inject_trap(&v->arch.hvm_vcpu.inject_trap); - v->arch.hvm_vcpu.inject_trap.vector = -1; + unsigned long ev; + + __vmread(VM_ENTRY_INTR_INFO, &ev); + + /* Check for already pending interrupts (races). */ + if ( !(ev & INTR_INFO_VALID_MASK) ) + { + hvm_inject_trap(&v->arch.hvm_vcpu.inject_trap); + v->arch.hvm_vcpu.inject_trap.vector = -1; + } } }