From patchwork Mon Sep 28 10:56:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803415 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD6751580 for ; Mon, 28 Sep 2020 10:57:41 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7EDEE2074F for ; Mon, 28 Sep 2020 10:57:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="CBAgCk51" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7EDEE2074F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqpP-0004DF-83; Mon, 28 Sep 2020 10:56:19 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqpO-0004D9-C7 for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 10:56:18 +0000 X-Inumbo-ID: 809d7d92-3d87-4b15-b2fd-40239f5670d4 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 809d7d92-3d87-4b15-b2fd-40239f5670d4; Mon, 28 Sep 2020 10:56:17 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290577; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SzaITT0sR3eKgr9vS2zQdHUk+Tx53cRBKfsTil/SkmQ=; b=CBAgCk51gqJC4Lfk5Vurzo8R4RD8XfWV6Kbt8GMm1/aUv4ihrpKPqYjok2HK3cYppB1VHw sqOJQBPZMvaPlhRx+eQpwXZ6ynX04iW6itpTnlmPvLT+tECb3RW6LELgutCldqKilExT80 X+XX8l1ojLkhnSK3/rjPGdfI7sG+SE0= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id ED72BAD82; Mon, 28 Sep 2020 10:56:16 +0000 (UTC) Subject: [PATCH 01/12] evtchn: refuse EVTCHNOP_status for Xen-bound event channels From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: Date: Mon, 28 Sep 2020 12:56:15 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Callers have no business knowing the state of the Xen end of an event channel. Signed-off-by: Jan Beulich Acked-by: Julien Grall Reviewed-by: Paul Durrant --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -933,6 +933,11 @@ int evtchn_status(evtchn_status_t *statu } chn = evtchn_from_port(d, port); + if ( consumer_is_xen(chn) ) + { + rc = -EACCES; + goto out; + } rc = xsm_evtchn_status(XSM_TARGET, d, chn); if ( rc ) From patchwork Mon Sep 28 10:56:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803413 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17534618 for ; Mon, 28 Sep 2020 10:57:14 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D3AB02074F for ; Mon, 28 Sep 2020 10:57:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="TX8K3Tpz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D3AB02074F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqpt-0004Gb-HQ; Mon, 28 Sep 2020 10:56:49 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqps-0004GP-5j for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 10:56:48 +0000 X-Inumbo-ID: ab44c1a4-0de3-4344-a2d2-3ea9938d2d36 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id ab44c1a4-0de3-4344-a2d2-3ea9938d2d36; Mon, 28 Sep 2020 10:56:46 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290606; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iIibg70Z0Q3OYMfIABU4YuhHLrFzABuQ6LAinJpQr0A=; b=TX8K3Tpzn/+GVO1e9eEBy6N5AzwXkssDeYfUMZS9tELnLkKWaDCjebetJeBz0QeeuxS584 w9BKIxJaTSKwVLQuW+pbHhGyh/F9CxNBKfDmFWiG0ErJj7yrLlv3jEYZnekcNchANxLO+Y xIzETASNPc1shtBDmMWvFJjE6SE6PKc= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id F13DEAD82; Mon, 28 Sep 2020 10:56:45 +0000 (UTC) Subject: [PATCH 02/12] evtchn: avoid race in get_xen_consumer() From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: Date: Mon, 28 Sep 2020 12:56:45 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" There's no global lock around the updating of this global piece of data. Make use of cmpxchg() to avoid two entities racing with their updates. Signed-off-by: Jan Beulich --- TBD: Initially I used cmpxchgptr() here, until realizing Arm doesn't have it. It's slightly more type-safe than cmpxchg() (requiring all arguments to actually be pointers), so I now wonder whether Arm should gain it (perhaps simply by moving the x86 implementation to xen/lib.h), or whether we should purge it from x86 as being pointless. --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -57,7 +57,8 @@ * with a pointer, we stash them dynamically in a small lookup array which * can be indexed by a small integer. */ -static xen_event_channel_notification_t xen_consumers[NR_XEN_CONSUMERS]; +static xen_event_channel_notification_t __read_mostly + xen_consumers[NR_XEN_CONSUMERS]; /* Default notification action: wake up from wait_on_xen_event_channel(). */ static void default_xen_notification_fn(struct vcpu *v, unsigned int port) @@ -81,7 +82,7 @@ static uint8_t get_xen_consumer(xen_even for ( i = 0; i < ARRAY_SIZE(xen_consumers); i++ ) { if ( xen_consumers[i] == NULL ) - xen_consumers[i] = fn; + (void)cmpxchg(&xen_consumers[i], NULL, fn); if ( xen_consumers[i] == fn ) break; } From patchwork Mon Sep 28 10:57:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803417 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AFD21746 for ; Mon, 28 Sep 2020 10:57:43 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4EDB42074F for ; Mon, 28 Sep 2020 10:57:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="bnti2IOt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4EDB42074F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqqF-0004Le-WA; Mon, 28 Sep 2020 10:57:11 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqqE-0004LL-Nc for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 10:57:10 +0000 X-Inumbo-ID: cafb11e5-e6f6-47d6-9e94-9fe138bc1c27 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id cafb11e5-e6f6-47d6-9e94-9fe138bc1c27; Mon, 28 Sep 2020 10:57:09 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290629; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oeCJCQsWUnJ0xjjxNtZRmFI9KK964X3ka2Xyj60DtEQ=; b=bnti2IOtyM2W4WfYf/Prhu45p6nr9u8vrTkjwV+0qMWV+UF24V2gFejm1p5coOhMYc2GlY oZfJ0YtwbDGj2471BPS7ETTlBtfblEp+RNmAsrouBQaOb4aGgSlY5v+LYyLvjsH67xAtrz jIaf+nJEqYkOP9vJJ8hfmx3wZZNIePM= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 229ADACAC; Mon, 28 Sep 2020 10:57:09 +0000 (UTC) Subject: [PATCH 03/12] evtchn: don't call Xen consumer callback with per-channel lock held From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: <1bf3959d-c097-f8ef-cce4-3a325d0984c4@suse.com> Date: Mon, 28 Sep 2020 12:57:08 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" While there don't look to be any problems with this right now, the lock order implications from holding the lock can be very difficult to follow (and may be easy to violate unknowingly). The present callbacks don't (and no such callback should) have any need for the lock to be held. Signed-off-by: Jan Beulich --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -746,9 +746,18 @@ int evtchn_send(struct domain *ld, unsig rport = lchn->u.interdomain.remote_port; rchn = evtchn_from_port(rd, rport); if ( consumer_is_xen(rchn) ) - xen_notification_fn(rchn)(rd->vcpu[rchn->notify_vcpu_id], rport); - else - evtchn_port_set_pending(rd, rchn->notify_vcpu_id, rchn); + { + /* Don't keep holding the lock for the call below. */ + xen_event_channel_notification_t fn = xen_notification_fn(rchn); + struct vcpu *rv = rd->vcpu[rchn->notify_vcpu_id]; + + rcu_lock_domain(rd); + spin_unlock_irqrestore(&lchn->lock, flags); + fn(rv, rport); + rcu_unlock_domain(rd); + return 0; + } + evtchn_port_set_pending(rd, rchn->notify_vcpu_id, rchn); break; case ECS_IPI: evtchn_port_set_pending(ld, lchn->notify_vcpu_id, lchn); From patchwork Mon Sep 28 10:57:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803419 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 91C7D1580 for ; Mon, 28 Sep 2020 10:58:25 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5EF492074F for ; Mon, 28 Sep 2020 10:58:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="RVVKkHlZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5EF492074F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqqg-0004Qt-90; Mon, 28 Sep 2020 10:57:38 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqqe-0004Qa-MK for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 10:57:36 +0000 X-Inumbo-ID: 466685f1-d6eb-4bf2-96dd-3d519d94c68a Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 466685f1-d6eb-4bf2-96dd-3d519d94c68a; Mon, 28 Sep 2020 10:57:35 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290654; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J/GpFR4UxC1zaNaCEb95T79spF3BCjvGuD9y1xyT+uQ=; b=RVVKkHlZS/xEOnOAPgxTCv0lxzuXd/cOPGTEQ2poXmT0bsBTY17chax2fe9swxBudLMq3u FB/Xuri9zDqF1uLv5u/xPqTQjtp7iVASdG7HeJhgJOqtq9vlyAyQFOFaUxLRqIaUcYUSNe xVS61sxQ7RkmvpaEE2QvG9jpGl3ZSKI= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 5A0C6ACAC; Mon, 28 Sep 2020 10:57:34 +0000 (UTC) Subject: [PATCH 04/12] evtchn: evtchn_set_priority() needs to acquire the per-channel lock From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: <5b1700a8-7900-9450-1c21-323bcde1fccc@suse.com> Date: Mon, 28 Sep 2020 12:57:33 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" evtchn_fifo_set_pending() (invoked with the per-channel lock held) has two uses of the channel's priority field. The field gets updated by evtchn_fifo_set_priority() with only the per-domain event_lock held, i.e. the two reads may observe two different values. While the 2nd use could - afaict - in principle be replaced by q->priority, I think evtchn_set_priority() should acquire the per-channel lock in any event. Signed-off-by: Jan Beulich --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -1132,7 +1132,9 @@ static long evtchn_set_priority(const st { struct domain *d = current->domain; unsigned int port = set_priority->port; + struct evtchn *chn; long ret; + unsigned long flags; spin_lock(&d->event_lock); @@ -1142,8 +1144,10 @@ static long evtchn_set_priority(const st return -EINVAL; } - ret = evtchn_port_set_priority(d, evtchn_from_port(d, port), - set_priority->priority); + chn = evtchn_from_port(d, port); + spin_lock_irqsave(&chn->lock, flags); + ret = evtchn_port_set_priority(d, chn, set_priority->priority); + spin_unlock_irqrestore(&chn->lock, flags); spin_unlock(&d->event_lock); From patchwork Mon Sep 28 10:58:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803421 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5649D1580 for ; Mon, 28 Sep 2020 10:59:43 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 251FD2074F for ; Mon, 28 Sep 2020 10:59:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="gfxh9ZIO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 251FD2074F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqrs-0004aq-L6; Mon, 28 Sep 2020 10:58:52 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqrs-0004ak-0L for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 10:58:52 +0000 X-Inumbo-ID: 97f7edf5-d8a0-4cae-8d7a-53e25c6143e9 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 97f7edf5-d8a0-4cae-8d7a-53e25c6143e9; Mon, 28 Sep 2020 10:58:51 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290730; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fXan/rIQLrp6e3jD5KfkNxF5GmzyEUYe+HppsckrcEs=; b=gfxh9ZIOQVCsGQ8RpS0yY7MifqXvz5qhcRNXcNlqwKTY6iU07pjr2zEGxw39lKgTGh1xDZ bGnina5DUQNwP2+aKjsaRWKyM01UNpVfO7jA6OyAOY0dSHA8Y2BR2h6GzSx7NclhwTbUVz YqY5D98wSVTR+xCbGX7zCnflNBpXFNU= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 666F1AE07; Mon, 28 Sep 2020 10:58:50 +0000 (UTC) Subject: [PATCH 05/12] evtchn/sched: reject poll requests for unusable ports From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini , Dario Faggioli References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: <802a0866-6bcf-cb52-1c3f-edb628fbc9c7@suse.com> Date: Mon, 28 Sep 2020 12:58:49 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Before and after XSA-342 there has been an asymmetry in how not really usable ports get treated in do_poll(): Ones beyond a certain boundary (max_evtchns originally, valid_evtchns subsequently) did get refused with -EINVAL, while lower ones were accepted despite there potentially being no way to wake the vCPU again from its polling state. Arrange to also honor evtchn_usable() output in the decision. Requested-by: Andrew Cooper Signed-off-by: Jan Beulich Reviewed-by: Julien Grall Reviewed-by: Dario Faggioli Reviewed-by: Paul Durrant --- a/xen/common/sched/core.c +++ b/xen/common/sched/core.c @@ -1427,13 +1427,13 @@ static long do_poll(struct sched_poll *s if ( __copy_from_guest_offset(&port, sched_poll->ports, i, 1) ) goto out; - rc = -EINVAL; - if ( !port_is_valid(d, port) ) - goto out; - - rc = 0; - if ( evtchn_port_is_pending(d, port) ) + rc = evtchn_port_poll(d, port); + if ( rc ) + { + if ( rc > 0 ) + rc = 0; goto out; + } } if ( sched_poll->nr_ports == 1 ) --- a/xen/include/xen/event.h +++ b/xen/include/xen/event.h @@ -240,19 +240,6 @@ static inline bool evtchn_is_pending(con return evtchn_usable(evtchn) && d->evtchn_port_ops->is_pending(d, evtchn); } -static inline bool evtchn_port_is_pending(struct domain *d, evtchn_port_t port) -{ - struct evtchn *evtchn = evtchn_from_port(d, port); - bool rc; - unsigned long flags; - - spin_lock_irqsave(&evtchn->lock, flags); - rc = evtchn_is_pending(d, evtchn); - spin_unlock_irqrestore(&evtchn->lock, flags); - - return rc; -} - static inline bool evtchn_is_masked(const struct domain *d, const struct evtchn *evtchn) { @@ -279,6 +266,24 @@ static inline bool evtchn_is_busy(const d->evtchn_port_ops->is_busy(d, evtchn); } +static inline int evtchn_port_poll(struct domain *d, evtchn_port_t port) +{ + int rc = -EINVAL; + + if ( port_is_valid(d, port) ) + { + struct evtchn *evtchn = evtchn_from_port(d, port); + unsigned long flags; + + spin_lock_irqsave(&evtchn->lock, flags); + if ( evtchn_usable(evtchn) ) + rc = evtchn_is_pending(d, evtchn); + spin_unlock_irqrestore(&evtchn->lock, flags); + } + + return rc; +} + static inline int evtchn_port_set_priority(struct domain *d, struct evtchn *evtchn, unsigned int priority) From patchwork Mon Sep 28 10:59:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803425 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EEA82618 for ; Mon, 28 Sep 2020 11:00:34 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B444221775 for ; Mon, 28 Sep 2020 11:00:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="ji1VVHJq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B444221775 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqsD-0004dP-U2; Mon, 28 Sep 2020 10:59:13 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqsD-0004dH-64 for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 10:59:13 +0000 X-Inumbo-ID: 3e66c6d3-274f-4454-a73d-b962ecfa2abc Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 3e66c6d3-274f-4454-a73d-b962ecfa2abc; Mon, 28 Sep 2020 10:59:12 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290751; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZHDyMtgX+8dhvc5uYQZOZEiupXAKqbAo9RNgQWlQ91k=; b=ji1VVHJqQsIf2fqUMHdMRoX1moyqYV7lyuG7ffEh9HGB2Rn9yRDo3ziDDBdHTnCnN/jXXU LTsTXPsQYshX2m0Po8wSrCLEWrqfZ/mk83uv3Gt3X5FJi8YU1Giq2lAKSV/Zg3xn83/QbQ Vu8mT5zrmtYGmkxCUyuAHUcTZPai3Vw= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id A9CD1AD82; Mon, 28 Sep 2020 10:59:11 +0000 (UTC) Subject: [PATCH 06/12] evtchn: don't bypass unlinking pIRQ when closing port From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: <6add36f5-93de-dc8e-7c14-dc5ae1c794eb@suse.com> Date: Mon, 28 Sep 2020 12:59:11 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" There's no other path causing a terminal unlink_pirq_port() to be called (evtchn_bind_vcpu() relinks it right away) and hence _if_ pirq can indeed be NULL when closing the port, list corruption would occur when bypassing the unlink (unless the structure never gets linked again). As we can't come here after evtchn_destroy() anymore, (late) domain destruction also isn't a reason for a possible exception, and hence the only alternative looks to be that the check was pointless in the first place. While I haven't observed the case, from code inspection I'm far from sure I can exclude this being possible, so it feels more safe to re-arrange the code instead. Fixes: c24536b636f2 ("replace d->nr_pirqs sized arrays with radix tree") Signed-off-by: Jan Beulich Reviewed-by: Paul Durrant --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -615,17 +615,18 @@ int evtchn_close(struct domain *d1, int case ECS_PIRQ: { struct pirq *pirq = pirq_info(d1, chn1->u.pirq.irq); - if ( !pirq ) - break; - if ( !is_hvm_domain(d1) ) - pirq_guest_unbind(d1, pirq); - pirq->evtchn = 0; - pirq_cleanup_check(pirq, d1); - unlink_pirq_port(chn1, d1->vcpu[chn1->notify_vcpu_id]); + if ( pirq ) + { + if ( !is_hvm_domain(d1) ) + pirq_guest_unbind(d1, pirq); + pirq->evtchn = 0; + pirq_cleanup_check(pirq, d1); #ifdef CONFIG_X86 - if ( is_hvm_domain(d1) && domain_pirq_to_irq(d1, pirq->pirq) > 0 ) - unmap_domain_pirq_emuirq(d1, pirq->pirq); + if ( is_hvm_domain(d1) && domain_pirq_to_irq(d1, pirq->pirq) > 0 ) + unmap_domain_pirq_emuirq(d1, pirq->pirq); #endif + } + unlink_pirq_port(chn1, d1->vcpu[chn1->notify_vcpu_id]); break; } From patchwork Mon Sep 28 11:00:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803427 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BBAFD618 for ; Mon, 28 Sep 2020 11:00:40 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8F7D52080A for ; Mon, 28 Sep 2020 11:00:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="hXL/yJOo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F7D52080A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqt3-0005A5-7i; Mon, 28 Sep 2020 11:00:05 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqt2-00051E-7k for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 11:00:04 +0000 X-Inumbo-ID: 16044122-9a3f-4be3-aa42-7528e02c1c0d Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 16044122-9a3f-4be3-aa42-7528e02c1c0d; Mon, 28 Sep 2020 11:00:03 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290802; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A+5Stl0rmqhXhbs7D3G+1NwS1Mh9/voMzMez3S4+mu8=; b=hXL/yJOotVbgPrTfvoOP41zdArxO4ahxUVJiXDAFvF6dSbVcUSlLFapdUBGSTIAXxHnEYl Qn9ThH3rsV0VdHgXMuVma4oTsBAMrJ+ObPSPMsMIAwG5wPadZbQ0ItQjZVr63bBdoabIjZ H7Xms+O9Uk2++29LErYL9G7Z+KrHCeo= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 4D813B012; Mon, 28 Sep 2020 11:00:02 +0000 (UTC) Subject: [PATCH 07/12] evtchn: cut short evtchn_reset()'s loop in the common case From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: <0577c62d-b349-6a60-d8bc-5b23a74342e0@suse.com> Date: Mon, 28 Sep 2020 13:00:01 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The general expectation is that there are only a few open ports left when a domain asks its event channel configuration to be reset. Similarly on average half a bucket worth of event channels can be expected to be inactive. Try to avoid iterating over all channels, by utilizing usage data we're maintaining anyway. Signed-off-by: Jan Beulich Reviewed-by: Paul Durrant Acked-by: Julien Grall --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -232,7 +232,11 @@ void evtchn_free(struct domain *d, struc evtchn_port_clear_pending(d, chn); if ( consumer_is_xen(chn) ) + { write_atomic(&d->xen_evtchns, d->xen_evtchns - 1); + /* Decrement ->xen_evtchns /before/ ->active_evtchns. */ + smp_wmb(); + } write_atomic(&d->active_evtchns, d->active_evtchns - 1); /* Reset binding to vcpu0 when the channel is freed. */ @@ -1073,6 +1077,19 @@ int evtchn_unmask(unsigned int port) return 0; } +static bool has_active_evtchns(const struct domain *d) +{ + unsigned int xen = read_atomic(&d->xen_evtchns); + + /* + * Read ->xen_evtchns /before/ active_evtchns, to prevent + * evtchn_reset() exiting its loop early. + */ + smp_rmb(); + + return read_atomic(&d->active_evtchns) > xen; +} + int evtchn_reset(struct domain *d, bool resuming) { unsigned int i; @@ -1097,7 +1114,7 @@ int evtchn_reset(struct domain *d, bool if ( !i ) return -EBUSY; - for ( ; port_is_valid(d, i); i++ ) + for ( ; port_is_valid(d, i) && has_active_evtchns(d); i++ ) { evtchn_close(d, i, 1); @@ -1340,6 +1357,10 @@ int alloc_unbound_xen_event_channel( spin_unlock_irqrestore(&chn->lock, flags); + /* + * Increment ->xen_evtchns /after/ ->active_evtchns. No explicit + * barrier needed due to spin-locked region just above. + */ write_atomic(&ld->xen_evtchns, ld->xen_evtchns + 1); out: From patchwork Mon Sep 28 11:00:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803429 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC09C6CB for ; Mon, 28 Sep 2020 11:00:53 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9909A21775 for ; Mon, 28 Sep 2020 11:00:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="ShrjWvlM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9909A21775 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqtK-0005Uv-Gl; Mon, 28 Sep 2020 11:00:22 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqtI-0005Uc-SY for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 11:00:20 +0000 X-Inumbo-ID: edc434d1-1fe6-418c-a800-fea8123e2a1b Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id edc434d1-1fe6-418c-a800-fea8123e2a1b; Mon, 28 Sep 2020 11:00:20 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290819; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mAH7mcn+I8c4u1B7EFxsTLHZFEBEuLLXXQ/mROQ/yrA=; b=ShrjWvlMN6Ip19eLagVUXToTY/0OK51n5x0M+DI+7lzSGDrdYZXG/2a/IqxJj/jJnUMlS8 jakO6ftepvMmUaTYfGv0jta5401tAzNy5Zk4FmgPMwG3W/4ErtrV8UwPk9MhQNZGgrhUq0 AokR/cIRmiISyfddiPlGSXukxwS1f0A= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 6891AAE07; Mon, 28 Sep 2020 11:00:19 +0000 (UTC) Subject: [PATCH 08/12] evtchn: ECS_CLOSED => ECS_FREE From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: Date: Mon, 28 Sep 2020 13:00:18 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" There's no ECS_CLOSED; correct a comment naming it. Signed-off-by: Jan Beulich Acked-by: Julien Grall Reviewed-by: Paul Durrant --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -673,7 +673,7 @@ int evtchn_close(struct domain *d1, int * We can only get here if the port was closed and re-bound after * unlocking d1 but before locking d2 above. We could retry but * it is easier to return the same error as if we had seen the - * port in ECS_CLOSED. It must have passed through that state for + * port in ECS_FREE. It must have passed through that state for * us to end up here, so it's a valid error to return. */ rc = -EINVAL; From patchwork Mon Sep 28 11:00:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803431 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8688B6CB for ; Mon, 28 Sep 2020 11:01:45 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4F69C2080A for ; Mon, 28 Sep 2020 11:01:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="JzrFbwMi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4F69C2080A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqti-0005aY-Qi; Mon, 28 Sep 2020 11:00:46 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqth-0005Zc-IQ for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 11:00:45 +0000 X-Inumbo-ID: 83155394-b879-44aa-8d8b-10939de9ade9 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 83155394-b879-44aa-8d8b-10939de9ade9; Mon, 28 Sep 2020 11:00:44 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290843; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3olIW+XDYq9LhGrju2pwubOlX/CoG8pdd5sHxB6Izf8=; b=JzrFbwMirq7i4Y+9Ww9eJz+8aieMIvflIsJqg5T8QZSCQrpHPPde+/Bop3AdyjSBdEPSgP BNQTHcA/MA9wBSbq6aQbblJ2XuBFAt1uV68MUp9uN8ul3aXhsYMiD+y7HpQZ2YRFtmbE6q OZFDLM7W0f6QPXocv94cunGttDMnC18= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id B1A77AE07; Mon, 28 Sep 2020 11:00:43 +0000 (UTC) Subject: [PATCH 09/12] evtchn: move FIFO-private struct declarations From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: Date: Mon, 28 Sep 2020 13:00:43 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" There's no need to expose them. Signed-off-by: Jan Beulich Acked-by: Julien Grall --- I wonder whether we shouldn't do away with event_fifo.h altogether. --- a/xen/common/event_fifo.c +++ b/xen/common/event_fifo.c @@ -21,6 +21,27 @@ #include +struct evtchn_fifo_queue { + uint32_t *head; /* points into control block */ + uint32_t tail; + uint8_t priority; + spinlock_t lock; +}; + +struct evtchn_fifo_vcpu { + struct evtchn_fifo_control_block *control_block; + struct evtchn_fifo_queue queue[EVTCHN_FIFO_MAX_QUEUES]; +}; + +#define EVTCHN_FIFO_EVENT_WORDS_PER_PAGE (PAGE_SIZE / sizeof(event_word_t)) +#define EVTCHN_FIFO_MAX_EVENT_ARRAY_PAGES \ + (EVTCHN_FIFO_NR_CHANNELS / EVTCHN_FIFO_EVENT_WORDS_PER_PAGE) + +struct evtchn_fifo_domain { + event_word_t *event_array[EVTCHN_FIFO_MAX_EVENT_ARRAY_PAGES]; + unsigned int num_evtchns; +}; + static inline event_word_t *evtchn_fifo_word_from_port(const struct domain *d, unsigned int port) { --- a/xen/include/xen/event_fifo.h +++ b/xen/include/xen/event_fifo.h @@ -9,27 +9,6 @@ #ifndef __XEN_EVENT_FIFO_H__ #define __XEN_EVENT_FIFO_H__ -struct evtchn_fifo_queue { - uint32_t *head; /* points into control block */ - uint32_t tail; - uint8_t priority; - spinlock_t lock; -}; - -struct evtchn_fifo_vcpu { - struct evtchn_fifo_control_block *control_block; - struct evtchn_fifo_queue queue[EVTCHN_FIFO_MAX_QUEUES]; -}; - -#define EVTCHN_FIFO_EVENT_WORDS_PER_PAGE (PAGE_SIZE / sizeof(event_word_t)) -#define EVTCHN_FIFO_MAX_EVENT_ARRAY_PAGES \ - (EVTCHN_FIFO_NR_CHANNELS / EVTCHN_FIFO_EVENT_WORDS_PER_PAGE) - -struct evtchn_fifo_domain { - event_word_t *event_array[EVTCHN_FIFO_MAX_EVENT_ARRAY_PAGES]; - unsigned int num_evtchns; -}; - int evtchn_fifo_init_control(struct evtchn_init_control *init_control); int evtchn_fifo_expand_array(const struct evtchn_expand_array *expand_array); void evtchn_fifo_destroy(struct domain *domain); From patchwork Mon Sep 28 11:01:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803433 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CCDF26CB for ; Mon, 28 Sep 2020 11:02:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9F3AF2080A for ; Mon, 28 Sep 2020 11:02:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="WaXq3i1g" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F3AF2080A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMquf-0005lV-94; Mon, 28 Sep 2020 11:01:45 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMque-0005lK-Cn for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 11:01:44 +0000 X-Inumbo-ID: 726871cb-ce63-440d-aab5-dea35f89a6c1 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 726871cb-ce63-440d-aab5-dea35f89a6c1; Mon, 28 Sep 2020 11:01:43 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290903; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CenzWT0+htNohYcB8mL2GQ4JbyBefC7VjC6YZI9XB+U=; b=WaXq3i1g9HUvqVRp+FzE6BIfJ8w1eBx6dEdhiAop3tsEgkutz4My2ymTxxh3nocMyrsVN9 zwqAzhtoAUayXWtrNOKxS7rpB7VgULlOmKbs+HAAjPGnn/EvGHFzHzyN+njIFEZNAGPKzY 9RdVbdCD10azjIygnBBIblVvtD4E0pw= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id E9DD5AD82; Mon, 28 Sep 2020 11:01:42 +0000 (UTC) Subject: [PATCH 10/12] evtchn/fifo: use stable fields when recording "last queue" information From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: Date: Mon, 28 Sep 2020 13:01:42 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Both evtchn->priority and evtchn->notify_vcpu_id could, prior to recent locking adjustments, change behind the back of evtchn_fifo_set_pending(). Neither the queue's priority nor the vCPU's vcpu_id fields have similar properties, so they seem better suited for the purpose. In particular they reflect the respective evtchn fields' values at the time they were used to determine queue and vCPU. Signed-off-by: Jan Beulich Reviewed-by: Julien Grall Reviewed-by: Paul Durrant --- a/xen/common/event_fifo.c +++ b/xen/common/event_fifo.c @@ -246,8 +246,8 @@ static void evtchn_fifo_set_pending(stru /* Moved to a different queue? */ if ( old_q != q ) { - evtchn->last_vcpu_id = evtchn->notify_vcpu_id; - evtchn->last_priority = evtchn->priority; + evtchn->last_vcpu_id = v->vcpu_id; + evtchn->last_priority = q->priority; spin_unlock_irqrestore(&old_q->lock, flags); spin_lock_irqsave(&q->lock, flags); From patchwork Mon Sep 28 11:02:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803435 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60C7B618 for ; Mon, 28 Sep 2020 11:02:49 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 29D7A2080A for ; Mon, 28 Sep 2020 11:02:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="a2b8fPSu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 29D7A2080A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqux-0005oz-I1; Mon, 28 Sep 2020 11:02:03 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMquw-0005oq-Os for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 11:02:02 +0000 X-Inumbo-ID: b7436e7f-fa02-4ee2-aefa-6a33848f9953 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id b7436e7f-fa02-4ee2-aefa-6a33848f9953; Mon, 28 Sep 2020 11:02:01 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9eNVqXwgS1BNCxawrz6W6Me07pTESeUHMUnTZhjOhss=; b=a2b8fPSun+tjg695SoOs9cuQix3wc49GlgI1DNezAa1JNCyyOZKGs/3hGVZu7TLcX7fI3w ZIue0jIIX2RiT3A0UF2rTG9q0SSlKgMTcI3GTl4gBDr2g0L2I3D+8xuFQt3fv52ZuIrGz4 GGmPAn7AA4qAv54txKdNsjKvv3nb9sY= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 1489EAE07; Mon, 28 Sep 2020 11:02:01 +0000 (UTC) Subject: [PATCH 11/12] evtchn: convert vIRQ lock to an r/w one From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: <6e529147-2a76-bc28-ac16-21fc9a2c8f03@suse.com> Date: Mon, 28 Sep 2020 13:02:00 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" There's no need to serialize all sending of vIRQ-s; all that's needed is serialization against the closing of the respective event channels (by means of a barrier). To facilitate the conversion, introduce a new rw_barrier(). Signed-off-by: Jan Beulich --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -160,7 +160,7 @@ struct vcpu *vcpu_create(struct domain * v->vcpu_id = vcpu_id; v->dirty_cpu = VCPU_CPU_CLEAN; - spin_lock_init(&v->virq_lock); + rwlock_init(&v->virq_lock); tasklet_init(&v->continue_hypercall_tasklet, NULL, NULL); --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -640,7 +640,7 @@ int evtchn_close(struct domain *d1, int if ( v->virq_to_evtchn[chn1->u.virq] != port1 ) continue; v->virq_to_evtchn[chn1->u.virq] = 0; - spin_barrier(&v->virq_lock); + rw_barrier(&v->virq_lock); } break; @@ -794,7 +794,7 @@ void send_guest_vcpu_virq(struct vcpu *v ASSERT(!virq_is_global(virq)); - spin_lock_irqsave(&v->virq_lock, flags); + read_lock_irqsave(&v->virq_lock, flags); port = v->virq_to_evtchn[virq]; if ( unlikely(port == 0) ) @@ -807,7 +807,7 @@ void send_guest_vcpu_virq(struct vcpu *v spin_unlock(&chn->lock); out: - spin_unlock_irqrestore(&v->virq_lock, flags); + read_unlock_irqrestore(&v->virq_lock, flags); } void send_guest_global_virq(struct domain *d, uint32_t virq) @@ -826,7 +826,7 @@ void send_guest_global_virq(struct domai if ( unlikely(v == NULL) ) return; - spin_lock_irqsave(&v->virq_lock, flags); + read_lock_irqsave(&v->virq_lock, flags); port = v->virq_to_evtchn[virq]; if ( unlikely(port == 0) ) @@ -838,7 +838,7 @@ void send_guest_global_virq(struct domai spin_unlock(&chn->lock); out: - spin_unlock_irqrestore(&v->virq_lock, flags); + read_unlock_irqrestore(&v->virq_lock, flags); } void send_guest_pirq(struct domain *d, const struct pirq *pirq) --- a/xen/common/spinlock.c +++ b/xen/common/spinlock.c @@ -2,7 +2,7 @@ #include #include #include -#include +#include #include #include #include @@ -334,6 +334,12 @@ void _spin_unlock_recursive(spinlock_t * } } +void _rw_barrier(rwlock_t *lock) +{ + check_barrier(&lock->lock.debug); + do { smp_mb(); } while ( _rw_is_locked(lock) ); +} + #ifdef CONFIG_DEBUG_LOCK_PROFILE struct lock_profile_anc { --- a/xen/include/xen/rwlock.h +++ b/xen/include/xen/rwlock.h @@ -237,6 +237,8 @@ static inline int _rw_is_write_locked(rw return (atomic_read(&lock->cnts) & _QW_WMASK) == _QW_LOCKED; } +void _rw_barrier(rwlock_t *lock); + #define read_lock(l) _read_lock(l) #define read_lock_irq(l) _read_lock_irq(l) #define read_lock_irqsave(l, f) \ @@ -266,6 +268,7 @@ static inline int _rw_is_write_locked(rw #define rw_is_locked(l) _rw_is_locked(l) #define rw_is_write_locked(l) _rw_is_write_locked(l) +#define rw_barrier(l) _rw_barrier(l) typedef struct percpu_rwlock percpu_rwlock_t; --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -235,7 +235,7 @@ struct vcpu /* IRQ-safe virq_lock protects against delivering VIRQ to stale evtchn. */ evtchn_port_t virq_to_evtchn[NR_VIRQS]; - spinlock_t virq_lock; + rwlock_t virq_lock; /* Tasklet for continue_hypercall_on_cpu(). */ struct tasklet continue_hypercall_tasklet; From patchwork Mon Sep 28 11:02:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11803437 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E7E16CB for ; Mon, 28 Sep 2020 11:04:22 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 158042080A for ; Mon, 28 Sep 2020 11:04:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="TQdirruI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 158042080A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqvg-0005vs-T3; Mon, 28 Sep 2020 11:02:48 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kMqvf-0005ve-Ci for xen-devel@lists.xenproject.org; Mon, 28 Sep 2020 11:02:47 +0000 X-Inumbo-ID: 420ec5e0-d2bc-466a-85e6-08a73cdb0abf Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 420ec5e0-d2bc-466a-85e6-08a73cdb0abf; Mon, 28 Sep 2020 11:02:45 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601290964; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4dfFB0Sz8C52D151IxsP7GS6hA2PzfNXUTCOXctxZ/g=; b=TQdirruIPI1JxkqR3bwGFTokQrsIqS8ls5bahS8w5ki+BNddexZ3S/c0IUrzNDyZnXvpt9 yPlGjIaQ8lFuqHctMFoKkAKil9qH1cfhAQWUStHirkep/EILOvf4QKlEZDPzCcEJWXTYwW 6IZqfZSP5yH85patqxTCK3yTPKcH3vE= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 65A06AE07; Mon, 28 Sep 2020 11:02:44 +0000 (UTC) Subject: [PATCH 12/12] evtchn: convert domain event lock to an r/w one From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Wei Liu , Stefano Stabellini , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Message-ID: <5fee2432-7b94-2f91-5f17-c9eb3ec9f126@suse.com> Date: Mon, 28 Sep 2020 13:02:43 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <0d5ffc89-4b04-3e06-e950-f0cb171c7419@suse.com> Content-Language: en-US X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Especially for the use in evtchn_move_pirqs() (called when moving a vCPU across pCPU-s) and the ones in EOI handling in PCI pass-through code, serializing perhaps an entire domain isn't helpful when no state (which isn't e.g. further protected by the per-channel lock) changes. Unfortunately this implies dropping of lock profiling for this lock, until r/w locks may get enabled for such functionality. While ->notify_vcpu_id is now meant to be consistently updated with the per-channel lock held for writing, an extension applies to ECS_PIRQ: The field is also guaranteed to not change with the per-domain event lock held. Therefore the unlink_pirq_port() call from evtchn_bind_vcpu() as well as the link_pirq_port() one from evtchn_bind_pirq() could in principle be moved out of the per-channel locked regions, but this further code churn didn't seem worth it. Signed-off-by: Jan Beulich --- RFC: * In evtchn_bind_vcpu() the question is whether limiting the use of write_lock() to just the ECS_PIRQ case is really worth it. * In flask_get_peer_sid() the question is whether we wouldn't better switch to using the per-channel lock. --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -909,7 +909,7 @@ int arch_domain_soft_reset(struct domain if ( !is_hvm_domain(d) ) return -EINVAL; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); for ( i = 0; i < d->nr_pirqs ; i++ ) { if ( domain_pirq_to_emuirq(d, i) != IRQ_UNBOUND ) @@ -919,7 +919,7 @@ int arch_domain_soft_reset(struct domain break; } } - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); if ( ret ) return ret; --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -528,9 +528,9 @@ void hvm_migrate_pirqs(struct vcpu *v) if ( !is_iommu_enabled(d) || !hvm_domain_irq(d)->dpci ) return; - spin_lock(&d->event_lock); + read_lock(&d->event_lock); pt_pirq_iterate(d, migrate_pirq, v); - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); } static bool hvm_get_pending_event(struct vcpu *v, struct x86_event *info) --- a/xen/arch/x86/hvm/irq.c +++ b/xen/arch/x86/hvm/irq.c @@ -404,9 +404,9 @@ int hvm_inject_msi(struct domain *d, uin { int rc; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); rc = map_domain_emuirq_pirq(d, pirq, IRQ_MSI_EMU); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); if ( rc ) return rc; info = pirq_info(d, pirq); --- a/xen/arch/x86/hvm/vioapic.c +++ b/xen/arch/x86/hvm/vioapic.c @@ -203,9 +203,9 @@ static int vioapic_hwdom_map_gsi(unsigne { gprintk(XENLOG_WARNING, "vioapic: error binding GSI %u: %d\n", gsi, ret); - spin_lock(&currd->event_lock); + write_lock(&currd->event_lock); unmap_domain_pirq(currd, pirq); - spin_unlock(&currd->event_lock); + write_unlock(&currd->event_lock); } pcidevs_unlock(); --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -465,7 +465,7 @@ int msixtbl_pt_register(struct domain *d int r = -EINVAL; ASSERT(pcidevs_locked()); - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); if ( !msixtbl_initialised(d) ) return -ENODEV; @@ -535,7 +535,7 @@ void msixtbl_pt_unregister(struct domain struct msixtbl_entry *entry; ASSERT(pcidevs_locked()); - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); if ( !msixtbl_initialised(d) ) return; @@ -589,13 +589,13 @@ void msixtbl_pt_cleanup(struct domain *d if ( !msixtbl_initialised(d) ) return; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); list_for_each_entry_safe( entry, temp, &d->arch.hvm.msixtbl_list, list ) del_msixtbl_entry(entry); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); } void msix_write_completion(struct vcpu *v) @@ -719,9 +719,9 @@ int vpci_msi_arch_update(struct vpci_msi msi->arch.pirq, msi->mask); if ( rc ) { - spin_lock(&pdev->domain->event_lock); + write_lock(&pdev->domain->event_lock); unmap_domain_pirq(pdev->domain, msi->arch.pirq); - spin_unlock(&pdev->domain->event_lock); + write_unlock(&pdev->domain->event_lock); pcidevs_unlock(); msi->arch.pirq = INVALID_PIRQ; return rc; @@ -760,9 +760,9 @@ static int vpci_msi_enable(const struct rc = vpci_msi_update(pdev, data, address, vectors, pirq, mask); if ( rc ) { - spin_lock(&pdev->domain->event_lock); + write_lock(&pdev->domain->event_lock); unmap_domain_pirq(pdev->domain, pirq); - spin_unlock(&pdev->domain->event_lock); + write_unlock(&pdev->domain->event_lock); pcidevs_unlock(); return rc; } @@ -807,9 +807,9 @@ static void vpci_msi_disable(const struc ASSERT(!rc); } - spin_lock(&pdev->domain->event_lock); + write_lock(&pdev->domain->event_lock); unmap_domain_pirq(pdev->domain, pirq); - spin_unlock(&pdev->domain->event_lock); + write_unlock(&pdev->domain->event_lock); pcidevs_unlock(); } --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -2413,10 +2413,10 @@ int ioapic_guest_write(unsigned long phy } if ( pirq >= 0 ) { - spin_lock(&hardware_domain->event_lock); + write_lock(&hardware_domain->event_lock); ret = map_domain_pirq(hardware_domain, pirq, irq, MAP_PIRQ_TYPE_GSI, NULL); - spin_unlock(&hardware_domain->event_lock); + write_unlock(&hardware_domain->event_lock); if ( ret < 0 ) return ret; } --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -1536,7 +1536,7 @@ int pirq_guest_bind(struct vcpu *v, stru irq_guest_action_t *action, *newaction = NULL; int rc = 0; - WARN_ON(!spin_is_locked(&v->domain->event_lock)); + WARN_ON(!rw_is_write_locked(&v->domain->event_lock)); BUG_ON(!local_irq_is_enabled()); retry: @@ -1756,7 +1756,7 @@ void pirq_guest_unbind(struct domain *d, struct irq_desc *desc; int irq = 0; - WARN_ON(!spin_is_locked(&d->event_lock)); + WARN_ON(!rw_is_write_locked(&d->event_lock)); BUG_ON(!local_irq_is_enabled()); desc = pirq_spin_lock_irq_desc(pirq, NULL); @@ -1793,7 +1793,7 @@ static bool pirq_guest_force_unbind(stru unsigned int i; bool bound = false; - WARN_ON(!spin_is_locked(&d->event_lock)); + WARN_ON(!rw_is_write_locked(&d->event_lock)); BUG_ON(!local_irq_is_enabled()); desc = pirq_spin_lock_irq_desc(pirq, NULL); @@ -2037,7 +2037,7 @@ int get_free_pirq(struct domain *d, int { int i; - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); if ( type == MAP_PIRQ_TYPE_GSI ) { @@ -2062,7 +2062,7 @@ int get_free_pirqs(struct domain *d, uns { unsigned int i, found = 0; - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); for ( i = d->nr_pirqs - 1; i >= nr_irqs_gsi; --i ) if ( is_free_pirq(d, pirq_info(d, i)) ) @@ -2090,7 +2090,7 @@ int map_domain_pirq( DECLARE_BITMAP(prepared, MAX_MSI_IRQS) = {}; DECLARE_BITMAP(granted, MAX_MSI_IRQS) = {}; - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); if ( !irq_access_permitted(current->domain, irq)) return -EPERM; @@ -2309,7 +2309,7 @@ int unmap_domain_pirq(struct domain *d, return -EINVAL; ASSERT(pcidevs_locked()); - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); info = pirq_info(d, pirq); if ( !info || (irq = info->arch.irq) <= 0 ) @@ -2436,13 +2436,13 @@ void free_domain_pirqs(struct domain *d) int i; pcidevs_lock(); - spin_lock(&d->event_lock); + write_lock(&d->event_lock); for ( i = 0; i < d->nr_pirqs; i++ ) if ( domain_pirq_to_irq(d, i) > 0 ) unmap_domain_pirq(d, i); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); pcidevs_unlock(); } @@ -2685,7 +2685,7 @@ int map_domain_emuirq_pirq(struct domain int old_emuirq = IRQ_UNBOUND, old_pirq = IRQ_UNBOUND; struct pirq *info; - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); if ( !is_hvm_domain(d) ) return -EINVAL; @@ -2751,7 +2751,7 @@ int unmap_domain_pirq_emuirq(struct doma if ( (pirq < 0) || (pirq >= d->nr_pirqs) ) return -EINVAL; - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); emuirq = domain_pirq_to_emuirq(d, pirq); if ( emuirq == IRQ_UNBOUND ) @@ -2799,7 +2799,7 @@ static int allocate_pirq(struct domain * { int current_pirq; - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); current_pirq = domain_irq_to_pirq(d, irq); if ( pirq < 0 ) { @@ -2871,7 +2871,7 @@ int allocate_and_map_gsi_pirq(struct dom } /* Verify or get pirq. */ - spin_lock(&d->event_lock); + write_lock(&d->event_lock); pirq = allocate_pirq(d, index, *pirq_p, irq, MAP_PIRQ_TYPE_GSI, NULL); if ( pirq < 0 ) { @@ -2884,7 +2884,7 @@ int allocate_and_map_gsi_pirq(struct dom *pirq_p = pirq; done: - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return ret; } @@ -2925,7 +2925,7 @@ int allocate_and_map_msi_pirq(struct dom pcidevs_lock(); /* Verify or get pirq. */ - spin_lock(&d->event_lock); + write_lock(&d->event_lock); pirq = allocate_pirq(d, index, *pirq_p, irq, type, &msi->entry_nr); if ( pirq < 0 ) { @@ -2938,7 +2938,7 @@ int allocate_and_map_msi_pirq(struct dom *pirq_p = pirq; done: - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); pcidevs_unlock(); if ( ret ) { --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -34,7 +34,7 @@ static int physdev_hvm_map_pirq( ASSERT(!is_hardware_domain(d)); - spin_lock(&d->event_lock); + write_lock(&d->event_lock); switch ( type ) { case MAP_PIRQ_TYPE_GSI: { @@ -84,7 +84,7 @@ static int physdev_hvm_map_pirq( break; } - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return ret; } @@ -154,18 +154,18 @@ int physdev_unmap_pirq(domid_t domid, in if ( is_hvm_domain(d) && has_pirq(d) ) { - spin_lock(&d->event_lock); + write_lock(&d->event_lock); if ( domain_pirq_to_emuirq(d, pirq) != IRQ_UNBOUND ) ret = unmap_domain_pirq_emuirq(d, pirq); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); if ( domid == DOMID_SELF || ret ) goto free_domain; } pcidevs_lock(); - spin_lock(&d->event_lock); + write_lock(&d->event_lock); ret = unmap_domain_pirq(d, pirq); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); pcidevs_unlock(); free_domain: @@ -192,10 +192,10 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_H ret = -EINVAL; if ( eoi.irq >= currd->nr_pirqs ) break; - spin_lock(&currd->event_lock); + read_lock(&currd->event_lock); pirq = pirq_info(currd, eoi.irq); if ( !pirq ) { - spin_unlock(&currd->event_lock); + read_unlock(&currd->event_lock); break; } if ( currd->arch.auto_unmask ) @@ -214,7 +214,7 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_H && hvm_irq->gsi_assert_count[gsi] ) send_guest_pirq(currd, pirq); } - spin_unlock(&currd->event_lock); + read_unlock(&currd->event_lock); ret = 0; break; } @@ -626,7 +626,7 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_H if ( copy_from_guest(&out, arg, 1) != 0 ) break; - spin_lock(&currd->event_lock); + write_lock(&currd->event_lock); ret = get_free_pirq(currd, out.type); if ( ret >= 0 ) @@ -639,7 +639,7 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_H ret = -ENOMEM; } - spin_unlock(&currd->event_lock); + write_unlock(&currd->event_lock); if ( ret >= 0 ) { --- a/xen/arch/x86/pv/shim.c +++ b/xen/arch/x86/pv/shim.c @@ -448,7 +448,7 @@ static long pv_shim_event_channel_op(int if ( rc ) \ break; \ \ - spin_lock(&d->event_lock); \ + write_lock(&d->event_lock); \ rc = evtchn_allocate_port(d, op.port_field); \ if ( rc ) \ { \ @@ -457,7 +457,7 @@ static long pv_shim_event_channel_op(int } \ else \ evtchn_reserve(d, op.port_field); \ - spin_unlock(&d->event_lock); \ + write_unlock(&d->event_lock); \ \ if ( !rc && __copy_to_guest(arg, &op, 1) ) \ rc = -EFAULT; \ @@ -585,11 +585,11 @@ static long pv_shim_event_channel_op(int if ( rc ) break; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); rc = evtchn_allocate_port(d, ipi.port); if ( rc ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); close.port = ipi.port; BUG_ON(xen_hypercall_event_channel_op(EVTCHNOP_close, &close)); @@ -598,7 +598,7 @@ static long pv_shim_event_channel_op(int evtchn_assign_vcpu(d, ipi.port, ipi.vcpu); evtchn_reserve(d, ipi.port); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); if ( __copy_to_guest(arg, &ipi, 1) ) rc = -EFAULT; --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -260,7 +260,7 @@ static long evtchn_alloc_unbound(evtchn_ if ( d == NULL ) return -ESRCH; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); if ( (port = get_free_port(d)) < 0 ) ERROR_EXIT_DOM(port, d); @@ -283,7 +283,7 @@ static long evtchn_alloc_unbound(evtchn_ out: check_free_port(d, port); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); rcu_unlock_domain(d); return rc; @@ -336,14 +336,14 @@ static long evtchn_bind_interdomain(evtc /* Avoid deadlock by first acquiring lock of domain with smaller id. */ if ( ld < rd ) { - spin_lock(&ld->event_lock); - spin_lock(&rd->event_lock); + write_lock(&ld->event_lock); + read_lock(&rd->event_lock); } else { if ( ld != rd ) - spin_lock(&rd->event_lock); - spin_lock(&ld->event_lock); + read_lock(&rd->event_lock); + write_lock(&ld->event_lock); } if ( (lport = get_free_port(ld)) < 0 ) @@ -384,9 +384,9 @@ static long evtchn_bind_interdomain(evtc out: check_free_port(ld, lport); - spin_unlock(&ld->event_lock); + write_unlock(&ld->event_lock); if ( ld != rd ) - spin_unlock(&rd->event_lock); + read_unlock(&rd->event_lock); rcu_unlock_domain(rd); @@ -418,7 +418,7 @@ int evtchn_bind_virq(evtchn_bind_virq_t if ( (v = domain_vcpu(d, vcpu)) == NULL ) return -ENOENT; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); if ( v->virq_to_evtchn[virq] != 0 ) ERROR_EXIT(-EEXIST); @@ -451,7 +451,7 @@ int evtchn_bind_virq(evtchn_bind_virq_t v->virq_to_evtchn[virq] = bind->port = port; out: - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return rc; } @@ -468,7 +468,7 @@ static long evtchn_bind_ipi(evtchn_bind_ if ( domain_vcpu(d, vcpu) == NULL ) return -ENOENT; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); if ( (port = get_free_port(d)) < 0 ) ERROR_EXIT(port); @@ -486,7 +486,7 @@ static long evtchn_bind_ipi(evtchn_bind_ bind->port = port; out: - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return rc; } @@ -533,7 +533,7 @@ static long evtchn_bind_pirq(evtchn_bind if ( !is_hvm_domain(d) && !pirq_access_permitted(d, pirq) ) return -EPERM; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); if ( pirq_to_evtchn(d, pirq) != 0 ) ERROR_EXIT(-EEXIST); @@ -573,7 +573,7 @@ static long evtchn_bind_pirq(evtchn_bind out: check_free_port(d, port); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return rc; } @@ -589,7 +589,7 @@ int evtchn_close(struct domain *d1, int unsigned long flags; again: - spin_lock(&d1->event_lock); + write_lock(&d1->event_lock); if ( !port_is_valid(d1, port1) ) { @@ -657,13 +657,11 @@ int evtchn_close(struct domain *d1, int BUG(); if ( d1 < d2 ) - { - spin_lock(&d2->event_lock); - } + read_lock(&d2->event_lock); else if ( d1 != d2 ) { - spin_unlock(&d1->event_lock); - spin_lock(&d2->event_lock); + write_unlock(&d1->event_lock); + read_lock(&d2->event_lock); goto again; } } @@ -710,11 +708,11 @@ int evtchn_close(struct domain *d1, int if ( d2 != NULL ) { if ( d1 != d2 ) - spin_unlock(&d2->event_lock); + read_unlock(&d2->event_lock); put_domain(d2); } - spin_unlock(&d1->event_lock); + write_unlock(&d1->event_lock); return rc; } @@ -939,7 +937,7 @@ int evtchn_status(evtchn_status_t *statu if ( d == NULL ) return -ESRCH; - spin_lock(&d->event_lock); + read_lock(&d->event_lock); if ( !port_is_valid(d, port) ) { @@ -992,7 +990,7 @@ int evtchn_status(evtchn_status_t *statu status->vcpu = chn->notify_vcpu_id; out: - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); rcu_unlock_domain(d); return rc; @@ -1005,20 +1003,19 @@ long evtchn_bind_vcpu(unsigned int port, struct evtchn *chn; long rc = 0; struct vcpu *v; + bool write_locked = false; + unsigned long flags; /* Use the vcpu info to prevent speculative out-of-bound accesses */ if ( (v = domain_vcpu(d, vcpu_id)) == NULL ) return -ENOENT; - spin_lock(&d->event_lock); - if ( !port_is_valid(d, port) ) - { - rc = -EINVAL; - goto out; - } + return -EINVAL; chn = evtchn_from_port(d, port); + again: + spin_lock_irqsave(&chn->lock, flags); /* Guest cannot re-bind a Xen-attached event channel. */ if ( unlikely(consumer_is_xen(chn)) ) @@ -1042,19 +1039,32 @@ long evtchn_bind_vcpu(unsigned int port, case ECS_PIRQ: if ( chn->notify_vcpu_id == v->vcpu_id ) break; + if ( !write_locked ) + { + spin_unlock_irqrestore(&chn->lock, flags); + write_lock(&d->event_lock); + write_locked = true; + goto again; + } + unlink_pirq_port(chn, d->vcpu[chn->notify_vcpu_id]); chn->notify_vcpu_id = v->vcpu_id; + spin_unlock_irqrestore(&chn->lock, flags); pirq_set_affinity(d, chn->u.pirq.irq, cpumask_of(v->processor)); link_pirq_port(port, chn, v); - break; + + write_unlock(&d->event_lock); + return 0; default: rc = -EINVAL; break; } out: - spin_unlock(&d->event_lock); + spin_unlock_irqrestore(&chn->lock, flags); + if ( write_locked ) + write_unlock(&d->event_lock); return rc; } @@ -1098,7 +1108,7 @@ int evtchn_reset(struct domain *d, bool if ( d != current->domain && !d->controller_pause_count ) return -EINVAL; - spin_lock(&d->event_lock); + read_lock(&d->event_lock); /* * If we are resuming, then start where we stopped. Otherwise, check @@ -1109,7 +1119,7 @@ int evtchn_reset(struct domain *d, bool if ( i > d->next_evtchn ) d->next_evtchn = i; - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); if ( !i ) return -EBUSY; @@ -1121,14 +1131,14 @@ int evtchn_reset(struct domain *d, bool /* NB: Choice of frequency is arbitrary. */ if ( !(i & 0x3f) && hypercall_preempt_check() ) { - spin_lock(&d->event_lock); + write_lock(&d->event_lock); d->next_evtchn = i; - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return -ERESTART; } } - spin_lock(&d->event_lock); + write_lock(&d->event_lock); d->next_evtchn = 0; @@ -1141,7 +1151,7 @@ int evtchn_reset(struct domain *d, bool evtchn_2l_init(d); } - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return rc; } @@ -1154,11 +1164,11 @@ static long evtchn_set_priority(const st long ret; unsigned long flags; - spin_lock(&d->event_lock); + read_lock(&d->event_lock); if ( !port_is_valid(d, port) ) { - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); return -EINVAL; } @@ -1167,7 +1177,7 @@ static long evtchn_set_priority(const st ret = evtchn_port_set_priority(d, chn, set_priority->priority); spin_unlock_irqrestore(&chn->lock, flags); - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); return ret; } @@ -1337,7 +1347,7 @@ int alloc_unbound_xen_event_channel( int port, rc; unsigned long flags; - spin_lock(&ld->event_lock); + write_lock(&ld->event_lock); port = rc = get_free_port(ld); if ( rc < 0 ) @@ -1365,7 +1375,7 @@ int alloc_unbound_xen_event_channel( out: check_free_port(ld, port); - spin_unlock(&ld->event_lock); + write_unlock(&ld->event_lock); return rc < 0 ? rc : port; } @@ -1453,7 +1463,8 @@ int evtchn_init(struct domain *d, unsign return -ENOMEM; d->valid_evtchns = EVTCHNS_PER_BUCKET; - spin_lock_init_prof(d, event_lock); + rwlock_init(&d->event_lock); + if ( get_free_port(d) != 0 ) { free_evtchn_bucket(d, d->evtchn); @@ -1480,7 +1491,7 @@ int evtchn_destroy(struct domain *d) /* After this barrier no new event-channel allocations can occur. */ BUG_ON(!d->is_dying); - spin_barrier(&d->event_lock); + rw_barrier(&d->event_lock); /* Close all existing event channels. */ for ( i = d->valid_evtchns; --i; ) @@ -1538,13 +1549,13 @@ void evtchn_move_pirqs(struct vcpu *v) unsigned int port; struct evtchn *chn; - spin_lock(&d->event_lock); + read_lock(&d->event_lock); for ( port = v->pirq_evtchn_head; port; port = chn->u.pirq.next_port ) { chn = evtchn_from_port(d, port); pirq_set_affinity(d, chn->u.pirq.irq, mask); } - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); } @@ -1557,7 +1568,7 @@ static void domain_dump_evtchn_info(stru "Polling vCPUs: {%*pbl}\n" " port [p/m/s]\n", d->domain_id, d->max_vcpus, d->poll_mask); - spin_lock(&d->event_lock); + read_lock(&d->event_lock); for ( port = 1; port_is_valid(d, port); ++port ) { @@ -1604,7 +1615,7 @@ static void domain_dump_evtchn_info(stru } } - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); } static void dump_evtchn_info(unsigned char key) --- a/xen/common/event_fifo.c +++ b/xen/common/event_fifo.c @@ -561,7 +561,7 @@ int evtchn_fifo_init_control(struct evtc if ( offset & (8 - 1) ) return -EINVAL; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); /* * If this is the first control block, setup an empty event array @@ -593,13 +593,13 @@ int evtchn_fifo_init_control(struct evtc else rc = map_control_block(v, gfn, offset); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return rc; error: evtchn_fifo_destroy(d); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return rc; } @@ -652,9 +652,9 @@ int evtchn_fifo_expand_array(const struc if ( !d->evtchn_fifo ) return -EOPNOTSUPP; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); rc = add_page_to_event_array(d, expand_array->array_gfn); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return rc; } --- a/xen/drivers/passthrough/io.c +++ b/xen/drivers/passthrough/io.c @@ -105,7 +105,7 @@ static void pt_pirq_softirq_reset(struct { struct domain *d = pirq_dpci->dom; - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_write_locked(&d->event_lock)); switch ( cmpxchg(&pirq_dpci->state, 1 << STATE_SCHED, 0) ) { @@ -162,7 +162,7 @@ static void pt_irq_time_out(void *data) const struct hvm_irq_dpci *dpci; const struct dev_intx_gsi_link *digl; - spin_lock(&irq_map->dom->event_lock); + read_lock(&irq_map->dom->event_lock); if ( irq_map->flags & HVM_IRQ_DPCI_IDENTITY_GSI ) { @@ -177,7 +177,7 @@ static void pt_irq_time_out(void *data) hvm_gsi_deassert(irq_map->dom, dpci_pirq(irq_map)->pirq); irq_map->flags |= HVM_IRQ_DPCI_EOI_LATCH; pt_irq_guest_eoi(irq_map->dom, irq_map, NULL); - spin_unlock(&irq_map->dom->event_lock); + read_unlock(&irq_map->dom->event_lock); return; } @@ -185,7 +185,7 @@ static void pt_irq_time_out(void *data) if ( unlikely(!dpci) ) { ASSERT_UNREACHABLE(); - spin_unlock(&irq_map->dom->event_lock); + read_unlock(&irq_map->dom->event_lock); return; } list_for_each_entry ( digl, &irq_map->digl_list, list ) @@ -204,7 +204,7 @@ static void pt_irq_time_out(void *data) pt_pirq_iterate(irq_map->dom, pt_irq_guest_eoi, NULL); - spin_unlock(&irq_map->dom->event_lock); + read_unlock(&irq_map->dom->event_lock); } struct hvm_irq_dpci *domain_get_irq_dpci(const struct domain *d) @@ -288,7 +288,7 @@ int pt_irq_create_bind( return -EINVAL; restart: - spin_lock(&d->event_lock); + write_lock(&d->event_lock); hvm_irq_dpci = domain_get_irq_dpci(d); if ( !hvm_irq_dpci && !is_hardware_domain(d) ) @@ -304,7 +304,7 @@ int pt_irq_create_bind( hvm_irq_dpci = xzalloc(struct hvm_irq_dpci); if ( hvm_irq_dpci == NULL ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return -ENOMEM; } for ( i = 0; i < NR_HVM_DOMU_IRQS; i++ ) @@ -316,7 +316,7 @@ int pt_irq_create_bind( info = pirq_get_info(d, pirq); if ( !info ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return -ENOMEM; } pirq_dpci = pirq_dpci(info); @@ -331,7 +331,7 @@ int pt_irq_create_bind( */ if ( pt_pirq_softirq_active(pirq_dpci) ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); cpu_relax(); goto restart; } @@ -389,7 +389,7 @@ int pt_irq_create_bind( pirq_dpci->dom = NULL; pirq_dpci->flags = 0; pirq_cleanup_check(info, d); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return rc; } } @@ -399,7 +399,7 @@ int pt_irq_create_bind( if ( (pirq_dpci->flags & mask) != mask ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return -EBUSY; } @@ -423,7 +423,7 @@ int pt_irq_create_bind( dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode); pirq_dpci->gmsi.dest_vcpu_id = dest_vcpu_id; - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); pirq_dpci->gmsi.posted = false; vcpu = (dest_vcpu_id >= 0) ? d->vcpu[dest_vcpu_id] : NULL; @@ -483,7 +483,7 @@ int pt_irq_create_bind( if ( !digl || !girq ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); xfree(girq); xfree(digl); return -ENOMEM; @@ -510,7 +510,7 @@ int pt_irq_create_bind( if ( pt_irq_bind->irq_type != PT_IRQ_TYPE_PCI || pirq >= hvm_domain_irq(d)->nr_gsis ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return -EINVAL; } @@ -546,7 +546,7 @@ int pt_irq_create_bind( if ( mask < 0 || trigger_mode < 0 ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); ASSERT_UNREACHABLE(); return -EINVAL; @@ -594,14 +594,14 @@ int pt_irq_create_bind( } pirq_dpci->flags = 0; pirq_cleanup_check(info, d); - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); xfree(girq); xfree(digl); return rc; } } - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); if ( iommu_verbose ) { @@ -619,7 +619,7 @@ int pt_irq_create_bind( } default: - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return -EOPNOTSUPP; } @@ -672,13 +672,13 @@ int pt_irq_destroy_bind( return -EOPNOTSUPP; } - spin_lock(&d->event_lock); + write_lock(&d->event_lock); hvm_irq_dpci = domain_get_irq_dpci(d); if ( !hvm_irq_dpci && !is_hardware_domain(d) ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return -EINVAL; } @@ -711,7 +711,7 @@ int pt_irq_destroy_bind( if ( girq ) { - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return -EINVAL; } @@ -755,7 +755,7 @@ int pt_irq_destroy_bind( pirq_cleanup_check(pirq, d); } - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); if ( what && iommu_verbose ) { @@ -799,7 +799,7 @@ int pt_pirq_iterate(struct domain *d, unsigned int pirq = 0, n, i; struct pirq *pirqs[8]; - ASSERT(spin_is_locked(&d->event_lock)); + ASSERT(rw_is_locked(&d->event_lock)); do { n = radix_tree_gang_lookup(&d->pirq_tree, (void **)pirqs, pirq, @@ -880,9 +880,9 @@ void hvm_dpci_msi_eoi(struct domain *d, (!hvm_domain_irq(d)->dpci && !is_hardware_domain(d)) ) return; - spin_lock(&d->event_lock); + read_lock(&d->event_lock); pt_pirq_iterate(d, _hvm_dpci_msi_eoi, (void *)(long)vector); - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); } static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci) @@ -893,7 +893,7 @@ static void hvm_dirq_assist(struct domai return; } - spin_lock(&d->event_lock); + read_lock(&d->event_lock); if ( test_and_clear_bool(pirq_dpci->masked) ) { struct pirq *pirq = dpci_pirq(pirq_dpci); @@ -947,7 +947,7 @@ static void hvm_dirq_assist(struct domai } out: - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); } static void hvm_pirq_eoi(struct pirq *pirq, @@ -1012,7 +1012,7 @@ void hvm_dpci_eoi(struct domain *d, unsi if ( is_hardware_domain(d) ) { - spin_lock(&d->event_lock); + read_lock(&d->event_lock); hvm_gsi_eoi(d, guest_gsi, ent); goto unlock; } @@ -1023,7 +1023,7 @@ void hvm_dpci_eoi(struct domain *d, unsi return; } - spin_lock(&d->event_lock); + read_lock(&d->event_lock); hvm_irq_dpci = domain_get_irq_dpci(d); if ( !hvm_irq_dpci ) @@ -1033,7 +1033,7 @@ void hvm_dpci_eoi(struct domain *d, unsi __hvm_dpci_eoi(d, girq, ent); unlock: - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); } /* --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -883,7 +883,7 @@ static int pci_clean_dpci_irqs(struct do if ( !is_hvm_domain(d) ) return 0; - spin_lock(&d->event_lock); + write_lock(&d->event_lock); hvm_irq_dpci = domain_get_irq_dpci(d); if ( hvm_irq_dpci != NULL ) { @@ -901,14 +901,14 @@ static int pci_clean_dpci_irqs(struct do ret = pt_pirq_iterate(d, pci_clean_dpci_irq, NULL); if ( ret ) { - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); return ret; } hvm_domain_irq(d)->dpci = NULL; free_hvm_irq_dpci(hvm_irq_dpci); } - spin_unlock(&d->event_lock); + write_unlock(&d->event_lock); return 0; } --- a/xen/drivers/passthrough/vtd/x86/hvm.c +++ b/xen/drivers/passthrough/vtd/x86/hvm.c @@ -54,7 +54,7 @@ void hvm_dpci_isairq_eoi(struct domain * if ( !is_iommu_enabled(d) ) return; - spin_lock(&d->event_lock); + read_lock(&d->event_lock); dpci = domain_get_irq_dpci(d); @@ -63,5 +63,5 @@ void hvm_dpci_isairq_eoi(struct domain * /* Multiple mirq may be mapped to one isa irq */ pt_pirq_iterate(d, _hvm_dpci_isairq_eoi, (void *)(long)isairq); } - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); } --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -373,7 +373,7 @@ struct domain unsigned int xen_evtchns; /* Port to resume from in evtchn_reset(), when in a continuation. */ unsigned int next_evtchn; - spinlock_t event_lock; + rwlock_t event_lock; const struct evtchn_port_ops *evtchn_port_ops; struct evtchn_fifo_domain *evtchn_fifo; --- a/xen/xsm/flask/flask_op.c +++ b/xen/xsm/flask/flask_op.c @@ -555,7 +555,7 @@ static int flask_get_peer_sid(struct xen struct evtchn *chn; struct domain_security_struct *dsec; - spin_lock(&d->event_lock); + read_lock(&d->event_lock); if ( !port_is_valid(d, arg->evtchn) ) goto out; @@ -573,7 +573,7 @@ static int flask_get_peer_sid(struct xen rv = 0; out: - spin_unlock(&d->event_lock); + read_unlock(&d->event_lock); return rv; }