From patchwork Fri Nov 15 20:10:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 11247087 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DA856C1 for ; Fri, 15 Nov 2019 20:12:18 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 68C402073A for ; Fri, 15 Nov 2019 20:12:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 68C402073A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=dornerworks.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iVhvw-0006zA-KR; Fri, 15 Nov 2019 20:11:08 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iVhvv-0006yc-0h for xen-devel@lists.xenproject.org; Fri, 15 Nov 2019 20:11:07 +0000 X-Inumbo-ID: 04d86d92-07e4-11ea-b678-bc764e2007e4 Received: from webmail.dornerworks.com (unknown [12.207.209.150]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 04d86d92-07e4-11ea-b678-bc764e2007e4; Fri, 15 Nov 2019 20:10:51 +0000 (UTC) From: Stewart Hildebrand To: Date: Fri, 15 Nov 2019 15:10:35 -0500 Message-ID: <20191115201037.44982-5-stewart.hildebrand@dornerworks.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191115200115.44890-1-stewart.hildebrand@dornerworks.com> References: <20191115200115.44890-1-stewart.hildebrand@dornerworks.com> MIME-Version: 1.0 X-Originating-IP: [172.27.14.58] X-ClientProxiedBy: Mcbain.dw.local (172.27.1.45) To Mcbain.dw.local (172.27.1.45) X-spam-status: No, score=-2.9 required=3.5 tests=ALL_TRUSTED, BAYES_00, MAILSHELL_SCORE_10_69 X-Spam-Flag: NO Subject: [Xen-devel] [XEN PATCH v3 09/11] xen: arm: gic: supporting routing a PPI to the current vcpu. X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Volodymyr Babchuk , Stefano Stabellini , Julien Grall , Ian Campbell Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Ian Campbell That is whichever vcpu is resident when the interrupt fires. An interrupt is in this state when both IRQ_GUEST and IRQ_PER_CPU are set in the descriptor status. Only PPIs can be in this mode. This requires some peripheral specific code to make use of the previously introduced functionality to save and restore the PPI state. The vtimer driver will do so shortly. Signed-off-by: Ian Campbell Signed-off-by: Stewart Hildebrand --- v3: * Change calls to gic_set_irq_properties() to gic_set_irq_type() and gic_set_irq_priority() due to following commits: 16580cde5a xen/arm: gic: Do not configure affinity during routing 23e8118b8e xen/arm: gic: split set_irq_properties * Partially address feedback from v2 [1]: * Clarify a comment. * Switch loglevel back to XENLOG_G_ERR and bump a parameter to the next line to comply with line length coding style. * Call vgic_get_hw_irq_desc from gic_save_and_mask_hwppi * Call vgic_connect_hw_irq from gic_restore_hwppi --- Note: I have not yet addressed feedback from [1] regarding differentiating between CPU0/CPU1 in the error message. I also have not yet given much thought to Julien's comment in [1] "Why do you set the parameter virq to irq?" I hope to investigate further if time allows, but if anyone has any input I'd like to hear it. [1] https://lists.xenproject.org/archives/html/xen-devel/2015-11/msg01064.html --- xen/arch/arm/gic.c | 33 ++++++++++++++-- xen/arch/arm/irq.c | 80 +++++++++++++++++++++++++++++++++++---- xen/include/asm-arm/gic.h | 2 + xen/include/asm-arm/irq.h | 1 + 4 files changed, 104 insertions(+), 12 deletions(-) diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index 75921724dd..982afaadbd 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -92,8 +92,7 @@ void gic_save_state(struct vcpu *v) void gic_save_and_mask_hwppi(struct vcpu *v, const unsigned virq, struct hwppi_state *s) { - struct pending_irq *p = irq_to_pending(v, virq); - struct irq_desc *desc = p->desc; + struct irq_desc *desc = vgic_get_hw_irq_desc(NULL, v, virq); spin_lock(&desc->lock); @@ -123,7 +122,6 @@ void gic_restore_hwppi(struct vcpu *v, const unsigned virq, const struct hwppi_state *s) { - struct pending_irq *p = irq_to_pending(v, virq); struct irq_desc *desc = irq_to_desc(s->irq); spin_lock(&desc->lock); @@ -131,7 +129,8 @@ void gic_restore_hwppi(struct vcpu *v, ASSERT(virq >= 16 && virq < 32); ASSERT(!is_idle_vcpu(v)); - p->desc = desc; /* Migrate to new physical processor */ + /* Migrate to new physical processor */ + vgic_connect_hw_irq(v->domain, v, virq, desc, true); irq_set_virq(desc, virq); @@ -178,6 +177,32 @@ void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority) gic_set_irq_priority(desc, priority); } +/* + * Program the GIC to route an interrupt to the current guest. + * + * That is, the IRQ is delivered to whichever VCPU happens to be + * resident on the PCPU when the interrupt arrives. + * + * Currently the interrupt *must* be a PPI and the code responsible + * for the related hardware must save and restore the PPI with + * gic_save_and_mask_hwppi/gic_restore_hwppi. + */ +int gic_route_irq_to_current_guest(struct irq_desc *desc, + unsigned int priority) +{ + ASSERT(spin_is_locked(&desc->lock)); + ASSERT(desc->irq >= 16 && desc->irq < 32); + + desc->handler = gic_hw_ops->gic_guest_irq_type; + set_bit(_IRQ_GUEST, &desc->status); + set_bit(_IRQ_PER_CPU, &desc->status); + + gic_set_irq_type(desc, desc->arch.type); + gic_set_irq_priority(desc, GIC_PRI_IRQ); + + return 0; +} + /* Program the GIC to route an interrupt to a guest * - desc.lock must be held */ diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c index 1a8e599c2e..17dec64203 100644 --- a/xen/arch/arm/irq.c +++ b/xen/arch/arm/irq.c @@ -236,6 +236,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq) if ( test_bit(_IRQ_GUEST, &desc->status) ) { struct irq_guest *info = irq_get_guest_info(desc); + struct vcpu *v; perfc_incr(guest_irqs); desc->handler->end(desc); @@ -243,10 +244,15 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq) set_bit(_IRQ_INPROGRESS, &desc->status); /* - * The irq cannot be a PPI, we only support delivery of SPIs to - * guests. + * A PPI exposed to a guest must always be in IRQ_GUEST|IRQ_PER_CPU + * mode ("route to active VCPU"), so we use current. + * + * For SPI, we use NULL. In this case, vgic_inject_irq() will look up + * the required target for delivery to a specific guest. */ - vgic_inject_irq(info->d, NULL, info->virq, true); + v = test_bit(_IRQ_PER_CPU, &desc->status) ? current : NULL; + vgic_inject_irq(info->d, v, info->virq, true); + goto out_no_end; } @@ -362,11 +368,15 @@ int setup_irq(unsigned int irq, unsigned int irqflags, struct irqaction *new) if ( test_bit(_IRQ_GUEST, &desc->status) ) { - struct domain *d = irq_get_domain(desc); + struct irq_guest *info = irq_get_guest_info(desc); spin_unlock_irqrestore(&desc->lock, flags); - printk(XENLOG_ERR "ERROR: IRQ %u is already in use by the domain %u\n", - irq, d->domain_id); + if ( !test_bit(_IRQ_PER_CPU, &desc->status) ) + printk(XENLOG_ERR "ERROR: IRQ %u is already in use by domain %u\n", + irq, info->d->domain_id); + else + printk(XENLOG_ERR + "ERROR: IRQ %u is already in use by \n", irq); return -EBUSY; } @@ -450,8 +460,14 @@ static int setup_guest_irq(struct irq_desc *desc, unsigned int virq, if ( d != ad ) { - printk(XENLOG_G_ERR "IRQ %u is already used by domain %u\n", - irq, ad->domain_id); + if ( !test_bit(_IRQ_PER_CPU, &desc->status) ) + printk(XENLOG_G_ERR + "ERROR: IRQ %u is already used by domain %u\n", + irq, ad->domain_id); + else + printk(XENLOG_G_ERR + "ERROR: IRQ %u is already used by \n", + irq); retval = -EBUSY; } else if ( irq_get_guest_info(desc)->virq != virq ) @@ -552,6 +568,54 @@ free_info: return retval; } +/* + * Route a PPI such that it is always delivered to the current vcpu on + * the pcpu. The driver for the peripheral must use + * gic_{save_and_mask,restore}_hwppi as part of the context switch. + */ +int route_hwppi_to_current_vcpu(unsigned int irq, const char *devname) +{ + struct irq_guest *info; + struct irq_desc *desc; + unsigned long flags; + int retval = 0; + + /* Can only route PPIs to current VCPU */ + if ( irq < 16 || irq >= 32 ) + return -EINVAL; + + desc = irq_to_desc(irq); + + info = xmalloc(struct irq_guest); + if ( !info ) + return -ENOMEM; + + info->d = NULL; /* Routed to current vcpu, so no specific domain */ + /* info->virq is set by gic_restore_hwppi. */ + + spin_lock_irqsave(&desc->lock, flags); + + retval = setup_guest_irq(desc, irq, flags, info, devname); + if ( retval ) + { + xfree(info); + return retval; + } + + retval = gic_route_irq_to_current_guest(desc, GIC_PRI_IRQ); + + spin_unlock_irqrestore(&desc->lock, flags); + + if ( retval ) + { + release_irq(desc->irq, info); + xfree(info); + return retval; + } + + return 0; +} + int release_guest_irq(struct domain *d, unsigned int virq) { struct irq_desc *desc; diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h index 1164e0c7a6..6a0910e13e 100644 --- a/xen/include/asm-arm/gic.h +++ b/xen/include/asm-arm/gic.h @@ -244,6 +244,8 @@ extern void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority); extern int gic_route_irq_to_guest(struct domain *, unsigned int virq, struct irq_desc *desc, unsigned int priority); +int gic_route_irq_to_current_guest(struct irq_desc *desc, + unsigned int priority); /* Remove an IRQ passthrough to a guest */ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq, diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h index 367fe6269c..c51265180b 100644 --- a/xen/include/asm-arm/irq.h +++ b/xen/include/asm-arm/irq.h @@ -77,6 +77,7 @@ void init_secondary_IRQ(void); int route_irq_to_guest(struct domain *d, unsigned int virq, unsigned int irq, const char *devname); +int route_hwppi_to_current_vcpu(unsigned int irq, const char *devname); int release_guest_irq(struct domain *d, unsigned int irq); void arch_move_irqs(struct vcpu *v);