| Message ID | 1453254177-103002-4-git-send-email-feng.wu@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
2016-01-20 09:42+0800, Feng Wu: > Use vector-hashing to deliver lowest-priority interrupts for > VT-d posted-interrupts. This patch extends kvm_intr_is_single_vcpu() > to support lowest-priority handling. > > Signed-off-by: Feng Wu <feng.wu@intel.com> > --- > v3: > - Remove unnecessary check in fast irq delivery patch > - print a error message only once for each guest when we find hardware > disabled LAPIC during interrupt injection. > > diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h > @@ -1316,8 +1316,8 @@ int x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size); > bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu); > bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu); > > -bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq, > - struct kvm_vcpu **dest_vcpu); > +bool kvm_intr_can_posting(struct kvm *kvm, struct kvm_lapic_irq *irq, I prefer the original one; I think it's better to describe function than intent in names -- intention should be obvious from its use. > + struct kvm_vcpu **dest_vcpu); > > void kvm_set_msi_irq(struct kvm_kernel_irq_routing_entry *e, > struct kvm_lapic_irq *irq); > diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c > @@ -300,13 +300,13 @@ out: > return r; > } > > -bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq, > - struct kvm_vcpu **dest_vcpu) > +bool kvm_intr_can_posting(struct kvm *kvm, struct kvm_lapic_irq *irq, > + struct kvm_vcpu **dest_vcpu) > { > int i, r = 0; > struct kvm_vcpu *vcpu; > > - if (kvm_intr_is_single_vcpu_fast(kvm, irq, dest_vcpu)) > + if (kvm_intr_can_posting_fast(kvm, irq, dest_vcpu)) > return true; There is one pitfall: xAPIC flat logical broadcast returns false, but lowest priority is defined for it (practically isn't a broadcast) and the rest of this function doesn't check for lowest priority, so the interrupt won't be posted. We could modify our _fast functions to cover 0xff in flat logical, but ignoring this case isn't bad either ... it can happen only with 8 VCPU guests. I'd just comment that we did it on purpose. :) > > kvm_for_each_vcpu(i, vcpu, kvm) { [Comments for the rest mirror [2/4].] -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> -----Original Message----- > From: Radim Kr?má? [mailto:rkrcmar@redhat.com] > Sent: Friday, January 22, 2016 4:17 AM > To: Wu, Feng <feng.wu@intel.com> > Cc: pbonzini@redhat.com; linux-kernel@vger.kernel.org; > kvm@vger.kernel.org > Subject: Re: [PATCH v3 3/4] KVM: x86: Add lowest-priority support for vt-d > posted-interrupts > > 2016-01-20 09:42+0800, Feng Wu: > > Use vector-hashing to deliver lowest-priority interrupts for > > VT-d posted-interrupts. This patch extends kvm_intr_is_single_vcpu() > > to support lowest-priority handling. > > > > Signed-off-by: Feng Wu <feng.wu@intel.com> > > --- > > v3: > > - Remove unnecessary check in fast irq delivery patch > > - print a error message only once for each guest when we find hardware > > disabled LAPIC during interrupt injection. > > > > diff --git a/arch/x86/include/asm/kvm_host.h > b/arch/x86/include/asm/kvm_host.h > > @@ -1316,8 +1316,8 @@ int x86_set_memory_region(struct kvm *kvm, int > id, gpa_t gpa, u32 size); > > bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu); > > bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu); > > > > -bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq, > > - struct kvm_vcpu **dest_vcpu); > > +bool kvm_intr_can_posting(struct kvm *kvm, struct kvm_lapic_irq *irq, > > I prefer the original one; I think it's better to describe function > than intent in names -- intention should be obvious from its use. > > > + struct kvm_vcpu **dest_vcpu); > > > > void kvm_set_msi_irq(struct kvm_kernel_irq_routing_entry *e, > > struct kvm_lapic_irq *irq); > > diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c > > @@ -300,13 +300,13 @@ out: > > return r; > > } > > > > -bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq, > > - struct kvm_vcpu **dest_vcpu) > > +bool kvm_intr_can_posting(struct kvm *kvm, struct kvm_lapic_irq *irq, > > + struct kvm_vcpu **dest_vcpu) > > { > > int i, r = 0; > > struct kvm_vcpu *vcpu; > > > > - if (kvm_intr_is_single_vcpu_fast(kvm, irq, dest_vcpu)) > > + if (kvm_intr_can_posting_fast(kvm, irq, dest_vcpu)) > > return true; > > There is one pitfall: xAPIC flat logical broadcast returns false, Do you mean kvm_intr_can_posting_fast() returns false for xAPIC flat logical lowest-priority broadcast? After carefully read the code for several times, I still cannot find the reason, could you please give more hints? BTW, I noticed there is a "if(irq->dest_id == 0xFF) goto out;" in this function, but it is for the physical dest mode. I am not sure you mean this. > but lowest priority is defined for it (practically isn't a broadcast) and > the rest of this function doesn't check for lowest priority, so the > interrupt won't be posted. > > We could modify our _fast functions to cover 0xff in flat logical, but > ignoring this case isn't bad either ... it can happen only with 8 VCPU > guests. Could you please elaborate a bit more why only for the 8 VCPU guests? Thanks a lot! Thanks, Feng -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2016-01-22 05:12+0000, Wu, Feng: >> From: Radim Kr?má? [mailto:rkrcmar@redhat.com] >> 2016-01-20 09:42+0800, Feng Wu: >>> - if (kvm_intr_is_single_vcpu_fast(kvm, irq, dest_vcpu)) >>> + if (kvm_intr_can_posting_fast(kvm, irq, dest_vcpu)) >>> return true; >> >> There is one pitfall: xAPIC flat logical broadcast returns false, > > Do you mean kvm_intr_can_posting_fast() returns false for > xAPIC flat logical lowest-priority broadcast? I did. > After carefully read the code for several times, I still cannot > find the reason, could you please give more hints? You are right, there isn't a problem in the code. > BTW, I noticed there is a "if(irq->dest_id == 0xFF) goto out;" in > this function, but it is for the physical dest mode. I am not > sure you mean this. I didn't check if my assumptions were wrong. I'm sorry. >> but lowest priority is defined for it (practically isn't a broadcast) and >> the rest of this function doesn't check for lowest priority, so the >> interrupt won't be posted. >> >> We could modify our _fast functions to cover 0xff in flat logical, but >> ignoring this case isn't bad either ... it can happen only with 8 VCPU >> guests. > > Could you please elaborate a bit more why only for the 8 VCPU guests? xAPIC flat logical doesn't forbid lowest priority broadcasts, but lowest priority delivery still needs to have all destinations enabled, which can only happen with 8 VCPUs. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 5054810..ce5a2ea 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1316,8 +1316,8 @@ int x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size); bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu); bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu); -bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq, - struct kvm_vcpu **dest_vcpu); +bool kvm_intr_can_posting(struct kvm *kvm, struct kvm_lapic_irq *irq, + struct kvm_vcpu **dest_vcpu); void kvm_set_msi_irq(struct kvm_kernel_irq_routing_entry *e, struct kvm_lapic_irq *irq); diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c index 062e907..75cd02b 100644 --- a/arch/x86/kvm/irq_comm.c +++ b/arch/x86/kvm/irq_comm.c @@ -300,13 +300,13 @@ out: return r; } -bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq, - struct kvm_vcpu **dest_vcpu) +bool kvm_intr_can_posting(struct kvm *kvm, struct kvm_lapic_irq *irq, + struct kvm_vcpu **dest_vcpu) { int i, r = 0; struct kvm_vcpu *vcpu; - if (kvm_intr_is_single_vcpu_fast(kvm, irq, dest_vcpu)) + if (kvm_intr_can_posting_fast(kvm, irq, dest_vcpu)) return true; kvm_for_each_vcpu(i, vcpu, kvm) { @@ -325,7 +325,7 @@ bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq, return r == 1; } -EXPORT_SYMBOL_GPL(kvm_intr_is_single_vcpu); +EXPORT_SYMBOL_GPL(kvm_intr_can_posting); #define IOAPIC_ROUTING_ENTRY(irq) \ { .gsi = irq, .type = KVM_IRQ_ROUTING_IRQCHIP, \ diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index e1a449da..d22e87e 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -800,7 +800,22 @@ out: return ret; } -bool kvm_intr_is_single_vcpu_fast(struct kvm *kvm, struct kvm_lapic_irq *irq, +/* + * This routine tries to handler interrupts in posted mode, here is how + * it deals with different cases: + * - For single-destination interrupts, handle it in posted mode + * - Else if vector hashing is enabled and it is a lowest-priority + * interrupt, handle it in posted mode and use the following mechanism + * to find the destinaiton vCPU. + * 1. For lowest-priority interrupts, store all the possible + * destination vCPUs in an array. + * 2. Use "guest vector % max number of destination vCPUs" to find + * the right destination vCPU in the array for the lowest-priority + * interrupt. + * - Otherwise, use remapped mode to inject the interrupt. + */ + +bool kvm_intr_can_posting_fast(struct kvm *kvm, struct kvm_lapic_irq *irq, struct kvm_vcpu **dest_vcpu) { struct kvm_apic_map *map; @@ -841,16 +856,44 @@ bool kvm_intr_is_single_vcpu_fast(struct kvm *kvm, struct kvm_lapic_irq *irq, if (cid >= ARRAY_SIZE(map->logical_map)) goto out; - for_each_set_bit(i, &bitmap, 16) { - dst = map->logical_map[cid][i]; - if (++r == 2) + if (kvm_vector_hashing_enabled() && + kvm_lowest_prio_delivery(irq)) { + int idx = 0; + unsigned int dest_vcpus = 0; + + dest_vcpus = hweight16(bitmap); + if (dest_vcpus == 0) goto out; - } - if (dst && kvm_apic_present(dst->vcpu)) + idx = kvm_vector_2_index(irq->vector, dest_vcpus, + &bitmap, 16); + + /* + * We may find a hardware disabled LAPIC here, if that + * is the case, print out a error message once for each + * guest and return + */ + dst = map->logical_map[cid][idx-1]; + if (!dst && (kvm->arch.disabled_lapic_found == 0)) { + kvm->arch.disabled_lapic_found = 1; + printk(KERN_ERR + "Disabled LAPIC found during irq injection\n"); + goto out; + } + *dest_vcpu = dst->vcpu; - else - goto out; + } else { + for_each_set_bit(i, &bitmap, 16) { + dst = map->logical_map[cid][i]; + if (++r == 2) + goto out; + } + + if (dst && kvm_apic_present(dst->vcpu)) + *dest_vcpu = dst->vcpu; + else + goto out; + } } ret = true; diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index d864601..6145939 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -173,8 +173,8 @@ bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector); void wait_lapic_expire(struct kvm_vcpu *vcpu); -bool kvm_intr_is_single_vcpu_fast(struct kvm *kvm, struct kvm_lapic_irq *irq, - struct kvm_vcpu **dest_vcpu); +bool kvm_intr_can_posting_fast(struct kvm *kvm, struct kvm_lapic_irq *irq, + struct kvm_vcpu **dest_vcpu); int kvm_vector_2_index(u32 vector, u32 dest_vcpus, const unsigned long *bitmap, u32 bitmap_size); #endif diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 13d14d4..b909ea1 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -10764,7 +10764,7 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, */ kvm_set_msi_irq(e, &irq); - if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { + if (!kvm_intr_can_posting(kvm, &irq, &vcpu)) { /* * Make sure the IRTE is in remapped mode if * we don't handle it in posted mode.
Use vector-hashing to deliver lowest-priority interrupts for VT-d posted-interrupts. This patch extends kvm_intr_is_single_vcpu() to support lowest-priority handling. Signed-off-by: Feng Wu <feng.wu@intel.com> --- v3: - Remove unnecessary check in fast irq delivery patch - print a error message only once for each guest when we find hardware disabled LAPIC during interrupt injection. arch/x86/include/asm/kvm_host.h | 4 +-- arch/x86/kvm/irq_comm.c | 8 +++--- arch/x86/kvm/lapic.c | 59 +++++++++++++++++++++++++++++++++++------ arch/x86/kvm/lapic.h | 4 +-- arch/x86/kvm/vmx.c | 2 +- 5 files changed, 60 insertions(+), 17 deletions(-)