From patchwork Tue Oct 1 12:37:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6B291CE7D1B for ; Tue, 1 Oct 2024 12:38:48 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808067.1219867 (Exim 4.92) (envelope-from ) id 1svc9D-0003AC-QD; Tue, 01 Oct 2024 12:38:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808067.1219867; Tue, 01 Oct 2024 12:38:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9D-0003A5-N3; Tue, 01 Oct 2024 12:38:35 +0000 Received: by outflank-mailman (input) for mailman id 808067; Tue, 01 Oct 2024 12:38:34 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9C-0002va-1O for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:34 +0000 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [2a00:1450:4864:20::633]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 10f95c99-7ff2-11ef-99a2-01e77a169b0f; Tue, 01 Oct 2024 14:38:31 +0200 (CEST) Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-a8a706236bfso450422566b.0 for ; Tue, 01 Oct 2024 05:38:31 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:30 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 10f95c99-7ff2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786311; x=1728391111; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6CLVhmVmABNvSZWpsVfABNNcbL+sBo7kkWt6jjP+OBE=; b=NJ/szdGP36LpKNG2bVcKYbzIORaDGjl3I5iEdHoxURAQ+JQDnqAMWBSSWAd4XGInYj FUTUEwaXk46QijxCWi+MwoZna31DnD6ooBABZMFT0OfG2+QRx/Ei2vvvraqell4aAfMd WpgPdoe3mie20w0bFOYjwOAbdVcYY6XfD3XHk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786311; x=1728391111; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6CLVhmVmABNvSZWpsVfABNNcbL+sBo7kkWt6jjP+OBE=; b=YsYFmaE5QVS03GvPJYvGQAMWwWBavPBxXVQRhUCyGJY7YvrTFicZns+6L4nuSlLSPf iV9yMc1qvfthAZOv/qYPPwax28NAG/S/5/9RmcGimBmbMiu12N4QVTZBD1KosjLASQSq espHzA95wfEoHV88j7DHRkwU7YSfLW5KdAF+PcJdgBXCfw2R4f3ShaGSEunpica7rWWS SYJUeoJpA0duHq9OAz9CDOBLjQgxfUKItlmmvUiiRRT8OsgYY8qHGJLDdIZYMy+sd20X dcaZ5m6WdCJTccguUVcPUHcA+NsJZjJFy8g9vZh5b3tXmM/13s13XlvzATAywNfuHhSp GJQA== X-Gm-Message-State: AOJu0YzYinCn532Z0LEhSC4Dg12R30Adfnqi5AAxo3w46dTe9KQfNrdx Ag7SZz7+i02fsLxnJ+DzC3PO954TgJqB34i5RP95It9L1z6mO3IqeDwVvL7FoD/xzmSiL3yKuPo kA2U= X-Google-Smtp-Source: AGHT+IGBeJtPNsZn+19IIMd92YMaOk91ubtWYoKiFy2/Ft20p7KGSCFMPdE+4Gfcl9TA9oN7LlEddw== X-Received: by 2002:a05:6402:2743:b0:5c8:9553:a1fb with SMTP id 4fb4d7f45d1cf-5c89553a26emr13424439a12.9.1727786310712; Tue, 01 Oct 2024 05:38:30 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Anthony PERARD Subject: [PATCH v6 01/11] lib/x86: Relax checks about policy compatibility Date: Tue, 1 Oct 2024 13:37:57 +0100 Message-ID: <20241001123807.605-2-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Allow a guest policy have up to leaf 0xb even if the host doesn't. Otherwise it's not possible to show leaf 0xb to guests we're emulating an x2APIC for on old AMD machines. No externally visible changes though because toolstack doesn't yet populate that leaf. Signed-off-by: Alejandro Vallejo --- tools/tests/cpu-policy/test-cpu-policy.c | 6 +++++- xen/lib/x86/policy.c | 11 ++++++++++- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c index 301df2c00285..9216010b1c5d 100644 --- a/tools/tests/cpu-policy/test-cpu-policy.c +++ b/tools/tests/cpu-policy/test-cpu-policy.c @@ -586,6 +586,10 @@ static void test_is_compatible_success(void) .platform_info.cpuid_faulting = true, }, }, + { + .name = "Host missing leaf 0xb, Guest wanted", + .guest.basic.max_leaf = 0xb, + }, }; struct cpu_policy_errors no_errors = INIT_CPU_POLICY_ERRORS; @@ -614,7 +618,7 @@ static void test_is_compatible_failure(void) } tests[] = { { .name = "Host basic.max_leaf out of range", - .guest.basic.max_leaf = 1, + .guest.basic.max_leaf = 0xc, .e = { 0, -1, -1 }, }, { diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c index f033d22785be..63bc96451d2c 100644 --- a/xen/lib/x86/policy.c +++ b/xen/lib/x86/policy.c @@ -15,7 +15,16 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host, #define FAIL_MSR(m) \ do { e.msr = (m); goto out; } while ( 0 ) - if ( guest->basic.max_leaf > host->basic.max_leaf ) + /* + * Old AMD hardware doesn't expose topology information in leaf 0xb. We + * want to emulate that leaf with credible information because it must be + * present on systems in which we emulate the x2APIC. + * + * For that reason, allow the max basic guest leaf to be larger than the + * hosts' up until 0xb. + */ + if ( guest->basic.max_leaf > 0xb && + guest->basic.max_leaf > host->basic.max_leaf ) FAIL_CPUID(0, NA); if ( guest->feat.max_subleaf > host->feat.max_subleaf ) From patchwork Tue Oct 1 12:37:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817920 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 056CECE7D19 for ; Tue, 1 Oct 2024 12:38:47 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808069.1219882 (Exim 4.92) (envelope-from ) id 1svc9E-0003OB-Ku; Tue, 01 Oct 2024 12:38:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808069.1219882; Tue, 01 Oct 2024 12:38:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9E-0003L6-DF; Tue, 01 Oct 2024 12:38:36 +0000 Received: by outflank-mailman (input) for mailman id 808069; Tue, 01 Oct 2024 12:38:35 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9D-0002va-1i for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:35 +0000 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [2a00:1450:4864:20::52e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1186ec32-7ff2-11ef-99a2-01e77a169b0f; Tue, 01 Oct 2024 14:38:32 +0200 (CEST) Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-5c87853df28so6153650a12.3 for ; Tue, 01 Oct 2024 05:38:32 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:30 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1186ec32-7ff2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786311; x=1728391111; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UB02sToOMWoS+3+bvVBptWjoHbPyERDalpALvStk7iE=; b=C26rpQfH5gIZ6gzOlY/vAxSiFxOAwu9t2tAqKdVoQqAWCG81shbNyXSIuOuc/LhBDO agnwLrAiAU5AQ3FtoggFyG+zXSv1YKMnG3J5IuQkYDb43EWqV8wY1qgufm5rwl4ig1TE ub3ycllDRLekx/BIw46uQvgoyDKWE56yk0mXg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786311; x=1728391111; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UB02sToOMWoS+3+bvVBptWjoHbPyERDalpALvStk7iE=; b=aU0/p8yTG7zKpUHGkeWzAFWqsWoZIfhn2ocgDhAsmj2lZ1bUGbKedkpIqow0gbThK+ 3rb1eUoOS8vzIvBJhbPfMEuOUk6y9ZQ6iZn1G/dn/+kGlfO97tEBCC7VerTUYclJ2WWq tYzb6G3/5VbZT/LOuuIeXbLbkWsQmaeqPSd9DpYL6cjYXZq0sszGtodpNfZrAM4ylu1Y 6ywjEys9ac3oTbakS1m/qNvCjruWgdjv5r/jCymZQi2F52zQrVbGUCFKXRaMpZNW0ysj lmIHib+43l4X2wMOXjZ0nNBTnVbjBVoE0ac60Og7vtQrMC9YAZi479c52FDOdKOkAioe Wn3A== X-Gm-Message-State: AOJu0YzI7TJ2wcNG7QcveW5g2t1cODf35fWC2p2aLl3K2JBFsYl7KBxM K00XQN9gV6S1eB3yR6PMxAXFryjT2WyRHqHBXERmVryBms4NGN8WhgWBh9NRqDKYgB4ScYIsArk u6+k= X-Google-Smtp-Source: AGHT+IGO1GpmPkOQcfrmzXZzq9xBvb8Osa2Y450r5q4s9sfSXZx7bSEFH+ArJLFDVvvgsLXIky/HWQ== X-Received: by 2002:a05:6402:26c8:b0:5c4:1320:e5a3 with SMTP id 4fb4d7f45d1cf-5c8824e7972mr14658094a12.16.1727786311392; Tue, 01 Oct 2024 05:38:31 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v6 02/11] x86/vlapic: Move lapic migration checks to the check hooks Date: Tue, 1 Oct 2024 13:37:58 +0100 Message-ID: <20241001123807.605-3-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 While doing this, factor out checks common to architectural and hidden state. Signed-off-by: Alejandro Vallejo Reviewed-by: Roger Pau Monné --- Last reviewed in the topology series v3. Fell under the cracks. https://lore.kernel.org/xen-devel/ZlhP11Vvk6c1Ix36@macbook/ --- xen/arch/x86/hvm/vlapic.c | 84 ++++++++++++++++++++++++++------------- 1 file changed, 56 insertions(+), 28 deletions(-) diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c index 992355e511cd..101902cff889 100644 --- a/xen/arch/x86/hvm/vlapic.c +++ b/xen/arch/x86/hvm/vlapic.c @@ -1571,60 +1571,88 @@ static void lapic_load_fixup(struct vlapic *vlapic) v, vlapic->loaded.id, vlapic->loaded.ldr, good_ldr); } -static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h) -{ - unsigned int vcpuid = hvm_load_instance(h); - struct vcpu *v; - struct vlapic *s; +static int lapic_check_common(const struct domain *d, unsigned int vcpuid) +{ if ( !has_vlapic(d) ) return -ENODEV; /* Which vlapic to load? */ - if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL ) + if ( !domain_vcpu(d, vcpuid) ) { - dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n", + dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no vCPU %u\n", d->domain_id, vcpuid); return -EINVAL; } - s = vcpu_vlapic(v); + + return 0; +} + +static int cf_check lapic_check_hidden(const struct domain *d, + hvm_domain_context_t *h) +{ + unsigned int vcpuid = hvm_load_instance(h); + struct hvm_hw_lapic s; + int rc = lapic_check_common(d, vcpuid); + + if ( rc ) + return rc; + + if ( hvm_load_entry_zeroextend(LAPIC, h, &s) != 0 ) + return -ENODATA; + + /* EN=0 with EXTD=1 is illegal */ + if ( (s.apic_base_msr & (APIC_BASE_ENABLE | APIC_BASE_EXTD)) == + APIC_BASE_EXTD ) + return -EINVAL; + + return 0; +} + +static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h) +{ + unsigned int vcpuid = hvm_load_instance(h); + struct vcpu *v = d->vcpu[vcpuid]; + struct vlapic *s = vcpu_vlapic(v); if ( hvm_load_entry_zeroextend(LAPIC, h, &s->hw) != 0 ) + { + ASSERT_UNREACHABLE(); return -EINVAL; + } s->loaded.hw = 1; if ( s->loaded.regs ) lapic_load_fixup(s); - if ( !(s->hw.apic_base_msr & APIC_BASE_ENABLE) && - unlikely(vlapic_x2apic_mode(s)) ) - return -EINVAL; - hvm_update_vlapic_mode(v); return 0; } -static int cf_check lapic_load_regs(struct domain *d, hvm_domain_context_t *h) +static int cf_check lapic_check_regs(const struct domain *d, + hvm_domain_context_t *h) { unsigned int vcpuid = hvm_load_instance(h); - struct vcpu *v; - struct vlapic *s; + int rc; - if ( !has_vlapic(d) ) - return -ENODEV; + if ( (rc = lapic_check_common(d, vcpuid)) ) + return rc; - /* Which vlapic to load? */ - if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL ) - { - dprintk(XENLOG_G_ERR, "HVM restore: dom%d has no apic%u\n", - d->domain_id, vcpuid); - return -EINVAL; - } - s = vcpu_vlapic(v); + if ( !hvm_get_entry(LAPIC_REGS, h) ) + return -ENODATA; + + return 0; +} + +static int cf_check lapic_load_regs(struct domain *d, hvm_domain_context_t *h) +{ + unsigned int vcpuid = hvm_load_instance(h); + struct vcpu *v = d->vcpu[vcpuid]; + struct vlapic *s = vcpu_vlapic(v); if ( hvm_load_entry(LAPIC_REGS, h, s->regs) != 0 ) - return -EINVAL; + ASSERT_UNREACHABLE(); s->loaded.id = vlapic_get_reg(s, APIC_ID); s->loaded.ldr = vlapic_get_reg(s, APIC_LDR); @@ -1641,9 +1669,9 @@ static int cf_check lapic_load_regs(struct domain *d, hvm_domain_context_t *h) return 0; } -HVM_REGISTER_SAVE_RESTORE(LAPIC, lapic_save_hidden, NULL, +HVM_REGISTER_SAVE_RESTORE(LAPIC, lapic_save_hidden, lapic_check_hidden, lapic_load_hidden, 1, HVMSR_PER_VCPU); -HVM_REGISTER_SAVE_RESTORE(LAPIC_REGS, lapic_save_regs, NULL, +HVM_REGISTER_SAVE_RESTORE(LAPIC_REGS, lapic_save_regs, lapic_check_regs, lapic_load_regs, 1, HVMSR_PER_VCPU); int vlapic_init(struct vcpu *v) From patchwork Tue Oct 1 12:37:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817924 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 356D6CE7D1F for ; Tue, 1 Oct 2024 12:38:49 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808068.1219871 (Exim 4.92) (envelope-from ) id 1svc9E-0003Dh-31; Tue, 01 Oct 2024 12:38:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808068.1219871; Tue, 01 Oct 2024 12:38:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9D-0003Co-Ub; Tue, 01 Oct 2024 12:38:35 +0000 Received: by outflank-mailman (input) for mailman id 808068; Tue, 01 Oct 2024 12:38:34 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9C-0002vk-DU for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:34 +0000 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [2a00:1450:4864:20::530]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 11da5997-7ff2-11ef-a0ba-8be0dac302b0; Tue, 01 Oct 2024 14:38:33 +0200 (CEST) Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-5c88c9e45c2so4808893a12.0 for ; Tue, 01 Oct 2024 05:38:33 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:31 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 11da5997-7ff2-11ef-a0ba-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786312; x=1728391112; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D90t4YTJEGwdBnk8XSsABj6XnnDwF6CKUln1kNou/II=; b=hVDnUuSiS1c4cthZvqqIznEOGXf4QGMTY1zw0NpGLEoHXax0shQYClQHCVrU1TGV8S l0aOAXh7Ou/LAjUh7kDpyilFBiDUL4glmqK4A50SJK4m5hQ/ytpGYuue4dUGC9wJfYag S55jIESnsQMb/7wygt9dvlAOlVvhrPckST42I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786312; x=1728391112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D90t4YTJEGwdBnk8XSsABj6XnnDwF6CKUln1kNou/II=; b=pOPbnOq9VD+x90NdGlkyDTKTX2Ss71MKoew+0bVCdiF1GJ2xmQWTpcnGOtv7E6zWF3 1eBx8qJSWLJmAroSY8wcRmxItkCd3sB4l+ZezlucaAdKAJHYbifnU9yrDCqCNsGEHbkH yDyGacHVvggbH5XR71syS2pQiv0njE0WSbXkWlGUadyZ3V+ydmcclWUpZ6ezDTWgq24I 3I7XaGv2O5WlUDfU4YZJDgRuMnXgUJUKHU3TXoNCAa+8yPxM6gQTlIxxwd9aDubFBSVI QoSjmPEbKWDAAyOvQZVYRMDcuf1gZmiMhgOsY7YT5rK6hu6ZCJXXzhL+3Yne2jm3C88h 4dCg== X-Gm-Message-State: AOJu0YyZMzAVDbB8zWQRYms7fvbYW4AYwY1a8wp+j//Gs7VUqQ83FAQf AyCE/KKpF4xCocp5G5jdieR4i42MWRD1AaI59bsv2uQgWjaOlvuMhPhrt95Gr8JFwGzAJXqJhZS M5Ug= X-Google-Smtp-Source: AGHT+IGBNNwmvNBx4ZTClnC8czS1nkcnS+4eIgXWGX+u53rZMl9rdxum7DUhCOTHoUuAEXC+gEHh3g== X-Received: by 2002:a05:6402:3507:b0:5c8:9f3e:5efc with SMTP id 4fb4d7f45d1cf-5c8a2a04200mr3054439a12.6.1727786312168; Tue, 01 Oct 2024 05:38:32 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v6 03/11] xen/x86: Add initial x2APIC ID to the per-vLAPIC save area Date: Tue, 1 Oct 2024 13:37:59 +0100 Message-ID: <20241001123807.605-4-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 This allows the initial x2APIC ID to be sent on the migration stream. This allows further changes to topology and APIC ID assignment without breaking existing hosts. Given the vlapic data is zero-extended on restore, fix up migrations from hosts without the field by setting it to the old convention if zero. The hardcoded mapping x2apic_id=2*vcpu_id is kept for the time being, but it's meant to be overriden by toolstack on a later patch with appropriate values. Signed-off-by: Alejandro Vallejo --- xen/arch/x86/cpuid.c | 14 +++++--------- xen/arch/x86/hvm/vlapic.c | 22 ++++++++++++++++++++-- xen/arch/x86/include/asm/hvm/vlapic.h | 1 + xen/include/public/arch-x86/hvm/save.h | 2 ++ 4 files changed, 28 insertions(+), 11 deletions(-) diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c index 2a777436ee27..dcbdeabadce9 100644 --- a/xen/arch/x86/cpuid.c +++ b/xen/arch/x86/cpuid.c @@ -138,10 +138,9 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf, const struct cpu_user_regs *regs; case 0x1: - /* TODO: Rework topology logic. */ res->b &= 0x00ffffffu; if ( is_hvm_domain(d) ) - res->b |= (v->vcpu_id * 2) << 24; + res->b |= vlapic_x2apic_id(vcpu_vlapic(v)) << 24; /* TODO: Rework vPMU control in terms of toolstack choices. */ if ( vpmu_available(v) && @@ -311,18 +310,15 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf, case 0xb: /* - * In principle, this leaf is Intel-only. In practice, it is tightly - * coupled with x2apic, and we offer an x2apic-capable APIC emulation - * to guests on AMD hardware as well. - * - * TODO: Rework topology logic. + * Don't expose topology information to PV guests. Exposed on HVM + * along with x2APIC because they are tightly coupled. */ - if ( p->basic.x2apic ) + if ( is_hvm_domain(d) && p->basic.x2apic ) { *(uint8_t *)&res->c = subleaf; /* Fix the x2APIC identifier. */ - res->d = v->vcpu_id * 2; + res->d = vlapic_x2apic_id(vcpu_vlapic(v)); } break; diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c index 101902cff889..02570f9dd63a 100644 --- a/xen/arch/x86/hvm/vlapic.c +++ b/xen/arch/x86/hvm/vlapic.c @@ -1090,7 +1090,7 @@ static uint32_t x2apic_ldr_from_id(uint32_t id) static void set_x2apic_id(struct vlapic *vlapic) { const struct vcpu *v = vlapic_vcpu(vlapic); - uint32_t apic_id = v->vcpu_id * 2; + uint32_t apic_id = vlapic->hw.x2apic_id; uint32_t apic_ldr = x2apic_ldr_from_id(apic_id); /* @@ -1470,7 +1470,7 @@ void vlapic_reset(struct vlapic *vlapic) if ( v->vcpu_id == 0 ) vlapic->hw.apic_base_msr |= APIC_BASE_BSP; - vlapic_set_reg(vlapic, APIC_ID, (v->vcpu_id * 2) << 24); + vlapic_set_reg(vlapic, APIC_ID, SET_xAPIC_ID(vlapic->hw.x2apic_id)); vlapic_do_init(vlapic); } @@ -1538,6 +1538,16 @@ static void lapic_load_fixup(struct vlapic *vlapic) const struct vcpu *v = vlapic_vcpu(vlapic); uint32_t good_ldr = x2apic_ldr_from_id(vlapic->loaded.id); + /* + * Loading record without hw.x2apic_id in the save stream, calculate using + * the traditional "vcpu_id * 2" relation. There's an implicit assumption + * that vCPU0 always has x2APIC0, which is true for the old relation, and + * still holds under the new x2APIC generation algorithm. While that case + * goes through the conditional it's benign because it still maps to zero. + */ + if ( !vlapic->hw.x2apic_id ) + vlapic->hw.x2apic_id = v->vcpu_id * 2; + /* Skip fixups on xAPIC mode, or if the x2APIC LDR is already correct */ if ( !vlapic_x2apic_mode(vlapic) || (vlapic->loaded.ldr == good_ldr) ) @@ -1606,6 +1616,13 @@ static int cf_check lapic_check_hidden(const struct domain *d, APIC_BASE_EXTD ) return -EINVAL; + /* + * Fail migrations from newer versions of Xen where + * rsvd_zero is interpreted as something else. + */ + if ( s.rsvd_zero ) + return -EINVAL; + return 0; } @@ -1687,6 +1704,7 @@ int vlapic_init(struct vcpu *v) } vlapic->pt.source = PTSRC_lapic; + vlapic->hw.x2apic_id = 2 * v->vcpu_id; vlapic->regs_page = alloc_domheap_page(v->domain, MEMF_no_owner); if ( !vlapic->regs_page ) diff --git a/xen/arch/x86/include/asm/hvm/vlapic.h b/xen/arch/x86/include/asm/hvm/vlapic.h index 2c4ff94ae7a8..85c4a236b9f6 100644 --- a/xen/arch/x86/include/asm/hvm/vlapic.h +++ b/xen/arch/x86/include/asm/hvm/vlapic.h @@ -44,6 +44,7 @@ #define vlapic_xapic_mode(vlapic) \ (!vlapic_hw_disabled(vlapic) && \ !((vlapic)->hw.apic_base_msr & APIC_BASE_EXTD)) +#define vlapic_x2apic_id(vlapic) ((vlapic)->hw.x2apic_id) /* * Generic APIC bitmap vector update & search routines. diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h index 7ecacadde165..1c2ec669ffc9 100644 --- a/xen/include/public/arch-x86/hvm/save.h +++ b/xen/include/public/arch-x86/hvm/save.h @@ -394,6 +394,8 @@ struct hvm_hw_lapic { uint32_t disabled; /* VLAPIC_xx_DISABLED */ uint32_t timer_divisor; uint64_t tdt_msr; + uint32_t x2apic_id; + uint32_t rsvd_zero; }; DECLARE_HVM_SAVE_TYPE(LAPIC, 5, struct hvm_hw_lapic); From patchwork Tue Oct 1 12:38:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 12FF5CE7D1E for ; Tue, 1 Oct 2024 12:38:49 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808070.1219888 (Exim 4.92) (envelope-from ) id 1svc9F-0003UZ-1l; Tue, 01 Oct 2024 12:38:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808070.1219888; Tue, 01 Oct 2024 12:38:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9E-0003TD-PC; Tue, 01 Oct 2024 12:38:36 +0000 Received: by outflank-mailman (input) for mailman id 808070; Tue, 01 Oct 2024 12:38:35 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9D-0002vk-2Y for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:35 +0000 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [2a00:1450:4864:20::535]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 12413986-7ff2-11ef-a0ba-8be0dac302b0; Tue, 01 Oct 2024 14:38:33 +0200 (CEST) Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-5c88b5c375fso4186274a12.3 for ; Tue, 01 Oct 2024 05:38:33 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:32 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 12413986-7ff2-11ef-a0ba-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786313; x=1728391113; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GfxQ3gahpFHXiNK/v5LJIeKG4M7IQS/lIxTTHFsFmsQ=; b=WJP0XnI7o5Hq4PzXexDYaOXAjpeaJRb9nQlSR/vBUiyVjj2Ep7iB8NYG/7M0zkihB9 K4eNxacHOQ1CfvD9Odha4U9D5Q4P4kagHCDDn+u5fGm6NpMUPpr92fnjbRYNCC6zjD9C LfVBNfBnOJXKhFtrk516cZDbM7WAe9f5L9gTk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786313; x=1728391113; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GfxQ3gahpFHXiNK/v5LJIeKG4M7IQS/lIxTTHFsFmsQ=; b=Aso8bWHj947CPHtwYIkNOZpXwYJ6IcQlK85+GpDxT3WOFM6HpqYadMDBpCVJIkzMnU vZRcTf3QQ7uiFdHkdRKW+Dhqj0/1O+AHUN9QIjsIOZuQ2ZDuUIjsmvAUi8k9AfbGvv1z fc+jfTcfhtKMKIenJfbAFCzV/8BguvwD3a/OirbW3s/o4kABOIY4hlXmBBSA09+tWvxc bYqVJ0VpIZBylhx7ZxZoyXQc3BI7gdMO4ch/Ufd4LQ+VsDMyA6TuEwHIcuyWKeTf2nwT +1JKgRmLcTpDXkT7tdpcF9Sp1fLSRWnFSjaOt6griUdc0iU9EmzL2G5LD9OukHPrW5Tp gc/g== X-Gm-Message-State: AOJu0YxZCGfcXlylURgyAhidVIU9xN3Oo4ldDak4ALaOc/iyzDKFxCpf A67KuDR72rQ1azfLDMAYsMG3IFHE6SBFmzse+hW2PB74XyTWOvQ79Vxgpcok//03RLw9HT70cX6 u078= X-Google-Smtp-Source: AGHT+IHYv9UQ1NH0BPBBzMwPwVzMJinEFh4rgxIxc5jc8prduA3Nll6k7VEnn7OBCAcZK3PkJF/3yA== X-Received: by 2002:a05:6402:2812:b0:5c8:8538:b770 with SMTP id 4fb4d7f45d1cf-5c88538bbbfmr13313247a12.8.1727786312880; Tue, 01 Oct 2024 05:38:32 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v6 04/11] xen/x86: Add supporting code for uploading LAPIC contexts during domain create Date: Tue, 1 Oct 2024 13:38:00 +0100 Message-ID: <20241001123807.605-5-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 If toolstack were to upload LAPIC contexts as part of domain creation it would encounter a problem were the architectural state does not reflect the APIC ID in the hidden state. This patch ensures updates to the hidden state trigger an update in the architectural registers so the APIC ID in both is consistent. Signed-off-by: Alejandro Vallejo --- xen/arch/x86/hvm/vlapic.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c index 02570f9dd63a..a8183c3023da 100644 --- a/xen/arch/x86/hvm/vlapic.c +++ b/xen/arch/x86/hvm/vlapic.c @@ -1640,7 +1640,27 @@ static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h) s->loaded.hw = 1; if ( s->loaded.regs ) + { + /* + * We already processed architectural regs in lapic_load_regs(), so + * this must be a migration. Fix up inconsistencies from any older Xen. + */ lapic_load_fixup(s); + } + else + { + /* + * We haven't seen architectural regs so this could be a migration or a + * plain domain create. In the domain create case it's fine to modify + * the architectural state to align it to the APIC ID that was just + * uploaded and in the migrate case it doesn't matter because the + * architectural state will be replaced by the LAPIC_REGS ctx later on. + */ + if ( vlapic_x2apic_mode(s) ) + set_x2apic_id(s); + else + vlapic_set_reg(s, APIC_ID, SET_xAPIC_ID(s->hw.x2apic_id)); + } hvm_update_vlapic_mode(v); From patchwork Tue Oct 1 12:38:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817926 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1649CEB2D0 for ; Tue, 1 Oct 2024 12:38:50 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808071.1219904 (Exim 4.92) (envelope-from ) id 1svc9G-00043H-9c; Tue, 01 Oct 2024 12:38:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808071.1219904; Tue, 01 Oct 2024 12:38:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9G-000424-5E; Tue, 01 Oct 2024 12:38:38 +0000 Received: by outflank-mailman (input) for mailman id 808071; Tue, 01 Oct 2024 12:38:36 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9E-0002va-J8 for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:36 +0000 Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [2a00:1450:4864:20::136]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 12f91f00-7ff2-11ef-99a2-01e77a169b0f; Tue, 01 Oct 2024 14:38:35 +0200 (CEST) Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-539983beb19so2209963e87.3 for ; Tue, 01 Oct 2024 05:38:34 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:33 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 12f91f00-7ff2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786314; x=1728391114; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PVd0quTcANw6t1kPQZgBl2vPnkcivNSH35qvlQWbNfA=; b=d1p7yQe3xHatPjEYCzn6b2xgMpJ0fz4dbYZ11N/EwPxxg0+YPYeSu/rQDyinPWgD/y xyfdA5Cyqxb3P1mqKagn1u0BvTV8WaIKw0zsvsEMsGta6ucHyIptU+kROKjiluRjqg2I hwP6TO79KTfoFhiLEw4digJ1Uh/V+HRdUQI/M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786314; x=1728391114; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PVd0quTcANw6t1kPQZgBl2vPnkcivNSH35qvlQWbNfA=; b=wYoHyKABAPFzqjQ6TpAvUyKsZzhlTWfV5YE6cPrhqLF68f3n7oKULeXH7jg043ypx2 NJAZfU6gNlrV+oPLc16CLnvL6M+QFieD7Exu2VmsAUVfgTp7FQgPKQ0AoifnKHe08ce9 unP3UwPQistP5m4/l4Y3oUebC/nYMyqx7PunWK+6Ln+0NUkz+f9jD1lsxeweFeTIbTkZ bOmTKymlYQeugrzjicsaqbWuiNbzzC2/rUGAwPUxnaQSxcWfeSEC28t6lH+ITuO6eofe uG9roLs7mUtpGVzm3djjByU1Qtf2vhM8WYfWrrOFtkm5lBHAgH49N/E5KtBSZW8XoFzV uBdw== X-Gm-Message-State: AOJu0Yxs09mBlSn2FBWOODqkpcYgud/SlF+xn8X3glJuNZuHXNuDb3KG RpZugWBzZpzmYPdVamTJ/Hy03J0kGygdJkoaBCQdug5sGJ7F7r3alhc4y9vqVVSwDDSVANSerwk k0XU= X-Google-Smtp-Source: AGHT+IFIZdsgDe7OkBunGV418GKNfPWk9qD5xmnGo3xLqm5hKLnrUAbXKEYZofkVNRAbG8Z0A2xCSg== X-Received: by 2002:a05:6512:401e:b0:536:a4d8:917b with SMTP id 2adb3069b0e04-5389fc3a859mr8129082e87.19.1727786313930; Tue, 01 Oct 2024 05:38:33 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Anthony PERARD Subject: [PATCH v6 05/11] tools/hvmloader: Retrieve (x2)APIC IDs from the APs themselves Date: Tue, 1 Oct 2024 13:38:01 +0100 Message-ID: <20241001123807.605-6-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Make it so the APs expose their own APIC IDs in a LUT. We can use that LUT to populate the MADT, decoupling the algorithm that relates CPU IDs and APIC IDs from hvmloader. While at this also remove ap_callin, as writing the APIC ID may serve the same purpose. Signed-off-by: Alejandro Vallejo --- tools/firmware/hvmloader/config.h | 5 ++- tools/firmware/hvmloader/hvmloader.c | 6 +-- tools/firmware/hvmloader/mp_tables.c | 4 +- tools/firmware/hvmloader/smp.c | 54 ++++++++++++++++++++----- tools/firmware/hvmloader/util.c | 2 +- tools/include/xen-tools/common-macros.h | 5 +++ 6 files changed, 60 insertions(+), 16 deletions(-) diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h index cd716bf39245..17666a1fdb01 100644 --- a/tools/firmware/hvmloader/config.h +++ b/tools/firmware/hvmloader/config.h @@ -4,6 +4,8 @@ #include #include +#include + enum virtual_vga { VGA_none, VGA_std, VGA_cirrus, VGA_pt }; extern enum virtual_vga virtual_vga; @@ -48,8 +50,9 @@ extern uint8_t ioapic_version; #define IOAPIC_ID 0x01 +extern uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS]; + #define LAPIC_BASE_ADDRESS 0xfee00000 -#define LAPIC_ID(vcpu_id) ((vcpu_id) * 2) #define PCI_ISA_DEVFN 0x08 /* dev 1, fn 0 */ #define PCI_ISA_IRQ_MASK 0x0c20U /* ISA IRQs 5,10,11 are PCI connected */ diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c index f8af88fabf24..0ff190ff4ec0 100644 --- a/tools/firmware/hvmloader/hvmloader.c +++ b/tools/firmware/hvmloader/hvmloader.c @@ -224,7 +224,7 @@ static void apic_setup(void) /* 8259A ExtInts are delivered through IOAPIC pin 0 (Virtual Wire Mode). */ ioapic_write(0x10, APIC_DM_EXTINT); - ioapic_write(0x11, SET_APIC_ID(LAPIC_ID(0))); + ioapic_write(0x11, SET_APIC_ID(CPU_TO_X2APICID[0])); } struct bios_info { @@ -341,11 +341,11 @@ int main(void) printf("CPU speed is %u MHz\n", get_cpu_mhz()); + smp_initialise(); + apic_setup(); pci_setup(); - smp_initialise(); - perform_tests(); if ( bios->bios_info_setup ) diff --git a/tools/firmware/hvmloader/mp_tables.c b/tools/firmware/hvmloader/mp_tables.c index 77d3010406d0..494f5bb3d813 100644 --- a/tools/firmware/hvmloader/mp_tables.c +++ b/tools/firmware/hvmloader/mp_tables.c @@ -198,8 +198,10 @@ static void fill_mp_config_table(struct mp_config_table *mpct, int length) /* fills in an MP processor entry for VCPU 'vcpu_id' */ static void fill_mp_proc_entry(struct mp_proc_entry *mppe, int vcpu_id) { + ASSERT(CPU_TO_X2APICID[vcpu_id] < 0xFF ); + mppe->type = ENTRY_TYPE_PROCESSOR; - mppe->lapic_id = LAPIC_ID(vcpu_id); + mppe->lapic_id = CPU_TO_X2APICID[vcpu_id]; mppe->lapic_version = 0x11; mppe->cpu_flags = CPU_FLAG_ENABLED; if ( vcpu_id == 0 ) diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c index 1b940cefd071..b0d4da111904 100644 --- a/tools/firmware/hvmloader/smp.c +++ b/tools/firmware/hvmloader/smp.c @@ -29,7 +29,34 @@ #include -static int ap_callin; +/** + * Lookup table of x2APIC IDs. + * + * Each entry is populated its respective CPU as they come online. This is required + * for generating the MADT with minimal assumptions about ID relationships. + */ +uint32_t CPU_TO_X2APICID[HVM_MAX_VCPUS]; + +/** Tristate about x2apic being supported. -1=unknown */ +static int has_x2apic = -1; + +static uint32_t read_apic_id(void) +{ + uint32_t apic_id; + + if ( has_x2apic ) + cpuid(0xb, NULL, NULL, NULL, &apic_id); + else + { + cpuid(1, NULL, &apic_id, NULL, NULL); + apic_id >>= 24; + } + + /* Never called by cpu0, so should never return 0 */ + ASSERT(apic_id); + + return apic_id; +} static void cpu_setup(unsigned int cpu) { @@ -37,13 +64,17 @@ static void cpu_setup(unsigned int cpu) cacheattr_init(); printf("done.\n"); - if ( !cpu ) /* Used on the BSP too */ + /* The BSP exits early because its APIC ID is known to be zero */ + if ( !cpu ) return; wmb(); - ap_callin = 1; + ACCESS_ONCE(CPU_TO_X2APICID[cpu]) = read_apic_id(); - /* After this point, the BSP will shut us down. */ + /* + * After this point the BSP will shut us down. A write to + * CPU_TO_X2APICID[cpu] signals the BSP to bring down `cpu`. + */ for ( ;; ) asm volatile ( "hlt" ); @@ -54,10 +85,6 @@ static void boot_cpu(unsigned int cpu) static uint8_t ap_stack[PAGE_SIZE] __attribute__ ((aligned (16))); static struct vcpu_hvm_context ap; - /* Initialise shared variables. */ - ap_callin = 0; - wmb(); - /* Wake up the secondary processor */ ap = (struct vcpu_hvm_context) { .mode = VCPU_HVM_MODE_32B, @@ -90,10 +117,11 @@ static void boot_cpu(unsigned int cpu) BUG(); /* - * Wait for the secondary processor to complete initialisation. + * Wait for the secondary processor to complete initialisation, + * which is signaled by its x2APIC ID being written to the LUT. * Do not touch shared resources meanwhile. */ - while ( !ap_callin ) + while ( !ACCESS_ONCE(CPU_TO_X2APICID[cpu]) ) cpu_relax(); /* Take the secondary processor offline. */ @@ -104,6 +132,12 @@ static void boot_cpu(unsigned int cpu) void smp_initialise(void) { unsigned int i, nr_cpus = hvm_info->nr_vcpus; + uint32_t ecx; + + cpuid(1, NULL, NULL, &ecx, NULL); + has_x2apic = (ecx >> 21) & 1; + if ( has_x2apic ) + printf("x2APIC supported\n"); printf("Multiprocessor initialisation:\n"); cpu_setup(0); diff --git a/tools/firmware/hvmloader/util.c b/tools/firmware/hvmloader/util.c index d3b3f9038e64..7e1e105d79dd 100644 --- a/tools/firmware/hvmloader/util.c +++ b/tools/firmware/hvmloader/util.c @@ -827,7 +827,7 @@ static void acpi_mem_free(struct acpi_ctxt *ctxt, static uint32_t acpi_lapic_id(unsigned cpu) { - return LAPIC_ID(cpu); + return CPU_TO_X2APIC_ID[cpu]; } void hvmloader_acpi_build_tables(struct acpi_config *config, diff --git a/tools/include/xen-tools/common-macros.h b/tools/include/xen-tools/common-macros.h index 60912225cb7a..336c6309d96e 100644 --- a/tools/include/xen-tools/common-macros.h +++ b/tools/include/xen-tools/common-macros.h @@ -108,4 +108,9 @@ #define get_unaligned(ptr) get_unaligned_t(typeof(*(ptr)), ptr) #define put_unaligned(val, ptr) put_unaligned_t(typeof(*(ptr)), val, ptr) +#define __ACCESS_ONCE(x) ({ \ + (void)(typeof(x))0; /* Scalar typecheck. */ \ + (volatile typeof(x) *)&(x); }) +#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x)) + #endif /* __XEN_TOOLS_COMMON_MACROS__ */ From patchwork Tue Oct 1 12:38:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 05D30CF64BE for ; Tue, 1 Oct 2024 12:38:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808072.1219917 (Exim 4.92) (envelope-from ) id 1svc9I-0004Sl-Uv; Tue, 01 Oct 2024 12:38:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808072.1219917; Tue, 01 Oct 2024 12:38:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9I-0004Sa-Qg; Tue, 01 Oct 2024 12:38:40 +0000 Received: by outflank-mailman (input) for mailman id 808072; Tue, 01 Oct 2024 12:38:37 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9F-0002va-LL for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:37 +0000 Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com [2a00:1450:4864:20::232]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 13ab85fd-7ff2-11ef-99a2-01e77a169b0f; Tue, 01 Oct 2024 14:38:36 +0200 (CEST) Received: by mail-lj1-x232.google.com with SMTP id 38308e7fff4ca-2fad100dd9eso21654871fa.3 for ; Tue, 01 Oct 2024 05:38:36 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:34 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 13ab85fd-7ff2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786315; x=1728391115; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9yI0HSpcntLJIYW3VaOO9aCgh2JBlz3FO3H/5MUH+To=; b=RV9+Jl1CwV5JzEnT1G2R8/wf0h5dCEfOrP23lrcAV/KD3/oWmImePzp6UBlaTXYd44 6aHuIi6wSUqkzDCOOLQq0axEwsarPe3auS8mg96acvWo86dj6Z9A3fpeMICQG+S1Ycm9 pVWa9NWHBExH7fPG51ub/+xUwEhe9pIEbAutA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786315; x=1728391115; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9yI0HSpcntLJIYW3VaOO9aCgh2JBlz3FO3H/5MUH+To=; b=BqGVgZlrzyhPROVAJqDsV1eX3YAHTb/iAvgqpMZicSGg8ZDMXOeEWH1dZxODDvF5Rv /POfoaoMDmx4pqqNO/H3VMNEx+kgCynHwiLmJfxaig09lKCsEEqTlL9553TW+LyHhPmf 093EuwuUK8gh8UyPoc8t62kZgEzbk+H5uI1BycY50tbSXILcNH48HyRfYPNQ01ig8YyR NrKrjmuMgrvnMMnShRClXafG/KCqSwktGaOj7aD3uckytglaPpibUMDEDGdMaBFLOceB vTS+ZlqeBEe5tuENjNkUUDGnOZXYwlqjSiq8FtJqUE5oS9c+VJ+RZ3yBhcTh8u6C0ame hXYg== X-Gm-Message-State: AOJu0YzUtWnGzJbPJ7u471q/8dpfzjlntU0AvxzWG2n73qDuyz2E37yx csnpjgPBkIzHxRl+mVKhTk/ZBkXwNMieIVif89JKiWX/NphSfDX0c/VQLNZCgvJrlG57ojnHPEl tvY4= X-Google-Smtp-Source: AGHT+IGswhbqCcBlXpdoQJzmjI32zdk/Fk5jlyLkCAjcdzHhOpmY1QgcPMuevLyWV+yJ/44rfe98Vw== X-Received: by 2002:a2e:be0d:0:b0:2fa:c49b:d14f with SMTP id 38308e7fff4ca-2fac49bd8f2mr50165401fa.2.1727786315182; Tue, 01 Oct 2024 05:38:35 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Anthony PERARD , Juergen Gross Subject: [PATCH v6 06/11] tools/libacpi: Use LUT of APIC IDs rather than function pointer Date: Tue, 1 Oct 2024 13:38:02 +0100 Message-ID: <20241001123807.605-7-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Refactors libacpi so that a single LUT is the authoritative source of truth for the CPU to APIC ID mappings. This has a know-on effect in reducing complexity on future patches, as the same LUT can be used for configuring the APICs and configuring the ACPI tables for PVH. Not functional change intended, because the same mappings are preserved. Signed-off-by: Alejandro Vallejo --- tools/firmware/hvmloader/util.c | 7 +------ tools/include/xenguest.h | 5 +++++ tools/libacpi/build.c | 6 +++--- tools/libacpi/libacpi.h | 2 +- tools/libs/light/libxl_dom.c | 5 +++++ tools/libs/light/libxl_x86_acpi.c | 7 +------ 6 files changed, 16 insertions(+), 16 deletions(-) diff --git a/tools/firmware/hvmloader/util.c b/tools/firmware/hvmloader/util.c index 7e1e105d79dd..4a6303bbae8c 100644 --- a/tools/firmware/hvmloader/util.c +++ b/tools/firmware/hvmloader/util.c @@ -825,11 +825,6 @@ static void acpi_mem_free(struct acpi_ctxt *ctxt, /* ACPI builder currently doesn't free memory so this is just a stub */ } -static uint32_t acpi_lapic_id(unsigned cpu) -{ - return CPU_TO_X2APIC_ID[cpu]; -} - void hvmloader_acpi_build_tables(struct acpi_config *config, unsigned int physical) { @@ -859,7 +854,7 @@ void hvmloader_acpi_build_tables(struct acpi_config *config, } config->lapic_base_address = LAPIC_BASE_ADDRESS; - config->lapic_id = acpi_lapic_id; + config->cpu_to_apicid = CPU_TO_X2APICID; config->ioapic_base_address = IOAPIC_BASE_ADDRESS; config->ioapic_id = IOAPIC_ID; config->pci_isa_irq_mask = PCI_ISA_IRQ_MASK; diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h index e01f494b772a..aa50b78dfb89 100644 --- a/tools/include/xenguest.h +++ b/tools/include/xenguest.h @@ -22,6 +22,8 @@ #ifndef XENGUEST_H #define XENGUEST_H +#include "xen/hvm/hvm_info_table.h" + #define XC_NUMA_NO_NODE (~0U) #define XCFLAGS_LIVE (1 << 0) @@ -236,6 +238,9 @@ struct xc_dom_image { #if defined(__i386__) || defined(__x86_64__) struct e820entry *e820; unsigned int e820_entries; + + /* LUT mapping cpu id to (x2)APIC ID */ + uint32_t cpu_to_apicid[HVM_MAX_VCPUS]; #endif xen_pfn_t vuart_gfn; diff --git a/tools/libacpi/build.c b/tools/libacpi/build.c index 2f29863db154..2ad1d461a2ec 100644 --- a/tools/libacpi/build.c +++ b/tools/libacpi/build.c @@ -74,7 +74,7 @@ static struct acpi_20_madt *construct_madt(struct acpi_ctxt *ctxt, const struct hvm_info_table *hvminfo = config->hvminfo; int i, sz; - if ( config->lapic_id == NULL ) + if ( config->cpu_to_apicid == NULL ) return NULL; sz = sizeof(struct acpi_20_madt); @@ -148,7 +148,7 @@ static struct acpi_20_madt *construct_madt(struct acpi_ctxt *ctxt, lapic->length = sizeof(*lapic); /* Processor ID must match processor-object IDs in the DSDT. */ lapic->acpi_processor_id = i; - lapic->apic_id = config->lapic_id(i); + lapic->apic_id = config->cpu_to_apicid[i]; lapic->flags = (test_bit(i, hvminfo->vcpu_online) ? ACPI_LOCAL_APIC_ENABLED : 0); lapic++; @@ -236,7 +236,7 @@ static struct acpi_20_srat *construct_srat(struct acpi_ctxt *ctxt, processor->type = ACPI_PROCESSOR_AFFINITY; processor->length = sizeof(*processor); processor->domain = config->numa.vcpu_to_vnode[i]; - processor->apic_id = config->lapic_id(i); + processor->apic_id = config->cpu_to_apicid[i]; processor->flags = ACPI_LOCAL_APIC_AFFIN_ENABLED; processor++; } diff --git a/tools/libacpi/libacpi.h b/tools/libacpi/libacpi.h index deda39e5dbc4..8b010212448c 100644 --- a/tools/libacpi/libacpi.h +++ b/tools/libacpi/libacpi.h @@ -84,7 +84,7 @@ struct acpi_config { unsigned long rsdp; /* x86-specific parameters */ - uint32_t (*lapic_id)(unsigned cpu); + uint32_t *cpu_to_apicid; /* LUT mapping cpu id to (x2)APIC ID */ uint32_t lapic_base_address; uint32_t ioapic_base_address; uint16_t pci_isa_irq_mask; diff --git a/tools/libs/light/libxl_dom.c b/tools/libs/light/libxl_dom.c index 94fef374014e..7f9c6eaa8b24 100644 --- a/tools/libs/light/libxl_dom.c +++ b/tools/libs/light/libxl_dom.c @@ -1082,6 +1082,11 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid, dom->container_type = XC_DOM_HVM_CONTAINER; +#if defined(__i386__) || defined(__x86_64__) + for ( uint32_t i = 0; i < info->max_vcpus; i++ ) + dom->cpu_to_apicid[i] = 2 * i; /* TODO: Replace by topo calculation */ +#endif + /* The params from the configuration file are in Mb, which are then * multiplied by 1 Kb. This was then divided off when calling * the old xc_hvm_build_target_mem() which then turned them to bytes. diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x86_acpi.c index 5cf261bd6794..585d4c8755cb 100644 --- a/tools/libs/light/libxl_x86_acpi.c +++ b/tools/libs/light/libxl_x86_acpi.c @@ -75,11 +75,6 @@ static void acpi_mem_free(struct acpi_ctxt *ctxt, { } -static uint32_t acpi_lapic_id(unsigned cpu) -{ - return cpu * 2; -} - static int init_acpi_config(libxl__gc *gc, struct xc_dom_image *dom, const libxl_domain_build_info *b_info, @@ -144,7 +139,7 @@ static int init_acpi_config(libxl__gc *gc, config->hvminfo = hvminfo; config->lapic_base_address = LAPIC_BASE_ADDRESS; - config->lapic_id = acpi_lapic_id; + config->cpu_to_apicid = dom->cpu_to_apicid; config->acpi_revision = 5; rc = 0; From patchwork Tue Oct 1 12:38:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817921 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A60DCE7D12 for ; Tue, 1 Oct 2024 12:38:48 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808073.1219922 (Exim 4.92) (envelope-from ) id 1svc9J-0004WF-BC; Tue, 01 Oct 2024 12:38:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808073.1219922; Tue, 01 Oct 2024 12:38:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9J-0004V1-4f; Tue, 01 Oct 2024 12:38:41 +0000 Received: by outflank-mailman (input) for mailman id 808073; Tue, 01 Oct 2024 12:38:38 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9G-0002va-B5 for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:38 +0000 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [2a00:1450:4864:20::534]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 140ead2b-7ff2-11ef-99a2-01e77a169b0f; Tue, 01 Oct 2024 14:38:36 +0200 (CEST) Received: by mail-ed1-x534.google.com with SMTP id 4fb4d7f45d1cf-5c89f3f28b6so1950566a12.2 for ; Tue, 01 Oct 2024 05:38:36 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:35 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 140ead2b-7ff2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786316; x=1728391116; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YuObzAPBZ9nf14Sh2mL9KT1PN35RQIsHwQXw8tfgbuc=; b=PEukFMJAh4ga/5xZMwdG5tBito/abC3MmAc7XEKC8ymZE6meQz9CIZ7+SiDHPfV9Yg mJNwszAzZtcGHtegt/QMb73APYZ8xXPoXfDiS2GMRIhvHpg1gRti/GBhskeGGZ/GcRJk 8ghWpfZYTvUGo4j5oGDlDwXKm5wGvOqds/koc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786316; x=1728391116; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YuObzAPBZ9nf14Sh2mL9KT1PN35RQIsHwQXw8tfgbuc=; b=u+dbNmyZApopP81cLL6SGYi2REWfuBtBOuHY+BJJkQ9HV53ZwZSMW71xJ6zyr9Xrqq o2WU/YbGkr72e7nmi91kDlgh9YBdjQZcZcES1Hw9liQ5Vcg3HF23zwz3DYn9tLj6/GHl Xhu5PaQFJqWMtGWX8W3QX21fI6wFRNS9FAnfCOVQ30r2aKeUQvlFBESQ9aPKKR8IJuyO /UVxHHSSaAzEbYhR4Hzl5HKhTPBjZQqWsdMm/xhgzDqtnXscOQwFNOUFJLf6KKEtNwDG SHBKANQg15xjtS8vhaf4xoLf1EHC7g1Eba1RqHCCGxfT62ZtIrOo6jZMqMjmdIeDt+1p zI0A== X-Gm-Message-State: AOJu0YzoCzxlfJ3+DeniS8ptb0sTh7eH7PH010RlaNlhEKyyz/5yqyTm T/XRk0jbKJFvmHrR+PNXIv6ZDLYtLvOS0RXnd4ZUJ2Wxc4aa7NYbzPte5+QIhwdkcEsqSE8FkJw IVrM= X-Google-Smtp-Source: AGHT+IG4jKYDXno4LWgH1e+P0gCG5KDXkdl4Ia0ars1LxyfAinjMa0gyexBSI2sdIQS2YMCCSAaEcg== X-Received: by 2002:a05:6402:27c6:b0:5c7:1911:f134 with SMTP id 4fb4d7f45d1cf-5c8824e51camr13023489a12.9.1727786315912; Tue, 01 Oct 2024 05:38:35 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Anthony PERARD , Juergen Gross Subject: [PATCH v6 07/11] tools/libguest: Always set vCPU context in vcpu_hvm() Date: Tue, 1 Oct 2024 13:38:03 +0100 Message-ID: <20241001123807.605-8-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Currently used by PVH to set MTRR, will be used by a later patch to set APIC state. Unconditionally send the hypercall, and gate overriding the MTRR so it remains functionally equivalent. While at it, add a missing "goto out" to what was the error condition in the loop. In principle this patch shouldn't affect functionality. An extra record (the MTRR) is sent to the hypervisor per vCPU on HVM, but these records are identical to those retrieved in the first place so there's no expected functional change. Signed-off-by: Alejandro Vallejo --- tools/libs/guest/xg_dom_x86.c | 84 ++++++++++++++++++----------------- 1 file changed, 44 insertions(+), 40 deletions(-) diff --git a/tools/libs/guest/xg_dom_x86.c b/tools/libs/guest/xg_dom_x86.c index cba01384ae75..c98229317db7 100644 --- a/tools/libs/guest/xg_dom_x86.c +++ b/tools/libs/guest/xg_dom_x86.c @@ -989,6 +989,7 @@ const static void *hvm_get_save_record(const void *ctx, unsigned int type, static int vcpu_hvm(struct xc_dom_image *dom) { + /* Initialises the BSP */ struct { struct hvm_save_descriptor header_d; HVM_SAVE_TYPE(HEADER) header; @@ -997,6 +998,18 @@ static int vcpu_hvm(struct xc_dom_image *dom) struct hvm_save_descriptor end_d; HVM_SAVE_TYPE(END) end; } bsp_ctx; + /* Initialises APICs and MTRRs of every vCPU */ + struct { + struct hvm_save_descriptor header_d; + HVM_SAVE_TYPE(HEADER) header; + struct hvm_save_descriptor mtrr_d; + HVM_SAVE_TYPE(MTRR) mtrr; + struct hvm_save_descriptor end_d; + HVM_SAVE_TYPE(END) end; + } vcpu_ctx; + /* Context from full_ctx */ + const HVM_SAVE_TYPE(MTRR) *mtrr_record; + /* Raw context as taken from Xen */ uint8_t *full_ctx = NULL; int rc; @@ -1083,51 +1096,42 @@ static int vcpu_hvm(struct xc_dom_image *dom) bsp_ctx.end_d.instance = 0; bsp_ctx.end_d.length = HVM_SAVE_LENGTH(END); - /* TODO: maybe this should be a firmware option instead? */ - if ( !dom->device_model ) + /* TODO: maybe setting MTRRs should be a firmware option instead? */ + mtrr_record = hvm_get_save_record(full_ctx, HVM_SAVE_CODE(MTRR), 0); + + if ( !mtrr_record) { - struct { - struct hvm_save_descriptor header_d; - HVM_SAVE_TYPE(HEADER) header; - struct hvm_save_descriptor mtrr_d; - HVM_SAVE_TYPE(MTRR) mtrr; - struct hvm_save_descriptor end_d; - HVM_SAVE_TYPE(END) end; - } mtrr = { - .header_d = bsp_ctx.header_d, - .header = bsp_ctx.header, - .mtrr_d.typecode = HVM_SAVE_CODE(MTRR), - .mtrr_d.length = HVM_SAVE_LENGTH(MTRR), - .end_d = bsp_ctx.end_d, - .end = bsp_ctx.end, - }; - const HVM_SAVE_TYPE(MTRR) *mtrr_record = - hvm_get_save_record(full_ctx, HVM_SAVE_CODE(MTRR), 0); - unsigned int i; - - if ( !mtrr_record ) - { - xc_dom_panic(dom->xch, XC_INTERNAL_ERROR, - "%s: unable to get MTRR save record", __func__); - goto out; - } + xc_dom_panic(dom->xch, XC_INTERNAL_ERROR, + "%s: unable to get MTRR save record", __func__); + goto out; + } - memcpy(&mtrr.mtrr, mtrr_record, sizeof(mtrr.mtrr)); + vcpu_ctx.header_d = bsp_ctx.header_d; + vcpu_ctx.header = bsp_ctx.header; + vcpu_ctx.mtrr_d.typecode = HVM_SAVE_CODE(MTRR); + vcpu_ctx.mtrr_d.length = HVM_SAVE_LENGTH(MTRR); + vcpu_ctx.mtrr = *mtrr_record; + vcpu_ctx.end_d = bsp_ctx.end_d; + vcpu_ctx.end = bsp_ctx.end; - /* - * Enable MTRR, set default type to WB. - * TODO: add MMIO areas as UC when passthrough is supported. - */ - mtrr.mtrr.msr_mtrr_def_type = MTRR_TYPE_WRBACK | MTRR_DEF_TYPE_ENABLE; + /* + * Enable MTRR, set default type to WB. + * TODO: add MMIO areas as UC when passthrough is supported in PVH + */ + if ( !dom->device_model ) + vcpu_ctx.mtrr.msr_mtrr_def_type = MTRR_TYPE_WRBACK | MTRR_DEF_TYPE_ENABLE; + + for ( unsigned int i = 0; i < dom->max_vcpus; i++ ) + { + vcpu_ctx.mtrr_d.instance = i; - for ( i = 0; i < dom->max_vcpus; i++ ) + rc = xc_domain_hvm_setcontext(dom->xch, dom->guest_domid, + (uint8_t *)&vcpu_ctx, sizeof(vcpu_ctx)); + if ( rc != 0 ) { - mtrr.mtrr_d.instance = i; - rc = xc_domain_hvm_setcontext(dom->xch, dom->guest_domid, - (uint8_t *)&mtrr, sizeof(mtrr)); - if ( rc != 0 ) - xc_dom_panic(dom->xch, XC_INTERNAL_ERROR, - "%s: SETHVMCONTEXT failed (rc=%d)", __func__, rc); + xc_dom_panic(dom->xch, XC_INTERNAL_ERROR, + "%s: SETHVMCONTEXT failed (rc=%d)", __func__, rc); + goto out; } } From patchwork Tue Oct 1 12:38:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817928 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CC90ECE7D19 for ; Tue, 1 Oct 2024 12:38:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808074.1219928 (Exim 4.92) (envelope-from ) id 1svc9J-0004c5-Ty; Tue, 01 Oct 2024 12:38:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808074.1219928; Tue, 01 Oct 2024 12:38:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9J-0004Yf-Ek; Tue, 01 Oct 2024 12:38:41 +0000 Received: by outflank-mailman (input) for mailman id 808074; Tue, 01 Oct 2024 12:38:38 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9G-0002vk-7t for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:38 +0000 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [2a00:1450:4864:20::536]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 14728a14-7ff2-11ef-a0ba-8be0dac302b0; Tue, 01 Oct 2024 14:38:37 +0200 (CEST) Received: by mail-ed1-x536.google.com with SMTP id 4fb4d7f45d1cf-5c89bdb9019so2448858a12.0 for ; Tue, 01 Oct 2024 05:38:37 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:36 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 14728a14-7ff2-11ef-a0ba-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786316; x=1728391116; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=I0GTGzwtafoymMbnFaFTF4qE2IIeqafzC6RojYeuQKU=; b=AwiOS6qOydFTRHg9bXtAWrwaEWPgF+IjbmwKHWbkEXaUYTo9grNcGEaJJYEJDmD1EB 0tx6iKHiUn6cipPyuCetJn4aYk1st9mIVIWhWqXP/O9a10k7l7kxMM25ZDdwpBJxnmD2 BE02nwmLi488m2BaEOeYXx1x8pQONGklPUVPY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786316; x=1728391116; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I0GTGzwtafoymMbnFaFTF4qE2IIeqafzC6RojYeuQKU=; b=YgIHWBCDSsH2vq2LW3q+nsRY7nYbBx/uD47/6dG8KkjrDvg4QmuPSEvRQ10ezVqpok k+Eq0TsPXJXd7xyim4iSUdV4SLACPAW1Jy38nHQ9T2yZJfGdZjg4nTGIJeBCK4pJhmXB Zmt/cT9dX8siidW8pVyENlOWsiOLftDrJluDdEQnYCJyQryJ/VKx6OnfQCVkxi87d/aQ mlrywDIXcVkHnIHhc+0gMQbLPcuUulsH4JlQEgEBlQZ8hwZcKLNE2wEXWWqK7q6oU9RH oeeePA3WadzEPcHK8UrkfrrazOSE/PSJAUO9teYWuWZgomrDQ9j3tVkyATQ7ouLIL2TO 9aEw== X-Gm-Message-State: AOJu0YyZ7p31FyhP881H87HjulOqZdD6E1AteK7crd/qTO2Bar+sUjHT MUKmvmhz717rXpGGg2pooPktFENs0MbYKcCyCBLfZfb1FWiFgqWFOtEd0IjPusD2OShmaw3gEcm fTeY= X-Google-Smtp-Source: AGHT+IGWrWoMczYC8L5QGRHinswuT9xtxfIlrvUWsBpMhVKDaFNgWGraiocX4r5UXEHrGxfyPE6g9A== X-Received: by 2002:a50:951a:0:b0:5c8:81bd:ac90 with SMTP id 4fb4d7f45d1cf-5c8825fd826mr11229511a12.27.1727786316561; Tue, 01 Oct 2024 05:38:36 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Anthony PERARD Subject: [PATCH v6 08/11] xen/lib: Add topology generator for x86 Date: Tue, 1 Oct 2024 13:38:04 +0100 Message-ID: <20241001123807.605-9-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Add a helper to populate topology leaves in the cpu policy from threads/core and cores/package counts. It's unit-tested in test-cpu-policy.c, but it's not connected to the rest of the code yet. Adds the ASSERT() macro to xen/lib/x86/private.h, as it was missing. Signed-off-by: Alejandro Vallejo --- tools/tests/cpu-policy/test-cpu-policy.c | 133 +++++++++++++++++++++++ xen/include/xen/lib/x86/cpu-policy.h | 16 +++ xen/lib/x86/policy.c | 88 +++++++++++++++ xen/lib/x86/private.h | 4 + 4 files changed, 241 insertions(+) diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c index 9216010b1c5d..870f7ecee0e5 100644 --- a/tools/tests/cpu-policy/test-cpu-policy.c +++ b/tools/tests/cpu-policy/test-cpu-policy.c @@ -654,6 +654,137 @@ static void test_is_compatible_failure(void) } } +static void test_topo_from_parts(void) +{ + static const struct test { + unsigned int threads_per_core; + unsigned int cores_per_pkg; + struct cpu_policy policy; + } tests[] = { + { + .threads_per_core = 3, .cores_per_pkg = 1, + .policy = { + .x86_vendor = X86_VENDOR_AMD, + .topo.subleaf = { + { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, }, + { .nr_logical = 1, .level = 1, .type = 2, .id_shift = 2, }, + }, + }, + }, + { + .threads_per_core = 1, .cores_per_pkg = 3, + .policy = { + .x86_vendor = X86_VENDOR_AMD, + .topo.subleaf = { + { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, }, + { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, }, + }, + }, + }, + { + .threads_per_core = 7, .cores_per_pkg = 5, + .policy = { + .x86_vendor = X86_VENDOR_AMD, + .topo.subleaf = { + { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, }, + { .nr_logical = 5, .level = 1, .type = 2, .id_shift = 6, }, + }, + }, + }, + { + .threads_per_core = 2, .cores_per_pkg = 128, + .policy = { + .x86_vendor = X86_VENDOR_AMD, + .topo.subleaf = { + { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, }, + { .nr_logical = 128, .level = 1, .type = 2, + .id_shift = 8, }, + }, + }, + }, + { + .threads_per_core = 3, .cores_per_pkg = 1, + .policy = { + .x86_vendor = X86_VENDOR_INTEL, + .topo.subleaf = { + { .nr_logical = 3, .level = 0, .type = 1, .id_shift = 2, }, + { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, }, + }, + }, + }, + { + .threads_per_core = 1, .cores_per_pkg = 3, + .policy = { + .x86_vendor = X86_VENDOR_INTEL, + .topo.subleaf = { + { .nr_logical = 1, .level = 0, .type = 1, .id_shift = 0, }, + { .nr_logical = 3, .level = 1, .type = 2, .id_shift = 2, }, + }, + }, + }, + { + .threads_per_core = 7, .cores_per_pkg = 5, + .policy = { + .x86_vendor = X86_VENDOR_INTEL, + .topo.subleaf = { + { .nr_logical = 7, .level = 0, .type = 1, .id_shift = 3, }, + { .nr_logical = 35, .level = 1, .type = 2, .id_shift = 6, }, + }, + }, + }, + { + .threads_per_core = 2, .cores_per_pkg = 128, + .policy = { + .x86_vendor = X86_VENDOR_INTEL, + .topo.subleaf = { + { .nr_logical = 2, .level = 0, .type = 1, .id_shift = 1, }, + { .nr_logical = 256, .level = 1, .type = 2, + .id_shift = 8, }, + }, + }, + }, + }; + + printf("Testing topology synthesis from parts:\n"); + + for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i ) + { + const struct test *t = &tests[i]; + struct cpu_policy actual = { .x86_vendor = t->policy.x86_vendor }; + int rc = x86_topo_from_parts(&actual, t->threads_per_core, + t->cores_per_pkg); + + if ( rc || memcmp(&actual.topo, &t->policy.topo, sizeof(actual.topo)) ) + { +#define TOPO(n, f) t->policy.topo.subleaf[(n)].f, actual.topo.subleaf[(n)].f + fail("FAIL[%d] - '%s %u t/c, %u c/p'\n", + rc, + x86_cpuid_vendor_to_str(t->policy.x86_vendor), + t->threads_per_core, t->cores_per_pkg); + printf(" subleaf=%u expected_n=%u actual_n=%u\n" + " expected_lvl=%u actual_lvl=%u\n" + " expected_type=%u actual_type=%u\n" + " expected_shift=%u actual_shift=%u\n", + 0, + TOPO(0, nr_logical), + TOPO(0, level), + TOPO(0, type), + TOPO(0, id_shift)); + + printf(" subleaf=%u expected_n=%u actual_n=%u\n" + " expected_lvl=%u actual_lvl=%u\n" + " expected_type=%u actual_type=%u\n" + " expected_shift=%u actual_shift=%u\n", + 1, + TOPO(1, nr_logical), + TOPO(1, level), + TOPO(1, type), + TOPO(1, id_shift)); +#undef TOPO + } + } +} + int main(int argc, char **argv) { printf("CPU Policy unit tests\n"); @@ -671,6 +802,8 @@ int main(int argc, char **argv) test_is_compatible_success(); test_is_compatible_failure(); + test_topo_from_parts(); + if ( nr_failures ) printf("Done: %u failures\n", nr_failures); else diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h index f43e1a3b21e9..116b305a1d7f 100644 --- a/xen/include/xen/lib/x86/cpu-policy.h +++ b/xen/include/xen/lib/x86/cpu-policy.h @@ -542,6 +542,22 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host, const struct cpu_policy *guest, struct cpu_policy_errors *err); +/** + * Synthesise topology information in `p` given high-level constraints + * + * Topology is given in various fields accross several leaves, some of + * which are vendor-specific. This function uses the policy itself to + * derive such leaves from threads/core and cores/package. + * + * @param p CPU policy of the domain. + * @param threads_per_core threads/core. Doesn't need to be a power of 2. + * @param cores_per_package cores/package. Doesn't need to be a power of 2. + * @return 0 on success; -errno on failure + */ +int x86_topo_from_parts(struct cpu_policy *p, + unsigned int threads_per_core, + unsigned int cores_per_pkg); + #endif /* !XEN_LIB_X86_POLICIES_H */ /* diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c index 63bc96451d2c..16b09a427841 100644 --- a/xen/lib/x86/policy.c +++ b/xen/lib/x86/policy.c @@ -2,6 +2,94 @@ #include +static unsigned int order(unsigned int n) +{ + ASSERT(n); /* clz(0) is UB */ + + return 8 * sizeof(n) - __builtin_clz(n); +} + +int x86_topo_from_parts(struct cpu_policy *p, + unsigned int threads_per_core, + unsigned int cores_per_pkg) +{ + unsigned int threads_per_pkg = threads_per_core * cores_per_pkg; + unsigned int apic_id_size; + + if ( !p || !threads_per_core || !cores_per_pkg ) + return -EINVAL; + + p->basic.max_leaf = MAX(0xb, p->basic.max_leaf); + + memset(p->topo.raw, 0, sizeof(p->topo.raw)); + + /* thread level */ + p->topo.subleaf[0].nr_logical = threads_per_core; + p->topo.subleaf[0].id_shift = 0; + p->topo.subleaf[0].level = 0; + p->topo.subleaf[0].type = 1; + if ( threads_per_core > 1 ) + p->topo.subleaf[0].id_shift = order(threads_per_core - 1); + + /* core level */ + p->topo.subleaf[1].nr_logical = cores_per_pkg; + if ( p->x86_vendor == X86_VENDOR_INTEL ) + p->topo.subleaf[1].nr_logical = threads_per_pkg; + p->topo.subleaf[1].id_shift = p->topo.subleaf[0].id_shift; + p->topo.subleaf[1].level = 1; + p->topo.subleaf[1].type = 2; + if ( cores_per_pkg > 1 ) + p->topo.subleaf[1].id_shift += order(cores_per_pkg - 1); + + apic_id_size = p->topo.subleaf[1].id_shift; + + /* + * Contrary to what the name might seem to imply. HTT is an enabler for + * SMP and there's no harm in setting it even with a single vCPU. + */ + p->basic.htt = true; + p->basic.lppp = MIN(0xff, threads_per_pkg); + + switch ( p->x86_vendor ) + { + case X86_VENDOR_INTEL: { + struct cpuid_cache_leaf *sl = p->cache.subleaf; + + for ( size_t i = 0; sl->type && + i < ARRAY_SIZE(p->cache.raw); i++, sl++ ) + { + sl->cores_per_package = cores_per_pkg - 1; + sl->threads_per_cache = threads_per_core - 1; + if ( sl->type == 3 /* unified cache */ ) + sl->threads_per_cache = threads_per_pkg - 1; + } + break; + } + + case X86_VENDOR_AMD: + case X86_VENDOR_HYGON: + /* Expose p->basic.lppp */ + p->extd.cmp_legacy = true; + + /* Clip NC to the maximum value it can hold */ + p->extd.nc = MIN(0xff, threads_per_pkg - 1); + + /* TODO: Expose leaf e1E */ + p->extd.topoext = false; + + /* + * Clip APIC ID to 8 bits, as that's what high core-count machines do. + * + * That's what AMD EPYC 9654 does with >256 CPUs. + */ + p->extd.apic_id_size = MIN(8, apic_id_size); + + break; + } + + return 0; +} + int x86_cpu_policies_are_compatible(const struct cpu_policy *host, const struct cpu_policy *guest, struct cpu_policy_errors *err) diff --git a/xen/lib/x86/private.h b/xen/lib/x86/private.h index 60bb82a400b7..2ec9dbee33c2 100644 --- a/xen/lib/x86/private.h +++ b/xen/lib/x86/private.h @@ -4,6 +4,7 @@ #ifdef __XEN__ #include +#include #include #include #include @@ -17,6 +18,7 @@ #else +#include #include #include #include @@ -28,6 +30,8 @@ #include +#define ASSERT(x) assert(x) + static inline bool test_bit(unsigned int bit, const void *vaddr) { const char *addr = vaddr; From patchwork Tue Oct 1 12:38:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817927 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AAF4FCE7D12 for ; Tue, 1 Oct 2024 12:38:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808075.1219935 (Exim 4.92) (envelope-from ) id 1svc9K-0004ip-Bz; Tue, 01 Oct 2024 12:38:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808075.1219935; Tue, 01 Oct 2024 12:38:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9K-0004fx-0D; Tue, 01 Oct 2024 12:38:42 +0000 Received: by outflank-mailman (input) for mailman id 808075; Tue, 01 Oct 2024 12:38:40 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9I-0002va-5g for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:40 +0000 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [2a00:1450:4864:20::531]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 151b346c-7ff2-11ef-99a2-01e77a169b0f; Tue, 01 Oct 2024 14:38:38 +0200 (CEST) Received: by mail-ed1-x531.google.com with SMTP id 4fb4d7f45d1cf-5c88370ad7bso5521583a12.3 for ; Tue, 01 Oct 2024 05:38:38 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:36 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 151b346c-7ff2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786318; x=1728391118; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=H8EtNk4UqEXSM3dt95fWtQHR5BEwc5KpYhDQ/7z9ZIg=; b=goCwjj2dpIOVLsDW/6sF3h7MStEiElPR8T7ZoSYDURY6h8XAEzAq6aHSB4KYggVUyS yDhMyyP4ICusImwxclGhYCJ6DPskEc0VnwQzxZxpo6Yt2yDDainjHSHR5PYhrmxqHH80 3cVVqsc4iv6ZrSlGM2jxjNbrYLELGdz8Szt44= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786318; x=1728391118; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H8EtNk4UqEXSM3dt95fWtQHR5BEwc5KpYhDQ/7z9ZIg=; b=XN3DiPZ36q7UdDjy/S39tAG4YW8vPxHUzgU28wH7HAvim8fy71f0BjmvqKa0zcyNLf VW92inYJoUijoP2Qwz+ZUiHqwFxaWrIC3PqFoujIFlnWESfvqWlBJ1U3rki1TiGa6XCq gH/dRHvpIw0VdvQrROw9iONiNXLZE+rtfu6CSGE2Mb832SIjxHVtneiHb9Om32/byIzl mhWdJS4m+RnI7TgpfsoYTRF4yEkySj1RuxZah3MhkkjyI80U0mDHxyAJ0PtYFAT54Hzo b5P34n37HQL4w+atCY4kEV1PiRT8EMGMljKzf23hB6p1Zuk8NGML3gPhxIsBota8aJIs R4+A== X-Gm-Message-State: AOJu0Yxi7s+tHuJO9QbHaCZfvX8zFQ3pKqvYH2Km8+FYda2/CS5G4FG/ ZU/r3EQGKWbNXh0W9NmaHBOuvgGTfKFCNOx71yx+uJoLjnnoGHbmY9vdaawJqhO1ZUA4lRlz0zP cbiI= X-Google-Smtp-Source: AGHT+IFMXDOf/Otw2ehiWab//1ldD17rCzZeR9uZJz+L6BvKjmRUSVvVl6d+NaZy3/Zdw6iDz2ysIA== X-Received: by 2002:a05:6402:518b:b0:5c8:957a:b1e5 with SMTP id 4fb4d7f45d1cf-5c8957ab4fdmr6601697a12.16.1727786317202; Tue, 01 Oct 2024 05:38:37 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Anthony PERARD Subject: [PATCH v6 09/11] xen/x86: Derive topologically correct x2APIC IDs from the policy Date: Tue, 1 Oct 2024 13:38:05 +0100 Message-ID: <20241001123807.605-10-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Implements the helper for mapping vcpu_id to x2apic_id given a valid topology in a policy. The algo is written with the intention of extending it to leaves 0x1f and extended 0x26 in the future. Toolstack doesn't set leaf 0xb and the HVM default policy has it cleared, so the leaf is not implemented. In that case, the new helper just returns the legacy mapping. Signed-off-by: Alejandro Vallejo --- tools/tests/cpu-policy/test-cpu-policy.c | 68 +++++++++++++++++++++ xen/include/xen/lib/x86/cpu-policy.h | 11 ++++ xen/lib/x86/policy.c | 76 ++++++++++++++++++++++++ 3 files changed, 155 insertions(+) diff --git a/tools/tests/cpu-policy/test-cpu-policy.c b/tools/tests/cpu-policy/test-cpu-policy.c index 870f7ecee0e5..ae7fc46a47d2 100644 --- a/tools/tests/cpu-policy/test-cpu-policy.c +++ b/tools/tests/cpu-policy/test-cpu-policy.c @@ -785,6 +785,73 @@ static void test_topo_from_parts(void) } } +static void test_x2apic_id_from_vcpu_id_success(void) +{ + static const struct test { + unsigned int vcpu_id; + unsigned int threads_per_core; + unsigned int cores_per_pkg; + uint32_t x2apic_id; + uint8_t x86_vendor; + } tests[] = { + { + .vcpu_id = 3, .threads_per_core = 3, .cores_per_pkg = 8, + .x2apic_id = 1 << 2, + }, + { + .vcpu_id = 6, .threads_per_core = 3, .cores_per_pkg = 8, + .x2apic_id = 2 << 2, + }, + { + .vcpu_id = 24, .threads_per_core = 3, .cores_per_pkg = 8, + .x2apic_id = 1 << 5, + }, + { + .vcpu_id = 35, .threads_per_core = 3, .cores_per_pkg = 8, + .x2apic_id = (35 % 3) | (((35 / 3) % 8) << 2) | ((35 / 24) << 5), + }, + { + .vcpu_id = 96, .threads_per_core = 7, .cores_per_pkg = 3, + .x2apic_id = (96 % 7) | (((96 / 7) % 3) << 3) | ((96 / 21) << 5), + }, + }; + + const uint8_t vendors[] = { + X86_VENDOR_INTEL, + X86_VENDOR_AMD, + X86_VENDOR_CENTAUR, + X86_VENDOR_SHANGHAI, + X86_VENDOR_HYGON, + }; + + printf("Testing x2apic id from vcpu id success:\n"); + + /* Perform the test run on every vendor we know about */ + for ( size_t i = 0; i < ARRAY_SIZE(vendors); ++i ) + { + for ( size_t j = 0; j < ARRAY_SIZE(tests); ++j ) + { + struct cpu_policy policy = { .x86_vendor = vendors[i] }; + const struct test *t = &tests[j]; + uint32_t x2apic_id; + int rc = x86_topo_from_parts(&policy, t->threads_per_core, + t->cores_per_pkg); + + if ( rc ) { + fail("FAIL[%d] - 'x86_topo_from_parts() failed", rc); + continue; + } + + x2apic_id = x86_x2apic_id_from_vcpu_id(&policy, t->vcpu_id); + if ( x2apic_id != t->x2apic_id ) + fail("FAIL - '%s cpu%u %u t/c %u c/p'. bad x2apic_id: expected=%u actual=%u\n", + x86_cpuid_vendor_to_str(policy.x86_vendor), + t->vcpu_id, t->threads_per_core, t->cores_per_pkg, + t->x2apic_id, x2apic_id); + } + } +} + int main(int argc, char **argv) { printf("CPU Policy unit tests\n"); @@ -803,6 +870,7 @@ int main(int argc, char **argv) test_is_compatible_failure(); test_topo_from_parts(); + test_x2apic_id_from_vcpu_id_success(); if ( nr_failures ) printf("Done: %u failures\n", nr_failures); diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h index 116b305a1d7f..6fe19490d290 100644 --- a/xen/include/xen/lib/x86/cpu-policy.h +++ b/xen/include/xen/lib/x86/cpu-policy.h @@ -542,6 +542,17 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host, const struct cpu_policy *guest, struct cpu_policy_errors *err); +/** + * Calculates the x2APIC ID of a vCPU given a CPU policy + * + * If the policy lacks leaf 0xb falls back to legacy mapping of apic_id=cpu*2 + * + * @param p CPU policy of the domain. + * @param id vCPU ID of the vCPU. + * @returns x2APIC ID of the vCPU. + */ +uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id); + /** * Synthesise topology information in `p` given high-level constraints * diff --git a/xen/lib/x86/policy.c b/xen/lib/x86/policy.c index 16b09a427841..6dd9a2900ad7 100644 --- a/xen/lib/x86/policy.c +++ b/xen/lib/x86/policy.c @@ -2,6 +2,82 @@ #include +static uint32_t parts_per_higher_scoped_level(const struct cpu_policy *p, + size_t lvl) +{ + /* + * `nr_logical` reported by Intel is the number of THREADS contained in + * the next topological scope. For example, assuming a system with 2 + * threads/core and 3 cores/module in a fully symmetric topology, + * `nr_logical` at the core level will report 6. Because it's reporting + * the number of threads in a module. + * + * On AMD/Hygon, nr_logical is already normalized by the higher scoped + * level (cores/complex, etc) so we can return it as-is. + */ + if ( p->x86_vendor != X86_VENDOR_INTEL || !lvl ) + return p->topo.subleaf[lvl].nr_logical; + + return p->topo.subleaf[lvl].nr_logical / + p->topo.subleaf[lvl - 1].nr_logical; +} + +uint32_t x86_x2apic_id_from_vcpu_id(const struct cpu_policy *p, uint32_t id) +{ + uint32_t shift = 0, x2apic_id = 0; + + /* In the absence of topology leaves, fallback to traditional mapping */ + if ( !p->topo.subleaf[0].type ) + return id * 2; + + /* + * `id` means different things at different points of the algo + * + * At lvl=0: global thread_id (same as vcpu_id) + * At lvl=1: global core_id + * At lvl=2: global socket_id (actually complex_id in AMD, module_id + * in Intel, but the name is inconsequential) + * + * +--+ + * ____ |#0| ______ <= 1 socket + * / +--+ \+--+ + * __#0__ __|#1|__ <= 2 cores/socket + * / | \ +--+/ +-|+ \ + * #0 #1 #2 |#3| #4 #5 <= 3 threads/core + * +--+ + * + * ... and so on. Global in this context means that it's a unique + * identifier for the whole topology, and not relative to the level + * it's in. For example, in the diagram shown above, we're looking at + * thread #3 in the global sense, though it's #0 within its core. + * + * Note that dividing a global thread_id by the number of threads per + * core returns the global core id that contains it. e.g: 0, 1 or 2 + * divided by 3 returns core_id=0. 3, 4 or 5 divided by 3 returns core + * 1, and so on. An analogous argument holds for higher levels. This is + * the property we exploit to derive x2apic_id from vcpu_id. + * + * NOTE: `topo` is currently derived from leaf 0xb, which is bound to two + * levels, but once we track leaves 0x1f (or extended 0x26) there will be a + * few more. The algorithm is written to cope with that case. + */ + for ( uint32_t i = 0; i < ARRAY_SIZE(p->topo.raw); i++ ) + { + uint32_t nr_parts; + + if ( !p->topo.subleaf[i].type ) + /* sentinel subleaf */ + break; + + nr_parts = parts_per_higher_scoped_level(p, i); + x2apic_id |= (id % nr_parts) << shift; + id /= nr_parts; + shift = p->topo.subleaf[i].id_shift; + } + + return (id << shift) | x2apic_id; +} + static unsigned int order(unsigned int n) { ASSERT(n); /* clz(0) is UB */ From patchwork Tue Oct 1 12:38:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817929 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5954CE837A for ; Tue, 1 Oct 2024 12:38:52 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808077.1219950 (Exim 4.92) (envelope-from ) id 1svc9L-00057q-SQ; Tue, 01 Oct 2024 12:38:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808077.1219950; Tue, 01 Oct 2024 12:38:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9L-000568-Dt; Tue, 01 Oct 2024 12:38:43 +0000 Received: by outflank-mailman (input) for mailman id 808077; Tue, 01 Oct 2024 12:38:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9J-0002va-KC for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:41 +0000 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [2a00:1450:4864:20::52a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 15ebfffa-7ff2-11ef-99a2-01e77a169b0f; Tue, 01 Oct 2024 14:38:40 +0200 (CEST) Received: by mail-ed1-x52a.google.com with SMTP id 4fb4d7f45d1cf-5c5b954c359so6038464a12.1 for ; Tue, 01 Oct 2024 05:38:40 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:37 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 15ebfffa-7ff2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786318; x=1728391118; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8x1lOX8g1vjbtkMDhxskapyryrttGg/PykaAfYqF8tI=; b=baxaGIXcP3jQ933yqIwqo9lTTvGoIHP0N/hxtz3HgtD0xPMU8qCCc4+mAvkAELhrSs qtGoo30FCpRS0LoLJY2W4at9TnoCtg6gjIBs2e0RbhFcVGaLp+EDYVzSRE9Ae19jlZS5 0pw17vyu8mcfnSdCmmTwqLNf1oRdLTebJb8T0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786318; x=1728391118; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8x1lOX8g1vjbtkMDhxskapyryrttGg/PykaAfYqF8tI=; b=GetAM0vYdt1nqzwHJd9CQV0JlH8rGfIGdeH19jYVeBFLlXZto8D6GHy77JwSe7uRzH 3Wcz7oDtgbGrnFSmfqN+m71hDDTBdKWoaOFJvOhNUOkv83IJvAu+5QM0v+gVWCYZg/xK E4IDAVGy63GQVaYy+Twgw3xoi1Bn+JGdDbWrHFVhcC4w/RwsAkNiGcx8lXfSR+0dzaJa RZToGAzeJijzAPQLYoh31dFscLgeDwVuq3v3PPh58F8zs0qumbn/OLaP/IIl9BXaHUM1 vD9LpTGhTOWGK/JP4n1ZSkPZm+hhATkKWKrhATFxpWHjZqSMoPpoR9mW5vbzlsz9Wxa6 uFXg== X-Gm-Message-State: AOJu0YwLa2eaHso3jE51RSLZAVCiSIlr7OzEGmiNYmxfXhP/SqRnzRRx ERKGaQopARRLGSAIHTXkAzTBrpU9hNEGvBmZZ0XHV5dwAmMaD0kTtzmDK4qBGb90888SFeyfq4y nSV0= X-Google-Smtp-Source: AGHT+IFRKz3oQ8e7LGS9CnwBZ3WyAqbsPQuyesyNlrJd3Sb7NZ9423l2OBIfQ/GVRadeo1LeO5YZEQ== X-Received: by 2002:a05:6402:40c5:b0:5c0:b793:df4e with SMTP id 4fb4d7f45d1cf-5c8825fd1eemr13517406a12.22.1727786317828; Tue, 01 Oct 2024 05:38:37 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Anthony PERARD , Juergen Gross Subject: [PATCH v6 10/11] tools/libguest: Set distinct x2APIC IDs for each vCPU Date: Tue, 1 Oct 2024 13:38:06 +0100 Message-ID: <20241001123807.605-11-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Have toolstack populate the new x2APIC ID in the LAPIC save record with the proper IDs intended for each vCPU. Signed-off-by: Alejandro Vallejo --- v6: * Rely on the new LUT in xc_dom_image rather than recalculating APIC IDs. --- tools/libs/guest/xg_dom_x86.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/tools/libs/guest/xg_dom_x86.c b/tools/libs/guest/xg_dom_x86.c index c98229317db7..38486140ed15 100644 --- a/tools/libs/guest/xg_dom_x86.c +++ b/tools/libs/guest/xg_dom_x86.c @@ -1004,11 +1004,14 @@ static int vcpu_hvm(struct xc_dom_image *dom) HVM_SAVE_TYPE(HEADER) header; struct hvm_save_descriptor mtrr_d; HVM_SAVE_TYPE(MTRR) mtrr; + struct hvm_save_descriptor lapic_d; + HVM_SAVE_TYPE(LAPIC) lapic; struct hvm_save_descriptor end_d; HVM_SAVE_TYPE(END) end; } vcpu_ctx; - /* Context from full_ctx */ + /* Contexts from full_ctx */ const HVM_SAVE_TYPE(MTRR) *mtrr_record; + const HVM_SAVE_TYPE(LAPIC) *lapic_record; /* Raw context as taken from Xen */ uint8_t *full_ctx = NULL; int rc; @@ -1111,6 +1114,8 @@ static int vcpu_hvm(struct xc_dom_image *dom) vcpu_ctx.mtrr_d.typecode = HVM_SAVE_CODE(MTRR); vcpu_ctx.mtrr_d.length = HVM_SAVE_LENGTH(MTRR); vcpu_ctx.mtrr = *mtrr_record; + vcpu_ctx.lapic_d.typecode = HVM_SAVE_CODE(LAPIC); + vcpu_ctx.lapic_d.length = HVM_SAVE_LENGTH(LAPIC); vcpu_ctx.end_d = bsp_ctx.end_d; vcpu_ctx.end = bsp_ctx.end; @@ -1125,6 +1130,18 @@ static int vcpu_hvm(struct xc_dom_image *dom) { vcpu_ctx.mtrr_d.instance = i; + lapic_record = hvm_get_save_record(full_ctx, HVM_SAVE_CODE(LAPIC), i); + if ( !lapic_record ) + { + xc_dom_panic(dom->xch, XC_INTERNAL_ERROR, + "%s: unable to get LAPIC[%d] save record", __func__, i); + goto out; + } + + vcpu_ctx.lapic = *lapic_record; + vcpu_ctx.lapic.x2apic_id = dom->cpu_to_apicid[i]; + vcpu_ctx.lapic_d.instance = i; + rc = xc_domain_hvm_setcontext(dom->xch, dom->guest_domid, (uint8_t *)&vcpu_ctx, sizeof(vcpu_ctx)); if ( rc != 0 ) From patchwork Tue Oct 1 12:38:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13817930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5E77ACEB2FD for ; Tue, 1 Oct 2024 12:38:52 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808076.1219942 (Exim 4.92) (envelope-from ) id 1svc9L-00050p-8Q; Tue, 01 Oct 2024 12:38:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808076.1219942; Tue, 01 Oct 2024 12:38:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9K-0004wn-VF; Tue, 01 Oct 2024 12:38:42 +0000 Received: by outflank-mailman (input) for mailman id 808076; Tue, 01 Oct 2024 12:38:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1svc9J-0002va-D8 for xen-devel@lists.xenproject.org; Tue, 01 Oct 2024 12:38:41 +0000 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [2a00:1450:4864:20::52d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 15db92a5-7ff2-11ef-99a2-01e77a169b0f; Tue, 01 Oct 2024 14:38:39 +0200 (CEST) Received: by mail-ed1-x52d.google.com with SMTP id 4fb4d7f45d1cf-5c88e6926e5so3117056a12.3 for ; Tue, 01 Oct 2024 05:38:39 -0700 (PDT) Received: from mewpvdipd1023.corp.cloud.com ([52.166.251.127]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248ac03sm6132630a12.70.2024.10.01.05.38.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 05:38:38 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 15db92a5-7ff2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1727786319; x=1728391119; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=loD1s7lD9RQwYiYkR/KqljzHBIRiq3F0/w5l0wcOKk4=; b=lRwPWjC1+GOBooJw/ZmNT+yNRs05ArKUQsNLISw6uXzXqC2jdScaOMETJ81FFBCA8C DfQkVa5FjVyn1tPEgmncZFUIsno27vtgSuTx3XssHEK61CrC/mUGwnwttKPf3z7hXTBE DNO2L5DLtee9zSoSKDLmeL4UVVS+ZkGMlxPvQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727786319; x=1728391119; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=loD1s7lD9RQwYiYkR/KqljzHBIRiq3F0/w5l0wcOKk4=; b=v1EgebWgAmGafu0FaOvBXfPtB+1iw4StY87of7JJCCeDWxqpcxzF5NYZDF/9otwJeJ V0jr/o8k5VHun30fDICF5vvc5L+3YqBIj17pdzAW9H85jUA2cvcG1JG8z8fkp2vB2WUz b/lV8ROxLpx8ax05bNZ6jLrOmSqeSvYqer+UnuZIVwvgYavQ9rasMrPSLJ/ejTa7DfUp yrcUXqXOPchOEtnJYSTbv19eINEyRyWcPyrSL7tUII6+nbAoUhnTH0hF6Y/XPbKNa4pD UwooYLJaGw/cToir9qI1CyRIHPutKU2mVpVmliJtvs8heT2NnwxzWOcgDwhBbHuiIfQt 8EYA== X-Gm-Message-State: AOJu0Yz5ACvoeBRIlUQ8X4sHQDHGDqK6m6+qQqRPXSCxVZcm4SR4nzcm HRhTO/WnQ13pYAzr4utXLljHClfSSuWldrLn+juOcWcUXHAOaGty0miR+0ulPJH9i3kTOSytDtP vhUQ= X-Google-Smtp-Source: AGHT+IH7TXEWvP98DHoKzQLTgruBwNIYgU5z1sa88Jq6yvRRgPSXxfpVWGnb88wWJ82YX1JB5jB/dA== X-Received: by 2002:a05:6402:3584:b0:5c5:cf0b:b515 with SMTP id 4fb4d7f45d1cf-5c8824cc677mr11031275a12.1.1727786318964; Tue, 01 Oct 2024 05:38:38 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Anthony PERARD , Juergen Gross , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v6 11/11] tools/x86: Synthesise domain topologies Date: Tue, 1 Oct 2024 13:38:07 +0100 Message-ID: <20241001123807.605-12-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241001123807.605-1-alejandro.vallejo@cloud.com> References: <20241001123807.605-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Expose sensible topologies in leaf 0xb. At the moment it synthesises non-HT systems, in line with the previous code intent. Leaf 0xb in the host policy is no longer zapped and the guest {max,def} policies have their topology leaves zapped instead. The intent is for toolstack to populate them. There's no current use for the topology information in the host policy, but it makes no harm. Signed-off-by: Alejandro Vallejo --- v6: * Assign accurate APIC IDs to xc_dom_img->cpu_to_apicid * New field in v6. Allows ACPI generation to be correct on PVH too. --- tools/include/xenguest.h | 3 +++ tools/libs/guest/xg_cpuid_x86.c | 29 ++++++++++++++++++++++++++++- tools/libs/light/libxl_dom.c | 22 +++++++++++++++++++++- xen/arch/x86/cpu-policy.c | 9 ++++++--- 4 files changed, 58 insertions(+), 5 deletions(-) diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h index aa50b78dfb89..dcabf219b9cb 100644 --- a/tools/include/xenguest.h +++ b/tools/include/xenguest.h @@ -831,6 +831,9 @@ int xc_set_domain_cpu_policy(xc_interface *xch, uint32_t domid, uint32_t xc_get_cpu_featureset_size(void); +/* Returns the APIC ID of the `cpu`-th CPU according to `policy` */ +uint32_t xc_cpu_to_apicid(const xc_cpu_policy_t *policy, unsigned int cpu); + enum xc_static_cpu_featuremask { XC_FEATUREMASK_KNOWN, XC_FEATUREMASK_SPECIAL, diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c index 4453178100ad..c591f8732a1a 100644 --- a/tools/libs/guest/xg_cpuid_x86.c +++ b/tools/libs/guest/xg_cpuid_x86.c @@ -725,8 +725,16 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore, p->policy.basic.htt = test_bit(X86_FEATURE_HTT, host_featureset); p->policy.extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset); } - else + else if ( restore ) { + /* + * Reconstruct the topology exposed on Xen <= 4.13. It makes very little + * sense, but it's what those guests saw so it's set in stone now. + * + * Guests from Xen 4.14 onwards carry their own CPUID leaves in the + * migration stream so they don't need special treatment. + */ + /* * Topology for HVM guests is entirely controlled by Xen. For now, we * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT. @@ -782,6 +790,20 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore, break; } } + else + { + /* TODO: Expose the ability to choose a custom topology for HVM/PVH */ + unsigned int threads_per_core = 1; + unsigned int cores_per_pkg = di.max_vcpu_id + 1; + + rc = x86_topo_from_parts(&p->policy, threads_per_core, cores_per_pkg); + if ( rc ) + { + ERROR("Failed to generate topology: rc=%d t/c=%u c/p=%u", + rc, threads_per_core, cores_per_pkg); + goto out; + } + } nr_leaves = ARRAY_SIZE(p->leaves); rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves); @@ -1028,3 +1050,8 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host, return false; } + +uint32_t xc_cpu_to_apicid(const xc_cpu_policy_t *policy, unsigned int cpu) +{ + return x86_x2apic_id_from_vcpu_id(&policy->policy, cpu); +} diff --git a/tools/libs/light/libxl_dom.c b/tools/libs/light/libxl_dom.c index 7f9c6eaa8b24..8dbfc5b7df61 100644 --- a/tools/libs/light/libxl_dom.c +++ b/tools/libs/light/libxl_dom.c @@ -1063,6 +1063,9 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid, libxl_domain_build_info *const info = &d_config->b_info; struct xc_dom_image *dom = NULL; bool device_model = info->type == LIBXL_DOMAIN_TYPE_HVM ? true : false; +#if defined(__i386__) || defined(__x86_64__) + struct xc_cpu_policy *policy = NULL; +#endif xc_dom_loginit(ctx->xch); @@ -1083,8 +1086,22 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid, dom->container_type = XC_DOM_HVM_CONTAINER; #if defined(__i386__) || defined(__x86_64__) + policy = xc_cpu_policy_init(); + if (!policy) { + LOGE(ERROR, "xc_cpu_policy_get_domain failed d%u", domid); + rc = ERROR_NOMEM; + goto out; + } + + rc = xc_cpu_policy_get_domain(ctx->xch, domid, policy); + if (rc != 0) { + LOGE(ERROR, "xc_cpu_policy_get_domain failed d%u", domid); + rc = ERROR_FAIL; + goto out; + } + for ( uint32_t i = 0; i < info->max_vcpus; i++ ) - dom->cpu_to_apicid[i] = 2 * i; /* TODO: Replace by topo calculation */ + dom->cpu_to_apicid[i] = xc_cpu_to_apicid(policy, i); #endif /* The params from the configuration file are in Mb, which are then @@ -1214,6 +1231,9 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid, out: assert(rc != 0); if (dom != NULL) xc_dom_release(dom); +#if defined(__i386__) || defined(__x86_64__) + xc_cpu_policy_destroy(policy); +#endif return rc; } diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c index b6d9fad56773..39c0aba9ea36 100644 --- a/xen/arch/x86/cpu-policy.c +++ b/xen/arch/x86/cpu-policy.c @@ -266,9 +266,6 @@ static void recalculate_misc(struct cpu_policy *p) p->basic.raw[0x8] = EMPTY_LEAF; - /* TODO: Rework topology logic. */ - memset(p->topo.raw, 0, sizeof(p->topo.raw)); - p->basic.raw[0xc] = EMPTY_LEAF; p->extd.e1d &= ~CPUID_COMMON_1D_FEATURES; @@ -616,6 +613,9 @@ static void __init calculate_pv_max_policy(void) recalculate_xstate(p); p->extd.raw[0xa] = EMPTY_LEAF; /* No SVM for PV guests. */ + + /* Wipe host topology. Populated by toolstack */ + memset(p->topo.raw, 0, sizeof(p->topo.raw)); } static void __init calculate_pv_def_policy(void) @@ -779,6 +779,9 @@ static void __init calculate_hvm_max_policy(void) /* It's always possible to emulate CPUID faulting for HVM guests */ p->platform_info.cpuid_faulting = true; + + /* Wipe host topology. Populated by toolstack */ + memset(p->topo.raw, 0, sizeof(p->topo.raw)); } static void __init calculate_hvm_def_policy(void)