From patchwork Wed Nov 13 03:17:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13873136 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D7B3A166310; Wed, 13 Nov 2024 03:17:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467854; cv=none; b=Oxwq1RzOT1H/XklFveyAGNXPHqIJ2AogqtiGxg9scowjiUmeFt0Ekjak7WwQQF7JUepBfbC/TKh/78GV7OCEKm/dV7uO0eXrNl+I4f/R3gfWDheAyQfjoTncxbzJWNfpx+VR0K2OKNq+s/rxzPj+aYMdrENEZFXMe7Jgld6uezs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467854; c=relaxed/simple; bh=q5P7swPuEjyp3vJ2hRrjCtBVHumWvilkU8Jd0nUXiDE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=u1d88+EnymR4qFY9cSiWWtJk9CgNqJbRAtaMI7HpECdXUI9hJ+9WWxLZCFxqW2R8eZpZvVujhLDgufi+waIVdRY/oNVC/v1kOPBCMY4vJWa29cGpzgRAbldiEzZG1/FTLMt4jTYoslg5Ln9h5/YzxNougHd+/ZYEXG9EFoztoco= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8Ax6+FJGjRnTn08AA--.53415S3; Wed, 13 Nov 2024 11:17:29 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by front1 (Coremail) with SMTP id qMiowMAxDEdHGjRnX4VTAA--.14727S3; Wed, 13 Nov 2024 11:17:28 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RFC 1/5] LoongArch: KVM: Add vmid support for stage2 MMU Date: Wed, 13 Nov 2024 11:17:23 +0800 Message-Id: <20241113031727.2815628-2-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20241113031727.2815628-1-maobibo@loongson.cn> References: <20241113031727.2815628-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: qMiowMAxDEdHGjRnX4VTAA--.14727S3 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== LoongArch KVM hypervisor supports two-level MMU, vpid index is used for stage1 MMU and vmid index is used for stage2 MMU. On 3A5000, vmid must be the same with vpid. On 3A6000 platform vmid may separate from vpid. If vcpu migrate to different physical CPUs, vpid need change however vmid can keep the same with old value. Also vmid index of the while VM machine on physical CPU the same, all vCPUs on the VM can share the same vmid index on one physical CPU. Here vmid index is added and it keeps the same with vpid still. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/kvm_host.h | 3 +++ arch/loongarch/kernel/asm-offsets.c | 1 + arch/loongarch/kvm/main.c | 1 + arch/loongarch/kvm/switch.S | 5 ++--- arch/loongarch/kvm/tlb.c | 5 ++++- 5 files changed, 11 insertions(+), 4 deletions(-) diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h index d6bb72424027..6151c7c470d5 100644 --- a/arch/loongarch/include/asm/kvm_host.h +++ b/arch/loongarch/include/asm/kvm_host.h @@ -166,6 +166,9 @@ struct kvm_vcpu_arch { unsigned long host_tp; unsigned long host_pgd; + /* vmid info for guest VM */ + unsigned long vmid; + /* Host CSRs are used when handling exits from guest */ unsigned long badi; unsigned long badv; diff --git a/arch/loongarch/kernel/asm-offsets.c b/arch/loongarch/kernel/asm-offsets.c index bee9f7a3108f..4e9a9311afd3 100644 --- a/arch/loongarch/kernel/asm-offsets.c +++ b/arch/loongarch/kernel/asm-offsets.c @@ -307,6 +307,7 @@ static void __used output_kvm_defines(void) OFFSET(KVM_ARCH_HSP, kvm_vcpu_arch, host_sp); OFFSET(KVM_ARCH_HTP, kvm_vcpu_arch, host_tp); OFFSET(KVM_ARCH_HPGD, kvm_vcpu_arch, host_pgd); + OFFSET(KVM_ARCH_VMID, kvm_vcpu_arch, vmid); OFFSET(KVM_ARCH_HANDLE_EXIT, kvm_vcpu_arch, handle_exit); OFFSET(KVM_ARCH_HEENTRY, kvm_vcpu_arch, host_eentry); OFFSET(KVM_ARCH_GEENTRY, kvm_vcpu_arch, guest_eentry); diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c index 27e9b94c0a0b..8c16bff80053 100644 --- a/arch/loongarch/kvm/main.c +++ b/arch/loongarch/kvm/main.c @@ -212,6 +212,7 @@ static void kvm_update_vpid(struct kvm_vcpu *vcpu, int cpu) context->vpid_cache = vpid; vcpu->arch.vpid = vpid; + vcpu->arch.vmid = vcpu->arch.vpid & vpid_mask; } void kvm_check_vpid(struct kvm_vcpu *vcpu) diff --git a/arch/loongarch/kvm/switch.S b/arch/loongarch/kvm/switch.S index 0c292f818492..2774343f64d3 100644 --- a/arch/loongarch/kvm/switch.S +++ b/arch/loongarch/kvm/switch.S @@ -72,9 +72,8 @@ ldx.d t0, t1, t0 csrwr t0, LOONGARCH_CSR_PGDL - /* Mix GID and RID */ - csrrd t1, LOONGARCH_CSR_GSTAT - bstrpick.w t1, t1, CSR_GSTAT_GID_SHIFT_END, CSR_GSTAT_GID_SHIFT + /* Set VMID for gpa --> hpa mapping */ + ld.d t1, a2, KVM_ARCH_VMID csrrd t0, LOONGARCH_CSR_GTLBC bstrins.w t0, t1, CSR_GTLBC_TGID_SHIFT_END, CSR_GTLBC_TGID_SHIFT csrwr t0, LOONGARCH_CSR_GTLBC diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c index ebdbe9264e9c..38daf936021d 100644 --- a/arch/loongarch/kvm/tlb.c +++ b/arch/loongarch/kvm/tlb.c @@ -23,7 +23,10 @@ void kvm_flush_tlb_all(void) void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa) { + unsigned int vmid; + lockdep_assert_irqs_disabled(); gpa &= (PAGE_MASK << 1); - invtlb(INVTLB_GID_ADDR, read_csr_gstat() & CSR_GSTAT_GID, gpa); + vmid = (vcpu->arch.vmid << CSR_GSTAT_GID_SHIFT) & CSR_GSTAT_GID; + invtlb(INVTLB_GID_ADDR, vmid, gpa); } From patchwork Wed Nov 13 03:17:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13873135 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7E8491662E7; Wed, 13 Nov 2024 03:17:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467854; cv=none; b=VFeC1GPMw26YnoXqAIN8+o86wSfAmVJalvhNgvEFpNHE7zmyNCJy70aNpIEXt829DorqNijUqtXtarZgxR0GtchkZgfxMwi0gF7GC2Ul9UVtpRlbSGGIc0U5oaTUft7uz5kZr93rOd6SeyfEhZzKdDBBeKj5ATSOAk7UnFHb72E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467854; c=relaxed/simple; bh=ds2iCd9Y1vCDdtQRKzk60n9z1DNTpM4hNfiXD6U4dho=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dDex+mruONtQw2ZNpybg342+2ulx6bW+Q2WvfFjb0w5afwxku/rMg3gb5P+KJeqmA/kzHu8gyZEBjOkoSt4EPF0rNXKCciy11nyNThlBUa68aKErMCVj6dE45oPJtadjARHyNbb3Bo3BiRQA1OEBNzOCGEEBDT9nxK7Ai3/KfUA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8CxC+JKGjRnUn08AA--.53836S3; Wed, 13 Nov 2024 11:17:30 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by front1 (Coremail) with SMTP id qMiowMAxDEdHGjRnX4VTAA--.14727S4; Wed, 13 Nov 2024 11:17:29 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RFC 2/5] LoongArch: KVM: Add separate vmid feature support Date: Wed, 13 Nov 2024 11:17:24 +0800 Message-Id: <20241113031727.2815628-3-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20241113031727.2815628-1-maobibo@loongson.cn> References: <20241113031727.2815628-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: qMiowMAxDEdHGjRnX4VTAA--.14727S4 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== Feature cpu_has_guestid is used to check whether separate vmid/vpid is supported or not. Also add different vmid updating function. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/kvm_host.h | 2 ++ arch/loongarch/kvm/main.c | 30 +++++++++++++++++++++++---- arch/loongarch/kvm/tlb.c | 14 +++++++++++++ 3 files changed, 42 insertions(+), 4 deletions(-) diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h index 6151c7c470d5..92ec3660d221 100644 --- a/arch/loongarch/include/asm/kvm_host.h +++ b/arch/loongarch/include/asm/kvm_host.h @@ -279,6 +279,8 @@ static inline int kvm_get_pmu_num(struct kvm_vcpu_arch *arch) int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu); /* MMU handling */ +void kvm_flush_tlb_all_stage1(void); +void kvm_flush_tlb_all_stage2(void); void kvm_flush_tlb_all(void); void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa); int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long badv, bool write); diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c index 8c16bff80053..afb2e10eba68 100644 --- a/arch/loongarch/kvm/main.c +++ b/arch/loongarch/kvm/main.c @@ -207,15 +207,17 @@ static void kvm_update_vpid(struct kvm_vcpu *vcpu, int cpu) ++vpid; /* vpid 0 reserved for root */ /* start new vpid cycle */ - kvm_flush_tlb_all(); + if (!cpu_has_guestid) + kvm_flush_tlb_all(); + else + kvm_flush_tlb_all_stage1(); } context->vpid_cache = vpid; vcpu->arch.vpid = vpid; - vcpu->arch.vmid = vcpu->arch.vpid & vpid_mask; } -void kvm_check_vpid(struct kvm_vcpu *vcpu) +static void __kvm_check_vpid(struct kvm_vcpu *vcpu) { int cpu; bool migrated; @@ -243,7 +245,6 @@ void kvm_check_vpid(struct kvm_vcpu *vcpu) kvm_update_vpid(vcpu, cpu); trace_kvm_vpid_change(vcpu, vcpu->arch.vpid); vcpu->cpu = cpu; - kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu); } /* Restore GSTAT(0x50).vpid */ @@ -251,6 +252,27 @@ void kvm_check_vpid(struct kvm_vcpu *vcpu) change_csr_gstat(vpid_mask << CSR_GSTAT_GID_SHIFT, vpid); } +static void __kvm_check_vmid(struct kvm_vcpu *vcpu) +{ + unsigned long vmid; + + /* On some machines like 3A5000, vmid needs the same with vpid */ + if (!cpu_has_guestid) { + vmid = vcpu->arch.vpid & vpid_mask; + if (vcpu->arch.vmid != vmid) { + vcpu->arch.vmid = vmid; + kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu); + } + return; + } +} + +void kvm_check_vpid(struct kvm_vcpu *vcpu) +{ + __kvm_check_vpid(vcpu); + __kvm_check_vmid(vcpu); +} + void kvm_init_vmcs(struct kvm *kvm) { kvm->arch.vmcs = vmcs; diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c index 38daf936021d..1d95e2208e82 100644 --- a/arch/loongarch/kvm/tlb.c +++ b/arch/loongarch/kvm/tlb.c @@ -21,6 +21,20 @@ void kvm_flush_tlb_all(void) local_irq_restore(flags); } +/* Invalidate all stage1 TLB entries including GVA-->GPA mappings */ +void kvm_flush_tlb_all_stage1(void) +{ + lockdep_assert_irqs_disabled(); + invtlb_all(INVGTLB_ALLGID_GVA_TO_GPA, 0, 0); +} + +/* Invalidate all stage2 TLB entries including GPA-->HPA mappings */ +void kvm_flush_tlb_all_stage2(void) +{ + lockdep_assert_irqs_disabled(); + invtlb_all(INVTLB_ALLGID_GPA_TO_HPA, 0, 0); +} + void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa) { unsigned int vmid; From patchwork Wed Nov 13 03:17:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13873138 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 59880166F26; Wed, 13 Nov 2024 03:17:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467855; cv=none; b=nKtJSppucncEfQ601Z0qmq+rIia/E8zaGGzi5BpkZuNblvcas9jvC4zX2r7fcAbFo1+T/tal1ghXijspzJeaPpAGliCGI0fWsKYN/7umYN1Bp9jX5XmwMO0v598gUVvrgLKiNr/UtQdq2awcmS/EmLe17MBdyk4ltCT2RDvwRtg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467855; c=relaxed/simple; bh=4Dk6ZN5yXXy56uvJFbd8k8Z3qHYS9PGMSK0FZg+JjHM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MvyWLAOUyigJbpgnoA4UF6pnchQ/40j55ea0MmpKeCIFv29p0Deq+z44wjPDehM6Ps5hUI3W7hAZqKJ4R+qjSvajssCsqazR544nsLYvtULo4Un2T0+mJ93aVBx4rKt+XsDwy1O9BEtXBSNuCWCfyt+RFwouEMcB0InTcgvuSXc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8CxieBKGjRnVn08AA--.53729S3; Wed, 13 Nov 2024 11:17:30 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by front1 (Coremail) with SMTP id qMiowMAxDEdHGjRnX4VTAA--.14727S5; Wed, 13 Nov 2024 11:17:30 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RFC 3/5] LoongArch: KVM: implement vmid updating logic Date: Wed, 13 Nov 2024 11:17:25 +0800 Message-Id: <20241113031727.2815628-4-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20241113031727.2815628-1-maobibo@loongson.cn> References: <20241113031727.2815628-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: qMiowMAxDEdHGjRnX4VTAA--.14727S5 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== For every physical CPU, there is one vmid calculation method. For vCPUs on the same VM, vmid is the same. However for vCPUs on different VM, vmid is different. When vCPU is scheduled on the physical CPU, it checked vmid of this VM and the global cached vmid, and judge whether it is valid or not. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/kvm_host.h | 2 ++ arch/loongarch/kvm/main.c | 42 ++++++++++++++++++++++++++- 2 files changed, 43 insertions(+), 1 deletion(-) diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h index 92ec3660d221..725d9c4e1965 100644 --- a/arch/loongarch/include/asm/kvm_host.h +++ b/arch/loongarch/include/asm/kvm_host.h @@ -64,6 +64,7 @@ struct kvm_arch_memory_slot { #define HOST_MAX_PMNUM 16 struct kvm_context { unsigned long vpid_cache; + unsigned long vmid_cache; struct kvm_vcpu *last_vcpu; /* Host PMU CSR */ u64 perf_ctrl[HOST_MAX_PMNUM]; @@ -116,6 +117,7 @@ struct kvm_arch { unsigned long pv_features; s64 time_offset; + unsigned long vmid[NR_CPUS]; struct kvm_context __percpu *vmcs; }; diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c index afb2e10eba68..367653b49a35 100644 --- a/arch/loongarch/kvm/main.c +++ b/arch/loongarch/kvm/main.c @@ -252,9 +252,33 @@ static void __kvm_check_vpid(struct kvm_vcpu *vcpu) change_csr_gstat(vpid_mask << CSR_GSTAT_GID_SHIFT, vpid); } -static void __kvm_check_vmid(struct kvm_vcpu *vcpu) +static void kvm_update_vmid(struct kvm_vcpu *vcpu, int cpu) { unsigned long vmid; + struct kvm_context *context; + + context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu); + vmid = context->vmid_cache + 1; + if (!(vmid & vpid_mask)) { + /* finish round of vmid loop */ + if (unlikely(!vmid)) + vmid = vpid_mask + 1; + + ++vmid; /* vmid 0 reserved for root */ + + /* start new vmid cycle */ + kvm_flush_tlb_all_stage2(); + } + + context->vmid_cache = vmid; + vcpu->kvm->arch.vmid[cpu] = vmid; +} + +static void __kvm_check_vmid(struct kvm_vcpu *vcpu) +{ + int cpu; + unsigned long ver, old, vmid; + struct kvm_context *context; /* On some machines like 3A5000, vmid needs the same with vpid */ if (!cpu_has_guestid) { @@ -265,6 +289,21 @@ static void __kvm_check_vmid(struct kvm_vcpu *vcpu) } return; } + + cpu = smp_processor_id(); + context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu); + + /* + * Check if our vmid is of an older version + */ + ver = vcpu->kvm->arch.vmid[cpu] & ~vpid_mask; + old = context->vmid_cache & ~vpid_mask; + if (ver != old) { + kvm_update_vmid(vcpu, cpu); + kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu); + } + + vcpu->arch.vmid = vcpu->kvm->arch.vmid[cpu] & vpid_mask; } void kvm_check_vpid(struct kvm_vcpu *vcpu) @@ -386,6 +425,7 @@ static int kvm_loongarch_env_init(void) for_each_possible_cpu(cpu) { context = per_cpu_ptr(vmcs, cpu); context->vpid_cache = vpid_mask + 1; + context->vmid_cache = vpid_mask + 1; context->last_vcpu = NULL; } From patchwork Wed Nov 13 03:17:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13873139 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9CA35165F08; Wed, 13 Nov 2024 03:17:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467856; cv=none; b=TqC1SmWfydxJ02ovPyQY71LrNOzsv3jKwI3sQihpWEsSBAZVVQxFAjkUI/c3TPO5KuC+xvXCBC7rRgLIBw/BuG2qTy7Xqm9mxhzJTaJXgwsqCTVdPQr1CGPaQJZ5dctzr0YP71YqD7m88qlwN6vNTwziN8wNcFdRpjv9PCpRzQQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467856; c=relaxed/simple; bh=R7jYZqx7GSLjqcCUIxjDjiNRq78ObURiNWfb3mXRxaI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aMaNFKVfT+8KMRxNDLXH4+d4ZWV0CEzBURF3gNlVxTUya0sjS5SgvRYXhKzcPgGx7+45IGF63rv1ufcGoJ+OXiYBPpQ25JtQhF0fZlA48OzgcoYlhzL+Pvy8LaH9/ZGiEwsBAe65V++u10Qkc8XLLkcT71JAodCnJ8FJQgkMKXE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8DxmeFLGjRnWn08AA--.56251S3; Wed, 13 Nov 2024 11:17:31 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by front1 (Coremail) with SMTP id qMiowMAxDEdHGjRnX4VTAA--.14727S6; Wed, 13 Nov 2024 11:17:30 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RFC 4/5] LoongArch: KVM: Add remote tlb flushing support Date: Wed, 13 Nov 2024 11:17:26 +0800 Message-Id: <20241113031727.2815628-5-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20241113031727.2815628-1-maobibo@loongson.cn> References: <20241113031727.2815628-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: qMiowMAxDEdHGjRnX4VTAA--.14727S6 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== With remote tlb flushing, vpid index stays unchanged and only vmid index is updated, since remote tlb flushing is to flush TLBs relative GPA --> HPA. For flushing method, cpumask tlb_flush_pending is added and set for all possible CPUs. When vCPUs is sched on the physical CPU, vmid is updated and cpumask for this physical CPU is cleared. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/kvm_host.h | 3 +++ arch/loongarch/kvm/main.c | 3 +++ arch/loongarch/kvm/mmu.c | 17 +++++++++++++++++ arch/loongarch/kvm/vcpu.c | 7 ++++++- 4 files changed, 29 insertions(+), 1 deletion(-) diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h index 725d9c4e1965..8f4b4b9a4e3c 100644 --- a/arch/loongarch/include/asm/kvm_host.h +++ b/arch/loongarch/include/asm/kvm_host.h @@ -117,6 +117,7 @@ struct kvm_arch { unsigned long pv_features; s64 time_offset; + cpumask_t tlb_flush_pending; unsigned long vmid[NR_CPUS]; struct kvm_context __percpu *vmcs; }; @@ -317,6 +318,8 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {} static inline void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) {} void kvm_check_vpid(struct kvm_vcpu *vcpu); +#define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS +int kvm_arch_flush_remote_tlbs(struct kvm *kvm); enum hrtimer_restart kvm_swtimer_wakeup(struct hrtimer *timer); void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm, const struct kvm_memory_slot *memslot); void kvm_init_vmcs(struct kvm *kvm); diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c index 367653b49a35..f89d1df885d7 100644 --- a/arch/loongarch/kvm/main.c +++ b/arch/loongarch/kvm/main.c @@ -292,6 +292,9 @@ static void __kvm_check_vmid(struct kvm_vcpu *vcpu) cpu = smp_processor_id(); context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu); + if (cpumask_test_and_clear_cpu(cpu, &vcpu->kvm->arch.tlb_flush_pending)) + vcpu->kvm->arch.vmid[cpu] = 0; + /* * Check if our vmid is of an older version diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 28681dfb4b85..763021cf829d 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -947,6 +947,23 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { } +/* + * kvm_arch_flush_remote_tlbs() - flush all VM TLB entries + * @kvm: pointer to kvm structure. + */ +int kvm_arch_flush_remote_tlbs(struct kvm *kvm) +{ + /* + * Queue a TLB invalidation for each CPU to perform on next + * vcpu loading + */ + if (cpu_has_guestid) + cpumask_setall(&kvm->arch.tlb_flush_pending); + + /* Return 1 continue to send ipi to running vCPUs */ + return 1; +} + void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm, const struct kvm_memory_slot *memslot) { diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c index 174734a23d0a..703f5f2fbb31 100644 --- a/arch/loongarch/kvm/vcpu.c +++ b/arch/loongarch/kvm/vcpu.c @@ -208,7 +208,12 @@ static int kvm_check_requests(struct kvm_vcpu *vcpu) return RESUME_GUEST; if (kvm_check_request(KVM_REQ_TLB_FLUSH, vcpu)) - vcpu->arch.vpid = 0; /* Drop vpid for this vCPU */ + /* + * vpid need the same with vmid if vpid is not separated + * with vmid + */ + if (!cpu_has_guestid) + vcpu->arch.vpid = 0; if (kvm_dirty_ring_check_request(vcpu)) return RESUME_HOST; From patchwork Wed Nov 13 03:17:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13873137 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 13B40166F23; Wed, 13 Nov 2024 03:17:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467855; cv=none; b=iXO6SgmHxRpPE1kBpFaWtcUAIUt3T+TY92zS4anTOql84ZDlHUrYgw7X4wwMU1fgbfJau67Wm/T+2pA6Ra09wSti5sk+qf6oH5YY96s1SiM62sxxoHES+OtsFVt9OJ3a/GetW+iLn9/tzRIwGU0LYRWVXKkW3z+isaX0N6uyOLc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731467855; c=relaxed/simple; bh=6lEKy1RfUWoR2IORPrrqQmixVxn3B4fAvLaywOrL9bU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EsJxwwLl6NAxkEOzIjDJeCUBvdc7iKlJpbOkqY5tV2ob4WUPON1XhPSc86HkwCsC3sP66DHIQ83MXAYdsXBeqV5Jyfpy2tqf6+oHgA6ZN54oqim3wwDe1LO5EDOdTxCRXdGhlHFRbnHByQnutKBqZcRO0dnEt/rLStRvmmCUFbs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8AxEK9LGjRnXn08AA--.15146S3; Wed, 13 Nov 2024 11:17:31 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by front1 (Coremail) with SMTP id qMiowMAxDEdHGjRnX4VTAA--.14727S7; Wed, 13 Nov 2024 11:17:31 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RFC 5/5] LoongArch: KVM: Enable separate vmid feature Date: Wed, 13 Nov 2024 11:17:27 +0800 Message-Id: <20241113031727.2815628-6-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20241113031727.2815628-1-maobibo@loongson.cn> References: <20241113031727.2815628-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: qMiowMAxDEdHGjRnX4VTAA--.14727S7 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== With CSR GTLBC shortname for Guest TLB Control Register, separate vmid feature will be enabled if bit 14 CSR_GTLBC_USEVMID is set. Enable this feature if cpu_has_guestid is true when KVM module is loaded. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/loongarch.h | 2 ++ arch/loongarch/kvm/main.c | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h index 64ad277e096e..5fee5db3bea0 100644 --- a/arch/loongarch/include/asm/loongarch.h +++ b/arch/loongarch/include/asm/loongarch.h @@ -326,6 +326,8 @@ #define CSR_GTLBC_TGID_WIDTH 8 #define CSR_GTLBC_TGID_SHIFT_END (CSR_GTLBC_TGID_SHIFT + CSR_GTLBC_TGID_WIDTH - 1) #define CSR_GTLBC_TGID (_ULCAST_(0xff) << CSR_GTLBC_TGID_SHIFT) +#define CSR_GTLBC_USEVMID_SHIFT 14 +#define CSR_GTLBC_USEVMID (_ULCAST_(0x1) << CSR_GTLBC_USEVMID_SHIFT) #define CSR_GTLBC_TOTI_SHIFT 13 #define CSR_GTLBC_TOTI (_ULCAST_(0x1) << CSR_GTLBC_TOTI_SHIFT) #define CSR_GTLBC_USETGID_SHIFT 12 diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c index f89d1df885d7..50c977d8b414 100644 --- a/arch/loongarch/kvm/main.c +++ b/arch/loongarch/kvm/main.c @@ -336,7 +336,7 @@ int kvm_arch_enable_virtualization_cpu(void) write_csr_gcfg(0); write_csr_gstat(0); write_csr_gintc(0); - clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI); + clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI | CSR_GTLBC_USEVMID); /* * Enable virtualization features granting guest direct control of @@ -359,6 +359,8 @@ int kvm_arch_enable_virtualization_cpu(void) /* Enable using TGID */ set_csr_gtlbc(CSR_GTLBC_USETGID); + if (cpu_has_guestid) + set_csr_gtlbc(CSR_GTLBC_USEVMID); kvm_debug("GCFG:%lx GSTAT:%lx GINTC:%lx GTLBC:%lx", read_csr_gcfg(), read_csr_gstat(), read_csr_gintc(), read_csr_gtlbc());