From patchwork Mon Jun 24 07:14:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13709065 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2AB1A3D969; Mon, 24 Jun 2024 07:14:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213269; cv=none; b=g/ucjPn1ubJRKAEJ2o/wffvSxN0JxVo7x14duZk09WOxq45ayW9uE23PWeDJGOAEmizB7znzTWqn+qO/MK/NGsX81wLJTjKcSpwYNN9zUx79u/d04HhqvVY799teBCjySpnRIZzSTe36jPhky1eDcUdG8wiHsAAQPkHaH2iN+xA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213269; c=relaxed/simple; bh=mgO3Ffeu+JMPmyYW5o8I9ebUfnt0F2rVvC6Ya7pvejQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gjXTQziOq2Czkow6AQGyHJ2b78ZE7WFgWv+AHSJTzTITRWc3I08wMbGh0E7V1DV4l08B9DXtimOMXzHdj8Ia5RfPse7CV/w6e8cxBqUVAdfgaMkzMuwsQiBBBLV5/QeWVVuSnsBP5UeF+6iCkKEgPv4wRcABFJow3dxf2BUehz0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8Cxe+rQHHlmB3EJAA--.38358S3; Mon, 24 Jun 2024 15:14:24 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxMMTPHHlmftsuAA--.9847S3; Mon, 24 Jun 2024 15:14:23 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, WANG Rui Subject: [PATCH v3 1/7] LoongArch: KVM: Delay secondary mmu tlb flush until guest entry Date: Mon, 24 Jun 2024 15:14:16 +0800 Message-Id: <20240624071422.3473789-2-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240624071422.3473789-1-maobibo@loongson.cn> References: <20240624071422.3473789-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxMMTPHHlmftsuAA--.9847S3 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== With hardware assisted virtualization, there are two level HW mmu, one is GVA to GPA mapping, the other is GPA to HPA mapping which is called secondary mmu in generic. If there is page fault for secondary mmu, there needs tlb flush operation indexed with fault gpa address and VMID. VMID is stored at register CSR_GSTAT and will be reload or recalculated before guest entry. Currently CSR_GSTAT is not saved and restored during vcpu context switch, it is recalculated during guest entry. So CSR_GSTAT is in effect only when vcpu runs in guest mode, however it may be not in effected if vcpu exits to host mode, since register CSR_GSTAT may be stale, it maybe records VMID of last schedule-out vcpu, rather than current vcpu. Function kvm_flush_tlb_gpa() should be called with its real VMID, here move it to guest entrance. Also arch specific request id KVM_REQ_TLB_FLUSH_GPA is added to flush tlb for secondary mmu, and it can be optimized if VMID is updated, since all guest tlb entries will be invalid if VMID is updated. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/kvm_host.h | 2 ++ arch/loongarch/kvm/main.c | 1 + arch/loongarch/kvm/mmu.c | 4 ++-- arch/loongarch/kvm/tlb.c | 5 +---- arch/loongarch/kvm/vcpu.c | 18 ++++++++++++++++++ 5 files changed, 24 insertions(+), 6 deletions(-) diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h index c87b6ea0ec47..32c4948f534f 100644 --- a/arch/loongarch/include/asm/kvm_host.h +++ b/arch/loongarch/include/asm/kvm_host.h @@ -30,6 +30,7 @@ #define KVM_PRIVATE_MEM_SLOTS 0 #define KVM_HALT_POLL_NS_DEFAULT 500000 +#define KVM_REQ_TLB_FLUSH_GPA KVM_ARCH_REQ(0) #define KVM_GUESTDBG_SW_BP_MASK \ (KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_USE_SW_BP) @@ -190,6 +191,7 @@ struct kvm_vcpu_arch { /* vcpu's vpid */ u64 vpid; + gpa_t flush_gpa; /* Frequency of stable timer in Hz */ u64 timer_mhz; diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c index 86a2f2d0cb27..844736b99d38 100644 --- a/arch/loongarch/kvm/main.c +++ b/arch/loongarch/kvm/main.c @@ -242,6 +242,7 @@ void kvm_check_vpid(struct kvm_vcpu *vcpu) kvm_update_vpid(vcpu, cpu); trace_kvm_vpid_change(vcpu, vcpu->arch.vpid); vcpu->cpu = cpu; + kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu); } /* Restore GSTAT(0x50).vpid */ diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 98883aa23ab8..9e39d28fec35 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -908,8 +908,8 @@ int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) return ret; /* Invalidate this entry in the TLB */ - kvm_flush_tlb_gpa(vcpu, gpa); - + vcpu->arch.flush_gpa = gpa; + kvm_make_request(KVM_REQ_TLB_FLUSH_GPA, vcpu); return 0; } diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c index 02535df6b51f..ebdbe9264e9c 100644 --- a/arch/loongarch/kvm/tlb.c +++ b/arch/loongarch/kvm/tlb.c @@ -23,10 +23,7 @@ void kvm_flush_tlb_all(void) void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa) { - unsigned long flags; - - local_irq_save(flags); + lockdep_assert_irqs_disabled(); gpa &= (PAGE_MASK << 1); invtlb(INVTLB_GID_ADDR, read_csr_gstat() & CSR_GSTAT_GID, gpa); - local_irq_restore(flags); } diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c index 9e8030d45129..b747bd8bc037 100644 --- a/arch/loongarch/kvm/vcpu.c +++ b/arch/loongarch/kvm/vcpu.c @@ -51,6 +51,16 @@ static int kvm_check_requests(struct kvm_vcpu *vcpu) return RESUME_GUEST; } +static void kvm_late_check_requests(struct kvm_vcpu *vcpu) +{ + lockdep_assert_irqs_disabled(); + if (kvm_check_request(KVM_REQ_TLB_FLUSH_GPA, vcpu)) + if (vcpu->arch.flush_gpa != INVALID_GPA) { + kvm_flush_tlb_gpa(vcpu, vcpu->arch.flush_gpa); + vcpu->arch.flush_gpa = INVALID_GPA; + } +} + /* * Check and handle pending signal and vCPU requests etc * Run with irq enabled and preempt enabled @@ -101,6 +111,13 @@ static int kvm_pre_enter_guest(struct kvm_vcpu *vcpu) /* Make sure the vcpu mode has been written */ smp_store_mb(vcpu->mode, IN_GUEST_MODE); kvm_check_vpid(vcpu); + + /* + * Called after function kvm_check_vpid() + * Since it updates csr_gstat used by kvm_flush_tlb_gpa(), + * also it may clear KVM_REQ_TLB_FLUSH_GPA pending bit + */ + kvm_late_check_requests(vcpu); vcpu->arch.host_eentry = csr_read64(LOONGARCH_CSR_EENTRY); /* Clear KVM_LARCH_SWCSR_LATEST as CSR will change when enter guest */ vcpu->arch.aux_inuse &= ~KVM_LARCH_SWCSR_LATEST; @@ -994,6 +1011,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) struct loongarch_csrs *csr; vcpu->arch.vpid = 0; + vcpu->arch.flush_gpa = INVALID_GPA; hrtimer_init(&vcpu->arch.swtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_PINNED); vcpu->arch.swtimer.function = kvm_swtimer_wakeup; From patchwork Mon Jun 24 07:14:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13709062 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7F8EE3BBF2; Mon, 24 Jun 2024 07:14:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213268; cv=none; b=DN2b204yk2IkzFHoPIKY9O0m6pg8QFETwtnsHWsRiV5Jqo/Ja904faxnxmsHqg3ykyTcZAqh2zTmD/Z8wGkhfgaAeS9FnLlUEAeUyZRaxOnJV7koN6OaBE9zP+UznZr4+MtjksnG1O+xOCT39LTrMG5bCp/mEOHluYgczyHTTQE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213268; c=relaxed/simple; bh=5nNNwBKF40usngDFhlzxJZVkfQIOyHAzYjcY4WPZc4I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=edCdc8bpnGFqmQv8SM8NwXsSIKYYSDA+3qa+FKeMvlKknQ1bX33Be88k6kK9aiQCqj5F30jJHhmiopYhNz11hXyzkG+BLEwnMUNMeIoSaxM4eQv2mBkkiHPOrG2+nVS2hIU9bR66P7lZFK5WxQ7QZji8GBsbdn5YvfJ21Y9+o8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8CxvOrQHHlmC3EJAA--.37691S3; Mon, 24 Jun 2024 15:14:24 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxMMTPHHlmftsuAA--.9847S4; Mon, 24 Jun 2024 15:14:23 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, WANG Rui Subject: [PATCH v3 2/7] LoongArch: KVM: Select huge page only if secondary mmu supports it Date: Mon, 24 Jun 2024 15:14:17 +0800 Message-Id: <20240624071422.3473789-3-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240624071422.3473789-1-maobibo@loongson.cn> References: <20240624071422.3473789-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxMMTPHHlmftsuAA--.9847S4 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== Currently page level selection about secondary mmu depends on memory slot and page level about host mmu. There will be problem if page level of secondary mmu is zero already. Huge page cannot be selected if there is normal page mapped in secondary mmu already, since it is not supported to merge normal pages into huge page now. So page level selection should depend on the following three conditions. 1. Memslot is aligned for huge page and vm is not migrating. 2. Page level of host mmu is huge page also. 3. Page level of secondary mmu is suituable for huge page. Signed-off-by: Bibo Mao --- arch/loongarch/kvm/mmu.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 9e39d28fec35..c6351d13ca1b 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -858,10 +858,20 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) /* Disable dirty logging on HugePages */ level = 0; - if (!fault_supports_huge_mapping(memslot, hva, write)) { - level = 0; - } else { + if (fault_supports_huge_mapping(memslot, hva, write)) { + /* Check page level about host mmu*/ level = host_pfn_mapping_level(kvm, gfn, memslot); + if (level == 1) { + /* + * Check page level about secondary mmu + * Disable hugepage if it is normal page on + * secondary mmu already + */ + ptep = kvm_populate_gpa(kvm, NULL, gpa, 0); + if (ptep && !kvm_pte_huge(*ptep)) + level = 0; + } + if (level == 1) { gfn = gfn & ~(PTRS_PER_PTE - 1); pfn = pfn & ~(PTRS_PER_PTE - 1); From patchwork Mon Jun 24 07:14:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13709066 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A8CA03C684; Mon, 24 Jun 2024 07:14:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213269; cv=none; b=f7/KPaiuD42TNoI2tpGsnFxDdZ1jitwvJ1MtMiUFP9m1ZocRrmz7QlZIt3cSrxKIJUpLWQZaovHB1G1cRNZs+6oSihbrr6oPESGt/ixEBDa6+d3hQjGJOsY1X2kBy/6eNarIkdlyMaVm9e4rLjyjOxc2WZegicORs8Cjr1BrqUY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213269; c=relaxed/simple; bh=p4K7YFd9BlwS4GqQ12d7WLwQ1xvfFQEqslZSuqL+560=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XSBjbGkIyguy9kCiHVUeZMYxqmaPE9emmtrmEQfA47CKI0+vKxcLHP9NcRf0K6LbGCcatFkCqDNPgVz/zuOAuSFggA2ol5fcfCBGg94jglSrRcBEo/Hw0jVyXPc4Hdfq7sPf1yHrn1nfuo3ft3fHMIcUke8eZuVLZ5DSjhB29Wg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8Ax3erQHHlmEXEJAA--.38154S3; Mon, 24 Jun 2024 15:14:24 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxMMTPHHlmftsuAA--.9847S5; Mon, 24 Jun 2024 15:14:24 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, WANG Rui Subject: [PATCH v3 3/7] LoongArch: KVM: Discard dirty page tracking on readonly memslot Date: Mon, 24 Jun 2024 15:14:18 +0800 Message-Id: <20240624071422.3473789-4-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240624071422.3473789-1-maobibo@loongson.cn> References: <20240624071422.3473789-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxMMTPHHlmftsuAA--.9847S5 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== For readonly memslot such as UEFI bios or UEFI var space, guest can not write this memory space directly. So it is not necessary to track dirty pages for readonly memslot. Here there is such optimization in function kvm_arch_commit_memory_region(). Signed-off-by: Bibo Mao --- arch/loongarch/kvm/mmu.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index c6351d13ca1b..1690828bd44b 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -444,6 +444,17 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, enum kvm_mr_change change) { int needs_flush; + u32 old_flags = old ? old->flags : 0; + u32 new_flags = new ? new->flags : 0; + bool log_dirty_pages = new_flags & KVM_MEM_LOG_DIRTY_PAGES; + + /* only track memslot flags changed */ + if (change != KVM_MR_FLAGS_ONLY) + return; + + /* Discard dirty page tracking on readonly memslot */ + if ((old_flags & new_flags) & KVM_MEM_READONLY) + return; /* * If dirty page logging is enabled, write protect all pages in the slot @@ -454,9 +465,7 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, * MOVE/DELETE: The old mappings will already have been cleaned up by * kvm_arch_flush_shadow_memslot() */ - if (change == KVM_MR_FLAGS_ONLY && - (!(old->flags & KVM_MEM_LOG_DIRTY_PAGES) && - new->flags & KVM_MEM_LOG_DIRTY_PAGES)) { + if (!(old_flags & KVM_MEM_LOG_DIRTY_PAGES) && log_dirty_pages) { spin_lock(&kvm->mmu_lock); /* Write protect GPA page table entries */ needs_flush = kvm_mkclean_gpa_pt(kvm, new->base_gfn, From patchwork Mon Jun 24 07:14:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13709063 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 442C43EA83; Mon, 24 Jun 2024 07:14:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213268; cv=none; b=Wi0EhHtIcDGz55VgryuG1u9mFdFYJrA8lkKqrweYNIOZx/VpabVS5T7rpxgiOzQw2n8dGm/mfHaZR5KVodspdVQf9DABUMDaK6YK624xaaK/j3psiNEZknv074EINktDaNzd4cz3Mx3+qvJGBvg+3Arc+Z1BYpQq5r60UF87KNQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213268; c=relaxed/simple; bh=HWbNIuQgn/nRTB1yXhow8PHsF0bgZs2f2FXGivWWCDE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=FrwOQl+5JcKKaKOGpy46wczijZBDtLnZvXwMmfwhmjO64jzlu07/dn4DqPmebFKEYw4YPikjBKe1EqGeljLFNm9xcvJhbngU30/veLKsZKSa78ivyfkJODDU7d+GQjtEq07PSyEMMX/kzeiO2Ho/rnbtgDQP/kWbhzkA0IhZZVg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8CxcPDRHHlmFnEJAA--.38408S3; Mon, 24 Jun 2024 15:14:25 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxMMTPHHlmftsuAA--.9847S6; Mon, 24 Jun 2024 15:14:24 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, WANG Rui Subject: [PATCH v3 4/7] LoongArch: KVM: Add memory barrier before update pmd entry Date: Mon, 24 Jun 2024 15:14:19 +0800 Message-Id: <20240624071422.3473789-5-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240624071422.3473789-1-maobibo@loongson.cn> References: <20240624071422.3473789-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxMMTPHHlmftsuAA--.9847S6 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== When updating pmd entry such as allocating new pmd page or splitting huge page into normal page, it is necessary to firstly update all pte entries, and then update pmd entry. It is weak order with LoongArch system, there will be problem if other vcpus sees pmd update firstly however pte is not updated. Here smp_wmb() is added to assure this. Signed-off-by: Bibo Mao --- arch/loongarch/kvm/mmu.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 1690828bd44b..092c063e4e56 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -163,6 +163,7 @@ static kvm_pte_t *kvm_populate_gpa(struct kvm *kvm, child = kvm_mmu_memory_cache_alloc(cache); _kvm_pte_init(child, ctx.invalid_ptes[ctx.level - 1]); + smp_wmb(); /* make pte visible before pmd */ kvm_set_pte(entry, __pa(child)); } else if (kvm_pte_huge(*entry)) { return entry; @@ -746,6 +747,7 @@ static kvm_pte_t *kvm_split_huge(struct kvm_vcpu *vcpu, kvm_pte_t *ptep, gfn_t g val += PAGE_SIZE; } + smp_wmb(); /* make pte visible before pmd */ /* The later kvm_flush_tlb_gpa() will flush hugepage tlb */ kvm_set_pte(ptep, __pa(child)); From patchwork Mon Jun 24 07:14:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13709069 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 01832134B0; Mon, 24 Jun 2024 07:15:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213331; cv=none; b=FVo2181YebMv5RM1qLEx3Q+Ww3LAbkK9UGJzcPX7KRVpLLzttt+sXXNiwIa02s2r5UEDxsAJloXMnlpU/Pk/wBV7j0Mw558HjJwD/IDr8IBLGuoNFmJHDMsboLS2FrSdT4GHhC+gJgAgN1BZup0XZZBGhJqQS5YvsRcWsbn6FEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213331; c=relaxed/simple; bh=s5RP6fAfPMX36ZkNaZ8PMuFgshW+se53erDIX1Zn0RM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pyY8AFxiRnO5QrsXyz+mTKsNxewaUAcbzdAvpRaXbN9ipiN8aWMzab7U/Trl1LBW+W/K5sh5rX6coxmvnhpgI8LDDeK9talIfZl8Ih3tABgA+9/X/mc1XT7+5H7W/EJtnqxckC9AT72zSINhrHYItPfzxsn2QWBfSPBqzezU36A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8BxLuvRHHlmHHEJAA--.37947S3; Mon, 24 Jun 2024 15:14:25 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxMMTPHHlmftsuAA--.9847S7; Mon, 24 Jun 2024 15:14:24 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, WANG Rui Subject: [PATCH v3 5/7] LoongArch: KVM: Add dirty bitmap initially all set support Date: Mon, 24 Jun 2024 15:14:20 +0800 Message-Id: <20240624071422.3473789-6-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240624071422.3473789-1-maobibo@loongson.cn> References: <20240624071422.3473789-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxMMTPHHlmftsuAA--.9847S7 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== Add KVM_DIRTY_LOG_INITIALLY_SET support on LoongArch system, this feature comes from other architectures like x86 and arm64. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/kvm_host.h | 3 +++ arch/loongarch/kvm/mmu.c | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h index 32c4948f534f..309ce329b747 100644 --- a/arch/loongarch/include/asm/kvm_host.h +++ b/arch/loongarch/include/asm/kvm_host.h @@ -37,6 +37,9 @@ #define KVM_GUESTDBG_VALID_MASK \ (KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_USE_SW_BP | KVM_GUESTDBG_SINGLESTEP) +#define KVM_DIRTY_LOG_MANUAL_CAPS \ + (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | KVM_DIRTY_LOG_INITIALLY_SET) + struct kvm_vm_stat { struct kvm_vm_stat_generic generic; u64 pages; diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 092c063e4e56..10da8a63cd3d 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -467,6 +467,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, * kvm_arch_flush_shadow_memslot() */ if (!(old_flags & KVM_MEM_LOG_DIRTY_PAGES) && log_dirty_pages) { + /* + * Initially-all-set does not require write protecting any page + * because they're all assumed to be dirty. + */ + if (kvm_dirty_log_manual_protect_and_init_set(kvm)) + return; + spin_lock(&kvm->mmu_lock); /* Write protect GPA page table entries */ needs_flush = kvm_mkclean_gpa_pt(kvm, new->base_gfn, From patchwork Mon Jun 24 07:14:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13709067 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6A0AD446A2; Mon, 24 Jun 2024 07:14:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213269; cv=none; b=XL/D0WoufYEgZpZV9AVaf4NJr7sKmZhwGj7nWsrAHrntszJx1qLpEbBzvAnyD5Q8FPKtFAtn5sqkrlPkP4nJavPGS6lY554Ez+KFcevuim4eSw+0v0RYSXC8qEbeUfY7+arHbBnBW67DIl5wHVGLtf7CwT7H3xnMlAzA9PdAg8c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213269; c=relaxed/simple; bh=UhN2A2X3VUnlt0siqa6ZdR4twIOdjvhLcmCgzISf9Lk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RJTDCHOM1eMClsclUrb18udd19qj++l8sapou8dNzWeYR818Sv1ctPRpIbJbnv+eR+HwjXx2O7tbboZerO2C5y3EakLKmsd4gM+Q8NPdhDxmE6L7aNEmHJMoRAC19WD9LbuCj4UlcEDP2cL4HWiAuwkerVIrZ7JCZYTpOlGln4g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8Bx3+vRHHlmIXEJAA--.22281S3; Mon, 24 Jun 2024 15:14:25 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxMMTPHHlmftsuAA--.9847S8; Mon, 24 Jun 2024 15:14:25 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, WANG Rui Subject: [PATCH v3 6/7] LoongArch: KVM: Mark page accessed and dirty with page ref added Date: Mon, 24 Jun 2024 15:14:21 +0800 Message-Id: <20240624071422.3473789-7-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240624071422.3473789-1-maobibo@loongson.cn> References: <20240624071422.3473789-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxMMTPHHlmftsuAA--.9847S8 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== Function kvm_map_page_fast() is fast path of secondary mmu page fault flow, pfn is parsed from secondary mmu page table walker. However the corresponding page reference is not added, it is dangerious to access page out of mmu_lock. Here page ref is added inside mmu_lock, function kvm_set_pfn_accessed() and kvm_set_pfn_dirty() is called with page ref added, so that the page will not be freed by others. Also kvm_set_pfn_accessed() is removed here since it is called in the following function kvm_release_pfn_clean(). Signed-off-by: Bibo Mao --- arch/loongarch/kvm/mmu.c | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 10da8a63cd3d..7f744e6679a8 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -557,6 +557,7 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ gfn_t gfn = gpa >> PAGE_SHIFT; struct kvm *kvm = vcpu->kvm; struct kvm_memory_slot *slot; + struct page *page; spin_lock(&kvm->mmu_lock); @@ -599,19 +600,22 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ if (changed) { kvm_set_pte(ptep, new); pfn = kvm_pte_pfn(new); + page = kvm_pfn_to_refcounted_page(pfn); + if (page) + get_page(page); } spin_unlock(&kvm->mmu_lock); - /* - * Fixme: pfn may be freed after mmu_lock - * kvm_try_get_pfn(pfn)/kvm_release_pfn pair to prevent this? - */ - if (kvm_pte_young(changed)) - kvm_set_pfn_accessed(pfn); + if (changed) { + if (kvm_pte_young(changed)) + kvm_set_pfn_accessed(pfn); - if (kvm_pte_dirty(changed)) { - mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); + if (kvm_pte_dirty(changed)) { + mark_page_dirty(kvm, gfn); + kvm_set_pfn_dirty(pfn); + } + if (page) + put_page(page); } return ret; out: @@ -920,7 +924,6 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) kvm_set_pfn_dirty(pfn); } - kvm_set_pfn_accessed(pfn); kvm_release_pfn_clean(pfn); out: srcu_read_unlock(&kvm->srcu, srcu_idx); From patchwork Mon Jun 24 07:14:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13709068 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 017F23D393; Mon, 24 Jun 2024 07:15:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213331; cv=none; b=i6TGR1BRJeyRAKso3GuO5xaSZscJknjDkJNTQGF9Yil9D/0/DgvRs4OVJ17nLpbv+oNkVI379kArk5hF9CFeEJ3Ji1NDlKhiiA6shhj7SRBcl7WbyaVBVJ1dOxAdiwp8Z0Qtk8B42TrvNgw5Zoy1cFkWVXlKjffPG5MVw//hy7s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719213331; c=relaxed/simple; bh=tMij1kigYbysiBVjoK1zx02otlNc/5BcBakZSzod/Sw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZwEDr14GrPtlFXRdpVda3Zki6af2tCVKi+MkF+oUassSJ9xGPX6zy19FzObHqgw6sQB1pqo79pZC082WgzuJ8wHRLNLdqwpWV1v30ezMZRh47XeHmqlIQL2O8RuyoK0UNs8ENQRqNgkG/Dn81rXPHkSUANEUmImQGC9tTiBoEzo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8AxnOrSHHlmJnEJAA--.37961S3; Mon, 24 Jun 2024 15:14:26 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxMMTPHHlmftsuAA--.9847S9; Mon, 24 Jun 2024 15:14:25 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, WANG Rui Subject: [PATCH v3 7/7] LoongArch: KVM: Sync pending interrupt when getting ESTAT from user mode Date: Mon, 24 Jun 2024 15:14:22 +0800 Message-Id: <20240624071422.3473789-8-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240624071422.3473789-1-maobibo@loongson.cn> References: <20240624071422.3473789-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxMMTPHHlmftsuAA--.9847S9 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== Currently interrupt is posted and cleared with asynchronous mode, and it is saved in SW state vcpu::arch::irq_pending and vcpu::arch::irq_clear. When vcpu is ready to run, pending interrupt is written back to ESTAT CSR register from SW state vcpu::arch::irq_pending at guest entrance. During VM migration stage, vcpu is put into stopped state, however pending interrupt is not synced to ESTAT CSR register. So there will be interrupt lost when VCPU is migrated to other host machines. Here when ESTAT CSR register is read from VMM user mode, pending interrupt is synchronized to ESTAT also. So that vcpu can get correct pending interrupt. Signed-off-by: Bibo Mao --- arch/loongarch/kvm/vcpu.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c index b747bd8bc037..6b612b8390f7 100644 --- a/arch/loongarch/kvm/vcpu.c +++ b/arch/loongarch/kvm/vcpu.c @@ -371,6 +371,17 @@ static int _kvm_getcsr(struct kvm_vcpu *vcpu, unsigned int id, u64 *val) return -EINVAL; if (id == LOONGARCH_CSR_ESTAT) { + preempt_disable(); + vcpu_load(vcpu); + /* + * Sync pending interrupt into estat so that interrupt + * remains during migration stage + */ + kvm_deliver_intr(vcpu); + vcpu->arch.aux_inuse &= ~KVM_LARCH_SWCSR_LATEST; + vcpu_put(vcpu); + preempt_enable(); + /* ESTAT IP0~IP7 get from GINTC */ gintc = kvm_read_sw_gcsr(csr, LOONGARCH_CSR_GINTC) & 0xff; *val = kvm_read_sw_gcsr(csr, LOONGARCH_CSR_ESTAT) | (gintc << 2);