From patchwork Wed Jun 19 08:09:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13703506 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6353E78B50; Wed, 19 Jun 2024 08:09:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784592; cv=none; b=cEpmhyfUxdB2TpDvEn19/X0NyJKyLQk5kq+j64CbowHB7xdTN1561COYKH9yXhK0XyYsLI9Y96n5MPQcPyPi/DFchyjwcL+01i0eoZO5LRmIKc5oX5ce/PxJmVf5xn1fMYt0wYNU+dtiTTuLlWgmm2X8SF40RPyPvFT7yNWyMAo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784592; c=relaxed/simple; bh=JGqeHtnAtlaENJv5Lw+Iz95YDDPb7H8pltTHhLYoDwI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AtVUgwqt/Io54p4qsZLr5zAkWK3SWpSo34JsWhHv8xlcq+CvAF8RxZxkGAuyqNCzaTfz6QwwaDF4q19gL6REt/K755tuQEESugxXiIfY914PLUTkX3FqZIV/RC7Yc30UIIyEjqzzMYSSBy9oTHfGpEOsqIGrTNFhbdUwfI8gzak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8CxL_BFknJm4i4IAA--.33034S3; Wed, 19 Jun 2024 16:09:41 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8AxjsdEknJmFeMoAA--.32907S3; Wed, 19 Jun 2024 16:09:41 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/6] LoongArch: KVM: Delay secondary mmu tlb flush until guest entry Date: Wed, 19 Jun 2024 16:09:35 +0800 Message-Id: <20240619080940.2690756-2-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240619080940.2690756-1-maobibo@loongson.cn> References: <20240619080940.2690756-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8AxjsdEknJmFeMoAA--.32907S3 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== If there is page fault for secondary mmu, there needs tlb flush operation indexed with fault gpa address and VMID. VMID is stored at register CSR_GSTAT and will be reload or recalculated during guest entry. Currently CSR_GSTAT is not saved and restored during vcpu context switch, it is recalculated during guest entry. So CSR_GSTAT is in effect only when vcpu runs in guest mode, however it may be not in effected if vcpu exits to host mode, since register CSR_GSTAT may be stale, it maybe records VMID of last scheduled vcpu, rather than current vcpu. Function kvm_flush_tlb_gpa() should be called with its real VMID, here move it to guest entrance. Also arch specific request id KVM_REQ_TLB_FLUSH_GPA is added to flush tlb, and it can be optimized if VMID is updated, since all guest tlb entries will be invalid if VMID is updated. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/kvm_host.h | 2 ++ arch/loongarch/kvm/main.c | 1 + arch/loongarch/kvm/mmu.c | 4 ++-- arch/loongarch/kvm/tlb.c | 5 +---- arch/loongarch/kvm/vcpu.c | 18 ++++++++++++++++++ 5 files changed, 24 insertions(+), 6 deletions(-) diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h index c87b6ea0ec47..32c4948f534f 100644 --- a/arch/loongarch/include/asm/kvm_host.h +++ b/arch/loongarch/include/asm/kvm_host.h @@ -30,6 +30,7 @@ #define KVM_PRIVATE_MEM_SLOTS 0 #define KVM_HALT_POLL_NS_DEFAULT 500000 +#define KVM_REQ_TLB_FLUSH_GPA KVM_ARCH_REQ(0) #define KVM_GUESTDBG_SW_BP_MASK \ (KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_USE_SW_BP) @@ -190,6 +191,7 @@ struct kvm_vcpu_arch { /* vcpu's vpid */ u64 vpid; + gpa_t flush_gpa; /* Frequency of stable timer in Hz */ u64 timer_mhz; diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c index 86a2f2d0cb27..844736b99d38 100644 --- a/arch/loongarch/kvm/main.c +++ b/arch/loongarch/kvm/main.c @@ -242,6 +242,7 @@ void kvm_check_vpid(struct kvm_vcpu *vcpu) kvm_update_vpid(vcpu, cpu); trace_kvm_vpid_change(vcpu, vcpu->arch.vpid); vcpu->cpu = cpu; + kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu); } /* Restore GSTAT(0x50).vpid */ diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 98883aa23ab8..9e39d28fec35 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -908,8 +908,8 @@ int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) return ret; /* Invalidate this entry in the TLB */ - kvm_flush_tlb_gpa(vcpu, gpa); - + vcpu->arch.flush_gpa = gpa; + kvm_make_request(KVM_REQ_TLB_FLUSH_GPA, vcpu); return 0; } diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c index 02535df6b51f..ebdbe9264e9c 100644 --- a/arch/loongarch/kvm/tlb.c +++ b/arch/loongarch/kvm/tlb.c @@ -23,10 +23,7 @@ void kvm_flush_tlb_all(void) void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa) { - unsigned long flags; - - local_irq_save(flags); + lockdep_assert_irqs_disabled(); gpa &= (PAGE_MASK << 1); invtlb(INVTLB_GID_ADDR, read_csr_gstat() & CSR_GSTAT_GID, gpa); - local_irq_restore(flags); } diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c index 9e8030d45129..b747bd8bc037 100644 --- a/arch/loongarch/kvm/vcpu.c +++ b/arch/loongarch/kvm/vcpu.c @@ -51,6 +51,16 @@ static int kvm_check_requests(struct kvm_vcpu *vcpu) return RESUME_GUEST; } +static void kvm_late_check_requests(struct kvm_vcpu *vcpu) +{ + lockdep_assert_irqs_disabled(); + if (kvm_check_request(KVM_REQ_TLB_FLUSH_GPA, vcpu)) + if (vcpu->arch.flush_gpa != INVALID_GPA) { + kvm_flush_tlb_gpa(vcpu, vcpu->arch.flush_gpa); + vcpu->arch.flush_gpa = INVALID_GPA; + } +} + /* * Check and handle pending signal and vCPU requests etc * Run with irq enabled and preempt enabled @@ -101,6 +111,13 @@ static int kvm_pre_enter_guest(struct kvm_vcpu *vcpu) /* Make sure the vcpu mode has been written */ smp_store_mb(vcpu->mode, IN_GUEST_MODE); kvm_check_vpid(vcpu); + + /* + * Called after function kvm_check_vpid() + * Since it updates csr_gstat used by kvm_flush_tlb_gpa(), + * also it may clear KVM_REQ_TLB_FLUSH_GPA pending bit + */ + kvm_late_check_requests(vcpu); vcpu->arch.host_eentry = csr_read64(LOONGARCH_CSR_EENTRY); /* Clear KVM_LARCH_SWCSR_LATEST as CSR will change when enter guest */ vcpu->arch.aux_inuse &= ~KVM_LARCH_SWCSR_LATEST; @@ -994,6 +1011,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) struct loongarch_csrs *csr; vcpu->arch.vpid = 0; + vcpu->arch.flush_gpa = INVALID_GPA; hrtimer_init(&vcpu->arch.swtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_PINNED); vcpu->arch.swtimer.function = kvm_swtimer_wakeup; From patchwork Wed Jun 19 08:09:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13703508 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B122378B4E; Wed, 19 Jun 2024 08:09:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784592; cv=none; b=cF0PMPyNocqNMBfjl+70yCPUqKNJigQePUSgPkHn5y8Qfi7pyR4zddCkGovOfBt4iROWU2t4dMr81KLi2LrN4dRCt58FGnxeP7fFkVzDWYgCte83c3PR+iXxwHiV0tj4qG9lMyMKwyrT1xu1eYsA7DGL5M3Zg3OoWRMVpGN60A8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784592; c=relaxed/simple; bh=VRGsuynlLOX3I/Mr6W6BKVr5Su0ws2ApaNc7yn8Faz8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uq/Iqm62hEKaPzzGpFLsFE6b0fGLO+buNC7bv8Cl+ASlG3TOgf3Ar7tM7nO4swMphHStia3W1Njo/Hd7fSQAAEUrPXjGy7uWCdRVh9vFtL0KMW/8sjBlSeNRa6k7KTjuXM02U0i/lwPoDVWdf+RGADWjebk1QNFw72XCLiAU6A8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8BxrOpGknJm5y4IAA--.32926S3; Wed, 19 Jun 2024 16:09:42 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8AxjsdEknJmFeMoAA--.32907S4; Wed, 19 Jun 2024 16:09:41 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/6] LoongArch: KVM: Select huge page only if secondary mmu supports it Date: Wed, 19 Jun 2024 16:09:36 +0800 Message-Id: <20240619080940.2690756-3-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240619080940.2690756-1-maobibo@loongson.cn> References: <20240619080940.2690756-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8AxjsdEknJmFeMoAA--.32907S4 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== Currently page level selection about secondary mmu depends on memory slot and page level about host mmu. There will be problem if page level of secondary mmu is zero already. So page level selection should depend on the following three conditions. 1. Memslot is aligned for huge page and vm is not migrating. 2. Page level of host mmu is huge page also. 3. Page level of secondary mmu is suituable for huge page, it cannot be normal page since it is not supported to merge normal pages into huge page now. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/kvm_mmu.h | 2 +- arch/loongarch/kvm/mmu.c | 16 +++++++++++++--- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/loongarch/include/asm/kvm_mmu.h b/arch/loongarch/include/asm/kvm_mmu.h index 099bafc6f797..d06ae0e0dde5 100644 --- a/arch/loongarch/include/asm/kvm_mmu.h +++ b/arch/loongarch/include/asm/kvm_mmu.h @@ -55,7 +55,7 @@ static inline void kvm_set_pte(kvm_pte_t *ptep, kvm_pte_t val) static inline int kvm_pte_write(kvm_pte_t pte) { return pte & _PAGE_WRITE; } static inline int kvm_pte_dirty(kvm_pte_t pte) { return pte & _PAGE_DIRTY; } static inline int kvm_pte_young(kvm_pte_t pte) { return pte & _PAGE_ACCESSED; } -static inline int kvm_pte_huge(kvm_pte_t pte) { return pte & _PAGE_HUGE; } +static inline int kvm_pte_huge(kvm_pte_t pte) { return !!(pte & _PAGE_HUGE); } static inline kvm_pte_t kvm_pte_mkyoung(kvm_pte_t pte) { diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 9e39d28fec35..c6351d13ca1b 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -858,10 +858,20 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) /* Disable dirty logging on HugePages */ level = 0; - if (!fault_supports_huge_mapping(memslot, hva, write)) { - level = 0; - } else { + if (fault_supports_huge_mapping(memslot, hva, write)) { + /* Check page level about host mmu*/ level = host_pfn_mapping_level(kvm, gfn, memslot); + if (level == 1) { + /* + * Check page level about secondary mmu + * Disable hugepage if it is normal page on + * secondary mmu already + */ + ptep = kvm_populate_gpa(kvm, NULL, gpa, 0); + if (ptep && !kvm_pte_huge(*ptep)) + level = 0; + } + if (level == 1) { gfn = gfn & ~(PTRS_PER_PTE - 1); pfn = pfn & ~(PTRS_PER_PTE - 1); From patchwork Wed Jun 19 08:09:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13703507 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A999178C64; Wed, 19 Jun 2024 08:09:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784592; cv=none; b=B6woQq5RpKY+RHEuidC/h1bXT9HZpSO+0rpm0KxYXvbjhyUPaxntq5sO++cnLXnL4rPPUT8Ha7JuxmRT1G+8of/NaakhKUi6JVBJkbUfjPffvqJ7589kYzSo7g+SLc08ivd95Jl/y1aC0VP+m4WkHVgO+ypT08qrspPnBkogEdE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784592; c=relaxed/simple; bh=p4K7YFd9BlwS4GqQ12d7WLwQ1xvfFQEqslZSuqL+560=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IFvPIjgV2HnjIrvHRNfGMHJDBzJdcMFrK1mtyOW/LE+GhiwJ4YSgcS0VuBsVoHPDHZM49q38c8eAWLM7XLmFd4yrdtVv5fpb0n/wglDgMwq5a0o/kjSHBkXX6FfeCHrGNJIiQzye6vXxoe7HJcCTCI0Ly8LFH6tVUjaqZsWT3GQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8Cx_epGknJm7C4IAA--.32789S3; Wed, 19 Jun 2024 16:09:42 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8AxjsdEknJmFeMoAA--.32907S5; Wed, 19 Jun 2024 16:09:42 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/6] LoongArch: KVM: Discard dirty page tracking on readonly memslot Date: Wed, 19 Jun 2024 16:09:37 +0800 Message-Id: <20240619080940.2690756-4-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240619080940.2690756-1-maobibo@loongson.cn> References: <20240619080940.2690756-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8AxjsdEknJmFeMoAA--.32907S5 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== For readonly memslot such as UEFI bios or UEFI var space, guest can not write this memory space directly. So it is not necessary to track dirty pages for readonly memslot. Here there is such optimization in function kvm_arch_commit_memory_region(). Signed-off-by: Bibo Mao --- arch/loongarch/kvm/mmu.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index c6351d13ca1b..1690828bd44b 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -444,6 +444,17 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, enum kvm_mr_change change) { int needs_flush; + u32 old_flags = old ? old->flags : 0; + u32 new_flags = new ? new->flags : 0; + bool log_dirty_pages = new_flags & KVM_MEM_LOG_DIRTY_PAGES; + + /* only track memslot flags changed */ + if (change != KVM_MR_FLAGS_ONLY) + return; + + /* Discard dirty page tracking on readonly memslot */ + if ((old_flags & new_flags) & KVM_MEM_READONLY) + return; /* * If dirty page logging is enabled, write protect all pages in the slot @@ -454,9 +465,7 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, * MOVE/DELETE: The old mappings will already have been cleaned up by * kvm_arch_flush_shadow_memslot() */ - if (change == KVM_MR_FLAGS_ONLY && - (!(old->flags & KVM_MEM_LOG_DIRTY_PAGES) && - new->flags & KVM_MEM_LOG_DIRTY_PAGES)) { + if (!(old_flags & KVM_MEM_LOG_DIRTY_PAGES) && log_dirty_pages) { spin_lock(&kvm->mmu_lock); /* Write protect GPA page table entries */ needs_flush = kvm_mkclean_gpa_pt(kvm, new->base_gfn, From patchwork Wed Jun 19 08:09:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13703509 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AD8DF78C66; Wed, 19 Jun 2024 08:09:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784592; cv=none; b=Z8h8pTGNviNMo/RBmkT/f4vUHJWRKJWw6Rpwtp6dIzfhK6LkTREiXFClRTU6iuQJ1Dzq4gRfIrpXwKXbE5iboHNw9ccj2srXQCGcfmeMdGV3gqoxOaaojtiYO4Gmel5X4xsciQXJtLDkRPx72W7gpTEsE3tsR/XBaZcBD0NkhFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784592; c=relaxed/simple; bh=y7i1eBvpBVWltUd1vbvl1kJJ1lAJm5onvJ/ngz+QX00=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BHkAOz2J+6tXjx54ztT7JlIn6xnFsIyB/ktJG8aPavHwl8h7bK6DryanSXPjjmTcRyL3gxN82EudObV5AMZfFch41wHCyiVogKKBMxzTAxUgHVvpGJAp9ItRE7YqetEyUIR5Xdw0YyynX22yjU0g148TIkuZnCUr9TDGXqMdC84= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8DxP_BGknJm8S4IAA--.33122S3; Wed, 19 Jun 2024 16:09:42 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8AxjsdEknJmFeMoAA--.32907S6; Wed, 19 Jun 2024 16:09:42 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 4/6] LoongArch: KVM: Add memory barrier before update pmd entry Date: Wed, 19 Jun 2024 16:09:38 +0800 Message-Id: <20240619080940.2690756-5-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240619080940.2690756-1-maobibo@loongson.cn> References: <20240619080940.2690756-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8AxjsdEknJmFeMoAA--.32907S6 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== When updating pmd entry such as allocating new pmd page or splitting huge page into normal page, it is necessary to firstly update all pte entries, and then update pmd entry. It is weak order with LoongArch system, there will be problem if other vcpus sees pmd update firstly however pte is not updated. Here smp_wmb() is added to assure this. Signed-off-by: Bibo Mao --- arch/loongarch/kvm/mmu.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 1690828bd44b..7f04edfbe428 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -163,6 +163,7 @@ static kvm_pte_t *kvm_populate_gpa(struct kvm *kvm, child = kvm_mmu_memory_cache_alloc(cache); _kvm_pte_init(child, ctx.invalid_ptes[ctx.level - 1]); + smp_wmb(); /* make pte visible before pmd */ kvm_set_pte(entry, __pa(child)); } else if (kvm_pte_huge(*entry)) { return entry; @@ -746,6 +747,7 @@ static kvm_pte_t *kvm_split_huge(struct kvm_vcpu *vcpu, kvm_pte_t *ptep, gfn_t g val += PAGE_SIZE; } + smp_wmb(); /* The later kvm_flush_tlb_gpa() will flush hugepage tlb */ kvm_set_pte(ptep, __pa(child)); From patchwork Wed Jun 19 08:09:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13703511 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BA26B78C6B; Wed, 19 Jun 2024 08:09:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784593; cv=none; b=X9fq+TytPTRiGSsy0tPOelas49r7tSufq0TRUQ23xtauHmqZl7llNXMyZrAKwuMlP0l9cBZ+YStLaSyiDRQZI+OkU8k0c6KUyPwCYERXKnArPYSjS7HS0OUdEFMoiwYj7+rRRb3B5EV/viysK0xXRrsVZE5hrzkcQkfQPMZRY80= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784593; c=relaxed/simple; bh=z9uXT3mjp7dVxjD9jHEieTpNwIbwtbk+ig9PZbKnsfg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UIB5RThr3JtE+AAzxvpqs4JsxRFc7WcvzBvHNoHlTBVdLTtFYAaZhFFpN92M4n+PmcDPh3TObFvgfQS2kiDo/ZgmngFuF/XzYIeNztf6Q4ngnVbnkiCq3iVkHn/CZWTbKiUWzXO+BaeXNsod9M3EgWAtEo7HEC7q9/bzNFmowAA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8Bx3+tHknJm9i4IAA--.17084S3; Wed, 19 Jun 2024 16:09:43 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8AxjsdEknJmFeMoAA--.32907S7; Wed, 19 Jun 2024 16:09:42 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 5/6] LoongArch: KVM: Add dirty bitmap initially all set support Date: Wed, 19 Jun 2024 16:09:39 +0800 Message-Id: <20240619080940.2690756-6-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240619080940.2690756-1-maobibo@loongson.cn> References: <20240619080940.2690756-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8AxjsdEknJmFeMoAA--.32907S7 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== Add KVM_DIRTY_LOG_INITIALLY_SET support on LoongArch system, this feature comes from other architectures like x86 and arm64. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/kvm_host.h | 3 +++ arch/loongarch/kvm/mmu.c | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h index 32c4948f534f..309ce329b747 100644 --- a/arch/loongarch/include/asm/kvm_host.h +++ b/arch/loongarch/include/asm/kvm_host.h @@ -37,6 +37,9 @@ #define KVM_GUESTDBG_VALID_MASK \ (KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_USE_SW_BP | KVM_GUESTDBG_SINGLESTEP) +#define KVM_DIRTY_LOG_MANUAL_CAPS \ + (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | KVM_DIRTY_LOG_INITIALLY_SET) + struct kvm_vm_stat { struct kvm_vm_stat_generic generic; u64 pages; diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 7f04edfbe428..3b862f3a72cb 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -467,6 +467,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, * kvm_arch_flush_shadow_memslot() */ if (!(old_flags & KVM_MEM_LOG_DIRTY_PAGES) && log_dirty_pages) { + /* + * Initially-all-set does not require write protecting any page + * because they're all assumed to be dirty. + */ + if (kvm_dirty_log_manual_protect_and_init_set(kvm)) + return; + spin_lock(&kvm->mmu_lock); /* Write protect GPA page table entries */ needs_flush = kvm_mkclean_gpa_pt(kvm, new->base_gfn, From patchwork Wed Jun 19 08:09:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bibo Mao X-Patchwork-Id: 13703512 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 39C9378C62; Wed, 19 Jun 2024 08:11:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=114.242.206.163 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784676; cv=none; b=DW60J+kcnyUWxvV7oC3GLDVMkPb70RdEL82nC6Pngz4JH+DtIV5wvnj6GcvEIHsPBQ9fjFT9++RKQzWxi/mWWPyvHyNF51DB+ylEeR6DpVRFI/rkJngVWjQl4mWLPBo5UdC5tXVDaX7bqPmx3Eo3eoNegJpXQE9qNxxZMhgh+yw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718784676; c=relaxed/simple; bh=Dj4caGZD85XgIzLrQI5RAXgZ3XwkUwK6rN5stW+Ojfw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qnuchPHqJitMtXkN7zjjq/6c9/Op7on0m/Epp+TsIzgeSR/90xyVEgZJ3V4griEmfPsT9OAX/dV71g0oPyCWJON8go0v7tIhuesE6Omb/5ghgT/MXCi5xzQQ6u3XmijZ9oXotrJ254xlFkbT1m/62GZGfF/0USdZlC8f0Exr2bs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn; spf=pass smtp.mailfrom=loongson.cn; arc=none smtp.client-ip=114.242.206.163 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8DxzOpHknJm+y4IAA--.33045S3; Wed, 19 Jun 2024 16:09:43 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by localhost.localdomain (Coremail) with SMTP id AQAAf8AxjsdEknJmFeMoAA--.32907S8; Wed, 19 Jun 2024 16:09:43 +0800 (CST) From: Bibo Mao To: Tianrui Zhao , Huacai Chen Cc: WANG Xuerui , Sean Christopherson , kvm@vger.kernel.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 6/6] LoongArch: KVM: Mark page accessed and dirty with page ref added Date: Wed, 19 Jun 2024 16:09:40 +0800 Message-Id: <20240619080940.2690756-7-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240619080940.2690756-1-maobibo@loongson.cn> References: <20240619080940.2690756-1-maobibo@loongson.cn> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: AQAAf8AxjsdEknJmFeMoAA--.32907S8 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== Function kvm_map_page_fast() is fast path of secondary mmu page fault flow, pfn is parsed from secondary mmu page table walker. However the corresponding page reference is not added, it is dangerious to access page out of mmu_lock. Here page ref is added inside mmu_lock, function kvm_set_pfn_accessed() and kvm_set_pfn_dirty() is called with page ref added, so that the page will not be freed by others. Also kvm_set_pfn_accessed() is removed here since it is called in the following function kvm_release_pfn_clean(). Signed-off-by: Bibo Mao --- arch/loongarch/kvm/mmu.c | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 3b862f3a72cb..5a820a81fd97 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -557,6 +557,7 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ gfn_t gfn = gpa >> PAGE_SHIFT; struct kvm *kvm = vcpu->kvm; struct kvm_memory_slot *slot; + struct page *page; spin_lock(&kvm->mmu_lock); @@ -599,19 +600,22 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ if (changed) { kvm_set_pte(ptep, new); pfn = kvm_pte_pfn(new); + page = kvm_pfn_to_refcounted_page(pfn); + if (page) + get_page(page); } spin_unlock(&kvm->mmu_lock); - /* - * Fixme: pfn may be freed after mmu_lock - * kvm_try_get_pfn(pfn)/kvm_release_pfn pair to prevent this? - */ - if (kvm_pte_young(changed)) - kvm_set_pfn_accessed(pfn); + if (changed) { + if (kvm_pte_young(changed)) + kvm_set_pfn_accessed(pfn); - if (kvm_pte_dirty(changed)) { - mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); + if (kvm_pte_dirty(changed)) { + mark_page_dirty(kvm, gfn); + kvm_set_pfn_dirty(pfn); + } + if (page) + put_page(page); } return ret; out: @@ -920,7 +924,6 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) kvm_set_pfn_dirty(pfn); } - kvm_set_pfn_accessed(pfn); kvm_release_pfn_clean(pfn); out: srcu_read_unlock(&kvm->srcu, srcu_idx);