From patchwork Wed Apr 14 06:51:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: zhukeqian X-Patchwork-Id: 12202041 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC8D5C433ED for ; Wed, 14 Apr 2021 06:54:21 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 72C3E6105A for ; Wed, 14 Apr 2021 06:54:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 72C3E6105A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID:Date: Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=oKqzoaSwzBrgZCXLMbltY8nOmvh2gKGq+7wHq5VBGe4=; b=P8+yqC7lHW3rrS+Y0YLvlgU2P Ur1yhLDYLMvmppVH58utNOtOkvjG/dZ2VehM2HzbAAi3v9sb5TobLgzkqeeU9ZbUdct8BY9NC/IXb 36ZMcr/4h+lGnYmChI3qHmQ46Nsmwsh+/tPne/k7q+2fe6UqbHM61fmi3pSf1mbBDhusJPYpHLsF6 QWnPobTsWRtDB4I21mAYPkTdfnxh5/zpibhwaOvlL/NAzHGYpuex7C9R4dkVyBhOm4nKyU9w5Cq8Q RU9mjQmvuHcNOb0VDNY6Jvv6pCOFhLiDJOU1bArLLdKCX+Wfd+IKhejdJEQ431Vqr8A4/UfQ48jNH dkUCg+Q2g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lWZOI-00Bjd1-EC; Wed, 14 Apr 2021 06:52:46 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWZNh-00BjXi-0I for linux-arm-kernel@desiato.infradead.org; Wed, 14 Apr 2021 06:52:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=dQU3TRYfiftfdDOKqxkBxs0ABqksq7ZJOY/kXQ3jrJ4=; b=pNjRxUsLL8flRzgNJSjfuXmGdF bZ4inh3AklGcHcwEY5WCycyhR3YBTSpRPnTl2wCTIq3tbqFHjPBpKeH0pmzVt4tYKhiQ1g6hlvMU/ 4bkIDWNKcPws0aEAwKRsRh7TzOYj4ojOY3LsucHDsL/1xJlsU47PJYEjSw11Vt1RFWzFabJvOaDbl gVTbUts9z00wYzRb2GezVsa+2VPG8zLauZR8DrKH3CtOdyF2WdKXWwDeDu6uo0FfnDuLnZGvF3Nhv gAL81Wj2wUtB5SZCamcPwYifP/TP3XnQMqa9FrBGPVsv+7PAZ4xvalJH4gu+UH19KMs57tihsfmiz T3AxofGQ==; Received: from szxga05-in.huawei.com ([45.249.212.191]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWZNe-007Xt1-1E for linux-arm-kernel@lists.infradead.org; Wed, 14 Apr 2021 06:52:07 +0000 Received: from DGGEMS402-HUB.china.huawei.com (unknown [172.30.72.58]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4FKtNz4ydKzNw04; Wed, 14 Apr 2021 14:49:07 +0800 (CST) Received: from DESKTOP-5IS4806.china.huawei.com (10.174.187.224) by DGGEMS402-HUB.china.huawei.com (10.3.19.202) with Microsoft SMTP Server id 14.3.498.0; Wed, 14 Apr 2021 14:51:54 +0800 From: Keqian Zhu To: , , , , Marc Zyngier CC: Subject: [PATCH v3 1/2] kvm/arm64: Remove the creation time's mapping of MMIO regions Date: Wed, 14 Apr 2021 14:51:08 +0800 Message-ID: <20210414065109.8616-2-zhukeqian1@huawei.com> X-Mailer: git-send-email 2.8.4.windows.1 In-Reply-To: <20210414065109.8616-1-zhukeqian1@huawei.com> References: <20210414065109.8616-1-zhukeqian1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.187.224] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210413_235206_266421_A69C451A X-CRM114-Status: GOOD ( 11.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The MMIO regions may be unmapped for many reasons and can be remapped by stage2 fault path. Map MMIO regions at creation time becomes a minor optimization and makes these two mapping path hard to sync. Remove the mapping code while keep the useful sanity check. Signed-off-by: Keqian Zhu --- arch/arm64/kvm/mmu.c | 38 +++----------------------------------- 1 file changed, 3 insertions(+), 35 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 8711894db8c2..c59af5ca01b0 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1301,7 +1301,6 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, { hva_t hva = mem->userspace_addr; hva_t reg_end = hva + mem->memory_size; - bool writable = !(mem->flags & KVM_MEM_READONLY); int ret = 0; if (change != KVM_MR_CREATE && change != KVM_MR_MOVE && @@ -1318,8 +1317,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, mmap_read_lock(current->mm); /* * A memory region could potentially cover multiple VMAs, and any holes - * between them, so iterate over all of them to find out if we can map - * any of them right now. + * between them, so iterate over all of them. * * +--------------------------------------------+ * +---------------+----------------+ +----------------+ @@ -1330,50 +1328,20 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, */ do { struct vm_area_struct *vma = find_vma(current->mm, hva); - hva_t vm_start, vm_end; if (!vma || vma->vm_start >= reg_end) break; - /* - * Take the intersection of this VMA with the memory region - */ - vm_start = max(hva, vma->vm_start); - vm_end = min(reg_end, vma->vm_end); - if (vma->vm_flags & VM_PFNMAP) { - gpa_t gpa = mem->guest_phys_addr + - (vm_start - mem->userspace_addr); - phys_addr_t pa; - - pa = (phys_addr_t)vma->vm_pgoff << PAGE_SHIFT; - pa += vm_start - vma->vm_start; - /* IO region dirty page logging not allowed */ if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { ret = -EINVAL; - goto out; - } - - ret = kvm_phys_addr_ioremap(kvm, gpa, pa, - vm_end - vm_start, - writable); - if (ret) break; + } } - hva = vm_end; + hva = min(reg_end, vma->vm_end); } while (hva < reg_end); - if (change == KVM_MR_FLAGS_ONLY) - goto out; - - spin_lock(&kvm->mmu_lock); - if (ret) - unmap_stage2_range(&kvm->arch.mmu, mem->guest_phys_addr, mem->memory_size); - else if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB)) - stage2_flush_memslot(kvm, memslot); - spin_unlock(&kvm->mmu_lock); -out: mmap_read_unlock(current->mm); return ret; } From patchwork Wed Apr 14 06:51:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: zhukeqian X-Patchwork-Id: 12202039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1703BC433ED for ; Wed, 14 Apr 2021 06:54:09 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 709E96113B for ; Wed, 14 Apr 2021 06:54:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 709E96113B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID:Date: Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2DH5rNDEah2kYLJV+I1Va6d9XOdtrHy08ja0Opjk+/A=; b=ln7MT+k3gE/5+2Go6ooK8ogi7 OvRcirAKa9LDlwf0AMcF4nnw34nWQ7rlyUM+iNubM2VCjJlL8o7Uyvq7VorFzRPRxNSvPQGurf593 4s0uhgPfLJoVte5mVLZOpaKjZflppVxwu+W01TB6jovd6GxLuMgQEgVNnGsNEKYmI44tSzUJWx+Hu FoGBwaSTYQP8rxNF0QuhcSKbZfmImm9efvsCw4sp+wRH49DZn8hRb41lWT1Gnd9sFFZ5eO0rV+Psv O6YrLSm/1fIKBY6kDc72IF4P0pJfaD4LpR4Es1zx07VB6+A2ZaOxiPNRmAuMiizxtKk1n6pHTGSxG 0Iiy96+dw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lWZO6-00Bja5-CJ; Wed, 14 Apr 2021 06:52:34 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWZNi-00BjXs-6g for linux-arm-kernel@desiato.infradead.org; Wed, 14 Apr 2021 06:52:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=aFd/Nyxe6j5ME1fW2k09da9gKmWx38vr/j6c7s3C2iU=; b=qalLE7i5buENaxe4IuyQQoANwj 5SeTm97EaNGF0pUL8dRoknAm0pGN2OrfjAqfxRefcH8+H7YMoFsDoforXyLU1OhMkr0MwzUYC2OIP JorOggvGHA45uUOPFbiLsE8KZ7lsE04kEKl3tDG0XkMcaOTU/kdAlmzyEw9HH2R+8n9s5fOv26nTL ynALE/4aW/1sfI9h3EWDXCcxui0u57PbxATxYrLTLHpRHUi3TgJmERMM4O8lKCC2B7TlW65T3k4GP wZBQxHTwHKNeA1aDtBymxYO45Reg+ZDGOeOv2eNJYFm1J7SIfOQrebuIXuOWNp9ImZUQokvdpqcJS Ym5WtilQ==; Received: from szxga05-in.huawei.com ([45.249.212.191]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWZNe-007XtU-1F for linux-arm-kernel@lists.infradead.org; Wed, 14 Apr 2021 06:52:09 +0000 Received: from DGGEMS402-HUB.china.huawei.com (unknown [172.30.72.58]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4FKtNz4Np5zNw2V; Wed, 14 Apr 2021 14:49:07 +0800 (CST) Received: from DESKTOP-5IS4806.china.huawei.com (10.174.187.224) by DGGEMS402-HUB.china.huawei.com (10.3.19.202) with Microsoft SMTP Server id 14.3.498.0; Wed, 14 Apr 2021 14:51:55 +0800 From: Keqian Zhu To: , , , , Marc Zyngier CC: Subject: [PATCH v3 2/2] kvm/arm64: Try stage2 block mapping for host device MMIO Date: Wed, 14 Apr 2021 14:51:09 +0800 Message-ID: <20210414065109.8616-3-zhukeqian1@huawei.com> X-Mailer: git-send-email 2.8.4.windows.1 In-Reply-To: <20210414065109.8616-1-zhukeqian1@huawei.com> References: <20210414065109.8616-1-zhukeqian1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.187.224] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210413_235206_434207_D08CE7E6 X-CRM114-Status: GOOD ( 15.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The MMIO region of a device maybe huge (GB level), try to use block mapping in stage2 to speedup both map and unmap. Compared to normal memory mapping, we should consider two more points when try block mapping for MMIO region: 1. For normal memory mapping, the PA(host physical address) and HVA have same alignment within PUD_SIZE or PMD_SIZE when we use the HVA to request hugepage, so we don't need to consider PA alignment when verifing block mapping. But for device memory mapping, the PA and HVA may have different alignment. 2. For normal memory mapping, we are sure hugepage size properly fit into vma, so we don't check whether the mapping size exceeds the boundary of vma. But for device memory mapping, we should pay attention to this. This adds device_rough_page_shift() to check these two points when selecting block mapping size. Signed-off-by: Keqian Zhu --- arch/arm64/kvm/mmu.c | 37 +++++++++++++++++++++++++++++++++---- 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index c59af5ca01b0..1a6d96169d60 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -624,6 +624,31 @@ static void kvm_send_hwpoison_signal(unsigned long address, short lsb) send_sig_mceerr(BUS_MCEERR_AR, (void __user *)address, lsb, current); } +/* + * Find a max mapping size that properly insides the vma. And hva and pa must + * have the same alignment to this mapping size. It's rough as there are still + * other restrictions, will be checked by fault_supports_stage2_huge_mapping(). + */ +static short device_rough_page_shift(struct vm_area_struct *vma, + unsigned long hva) +{ + phys_addr_t pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start); + +#ifndef __PAGETABLE_PMD_FOLDED + if ((hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) && + ALIGN_DOWN(hva, PUD_SIZE) >= vma->vm_start && + ALIGN(hva, PUD_SIZE) <= vma->vm_end) + return PUD_SHIFT; +#endif + + if ((hva & (PMD_SIZE - 1)) == (pa & (PMD_SIZE - 1)) && + ALIGN_DOWN(hva, PMD_SIZE) >= vma->vm_start && + ALIGN(hva, PMD_SIZE) <= vma->vm_end) + return PMD_SHIFT; + + return PAGE_SHIFT; +} + static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot, unsigned long hva, unsigned long map_size) @@ -769,7 +794,10 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, return -EFAULT; } - /* Let's check if we will get back a huge page backed by hugetlbfs */ + /* + * Let's check if we will get back a huge page backed by hugetlbfs, or + * get block mapping for device MMIO region. + */ mmap_read_lock(current->mm); vma = find_vma_intersection(current->mm, hva, hva + 1); if (unlikely(!vma)) { @@ -780,11 +808,12 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, if (is_vm_hugetlb_page(vma)) vma_shift = huge_page_shift(hstate_vma(vma)); + else if (vma->vm_flags & VM_PFNMAP) + vma_shift = device_rough_page_shift(vma, hva); else vma_shift = PAGE_SHIFT; - if (logging_active || - (vma->vm_flags & VM_PFNMAP)) { + if (logging_active) { force_pte = true; vma_shift = PAGE_SHIFT; } @@ -855,7 +884,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, if (kvm_is_device_pfn(pfn)) { device = true; - force_pte = true; + force_pte = (vma_pagesize == PAGE_SIZE); } else if (logging_active && !write_fault) { /* * Only actually map the page as writable if this was a write