From patchwork Tue Sep 26 18:19:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Donnefort X-Patchwork-Id: 13399517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 43B6CE7E654 for ; Tue, 26 Sep 2023 18:20:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=z/pTrRQGytWuWP2KftYKx71nDAk+tupSotglOAPS6Lw=; b=IXtNvosUGTYN+Rozasw1UePc4Y ZPSAQ9wU9Lg6B4TEm20kF8TzVKh1VJ0RNS/bKKvWLlySbXKcXJIlJc9gJ+33M5o1/09xqIOjew7HD u45u5MtkyPsY7Ku9hnovjdZDUq1MhPgG95iKwVqiT3O49AbzX8g1TUxVld0fwMxGRFmsTnvqVkZOI eyElpNEj6RlprKDFT8sllgLDWQIprPFps6b+jYIemWRlDYOWPV0ehEs4MQy+aXNmlxBFH8gXbCuXx tOe3MyxkkLSTXsVrUXAAJ9LVFcmd7OW8IJ1xukfh7lL4YAXPstsZrgLajdS6eoP+mmc7KC5JYKGRd LIeRLMmg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qlCf1-00Gpgt-2E; Tue, 26 Sep 2023 18:19:51 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qlCez-00GpfT-1F for linux-arm-kernel@lists.infradead.org; Tue, 26 Sep 2023 18:19:50 +0000 Received: by mail-wr1-x449.google.com with SMTP id ffacd0b85a97d-31fe49c6bb1so7119990f8f.0 for ; Tue, 26 Sep 2023 11:19:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695752386; x=1696357186; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DGKOexKeYx3KIrtoFBMpiZrvvlVOfn3NLusWWIToTBE=; b=Sdz8qTX0/83Mt7gODR2aR0otgyb0tBtkkw20by82VRsKQj3a3s+wYTaWn89A2b5PoR TxtLsWs7bRLTeeIJkaBVDFe4hjpuydSgjIwZuv13wo/wTPpNhERX60TdPwQO/JEfR7h7 +zItl0IYzRvQm4qOerKKWWdoA7YRw3ffa6CC2srrCDyJOI61RTI0FBkzdofx4V+JLRDI b4sH0RsKUemXdqCahzYvxqzSgFfDynkssdo8mvEnBDPElMEfpXUory5j6SmFLXIW2yEl d+O7bAd0wPrKgS6YD2i+k1keaH1u+3U6Wpl0crBKD2N9Yk0ZJfsPPs871DpkoDH0oZ4+ QEhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695752386; x=1696357186; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DGKOexKeYx3KIrtoFBMpiZrvvlVOfn3NLusWWIToTBE=; b=MCqKr8z58legxtCAZEadp6YB0ogWxObTs3HKmGFk3dEw+DAASzAE2XxMMCt3Ivfo5Y P2lv78xX41thk2V0uH+oB2B5i+8gwm47R/dLXiOXX8UVS3HnbAhyNURbHPQXkHSku6h+ NwvIMyXMmjkrWdDZSnZkkBCyKMBct8SP3cLVojBA+LETi5wk70QmjPvyHr2fOFfcNq9s qkIE9V6j4VyHi7wwF2BbBOfSwgNg+Q2+cUl2i6V8+3QgVs374+E3ce+l/7eGpqDWC1Kr YW/sP+ePHz4oiAVL5Ws01pRJfL/K4Bc9vNPgjEogdhN1FzufF7jgKzjPml/GBRLE2jnx ze3Q== X-Gm-Message-State: AOJu0Ywo0pjM5VfwIVPrr8iL4DYegnSL2OT3mhGu8hFmrQGd3VhTiduB kLA8hPU6oKLy85LlAZNVUVcya4OQ1d3Z1CsI X-Google-Smtp-Source: AGHT+IEqc0pyzkMuy8FapsV9SHMmWLYEASvTHx0NsjF40Te7/goAR7lbuc+BSqE3nTWIs3RuYvucpCcOwRI5yu9u X-Received: from vdonnefort.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:2eea]) (user=vdonnefort job=sendgmr) by 2002:a5d:6446:0:b0:31f:b4a3:db12 with SMTP id d6-20020a5d6446000000b0031fb4a3db12mr55390wrw.6.1695752385720; Tue, 26 Sep 2023 11:19:45 -0700 (PDT) Date: Tue, 26 Sep 2023 19:19:31 +0100 In-Reply-To: <20230926181932.1650692-1-vdonnefort@google.com> Mime-Version: 1.0 References: <20230926181932.1650692-1-vdonnefort@google.com> X-Mailer: git-send-email 2.42.0.515.g380fc7ccd1-goog Message-ID: <20230926181932.1650692-2-vdonnefort@google.com> Subject: [PATCH v1 1/2] KVM: arm64: Do not transfer page refcount for THP adjustment From: Vincent Donnefort To: maz@kernel.org, oliver.upton@linux.dev Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kernel-team@android.com, will@kernel.org, willy@infradead.org, Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230926_111949_428570_ED292F42 X-CRM114-Status: GOOD ( 11.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With folios, GUP affects a refcount common to all pages forming the THP. There is therefore no need to move the refcount from the tail to the head page. Under the hood it decrements and increments the same counter. Signed-off-by: Vincent Donnefort diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 587a104f66c3..de5e5148ef5d 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1295,28 +1295,8 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot, if (sz < PMD_SIZE) return PAGE_SIZE; - /* - * The address we faulted on is backed by a transparent huge - * page. However, because we map the compound huge page and - * not the individual tail page, we need to transfer the - * refcount to the head page. We have to be careful that the - * THP doesn't start to split while we are adjusting the - * refcounts. - * - * We are sure this doesn't happen, because mmu_invalidate_retry - * was successful and we are holding the mmu_lock, so if this - * THP is trying to split, it will be blocked in the mmu - * notifier before touching any of the pages, specifically - * before being able to call __split_huge_page_refcount(). - * - * We can therefore safely transfer the refcount from PG_tail - * to PG_head and switch the pfn from a tail page to the head - * page accordingly. - */ *ipap &= PMD_MASK; - kvm_release_pfn_clean(pfn); pfn &= ~(PTRS_PER_PMD - 1); - get_page(pfn_to_page(pfn)); *pfnp = pfn; return PMD_SIZE; From patchwork Tue Sep 26 18:19:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Donnefort X-Patchwork-Id: 13399516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 58758E7E62B for ; Tue, 26 Sep 2023 18:20:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=L3305J+cZ+wGBTivc/stj/5Jbmg+iSekhtrpZ2Uj6Ig=; b=pcc8knIdNRENhZvEW1kMGNNKpC hXHNedu6VqtY5/tfrKerLxVP+QBMJwlhHMxHgWOwgzXtEHey0bjIzGdxCA5kqmheqQ25+oAXguY0l BJqOQGtNUl92b7sZBG19Nc6yF4R+acm4adR+eQUf3T7XUd2tSNFSBo4LZSPIe9H+e1trZZtwl4n0B p6IUphM+tCGaiD4YHMBGcGXvbGdU4PKhCzuBtUJG1xAgracFaqIqDZb8Z477zuMPUhVmEPG+PSAI0 DQ0xIgKXKIbykWEbA8L4mDOlMYSgxhmDbpUgtGmVi+2/y/mbDiFu9gvO+t2Sujtv1Hgj+I7NlMqjL JrOpiJAg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qlCfA-00GpiW-09; Tue, 26 Sep 2023 18:20:00 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qlCf0-00Gpfa-2C for linux-arm-kernel@lists.infradead.org; Tue, 26 Sep 2023 18:19:52 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-59f67676065so108580537b3.0 for ; Tue, 26 Sep 2023 11:19:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695752388; x=1696357188; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7evrq3PKBiHzaLA4DGci/aYqgw5Y4nsW4sL/8l2Nq7I=; b=T3DiBQv2ZnyIWcXSYFbLKMwJAJfi1OyDWNxRc3VG5a9SVWWCuQ5cCA+VCY6KDzRuhs RiwuWLbjhaG+Obdg9l3s1gGcAQjB3X3GiLGPkXVuubL7aRB7qMwrNA5mwoxxKFO4SmSE u+mlK7OC9Slfsj2UWLykql6W5UuX1KyHGi/cCEJ7fGr61p13pnb/6oumszBlcXHunNRk QCG+oAqrUxi/1BmFRiQNrRg8Fn3nRFMiPwh+aETiHVZn9wZTUQc3oUB5CiF/5eErMsjC K2S23AJiI08q9WAqyetFkraRHM9gT7qX5QUCDgQlSpLGV/3GrkS2pMcOSemucypEK1bd bqxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695752388; x=1696357188; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7evrq3PKBiHzaLA4DGci/aYqgw5Y4nsW4sL/8l2Nq7I=; b=KeaaULqcelDx0D+l9r4Cs6URnZiA/HVLQCrk7GAqXHp1wN4J9ddUct2Sp2i4xb1ECy j3sYYqu6L8Z7LK6JiR7xTUAbfxTHwrQts8XklyQk8zfpGYFKeGybRGgiksOg4qIcsImf FODif8fKiJUygq57qZcYqv1A3AYuovrD3/TGu/xXUnVnqzUz4lULcleNb0tuFjjurlAI rfhX5Y75HWlyGaURrQ6SyMiJZwl73e+qx6hBrZLYDg9884hlU+fznoMHRNaUZ5m7OJgz pSdIyA+d5SoGAR8uVVgPB9oiLGC/jSMIFFfxoB/lqIlFbR7rdih36/eLc58iRimtIjCe Uq6Q== X-Gm-Message-State: AOJu0YwJUcCsRyZ2asJVRhn9w3nYnxY6NuYsjtZ37Sj93PSGdDMqtkJN wREzwGAsCOvM+6UOpBAJDdphPBu/q+wL+sXi X-Google-Smtp-Source: AGHT+IFsBLgs3gQ7K5g/UspJwVgIViCv0ygjK5w+Pz4KI1o7o1An2/fTMUVh8EL5Mm2ZDlEZ0xrd5xxhWPj51vn2 X-Received: from vdonnefort.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:2eea]) (user=vdonnefort job=sendgmr) by 2002:a0d:ec10:0:b0:59b:e1db:562c with SMTP id q16-20020a0dec10000000b0059be1db562cmr140768ywn.5.1695752388237; Tue, 26 Sep 2023 11:19:48 -0700 (PDT) Date: Tue, 26 Sep 2023 19:19:32 +0100 In-Reply-To: <20230926181932.1650692-1-vdonnefort@google.com> Mime-Version: 1.0 References: <20230926181932.1650692-1-vdonnefort@google.com> X-Mailer: git-send-email 2.42.0.515.g380fc7ccd1-goog Message-ID: <20230926181932.1650692-3-vdonnefort@google.com> Subject: [PATCH v1 2/2] KVM: arm64: Use folio for THP adjustment From: Vincent Donnefort To: maz@kernel.org, oliver.upton@linux.dev Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kernel-team@android.com, will@kernel.org, willy@infradead.org, Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230926_111950_727533_77B3C74D X-CRM114-Status: GOOD ( 14.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now transparent huge pages use folios, we can check efficiently if a page is mapped by a block by checking the folio size. This is saving a page table walk. It is safe to read the folio in this path. We've just increased its refcount (GUP from __gfn_to_pfn_memslot()). This will prevent attempts of splitting the huge page. Signed-off-by: Vincent Donnefort diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index de5e5148ef5d..69fcbcc7aca5 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -791,51 +791,6 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size, return 0; } -static struct kvm_pgtable_mm_ops kvm_user_mm_ops = { - /* We shouldn't need any other callback to walk the PT */ - .phys_to_virt = kvm_host_va, -}; - -static int get_user_mapping_size(struct kvm *kvm, u64 addr) -{ - struct kvm_pgtable pgt = { - .pgd = (kvm_pteref_t)kvm->mm->pgd, - .ia_bits = vabits_actual, - .start_level = (KVM_PGTABLE_MAX_LEVELS - - CONFIG_PGTABLE_LEVELS), - .mm_ops = &kvm_user_mm_ops, - }; - unsigned long flags; - kvm_pte_t pte = 0; /* Keep GCC quiet... */ - u32 level = ~0; - int ret; - - /* - * Disable IRQs so that we hazard against a concurrent - * teardown of the userspace page tables (which relies on - * IPI-ing threads). - */ - local_irq_save(flags); - ret = kvm_pgtable_get_leaf(&pgt, addr, &pte, &level); - local_irq_restore(flags); - - if (ret) - return ret; - - /* - * Not seeing an error, but not updating level? Something went - * deeply wrong... - */ - if (WARN_ON(level >= KVM_PGTABLE_MAX_LEVELS)) - return -EFAULT; - - /* Oops, the userspace PTs are gone... Replay the fault */ - if (!kvm_pte_valid(pte)) - return -EAGAIN; - - return BIT(ARM64_HW_PGTABLE_LEVEL_SHIFT(level)); -} - static struct kvm_pgtable_mm_ops kvm_s2_mm_ops = { .zalloc_page = stage2_memcache_zalloc_page, .zalloc_pages_exact = kvm_s2_zalloc_pages_exact, @@ -1274,7 +1229,7 @@ static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot, * * Returns the size of the mapping. */ -static long +static unsigned long transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot, unsigned long hva, kvm_pfn_t *pfnp, phys_addr_t *ipap) @@ -1287,10 +1242,7 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot, * block map is contained within the memslot. */ if (fault_supports_stage2_huge_mapping(memslot, hva, PMD_SIZE)) { - int sz = get_user_mapping_size(kvm, hva); - - if (sz < 0) - return sz; + size_t sz = folio_size(pfn_folio(pfn)); if (sz < PMD_SIZE) return PAGE_SIZE; @@ -1385,7 +1337,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, kvm_pfn_t pfn; bool logging_active = memslot_is_logging(memslot); unsigned long fault_level = kvm_vcpu_trap_get_fault_level(vcpu); - long vma_pagesize, fault_granule; + unsigned long vma_pagesize, fault_granule; enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt; @@ -1530,11 +1482,6 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, vma_pagesize = transparent_hugepage_adjust(kvm, memslot, hva, &pfn, &fault_ipa); - - if (vma_pagesize < 0) { - ret = vma_pagesize; - goto out_unlock; - } } if (fault_status != ESR_ELx_FSC_PERM && !device && kvm_has_mte(kvm)) {