From patchwork Thu Dec 8 19:38:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Matlack X-Patchwork-Id: 13068802 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CDDD3C4332F for ; Thu, 8 Dec 2022 19:50:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=NJDpr3Lcn2fuEGPf/jG+9AytPqIpoPo6wcmEa+lNTho=; b=k6a+NjJHTaDVZaqLPe8o+2Tr5+ KkWAY5u+1K7Yc773sQygT6jxjkj61ee/ukDP/YfQP/e+RDlmI/eeV9LhhAyKjZtCCyRIh66qabila z7ENBLR3BdcQUer+vQkNd3G5l3fSH++cXTvqCLpGLm+zdtaHmlI0fSq4jnoHPbdq1q4MOayTrA+bs e8yI20gJ7LQd86868pY+9D5TFkmSzIqtKgsYrRH/XubEyQ7NSBMxoPZNWapRDvQBdOxcUfm6aUmwG EiiIseQrdTTqnFTYHMi7zPu3pbMlyREJ0k2Op4Nhmiqp6lbZm2QyErV2Bo8Zd4mXBdJxmDUN8MYdE T3Y+6kTw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p3MtS-00A3Me-AK; Thu, 08 Dec 2022 19:49:19 +0000 Received: from mail-ot1-x34a.google.com ([2607:f8b0:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p3Mr9-00A1kO-2S for linux-arm-kernel@lists.infradead.org; Thu, 08 Dec 2022 19:46:56 +0000 Received: by mail-ot1-x34a.google.com with SMTP id m12-20020a9d6acc000000b006707706d25dso144936otq.22 for ; Thu, 08 Dec 2022 11:46:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=T0+hU8k+RyCmVBvBo6qJA+pYHpeWz4BnzaTdktBKy2A=; b=j2HX3lJ7u8ucEzkeGBQfuBx9Yb7N2Qy9oTH39ulAVDzwqd+G0fIW4WjnRqotxR/RTx o/pna9eXTXuMdJmAYL1UVpM36PkT4GZM2h0bpfrNAurm7bBa7yOIol4vLsYL261GeqzA 3HOdEsZ41tJib1TeC34Adynr+Edi81xeW1SDkV4CdXiQGoTOGOVJ4WxSxzRqIQRX0/iB RSZpalWLzZZk3q25mDyKyCcOP/lgTKFRKvpNiWk49MLHspz7m3Pd9fzp8sxVKQa7WFIV Cq+v/eIrOgIUSWXVgUqA7VpSfMfnDwSrS/lkhXxrcLWaE6ZViWl/62plvpN2oVhZ91U+ +4vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=T0+hU8k+RyCmVBvBo6qJA+pYHpeWz4BnzaTdktBKy2A=; b=AfkXQKRAdTgtg5+zXX7G2qOnZFLmES9ehlfuta0MVCGHkighhlb93lZKJw0wixo24K kKM+ZyIt+rpLuyecmx/7eOK5PPoYgB8R0/YJ8hhQQl9v4kWgjNUAQBWyPeq/ExmEbnst K1GoEBpvMjs2REdlUs8zstDbziWyW+ifmsfZ1R8EXdWmUjUShQjzRy4qCtaz2XyI16d8 bDl6+SQQuQSCF/CiZ4ThT1NhMwpDrsAUnqgiY9s+L1uww/j7hKU1br96uTZN1kTa6s8v DB6gdOdR7sxBQZVpJlivwaw4Gh8ucvuzZDXoaWxvlA+IFFzgny0xS/e6QpzQLH8OgLEI OyjQ== X-Gm-Message-State: ANoB5pmf3RXmTUX+KgHuIWN+mN2emWdErJv8L0IzPxtEw2eDlBQFWalR X+g9opi+NrqYAYAyVcMSRf4+zCsQewdicQ== X-Google-Smtp-Source: AA0mqf4KN0ToQ1wmZzmflZp36b4L5Nl4FpG0VJHdW80YwtBlwu869ju+Ri2dl6OmwZR+dYAtfRM+xdMRlNvXFw== X-Received: from dmatlack-n2d-128.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:1309]) (user=dmatlack job=sendgmr) by 2002:a25:6994:0:b0:6f3:c535:4ea3 with SMTP id e142-20020a256994000000b006f3c5354ea3mr55731660ybc.498.1670528366862; Thu, 08 Dec 2022 11:39:26 -0800 (PST) Date: Thu, 8 Dec 2022 11:38:32 -0800 In-Reply-To: <20221208193857.4090582-1-dmatlack@google.com> Mime-Version: 1.0 References: <20221208193857.4090582-1-dmatlack@google.com> X-Mailer: git-send-email 2.39.0.rc1.256.g54fd8350bd-goog Message-ID: <20221208193857.4090582-13-dmatlack@google.com> Subject: [RFC PATCH 12/37] KVM: x86/mmu: Use PG_LEVEL_{PTE,PMD,PUD} in the TDP MMU From: David Matlack To: Paolo Bonzini Cc: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Huacai Chen , Aleksandar Markovic , Anup Patel , Atish Patra , Paul Walmsley , Palmer Dabbelt , Albert Ou , Sean Christopherson , Andrew Morton , David Matlack , Anshuman Khandual , Nadav Amit , "Matthew Wilcox (Oracle)" , Vlastimil Babka , "Liam R. Howlett" , Suren Baghdasaryan , Peter Xu , xu xin , Arnd Bergmann , Yu Zhao , Colin Cross , Hugh Dickins , Ben Gardon , Mingwei Zhang , Krish Sadhukhan , Ricardo Koller , Jing Zhang , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221208_114655_133207_37A6D91C X-CRM114-Status: GOOD ( 14.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use PG_LEVEL_{PTE,PMD,PUD} in the TDP MMU instead of the x86-specific PG_LEVEL_{4K,2M,1G} aliases. This prepares for moving the TDP MMU to common code, where not all architectures will have 4K PAGE_SIZE. No functional change intended. Signed-off-by: David Matlack --- arch/x86/kvm/mmu/tdp_iter.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 16 ++++++++-------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_iter.h b/arch/x86/kvm/mmu/tdp_iter.h index f0af385c56e0..892c078aab58 100644 --- a/arch/x86/kvm/mmu/tdp_iter.h +++ b/arch/x86/kvm/mmu/tdp_iter.h @@ -106,7 +106,7 @@ struct tdp_iter { tdp_iter_next(&iter)) #define for_each_tdp_pte(iter, root, start, end) \ - for_each_tdp_pte_min_level(iter, root, PG_LEVEL_4K, start, end) + for_each_tdp_pte_min_level(iter, root, PG_LEVEL_PTE, start, end) tdp_ptep_t spte_to_child_pt(u64 pte, int level); diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 4940413d3767..bce0566f2d94 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -347,7 +347,7 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn, bool pfn_changed; struct kvm_memory_slot *slot; - if (level > PG_LEVEL_4K) + if (level > PG_LEVEL_PTE) return; pfn_changed = spte_to_pfn(old_spte) != spte_to_pfn(new_spte); @@ -526,7 +526,7 @@ static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, bool pfn_changed = spte_to_pfn(old_spte) != spte_to_pfn(new_spte); WARN_ON(level > PT64_ROOT_MAX_LEVEL); - WARN_ON(level < PG_LEVEL_4K); + WARN_ON(level < PG_LEVEL_PTE); WARN_ON(gfn & (KVM_PAGES_PER_HPAGE(level) - 1)); /* @@ -897,9 +897,9 @@ static void tdp_mmu_zap_root(struct kvm *kvm, struct kvm_mmu_page *root, * inducing a stall to allow in-place replacement with a 1gb hugepage. * * Because zapping a SP recurses on its children, stepping down to - * PG_LEVEL_4K in the iterator itself is unnecessary. + * PG_LEVEL_PTE in the iterator itself is unnecessary. */ - __tdp_mmu_zap_root(kvm, root, shared, PG_LEVEL_1G); + __tdp_mmu_zap_root(kvm, root, shared, PG_LEVEL_PUD); __tdp_mmu_zap_root(kvm, root, shared, root->role.level); rcu_read_unlock(); @@ -944,7 +944,7 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, rcu_read_lock(); - for_each_tdp_pte_min_level(iter, root, PG_LEVEL_4K, start, end) { + for_each_tdp_pte_min_level(iter, root, PG_LEVEL_PTE, start, end) { if (can_yield && tdp_mmu_iter_cond_resched(kvm, &iter, flush, false)) { flush = false; @@ -1303,7 +1303,7 @@ static bool set_spte_gfn(struct kvm *kvm, struct tdp_iter *iter, /* Huge pages aren't expected to be modified without first being zapped. */ WARN_ON(pte_huge(range->pte) || range->start + 1 != range->end); - if (iter->level != PG_LEVEL_4K || + if (iter->level != PG_LEVEL_PTE || !is_shadow_present_pte(iter->old_spte)) return false; @@ -1672,7 +1672,7 @@ static void clear_dirty_pt_masked(struct kvm *kvm, struct kvm_mmu_page *root, if (!mask) break; - if (iter.level > PG_LEVEL_4K || + if (iter.level > PG_LEVEL_PTE || !(mask & (1UL << (iter.gfn - gfn)))) continue; @@ -1726,7 +1726,7 @@ static void zap_collapsible_spte_range(struct kvm *kvm, rcu_read_lock(); - for_each_tdp_pte_min_level(iter, root, PG_LEVEL_2M, start, end) { + for_each_tdp_pte_min_level(iter, root, PG_LEVEL_PMD, start, end) { retry: if (tdp_mmu_iter_cond_resched(kvm, &iter, false, true)) continue;