From patchwork Thu Oct 10 18:23:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830844 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BFB4BCFC5EC for ; Thu, 10 Oct 2024 18:37:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=MTKe1beU7gaOEt5N0uCG5+dvHKG2PDFM6i+SzQjP9lI=; b=mOMLiR9tAhx5lL oyON5Y7i1LLu2XdQKU/ydzglqa4mu+0/xv8dvTxcefI/6ZKTfaoWHA/VnV0uat3qL/biIvvIBPbhU 5eOprBmKNVGq0ZnmWzSRTGaWVVTssQ1bkn8998B7Lxj+PJHD5tw2VqIqeX1PvM2xI40YuYBuepu9r BO9kBy4JAFPlREUWUx/YWfdboxegPHzvCpvPsk4+yHgIxu5qwyeQ38sx7V8iQgZgUFMLjQuE7zygC UZtXdAATbqVWPE9H1v2nLJ+g/Cq4ZRHsCapC7ZJjEsoXQjEb8spjzcej2R/vMxQ267lS0qAyJ/TFG Oa1uTFFDPEW7FNlv9eMA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy2e-0000000Dw4H-2SSy; Thu, 10 Oct 2024 18:37:40 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqM-0000000DpfE-2L6Q for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:24:59 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e26ba37314so24841917b3.0 for ; Thu, 10 Oct 2024 11:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584697; x=1729189497; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=M7Snr5N22pikPCYYXMJYsOlgIEBf4+x3Uqg6sVxeZVadYTICrG4ulpFRgPtXVG0X0n JZ2/nX9phcaEtKWHj1yPru+AarfrxgYbtzXswb2v92ZPGKNmtdah06jEFW/mllb+oibh CBKgB1cMUdcgVGm+aTjLPjjrmXe2Hfze6XuIV0ZeaSFds53FKVgiaPzajxfWki+VusBG 0ZxzebIV31ZFXuzuGmx7TJqF8oZ/P8kgCd05nBiQQ6We33hwrDG5iyqppNKEN7BXJoyY U1tRYYoYlyeYl0xm2A4lPSen0EvpIggWblDdQu+MQKURLXfflhNdB/DNtQyuAg89xkyJ 7Jtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584697; x=1729189497; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=HQlOybqBxZtTAd788lqNVwBs+W8K+dUAgRMhE7JBP/KeEe/oizCG6Vh19LoAB2WmwO G7kGNO10oGIPLLbh8kSYHLgKWalotUD0AkH4sffOcjqjZDOBo+SYo/6s4V+iqrt3RFzJ yBMODAJHj84U+QvxV7N290ffyoB9i5y5UY2Ni2HfajCu9xX5dJDyEXFbFj1pL78JGPgs 5OLoqMJ3pk9qcBRp2OP+jKtOl3Cvg4wEQ26KBUFa6h8ZGS5n+VWWC3SnpU0Td18HNy8y yqr3KDrUcpWQWyXtwucV8MAGlD99Zwr5Vw6Npb0QQ+MOLzEHeAZBPmRDL+KNCrKmzGNV gTPA== X-Forwarded-Encrypted: i=1; AJvYcCWcwY32ODg1EzKgd1N0WCEkHElAu6EsAXrJenGJH499ajikO5qZ8Uqiq4tndRMKNZx16eYhzeJC9hKADA==@lists.infradead.org X-Gm-Message-State: AOJu0Yxchsze+JlWmzL/lIcYoPbo7OyFe4IkGEUfBNV4cw28a0GrU5UU wth7ATydSMfMv9cSPEchfE8C3OiPyVGAh3f6NI3vvaOBIb/iGMXaL5t/OxcT2Ne4Yof/IRHPFQO zww== X-Google-Smtp-Source: AGHT+IGrULPD5dT5NWC9dFAfgTsoL9JIWGzHU9t8+6e7flVyO5QHPCRlr/jDUZCu+nhXBHYl+WGzejB3KuQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:e486:0:b0:6e2:371f:4aef with SMTP id 00721157ae682-6e322168931mr204147b3.3.1728584696651; Thu, 10 Oct 2024 11:24:56 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-5-seanjc@google.com> Subject: [PATCH v13 04/85] KVM: x86/mmu: Skip the "try unsync" path iff the old SPTE was a leaf SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112458_676789_343792F5 X-CRM114-Status: GOOD ( 12.18 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Apply make_spte()'s optimization to skip trying to unsync shadow pages if and only if the old SPTE was a leaf SPTE, as non-leaf SPTEs in direct MMUs are always writable, i.e. could trigger a false positive and incorrectly lead to KVM creating a SPTE without write-protecting or marking shadow pages unsync. This bug only affects the TDP MMU, as the shadow MMU only overwrites a shadow-present SPTE when synchronizing SPTEs (and only 4KiB SPTEs can be unsync). Specifically, mmu_set_spte() drops any non-leaf SPTEs *before* calling make_spte(), whereas the TDP MMU can do a direct replacement of a page table with the leaf SPTE. Opportunistically update the comment to explain why skipping the unsync stuff is safe, as opposed to simply saying "it's someone else's problem". Cc: stable@vger.kernel.org Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/spte.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8f7eb3ad88fc..5521608077ec 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -226,12 +226,20 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= PT_WRITABLE_MASK | shadow_mmu_writable_mask; /* - * Optimization: for pte sync, if spte was writable the hash - * lookup is unnecessary (and expensive). Write protection - * is responsibility of kvm_mmu_get_page / kvm_mmu_sync_roots. - * Same reasoning can be applied to dirty page accounting. + * When overwriting an existing leaf SPTE, and the old SPTE was + * writable, skip trying to unsync shadow pages as any relevant + * shadow pages must already be unsync, i.e. the hash lookup is + * unnecessary (and expensive). + * + * The same reasoning applies to dirty page/folio accounting; + * KVM will mark the folio dirty using the old SPTE, thus + * there's no need to immediately mark the new SPTE as dirty. + * + * Note, both cases rely on KVM not changing PFNs without first + * zapping the old SPTE, which is guaranteed by both the shadow + * MMU and the TDP MMU. */ - if (is_writable_pte(old_spte)) + if (is_last_spte(old_spte, level) && is_writable_pte(old_spte)) goto out; /*