From patchwork Thu Oct 10 18:23:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830840 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 020B8CFC5F0 for ; Thu, 10 Oct 2024 18:37:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=jxW7uER1jDhEn8 341qklki9xvztKXeK4RfYuk8QJpT6zDsVA3Mp5S2D+lwi5SUAF6hzNsAPVyv3ruA/duJhztQcqOga QMkYFrGWQUpRzLnhPtdPvUKzV23JAymcSdSaNdAd5zPuP8b+ajZKRF4pLImdyvtXwQeZgHv+FNgsh tyviEgosrN1LZgSfbvLf+bewDkdYHFoN7quXuNmqk+qFlGybfkevmlYBJDTttVZD3s1pMIniKSuBM wkVhH0YO3uV+rclYEjZhGUOnNrcQLf6qHpT/i9UV/PULpqaRvx7+1rMnoHy3KJRr+88ztwOfGJQ8B iONBfWJPYWUFuLfju1Uw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy2d-0000000Dw3T-1LRe; Thu, 10 Oct 2024 18:37:39 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqL-0000000DpfC-46t3 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:24:59 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e26ba37314so24841897b3.0 for ; Thu, 10 Oct 2024 11:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584697; x=1729189497; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=M7Snr5N22pikPCYYXMJYsOlgIEBf4+x3Uqg6sVxeZVadYTICrG4ulpFRgPtXVG0X0n JZ2/nX9phcaEtKWHj1yPru+AarfrxgYbtzXswb2v92ZPGKNmtdah06jEFW/mllb+oibh CBKgB1cMUdcgVGm+aTjLPjjrmXe2Hfze6XuIV0ZeaSFds53FKVgiaPzajxfWki+VusBG 0ZxzebIV31ZFXuzuGmx7TJqF8oZ/P8kgCd05nBiQQ6We33hwrDG5iyqppNKEN7BXJoyY U1tRYYoYlyeYl0xm2A4lPSen0EvpIggWblDdQu+MQKURLXfflhNdB/DNtQyuAg89xkyJ 7Jtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584697; x=1729189497; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=BtVr5cyHmr71nDrfDfA225ncOmiYBalFYQ3w6vhrSS0T6EyQu3AbPF4wamnPE6bnoQ KGeW/6qKgFKyFcjgAf5Kko+FQ5fNiIr6ZQCpPK1XKY7mr48prR4lQzijhvfnggmSTRrl iVkAwjuFaeL/bz206r0LX+LEuAjJ4tLw03ImSgci/5/4Ddyn1IxBbi6TuPtiBwZ37LA8 VRS6Nkrj/LzsWASsmYJCyJqui2Ckot3eqysLIJFsAyeUbal3MUliiHIxWZO8o3/oub00 m8Orfl5PIW0QBw1pt/a7sGXd/PKf5iZQTpv3HC8MtO1iEQfaVd7UnMkG8+JaWUn7lEri X+gg== X-Forwarded-Encrypted: i=1; AJvYcCWBLCE3bINxMS4N5E8yKa1gg6Kl2YwcT09KJUsFbwTz9dcUT+0VLRQ4C00G9ZJHOYQpHAl41XGsIX4GmuJDiHvt@lists.infradead.org X-Gm-Message-State: AOJu0YyWy2cCvpbnCZkKokwdR6tGpotlVVVmk2iVa4enB1EYzRppDRgr O1dxxBBnkHBbH8wOalNSEqZjvtXsZBCPK6heg4zmzESczclucPEXlGVtqRs3U3Y0KM2l15HJQfv ZvQ== X-Google-Smtp-Source: AGHT+IGrULPD5dT5NWC9dFAfgTsoL9JIWGzHU9t8+6e7flVyO5QHPCRlr/jDUZCu+nhXBHYl+WGzejB3KuQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:e486:0:b0:6e2:371f:4aef with SMTP id 00721157ae682-6e322168931mr204147b3.3.1728584696651; Thu, 10 Oct 2024 11:24:56 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-5-seanjc@google.com> Subject: [PATCH v13 04/85] KVM: x86/mmu: Skip the "try unsync" path iff the old SPTE was a leaf SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112458_044660_D6147033 X-CRM114-Status: GOOD ( 13.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Apply make_spte()'s optimization to skip trying to unsync shadow pages if and only if the old SPTE was a leaf SPTE, as non-leaf SPTEs in direct MMUs are always writable, i.e. could trigger a false positive and incorrectly lead to KVM creating a SPTE without write-protecting or marking shadow pages unsync. This bug only affects the TDP MMU, as the shadow MMU only overwrites a shadow-present SPTE when synchronizing SPTEs (and only 4KiB SPTEs can be unsync). Specifically, mmu_set_spte() drops any non-leaf SPTEs *before* calling make_spte(), whereas the TDP MMU can do a direct replacement of a page table with the leaf SPTE. Opportunistically update the comment to explain why skipping the unsync stuff is safe, as opposed to simply saying "it's someone else's problem". Cc: stable@vger.kernel.org Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/spte.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8f7eb3ad88fc..5521608077ec 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -226,12 +226,20 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= PT_WRITABLE_MASK | shadow_mmu_writable_mask; /* - * Optimization: for pte sync, if spte was writable the hash - * lookup is unnecessary (and expensive). Write protection - * is responsibility of kvm_mmu_get_page / kvm_mmu_sync_roots. - * Same reasoning can be applied to dirty page accounting. + * When overwriting an existing leaf SPTE, and the old SPTE was + * writable, skip trying to unsync shadow pages as any relevant + * shadow pages must already be unsync, i.e. the hash lookup is + * unnecessary (and expensive). + * + * The same reasoning applies to dirty page/folio accounting; + * KVM will mark the folio dirty using the old SPTE, thus + * there's no need to immediately mark the new SPTE as dirty. + * + * Note, both cases rely on KVM not changing PFNs without first + * zapping the old SPTE, which is guaranteed by both the shadow + * MMU and the TDP MMU. */ - if (is_writable_pte(old_spte)) + if (is_last_spte(old_spte, level) && is_writable_pte(old_spte)) goto out; /*