From patchwork Fri Oct 27 18:21:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13438995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9518DC25B72 for ; Fri, 27 Oct 2023 18:23:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+9lFAwOgGHPCkjLIeb+HJZL500E1c/550lFPIV5Kifk=; b=bLecDQOL7rTsa2 yKQsgssXyIHEZeoTSJTk3wpsSDnO8RVIo1z/VGEG45fip8vQ1cweGqMQuqmOUfz4AW7bL6q0ax4TD L3cKDHpBJuZ0rFenHZqN4zwLZo+v2f21IqQ9DYnBPAsrZyfVyWMQbfhaV5FovBGngWygJD2+K9Ehr /ya7sc+yRsbB+56ZhaYmeTcXn576tJuJoy/KdVL/cnYGeyYrM/5GP526fq784WME8WtRUaZwsiZ6X 1W5G8ZeDtgYMJZBOak91OYaeivnuaMC04dFr8WfxfJzgD1qlN1wjz1Sv3+AHSBmUe/nuiotsFrcrG 70YmrD6iT4IGGU9/nTVw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qwRTp-00GwxV-0k; Fri, 27 Oct 2023 18:22:45 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qwRTa-00GwjI-2y for linux-arm-kernel@lists.infradead.org; Fri, 27 Oct 2023 18:22:33 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-da04776a869so1685176276.0 for ; Fri, 27 Oct 2023 11:22:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1698430949; x=1699035749; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Fn/2IBJw8IgyOIqoe5CTcghLJbfyq3WPHSyzyGJhcug=; b=Y1K00eqadYQft1+SyCN9DHLHlRpkUAtGhjhL87ht7MELbqOyRV1KcgVbc3WL7hMzVA WU16cPdog7ARbZhyigEx/yrTSb+7Eo5JjciXrhhn6tCnojN66m72x+C1cA+jjrk5Zs+m QAGUTcJejMXILnKx4mlG0yzchd7QZCCeV5PZ2kVS/uUzJe97DnhDk5831jxAn2dONNYT 8gUoSvo5OP4k5hfZ14ylJeJezKQe+HVAK3+AMczOJeMUWOQ5EdsHF9ManAoehDq5iOca f2SiKm9BGHoDEWPKEAmJjLYVnAJAfwpBhGE81YilbE8ZR5L6NSZuyYmoY6CQbLd+3E2P +ECQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698430949; x=1699035749; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Fn/2IBJw8IgyOIqoe5CTcghLJbfyq3WPHSyzyGJhcug=; b=h1DgFZcSUqta3oWIKPjx9/9wfUQjeiQlExBNJTF4e5cx7BubG/1F4zDuNQxNFVxpYB 3gA+QPK2ru975rL+tGY74V3ZJOUH24Vug6PiwzggC0E6jys2yiI4TNbfnmHBdjJDRgND 2cqcuUWhe6m5/sWwXsMUSyJoxikkYp0+2fBMsapPD09VlAh7157L0DS5hK5ZUCs+STlh fRl1Uzhdh6wRCavSubW2dhfPIcocU6U+cWRCF+Ty58TokAxKUUbd1aXweSiL/Ulkih2i iJ/+yHxgbTq2K7dTLz6uZZfAVxBKyup5ocW7i21rCPXqKvRdUIEWgicPyErxw6kkZTun Er/Q== X-Gm-Message-State: AOJu0YzN7jaQizISg7LgQcvAP5RQ9Huz6nw9RComuV5YvKzhe0a3HrEb tkZfGGaFJkcsiLlf3T57/RQ3e92rs1I= X-Google-Smtp-Source: AGHT+IE/2A67BLmFvNwLQGxAeqfnMv4JVTEXFJ3uXVKssW9yJHFYWQUNgIG+FyGEOnQbWZWT52otjVeFu0A= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:1788:b0:da0:c9a5:b529 with SMTP id ca8-20020a056902178800b00da0c9a5b529mr57775ybb.12.1698430949628; Fri, 27 Oct 2023 11:22:29 -0700 (PDT) Date: Fri, 27 Oct 2023 11:21:44 -0700 In-Reply-To: <20231027182217.3615211-1-seanjc@google.com> Mime-Version: 1.0 References: <20231027182217.3615211-1-seanjc@google.com> X-Mailer: git-send-email 2.42.0.820.g83a721a137-goog Message-ID: <20231027182217.3615211-3-seanjc@google.com> Subject: [PATCH v13 02/35] KVM: Assert that mmu_invalidate_in_progress *never* goes negative From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Sean Christopherson , Alexander Viro , Christian Brauner , "Matthew Wilcox (Oracle)" , Andrew Morton Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Xu Yilun , Chao Peng , Fuad Tabba , Jarkko Sakkinen , Anish Moorthy , David Matlack , Yu Zhang , Isaku Yamahata , " =?utf-8?q?Micka=C3=ABl_Sala?= =?utf-8?q?=C3=BCn?= " , Vlastimil Babka , Vishal Annapurve , Ackerley Tng , Maciej Szmigiero , David Hildenbrand , Quentin Perret , Michael Roth , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231027_112230_969180_F254956D X-CRM114-Status: GOOD ( 13.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the assertion on the in-progress invalidation count from the primary MMU's notifier path to KVM's common notification path, i.e. assert that the count doesn't go negative even when the invalidation is coming from KVM itself. Opportunistically convert the assertion to a KVM_BUG_ON(), i.e. kill only the affected VM, not the entire kernel. A corrupted count is fatal to the VM, e.g. the non-zero (negative) count will cause mmu_invalidate_retry() to block any and all attempts to install new mappings. But it's far from guaranteed that an end() without a start() is fatal or even problematic to anything other than the target VM, e.g. the underlying bug could simply be a duplicate call to end(). And it's much more likely that a missed invalidation, i.e. a potential use-after-free, would manifest as no notification whatsoever, not an end() without a start(). Signed-off-by: Sean Christopherson Reviewed-by: Paolo Bonzini Reviewed-by: Fuad Tabba Tested-by: Fuad Tabba --- virt/kvm/kvm_main.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 0524933856d4..5a97e6c7d9c2 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -833,6 +833,7 @@ void kvm_mmu_invalidate_end(struct kvm *kvm, unsigned long start, * in conjunction with the smp_rmb in mmu_invalidate_retry(). */ kvm->mmu_invalidate_in_progress--; + KVM_BUG_ON(kvm->mmu_invalidate_in_progress < 0, kvm); } static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, @@ -863,8 +864,6 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, */ if (wake) rcuwait_wake_up(&kvm->mn_memslots_update_rcuwait); - - BUG_ON(kvm->mmu_invalidate_in_progress < 0); } static int kvm_mmu_notifier_clear_flush_young(struct mmu_notifier *mn,