From patchwork Tue Mar 18 16:20:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 14021250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34388C28B28 for ; Tue, 18 Mar 2025 16:21:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9EEEB280016; Tue, 18 Mar 2025 12:21:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FF6228000B; Tue, 18 Mar 2025 12:21:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B649280016; Tue, 18 Mar 2025 12:21:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 29E9128000B for ; Tue, 18 Mar 2025 12:21:02 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E96781206ED for ; Tue, 18 Mar 2025 16:21:02 +0000 (UTC) X-FDA: 83235185964.13.548A69B Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf30.hostedemail.com (Postfix) with ESMTP id 1750A8000F for ; Tue, 18 Mar 2025 16:21:00 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Y6yGwOXD; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of 3a53ZZwUKCFIDuvvu08805y.w86527EH-664Fuw4.8B0@flex--tabba.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3a53ZZwUKCFIDuvvu08805y.w86527EH-664Fuw4.8B0@flex--tabba.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742314861; a=rsa-sha256; cv=none; b=QywdvXu/UjaLHWSoE69A14poU1xJltleiuUBbpO0JgVUd9EdrdY9LohkPlZiMVf2gKxtVC IbrLPDZLq6Gq6lCKKefUdhT3JeOgz4ypqtv75ONV0UNrwJUVM+etjkgYEhqePKZLA5BpYq Gj8SmlQ+CL29kUwl1ITW/2cvuVA5o54= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Y6yGwOXD; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of 3a53ZZwUKCFIDuvvu08805y.w86527EH-664Fuw4.8B0@flex--tabba.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3a53ZZwUKCFIDuvvu08805y.w86527EH-664Fuw4.8B0@flex--tabba.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742314861; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=G9KWHi/KBkZHmOxvUg1u2dLw8hruOCqp8/fshelQJPQ=; b=3p+pTg3B8STv4kcX1XWOl4D1x+Q6FpY/ebJkHUkcoVXVvwztB5XE3oGlFg0y4uNdQpM7ya CBWEoZNkww6meuBqKBj4tMNc6stmH88KOZyRaX4zPKmeDxIbvPPYdvb2kX3ei0Ht9RwiNC mjc5mC7e5eZuN4txOkR1nyfTkzZtAh4= Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43cf172ffe1so30697805e9.3 for ; Tue, 18 Mar 2025 09:21:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742314860; x=1742919660; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=G9KWHi/KBkZHmOxvUg1u2dLw8hruOCqp8/fshelQJPQ=; b=Y6yGwOXDp8pTHuXnQnsj2cKP8/TiFv4S7wCxGGWoN6qOs93BL2VCUK6d06yjK0IXuB myfoltkZ1T1wUsxUx1KsmWDVBlCpwuiloJrHb6Dbp4beXn6xMdaVmVj41X2QNof2PO4u KyxZwGyRmjKj77b3izjVJgyjo83rXE7/jRA8fyTicFmgO3TBUkh274A/akjx4zMNeAzA A0J2fXLBLEdnllNAuMsYqHRf46C7Q8/qiul301NigMUOI0f9M2ZHD0PEsnDriXSPfZFn PlHHipcf+ASY327odTyKI4o2nTPYIZZ9ikt4XjM74rnXOZ9+XxbNzeDQpq1T0H4iA+w4 ntAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742314860; x=1742919660; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=G9KWHi/KBkZHmOxvUg1u2dLw8hruOCqp8/fshelQJPQ=; b=QG1uxtcYSF1TYVk57xkgMQBHUYYtFu9sLSAhXZmEhupJZ1e0qfEECOTwEhhjlSQ9tV vT2t5wfqHszeHHQj0VHdymkPHIB3NzsLPn16LRLLB8illMDnZM0t7AdF667PG61auSil VqDBGm93LylwPWgp284i3msz+KWXossNdgJR4KsihKzexrTSpTnMNXlG1gV9olx0TXzk 9aTuAXRxn4uTcFpJIJUUpTBxB3IyiUif7F6Q9VqPfa5/a3Elvcx/Il/7RZGsbnwMSqhG 7QgLBRhwU3tjA6fq8mvIIH+MJ4Y6gQN3aZLU/zkiR9FOHMB0VF0IV5KcGzmchcSbATkt oy8A== X-Forwarded-Encrypted: i=1; AJvYcCVbX7EuoDDzzGnV/SxADV8vHlV9FfL8XqkhqcCVqqzyllNTzOmRv1ie/VFUB4nDCrsrwbJNc0CoQA==@kvack.org X-Gm-Message-State: AOJu0YzjEwzPNt4kN88DWkfrkJzWdzFQcegZ2TSsVfgf7NvrrA4S+5g5 rlpDyH/XRx8YwSMEKSa721zwFlwcLdbhB8TaCIiO47p8O7+RxHkhqcaJ0Ae7ZHDa5PiDgqRVlQ= = X-Google-Smtp-Source: AGHT+IEh2X4kUYxpJwUkKLEJgfMN+mg0RefgQxFOkAM1n9rRoUkko5MREaX9qI+5+A7CeT0+SuksFzPCZg== X-Received: from wmqe11.prod.google.com ([2002:a05:600c:4e4b:b0:43d:4038:9229]) (user=tabba job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:848d:b0:43d:94:cff0 with SMTP id 5b1f17b1804b1-43d3b9dc3b0mr24262095e9.19.1742314859735; Tue, 18 Mar 2025 09:20:59 -0700 (PDT) Date: Tue, 18 Mar 2025 16:20:45 +0000 In-Reply-To: <20250318162046.4016367-1-tabba@google.com> Mime-Version: 1.0 References: <20250318162046.4016367-1-tabba@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250318162046.4016367-7-tabba@google.com> Subject: [PATCH v6 6/7] KVM: guest_memfd: Handle invalidation of shared memory From: Fuad Tabba To: kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mm@kvack.org Cc: pbonzini@redhat.com, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, seanjc@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, willy@infradead.org, akpm@linux-foundation.org, xiaoyao.li@intel.com, yilun.xu@intel.com, chao.p.peng@linux.intel.com, jarkko@kernel.org, amoorthy@google.com, dmatlack@google.com, isaku.yamahata@intel.com, mic@digikod.net, vbabka@suse.cz, vannapurve@google.com, ackerleytng@google.com, mail@maciej.szmigiero.name, david@redhat.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com, suzuki.poulose@arm.com, steven.price@arm.com, quic_eberman@quicinc.com, quic_mnalajal@quicinc.com, quic_tsoni@quicinc.com, quic_svaddagi@quicinc.com, quic_cvanscha@quicinc.com, quic_pderrin@quicinc.com, quic_pheragu@quicinc.com, catalin.marinas@arm.com, james.morse@arm.com, yuzenghui@huawei.com, oliver.upton@linux.dev, maz@kernel.org, will@kernel.org, qperret@google.com, keirf@google.com, roypat@amazon.co.uk, shuah@kernel.org, hch@infradead.org, jgg@nvidia.com, rientjes@google.com, jhubbard@nvidia.com, fvdl@google.com, hughd@google.com, jthoughton@google.com, peterx@redhat.com, tabba@google.com X-Rspam-User: X-Rspamd-Queue-Id: 1750A8000F X-Stat-Signature: jqexo8fdqwgtf37r7ecyudshna9f3wm3 X-Rspamd-Server: rspam06 X-HE-Tag: 1742314860-272917 X-HE-Meta: U2FsdGVkX19Im7cMT7Oy3LL5uvlGCjYfTlDVZd6pXUOtzeMs0nzlO/otdqE6m6Gp02BGTB89qXG7GkmyzBMtMjN9d2XIQ4CdE44bGzPIEPAK49b6hqshsU5Ir9gzY/2X6VHn//3q4+seGGmxmjj/1QwiftgZzMjXnEpc1HTTzBKiF8PyPw0v5j89fY3c1QPCiQiXcbfmhXgE5eg2ZwE45Z9Kw0efQNQqyKFy+JHwzBzZW1iSjw6R6wcCyNGH6d3W5P2qVx5k8BYMBP7WHi6Q1zKnjOBz1MOx+r4cWFr7+4bbQsN0Q3hZrWmYElGAFCsuvnCr25SPMgEUjJX3JXU+eDvw+WONdoJb0nlcqHXpjXALx33yB6o4BUeuapcDmQT3G0iW84G/Okdo8I9Xht7TmGfFjcmTzlVJ42N7t6aouEHzQCB+QNy84uLGmOQUSYUHhOL0ON18BdRO3KJb7DIOxu/QfWDQYm9tPvcT17TsfPIqkFk//m1CFelx5TReHoda3mAuTib8HaIINb4+/o42ILADu0JSde12l8NUxiYUyraShIhTZdkG+Y+xgMx8Q5nmDWnoOk8t+TbARG0Euxr+oR5yIJudacxMXB7EVvjGEU8gUq7vXrAoBd+OT1kBKCGFvw2czMikdbBy+Jrq+wDEGMoZPq8NFFDLnM3Lpf0KYFUgN5AS5U2vMCxIhoWbothV2ihNG2yxSzV88pU+o8r7hE5O9Phb3LD/HKpu1bD3g4D9pWTC9ZMwq9gqzbVxJlQQUFnDv0FV/NKz7w0ZcFcgCeHUKkkd1Vuo85dX+VqHUfktgTE6b8fAp7apZV2PRqu3vqdywl/VGuaY3ZKBggwZXGVyZfpiw3dTQ9DH8CHc1ecorrXPg6UGFveWxMz9v3VhWWO7iBz/d6eXpkiowTcaIjjctE6EThM8frqRLDfxDImC0c3Gmb5WPcWOel8m80QrkJKEaygD1bLLz37Bl1z OftVLOut +/d03CjMGLgIJT8WzhAqtQFyKWd8H2BL1slXjIF3qF9vscOtp9wG7uMHGwUx11iHvj2DcYgiameI1BbBMsANKeZL/8b8l4Ewt4+JgrbcLGbt2Co3I+UxjqL2E/PUpBbm3CydgarBvsob5qXxmToH6wzaH7f75mwtA3JVhY6PYj1BSprhj8STNDXUwseptTQehhls5Yj6vwxnsaWByCUDSnYl3s2k/vjd4VmTjrVLFe/Isfqo7EeHm9/e0tOcJXtKQvr8Qd/PsGq4dqYe0hVG2dFd7wn1JA0e9O3WByjUY2kV0cGNkJfgvUylEMWCEwUCOXif3pWgKHlS8F9eJ+t4ip+8/77k8ha+pvw9rHsVFlgT28/pYaaYYd8wpGMQL3aJMYknGYwvPCpXd4AuEAa6u8xKUfj/i8/MMrGzZLITR9AtFzXpnD0Dlm4v0N6SAr+v4Wse9Z3id5B1aqRZIRft2HoAUWuiUcV7LG0RMNzU6CIZ+mm4299YIGS8raxlhvtSQNFJmsF8j26yAg+pofaveIg0szLedsRUHGoJ//ksxAE4YDtqLPU0iGw/LmVWOEoA7VoIgUptCp8j3B2Z8DvyBFCycSmFJK99EPTxW9C3QQD+k8WMYx61IiNPxrxk+PXK2+KRhm6HTDBWomneTFfbJpKQO1g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When guest_memfd backed memory is invalidated, e.g., on punching holes, releasing, ensure that the sharing states are updated and that any folios in a transient state are restored to an appropriate state. Signed-off-by: Fuad Tabba --- virt/kvm/guest_memfd.c | 56 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 4fd9e5760503..0487a08615f0 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -117,6 +117,16 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index) return filemap_grab_folio(inode->i_mapping, index); } +#ifdef CONFIG_KVM_GMEM_SHARED_MEM +static void kvm_gmem_offset_range_invalidate_shared(struct inode *inode, + pgoff_t start, pgoff_t end); +#else +static inline void kvm_gmem_offset_range_invalidate_shared(struct inode *inode, + pgoff_t start, pgoff_t end) +{ +} +#endif + static void kvm_gmem_invalidate_begin(struct kvm_gmem *gmem, pgoff_t start, pgoff_t end) { @@ -126,6 +136,7 @@ static void kvm_gmem_invalidate_begin(struct kvm_gmem *gmem, pgoff_t start, unsigned long index; xa_for_each_range(&gmem->bindings, index, slot, start, end - 1) { + struct file *file = READ_ONCE(slot->gmem.file); pgoff_t pgoff = slot->gmem.pgoff; struct kvm_gfn_range gfn_range = { @@ -145,6 +156,16 @@ static void kvm_gmem_invalidate_begin(struct kvm_gmem *gmem, pgoff_t start, } flush |= kvm_mmu_unmap_gfn_range(kvm, &gfn_range); + + /* + * If this gets called after kvm_gmem_unbind() it means that all + * in-flight operations are gone, and the file has been closed. + */ + if (file) { + kvm_gmem_offset_range_invalidate_shared(file_inode(file), + gfn_range.start, + gfn_range.end); + } } if (flush) @@ -509,6 +530,41 @@ static int kvm_gmem_offset_clear_shared(struct inode *inode, pgoff_t index) return r; } +/* + * Callback when invalidating memory that is potentially shared. + * + * Must be called with the filemap (inode->i_mapping) invalidate_lock held. + */ +static void kvm_gmem_offset_range_invalidate_shared(struct inode *inode, + pgoff_t start, pgoff_t end) +{ + struct xarray *shared_offsets = &kvm_gmem_private(inode)->shared_offsets; + pgoff_t i; + + rwsem_assert_held_write_nolockdep(&inode->i_mapping->invalidate_lock); + + for (i = start; i < end; i++) { + /* + * If the folio is NONE_SHARED, it indicates that it is + * transitioning to private (GUEST_SHARED). Transition it to + * shared (ALL_SHARED) and remove the callback. + */ + if (xa_to_value(xa_load(shared_offsets, i)) == KVM_GMEM_NONE_SHARED) { + struct folio *folio = folio = filemap_lock_folio(inode->i_mapping, i); + + if (!WARN_ON_ONCE(IS_ERR(folio))) { + if (folio_test_guestmem(folio)) + kvm_gmem_restore_pending_folio(folio, inode); + + folio_unlock(folio); + folio_put(folio); + } + } + + xa_erase(shared_offsets, i); + } +} + /* * Marks the range [start, end) as not shared with the host. If the host doesn't * have any references to a particular folio, then that folio is marked as