From patchwork Wed Dec 11 10:37:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13903322 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F2FEE77182 for ; Wed, 11 Dec 2024 10:37:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D0C166B0112; Wed, 11 Dec 2024 05:37:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C95526B0113; Wed, 11 Dec 2024 05:37:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FC576B0114; Wed, 11 Dec 2024 05:37:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7BB536B0112 for ; Wed, 11 Dec 2024 05:37:38 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3372AAE97B for ; Wed, 11 Dec 2024 10:37:38 +0000 (UTC) X-FDA: 82882326282.10.48212D7 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf08.hostedemail.com (Postfix) with ESMTP id ECC0B160002 for ; Wed, 11 Dec 2024 10:37:20 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=uo96PtYS; spf=pass (imf08.hostedemail.com: domain of 3bmtZZwkKCOsNYVPRelUYTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3bmtZZwkKCOsNYVPRelUYTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733913431; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eeSYOeE5pMMeyg7ZhXQAlvtxanJpaTuPefZQ5vxq80o=; b=dUwIM6UKzFz1zTYJ9Dnpw6xV/ElSf98EWW3qehIavytTNuOQtRcrrL9cnVxNZ/HN3zsjKT A0KiDOWUiPP/fOQNZIY0OJYovnp1uVOzhk4qoEMlhgV5wWcLZX/8aqlgEcFOr0HC4s18bq aULgtT7ScbE/HuwjztIOmna7VS8KahQ= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=uo96PtYS; spf=pass (imf08.hostedemail.com: domain of 3bmtZZwkKCOsNYVPRelUYTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3bmtZZwkKCOsNYVPRelUYTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733913432; a=rsa-sha256; cv=none; b=jwQGG/VBeDZsl44EkgTqkQt3fc8boq2nNrdFv156hHbuN+5dL+F0nJWmAD1EpA0HeMDXBr U+A15ObC4Japw7AGEr+9YCwdIUC83Ye5pY0u/xJe5bMQrew7MscoZp3j9lJmwZlsYFR/Bg 13fKZPL4J5MBm354UKqlDUn4NSLy4nc= Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-434f852cb35so2245185e9.0 for ; Wed, 11 Dec 2024 02:37:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1733913455; x=1734518255; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=eeSYOeE5pMMeyg7ZhXQAlvtxanJpaTuPefZQ5vxq80o=; b=uo96PtYSpX3U9Gx7sqNHT+wIo7h5tk8x3IFcF2eeXmQ/dCU1VOs5z1MN0+OylDAnS1 gtVfhq1mQY0wFgpTmI22VrLG7bpIGLEkFL+TfpnEMUFJ1BGRNfx5CjIuUco3nzVMPC6v EVkhQmLoxKvRveHirb4jDPVw1twmJWbDxMCWOF8Du3V8+1rIRtHfXtlPrd8teuzeJDe3 oVbZ/r6M/oru23Dit3nrjRZpZu5fArOGx+rEyjHGCbhgrnIhuBhFXECkPqWX4GoWTvxv MwlfFhV3kuAl1yDyUZe/SFZlmCiBQIvbqG4Bv2uTilibU6F8obHfVAxV1GcKEyRoBS+j xA6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733913455; x=1734518255; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eeSYOeE5pMMeyg7ZhXQAlvtxanJpaTuPefZQ5vxq80o=; b=uDl5BrKSllsLQMzY1ecJeaoQ9OC271MPAqOQDd4qi415HswtZ1Ny4REiB7RXuJ7/aD mP/9+ulnyLXGv8FI2nsJ0yAQSSp2nSZOJZbCt+5BY/dOAv7ps2T4KdvJ8X/i4X4bQQwI m+QMVirfaQTXS2rW3+3R+DwAndWFmadh9Sw9i/Uf4JveQCobvc++PmKx97uyWg9pHNRa QlJQTmjSqM1jP6y5wZ8GTL+5ycbnLWmBB4QAadPGQMVHB6WIAHqMRq3kvmxNTFhiEmuO VOzTn9SQc0Rr8B+7+0yydAVpnbaArryoZUeqtDadLUHkFiMWDzulkKoR8lJsNu+zncuG shfw== X-Forwarded-Encrypted: i=1; AJvYcCU8ae75MG28uVim7lPI1EV8Eaki12AL+f6R8CZ7Ri4xp7HbhOP2I3i52iCIxdUVVHeEtSSf1icIBA==@kvack.org X-Gm-Message-State: AOJu0YzBWLvqcraMaainiDDq5TR8gOIbxO/vPDZwMyniHBTlLmx+Mt1j MFOfvNvW2a5smm35RrzrceBAFNT9hKZrNhzQ7PhxS/Pm7PyKDaedv1Th4JJFsbpEwbybV8sXPX7 1uL5MpX5khtdelg== X-Google-Smtp-Source: AGHT+IGlVGDsXAS8QTt20ZhjhUvb4+iLhx6imu5M/qvMwlRkpVFZ0hN9qToGEpQgPFG5rFlBMfq0S5s6OEGbDoM= X-Received: from wmbd2.prod.google.com ([2002:a05:600c:58c2:b0:434:f513:bb24]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:b86:b0:434:f623:9ff3 with SMTP id 5b1f17b1804b1-4361c382ce3mr18383385e9.15.1733913454954; Wed, 11 Dec 2024 02:37:34 -0800 (PST) Date: Wed, 11 Dec 2024 10:37:05 +0000 In-Reply-To: <20241211-vma-v11-0-466640428fc3@google.com> Mime-Version: 1.0 References: <20241211-vma-v11-0-466640428fc3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=10055; i=aliceryhl@google.com; h=from:subject:message-id; bh=BP0rHFtdcn3zxhmWpCsC4fRZsGWpHTVijKhPN/V33Gw=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnWWtmTO1N1K8DUC6NoDXf02ue/qobHQiKrkrWy Qgkt4oEYx2JAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ1lrZgAKCRAEWL7uWMY5 Rk37D/wKqutxvgqTXRQSIuDKk6I9CScFzIL8g8BqDE1KNEKiN6wrIWLFEcpw9BSH85DoRLdm0y2 XGydQAJ5iIJzfHWTw4k5RvMDkr8JHrdMTYTNLo9w49UHdABMFkiSk2ZRQ4p97t2rcPk4DciGYtB LYzQdCJPLhvlnk4j125r3wgO+8a1PgVElFmYrZZG42nFk3Do3gamix8Db9R8M6/l5Zl9o2uObsA ONIlvWmIedb1NJ5V/LfYc/HhYoSZT3wYNQRjET2d0352qRxyipt4oXDJfEOnf4PUc9WElAfZaEo Itha4cyfEjOZjky9RLsKXWzEQFyadVxgjxdtYQwzOScECwI9iVKIvRhYYHdjLqmmjwW0rkktYVb DDUuSOD3u9KPp1UEtAxW/wWaPN3JUFL2Ng+nkc5V7nbSzlMXF22g5ytMA0rpK1QOC5NIhqVXDup SbLgi25Ot6vGwTcrGyxuBbQjXTVfV51yZpVQPW96b3IF30Pd8OHeMa2bCusmjZU+Po5vkEtOpoY 8GLg8cf1on2Jt21bMRjq6S4gH4r9TLiZ5WgNBhTp1CAc443b92p/AVEzZshjGavyrKiMhSkBnbR j7QpB7Q3pSWCg1HJHzVSwr/MX9VStblyNEDSRZL9+R4YR7h3FeeWbC+qJWw7PscIJWVRRd+QSWf a+c0FhQRhWjjrTg== X-Mailer: b4 0.13.0 Message-ID: <20241211-vma-v11-1-466640428fc3@google.com> Subject: [PATCH v11 1/8] mm: rust: add abstraction for struct mm_struct From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: ECC0B160002 X-Rspam-User: X-Stat-Signature: 3ogud9wobss5dqb874bftkacuyn6njdt X-HE-Tag: 1733913440-6392 X-HE-Meta: U2FsdGVkX1+q4/tTdIPyIGT62HQFtHV3e/NL/dL82ICnpdNkZRSEs4RDDL1mxrKYz0AyollF8+2w4qRK1tGonUy+xWwH+KzvYMyfWpviqL2fL3lZ0rhzbENsbtBwlQxvprsfjhvi+bmbRDF+n2IHTUsbf85HG4axYpKUQ1zaSF3B6AUSkjWYDf49DwvGmxMypzlSCC9BQwXLUKqs0VDt+cznlclhJFgDhtFMMpKBEQy6rN4cTFH1BUiauWiB4fEoRP6Obn24pPq/LDIbeSp41lVgdiBSKkIVm/mrdPwNdsDVBowScV4P6WfCUIwHijvB7WE4MgzbcthZc4YKwSoZ64os85DE/mslbIWbRV+9JLQQAw4MzA9MrqnPN/JC31IyS9S9GKSDQ3J764kUrCWoAWntOye5J02ns19s+R8ZB/PTaSsBvto1m7l/c5NCbYmZhDtEEdCantkHkl+yvqQt+jwv6i+aTi5JzTVCHEHxs2a9eePdoJ0f5ucVG0/MjAaM1W7ctZ2/vpLNBU/vAKs2IEAt2kg8R0wZZheWD+ANo883OMnGur72dPL9ZmtFG79+HBrK7HQQwSltIZ+XMl8Ou2MfRifkYXyf7AAHva9KA92aNqdaWSg5hNVz5oNmQblHkEVx4W69hiOaJXRdvViaHBwKlUqH4jvjG0qM/wj1m22qZ3oK+0v9ar9/Vk9SErgltzgOdz4WNQfUZyKrvg6Riw14x3zhQwK1+jOZ4i37IXifJzqlVxZSQ/bMhaxkSDilmVQXY2zcJE1YW2E+QGpnBZ7eumkcslBQ9qhWmcjES/ATKM3BZb9Cr3Se+VtxgIcTTtO+IVUVLZtOJXoA5Emn5vDllv4fpFC2DC8uO50fS1Gbift2tFUbhCz3uFthVYdSWHFfMJ0g3aQMDPZlD2FaFpsP0YM4Zh2T1GKoMNy9q17hnCRuDn4vGcWFNb0yQ/b/eo+bLuQtcHgXwME/cNK WreiORmh NbG6JMxzG3VN3WWOPfIS1hAB1TwXRxMvSeuEsjJmXaeWPvufCBfP+zuHnOuVkFaDa9KL2+faGmEBXZ/acDGJ/1+rNivaoW/laK0RR5e7JCts4GWb1+ggs6QdYMyCbIuDPzerV1MbVWaJEuCGaFaTzOGHcUJgo283zJTa0FkhBXgmTIlx1FEQXu56TyhAPygRJy8dPsQhJ2AjQXtwoMpT64fZuR0NCh1Hq50yRx8kxRaPlg7hAbLsU126+MVGypqMfD8YzTydl91biZeKfIyPUmIf+GfnYgTxm9y+D9OJR3e03CHJzg0/7w53xKbD5maFR8z8GAs3uSKvhSp3HP5KP5HwR8CwYkvFsZ+g3y3w8t7FFVchUCJDeOuqPIUTHQoSrW9Y6mVvxwleICYrgtVL+Hkg4UDOA2J9Ny2nvNzfvz8BtPAr4ws8zRgJrIYVTUMUPhdk1LMQ68nds7gLqDGLeIB+S18Gq9vHGG2Izr1JHb/SbConB8nrBz5BUuK7AIjoHrgeK2ilRi5Dz176bUCb1cbDLO9agkiiW/c2P2d4KsIPWdIW7w2bcz8ijhu7/cIuuRrvMOh5usNvTnxZP8HhaCJ0uQA51N661O+VHiQPhnRsJDfxsnz9yr1nDjAQwuX+bQUnkWX4UBcHcHTdvn8a5Ktd9jMo/kNub2pEesibXRXD1KI5xtSq9AWoPETALqBnYD3JhUy68Y2mzVQA= X-Bogosity: Unsure, tests=bogofilter, spamicity=0.499547, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: These abstractions allow you to reference a `struct mm_struct` using both mmgrab and mmget refcounts. This is done using two Rust types: * Mm - represents an mm_struct where you don't know anything about the value of mm_users. * MmWithUser - represents an mm_struct where you know at compile time that mm_users is non-zero. This allows us to encode in the type system whether a method requires that mm_users is non-zero or not. For instance, you can always call `mmget_not_zero` but you can only call `mmap_read_lock` when mm_users is non-zero. It's possible to access current->mm without a refcount increment, but that is added in a later patch of this series. Acked-by: Lorenzo Stoakes (for mm bits) Signed-off-by: Alice Ryhl --- rust/helpers/helpers.c | 1 + rust/helpers/mm.c | 39 +++++++++ rust/kernel/lib.rs | 1 + rust/kernel/mm.rs | 219 +++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 260 insertions(+) diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c index dcf827a61b52..9d748ec845b3 100644 --- a/rust/helpers/helpers.c +++ b/rust/helpers/helpers.c @@ -16,6 +16,7 @@ #include "fs.c" #include "jump_label.c" #include "kunit.c" +#include "mm.c" #include "mutex.c" #include "page.c" #include "pid_namespace.c" diff --git a/rust/helpers/mm.c b/rust/helpers/mm.c new file mode 100644 index 000000000000..7201747a5d31 --- /dev/null +++ b/rust/helpers/mm.c @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +void rust_helper_mmgrab(struct mm_struct *mm) +{ + mmgrab(mm); +} + +void rust_helper_mmdrop(struct mm_struct *mm) +{ + mmdrop(mm); +} + +void rust_helper_mmget(struct mm_struct *mm) +{ + mmget(mm); +} + +bool rust_helper_mmget_not_zero(struct mm_struct *mm) +{ + return mmget_not_zero(mm); +} + +void rust_helper_mmap_read_lock(struct mm_struct *mm) +{ + mmap_read_lock(mm); +} + +bool rust_helper_mmap_read_trylock(struct mm_struct *mm) +{ + return mmap_read_trylock(mm); +} + +void rust_helper_mmap_read_unlock(struct mm_struct *mm) +{ + mmap_read_unlock(mm); +} diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index e1065a7551a3..6555e0847192 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -46,6 +46,7 @@ pub mod kunit; pub mod list; pub mod miscdevice; +pub mod mm; #[cfg(CONFIG_NET)] pub mod net; pub mod page; diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs new file mode 100644 index 000000000000..84cba581edaa --- /dev/null +++ b/rust/kernel/mm.rs @@ -0,0 +1,219 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2024 Google LLC. + +//! Memory management. +//! +//! C header: [`include/linux/mm.h`](srctree/include/linux/mm.h) + +use crate::{ + bindings, + types::{ARef, AlwaysRefCounted, NotThreadSafe, Opaque}, +}; +use core::{ops::Deref, ptr::NonNull}; + +/// A wrapper for the kernel's `struct mm_struct`. +/// +/// Since `mm_users` may be zero, the associated address space may not exist anymore. You can use +/// [`mmget_not_zero`] to be able to access the address space. +/// +/// The `ARef` smart pointer holds an `mmgrab` refcount. Its destructor may sleep. +/// +/// # Invariants +/// +/// Values of this type are always refcounted using `mmgrab`. +/// +/// [`mmget_not_zero`]: Mm::mmget_not_zero +#[repr(transparent)] +pub struct Mm { + mm: Opaque, +} + +// SAFETY: It is safe to call `mmdrop` on another thread than where `mmgrab` was called. +unsafe impl Send for Mm {} +// SAFETY: All methods on `Mm` can be called in parallel from several threads. +unsafe impl Sync for Mm {} + +// SAFETY: By the type invariants, this type is always refcounted. +unsafe impl AlwaysRefCounted for Mm { + #[inline] + fn inc_ref(&self) { + // SAFETY: The pointer is valid since self is a reference. + unsafe { bindings::mmgrab(self.as_raw()) }; + } + + #[inline] + unsafe fn dec_ref(obj: NonNull) { + // SAFETY: The caller is giving up their refcount. + unsafe { bindings::mmdrop(obj.cast().as_ptr()) }; + } +} + +/// A wrapper for the kernel's `struct mm_struct`. +/// +/// This type is like [`Mm`], but with non-zero `mm_users`. It can only be used when `mm_users` can +/// be proven to be non-zero at compile-time, usually because the relevant code holds an `mmget` +/// refcount. It can be used to access the associated address space. +/// +/// The `ARef` smart pointer holds an `mmget` refcount. Its destructor may sleep. +/// +/// # Invariants +/// +/// Values of this type are always refcounted using `mmget`. The value of `mm_users` is non-zero. +#[repr(transparent)] +pub struct MmWithUser { + mm: Mm, +} + +// SAFETY: It is safe to call `mmput` on another thread than where `mmget` was called. +unsafe impl Send for MmWithUser {} +// SAFETY: All methods on `MmWithUser` can be called in parallel from several threads. +unsafe impl Sync for MmWithUser {} + +// SAFETY: By the type invariants, this type is always refcounted. +unsafe impl AlwaysRefCounted for MmWithUser { + #[inline] + fn inc_ref(&self) { + // SAFETY: The pointer is valid since self is a reference. + unsafe { bindings::mmget(self.as_raw()) }; + } + + #[inline] + unsafe fn dec_ref(obj: NonNull) { + // SAFETY: The caller is giving up their refcount. + unsafe { bindings::mmput(obj.cast().as_ptr()) }; + } +} + +// Make all `Mm` methods available on `MmWithUser`. +impl Deref for MmWithUser { + type Target = Mm; + + #[inline] + fn deref(&self) -> &Mm { + &self.mm + } +} + +// These methods are safe to call even if `mm_users` is zero. +impl Mm { + /// Call `mmgrab` on `current.mm`. + #[inline] + pub fn mmgrab_current() -> Option> { + // SAFETY: It's safe to get the `mm` field from current. + let mm = unsafe { + let current = bindings::get_current(); + (*current).mm + }; + + if mm.is_null() { + return None; + } + + // SAFETY: The value of `current->mm` is guaranteed to be null or a valid `mm_struct`. We + // just checked that it's not null. Furthermore, the returned `&Mm` is valid only for the + // duration of this function, and `current->mm` will stay valid for that long. + let mm = unsafe { Mm::from_raw(mm) }; + + // This increments the refcount using `mmgrab`. + Some(ARef::from(mm)) + } + + /// Returns a raw pointer to the inner `mm_struct`. + #[inline] + pub fn as_raw(&self) -> *mut bindings::mm_struct { + self.mm.get() + } + + /// Obtain a reference from a raw pointer. + /// + /// # Safety + /// + /// The caller must ensure that `ptr` points at an `mm_struct`, and that it is not deallocated + /// during the lifetime 'a. + #[inline] + pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a Mm { + // SAFETY: Caller promises that the pointer is valid for 'a. Layouts are compatible due to + // repr(transparent). + unsafe { &*ptr.cast() } + } + + /// Calls `mmget_not_zero` and returns a handle if it succeeds. + #[inline] + pub fn mmget_not_zero(&self) -> Option> { + // SAFETY: The pointer is valid since self is a reference. + let success = unsafe { bindings::mmget_not_zero(self.as_raw()) }; + + if success { + // SAFETY: We just created an `mmget` refcount. + Some(unsafe { ARef::from_raw(NonNull::new_unchecked(self.as_raw().cast())) }) + } else { + None + } + } +} + +// These methods require `mm_users` to be non-zero. +impl MmWithUser { + /// Obtain a reference from a raw pointer. + /// + /// # Safety + /// + /// The caller must ensure that `ptr` points at an `mm_struct`, and that `mm_users` remains + /// non-zero for the duration of the lifetime 'a. + #[inline] + pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser { + // SAFETY: Caller promises that the pointer is valid for 'a. The layout is compatible due + // to repr(transparent). + unsafe { &*ptr.cast() } + } + + /// Lock the mmap read lock. + #[inline] + pub fn mmap_read_lock(&self) -> MmapReadGuard<'_> { + // SAFETY: The pointer is valid since self is a reference. + unsafe { bindings::mmap_read_lock(self.as_raw()) }; + + // INVARIANT: We just acquired the read lock. + MmapReadGuard { + mm: self, + _nts: NotThreadSafe, + } + } + + /// Try to lock the mmap read lock. + #[inline] + pub fn mmap_read_trylock(&self) -> Option> { + // SAFETY: The pointer is valid since self is a reference. + let success = unsafe { bindings::mmap_read_trylock(self.as_raw()) }; + + if success { + // INVARIANT: We just acquired the read lock. + Some(MmapReadGuard { + mm: self, + _nts: NotThreadSafe, + }) + } else { + None + } + } +} + +/// A guard for the mmap read lock. +/// +/// # Invariants +/// +/// This `MmapReadGuard` guard owns the mmap read lock. +pub struct MmapReadGuard<'a> { + mm: &'a MmWithUser, + // `mmap_read_lock` and `mmap_read_unlock` must be called on the same thread + _nts: NotThreadSafe, +} + +impl Drop for MmapReadGuard<'_> { + #[inline] + fn drop(&mut self) { + // SAFETY: We hold the read lock by the type invariants. + unsafe { bindings::mmap_read_unlock(self.mm.as_raw()) }; + } +} From patchwork Wed Dec 11 10:37:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13903323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DEB4E77180 for ; Wed, 11 Dec 2024 10:37:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0658D6B0114; Wed, 11 Dec 2024 05:37:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EDF646B0116; Wed, 11 Dec 2024 05:37:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D31036B0118; Wed, 11 Dec 2024 05:37:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AAC506B0114 for ; Wed, 11 Dec 2024 05:37:40 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 369C5425BD for ; Wed, 11 Dec 2024 10:37:40 +0000 (UTC) X-FDA: 82882326114.27.CF0141A Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf15.hostedemail.com (Postfix) with ESMTP id 4DDFFA0006 for ; Wed, 11 Dec 2024 10:37:13 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="ZcK/0+MX"; spf=pass (imf15.hostedemail.com: domain of 3cWtZZwkKCO4QbYSUhoXbWeeWbU.SecbYdkn-ccalQSa.ehW@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3cWtZZwkKCO4QbYSUhoXbWeeWbU.SecbYdkn-ccalQSa.ehW@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733913442; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ptCVgSajmLGd+33xQ7/1wwSdfyjzIQ8tWfofuzm1Y9U=; b=OYhaSOo6C6J+ASmF/8wo6daPRlTdinUKuJ6G05ASrhDo1BcnHnaSICKapcKGHqxFWRH3Bu 2WxJZxsv3NaaAN0aJAkapYaVfoEhtE2bQ/4NFpllKSWKW9rOqwfjfKb/tuoPNaDSOjVh5m OrD5KMOEynl4f+RXdBq1LrmshGHmXko= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="ZcK/0+MX"; spf=pass (imf15.hostedemail.com: domain of 3cWtZZwkKCO4QbYSUhoXbWeeWbU.SecbYdkn-ccalQSa.ehW@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3cWtZZwkKCO4QbYSUhoXbWeeWbU.SecbYdkn-ccalQSa.ehW@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733913442; a=rsa-sha256; cv=none; b=nm8s306EXmB7P45uEG5PHDIAkhoY+9bdWAMC4cANHmshDJ1H2Hygn6xXfUkosFUaUK7gGK 444zJ8GwRZQMKtQia5z0eYBZxLjTYHM2G13ZZx6G9nobC8+IcUBxsEY6VYn4fZOEKskvHP lTSa03By334d8xJ9FdBHxTs7Oo+iYGs= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-434e8beeed9so37307355e9.1 for ; Wed, 11 Dec 2024 02:37:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1733913457; x=1734518257; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ptCVgSajmLGd+33xQ7/1wwSdfyjzIQ8tWfofuzm1Y9U=; b=ZcK/0+MXlNLYHhV8XckhhL9wssryRwNcK1VMrd6Pb+aDhQR8aJfs6+LmPQzYCv9yGi Fl23ZLn8NNPjksk5hbpo8BA9LhLyoDyLPffuGTTH51wi3m+SwE98dmL+Gc6zZ5fTM++H RHiXCSDP8CyBpCwebUd003LMjaH8jGQBnynNuhxlshzc4BJFRtbe5uNY2GmbpzO61Ytj XjCJJoKvKtqhl535A8DxyjGDXRzIG3Lb4cI50k0TtK4qC/Qe1dN8C3jgkEPa9uaVnoYi BtVxsZ45kKAow1TtCay5Bx1GLOvM9O+TlvcauejrSMUzbcPUp4+u0ESJankpAU2Zt2Wr rR4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733913457; x=1734518257; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ptCVgSajmLGd+33xQ7/1wwSdfyjzIQ8tWfofuzm1Y9U=; b=RG6rmw9xpJ4MtOHOOw+VmBnJKf5YyIvh8AVZmQdNuPOAUGSbfYMzTRhJIdWsZeTT7+ pehnynfAeIwMmd7rji6fzJ064lLl21LauVbOuaK/4brfphCugqDdU49LyUfov2OMvt8o Z5oQ0zG+kTIcyWd5+9bFhJAVDNo+WDRXOUq77TfXCtKDQLPNP2RcvCWYBG9cJxO5e6aq P8QNaNjDPltE7bR/eiR+QFaN4jzg/JILkqJdVzRlIPEBy0ogv/46X0x4Opw0QcU6DcgB 8sZKp+/rkvsNK+AJEGRoUqGCYxZ7GQ+p/mTj5tyIpnBgAc8t0LUWIGa2v3wh4vMvLBm7 iqvw== X-Forwarded-Encrypted: i=1; AJvYcCUxRiRKKxGwz4Yom5VVwXBFjB6EvkzZw+Ashxa3EYA6FU8IZ5m8Yqc15VU8Ih9FjaV3Rh226XiH4Q==@kvack.org X-Gm-Message-State: AOJu0YxMyDvxYeObdJu6tKS1UWDqCS235iSrmWF8Bhyx9lfnZFMlJ2Ka YskLwIy4g8cZlWQuM/uCAhTxJTFEE1aKiM7c89hoVleQzGsoGCvUf3kbdtXzZZHs0cV0UdHj0ff lia0ZSzvWm064Jw== X-Google-Smtp-Source: AGHT+IF2oHVYbWX6H19GjEN2XS8y+OVVbBpLGUljSwx1ScMsBgY3j4qEcrJlD9I/H+Lzexp4LvuOcuGlpE8PUZM= X-Received: from wmpd23.prod.google.com ([2002:a05:600c:4c17:b0:436:16c6:831]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1d20:b0:434:f218:e1a8 with SMTP id 5b1f17b1804b1-4361c3c70eamr16834795e9.19.1733913457042; Wed, 11 Dec 2024 02:37:37 -0800 (PST) Date: Wed, 11 Dec 2024 10:37:06 +0000 In-Reply-To: <20241211-vma-v11-0-466640428fc3@google.com> Mime-Version: 1.0 References: <20241211-vma-v11-0-466640428fc3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=10059; i=aliceryhl@google.com; h=from:subject:message-id; bh=UELtGY1YbfyD3E/jKfzFIZ073fFyrrrn2baNLFYoZH8=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnWWtmJY7Ptcy7pNZrWwIGonxUXNeWa5B1NRzbo /1Y1X00++WJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ1lrZgAKCRAEWL7uWMY5 RisoD/9s1GclB/tPV2RQgJj7Nws8WY0RQiMF3+XoqpaXLo7CBWgXqFtqDRnmEDtnPSRjPNjZDUw vn76Q3hqvoFexzTpBLEIXNpq+0aW62Rs4WYTSB9EK4l03q8OIXaLWOuNzNlIFnkk5FWJQnnVctY 4WaWo/DVhnFu3CckEwyhSjyZaC6tPRmjhEOV0MV9Zbc8e1VQXzd6gIuukpFAHK46aLIKXZwmPa7 J9pTslBugUwDx0t/QdX6IDb8rto2Ux9i/12Ph2fwYr/mmAY0XkIqc3MavawP8NkoK9IqXgWDVfH OAk1I8Nd9M4GaTxeIr8Wqn3+VcYuY2axQuPSdU+N+/CI/5kSIxDN++PqFJG1ohgzX3zgVUHioTH 5ZwPVM+FGEGb2b6Lv4iIn2uoVLcgsE3qxZetWkS/wYejQJ8GS0tvdtyuY0zkLLM5uQqH6ZBqFMR VHC+va0R5zAofKnL4ktwO3H/+1SBj2euS1cjsFFE2EIDCz3ENNd32tgOwboechRxR0AXiyPxyuE Iw12dKxpN9KICnBsQaXZZMbK3xJQ5wsZ74pST127xzw0893fVbw2sKL5BfGFNVccPJv/QWbMq41 de5sNMp11t7DLip+VQnwMBl5Qlb/GCogJOR7gtVxW7AA1y9Eaqfqjj38Lo9unyrdcERTCEq0e+I leQBcaYv82zdrBw== X-Mailer: b4 0.13.0 Message-ID: <20241211-vma-v11-2-466640428fc3@google.com> Subject: [PATCH v11 2/8] mm: rust: add vm_area_struct methods that require read access From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam05 X-Stat-Signature: miqrh1hhzamkujxhbcggrdxd8z1t5ckn X-Rspamd-Queue-Id: 4DDFFA0006 X-Rspam-User: X-HE-Tag: 1733913433-872481 X-HE-Meta: U2FsdGVkX18HdtWaIHR0tJkQ2PHseFbR3JdkX1O6wRRYNWyBe+VD4hqmuR5ZTcMmz6+b8owD7JI++fE8y8m6tCy1SwgNHmeYam1PH3C0MV67VXsNkz9Wh8D95X3OcAtLa42g26y/X8vvWvU1TVOltpr7uITbwkoa/8oclW+6pN3kAC2q6gT9WLenMRpUft/nl2pqNdJ0SLFGfgT4E/Giu05XZn0eJllrHHmHqdcCs+5kvfjORfWwaQR5Gk7tD2AOnbE2E29dXMjzguxiyl9PV8ClcVSZRSO/JA131UFtWKJ/E0MvCtUDBTl7B4ZYsD0argclY4Te60Kxkj8wu5Ki09oFIrfT6IbMsify4Fn2uVyCNm48tXEi58YRu7kh3BD1pHVjagjLFADN/yTFUJtqio5J9MeuBno68hqRS554EOHa0C0mcvRRX0aeW2+/MwynHcO0bFpHaGCKKOkgfx1lhaG+LMkrFo+9HdtEA1Bua7rPR3atbqlWwyWtvUy8pS6/sC0HsVXAfQdsiqBRil8gw1XSfw0bLpET2D6kGqztT89B7FiGSUyDx7RuZ5JE6qB2HjRu7/lgh0aumVjNOBmf5LYaGOStltW4vPxDP/dBXmKwCdpCeDpKlqyfwR7llv18pGTpwIQIDAN+E4zHM4JZdfjVcfjTN9dWkr28L0OyFrG63B2gGmK3QQyGEYNwH/AR35MsuFcmjuwenp4qhLGircIlCcmfPRucl/VioV2U4UqmzLClGg1tltjVeM7vdJA4LSVFVRSuk5vOAPW2Vy1vtMgz9QFaqUsbWhF+aFza0qU9ofQbvmqpaAamBdP2w8X7OyZghm0fNqfDLvADteDxvvLUG0rwoSXf4hj7Oo1z2Di3mrxb4R2pj4v6WDkXceO/w9OKg8sGVEqvbaG1ke1pUH5K7woAs7w2kgQ100iPPGsSSsx3/Bdww7roSXEihcfkcliyH+TmxqdiRQop799 e+SovfxA CdleTzZkVQJcxUNf3pBdVJ3WUXlfyZwUMpjqOdlV3sRd3c8KC+bsDPtJvTPVMxsyPRBXsCMMLRLRaMLm04RcAqStbvK8jChM7juoJ/RdC6L+QT7wKKv2/afafEA3bPa0mXdAkgnDqXE+xtG3t+rbIXT1zJ36GyRGH8fUNiEUfCUGiFYTsV8pfd1AvygLVZAVdhhPzPrtGgMc7Qayadnhh3A5ys866EWKy4t/8KxsPSghUgMkLoMQ+62SMN1G6vcb6uZSsU1YHnTFBUMAMq9Jw4mGVJj9/bHgr0q8Mhed03QgEdseI1fpTiT+V1CN1IS8vmlQbctYsm/8GJhSphqmHtIBI6i0mqDYQwxwVqZ+foK767LXjj3beAg582xkGqRKDtPu6M//h9UsuiIZseTtWxTemD0n4Lf2EUeDgkNqUgf+tHJhDx6BD6oKJl6QjYaSrKMZy1muI6nrHy4mZH4P0izHkPeKCz9GDyfLkEbc9jWhih3ZmVV4LYWhFkrauAxU3dw7Sn++vcMi/2JGcyTCtQQi0H3Ji2IfaBuuoLn/biWmzs2ZterW/GWyYBSx3ncjLYjSEhIpONXarPLoosyQJN+tmRAiYFUKQTFmlDX679UPNbWOSw11smXmfanOBvtmFxWp2El0V33/an/AwcTo/nE1bRKsek9y4Pubzau+dXDS1xi6PqrQXVSXbdVQEwU025+/B11FWfg41+Fo= X-Bogosity: Unsure, tests=bogofilter, spamicity=0.492267, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This adds a type called VmAreaRef which is used when referencing a vma that you have read access to. Here, read access means that you hold either the mmap read lock or the vma read lock (or stronger). Additionally, a vma_lookup method is added to the mmap read guard, which enables you to obtain a &VmAreaRef in safe Rust code. This patch only provides a way to lock the mmap read lock, but a follow-up patch also provides a way to just lock the vma read lock. Acked-by: Lorenzo Stoakes (for mm bits) Reviewed-by: Jann Horn Signed-off-by: Alice Ryhl --- rust/helpers/mm.c | 6 ++ rust/kernel/mm.rs | 21 ++++++ rust/kernel/mm/virt.rs | 191 +++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 218 insertions(+) diff --git a/rust/helpers/mm.c b/rust/helpers/mm.c index 7201747a5d31..7b72eb065a3e 100644 --- a/rust/helpers/mm.c +++ b/rust/helpers/mm.c @@ -37,3 +37,9 @@ void rust_helper_mmap_read_unlock(struct mm_struct *mm) { mmap_read_unlock(mm); } + +struct vm_area_struct *rust_helper_vma_lookup(struct mm_struct *mm, + unsigned long addr) +{ + return vma_lookup(mm, addr); +} diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs index 84cba581edaa..ace8e7d57afe 100644 --- a/rust/kernel/mm.rs +++ b/rust/kernel/mm.rs @@ -12,6 +12,8 @@ }; use core::{ops::Deref, ptr::NonNull}; +pub mod virt; + /// A wrapper for the kernel's `struct mm_struct`. /// /// Since `mm_users` may be zero, the associated address space may not exist anymore. You can use @@ -210,6 +212,25 @@ pub struct MmapReadGuard<'a> { _nts: NotThreadSafe, } +impl<'a> MmapReadGuard<'a> { + /// Look up a vma at the given address. + #[inline] + pub fn vma_lookup(&self, vma_addr: usize) -> Option<&virt::VmAreaRef> { + // SAFETY: We hold a reference to the mm, so the pointer must be valid. Any value is okay + // for `vma_addr`. + let vma = unsafe { bindings::vma_lookup(self.mm.as_raw(), vma_addr as _) }; + + if vma.is_null() { + None + } else { + // SAFETY: We just checked that a vma was found, so the pointer is valid. Furthermore, + // the returned area will borrow from this read lock guard, so it can only be used + // while the mmap read lock is still held. + unsafe { Some(virt::VmAreaRef::from_raw(vma)) } + } + } +} + impl Drop for MmapReadGuard<'_> { #[inline] fn drop(&mut self) { diff --git a/rust/kernel/mm/virt.rs b/rust/kernel/mm/virt.rs new file mode 100644 index 000000000000..68c763169cf0 --- /dev/null +++ b/rust/kernel/mm/virt.rs @@ -0,0 +1,191 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2024 Google LLC. + +//! Virtual memory. + +use crate::{bindings, mm::MmWithUser, types::Opaque}; + +/// A wrapper for the kernel's `struct vm_area_struct` with read access. +/// +/// It represents an area of virtual memory. +/// +/// # Invariants +/// +/// The caller must hold the mmap read lock or the vma read lock. +#[repr(transparent)] +pub struct VmAreaRef { + vma: Opaque, +} + +// Methods you can call when holding the mmap or vma read lock (or strong). They must be usable no +// matter what the vma flags are. +impl VmAreaRef { + /// Access a virtual memory area given a raw pointer. + /// + /// # Safety + /// + /// Callers must ensure that `vma` is valid for the duration of 'a, and that the mmap or vma + /// read lock (or stronger) is held for at least the duration of 'a. + #[inline] + pub unsafe fn from_raw<'a>(vma: *const bindings::vm_area_struct) -> &'a Self { + // SAFETY: The caller ensures that the invariants are satisfied for the duration of 'a. + unsafe { &*vma.cast() } + } + + /// Returns a raw pointer to this area. + #[inline] + pub fn as_ptr(&self) -> *mut bindings::vm_area_struct { + self.vma.get() + } + + /// Access the underlying `mm_struct`. + #[inline] + pub fn mm(&self) -> &MmWithUser { + // SAFETY: By the type invariants, this `vm_area_struct` is valid and we hold the mmap/vma + // read lock or stronger. This implies that the underlying mm has a non-zero value of + // `mm_users`. + unsafe { MmWithUser::from_raw((*self.as_ptr()).vm_mm) } + } + + /// Returns the flags associated with the virtual memory area. + /// + /// The possible flags are a combination of the constants in [`flags`]. + #[inline] + pub fn flags(&self) -> vm_flags_t { + // SAFETY: By the type invariants, the caller holds at least the mmap read lock, so this + // access is not a data race. + unsafe { (*self.as_ptr()).__bindgen_anon_2.vm_flags as _ } + } + + /// Returns the (inclusive) start address of the virtual memory area. + #[inline] + pub fn start(&self) -> usize { + // SAFETY: By the type invariants, the caller holds at least the mmap read lock, so this + // access is not a data race. + unsafe { (*self.as_ptr()).__bindgen_anon_1.__bindgen_anon_1.vm_start as _ } + } + + /// Returns the (exclusive) end address of the virtual memory area. + #[inline] + pub fn end(&self) -> usize { + // SAFETY: By the type invariants, the caller holds at least the mmap read lock, so this + // access is not a data race. + unsafe { (*self.as_ptr()).__bindgen_anon_1.__bindgen_anon_1.vm_end as _ } + } + + /// Zap pages in the given page range. + /// + /// This clears page table mappings for the range at the leaf level, leaving all other page + /// tables intact, and freeing any memory referenced by the VMA in this range. That is, + /// anonymous memory is completely freed, file-backed memory has its reference count on page + /// cache folio's dropped, any dirty data will still be written back to disk as usual. + #[inline] + pub fn zap_page_range_single(&self, address: usize, size: usize) { + let (end, did_overflow) = address.overflowing_add(size); + if did_overflow || address < self.start() || self.end() < end { + // TODO: call WARN_ONCE once Rust version of it is added + return; + } + + // SAFETY: By the type invariants, the caller has read access to this VMA, which is + // sufficient for this method call. This method has no requirements on the vma flags. The + // address range is checked to be within the vma. + unsafe { + bindings::zap_page_range_single( + self.as_ptr(), + address as _, + size as _, + core::ptr::null_mut(), + ) + }; + } +} + +/// The integer type used for vma flags. +#[doc(inline)] +pub use bindings::vm_flags_t; + +/// All possible flags for [`VmAreaRef`]. +pub mod flags { + use super::vm_flags_t; + use crate::bindings; + + /// No flags are set. + pub const NONE: vm_flags_t = bindings::VM_NONE as _; + + /// Mapping allows reads. + pub const READ: vm_flags_t = bindings::VM_READ as _; + + /// Mapping allows writes. + pub const WRITE: vm_flags_t = bindings::VM_WRITE as _; + + /// Mapping allows execution. + pub const EXEC: vm_flags_t = bindings::VM_EXEC as _; + + /// Mapping is shared. + pub const SHARED: vm_flags_t = bindings::VM_SHARED as _; + + /// Mapping may be updated to allow reads. + pub const MAYREAD: vm_flags_t = bindings::VM_MAYREAD as _; + + /// Mapping may be updated to allow writes. + pub const MAYWRITE: vm_flags_t = bindings::VM_MAYWRITE as _; + + /// Mapping may be updated to allow execution. + pub const MAYEXEC: vm_flags_t = bindings::VM_MAYEXEC as _; + + /// Mapping may be updated to be shared. + pub const MAYSHARE: vm_flags_t = bindings::VM_MAYSHARE as _; + + /// Page-ranges managed without `struct page`, just pure PFN. + pub const PFNMAP: vm_flags_t = bindings::VM_PFNMAP as _; + + /// Memory mapped I/O or similar. + pub const IO: vm_flags_t = bindings::VM_IO as _; + + /// Do not copy this vma on fork. + pub const DONTCOPY: vm_flags_t = bindings::VM_DONTCOPY as _; + + /// Cannot expand with mremap(). + pub const DONTEXPAND: vm_flags_t = bindings::VM_DONTEXPAND as _; + + /// Lock the pages covered when they are faulted in. + pub const LOCKONFAULT: vm_flags_t = bindings::VM_LOCKONFAULT as _; + + /// Is a VM accounted object. + pub const ACCOUNT: vm_flags_t = bindings::VM_ACCOUNT as _; + + /// Should the VM suppress accounting. + pub const NORESERVE: vm_flags_t = bindings::VM_NORESERVE as _; + + /// Huge TLB Page VM. + pub const HUGETLB: vm_flags_t = bindings::VM_HUGETLB as _; + + /// Synchronous page faults. (DAX-specific) + pub const SYNC: vm_flags_t = bindings::VM_SYNC as _; + + /// Architecture-specific flag. + pub const ARCH_1: vm_flags_t = bindings::VM_ARCH_1 as _; + + /// Wipe VMA contents in child on fork. + pub const WIPEONFORK: vm_flags_t = bindings::VM_WIPEONFORK as _; + + /// Do not include in the core dump. + pub const DONTDUMP: vm_flags_t = bindings::VM_DONTDUMP as _; + + /// Not soft dirty clean area. + pub const SOFTDIRTY: vm_flags_t = bindings::VM_SOFTDIRTY as _; + + /// Can contain `struct page` and pure PFN pages. + pub const MIXEDMAP: vm_flags_t = bindings::VM_MIXEDMAP as _; + + /// MADV_HUGEPAGE marked this vma. + pub const HUGEPAGE: vm_flags_t = bindings::VM_HUGEPAGE as _; + + /// MADV_NOHUGEPAGE marked this vma. + pub const NOHUGEPAGE: vm_flags_t = bindings::VM_NOHUGEPAGE as _; + + /// KSM may merge identical pages. + pub const MERGEABLE: vm_flags_t = bindings::VM_MERGEABLE as _; +} From patchwork Wed Dec 11 10:37:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13903324 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31A5BE7717D for ; Wed, 11 Dec 2024 10:37:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E92B78D0013; Wed, 11 Dec 2024 05:37:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E1C3F6B0119; Wed, 11 Dec 2024 05:37:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C23358D0013; Wed, 11 Dec 2024 05:37:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 986456B0118 for ; Wed, 11 Dec 2024 05:37:42 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 55E64A0C57 for ; Wed, 11 Dec 2024 10:37:42 +0000 (UTC) X-FDA: 82882326072.08.6B3AFB3 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf12.hostedemail.com (Postfix) with ESMTP id 657D54000C for ; Wed, 11 Dec 2024 10:37:30 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ZH4HiOZa; spf=pass (imf12.hostedemail.com: domain of 3c2tZZwkKCPASdaUWjqZdYggYdW.Ugedafmp-eecnSUc.gjY@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3c2tZZwkKCPASdaUWjqZdYggYdW.Ugedafmp-eecnSUc.gjY@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733913445; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xdRoZ/FXvJ1iNt28gv3dQ+sMcWS78rYWNAmYz+QTguo=; b=eo1pMEUyWmkpCJuLTld21RCDx8W/n/8y16sqkU01sIOc8L5OGNTFonxQoUq5FtpPvVvHt6 cfhwapEnJ1W/Jh4fqXWho5iQD/orq02PJnKdD/aHlIfJ+o3klAo28FuHzeZjqhfGQG0q3G kXJaRUiEkPEgE0ps4gLcPUMs9Q+lkU8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733913445; a=rsa-sha256; cv=none; b=wUmo5aYAKTskzEKp5C6b3JbdEuDkoZdxH/HVfJPhjA1ln2DLP60ZdOW9OC18wQUmcUWnoY yBXAzGMIZqtKNFzu8naPS2NwVYNowxKlVdY9sdHAVJMP/mHDSFiHTYE2bifeleBHaIJHjn 1W4V9sTBzaPYB+4fM1qTRw6Aa4o7yD8= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ZH4HiOZa; spf=pass (imf12.hostedemail.com: domain of 3c2tZZwkKCPASdaUWjqZdYggYdW.Ugedafmp-eecnSUc.gjY@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3c2tZZwkKCPASdaUWjqZdYggYdW.Ugedafmp-eecnSUc.gjY@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4361f371908so3199135e9.0 for ; Wed, 11 Dec 2024 02:37:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1733913459; x=1734518259; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xdRoZ/FXvJ1iNt28gv3dQ+sMcWS78rYWNAmYz+QTguo=; b=ZH4HiOZa/RtMZwof2m2ZKC2rZyOIAmDsIZSuhiMB9/tEtvl63mIq+p+X26iH/mNxKc TqBN+RhY8sTGeGpVL02TATdaqoA64JRM7bZw5AvMM4jVCnOm4IqdcEthtKhDdlz+ERZS SEmLr3o8REzrCcaqoGXb9oIh1ogiidEyjXX04nkCRVVxrga7hCqI1pefcnKuV8ZRGwyS ephoRpPatWbYV//9l5JJknOQY4vGUdJ4HSSzxSYsT++BsW7dKbbbv2SiPi2yWI2TgYVm Wzfm3PswHo7hudGa63c+WVRq4ngC0F5mv75p0MBi+xJQqb7RjETChnLQo9kSnz3IatCI 0LzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733913459; x=1734518259; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xdRoZ/FXvJ1iNt28gv3dQ+sMcWS78rYWNAmYz+QTguo=; b=INRwxKRp5qEe56hsj5i0nmlI1dZJmpf0v5hhVK/PetdPAW2Hux1CFjtqGsRVCtUXkC 6Ur8OVeIbDeShiydW1uwG+Z+0rVUqFPihfccJOeV+Ec66VMv7dMf94cJslZi20gGyUkd pfxOUKWjadXcn2jrxQMGB8S18SBNFqToMQa5tIlQrPR6eo7clnmaPTnu0vvHbpiBGtPL EDtTVIYOkdaDysITT2rqemZvNASHMem+u/k1WOpUyN1t7dZSumf/bs96myIl9ih//4Jd GdMWasBx1vRoLY0iK7iO+7C5rm4h5zQKG9qrpV+pADqq71iwUmcog54sev3aANdPeBsk JEaw== X-Forwarded-Encrypted: i=1; AJvYcCUyAe7/+5Sxu63WGGa/XHvY+oCOqksbTrHOgZrEwOwqDDE8wJxt7aU9JtiO1AxjRbocvYKrflCD1w==@kvack.org X-Gm-Message-State: AOJu0YxwXMIRkYu0yF+ZpPfxvrv8GnGOZ/BoXCGtHSnaHIiF/eG25XDB +usJSKfcznjZTwo8bDnm++I1EsTAoUwvnF0WULZEndPOV+AB19VebVt0udrEahsR7/+bxAoL8xO NbNoyvRPRG1nCtQ== X-Google-Smtp-Source: AGHT+IFiaOcL2ApjopemuAF+csic5VYHrPo4suLSy1p/57Ja4FGM8Qf+KoTAcXifpYCYix4DnL3IX5KOAs2s7jk= X-Received: from wmoy22.prod.google.com ([2002:a05:600c:17d6:b0:436:1995:1888]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4f11:b0:435:23c:e23e with SMTP id 5b1f17b1804b1-4361c3ab01fmr16964065e9.12.1733913459182; Wed, 11 Dec 2024 02:37:39 -0800 (PST) Date: Wed, 11 Dec 2024 10:37:07 +0000 In-Reply-To: <20241211-vma-v11-0-466640428fc3@google.com> Mime-Version: 1.0 References: <20241211-vma-v11-0-466640428fc3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3734; i=aliceryhl@google.com; h=from:subject:message-id; bh=6t+xF+P5/IR/lCGOL4lVeujRJ3nIoK+5J94bGPLjczU=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnWWtn4X75kf6LePY+XZjlPFNolUZoJJ13E+kKv AdV5RmQ4cmJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ1lrZwAKCRAEWL7uWMY5 RsYAEAClhhBwJB76Jj2L7B1ET41hYpXogDIazTH+rKg3i7QZObgXoaE2MPemKRlrV4yMInqgRij eWQIwr4ncHv/JV7zPTLkVOZ3LcVB0UPpgYcTPZ9/18yIS3+Db7En1gZRHrU9YmfFe4Incwr2JQK DB4vDQ2gtEQzfpx2s6Z77STq/wcZLrfnk6guaSaocfPSmf79vOKr18vnHn05DDVEtV6kV65amQT fEDULMlPeyXghGqiA1Xhv7wIq/jdIoFWmXTMgn46Zr5XpBKTwJAyh8GgaS0NrVLXXarNrOgpqjo ygUHM9Opb8NqrgNOae6CNtoBy3bVIE1Ftn8SGnhULvbvGQ2+C2utiYsAkWlaDVLLCmE0v4bow13 BUR5B8FS3dRzJ3d+n5fQkBdXnMw8Y2ovTwu+71t4Vz7hWt1OdmZ+IQCRoqx/DoPrJHRpR2hwmyW R/ZHCvJEbjPPbt+8KnCYToVtc0DTAJGW3I5t3TE+BMC/qBFk0kJ80vj9lwBRD8zh5j3KMToFUqm IBiRpwF1dIyfMCAaWSr6GoixEibdzj31iE3iTylBVC8DbZIl0BzB+K/rxomImUe7AIrEDRAd5R6 1RsaczeisEh5okvkLlrWcVFYbfxpa1IJJuHTKwlA1sHNehU3jTTg+NuvSPAnkHJe4ZL5Zy+MAuk tt79oV/6PfAzmtw== X-Mailer: b4 0.13.0 Message-ID: <20241211-vma-v11-3-466640428fc3@google.com> Subject: [PATCH v11 3/8] mm: rust: add vm_insert_page From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 657D54000C X-Stat-Signature: th9mcdtuiprd8hjxrj1t467ehri5hb13 X-Rspam-User: X-HE-Tag: 1733913450-261983 X-HE-Meta: U2FsdGVkX19zxNPNqXdzzI5SW8I8tQNEoE78IiQ3MuV/zOl7ELAR6k0lGVogaXy8UMfLgX9Sl2wIQNqEsrYAnHS3cyIcTmM6MCJAnVAJpTTO+Alkjdas+zXRa7SAMH3enEhIXO106nDuBc70kUvQCTe/6K75KlY+VIya/y/NhMk0UvoQWpfhKC8BTNmRYCee0tKRBoWNcryWRcU1+eRkQv1h53kBB7KEke95QhfhmU5TgpW5k68TYeJr6uRn/uNjtBOe7eQ3RoYL3dW6YxhEJfyMIqkWmITHH/xKEbsyQsq1xg4DokhYKBFlV6+TeRE+esomwQgaIODtvCFlP+PYSXNxMkQiCjNFB3KwdvYPy2LhtZmOXMMnII3KryZCmbeRbqBM8RqR+Qb6G9k7aik5Y2xfrMsUsAkedPBQ8Ot/a2hdOCOXqNsxfI6DS8HjX1SQVg8ZZ1WLt8U30/B3uUGWgnCzKU0PT4ymwWD5GWMDS2V4x+CggepEY5n0+Y/fTD74DfYWfLPDkMl0WmWVEXkIQKAGu2c4XB+Z1DZr1B1dilREX1jmywp5nrgIwci7DU4bN30JqPSpvhVOwW5xaz4Ri3hkMPJmKKaurPBsxYeglFr1+NxPQpIYZrlUgSfx571QTSFc9CM6pgrsM2eKJb5vKx6J34+UZh83FRBqwQ70fmxns5ahBfwHfZjKGkXk5ZkZ7CSg+NndMA8iWRbo4LunuHMBRqVuE6Vo4qCqqds1oRn4Fg2rUwwKW5Yo0tKj9hkKDrLr6znGPqfA4J60vNsRztpmzRuttG56t+fnF1kiwuI7StCSvTU/g4UEH3/3pGvMtqL7uwTaqNLPu4Tj9DFAKQyiBlclnnKRb0M7uL1DyJe3dRFGH9FSGi3xXoi9hkJ5vReTICOB1itx1c/lLidmkmW3Zr/4GcC1uUUW4YU0FIsWZH3+f6mzJTUwNP+g0EY20SYosyGlFBk7Ck7+xnH GLIojVbp BrlJ8+rbM4JTE36IjpYfwN5+WQ2j4K3DJRccgzwmEy87aTv214Xw4DjLHx/y8tG99kUYYPFjflM5RkkqzxjJwepl/2T+BrQMvnobX5547D0LaJ1FCMYxJLmGxyClPMK+RSTV/qZg+wwCM7FpvV96gXNANM5M6Xik3xzXgUjKYeGMnI2YEGvmwAeXDw8H6jG1vAEsXiRFtZHCCxUPuRBlazh/0dGUPNHJev9WHTH6m2kcumeqmdTIUUXaUP5fe2dl+CKiPgy0ynjiG/QeHuzHZO1b5KwYcIvcYteWfYu4KUuKjlLLBrBunwA/qBMwhcXKoigfGP3JVFTAQinlfoXg1Qabdjw/ZcxAuZYX4OF48snPHKzLCQT438F//GlfqKXFsMzACRQQleca7uj0EjMw02PUp4JMTH8YbEwQ28SV5PJgDUdujg9BGjw/jWSsnZYAtpLWQjOxoMk8I+/gW5M/5gjVzdYi9n+x4KXA/Qe8jd0ekCopXtLokFtSNhv9+ecNBP5DCL9GpqCDcTw1Wx5FDyPX5kUkwOamTCjZfzxa4E78WmS9jFhW6gDDn2ubGz4JVzz4LFdPQENl0PC6QJQHyxqQ1tACr/tTh33kE21tPDrSLO+2CCBAKIXXSleL4B7I0/uSrXZppIIsECPIruJnplfjeBLdTWJBW/qCEdDvNUMAFrH9yABI/zLjf2QCXhEyvwRQDpKZcR3OZgxyP6pZiduqwlaKIIE0ecM2w7TXH5wJcFaomcCBj+rqwv9kKSK8TbplQO/bZ4PUnJjvywqMNiNZE3DfVFQ8WM5Qy X-Bogosity: Unsure, tests=bogofilter, spamicity=0.499856, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The vm_insert_page method is only usable on vmas with the VM_MIXEDMAP flag, so we introduce a new type to keep track of such vmas. The approach used in this patch assumes that we will not need to encode many flag combinations in the type. I don't think we need to encode more than VM_MIXEDMAP and VM_PFNMAP as things are now. However, if that becomes necessary, using generic parameters in a single type would scale better as the number of flags increases. Acked-by: Lorenzo Stoakes (for mm bits) Signed-off-by: Alice Ryhl --- rust/kernel/mm/virt.rs | 71 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 70 insertions(+), 1 deletion(-) diff --git a/rust/kernel/mm/virt.rs b/rust/kernel/mm/virt.rs index 68c763169cf0..3a23854e14f4 100644 --- a/rust/kernel/mm/virt.rs +++ b/rust/kernel/mm/virt.rs @@ -4,7 +4,15 @@ //! Virtual memory. -use crate::{bindings, mm::MmWithUser, types::Opaque}; +use crate::{ + bindings, + error::{to_result, Result}, + mm::MmWithUser, + page::Page, + types::Opaque, +}; + +use core::ops::Deref; /// A wrapper for the kernel's `struct vm_area_struct` with read access. /// @@ -100,6 +108,67 @@ pub fn zap_page_range_single(&self, address: usize, size: usize) { ) }; } + + /// Check whether the `VM_MIXEDMAP` flag is set. + /// + /// This can be used to access methods that require `VM_MIXEDMAP` to be set. + #[inline] + pub fn as_mixedmap_vma(&self) -> Option<&VmAreaMixedMap> { + if self.flags() & flags::MIXEDMAP != 0 { + // SAFETY: We just checked that `VM_MIXEDMAP` is set. All other requirements are + // satisfied by the type invariants of `VmAreaRef`. + Some(unsafe { VmAreaMixedMap::from_raw(self.as_ptr()) }) + } else { + None + } + } +} + +/// A wrapper for the kernel's `struct vm_area_struct` with read access and `VM_MIXEDMAP` set. +/// +/// It represents an area of virtual memory. +/// +/// # Invariants +/// +/// The caller must hold the mmap read lock or the vma read lock. The `VM_MIXEDMAP` flag must be +/// set. +#[repr(transparent)] +pub struct VmAreaMixedMap { + vma: VmAreaRef, +} + +// Make all `VmAreaRef` methods available on `VmAreaMixedMap`. +impl Deref for VmAreaMixedMap { + type Target = VmAreaRef; + + #[inline] + fn deref(&self) -> &VmAreaRef { + &self.vma + } +} + +impl VmAreaMixedMap { + /// Access a virtual memory area given a raw pointer. + /// + /// # Safety + /// + /// Callers must ensure that `vma` is valid for the duration of 'a, and that the mmap read lock + /// (or stronger) is held for at least the duration of 'a. The `VM_MIXEDMAP` flag must be set. + #[inline] + pub unsafe fn from_raw<'a>(vma: *const bindings::vm_area_struct) -> &'a Self { + // SAFETY: The caller ensures that the invariants are satisfied for the duration of 'a. + unsafe { &*vma.cast() } + } + + /// Maps a single page at the given address within the virtual memory area. + /// + /// This operation does not take ownership of the page. + #[inline] + pub fn vm_insert_page(&self, address: usize, page: &Page) -> Result { + // SAFETY: The caller has read access and has verified that `VM_MIXEDMAP` is set. The page + // is order 0. The address is checked on the C side so it can take any value. + to_result(unsafe { bindings::vm_insert_page(self.as_ptr(), address as _, page.as_ptr()) }) + } } /// The integer type used for vma flags. From patchwork Wed Dec 11 10:37:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13903325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6C24E77182 for ; Wed, 11 Dec 2024 10:37:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20FEF8D0015; Wed, 11 Dec 2024 05:37:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 19A966B0119; Wed, 11 Dec 2024 05:37:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6A318D0015; Wed, 11 Dec 2024 05:37:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C2FA26B0118 for ; Wed, 11 Dec 2024 05:37:44 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7C195160881 for ; Wed, 11 Dec 2024 10:37:44 +0000 (UTC) X-FDA: 82882326198.19.15A4493 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf25.hostedemail.com (Postfix) with ESMTP id 0BAA8A0002 for ; Wed, 11 Dec 2024 10:37:25 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CEY6PZvN; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3dWtZZwkKCPIUfcWYlsbfaiiafY.Wigfchor-ggepUWe.ila@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3dWtZZwkKCPIUfcWYlsbfaiiafY.Wigfchor-ggepUWe.ila@flex--aliceryhl.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733913441; a=rsa-sha256; cv=none; b=fVufVDPS5jEi6ixDQTrkxt4B7VsSG+6qONG3uOZwTa+pjbdmvtsS2267pbCNrOVNxMNnDm eFAjFG3l9FWUj36hOr8LUqnnu1kEWLFXucnm1WGu6qrTla7OwEWtPi2R7BcW7+UVveimw5 jtQOZ1FF3FmW1M/Z4yF6c3xjqHOMclI= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CEY6PZvN; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3dWtZZwkKCPIUfcWYlsbfaiiafY.Wigfchor-ggepUWe.ila@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3dWtZZwkKCPIUfcWYlsbfaiiafY.Wigfchor-ggepUWe.ila@flex--aliceryhl.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733913441; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jtfdo1hWu/gkdW8i64KIMP+Laz0H38tgBK8KPM9++n4=; b=sCYsunuUnBcxE55P5b422KC9Y/PfBbKb/do5VM/NNAiEV40UGCbUZgVu+qAbvp7xoXxl8a 6iPg1koxBSvCLMu1wIaNW7jv0GTcEgqSu6vCO5N5KCxhCAjLJUu23lOZcWlu2DYwRZZ4mz nReXY1uKU2YUCyCbG1ia7JXesgyDqPk= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4362153dcd6so202075e9.2 for ; Wed, 11 Dec 2024 02:37:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1733913461; x=1734518261; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jtfdo1hWu/gkdW8i64KIMP+Laz0H38tgBK8KPM9++n4=; b=CEY6PZvNipHuFvOgi1Ez3vgMiwXKxmwlip88IwSRhQ24HsWLg0BomempS4kPTnj6ul FGf8B4IOdltZVZY3Db6FFylh46AGFzzoUcAE3nTZ80Qy8prrdSo8nDmDOkaxkXC2M8w2 ZF0znoPmK+INT1ki42aejWsEUc0hkAEZdMiyAzVqEWfZ4f7IKYT8F+5rUpKQTnIdkYQ7 3kKU4wH2+71RWeYlDKpyF+U5CQjmYWXQtuP8W/xO1CddV3CfnYh0xzCpEMUlPPri1vPN 9ty31rr9LLtYzrhDuRgcXILnNJb90z3XcfN7ZB6o9uLnxeQDsrbel/0rKfk3kkz4DWv8 J2lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733913461; x=1734518261; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jtfdo1hWu/gkdW8i64KIMP+Laz0H38tgBK8KPM9++n4=; b=ig2SI2umLMw52+AYV9Kq32iYsDZleM9rk9TLaMZldTcamgIa2kTxhXb60UuGseGgvS MRCpJC9OKaZoy0juwWQ8ox/C96uPL5GFMtYnBomDiZPUEN6MnNUGhcLLOzl15NzpvSsV wyY864MbUB0K23Z5gzbv1LdljeXvQqHLIFA7AMOGthvQuXwIkmbaFGVDe/0Z1jpfh3wW HxP4i2fHUuUG2qhlm4X/9GUvhakSP9WmmcD3pn22J7IybVFpU5VEylhHfADWaFvG8jn+ foNErrHcuf32TkC4UMNDMUjJf5oA9qDETK+RG0aJddU2NZBbiXy9bCGTi8cymirYYjHB CBNw== X-Forwarded-Encrypted: i=1; AJvYcCU4eI/gN9+p9171Gek3aEyw6tPQqJTsPE9oaQQlUJSIwYNDFpFJRueoXESEhWpWXBrhQxoJ01SPIQ==@kvack.org X-Gm-Message-State: AOJu0Yw1MqCLyEPaDrhvqsxkOxntEgQoDJKbrJX7KMu4f0mTn/uYJRC2 bmkBw/xnpfAEW1KfgAeMVC0IaEsZURInR/CkpaO2cSZwVIngKsqfjSFL63dupHgs15nAdGhYL11 ivgfA7m/JJ1A3wg== X-Google-Smtp-Source: AGHT+IEPSG2Hqz3P3TcvwAF16etBCufMRdEZ09D0s4Xc1JzGR3A143l7IWc1m3i2YQwgadn0owNcK8U+CXDiey4= X-Received: from wmok4.prod.google.com ([2002:a05:600c:4784:b0:434:a2c3:d51b]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:4023:b0:385:eb17:cd3d with SMTP id ffacd0b85a97d-3864ce49640mr1860462f8f.8.1733913461236; Wed, 11 Dec 2024 02:37:41 -0800 (PST) Date: Wed, 11 Dec 2024 10:37:08 +0000 In-Reply-To: <20241211-vma-v11-0-466640428fc3@google.com> Mime-Version: 1.0 References: <20241211-vma-v11-0-466640428fc3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3946; i=aliceryhl@google.com; h=from:subject:message-id; bh=PSMNvAHImS1yuLQzsvSoD2e3+5nrD/G5jvs5aJHnybQ=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnWWtoiZME2oih+BwGJvgLeV9XdV6aNRW+Q12xj brbx3/LqYWJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ1lraAAKCRAEWL7uWMY5 RhRpEACoHtQCpffN1jELk1C4qWvx35jHIqYWia49W6YKDfgnOKbSh/PaQNsmFLQ5xqVgbm7v/UK aBQwg3EiD7cArhLxL66nlarZSNU4xAyIYRagufeEs62wVM8fiHrN0bLBQZaTeVR45ka6Dr/NpjT JDvwDHSpm37SvlfAmofc9S+xU4wW8bYutsCq3jCcK0SwgR0BZRcT+q5yYgBnR06vTwCt8Eo32tF 4eCgLuB09ENw8ozEaZuy3U/nIsKDw/Fs1cyweaV3JFTRhZu08ZCQFCMZIItw+z7S9L6SHZ7QOqo 2aszTbLLzdzJChsBBjyQ7S46UghGprkQh+8+iJ/+OOOK9T4A71euURV7yzgU5+I7pOVq7uuEUvc xMxwyyWyeR/cf/dst1SzUffc7jT2e/Ap4W1LCtdaFkEgNXR6/6vTsJ9ffKl5VQYaw5CW+v+L/NY BXwxO+E6cGtAf9afhviyTbXnazRnar/c2501rI4RBu/3zB+0LuIiuSBH4ZyHIkanDFBbekdJzfe j2fNRFns1Rv36w8SKGtQokraNhC7EX9sVaWLGkn778X7n00EcmfFH5uMSpERmffICVt/3aJa2rW nW09v5vj355pS053y8y4wrt1fWYw/tuQYoBi21rp2L+yWlr+nKcIRcoshluSVLDbH40G2H3jp6u boRup1nXBHzGN5w== X-Mailer: b4 0.13.0 Message-ID: <20241211-vma-v11-4-466640428fc3@google.com> Subject: [PATCH v11 4/8] mm: rust: add lock_vma_under_rcu From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Stat-Signature: zyeh4ta56khr9bp5r7zitgfdi4jy1seu X-Rspam-User: X-Rspamd-Queue-Id: 0BAA8A0002 X-Rspamd-Server: rspam08 X-HE-Tag: 1733913445-958195 X-HE-Meta: U2FsdGVkX1+PvWUNhlmpTOHyh0MpffJThLEn8Rcj0eYINvhUhzY4aHOd/p5RW0uqfJ/pE6Ttj8D5M09gdeULBMJDbIVXOdIMVAFCEsPFTzxg5rj5t0fXakcLlH38sFVrEquMrmjuH66VB9Ng/ZeIxKnOvaV3EEqmQvTfnkDwgTuImysIHiFBlfyvk3UfLeZuCEhxx+gAWf9iHe3mFtr/BvH1FH2LLHZS+cI717h41e7VnwFw+B/g+hHziSvv1arqo9zCCP3amMcjvJgC6to8arPNtPrHjPIpgLIRyv8wo7Vc95r9N00N4eqEJKRPr7cIjeqsvkvZxUl2yd+p1xvI/k0O0ubg3JJZegt3VsrrEo7mX/jMm7lUx42lfACPvPrYEZzAVSagSJhHSsOpflmCBf5+QuKLzuzBErZqq8E8RBmdNwW3rDACSlPdhTXPd8f0cU5YDIIVLX8lF0PndE+53iHTVdGTSx0bkxBfZBBplszumtHcXBEzfvu+Oc2XX5kiY3Fm9viyZ9iNVGv3rQw+bM2/yYR2fAVmtO1boMYquyKe/HglM795naVfyOBPL9mlw8yfj6re6JpJ1neCcOwPZ0ZGeSijtd2WxgjnZAuwjGnrPEcYwSXfdUZfxE8tEW1ZBGiC1tVsOlI1ywLH1ZM+r1yYxx4uku/fvxxdIGUSKM0WufJPbyiH3A0t4wvJyeglAFy/JS/sP6VJenphuFfLLDgLj6heXzACcg6jRa1y8vLPzeEgCNmf7Xe/Xg4xV+w8aFkwDuQpwylly3bUSmQOGr/6AHpVCDukLlkblq06Zyka6DXDWfv8A5fhMICpvnYFqjplmQJ5kp1vnA5SYxQ1n89ria7aPv2Q8VqOstWYxdanGMjZqB1I4JQWhMUai4BvpocHYlM9LkOL5t1Cb3W6uG02+SGXp9Op6alD1IZkqp1xRT/x9/ZNl6tb8cuyik8WXQPcjHP7591kGog7T6p 51JW+Ed6 QnEPdtrjIcC3TlPFN9sT0GL8VMfR/dbwKHzdRdnNmDvcS50woz2jHRf0xMcfiW/cd17DoU+Y/oflwQ091W1VCL3RhiQ9YIVFOuwgukfQoMyoJWvkmsQhLtodN5kit0k3PeEHSYpcAw0cSWZ3Mvf8YJDATxx37qYFC1DQUm3M+Era4FcHAgeBcap/P2cCZBipEFoXb4XbBV62b6qwNsDuHAjtFnWV3oXFvesoNeNXW2cxhfEqSwzjmorAoedXN82MveqGyvK4vQIuclK2MKp2vAa/DDraT8azQ0FIjQARw6cfig3Q2yOw+vnmQP0kQSRaK2F24reAIHDg2iQwUIy66KgrEAoaX868K2hUOO8flVkXGMfUSdKid+2HekZr+tD2SkOExjbQkfgSjWK+adassufJWVSlQ04OKjaGyu0lelQBZ0afGw9HqEukDNrQldLMbjFQtByRXnMTzFptmfZ7XERq0Cc/M/ocv0LxM19jnP/fn0PgPKEoQJbnzY8ydAIFYkAGD/cDH4Fp+q1SA6Me5oInucjtRKI56ij5gQVU0Gd1qD5NwrPPthcBJvjya2YS1i9hiHTBlczJEVTksZ52AZuI0otpywHFenCJrhbnAp7gd738zgqB8TRf7XCX9UBrINijNmKyikmN4zhxDSckrfXppKhLlYUuZiGQkdYlxm4JZAq+NFsW89zDTQOmzRz4pmhh2Tyq6BS48QwA= X-Bogosity: Unsure, tests=bogofilter, spamicity=0.499337, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the binder driver always uses the mmap lock to make changes to its vma. Because the mmap lock is global to the process, this can involve significant contention. However, the kernel has a feature called per-vma locks, which can significantly reduce contention. For example, you can take a vma lock in parallel with an mmap write lock. This is important because contention on the mmap lock has been a long-term recurring challenge for the Binder driver. This patch introduces support for using `lock_vma_under_rcu` from Rust. The Rust Binder driver will be able to use this to reduce contention on the mmap lock. Acked-by: Lorenzo Stoakes (for mm bits) Reviewed-by: Jann Horn Signed-off-by: Alice Ryhl --- rust/helpers/mm.c | 5 +++++ rust/kernel/mm.rs | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/rust/helpers/mm.c b/rust/helpers/mm.c index 7b72eb065a3e..81b510c96fd2 100644 --- a/rust/helpers/mm.c +++ b/rust/helpers/mm.c @@ -43,3 +43,8 @@ struct vm_area_struct *rust_helper_vma_lookup(struct mm_struct *mm, { return vma_lookup(mm, addr); } + +void rust_helper_vma_end_read(struct vm_area_struct *vma) +{ + vma_end_read(vma); +} diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs index ace8e7d57afe..425b73a9dfe6 100644 --- a/rust/kernel/mm.rs +++ b/rust/kernel/mm.rs @@ -13,6 +13,7 @@ use core::{ops::Deref, ptr::NonNull}; pub mod virt; +use virt::VmAreaRef; /// A wrapper for the kernel's `struct mm_struct`. /// @@ -170,6 +171,32 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser { unsafe { &*ptr.cast() } } + /// Attempt to access a vma using the vma read lock. + /// + /// This is an optimistic trylock operation, so it may fail if there is contention. In that + /// case, you should fall back to taking the mmap read lock. + /// + /// When per-vma locks are disabled, this always returns `None`. + #[inline] + pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option> { + #[cfg(CONFIG_PER_VMA_LOCK)] + { + // SAFETY: Calling `bindings::lock_vma_under_rcu` is always okay given an mm where + // `mm_users` is non-zero. + let vma = unsafe { bindings::lock_vma_under_rcu(self.as_raw(), vma_addr as _) }; + if !vma.is_null() { + return Some(VmaReadGuard { + // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a + // valid vma. The vma is stable for as long as the vma read lock is held. + vma: unsafe { VmAreaRef::from_raw(vma) }, + _nts: NotThreadSafe, + }); + } + } + + None + } + /// Lock the mmap read lock. #[inline] pub fn mmap_read_lock(&self) -> MmapReadGuard<'_> { @@ -238,3 +265,32 @@ fn drop(&mut self) { unsafe { bindings::mmap_read_unlock(self.mm.as_raw()) }; } } + +/// A guard for the vma read lock. +/// +/// # Invariants +/// +/// This `VmaReadGuard` guard owns the vma read lock. +pub struct VmaReadGuard<'a> { + vma: &'a VmAreaRef, + // `vma_end_read` must be called on the same thread as where the lock was taken + _nts: NotThreadSafe, +} + +// Make all `VmAreaRef` methods available on `VmaReadGuard`. +impl Deref for VmaReadGuard<'_> { + type Target = VmAreaRef; + + #[inline] + fn deref(&self) -> &VmAreaRef { + self.vma + } +} + +impl Drop for VmaReadGuard<'_> { + #[inline] + fn drop(&mut self) { + // SAFETY: We hold the read lock by the type invariants. + unsafe { bindings::vma_end_read(self.vma.as_ptr()) }; + } +} From patchwork Wed Dec 11 10:37:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13903326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 538EDE7717D for ; Wed, 11 Dec 2024 10:37:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 594388D0016; Wed, 11 Dec 2024 05:37:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F4FE6B011A; Wed, 11 Dec 2024 05:37:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AC448D0016; Wed, 11 Dec 2024 05:37:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E96546B0119 for ; Wed, 11 Dec 2024 05:37:46 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 92BF980853 for ; Wed, 11 Dec 2024 10:37:46 +0000 (UTC) X-FDA: 82882326282.19.63BE6D0 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf29.hostedemail.com (Postfix) with ESMTP id 7E40612000B for ; Wed, 11 Dec 2024 10:37:12 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=3XTPYON2; spf=pass (imf29.hostedemail.com: domain of 3d2tZZwkKCPQWheYanudhckkcha.Ykihejqt-iigrWYg.knc@flex--aliceryhl.bounces.google.com designates 209.85.221.74 as permitted sender) smtp.mailfrom=3d2tZZwkKCPQWheYanudhckkcha.Ykihejqt-iigrWYg.knc@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733913449; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N4tIdMdsO0J6lrUvneIwXSrF/KOin5KwKJIBsiYufGA=; b=t0OTszYjW/nu9Ir+VNqCMchhdgDC7Q5okdGYBKAh9V3D+ie+yhG3TNefGdMX0GrJ/yy+6e 68GQyO+dZ91+ch3zFAO7U6m2aC+aKTz/K86crXPjdlcwplAmbKr4DsQaX1uMPuFOLZdzOB ptU5Gfs9uvRlHTaPLuSAQD/YG7GifkE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733913449; a=rsa-sha256; cv=none; b=rnE79EGNAyiz3yHMokQRj21lVT/m7QdEt0s36ZJSZhdjH788hQVGxQlGSYg5MbVrkF6lek Fes+QVt+MYlZgyUH2SX7M9M7ZKMNIie8T2/uyEkTlntckFwaC2ivNkt+/0E/yErthXj0BV YQpB9Z/o+bU1ptCXHwoGwFkqPnAWmIc= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=3XTPYON2; spf=pass (imf29.hostedemail.com: domain of 3d2tZZwkKCPQWheYanudhckkcha.Ykihejqt-iigrWYg.knc@flex--aliceryhl.bounces.google.com designates 209.85.221.74 as permitted sender) smtp.mailfrom=3d2tZZwkKCPQWheYanudhckkcha.Ykihejqt-iigrWYg.knc@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-3862e986d17so1711730f8f.3 for ; Wed, 11 Dec 2024 02:37:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1733913463; x=1734518263; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=N4tIdMdsO0J6lrUvneIwXSrF/KOin5KwKJIBsiYufGA=; b=3XTPYON28vQjz0IwzIAaJo+FJqyIb2vilBybcJ4OPNasyWzwHYIW7BsZDq1AT9sf4/ CiSdS7SDQhVMC5BuYXwujmbtNeRB7AOYcLtLXZGAWZdC6pQ2K6lgth0Wv5NOd/UKXOJn +TZ5UP1X0uidZAAb7y31urI2HjuaLJe9xEFWG97+0GV1hi/DMTOUBmM+0ACr0SGGLDL4 gNwl1OOXDuYxwmmfSP6nBv2ZLoIE9eSMLY6JQ6kpyK9L7ulXIHtfq70oVd5rgdj6JxTu 5Wyrf+K0URuR9rJEkHYbw4zEMHum+XA24S7XOoDt3Ers1hJJ/UO1SRmQB0DWpWP9SNRw G75Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733913463; x=1734518263; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=N4tIdMdsO0J6lrUvneIwXSrF/KOin5KwKJIBsiYufGA=; b=CHR6VoZ6ihXtv7akXpSuqRl48stXROnhe3DcXhEPGXd873ToH5fvSqVoDJ58VmVYMG sPN5tH+gZq4NDjp5Szdyvt3q6AAOJcGkQVPrkmL+LX9N5HjCd7NTpzBL4L1OLmeOK+4y E/UwWItyJ6ksRK+cRG5fk/KMW4JecnKdb0avdIRHlkpK9KviKfQq0y0dy741pW1wWDnJ I2HAP5nDiSgDgSwUw1M4GuocNbpE7ut/pZtwLdFTjOQ09wa17ZS8Jo5sMFRj57uZT/Hd GiRpMRB4/fie5FfmoItBXuh6Ggy7eFZNEnwxktsWdVmJggoBRFMpmVdpqlO1YKsEsee1 wLAg== X-Forwarded-Encrypted: i=1; AJvYcCXmFmSg3NTxN4ce2LQ+jqROiD/mAg5e34GwTu9FpuCPSCcyCYH46iD3qEeXXxCpnBBPmhRyD3PEsQ==@kvack.org X-Gm-Message-State: AOJu0YzyMGXeBpqoUzDPQk1Rl7mT1LN+9REP1q/DXuIBIoczR3qFE+jG bKBIUU/lui9ABnbcR0q7T8LVVkWXEvu5Gmq5drAWgE6f8bVDIQte1Wu6nPtE8QrHaRZ4HhDmP3x xgCgw/87XuP7vkQ== X-Google-Smtp-Source: AGHT+IGvJYqc2Db+rertFy18yncLZyz9Adh52dHN4rQJSEpkQ1PGruhxbsByd/HWJepWTHQaE7E4uK9LRFm4JnY= X-Received: from wmpr10.prod.google.com ([2002:a05:600c:320a:b0:435:dde5:2c3b]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:2d84:b0:385:f979:7664 with SMTP id ffacd0b85a97d-3864ced3035mr1190487f8f.58.1733913463310; Wed, 11 Dec 2024 02:37:43 -0800 (PST) Date: Wed, 11 Dec 2024 10:37:09 +0000 In-Reply-To: <20241211-vma-v11-0-466640428fc3@google.com> Mime-Version: 1.0 References: <20241211-vma-v11-0-466640428fc3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3090; i=aliceryhl@google.com; h=from:subject:message-id; bh=o52ixmPVakbGqHXkdZGKle8ANx4alFbrYcfoTBoaBeY=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnWWtoRxaCSkuHkzNXUxE2qqNVnv4EQP+W4uD5c UetmRby79SJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ1lraAAKCRAEWL7uWMY5 RmGCD/9Xgd51fs3bqaJoNH1j26BxFH1sX0Yp3b23v/LQVFSNhncGa6OGK550wg7ov0OeRIedPKw vfoO7FS+v9j13zOMi4A9nogfpPp0lYkqOTWrjQl4lIX68/kck8B5nybxGBMNAvS2CWxD8J0Oomq 6SQ+cb/xgGWhD7xV2WvIifQANQyuin8/tmalAHCl1YX+yIhv+kuE9c0KrvQTAZoWjdTvymoHJIf 1HJepqC3TOgcM0g6dYzn1eNvyFJ9ohJMe9U/AuPZyQf7j0uZPdI87WRmSPQo4Nz3eWY1OLsDC3O UXjBgTWPCBw2u6qNS1ZBOlc/qK8zjohbdQlEjtXZmBJySD+w3BE4xtEeY3yFUM4dz74jwcOQlWK WDLxpWyPhDtHAwUZApsiExI7rM4qZ5YDprePBLCPvtvFW0iPN05pOnPqHip3EZB571ZotEyrQqH M6TPjtBZ0so7nD5X7bMEUx1AWeXuWpfqU9K+OuvbpcPa1bSLiKddGL3/aXfKcp89ZoPPHKRWgE4 qQKlEHkqh+G265jkYzw0ENNNu9TBPO/eqNsH9Vz4XpCRGS3WC0l5ZbXStkimQTxSBRvxIimx72u 12CtVhWEgk2lytcOj92QGdVSVd3m1qTGXFcJ2PHYgnugqt12OMZ/3fsD9gyqGz17LTTFnjZ+Rls mQogTRrM6ybWoCA== X-Mailer: b4 0.13.0 Message-ID: <20241211-vma-v11-5-466640428fc3@google.com> Subject: [PATCH v11 5/8] mm: rust: add mmput_async support From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 7E40612000B X-Stat-Signature: wgeki17dkhy9bpixyey6eaqm6ma9zxs8 X-Rspam-User: X-HE-Tag: 1733913432-604707 X-HE-Meta: U2FsdGVkX1/assSDdVWbK0d3BgzelEtA/y3ipS6TBodRysRXiR/wGMiUL4V8tI8QbwsFNuP+JWDjuuK2/EmE/HHdmdxB4fICtNtosfGvoyTzJqjJvSKW+GplGONsUcBDUhIMVleMIgogC6STbmrJVVPtnv7DEYthZc22HD7PX1NVHJF+4bazwelBDDBwUd1d0mxYtmN8XNgW0W5+ktCznMmE25rk9pXDTQi2dfHWJ8xtypaJ/DMoeeNHn9gqAAVzcKCH+UzTjDtGt4DpdQacuDlsoW+XAlQyqghE83mf7WT1hMzp3DF/MrUNSggQ6lsm3sWFukMg3YWo+3oDlOFZ2Qmcgp2Ijrv8gpedu+FNScGMgD4nfNsdiopSxtpgrsQClxCkZwD2oU9MjTWv1LpzrYOfhrernXA/yaknB4VN6UrIBRuS0B1H6hv6Z898Wan5MVeGtndB7IP+foeGjKOdj460sjNOWC1UA3QmWP8Jhu2OnT7QfXynqdD4jiKk0IzC7kXJPPv3NzZdFCy0f4m7c1ZUISo3lXiFUxG8VRjh/CHbVsO6CbcBSfOprjQAY7lsaRl5T0wEG+G8U/I7t9w9Bfdvr5CWidZKuuQZ0Pd5aRGR0+UZGhl9cAD69w4+osSS0qAYjA1ueus/tsd153RwlsbhC14OiVG5IduiHc3yOIRsX8Bguvl8KC5djdJvdWFrQT0VC1tqEiRchVyeSqjjp6vSkFJTKh4R1UXAm3EF8Ga/uAsXtDv6OQOZiUTR50GnO/TxZ3WM7KRqi+ZMqI64+LYeqt5aXWYssuSjism+327aMrpfCbu+FF4xCIb0xGMT3Ut3XS4u1Tt5g7wPT5cZcxfXweG9kuxelWpp5tb0qlDYuIpta5qQi+gE3BCIiWiMOjmcRoyeXyVV2bsi1hYH9z0csvoFwRwikzt4Tyj2nGnXA2yatfUPXXZBtMZ/MTL/m9TCRFWUiM6RcgXO9qQ VeelwkMT 41olKgAcROCJp8hYwE2hV0uqhQeH4ecE2utaP1m3bnmHqhGmmMJitApBj8g000IplWeOqnAn8yWnwRKB1ykxFciqEymCrkMCcQYQ4Vm+o8QK+UhrUYS99341/7z6J3pWhvDDfIZr2EWTCGtNimHbL6a299q2C6ccMIhb0mjrtwqw4VYwktzGvM7issyb1oKHC9usKCGgWgVvhZXBHtqFgTJvVjZIbe8n3IZ7pWssZ/CpIzYMsjKsYkwXBJixcFmUPU92a1uDMGA+uMCz79MWPSscwmoaeiTwDlUL9yAf14V+BciydgYVmpuJhQvsi26S5kIFOlHl0ng1RS3CNfGAjHR2cLN481HOkENAC894wjV1JBrG7g5ndM3QTHtyNntrYf3naUMQk3wHrK04e+jZt9mSKtFu/oWVVSE7SImce4rpadOb1W/QzIXGTU9flN7izgx3/dz9u5OUCDL5oM1wbxM3As+yn74DW+DUUUqcd0KhQPeHct7CDBIQSMvI+9N0YmXcEIJ+6DhH69a69EssRu83qwpCX7c3lhJCKtWD1n0Z5HK6g8yPVR/QMwQn9SlfyvGF2IBMl27KyhDtmjaWB/foenE4uD7TWJkXppea8j5MyKwRRboKNFrFkLk5QU24OZC5XRdxI9FjTeL0Kov+2YB2WdmcHZMRt4D7mI7gheGHCJqFVmazeEJVDQo5euUJbXedv84RBaWwnrSdtAaPl9vG1ewY/jkGusM1+TkoZ08ZBgdwo1w6fFfM6NnUG+l/fpVyONUvtJhNyyyKSr2SJkvJNtIo/Gz1ZVCWN X-Bogosity: Unsure, tests=bogofilter, spamicity=0.499322, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Adds an MmWithUserAsync type that uses mmput_async when dropped but is otherwise identical to MmWithUser. This has to be done using a separate type because the thing we are changing is the destructor. Rust Binder needs this to avoid a certain deadlock. See commit 9a9ab0d96362 ("binder: fix race between mmput() and do_exit()") for details. It's also needed in the shrinker to avoid cleaning up the mm in the shrinker's context. Acked-by: Lorenzo Stoakes (for mm bits) Signed-off-by: Alice Ryhl --- rust/kernel/mm.rs | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs index 425b73a9dfe6..50f4861ae4b9 100644 --- a/rust/kernel/mm.rs +++ b/rust/kernel/mm.rs @@ -98,6 +98,48 @@ fn deref(&self) -> &Mm { } } +/// A wrapper for the kernel's `struct mm_struct`. +/// +/// This type is identical to `MmWithUser` except that it uses `mmput_async` when dropping a +/// refcount. This means that the destructor of `ARef` is safe to call in atomic +/// context. +/// +/// # Invariants +/// +/// Values of this type are always refcounted using `mmget`. The value of `mm_users` is non-zero. +#[repr(transparent)] +pub struct MmWithUserAsync { + mm: MmWithUser, +} + +// SAFETY: It is safe to call `mmput_async` on another thread than where `mmget` was called. +unsafe impl Send for MmWithUserAsync {} +// SAFETY: All methods on `MmWithUserAsync` can be called in parallel from several threads. +unsafe impl Sync for MmWithUserAsync {} + +// SAFETY: By the type invariants, this type is always refcounted. +unsafe impl AlwaysRefCounted for MmWithUserAsync { + fn inc_ref(&self) { + // SAFETY: The pointer is valid since self is a reference. + unsafe { bindings::mmget(self.as_raw()) }; + } + + unsafe fn dec_ref(obj: NonNull) { + // SAFETY: The caller is giving up their refcount. + unsafe { bindings::mmput_async(obj.cast().as_ptr()) }; + } +} + +// Make all `MmWithUser` methods available on `MmWithUserAsync`. +impl Deref for MmWithUserAsync { + type Target = MmWithUser; + + #[inline] + fn deref(&self) -> &MmWithUser { + &self.mm + } +} + // These methods are safe to call even if `mm_users` is zero. impl Mm { /// Call `mmgrab` on `current.mm`. @@ -171,6 +213,13 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser { unsafe { &*ptr.cast() } } + /// Use `mmput_async` when dropping this refcount. + #[inline] + pub fn into_mmput_async(me: ARef) -> ARef { + // SAFETY: The layouts and invariants are compatible. + unsafe { ARef::from_raw(ARef::into_raw(me).cast()) } + } + /// Attempt to access a vma using the vma read lock. /// /// This is an optimistic trylock operation, so it may fail if there is contention. In that From patchwork Wed Dec 11 10:37:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13903327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBF25E77182 for ; Wed, 11 Dec 2024 10:37:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56C0C6B011A; Wed, 11 Dec 2024 05:37:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4800A8D001B; Wed, 11 Dec 2024 05:37:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20E778D0017; Wed, 11 Dec 2024 05:37:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EABD96B011A for ; Wed, 11 Dec 2024 05:37:48 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A34E0A0C87 for ; Wed, 11 Dec 2024 10:37:48 +0000 (UTC) X-FDA: 82882326744.12.6BD787D Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf13.hostedemail.com (Postfix) with ESMTP id DD06E20019 for ; Wed, 11 Dec 2024 10:37:23 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="MkhoqML/"; spf=pass (imf13.hostedemail.com: domain of 3eWtZZwkKCPYYjgacpwfjemmejc.amkjglsv-kkitYai.mpe@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3eWtZZwkKCPYYjgacpwfjemmejc.amkjglsv-kkitYai.mpe@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733913456; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H9J7APDOlzo7bms4ez8DHMnNHWyqhL35jyR1kYpr9QI=; b=FyiFBVlnujKeILRgWg21avG18e8+kmb1PCc0veW/XXgLPyo7uLWa3KuLBXmOszr4PIk4vm uyDncDbz/1GFjaqkRThm6wpu4eDq0B+z/7NmxTWySsBJ2VDUKSmwOybpuL+eg8iCWyFqth c/L6wSKq1H9GYzytxGR5aPGkUMqQOFU= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="MkhoqML/"; spf=pass (imf13.hostedemail.com: domain of 3eWtZZwkKCPYYjgacpwfjemmejc.amkjglsv-kkitYai.mpe@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3eWtZZwkKCPYYjgacpwfjemmejc.amkjglsv-kkitYai.mpe@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733913456; a=rsa-sha256; cv=none; b=H2SuiAff4YDGt+qs0ab4UcvCCeem+XX4w5lWCBHjPAx/Yad3wLVwkvai3cM7QkuGMQmlXn 1enW0Vipc+x8zGl0a5UUV1s/uisEdoVJthh6cqJzRlXe5NJINKynDYUwCiJwsy/iL7M4ZA 68/npIh9PM9lQV7A7aNOCSi9vEhFiiA= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-434e9e1b4d7so2245595e9.3 for ; Wed, 11 Dec 2024 02:37:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1733913465; x=1734518265; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=H9J7APDOlzo7bms4ez8DHMnNHWyqhL35jyR1kYpr9QI=; b=MkhoqML/cY+sZrM8fgtXKopU97S02fMMRo/iNq0Sk2Q5MVE+agtGL+I5np6dKR+wpx AA6GTyo3Mg2SF9sSncyAM6K1Lrb4L7kkYesMGVxwqe1k4V10rSFLRq841sXu9hEXXHUn NQOnY3UTfyZ21LxbrDXGz24RoWJTJ/LPSsQKS5jTTEGtKwDQxV2R1xrgJ4Em5F9l1ex/ 5ftIAjLQdlKwpBfqH9HypKUvbHWleBswAic2nnDrUEhSv1FsUtvcWhjNxz6yf94IoRpj 3ybU1TTJ1OJFQzus5NlaDhCi9pvsd+W+Uh1N8qlE2EelMDsz9MdQrHj0G2Hz2frKZSj5 04Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733913465; x=1734518265; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=H9J7APDOlzo7bms4ez8DHMnNHWyqhL35jyR1kYpr9QI=; b=g9noUT7Df/An9HeFOiNs+Q9IPK03Lw5J50dfOuZjajlpXfe2Ak/vIMIvFBdouYGYDJ Ab95M8S+75IpFa/MkHOpAbH/ki5rFY7F+/VXeF8zpkAeHw1RwRjj7m2drFzUm6GkIawA qHLoBFG+YdGRlu0Q8FtBgrRVbCtpt+KExuNbo63PRXcjR8v1klRauB7mhLJhfn1yWvhw RiTGWND41fmmh9hWwJeIxil7/5gPfhPfBZ0XAIv9/NdMPCtPMfkDQB3tg0/Q7g2PJD61 AbblW8MvCuiR4Zjc2R382IKHDCeaaBiRcwYkXxeNiEXuFpDyaNTvjFlO1Snx7VRuczp9 IBbg== X-Forwarded-Encrypted: i=1; AJvYcCUQToXtR69ScnlHGbGPDV8VM6lVl7Gs9NTJtZVKVlN5pPUmVwfMvDLU59zw4AXH2l/+jgsDCmYG/g==@kvack.org X-Gm-Message-State: AOJu0YxyDqLQTKt9IMEv0NQXFtGqQguP/oS8tOq5gD7XPTmnicGSic7i z0ehr0CR5195m1djHtEmnHlOyG0qONrl0sS+cJLTwXaMvMVu3auQkpN0JLKhKuql7qgXiYHlX/x kygkvcii2ix+Q6g== X-Google-Smtp-Source: AGHT+IEHXGYW3IxnBDufE3uW2/yO/pfGqLpiPsVXzHTtpypegQgEznYG6piRm0MwSTPxkbuG6wjEylcUkB+RemI= X-Received: from wmlu15.prod.google.com ([2002:a05:600c:210f:b0:434:f0d4:cbaf]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b94:b0:434:f0df:9f6 with SMTP id 5b1f17b1804b1-4361c346814mr18704535e9.3.1733913465503; Wed, 11 Dec 2024 02:37:45 -0800 (PST) Date: Wed, 11 Dec 2024 10:37:10 +0000 In-Reply-To: <20241211-vma-v11-0-466640428fc3@google.com> Mime-Version: 1.0 References: <20241211-vma-v11-0-466640428fc3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=8235; i=aliceryhl@google.com; h=from:subject:message-id; bh=seaPzXRr+ND8q/GQYnUjLUdeX+eDlOtmUd9qc8R35Uw=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnWWtph/aiDjcWSnagWFqg3c8CayEabkssTHofo IIZe48WPGWJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ1lraQAKCRAEWL7uWMY5 RhAmD/48pALXpz/NRk7/Fwz/9MCuo0u66DfT05kQmm6LINIciACvTECWnSEQ0tqAKRNpsp/3Fex M4Jpu48j5eDcPWEe/l1C35rsiOmHX4W4JVrnnbQZp+/CjgyqyKCcGdDeeAsbF9Fup1VX8msId3D /03PP6NhmDZYipxZdlby2oUlmR34VgBi5vhQUaLndkfR7m4sEFY9dttHnlypa8gl+V0gKxmY79U 9JhN1ZFWyvUIYXm8lClnAsbDLy08fgFrDbEUeFzSMhVR1kTYVdrrhv5dekv/RSP83A632PXlet0 PYeKGYna7WKapvDmtx8K7d0S/9kzlS2ohG7IDbWe5ZGerwfda1ceER8EtUwLQcpx6Lodt9LYOWU 0WrCa6ARlGG4I/YVU2U6dzSxkYI3Kozqnl6EzH58UkK9BXMGdSGAL+HX19s8k94rmcbVe5P7PHi W+05YwL+uuVwdmWJJ3JWrwvQTW1iF2nFdQXtgZ2iGluLiOIQYSiG9J8og/1HIcZ0hzT7VTzDTL7 8A3/CIHP3KV5PwM6wFMQXsbOnuL1jQF2ZQmrxOfLskPlDXn3meCorLLNaO9pkEG5rjCOyNHwD6p 7r6CLbjSBFYOiYh8+hJVc36KbYuQafNb+ARWM1sWBLGLoe0W8HWuM103Eetus0orFCyJ79zlJFI VIEfxrJdZ6KH4lQ== X-Mailer: b4 0.13.0 Message-ID: <20241211-vma-v11-6-466640428fc3@google.com> Subject: [PATCH v11 6/8] mm: rust: add VmAreaNew for f_ops->mmap() From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Queue-Id: DD06E20019 X-Stat-Signature: tr1ao9sz9gibzt1zpmqwgjcot8m5fi1p X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1733913443-31838 X-HE-Meta: U2FsdGVkX19l65nYFB934adfKhsmAkmTTdcQh1I2Jt+IMMHIZSSIw23Val44OlmhAz6dDVijKTFv5lVO5CTMi/AlwYHSR2Gbzui3TN75ecXFWPYfre1ZwCVKgyCrpzeCyfp8lCw8oncNK4yQBCAWJGATG0sa4WVxs9OopTFKLnqrQOuvIzXnZyPFGAgPTXarRzar59zpGrEQWwd0fA3O3GIwElmuHA9C/drecW4QCafglu6ra71Gh9J4uvICASqOj9GgD4Yv1tEEdUVblW7sBJK9000wggYkQ0XzKlMzmcnOVymDpNGaJwbOCvetBE42thu2qNqrsqXy+RtZIfs6HThY89kGjN9dnQO4lAl6xxWsZwpZ8V02ry2hKT1+UVsXADDqaOsp3FUSUCozalj+tpoKh10TGMHNjekclkgSKuFqPFDjP+yo7yhfI4Y9ox9fpUesvq4SFkYMl5BXjy8yLGuYBCok54Ya6YUTSCwhI/EJJ6iIcZ+2ANEWf8Wj0rf/KzVbD4L3ovJxm0T/A2Uzz5xG1XzHf0NJWuDWxHHpsYRP1LYWYGSjVrBL0g6q/tLTwpxFBmiwhgWiCYyz+UdsZ5AYG4YuxlqNCU3QZXPparEjaiRICVv+pEUKWvhRevaKUqChiQCqw131lEl5eRtkbouzrZPwzErrZD0M6eoHDgTau5eX16gLFxxelUNQzQpgnnnW9GxDIw0jHOCqaLg37xno3BAaE9kL5gd+nruiXcxm6bd1qXOtBuuYf+XDArLmRT/EfOEBXaZ2qK6bSbTYpXirjTakPE3H9VjD3ljb9g+7gMP/xAmWzdZcPAg+iMEtKDTqzSZCnK+ZmG/DGSXqZKssZZCHlBxJI2sBqT10tRS+ee/zh3+n2Z/ASDYmiWsE9Yyio9rOkWZPgJvCnYcwy1fT5IC/Qect4MnjZKsmoYKo7DUHKkYTwNTczpHSu5L0HfQ3mHjsjYbTc0YkQ30 U4Bx0lY4 yg/FidYwNaCaqgQ5LUDaTmSCY8it6OndOZqiCNSbZVMgD4NWJr4rsfWyeT2y+792LE6h5uR++n+Mwcj6vS6DYhOmDQTIIszK/1LAfegxHedjnRE8pZrS34QtULelZ4l/Pk6Daw+nvjupZqfTx5mfX5j2+dsLw5iT8S91n4HiTZ0IVum0WdQuQGR9r7dK47giA4+HFJ4GvS+zyxDM78sd5zRzy78m2PNLTd18S5QPz8sp2OozkmanzhgUajHNt0ZPrSd4TR+esk3WleemRoRa10iIIZf5CK3WoUldULk+VG+wm4buaMpmarYVxSg3nbI2qKLTYGi7EZXhzmIVewEHHHRx4N3CAIxvLKLx3RmZARynWjL70tr4DWqSwl7TLTlz0hXgrKb0/zkGPYQhKVEp4GJ/uBc5yEMINRivesMOl+3ubmlV3Iabg/KTJ3URj0pw7ifRgQTmOD8Rbi9V9OZDUO7El4rFBXcPrTVl5nGrMs9UFA6EDP1preI7P0b7671xy7JUurwiZ34d/e/9H6axSK5BQvSfNpD0iCGaFKqYnDcVemhmFbTr767nrDqiNx2I39XGd8kExawle5MXZ6BtEbonoGrQoelOC8PlbmprmRRz2aBm/JEhOI4CXZCMwthYEXYOcPZBdwjIqtOdXpzv9TPP9kFLNQ9JrTRco8PSIyW59nv4fkfLd9uwrqiMUbeU/bznCvwyrGhvdrgA= X-Bogosity: Unsure, tests=bogofilter, spamicity=0.496955, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This type will be used when setting up a new vma in an f_ops->mmap() hook. Using a separate type from VmAreaRef allows us to have a separate set of operations that you are only able to use during the mmap() hook. For example, the VM_MIXEDMAP flag must not be changed after the initial setup that happens during the f_ops->mmap() hook. To avoid setting invalid flag values, the methods for clearing VM_MAYWRITE and similar involve a check of VM_WRITE, and return an error if VM_WRITE is set. Trying to use `try_clear_maywrite` without checking the return value results in a compilation error because the `Result` type is marked #[must_use]. For now, there's only a method for VM_MIXEDMAP and not VM_PFNMAP. When we add a VM_PFNMAP method, we will need some way to prevent you from setting both VM_MIXEDMAP and VM_PFNMAP on the same vma. Acked-by: Lorenzo Stoakes (for mm bits) Reviewed-by: Jann Horn Signed-off-by: Alice Ryhl --- rust/kernel/mm/virt.rs | 181 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 180 insertions(+), 1 deletion(-) diff --git a/rust/kernel/mm/virt.rs b/rust/kernel/mm/virt.rs index 3a23854e14f4..6d9ba56d4f95 100644 --- a/rust/kernel/mm/virt.rs +++ b/rust/kernel/mm/virt.rs @@ -6,7 +6,7 @@ use crate::{ bindings, - error::{to_result, Result}, + error::{code::EINVAL, to_result, Result}, mm::MmWithUser, page::Page, types::Opaque, @@ -171,6 +171,185 @@ pub fn vm_insert_page(&self, address: usize, page: &Page) -> Result { } } +/// A builder for setting up a vma in an `f_ops->mmap()` hook. +/// +/// # Invariants +/// +/// For the duration of 'a, the referenced vma must be undergoing initialization in an +/// `f_ops->mmap()` hook. +pub struct VmAreaNew { + vma: VmAreaRef, +} + +// Make all `VmAreaRef` methods available on `VmAreaNew`. +impl Deref for VmAreaNew { + type Target = VmAreaRef; + + #[inline] + fn deref(&self) -> &VmAreaRef { + &self.vma + } +} + +impl VmAreaNew { + /// Access a virtual memory area given a raw pointer. + /// + /// # Safety + /// + /// Callers must ensure that `vma` is undergoing initial vma setup for the duration of 'a. + #[inline] + pub unsafe fn from_raw<'a>(vma: *const bindings::vm_area_struct) -> &'a Self { + // SAFETY: The caller ensures that the invariants are satisfied for the duration of 'a. + unsafe { &*vma.cast() } + } + + /// Internal method for updating the vma flags. + /// + /// # Safety + /// + /// This must not be used to set the flags to an invalid value. + #[inline] + unsafe fn update_flags(&self, set: vm_flags_t, unset: vm_flags_t) { + let mut flags = self.flags(); + flags |= set; + flags &= !unset; + + // SAFETY: This is not a data race: the vma is undergoing initial setup, so it's not yet + // shared. Additionally, `VmAreaNew` is `!Sync`, so it cannot be used to write in parallel. + // The caller promises that this does not set the flags to an invalid value. + unsafe { (*self.as_ptr()).__bindgen_anon_2.__vm_flags = flags }; + } + + /// Set the `VM_MIXEDMAP` flag on this vma. + /// + /// This enables the vma to contain both `struct page` and pure PFN pages. Returns a reference + /// that can be used to call `vm_insert_page` on the vma. + #[inline] + pub fn set_mixedmap(&self) -> &VmAreaMixedMap { + // SAFETY: We don't yet provide a way to set VM_PFNMAP, so this cannot put the flags in an + // invalid state. + unsafe { self.update_flags(flags::MIXEDMAP, 0) }; + + // SAFETY: We just set `VM_MIXEDMAP` on the vma. + unsafe { VmAreaMixedMap::from_raw(self.vma.as_ptr()) } + } + + /// Set the `VM_IO` flag on this vma. + /// + /// This is used for memory mapped IO and similar. The flag tells other parts of the kernel to + /// avoid looking at the pages. For memory mapped IO this is useful as accesses to the pages + /// could have side effects. + #[inline] + pub fn set_io(&self) { + // SAFETY: Setting the VM_IO flag is always okay. + unsafe { self.update_flags(flags::IO, 0) }; + } + + /// Set the `VM_DONTEXPAND` flag on this vma. + /// + /// This prevents the vma from being expanded with `mremap()`. + #[inline] + pub fn set_dontexpand(&self) { + // SAFETY: Setting the VM_DONTEXPAND flag is always okay. + unsafe { self.update_flags(flags::DONTEXPAND, 0) }; + } + + /// Set the `VM_DONTCOPY` flag on this vma. + /// + /// This prevents the vma from being copied on fork. This option is only permanent if `VM_IO` + /// is set. + #[inline] + pub fn set_dontcopy(&self) { + // SAFETY: Setting the VM_DONTCOPY flag is always okay. + unsafe { self.update_flags(flags::DONTCOPY, 0) }; + } + + /// Set the `VM_DONTDUMP` flag on this vma. + /// + /// This prevents the vma from being included in core dumps. This option is only permanent if + /// `VM_IO` is set. + #[inline] + pub fn set_dontdump(&self) { + // SAFETY: Setting the VM_DONTDUMP flag is always okay. + unsafe { self.update_flags(flags::DONTDUMP, 0) }; + } + + /// Returns whether `VM_READ` is set. + /// + /// This flag indicates whether userspace is mapping this vma as readable. + #[inline] + pub fn get_read(&self) -> bool { + (self.flags() & flags::READ) != 0 + } + + /// Try to clear the `VM_MAYREAD` flag, failing if `VM_READ` is set. + /// + /// This flag indicates whether userspace is allowed to make this vma readable with + /// `mprotect()`. + /// + /// Note that this operation is irreversible. Once `VM_MAYREAD` has been cleared, it can never + /// be set again. + #[inline] + pub fn try_clear_mayread(&self) -> Result { + if self.get_read() { + return Err(EINVAL); + } + // SAFETY: Clearing `VM_MAYREAD` is okay when `VM_READ` is not set. + unsafe { self.update_flags(0, flags::MAYREAD) }; + Ok(()) + } + + /// Returns whether `VM_WRITE` is set. + /// + /// This flag indicates whether userspace is mapping this vma as writable. + #[inline] + pub fn get_write(&self) -> bool { + (self.flags() & flags::WRITE) != 0 + } + + /// Try to clear the `VM_MAYWRITE` flag, failing if `VM_WRITE` is set. + /// + /// This flag indicates whether userspace is allowed to make this vma writable with + /// `mprotect()`. + /// + /// Note that this operation is irreversible. Once `VM_MAYWRITE` has been cleared, it can never + /// be set again. + #[inline] + pub fn try_clear_maywrite(&self) -> Result { + if self.get_write() { + return Err(EINVAL); + } + // SAFETY: Clearing `VM_MAYWRITE` is okay when `VM_WRITE` is not set. + unsafe { self.update_flags(0, flags::MAYWRITE) }; + Ok(()) + } + + /// Returns whether `VM_EXEC` is set. + /// + /// This flag indicates whether userspace is mapping this vma as executable. + #[inline] + pub fn get_exec(&self) -> bool { + (self.flags() & flags::EXEC) != 0 + } + + /// Try to clear the `VM_MAYEXEC` flag, failing if `VM_EXEC` is set. + /// + /// This flag indicates whether userspace is allowed to make this vma executable with + /// `mprotect()`. + /// + /// Note that this operation is irreversible. Once `VM_MAYEXEC` has been cleared, it can never + /// be set again. + #[inline] + pub fn try_clear_mayexec(&self) -> Result { + if self.get_exec() { + return Err(EINVAL); + } + // SAFETY: Clearing `VM_MAYEXEC` is okay when `VM_EXEC` is not set. + unsafe { self.update_flags(0, flags::MAYEXEC) }; + Ok(()) + } +} + /// The integer type used for vma flags. #[doc(inline)] pub use bindings::vm_flags_t; From patchwork Wed Dec 11 10:37:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13903328 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93A25E77180 for ; Wed, 11 Dec 2024 10:37:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D86E8D0017; Wed, 11 Dec 2024 05:37:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8EE386B011F; Wed, 11 Dec 2024 05:37:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6C9068D0017; Wed, 11 Dec 2024 05:37:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 378326B011D for ; Wed, 11 Dec 2024 05:37:51 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E1B1A425E0 for ; Wed, 11 Dec 2024 10:37:50 +0000 (UTC) X-FDA: 82882325862.20.94FF9B6 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf04.hostedemail.com (Postfix) with ESMTP id 9E67C40007 for ; Wed, 11 Dec 2024 10:37:24 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Pm0ZeKNy; spf=pass (imf04.hostedemail.com: domain of 3e2tZZwkKCPgfqnhjw3mqlttlqj.htrqnsz2-rrp0fhp.twl@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3e2tZZwkKCPgfqnhjw3mqlttlqj.htrqnsz2-rrp0fhp.twl@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733913444; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rTGvDMTG6tS84iSkKnwUvzTY8tCVI3+QbIdPlF0cheo=; b=UKLNQhX8E2SAQrbJSZbKvINawtECNfGjlUzIMyhJNiYJg1vMr4n9Q9GB3nOd31mFYP8grO CjshsyFSIt4WEFgmXUrO9gvMQQ4PdE6eo3dZ6JXpk36tsz+/RJVX+2MpJ0EFi0QGfimfYO z/Pe8KeN2B849TcbIfdkqgJp/y6Y3ss= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Pm0ZeKNy; spf=pass (imf04.hostedemail.com: domain of 3e2tZZwkKCPgfqnhjw3mqlttlqj.htrqnsz2-rrp0fhp.twl@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3e2tZZwkKCPgfqnhjw3mqlttlqj.htrqnsz2-rrp0fhp.twl@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733913444; a=rsa-sha256; cv=none; b=tapCOfesbK5yh2CqQWt1lyOfPmLUQIlWejlEC0VP2+89gps2eEn747v0JaL5bacYORjnLW 2VN6DpOsQqKuV4HKWEKryasihFm33Af2eyUQZJKOr458Jq5eOECTWXyk7qa8RaJBWkIARM 41K2xOwJaZZPzQ4PLDutHY3IuEvFep0= Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-434fe2b605eso16702975e9.2 for ; Wed, 11 Dec 2024 02:37:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1733913467; x=1734518267; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rTGvDMTG6tS84iSkKnwUvzTY8tCVI3+QbIdPlF0cheo=; b=Pm0ZeKNy9RT8QxBCVZwxrTpLTTc0+2SmRY2jKpqBcZUD51jGWs90Znlgcjb2vvVBXW zAeyECZAxlIetwrIQnrYdiXWkVFKhC3t6tZjkiKrcZE3SMzp2fqwRvMidCyhYLqYOYqj VbQ2TW15hkFXYrxLLHjoyQcRe/oXjnglUs1BrAG5aiZbbaAn6DjBXhtX0ayVZ1XL7GhV SMnpoRaJlZehNgVCqJa6dfZpCYdMv67sRD4pXy53Kv9rzZp6wK8MVTpfyy0KiGqz/MI2 rkM4DDODEshaToZTki2/3Q7inw/NXj4N5PQlz08oPHxSrX/+257z0iDAp/XeD/ifkvIN oeGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733913467; x=1734518267; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rTGvDMTG6tS84iSkKnwUvzTY8tCVI3+QbIdPlF0cheo=; b=LqxVt/0hpmEdSXLxXemzoi0LGa40TqNv5lNv91Yvr7bDAJWZJqiujtSbR8ILaC4Rbw Gs76JhkV9g5FAmlX1PJUXY24T+hdbALfUu9R7XYpk1B187AdCWTfMUK+gU9cipr6EC+r i2y6AUZEXXTr7UhDMVagF3mHhmTDb4AzFza5Jtj8vw4vlhTvX76rS2i0pQdRqbUZct/i Xb2dhsrPlb7nTJri4FW+lECwd4HQ4OBuglpmY2SlFIu9qOBdqhjAMVxITCDzBpkJqkpV mv6Q+YCxt/DajvjfCcfq4bCjsp/UBfaS46HT+dUBeoHNyVrnnUZFZM7yV881BmYLLOi8 pzqQ== X-Forwarded-Encrypted: i=1; AJvYcCUtEcElZYsHfjwFoAVZEzD2eJINma98L6E41wZEBwfr2sIZlCtZEFzY6ijl+zaUdIo/mK50zAaalg==@kvack.org X-Gm-Message-State: AOJu0YxbylDU/ew0aADVUjSYX/GCW1vcOxQfcWgO0ENJL4QzdoFBIk8J PQ5gdCSbskhvW6LPtfznl9H487HFMQrXKFUhfWvjqwYtuwOgwjNUyAlgVC+5ERvXjEto1b+tbEs SfL3v1+yGiT/x1A== X-Google-Smtp-Source: AGHT+IFbQovoxVXUDOnrp4PlHMXfzU8u47wUa/LBd5eLin9mhsNX8CNYJjENnzGGaQ/unC/QIRj6aAvvCl/6L5c= X-Received: from wmbf11.prod.google.com ([2002:a05:600c:594b:b0:434:a98d:6a1c]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:83cf:b0:434:edcf:7464 with SMTP id 5b1f17b1804b1-4361c418431mr18973185e9.30.1733913467697; Wed, 11 Dec 2024 02:37:47 -0800 (PST) Date: Wed, 11 Dec 2024 10:37:11 +0000 In-Reply-To: <20241211-vma-v11-0-466640428fc3@google.com> Mime-Version: 1.0 References: <20241211-vma-v11-0-466640428fc3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3113; i=aliceryhl@google.com; h=from:subject:message-id; bh=kzjS+ohxpZzNcxKXIXjmOFsrjiQU+XTutyXr8P3Ah4o=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnWWtphPlZQRQby61uOX0k3ecWUliVN/FZJFCoi bM4aIl7KVeJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ1lraQAKCRAEWL7uWMY5 RpGaD/wMi3M4it7Hk8rAcHr7ywdgfyToJ+g739ILqsktYqKVFQs/m+KIByJs+9mKBODgiyL10K5 PEZYsa0Ox8s8XKZPI69psKntnEhe3GjUUvu7px0f15cmt57bRZODDgo3vI59YfdTW5FZOESP9RD YhSPLC08wMnnoQBxpcm99TqMClCmmsWIlo4bXZfGvuzS2CJUipJ9sptfZzzrYVIKNO23klgaGPQ 8yZF6MBm59u1msMIvMt35PK1aBm/0EVWPzHCjgQnSqZTbQfrF0VdVZ5yjRfJiEzqf7NT36yIkz5 a89RzhZKbKMOqBVSMGJ7WHIpfsblZ/9wBgcrgDYLjQRb0FTgJQemp5aXCPWaXMNLS/rtMH/O0VQ eaYb9srV1MWgtGQOwbKFC8C0jIWhmSOiTdUabE96UYeBrzR4OkXqW1nPI/E6OaG6jx9o34JdWHQ nCIW7jvcdA2V19zfyaTQT+zOkrX6OJ3rfccVMAXlArE2lHiCUpKWFt7L7z1DlYoYj9PmjnUaTZn IIsFDF/0jEngVlAuF+Fw+EHPuOXm2I4uasrQubr3yOClmPU2OeM9hwv1p1GvOzU6QfgS0LZempt ng0nHj9CaNoclmMK2hvLhOPagmeyxUlGOVFA12epr2x5ASIL++FcelN0S6Uqpdlju+Q7hHbXi1C UQa8vdmTLIt2gug== X-Mailer: b4 0.13.0 Message-ID: <20241211-vma-v11-7-466640428fc3@google.com> Subject: [PATCH v11 7/8] rust: miscdevice: add mmap support From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9E67C40007 X-Rspam-User: X-Stat-Signature: 59wnsdxfum3grajrjxg9txe45razcs1n X-HE-Tag: 1733913444-701538 X-HE-Meta: U2FsdGVkX189JlWlr+AN/WQU5cFcL4niuffXAIkACTIVi4KbuAwWIfBwI/oJvh8pStap09JqMZ1eSx3GgENXNEhEcWydAR3nHGNxi57H5m+HnHfKGjHrp7XI/Lh2igVCaGRRQH4JoIswzMP3qUS559t85Uq4j7gWRFWGDX6RltRetdZ8aD1KiZoh0zOA6pZU9NTj1SzTHbyIYXBUBPTs6YIFpmEYz8rbpsFjdJBOV4HX4ctX0ztRLfkgke8HDUfXPef3WQ0BZa5PZE2RF3z43i00BA8k0Uvsk8uwXA+diMzbT9aVzdgXkI41KcKScF2n58GF6AKy4jVOzTdipxzEBIFSHP9Nyfv90/4Kf7dgSrSq5FpkFrScAmV+70WYro3HzofHPesY5gezfUoGn0+cAPB9L5ZDFokzPMf1a7cWT74n8SZfgZ+UvWq6kj+W1oNkq/HWfC0Wxa3FGK+YQvttmcINxnaYDAv2ZqEGE1mBBd6KHxmEyt7XU/u/cQUVXeVwk+Cq3WEhaJ/G0LNzEeqUdXyZSo4JC6TScna+HRAgu+LlgHrkLNbZdFKU2NHMqkA02gD/7Ts0/Wqc84qiWX8A9Ylxwd/GvrSNEMLjodykxhvrQz4mZqk00ejUrgYrBWjlbqBlJ2mCv37P+NkibZsz0/ej5FF4BGm+JGa/2/+a9PWA187sQbCeLHLkZX70T4CBjZxmc7Hz6AmBy3UcglZEGAFZvqRi1TEn7ZD9wyinn0LQ3UGT/5zh33+U8WbkhCNU5BjkpSjTZlZ6Rowx6y5oo+KPmSbkgwYfyj8ekZ9JaI1SKG1fnfs0C53wpYg3pysXZSW3MgTCGYzpJ5YbOCCk/6k5Hzn9eEucU389DLiV5SJeff5T6529M1qJ6SMc0s/W2U8F+jt3wNhwX8cd/jPkrgKACtcMJ1uM7FxwkpKYLja3G0pATlOiZKJQ0BJWxo4K39xla5+NhXvo2fM/C6O Rx49KpbP h7Qq82NRBO+usTPvfWUG6fMrcuoxpVPzmnufXgPbwxE8L/D85585AUNeqqW1WrmHmo9mynXhkJMVln9T+tqpDOlXfi6LBirbqauCwIXReNSlx8UCE+5qkHs5RejjN1BHuTflNtfA8Wz0/xr9UQVhcznfaMMRnBvVH7zFNvQwW2uU1tQDrvz0Mq13FHwGqrYMJNISUHJcGnvfu0yIW1o4ECU4L6YG8MM/0lOZ73jDKWR1lY8YrhSZ3j21+6JnZNEXwf5cIVKOCsRuAmhPLQf2fh2xXnTaiea4gnBBDXzG00LVVlamRF4jbgkbLtpKRvfTPXQYoCOlEhatAQoVXv2q2Vs348o3xfkSYPFYcqqF8u/tCPanf0dF8Z2seYr57njV84vOO/K4oT6iKy8NQfmDuEfY7xBaw6mh9ORb2smlAln1k75aNNavrFsPgXQngzTX25rYlmqvlPA2L/5y3tV4Bt6mWgV6IL/Zc9XHqR+/B8dct/7gs5LQmtNRbLiv5ftRiDvnbv/7wug57xNbuUzdPlFQ+nbnPpBxSKiFBcg3ynhhy92ojqm8zkt5vK7LgGfkgSw+py/EmiLGZWZWGhC60er3GedPLff4PEHNI4Xbs8IQ5cE1CdrxXUK01GgMHAg+MOCYaws2uK7Q9OQrc/c/3mlfYxyNJsRpYTciP X-Bogosity: Unsure, tests=bogofilter, spamicity=0.496817, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add the ability to write a file_operations->mmap hook in Rust when using the miscdevice abstraction. The `vma` argument to the `mmap` hook uses the `VmAreaNew` type from the previous commit; this type provides the correct set of operations for a file_operations->mmap hook. Acked-by: Lorenzo Stoakes (for mm bits) Signed-off-by: Alice Ryhl --- rust/kernel/miscdevice.rs | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/rust/kernel/miscdevice.rs b/rust/kernel/miscdevice.rs index 7e2a79b3ae26..e5366f9c6d7f 100644 --- a/rust/kernel/miscdevice.rs +++ b/rust/kernel/miscdevice.rs @@ -11,6 +11,8 @@ use crate::{ bindings, error::{to_result, Error, Result, VTABLE_DEFAULT_ERROR}, + fs::File, + mm::virt::VmAreaNew, prelude::*, str::CStr, types::{ForeignOwnable, Opaque}, @@ -110,6 +112,15 @@ fn release(device: Self::Ptr) { drop(device); } + /// Handle for mmap. + fn mmap( + _device: ::Borrowed<'_>, + _file: &File, + _vma: &VmAreaNew, + ) -> Result { + kernel::build_error!(VTABLE_DEFAULT_ERROR) + } + /// Handler for ioctls. /// /// The `cmd` argument is usually manipulated using the utilties in [`kernel::ioctl`]. @@ -156,6 +167,7 @@ impl VtableHelper { const VTABLE: bindings::file_operations = bindings::file_operations { open: Some(fops_open::), release: Some(fops_release::), + mmap: maybe_fn(T::HAS_MMAP, fops_mmap::), unlocked_ioctl: maybe_fn(T::HAS_IOCTL, fops_ioctl::), #[cfg(CONFIG_COMPAT)] compat_ioctl: if T::HAS_COMPAT_IOCTL { @@ -216,6 +228,31 @@ impl VtableHelper { 0 } +/// # Safety +/// +/// `file` must be a valid file that is associated with a `MiscDeviceRegistration`. +/// `vma` must be a vma that is currently being mmap'ed with this file. +unsafe extern "C" fn fops_mmap( + file: *mut bindings::file, + vma: *mut bindings::vm_area_struct, +) -> c_int { + // SAFETY: The mmap call of a file can access the private data. + let private = unsafe { (*file).private_data }; + // SAFETY: Mmap calls can borrow the private data of the file. + let device = unsafe { ::borrow(private) }; + // SAFETY: The caller provides a vma that is undergoing initial VMA setup. + let area = unsafe { VmAreaNew::from_raw(vma) }; + // SAFETY: + // * The file is valid for the duration of this call. + // * There is no active fdget_pos region on the file on this thread. + let file = unsafe { File::from_raw_file(file) }; + + match T::mmap(device, file, area) { + Ok(()) => 0, + Err(err) => err.to_errno() as c_int, + } +} + /// # Safety /// /// `file` must be a valid file that is associated with a `MiscDeviceRegistration`. From patchwork Wed Dec 11 10:37:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13903329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 549E5E77180 for ; Wed, 11 Dec 2024 10:37:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0FF7E8D001B; Wed, 11 Dec 2024 05:37:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 087016B0121; Wed, 11 Dec 2024 05:37:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D0E578D001B; Wed, 11 Dec 2024 05:37:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9C53E6B011F for ; Wed, 11 Dec 2024 05:37:53 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 19B3114093F for ; Wed, 11 Dec 2024 10:37:53 +0000 (UTC) X-FDA: 82882327080.05.D43557E Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf08.hostedemail.com (Postfix) with ESMTP id C1F3E160002 for ; Wed, 11 Dec 2024 10:37:35 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=uImWuYWI; spf=pass (imf08.hostedemail.com: domain of 3fWtZZwkKCPocnkegt0jniqqing.eqonkpwz-oomxcem.qti@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3fWtZZwkKCPocnkegt0jniqqing.eqonkpwz-oomxcem.qti@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733913455; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=COiqNiZZHGXDau8NrwsaWsQuxOeFpNqIFDuZo6Q7yxY=; b=f1+SRmEOzuWyskgP0uRy2Tmd+mi/WjuaI8tojBXAOWs+K1fbFv4D9F6WsUXetQp9Ki7p1y PIrN0AqTK+HtwAA5LmdjHFHk94ars/o6PpQq4KBx46TfzXtWK5ddLso/zw9BnYoVslD4Fv cRFo6Pw0k+KU5nNOhRBNDgGrsJTCDEs= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=uImWuYWI; spf=pass (imf08.hostedemail.com: domain of 3fWtZZwkKCPocnkegt0jniqqing.eqonkpwz-oomxcem.qti@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3fWtZZwkKCPocnkegt0jniqqing.eqonkpwz-oomxcem.qti@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733913455; a=rsa-sha256; cv=none; b=jrkJ0ar9KAgODM2osmLXv0xgz+cAptnXeOE46rWy5fcngqX67zLMsNKaQX0ernMuI+wh3j 9msRYU2Xx83viTKOKqGBNJHFrJtoqIRjnqHBGjt/ny2IsR8zDNgNGGMLD/cwBdYrwiObdy 9tiP2/+6dCDI1eKnKQB5oGKM64QzvLg= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-434f387f346so24613755e9.1 for ; Wed, 11 Dec 2024 02:37:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1733913470; x=1734518270; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=COiqNiZZHGXDau8NrwsaWsQuxOeFpNqIFDuZo6Q7yxY=; b=uImWuYWIvLFJbgNT/PgL3n1XYmYnowLbNtYGr32QWu8svXWPl9K0oCL9uCRQrqvAAD VLS6uQYQAG+zt+FAp9jNfw5mE2LZiKunWqAC4vAKCg88W1iefL3B3f2mNRQQZb9Ugyf3 vKUG6m2TQuKMFQEILsLv8tyAg38Fj/Jymh3FgTxCPFMWIFSK86lPE2P3wpSVbT8KM8M5 b+cv5LRappodShtpJIK806RD+T73yuUW/5lMpqoRgBM+Ky0RpTZ4uY0HxOe4N4NY+Q/F Oqj119jZwBABU5kYMWKlQjjRKBgqkW9lnfdcAXYeRCDbr7e0+r9ZgYcCqsQoJ6U5/ddU y0Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733913470; x=1734518270; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=COiqNiZZHGXDau8NrwsaWsQuxOeFpNqIFDuZo6Q7yxY=; b=PTeHAzXni3BqN0hlgZUWykJ6SLxwzOcViIA5F0m+3X7V4OGPk593xupqL1+F/5jJj6 GRUHwscvHilRlVeV32B4fHDlIRm//N5rXSgg3Iftl9YqBkdP1vViWawLSoF2YuATR085 D5rrok0NAVlHM97FN8mJEEnUZ3cG+16gLBtP0wznk0Jq8nIcakOQ3/Jg8dfUBA+U045g LkfOxDTeJCRL4HaOyxWogwwW5gox4AUCi8ZdZrrWWJVjLW1Zxrs9MmyqdY9U+pVUbW/S pWpepT2wE08FJdBnN3UFJ4RNnCo5EPJikFqH2vuoQEtljSStX2tx4HNxcrTUFTpsDO9r VS+A== X-Forwarded-Encrypted: i=1; AJvYcCUus1LF2qbHU1JPo68K07Iyg0OgasRCUqR2f8N9TmCMoAeJshEPbg2mpojIWYjGd54YEoHSbu6nRA==@kvack.org X-Gm-Message-State: AOJu0YyjDX4K2IKkPaBfJPRs619wCSXi1FJ+DE16ji36rJTQJ6rWIMXL 8fEtkv+vcjinDpZ4BUQCCryw5i3CRd/H63vZq5R3l4Vp//PAIveKeQ+V8Wy+zKO8qYFgwj0f2lf 6lHzuBUXHQbwXcg== X-Google-Smtp-Source: AGHT+IGJWksM6TqfuEAm6PR3f/upyLvC2qq3UeG2eYl7geK7S/29j1j2r63y2Gy0jB84JFF33EsS10vrIieEeLs= X-Received: from wmbju8.prod.google.com ([2002:a05:600c:56c8:b0:434:feb1:add1]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1d20:b0:431:3bf9:3ebb with SMTP id 5b1f17b1804b1-4361c429dedmr15419925e9.24.1733913469827; Wed, 11 Dec 2024 02:37:49 -0800 (PST) Date: Wed, 11 Dec 2024 10:37:12 +0000 In-Reply-To: <20241211-vma-v11-0-466640428fc3@google.com> Mime-Version: 1.0 References: <20241211-vma-v11-0-466640428fc3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=22122; i=aliceryhl@google.com; h=from:subject:message-id; bh=StMV4D93TpURmhDr1aRbRum370oMmECfCETgkAAobmQ=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnWWtqlOomS0h8JmMDkoglQIvfsmDNU4Ge9Cyvz cp4qxKVmm6JAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ1lragAKCRAEWL7uWMY5 Rtq+EACKcdjUdiMl3drn8z33X6oq2sVRiZ63h26GpBhdtT4Pu4XbJ5dsrLQpt4HBMohyfcNJBl2 glSq0DquhIXF07WfXfh+S7IfeQo1tt7RqvpzmWYCAMTkiriJIagcjydFT5WHBX2K3deRnH/0MJf UjOOVQ9c1TYAEDcZm89a/jbumC75IKTVNrxBdyAjUqP4xSrJkjcTQASEF2PIJ/TDHAyhaKa01d8 D8bBK040A5ugUvgSIHnS5hBCv/HHMcMlGJzNq8k+Ae//dPuNZQ63+HgHiB2kmn9HEAXPXZYZJPc cCL6So9vYr7FjhHwlbDKlpx9p+VEWVxStqbtm6i+464u9YIctLCnhvZNPVBnYy5CH3WWQiQaS0+ C1jQvLJbVAgSySJSpu4OEBlC1PUgzQK/HBKtH9KKwrLaLWBF+ZmTjEf0XQbV81DKQqjTBKvOWDZ XIYcVkJ4HRGiCu/CLWjPfVf2L21R+fTOYo/o44PVKF4PNCgjNXp4ZxY3qFxyE6RlSu9Th1omtBP DQ8F+MxWuFXuvZSfAat4vGvLP+Vi2aqKR8hTcbPUpcYXF0U8kXmvcv038Xpt6yU9una+eVhcbJl dpVbSc4WCKGTdvdmibZkmIJr9lAnu1NQpX02WegzXy1A6lh2lPZhJIXlxAdNg/d0IMCgIIFvDx4 /8ePkAu2/EzpLbQ== X-Mailer: b4 0.13.0 Message-ID: <20241211-vma-v11-8-466640428fc3@google.com> Subject: [PATCH v11 8/8] task: rust: rework how current is accessed From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam05 X-Stat-Signature: y6qom5jff8hd3nu4rx4m1t36kp4bwt7u X-Rspamd-Queue-Id: C1F3E160002 X-Rspam-User: X-HE-Tag: 1733913455-530367 X-HE-Meta: U2FsdGVkX1+/kD4Xcozf2EdeSsi8xxcB/uHyC4VJgwVagOGPEWDVDjjvRxPIKao/iTGXhbyBkeqlk9yMdcnY4v7Oo62ncu8Mn7XLoTDtGG9w7TAZ6apZh0FRHCH/bfKG1hoKu9XspE9JAUYWB3ZyJwr9OftndPOmWTXaR15HUOzEENRNu3X4Az/5Dat8/eg6HgmLLqkVB6flq99MoDqHj2G8df9SyyXwO/Z/ES5sYtV3ckKzIJtyN9d41a1vKhEmOYarFo4GtxM49MjhVwGIkKhKo7QlBrcGXNrK6vk9BFi+zRS3qt5X+UdNoYEmIcilfeJTw9K8GsKc28Qw2qvg/hYZf3suaZqtuKRBB8FGoHUKvkkVMZw/QXWUAx4RcrxoMsonGpTK+sNbylODZKo+P1Sc9Eziez4kaPl0l970Ld49WIGPrxlsFJrzqvPzpd/1D9Xj+J3cuxX/tvx2iwi3/lf8n5A/NSmhMxepOpYkzxnuxfTQDbcZwc0T8jrW2/JMC++iBa4QHJQU8LIh8EU8PWlj+pDWKjh4gshS/DKYzFNxj62JaR277oMb4zMokkvMoaOODcbYpwCK1eiXf0NuLoHcL0eCEzK86ngRoHJsXkS3QAkogOsCAiwEbo/HVnAVhYa4FZN38aCAnw0xNQoysZ8ZyCmEaVqW8KsVNTxulIXqRom9H3p9rL0Tuh8+tGO/m5Kay8UeL/uqfSH0+MKIOVJWfCgtY8La15gtqZ32ASwIU65ucMdb4GmRX4n6qJBInlKx/i+9QHcQl8SHuVIKKKpwLZxfoKOCn1vCfyd3Po0OvCKUBAFGC/Old3FtWrM+FmwmnzRmlWgzacGz018EYTbgy7iA5C7B6e+uzeU0EgHDyb01WOdxvCO9RrJ+rXVuTvAPGKMZYgkQY17E23JsfmNuqVywt8y5zloYcP/LfMFPBRvPw/oiPpd9bQg/GKqaRXqFTuTNTrEZmFld1jy vbQCV5kx 0wkIBOMSj9m5hOHe8RVH7hojsPbvCIeG7gDVuboCn5HlBl/paKrl369coEDkGyGBtZv9T/TR3fyFTxuJ+mRg3mWRmGxPC2RyjW8fVB5KmpcH1sIQlR1G7YesWNsQm1sPL4rskgCdCrhom8XJHh17wx7Ga57Hm3v1epcxFK8gK4sGSJgKLG5ybjztQDEFHdtP2BR1cAVGLngdiA+DC4dF8qxsSqkZrY40XBKuZMXYLeVj3U254hSyjh2vFnq4Hq1Uic7MwLqFMvxYeKFPAcX3B2rOrN1CwAqiWlfyJNU+Ny7nCY4uMNmO6g3IV6b7xAbvoFuVb4N0CawteTl16+0xGcjkEfeaqSZvxcbvsLaZUHlalbka/r2wHDM726ADzaeQKpkhLcwJpiMbpvkj/+epWa0VpdynDeMwJ7hDFO4nBHEjDptwOz2uknq2WAPR9wSpl0aw1VkxBKP3MkuxYQx1mqHkbFIn+7Qo4cWamL9UsZkOX0LJei+IYzWlmC2j6nYuLV31eHDG28wma+XEwfYE4RbRJ9aF7r1mXnz4h85eRaDl8Vht3jSB+Kycxy+1m2Xbp99JGpNdqAblhKM4vZUM59nsbQnB5Q6GN1njdB3ykVaDgaoqzZYJqDlRNHHmsAkNZ0O5dKj6hcqH46Z/9gGevSKtG9wjONztMWnwc X-Bogosity: Unsure, tests=bogofilter, spamicity=0.492989, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a new type called `CurrentTask` that lets you perform various operations that are only safe on the `current` task. Use the new type to provide a way to access the current mm without incrementing its refcount. With this change, you can write stuff such as let vma = current!().mm().lock_vma_under_rcu(addr); without incrementing any refcounts. This replaces the existing abstractions for accessing the current pid namespace. With the old approach, every field access to current involves both a macro and a unsafe helper function. The new approach simplifies that to a single safe function on the `CurrentTask` type. This makes it less heavy-weight to add additional current accessors in the future. That said, creating a `CurrentTask` type like the one in this patch requires that we are careful to ensure that it cannot escape the current task or otherwise access things after they are freed. To do this, I declared that it cannot escape the current "task context" where I defined a "task context" as essentially the region in which `current` remains unchanged. So e.g., release_task() or begin_new_exec() would leave the task context. If a userspace thread returns to userspace and later makes another syscall, then I consider the two syscalls to be different task contexts. This allows values stored in that task to be modified between syscalls, even if they're guaranteed to be immutable during a syscall. Ensuring correctness of `CurrentTask` is slightly tricky if we also want the ability to have a safe `kthread_use_mm()` implementation in Rust. To support that safely, there are two patterns we need to ensure are safe: // Case 1: current!() called inside the scope. let mm; kthread_use_mm(some_mm, || { mm = current!().mm(); }); drop(some_mm); mm.do_something(); // UAF and: // Case 2: current!() called before the scope. let mm; let task = current!(); kthread_use_mm(some_mm, || { mm = task.mm(); }); drop(some_mm); mm.do_something(); // UAF The existing `current!()` abstraction already natively prevents the first case: The `&CurrentTask` would be tied to the inner scope, so the borrow-checker ensures that no reference derived from it can escape the scope. Fixing the second case is a bit more tricky. The solution is to essentially pretend that the contents of the scope execute on an different thread, which means that only thread-safe types can cross the boundary. Since `CurrentTask` is marked `NotThreadSafe`, attempts to move it to another thread will fail, and this includes our fake pretend thread boundary. This has the disadvantage that other types that aren't thread-safe for reasons unrelated to `current` also cannot be moved across the `kthread_use_mm()` boundary. I consider this an acceptable tradeoff. Cc: Christian Brauner Signed-off-by: Alice Ryhl --- rust/kernel/mm.rs | 22 ---- rust/kernel/task.rs | 284 ++++++++++++++++++++++++++++++---------------------- 2 files changed, 167 insertions(+), 139 deletions(-) diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs index 50f4861ae4b9..f7d1079391ef 100644 --- a/rust/kernel/mm.rs +++ b/rust/kernel/mm.rs @@ -142,28 +142,6 @@ fn deref(&self) -> &MmWithUser { // These methods are safe to call even if `mm_users` is zero. impl Mm { - /// Call `mmgrab` on `current.mm`. - #[inline] - pub fn mmgrab_current() -> Option> { - // SAFETY: It's safe to get the `mm` field from current. - let mm = unsafe { - let current = bindings::get_current(); - (*current).mm - }; - - if mm.is_null() { - return None; - } - - // SAFETY: The value of `current->mm` is guaranteed to be null or a valid `mm_struct`. We - // just checked that it's not null. Furthermore, the returned `&Mm` is valid only for the - // duration of this function, and `current->mm` will stay valid for that long. - let mm = unsafe { Mm::from_raw(mm) }; - - // This increments the refcount using `mmgrab`. - Some(ARef::from(mm)) - } - /// Returns a raw pointer to the inner `mm_struct`. #[inline] pub fn as_raw(&self) -> *mut bindings::mm_struct { diff --git a/rust/kernel/task.rs b/rust/kernel/task.rs index 07bc22a7645c..8c1ee46c03eb 100644 --- a/rust/kernel/task.rs +++ b/rust/kernel/task.rs @@ -7,6 +7,7 @@ use crate::{ bindings, ffi::{c_int, c_long, c_uint}, + mm::MmWithUser, pid_namespace::PidNamespace, types::{ARef, NotThreadSafe, Opaque}, }; @@ -31,22 +32,20 @@ #[macro_export] macro_rules! current { () => { - // SAFETY: Deref + addr-of below create a temporary `TaskRef` that cannot outlive the - // caller. + // SAFETY: This expression creates a temporary value that is dropped at the end of the + // caller's scope. The following mechanisms ensure that the resulting `&CurrentTask` cannot + // leave current task context: + // + // * To return to userspace, the caller must leave the current scope. + // * Operations such as `begin_new_exec()` are necessarily unsafe and the caller of + // `begin_new_exec()` is responsible for safety. + // * Rust abstractions for things such as a `kthread_use_mm()` scope must require the + // closure to be `Send`, so the `NotThreadSafe` field of `CurrentTask` ensures that the + // `&CurrentTask` cannot cross the scope in either direction. unsafe { &*$crate::task::Task::current() } }; } -/// Returns the currently running task's pid namespace. -#[macro_export] -macro_rules! current_pid_ns { - () => { - // SAFETY: Deref + addr-of below create a temporary `PidNamespaceRef` that cannot outlive - // the caller. - unsafe { &*$crate::task::Task::current_pid_ns() } - }; -} - /// Wraps the kernel's `struct task_struct`. /// /// # Invariants @@ -105,6 +104,44 @@ unsafe impl Send for Task {} // synchronised by C code (e.g., `signal_pending`). unsafe impl Sync for Task {} +/// Represents the [`Task`] in the `current` global. +/// +/// This type exists to provide more efficient operations that are only valid on the current task. +/// For example, to retrieve the pid-namespace of a task, you must use rcu protection unless it is +/// the current task. +/// +/// # Invariants +/// +/// Each value of this type must only be accessed from the task context it was created within. +/// +/// Of course, every thread is in a different task context, but for the purposes of this invariant, +/// these operations also permanently leave the task context: +/// +/// * Returning to userspace from system call context. +/// * Calling `release_task()`. +/// * Calling `begin_new_exec()` in a binary format loader. +/// +/// Other operations temporarily create a new sub-context: +/// +/// * Calling `kthread_use_mm()` creates a new context, and `kthread_unuse_mm()` returns to the +/// old context. +/// +/// This means that a `CurrentTask` obtained before a `kthread_use_mm()` call may be used again +/// once `kthread_unuse_mm()` is called, but it must not be used between these two calls. +/// Conversely, a `CurrentTask` obtained between a `kthread_use_mm()`/`kthread_unuse_mm()` pair +/// must not be used after `kthread_unuse_mm()`. +#[repr(transparent)] +pub struct CurrentTask(Task, NotThreadSafe); + +// Make all `Task` methods available on `CurrentTask`. +impl Deref for CurrentTask { + type Target = Task; + #[inline] + fn deref(&self) -> &Task { + &self.0 + } +} + /// The type of process identifiers (PIDs). type Pid = bindings::pid_t; @@ -131,119 +168,29 @@ pub fn current_raw() -> *mut bindings::task_struct { /// /// # Safety /// - /// Callers must ensure that the returned object doesn't outlive the current task/thread. - pub unsafe fn current() -> impl Deref { - struct TaskRef<'a> { - task: &'a Task, - _not_send: NotThreadSafe, + /// Callers must ensure that the returned object is only used to access a [`CurrentTask`] + /// within the task context that was active when this function was called. For more details, + /// see the invariants section for [`CurrentTask`]. + pub unsafe fn current() -> impl Deref { + struct TaskRef { + task: *const CurrentTask, } - impl Deref for TaskRef<'_> { - type Target = Task; + impl Deref for TaskRef { + type Target = CurrentTask; fn deref(&self) -> &Self::Target { - self.task + // SAFETY: The returned reference borrows from this `TaskRef`, so it cannot outlive + // the `TaskRef`, which the caller of `Task::current()` has promised will not + // outlive the task/thread for which `self.task` is the `current` pointer. Thus, it + // is okay to return a `CurrentTask` reference here. + unsafe { &*self.task } } } - let current = Task::current_raw(); TaskRef { - // SAFETY: If the current thread is still running, the current task is valid. Given - // that `TaskRef` is not `Send`, we know it cannot be transferred to another thread - // (where it could potentially outlive the caller). - task: unsafe { &*current.cast() }, - _not_send: NotThreadSafe, - } - } - - /// Returns a PidNamespace reference for the currently executing task's/thread's pid namespace. - /// - /// This function can be used to create an unbounded lifetime by e.g., storing the returned - /// PidNamespace in a global variable which would be a bug. So the recommended way to get the - /// current task's/thread's pid namespace is to use the [`current_pid_ns`] macro because it is - /// safe. - /// - /// # Safety - /// - /// Callers must ensure that the returned object doesn't outlive the current task/thread. - pub unsafe fn current_pid_ns() -> impl Deref { - struct PidNamespaceRef<'a> { - task: &'a PidNamespace, - _not_send: NotThreadSafe, - } - - impl Deref for PidNamespaceRef<'_> { - type Target = PidNamespace; - - fn deref(&self) -> &Self::Target { - self.task - } - } - - // The lifetime of `PidNamespace` is bound to `Task` and `struct pid`. - // - // The `PidNamespace` of a `Task` doesn't ever change once the `Task` is alive. A - // `unshare(CLONE_NEWPID)` or `setns(fd_pidns/pidfd, CLONE_NEWPID)` will not have an effect - // on the calling `Task`'s pid namespace. It will only effect the pid namespace of children - // created by the calling `Task`. This invariant guarantees that after having acquired a - // reference to a `Task`'s pid namespace it will remain unchanged. - // - // When a task has exited and been reaped `release_task()` will be called. This will set - // the `PidNamespace` of the task to `NULL`. So retrieving the `PidNamespace` of a task - // that is dead will return `NULL`. Note, that neither holding the RCU lock nor holding a - // referencing count to - // the `Task` will prevent `release_task()` being called. - // - // In order to retrieve the `PidNamespace` of a `Task` the `task_active_pid_ns()` function - // can be used. There are two cases to consider: - // - // (1) retrieving the `PidNamespace` of the `current` task - // (2) retrieving the `PidNamespace` of a non-`current` task - // - // From system call context retrieving the `PidNamespace` for case (1) is always safe and - // requires neither RCU locking nor a reference count to be held. Retrieving the - // `PidNamespace` after `release_task()` for current will return `NULL` but no codepath - // like that is exposed to Rust. - // - // Retrieving the `PidNamespace` from system call context for (2) requires RCU protection. - // Accessing `PidNamespace` outside of RCU protection requires a reference count that - // must've been acquired while holding the RCU lock. Note that accessing a non-`current` - // task means `NULL` can be returned as the non-`current` task could have already passed - // through `release_task()`. - // - // To retrieve (1) the `current_pid_ns!()` macro should be used which ensure that the - // returned `PidNamespace` cannot outlive the calling scope. The associated - // `current_pid_ns()` function should not be called directly as it could be abused to - // created an unbounded lifetime for `PidNamespace`. The `current_pid_ns!()` macro allows - // Rust to handle the common case of accessing `current`'s `PidNamespace` without RCU - // protection and without having to acquire a reference count. - // - // For (2) the `task_get_pid_ns()` method must be used. This will always acquire a - // reference on `PidNamespace` and will return an `Option` to force the caller to - // explicitly handle the case where `PidNamespace` is `None`, something that tends to be - // forgotten when doing the equivalent operation in `C`. Missing RCU primitives make it - // difficult to perform operations that are otherwise safe without holding a reference - // count as long as RCU protection is guaranteed. But it is not important currently. But we - // do want it in the future. - // - // Note for (2) the required RCU protection around calling `task_active_pid_ns()` - // synchronizes against putting the last reference of the associated `struct pid` of - // `task->thread_pid`. The `struct pid` stored in that field is used to retrieve the - // `PidNamespace` of the caller. When `release_task()` is called `task->thread_pid` will be - // `NULL`ed and `put_pid()` on said `struct pid` will be delayed in `free_pid()` via - // `call_rcu()` allowing everyone with an RCU protected access to the `struct pid` acquired - // from `task->thread_pid` to finish. - // - // SAFETY: The current task's pid namespace is valid as long as the current task is running. - let pidns = unsafe { bindings::task_active_pid_ns(Task::current_raw()) }; - PidNamespaceRef { - // SAFETY: If the current thread is still running, the current task and its associated - // pid namespace are valid. `PidNamespaceRef` is not `Send`, so we know it cannot be - // transferred to another thread (where it could potentially outlive the current - // `Task`). The caller needs to ensure that the PidNamespaceRef doesn't outlive the - // current task/thread. - task: unsafe { PidNamespace::from_ptr(pidns) }, - _not_send: NotThreadSafe, + // CAST: The layout of `struct task_struct` and `CurrentTask` is identical. + task: Task::current_raw().cast(), } } @@ -326,6 +273,109 @@ pub fn wake_up(&self) { } } +impl CurrentTask { + /// Access the address space of the current task. + /// + /// This function does not touch the refcount of the mm. + #[inline] + pub fn mm(&self) -> Option<&MmWithUser> { + // SAFETY: The `mm` field of `current` is not modified from other threads, so reading it is + // not a data race. + let mm = unsafe { (*self.as_ptr()).mm }; + + if mm.is_null() { + return None; + } + + // SAFETY: If `current->mm` is non-null, then it references a valid mm with a non-zero + // value of `mm_users`. Furthermore, the returned `&MmWithUser` borrows from this + // `CurrentTask`, so it cannot escape the scope in which the current pointer was obtained. + // + // This is safe even if `kthread_use_mm()`/`kthread_unuse_mm()` are used. There are two + // relevant cases: + // * If the `&CurrentTask` was created before `kthread_use_mm()`, then it cannot be + // accessed during the `kthread_use_mm()`/`kthread_unuse_mm()` scope due to the + // `NotThreadSafe` field of `CurrentTask`. + // * If the `&CurrentTask` was created within a `kthread_use_mm()`/`kthread_unuse_mm()` + // scope, then the `&CurrentTask` cannot escape that scope, so the returned `&MmWithUser` + // also cannot escape that scope. + // In either case, it's not possible to read `current->mm` and keep using it after the + // scope is ended with `kthread_unuse_mm()`. + Some(unsafe { MmWithUser::from_raw(mm) }) + } + + /// Access the pid namespace of the current task. + /// + /// This function does not touch the refcount of the namespace or use RCU protection. + #[doc(alias = "task_active_pid_ns")] + #[inline] + pub fn active_pid_ns(&self) -> Option<&PidNamespace> { + // SAFETY: It is safe to call `task_active_pid_ns` without RCU protection when calling it + // on the current task. + let active_ns = unsafe { bindings::task_active_pid_ns(self.as_ptr()) }; + + if active_ns.is_null() { + return None; + } + + // The lifetime of `PidNamespace` is bound to `Task` and `struct pid`. + // + // The `PidNamespace` of a `Task` doesn't ever change once the `Task` is alive. A + // `unshare(CLONE_NEWPID)` or `setns(fd_pidns/pidfd, CLONE_NEWPID)` will not have an effect + // on the calling `Task`'s pid namespace. It will only effect the pid namespace of children + // created by the calling `Task`. This invariant guarantees that after having acquired a + // reference to a `Task`'s pid namespace it will remain unchanged. + // + // When a task has exited and been reaped `release_task()` will be called. This will set + // the `PidNamespace` of the task to `NULL`. So retrieving the `PidNamespace` of a task + // that is dead will return `NULL`. Note, that neither holding the RCU lock nor holding a + // referencing count to the `Task` will prevent `release_task()` being called. + // + // In order to retrieve the `PidNamespace` of a `Task` the `task_active_pid_ns()` function + // can be used. There are two cases to consider: + // + // (1) retrieving the `PidNamespace` of the `current` task + // (2) retrieving the `PidNamespace` of a non-`current` task + // + // From system call context retrieving the `PidNamespace` for case (1) is always safe and + // requires neither RCU locking nor a reference count to be held. Retrieving the + // `PidNamespace` after `release_task()` for current will return `NULL` but no codepath + // like that is exposed to Rust. + // + // Retrieving the `PidNamespace` from system call context for (2) requires RCU protection. + // Accessing `PidNamespace` outside of RCU protection requires a reference count that + // must've been acquired while holding the RCU lock. Note that accessing a non-`current` + // task means `NULL` can be returned as the non-`current` task could have already passed + // through `release_task()`. + // + // To retrieve (1) the `&CurrentTask` type should be used which ensures that the returned + // `PidNamespace` cannot outlive the current task context. The `CurrentTask::active_pid_ns` + // function allows Rust to handle the common case of accessing `current`'s `PidNamespace` + // without RCU protection and without having to acquire a reference count. + // + // For (2) the `task_get_pid_ns()` method must be used. This will always acquire a + // reference on `PidNamespace` and will return an `Option` to force the caller to + // explicitly handle the case where `PidNamespace` is `None`, something that tends to be + // forgotten when doing the equivalent operation in `C`. Missing RCU primitives make it + // difficult to perform operations that are otherwise safe without holding a reference + // count as long as RCU protection is guaranteed. But it is not important currently. But we + // do want it in the future. + // + // Note for (2) the required RCU protection around calling `task_active_pid_ns()` + // synchronizes against putting the last reference of the associated `struct pid` of + // `task->thread_pid`. The `struct pid` stored in that field is used to retrieve the + // `PidNamespace` of the caller. When `release_task()` is called `task->thread_pid` will be + // `NULL`ed and `put_pid()` on said `struct pid` will be delayed in `free_pid()` via + // `call_rcu()` allowing everyone with an RCU protected access to the `struct pid` acquired + // from `task->thread_pid` to finish. + // + // SAFETY: If `current`'s pid ns is non-null, then it references a valid pid ns. + // Furthermore, the returned `&PidNamespace` borrows from this `CurrentTask`, so it cannot + // escape the scope in which the current pointer was obtained. + Some(unsafe { PidNamespace::from_ptr(active_ns) }) + } +} + // SAFETY: The type invariants guarantee that `Task` is always refcounted. unsafe impl crate::types::AlwaysRefCounted for Task { fn inc_ref(&self) {