From patchwork Fri Nov 29 16:32:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13888842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E586AD729E7 for ; Fri, 29 Nov 2024 16:32:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78DFD6B0085; Fri, 29 Nov 2024 11:32:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 73E436B008C; Fri, 29 Nov 2024 11:32:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5BA006B0092; Fri, 29 Nov 2024 11:32:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3F1186B0085 for ; Fri, 29 Nov 2024 11:32:58 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id DBE8F161288 for ; Fri, 29 Nov 2024 16:32:57 +0000 (UTC) X-FDA: 82839676710.28.323A4EE Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf29.hostedemail.com (Postfix) with ESMTP id AAE5D120016 for ; Fri, 29 Nov 2024 16:32:43 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mv7FhHJw; spf=pass (imf29.hostedemail.com: domain of 3tuxJZwkKCPcZkhbdqxgkfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3tuxJZwkKCPcZkhbdqxgkfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732897969; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j9X1mSWROln1wrDBMjuahPQZdYG+tTYafJBbOIpF/U8=; b=NsjBL42Nhnl2qZ+stJY8pGfOagh7W8tIkmAM0Ona5Wifh+5tzqvbwDkLDJWpW/BNWuBPA0 NxsGbXFQf+c2fXssqqptas49z95FUlilFm15ixgA1KBLIgdPrLNf+f4qBA4YCs/cHHIHZ2 vSitWESf6KmWWqfmD2Hgwi1HbitrYWI= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mv7FhHJw; spf=pass (imf29.hostedemail.com: domain of 3tuxJZwkKCPcZkhbdqxgkfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3tuxJZwkKCPcZkhbdqxgkfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732897969; a=rsa-sha256; cv=none; b=rSOZnzb5TAMzCmHvRAWmlSWthaclPMLrKPNxbywhmk6PuWKQuUK3FBMUrqJY2oW1Sulk7J uevKlE3OZhW8/UYAjn6ciGRHR8YqF25GH2/1JSpBv8l0aJXm3SK8MNFCimxcCNnIKaL8PP 3EGlYK57ogoThbQuvIdtYg6dHsmQ6nA= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-434ad71268dso12152605e9.1 for ; Fri, 29 Nov 2024 08:32:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732897974; x=1733502774; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=j9X1mSWROln1wrDBMjuahPQZdYG+tTYafJBbOIpF/U8=; b=mv7FhHJwW249MIHVmjkqhpOwq+cMiqS8m7NAx4PG8TX98yAzGA2WIkAimFESSp5ADC NjQL09MnzgMq5Z+5e7xBc1jWyKZA9eSIQdrilaNsJ7mrrvdUwsD7vyn5Qt9C/NkCWJhi uDb+aR5We4yCBC84OO7sPGf8Cy+P8vF4OWQxH2Ygd+QPdq2dDWrw+tlJDT55qi/J4IKg /mN+AWrge05oN5pKHPIUJ5c5UITgM1lWLTmy/nFfsp5KGoNqlS7L0RGCOMWX+fP5Cbp4 TL9ZcpVZwCvkaOeGEFFN+I5x/Lt3RKbFt3AHnmorNBJn/B9wj9KuyVw8Sju2iUyZ5YZk AICw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732897974; x=1733502774; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=j9X1mSWROln1wrDBMjuahPQZdYG+tTYafJBbOIpF/U8=; b=kgEr2jrng1U+f8tQh2Igi0MScKq1yW/mLsX1/7sTCOM3Q4QMTF1BCvEomij2YqFVb5 yWPyxYaxXy95sLNHQjjB2HmgtlyIcY59xa0NynYpUxBGeoIx4EJdSAB9aeWspXfwTOCU zNF+a+NzI0i4J58zBQ81vH3UWqqdIbciaEPw6SYfq1IOnMtH3jssYsTYkvxZIwTuryMs XE7YGX+Tc2vGdYJvWjwnP2vC3p/v4BzlzNNovmtzVTZl37uVHISHdT78l1ii5dX3S9zv +iJQ5OfT+S7PTyGEii+DZNAwXm0QSlzkRyKcsGzzFS7ijVMNzUKIfJeNW8ew93Z7vGpQ coJw== X-Forwarded-Encrypted: i=1; AJvYcCUOT4oLM1jG+eoXYO1lQa4ITBD+Kdh17FpYOsKItE0xrZVKvmhoa1BJMEUY7/OEcn2DgwJ3Kf5QYA==@kvack.org X-Gm-Message-State: AOJu0Yy+RymidbctEDRQZkEoqU0ou6OnV32m1D+UkaiPp1k9PPVVW/lm BUBR/eedelXc+CAVfEik7PXd1XRgymWlMrW8r32SKtBJxJhUkSuZh3HW4aKezwjtHCdIWJadNRC rRUoztYcSv47MKA== X-Google-Smtp-Source: AGHT+IG1Q9DtpcqFchzTEpT4t81tXHCjVZL4m+DlejQopKd4Dv0OkuXm1Htn/deYdYiYHN+thVzGQZM8rcROeaE= X-Received: from wmjy22.prod.google.com ([2002:a7b:cd96:0:b0:434:9322:f580]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3589:b0:434:a4d3:31f0 with SMTP id 5b1f17b1804b1-434afb9533fmr77447825e9.3.1732897974563; Fri, 29 Nov 2024 08:32:54 -0800 (PST) Date: Fri, 29 Nov 2024 16:32:34 +0000 In-Reply-To: <20241129-vma-v10-0-4dfff05ba927@google.com> Mime-Version: 1.0 References: <20241129-vma-v10-0-4dfff05ba927@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=10055; i=aliceryhl@google.com; h=from:subject:message-id; bh=9dPy033+czHjobcXs31YH34rNSge6iKWvSxm8dMkcKg=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnSeytYAQEEiK6whr+TrOU4N31iHFdV607MGM6c jNKDH8RODmJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ0nsrQAKCRAEWL7uWMY5 Rl1BD/43K0CD3qubFqV+QMRL/6t3UCee4q4QUSFTTg5qJ7tu57qKFhi+Z5+h/98/rPE11mSQVnv fYVg0nh5ueazDKhTrdTjPqDDeP8J71WHLI2yDJpy9m/DEbtqHQjFi4Ev/ROD3Npv0ASTYYdhJ1Q N+Zdfdftn+jufRt3v3P0n631vzB/nKT+cTYg2is2VYOlthxxbg0z1CtgLpQ3wI1aOt4bgQ+IjC3 rBsVW/lx+D9CBKhMT6z+IZ9eXnCA6WF4Qs6nDe9CrBaKZWwlKxouff8zXtNnIfXntvQFb6Sx4ow v5uwVIjipKvKDEX7X/6URnX7sK93JbXcr63Dx4w3LYt/EcKFPCuECZ22T+HQ2+uznUOxICWI4MD 3RiM7iPMRNEV1h5Fv8JrqJUhpXtHeydK5iUL85qfjSVLeZNQs5OoWRWaCvzCs/Vyeji0aFob9S0 b44YQjzgcZlt71l5KwAz+yijXWgWzwbHXt0Mz0of2EP9WlhHEGH5cVEMGwNdv+lASz+s7hh4+iS aqHdqyjijCQLduXH1+7cDDfUfnepdbF3q9yAQ682CbbGM25oiDhGPY8I1vi+HAhVeHfeL9tEKMK 68mqnN0SeyMqOkYmDQ4deL+V4WAiCfBb/pm7+k/rXYPINeo6hFfAchVChGNWIge2r3lvRLuqPBt Egg70NbRZv5rQ4g== X-Mailer: b4 0.13.0 Message-ID: <20241129-vma-v10-1-4dfff05ba927@google.com> Subject: [PATCH v10 1/8] mm: rust: add abstraction for struct mm_struct From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Queue-Id: AAE5D120016 X-Rspamd-Server: rspam12 X-Stat-Signature: b8h5gr7hkzpwzgrn9zq684n6mmmxwqyh X-Rspam-User: X-HE-Tag: 1732897963-556679 X-HE-Meta: U2FsdGVkX19dPD1fjvnl3fos4ICcoIkvLdoBETrKjW8V/Ul/R2usuahJGUDbjIiybdH4VwSjoRG/DZWp4E1jW5v6Sx9BsqS2SAqYlGBk4EpyfPiBh8kpFjDbms4F6E7pmK2RJ0ihFOiNw2K52mbnpVO68wfK2TpS2Jn+c3DI17BCNJx8kgc7wYNhQOiTrNzd6/U/sa6GVvgUReEK2VtvRERkd6fX2BGP9U0XYbT7qtFUPkBNHaQTMhQ6XqO0ndPjUzWaV0w00eMZrAq/MgGK9+nFs9Avtc2PPm/BlgbwQa4PxAujOOBiSb02WvuxXpWFqsVwV+9i/yt+Dze2wOo5H5fjSCLKmSR8uA/tNI+UWX3s0dqva7nbGeoxwxoB51SUzD/afIQVJ4d4fnUIOY6DaHeJjyfiMLWtlymF1USSiTug+u+gJ5A6/jrrNikFH2iLBF8QR44w9r3ErHCHepG4pL76As/KRubsjgdlW5SNPw3nvk9Mp/v7ut6hNFa5Y98rOPbGfbkVsNdtiBWwTzJSQdQsz9o06bRzmRVLu7mcpRamdMwtMqwubNJJJLt6Kg8MO0bI37zKcWDKD/Q1YPhot6uhoISpQM6lEcjt4nBA9DN6oCWTFUt093NYbJIrCX5oLBp3uEFYNdF6rN/39WV9hSiYR77XTgIGluDoNHQ7fSAewLZEShbGDa+Ly+mex/U3nhpxi6qeCzHCUDbofuywEIUdcFCGbt964Z/NvafElCZsQdBlPK1oYcsjqx+6g21Sr9OPQytkcRG9KWZTNmM2tT1Bo5irxIZfPFOA6oIi8URlxhPMLy6ntDzqRN4Tn/O4hf6W1ljdM9QcrjN89dBZGtSccHggWFJaQQjpj0z/6ZiB1HuYo+alI+Bva6vK++7u6C6ZKrZ8bBz4QpW5x8wHpjjhZpXsCjv8I3OSs/zVmzM5N3ycqWlL5o0xUKaJLztR8PGpg3Syz+sTlzdfNyC 26h31n03 kvmf9rJavFhLZ6ZS7rQt/Tmuss2VINMdjGdwibt7bulsQQCA5c2BNtZprJ6luDJIRlSifTVNmkks8FOsQC3rTmYDj+oTxEk6YpXq+P5qBfAAR0Fpk1fEGhuzbu7yRYxXf5VVuwC4Ami8MOd/Uzfra5qdY/7YYM1kPl8Q9znxT1SVJERnr8+j6nJDHb6iPH8lAudoPh6JdaU/CxFbATvSw8yUgiKvMiX44ZbzLX71TNhd7WJMurzIRu0t9PoSjp1pxUVgaPXOER8FpaZMV05Cq3GE6djdHjSztV9XqANSbQ5/MMs3Ov4uwfAXpy4FqOvSnjKISNCnL2QTz3pBHvYhlQIRTMb9/XgVZfQQTfEiR6eFAmjrGKM3m131rt2bJh7JH7cmSwhKlcdnCNuytPj8LtY7e1q4QvkYCWj315JR+TNR7ad9S7ktekFOe6obAopsql+1abJSG9fSup3ILJK2oeceBhWfvYn8HMEOtkpjoxd1aJk6fbQzi5F5z9BmJ8hon2MgW3qHr6BTqrHKP6GB5hjIBUB21Epzj0ty8DxZ9kjOgNeO9s4tkkM3RclMFJ/KT3fkDDbx7vii/ATrcoi6hHpg9t3DlMkr3IijPMuJM8nZqvytcS/97xED0VifG2O4e/Bom32tuOSLWcAntla/z71EvkQfwwSmAsQBZmeqaBGX0ALQb2A+L0L8GrhnDa6G+U2ACtscq1KX5pNw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: These abstractions allow you to reference a `struct mm_struct` using both mmgrab and mmget refcounts. This is done using two Rust types: * Mm - represents an mm_struct where you don't know anything about the value of mm_users. * MmWithUser - represents an mm_struct where you know at compile time that mm_users is non-zero. This allows us to encode in the type system whether a method requires that mm_users is non-zero or not. For instance, you can always call `mmget_not_zero` but you can only call `mmap_read_lock` when mm_users is non-zero. It's possible to access current->mm without a refcount increment, but that is added in a later patch of this series. Acked-by: Lorenzo Stoakes (for mm bits) Signed-off-by: Alice Ryhl --- rust/helpers/helpers.c | 1 + rust/helpers/mm.c | 39 +++++++++ rust/kernel/lib.rs | 1 + rust/kernel/mm.rs | 219 +++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 260 insertions(+) diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c index dcf827a61b52..9d748ec845b3 100644 --- a/rust/helpers/helpers.c +++ b/rust/helpers/helpers.c @@ -16,6 +16,7 @@ #include "fs.c" #include "jump_label.c" #include "kunit.c" +#include "mm.c" #include "mutex.c" #include "page.c" #include "pid_namespace.c" diff --git a/rust/helpers/mm.c b/rust/helpers/mm.c new file mode 100644 index 000000000000..7201747a5d31 --- /dev/null +++ b/rust/helpers/mm.c @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +void rust_helper_mmgrab(struct mm_struct *mm) +{ + mmgrab(mm); +} + +void rust_helper_mmdrop(struct mm_struct *mm) +{ + mmdrop(mm); +} + +void rust_helper_mmget(struct mm_struct *mm) +{ + mmget(mm); +} + +bool rust_helper_mmget_not_zero(struct mm_struct *mm) +{ + return mmget_not_zero(mm); +} + +void rust_helper_mmap_read_lock(struct mm_struct *mm) +{ + mmap_read_lock(mm); +} + +bool rust_helper_mmap_read_trylock(struct mm_struct *mm) +{ + return mmap_read_trylock(mm); +} + +void rust_helper_mmap_read_unlock(struct mm_struct *mm) +{ + mmap_read_unlock(mm); +} diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index e1065a7551a3..6555e0847192 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -46,6 +46,7 @@ pub mod kunit; pub mod list; pub mod miscdevice; +pub mod mm; #[cfg(CONFIG_NET)] pub mod net; pub mod page; diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs new file mode 100644 index 000000000000..84cba581edaa --- /dev/null +++ b/rust/kernel/mm.rs @@ -0,0 +1,219 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2024 Google LLC. + +//! Memory management. +//! +//! C header: [`include/linux/mm.h`](srctree/include/linux/mm.h) + +use crate::{ + bindings, + types::{ARef, AlwaysRefCounted, NotThreadSafe, Opaque}, +}; +use core::{ops::Deref, ptr::NonNull}; + +/// A wrapper for the kernel's `struct mm_struct`. +/// +/// Since `mm_users` may be zero, the associated address space may not exist anymore. You can use +/// [`mmget_not_zero`] to be able to access the address space. +/// +/// The `ARef` smart pointer holds an `mmgrab` refcount. Its destructor may sleep. +/// +/// # Invariants +/// +/// Values of this type are always refcounted using `mmgrab`. +/// +/// [`mmget_not_zero`]: Mm::mmget_not_zero +#[repr(transparent)] +pub struct Mm { + mm: Opaque, +} + +// SAFETY: It is safe to call `mmdrop` on another thread than where `mmgrab` was called. +unsafe impl Send for Mm {} +// SAFETY: All methods on `Mm` can be called in parallel from several threads. +unsafe impl Sync for Mm {} + +// SAFETY: By the type invariants, this type is always refcounted. +unsafe impl AlwaysRefCounted for Mm { + #[inline] + fn inc_ref(&self) { + // SAFETY: The pointer is valid since self is a reference. + unsafe { bindings::mmgrab(self.as_raw()) }; + } + + #[inline] + unsafe fn dec_ref(obj: NonNull) { + // SAFETY: The caller is giving up their refcount. + unsafe { bindings::mmdrop(obj.cast().as_ptr()) }; + } +} + +/// A wrapper for the kernel's `struct mm_struct`. +/// +/// This type is like [`Mm`], but with non-zero `mm_users`. It can only be used when `mm_users` can +/// be proven to be non-zero at compile-time, usually because the relevant code holds an `mmget` +/// refcount. It can be used to access the associated address space. +/// +/// The `ARef` smart pointer holds an `mmget` refcount. Its destructor may sleep. +/// +/// # Invariants +/// +/// Values of this type are always refcounted using `mmget`. The value of `mm_users` is non-zero. +#[repr(transparent)] +pub struct MmWithUser { + mm: Mm, +} + +// SAFETY: It is safe to call `mmput` on another thread than where `mmget` was called. +unsafe impl Send for MmWithUser {} +// SAFETY: All methods on `MmWithUser` can be called in parallel from several threads. +unsafe impl Sync for MmWithUser {} + +// SAFETY: By the type invariants, this type is always refcounted. +unsafe impl AlwaysRefCounted for MmWithUser { + #[inline] + fn inc_ref(&self) { + // SAFETY: The pointer is valid since self is a reference. + unsafe { bindings::mmget(self.as_raw()) }; + } + + #[inline] + unsafe fn dec_ref(obj: NonNull) { + // SAFETY: The caller is giving up their refcount. + unsafe { bindings::mmput(obj.cast().as_ptr()) }; + } +} + +// Make all `Mm` methods available on `MmWithUser`. +impl Deref for MmWithUser { + type Target = Mm; + + #[inline] + fn deref(&self) -> &Mm { + &self.mm + } +} + +// These methods are safe to call even if `mm_users` is zero. +impl Mm { + /// Call `mmgrab` on `current.mm`. + #[inline] + pub fn mmgrab_current() -> Option> { + // SAFETY: It's safe to get the `mm` field from current. + let mm = unsafe { + let current = bindings::get_current(); + (*current).mm + }; + + if mm.is_null() { + return None; + } + + // SAFETY: The value of `current->mm` is guaranteed to be null or a valid `mm_struct`. We + // just checked that it's not null. Furthermore, the returned `&Mm` is valid only for the + // duration of this function, and `current->mm` will stay valid for that long. + let mm = unsafe { Mm::from_raw(mm) }; + + // This increments the refcount using `mmgrab`. + Some(ARef::from(mm)) + } + + /// Returns a raw pointer to the inner `mm_struct`. + #[inline] + pub fn as_raw(&self) -> *mut bindings::mm_struct { + self.mm.get() + } + + /// Obtain a reference from a raw pointer. + /// + /// # Safety + /// + /// The caller must ensure that `ptr` points at an `mm_struct`, and that it is not deallocated + /// during the lifetime 'a. + #[inline] + pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a Mm { + // SAFETY: Caller promises that the pointer is valid for 'a. Layouts are compatible due to + // repr(transparent). + unsafe { &*ptr.cast() } + } + + /// Calls `mmget_not_zero` and returns a handle if it succeeds. + #[inline] + pub fn mmget_not_zero(&self) -> Option> { + // SAFETY: The pointer is valid since self is a reference. + let success = unsafe { bindings::mmget_not_zero(self.as_raw()) }; + + if success { + // SAFETY: We just created an `mmget` refcount. + Some(unsafe { ARef::from_raw(NonNull::new_unchecked(self.as_raw().cast())) }) + } else { + None + } + } +} + +// These methods require `mm_users` to be non-zero. +impl MmWithUser { + /// Obtain a reference from a raw pointer. + /// + /// # Safety + /// + /// The caller must ensure that `ptr` points at an `mm_struct`, and that `mm_users` remains + /// non-zero for the duration of the lifetime 'a. + #[inline] + pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser { + // SAFETY: Caller promises that the pointer is valid for 'a. The layout is compatible due + // to repr(transparent). + unsafe { &*ptr.cast() } + } + + /// Lock the mmap read lock. + #[inline] + pub fn mmap_read_lock(&self) -> MmapReadGuard<'_> { + // SAFETY: The pointer is valid since self is a reference. + unsafe { bindings::mmap_read_lock(self.as_raw()) }; + + // INVARIANT: We just acquired the read lock. + MmapReadGuard { + mm: self, + _nts: NotThreadSafe, + } + } + + /// Try to lock the mmap read lock. + #[inline] + pub fn mmap_read_trylock(&self) -> Option> { + // SAFETY: The pointer is valid since self is a reference. + let success = unsafe { bindings::mmap_read_trylock(self.as_raw()) }; + + if success { + // INVARIANT: We just acquired the read lock. + Some(MmapReadGuard { + mm: self, + _nts: NotThreadSafe, + }) + } else { + None + } + } +} + +/// A guard for the mmap read lock. +/// +/// # Invariants +/// +/// This `MmapReadGuard` guard owns the mmap read lock. +pub struct MmapReadGuard<'a> { + mm: &'a MmWithUser, + // `mmap_read_lock` and `mmap_read_unlock` must be called on the same thread + _nts: NotThreadSafe, +} + +impl Drop for MmapReadGuard<'_> { + #[inline] + fn drop(&mut self) { + // SAFETY: We hold the read lock by the type invariants. + unsafe { bindings::mmap_read_unlock(self.mm.as_raw()) }; + } +} From patchwork Fri Nov 29 16:32:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13888843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF68FD729EC for ; Fri, 29 Nov 2024 16:33:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 852F76B0092; Fri, 29 Nov 2024 11:33:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 801AD6B0093; Fri, 29 Nov 2024 11:33:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 630576B0095; Fri, 29 Nov 2024 11:33:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4053A6B0092 for ; Fri, 29 Nov 2024 11:33:00 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E14BBA1311 for ; Fri, 29 Nov 2024 16:32:59 +0000 (UTC) X-FDA: 82839676626.21.B614FC1 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf28.hostedemail.com (Postfix) with ESMTP id 72D04C0002 for ; Fri, 29 Nov 2024 16:32:46 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=LqsI9Oe2; spf=pass (imf28.hostedemail.com: domain of 3uOxJZwkKCPkbmjdfszimhpphmf.dpnmjovy-nnlwbdl.psh@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3uOxJZwkKCPkbmjdfszimhpphmf.dpnmjovy-nnlwbdl.psh@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732897971; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a5YzoZTIOzo/0986DnSppxfBI0p8rQsJVoOP2q6LCvk=; b=ZCOKkI5YjgW6i0+TgPUxgxfTGT0YTSlHlYSugpRWGAowdiNPMFwbsyIcuLdUY7jYgw1XpQ 9rEPlF/hpkwGEMB+mlBMhNp9AqZAuh8KPT+URN6RWKSqUpSKSavVuI3NGRX3ec+eoE3f0s vETAypXXjjnIjbADFGebWQlec1L1lSs= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=LqsI9Oe2; spf=pass (imf28.hostedemail.com: domain of 3uOxJZwkKCPkbmjdfszimhpphmf.dpnmjovy-nnlwbdl.psh@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3uOxJZwkKCPkbmjdfszimhpphmf.dpnmjovy-nnlwbdl.psh@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732897971; a=rsa-sha256; cv=none; b=Uu1JfRoJXk4uggUqkmX46yd9NASATBJ1otnHqM8BLMTpxhD27Fbrh8elD3hSK4I0o0tjnc vnN9i+/wB8CqPj8onZWHyfXqodM/jw8R6GQRhC/Qr9OCwWZ9nizb6IppG7BBxwdccPfaXa TwgJLDsRFEyyvM1teX8iONmj2IG2QuY= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-434a9dce888so14163625e9.0 for ; Fri, 29 Nov 2024 08:32:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732897976; x=1733502776; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=a5YzoZTIOzo/0986DnSppxfBI0p8rQsJVoOP2q6LCvk=; b=LqsI9Oe2xV3TMSkqAVmdfVM6Ev1oRo1xPmwP2MjkNelLVRPhUQ8q6cZllPnW3Lz9yE fGlaH3z8DWry/KvEgoKanVKwblD2VonEP1ePSuFK8HF74u18UmsbAc3LSGaciqDB/qNC VfMce8FsV9DKYGX/ojlkrtMu2J6QlKdTvu9mu7w8Guz3cJMWlVThhZB9MObR8uqgAV/y U7J2EJ0lFOZQHhE2fZwNwQRiHwM38gK9TgCbyN5HLwEowh4b4JRKXwPSBI+3Nj4X1eBP Vgd83tLjc0H8LiW7r6XTdNwdidskpSKlth5Al+wvcVjnCWJZz1rHsvHqz8DF2RbOCXoY u8dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732897976; x=1733502776; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=a5YzoZTIOzo/0986DnSppxfBI0p8rQsJVoOP2q6LCvk=; b=HE7s8l3RgJrdUpWZTA1sdCKiU860gEnsx8dJUTOBFSGDZ3VuxG4dr9ayIJ9qf9mlFj Aoy3qI+jjKpPrXSDPLG4vz80LAcW4kf5Hth4C+g6YIUxRAZqhLlS2u2DxiiASBPPjexR KG5CPhrEWrQJnv1zujnb8dk7tdZlfiu9lK+KwTjacRW+7K5SKizazw/WRLnX3h1MyJKc 8hyRU4yDS056PAUx76P8nzZJDbWZ8zNwyxVMDFOTRfmo8oTGSVimIepfCs6KibBPQCVj EzxDdSorlAhQHT65zqav0qXidldIPOYywOxBcxIk3Ol/G3P/Ks9Fc1DXpMPmuxbJl4sD IZCA== X-Forwarded-Encrypted: i=1; AJvYcCUPpsEy1TqWtxFNYOOGO2ntZis5h3jD3o/CWIXYm3YQ0i2Ss1h+sKtR6kbg7p1PckzCaguu1aYKxA==@kvack.org X-Gm-Message-State: AOJu0YwINLP6aMeJv0kLgSbCVQ+sQLfCawqTBYCJETkuhaOhcrg8K38C I2eCrD1549loXzd7l+gmy9lbZdxRWbW7FS/xKVBrvxpOMkicZ+c6pzZwmrZ+hfmDdPsYKGVu3GH fD3q7tplgR0n+OA== X-Google-Smtp-Source: AGHT+IHze4h565Aoh2DoHrp99j4LSl0kVAIFqSGJmgywq3dMrDmKMH3RGZw03PbIllpNmKnyBzK6/JzPDiaVmPE= X-Received: from wmoq13.prod.google.com ([2002:a05:600c:46cd:b0:431:1260:2444]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3146:b0:434:a169:6ff8 with SMTP id 5b1f17b1804b1-434a9db8423mr128388385e9.3.1732897976603; Fri, 29 Nov 2024 08:32:56 -0800 (PST) Date: Fri, 29 Nov 2024 16:32:35 +0000 In-Reply-To: <20241129-vma-v10-0-4dfff05ba927@google.com> Mime-Version: 1.0 References: <20241129-vma-v10-0-4dfff05ba927@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=9643; i=aliceryhl@google.com; h=from:subject:message-id; bh=qfqYB5CY+zrrWoF1p5PA1wE61kagKg3f8qMfUMwrH2s=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnSeyunSmy45C3+dwg4sTBb0910JzVks9XKF/WA HFj/FmBAwyJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ0nsrgAKCRAEWL7uWMY5 Ro6+D/0eS03pE3ZODh9XPuyPKCvsQdzzedfXMU5VED+gXAd+veD/s0mctxSF5+dCrZH/WzcSeG0 E0wvbKQGYvTYjDOrABcB4VUKPw13cEX5BB5FFhslBKJaVfKdQjRib4r2lFNk8nm2hDydoigQEf5 Necf6GOJbCrOgVx9Rkn6E2leJwlSrSiobggvnXvvPuJjEcyVX02iG/xj9eaQBwg8R9x2cozehqe 5AZsxJl9JZqNPPCvnMquL3HhnS51Tx9OzpfikrhsWXxOYqQMbaSYCsq3JREbXVut+Wyf8plCNvF 8sUwhycV5tQES/p36rGUFikzr7Z4qUmQbkCF8XX/gOJshCUGnlnMOfXLf3cZQAUn7/FZWVwIVe2 07m+UyPyzz2M5AMMEaGHizR1WiL1I09EhEDgdNrnAqTA+LPowkPYhRE+urCovOVmwG2KEVjlu+B 6sNpFxFnHDbpUAMnIR+WBFeG6+ZeDfSrrC46g59620JfvBYSBcS17v0TvqAIsHsPC8w41FLcPPD O/6dcaOE9l5QO27XYeCm2uOSGn8Mn4SemMv9PK7a7V4ECtf9kuFUJ7CBD0KjnZwz94CrvGAxwPq dkZ21D6ljWsx+KrHr5XkZ4KglFbqxP/ZE6f8FCSkOH/53QWrOW//QlkjLpp08wULZs6JDDB43rw bJ2rL11wZz/I6WQ== X-Mailer: b4 0.13.0 Message-ID: <20241129-vma-v10-2-4dfff05ba927@google.com> Subject: [PATCH v10 2/8] mm: rust: add vm_area_struct methods that require read access From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Queue-Id: 72D04C0002 X-Rspamd-Server: rspam12 X-Stat-Signature: qqe5hdq4ueyjdsmey59w4dw1x4mwbzhx X-Rspam-User: X-HE-Tag: 1732897966-671861 X-HE-Meta: U2FsdGVkX1/ZoqYWI4nxsOIL3gW6MK/0QsURPEpjHiHVx6TBgn263MtZr8X6Zvhvm2mf1D7Vz3Q3WrzTzGh/QHp6ocBSUaGT2QGMVq5LywYEtTghkSaaoRQfR9U5dALZoimiV7O7f7i0fPXJYheyjIvN5tN2Yd4WLRoxbCYZx9pg6Hq6b8itPlR+qZJq/CVcUdwWexIad0107BM6Ky+6H376HGeKRxFRC+lyRZvTK2aB8dD3fQvz6YW2ytjW0AYjHarboRf6cokYrBCaYydp84XkyM4me4vYqo3OWZ4sdg3xbYch26RqKb7NsVNsuU/NWuZ/Kqoa/Gst+jtlqAmdPBf2bUMKk+xG6K2u2zrlvHvYEm+5itBqpwob4PFcjU/9A5i99lG//Ey4xOZB7GD1jCPUU1guV9zaf7M7nWS6wBicrI8cYHxYC+pz29QxiY1aI8tGoquSGniJ7ghoz3Z14xNaNn1imUbqhkutkTwT6tEY8cNMDMuFImb0KMYsn0/V1at7TwGUtw9wdDZ9qkmJnKt9uMAX7GCk7PiZ//xdMG8PltCIyNpGD0Nzb0/xzAnA4908xyw4eQ5p8hlbXJRz81lEHFdYUxULxtYTRfWdWCoQVq2+sEZ5fJomgseZjHFp5KJgskOvtL3lyvSEq7uTMEyzOBSfuJFu6weYfluVRBCS4gY4w3YFjAYJgeae1kX4MRjFcSAzKR0BNPWeLK5mHlBGEpPMWYNQfIKmWHItWv/m+noDyfhM1Bvp36liLJ32OafXd5ig3GolnmAKD77+NjMOemEcvUH+CNGxnt4460JtyYL34nriJ1XdlFG/tLRPRStvsuJuQnNqFg/na/PKtjyFeqIxfunSP8SzPuiUjIgWP+VXHxAkBe/1Hh/j91uIs0jEScdXgQqKLDFaylVnPuIyDShizp26jzrwwQViuneM3IhWzvUWtnUIleIF96K5TEB6Hh8sQ8nIyWocKpb 56z4SBI8 V3nDIQzI0zk1Q3o4OreV+llgJpN/jCebtviK6q+mb1xwrEwX3rX7PAv5U5/HeZ2TbUdFYYqfYXQDse/2oCShYN33028C6hSw5IcdvBukPxk+W63JwD/EL8i2XKbgAcQ6rPfpZSxqDh/oLT/JyTZImFrpO5HdHBP5P8at2p/0mnf5x22+J7RFhzJ2VyUDq+WR1L6xw72l8/OXizZtNaAAlw+2g7oK45THPLZvOANjgp4LXyx8g4EWQS9CjADtN1GU6IOSiFly6rG+cBkN/uOhohP90FNnrjbKPtz/YsO+mKdqecWqSlRBE9BAkc3Txruc2k/M8m/s96gL8mquiIjpJtPxHmIWeDJF4AV4jbC8bpoZkPODXufA9s7lxmM8/aAzvxox0KzT6JVb2OjOLPnQKJLfXEF97sz7mW1xtZItyhdSwpkI3EFMVWPHFGtAh7ogt0a/IwpTmq9jTyTpYKiOsowrtcAp/ILB8SKFRvd1sTXP5arDR+bEK4cwgcOn4BrI6a9bM4TfKaW7GzqV87KzwIZN2z6QH4GseRne3FqgMoV3uutzZNKVuqFEJL5jNOvPVAbIa9IS8+b+wd/6R/7BNhufqKKdU/45Est896TiOUOfl4W5cdWBcJPfeaytWs8mG2H2D3PjJsqCRmfVPB+ufdOGc/5jVtAt0eaTmiKZaUjvQ6mEN5C285rfLlHDkYwp5TaxkIKWm0GPud3a+GEtt1gzP6A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This adds a type called VmAreaRef which is used when referencing a vma that you have read access to. Here, read access means that you hold either the mmap read lock or the vma read lock (or stronger). Additionally, a vma_lookup method is added to the mmap read guard, which enables you to obtain a &VmAreaRef in safe Rust code. This patch only provides a way to lock the mmap read lock, but a follow-up patch also provides a way to just lock the vma read lock. Acked-by: Lorenzo Stoakes (for mm bits) Reviewed-by: Jann Horn Signed-off-by: Alice Ryhl --- rust/helpers/mm.c | 6 ++ rust/kernel/mm.rs | 21 ++++++ rust/kernel/mm/virt.rs | 182 +++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 209 insertions(+) diff --git a/rust/helpers/mm.c b/rust/helpers/mm.c index 7201747a5d31..7b72eb065a3e 100644 --- a/rust/helpers/mm.c +++ b/rust/helpers/mm.c @@ -37,3 +37,9 @@ void rust_helper_mmap_read_unlock(struct mm_struct *mm) { mmap_read_unlock(mm); } + +struct vm_area_struct *rust_helper_vma_lookup(struct mm_struct *mm, + unsigned long addr) +{ + return vma_lookup(mm, addr); +} diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs index 84cba581edaa..ace8e7d57afe 100644 --- a/rust/kernel/mm.rs +++ b/rust/kernel/mm.rs @@ -12,6 +12,8 @@ }; use core::{ops::Deref, ptr::NonNull}; +pub mod virt; + /// A wrapper for the kernel's `struct mm_struct`. /// /// Since `mm_users` may be zero, the associated address space may not exist anymore. You can use @@ -210,6 +212,25 @@ pub struct MmapReadGuard<'a> { _nts: NotThreadSafe, } +impl<'a> MmapReadGuard<'a> { + /// Look up a vma at the given address. + #[inline] + pub fn vma_lookup(&self, vma_addr: usize) -> Option<&virt::VmAreaRef> { + // SAFETY: We hold a reference to the mm, so the pointer must be valid. Any value is okay + // for `vma_addr`. + let vma = unsafe { bindings::vma_lookup(self.mm.as_raw(), vma_addr as _) }; + + if vma.is_null() { + None + } else { + // SAFETY: We just checked that a vma was found, so the pointer is valid. Furthermore, + // the returned area will borrow from this read lock guard, so it can only be used + // while the mmap read lock is still held. + unsafe { Some(virt::VmAreaRef::from_raw(vma)) } + } + } +} + impl Drop for MmapReadGuard<'_> { #[inline] fn drop(&mut self) { diff --git a/rust/kernel/mm/virt.rs b/rust/kernel/mm/virt.rs new file mode 100644 index 000000000000..032eea4d4690 --- /dev/null +++ b/rust/kernel/mm/virt.rs @@ -0,0 +1,182 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2024 Google LLC. + +//! Virtual memory. + +use crate::{bindings, types::Opaque}; + +/// A wrapper for the kernel's `struct vm_area_struct` with read access. +/// +/// It represents an area of virtual memory. +/// +/// # Invariants +/// +/// The caller must hold the mmap read lock or the vma read lock. +#[repr(transparent)] +pub struct VmAreaRef { + vma: Opaque, +} + +// Methods you can call when holding the mmap or vma read lock (or strong). They must be usable no +// matter what the vma flags are. +impl VmAreaRef { + /// Access a virtual memory area given a raw pointer. + /// + /// # Safety + /// + /// Callers must ensure that `vma` is valid for the duration of 'a, and that the mmap or vma + /// read lock (or stronger) is held for at least the duration of 'a. + #[inline] + pub unsafe fn from_raw<'a>(vma: *const bindings::vm_area_struct) -> &'a Self { + // SAFETY: The caller ensures that the invariants are satisfied for the duration of 'a. + unsafe { &*vma.cast() } + } + + /// Returns a raw pointer to this area. + #[inline] + pub fn as_ptr(&self) -> *mut bindings::vm_area_struct { + self.vma.get() + } + + /// Returns the flags associated with the virtual memory area. + /// + /// The possible flags are a combination of the constants in [`flags`]. + #[inline] + pub fn flags(&self) -> vm_flags_t { + // SAFETY: By the type invariants, the caller holds at least the mmap read lock, so this + // access is not a data race. + unsafe { (*self.as_ptr()).__bindgen_anon_2.vm_flags as _ } + } + + /// Returns the (inclusive) start address of the virtual memory area. + #[inline] + pub fn start(&self) -> usize { + // SAFETY: By the type invariants, the caller holds at least the mmap read lock, so this + // access is not a data race. + unsafe { (*self.as_ptr()).__bindgen_anon_1.__bindgen_anon_1.vm_start as _ } + } + + /// Returns the (exclusive) end address of the virtual memory area. + #[inline] + pub fn end(&self) -> usize { + // SAFETY: By the type invariants, the caller holds at least the mmap read lock, so this + // access is not a data race. + unsafe { (*self.as_ptr()).__bindgen_anon_1.__bindgen_anon_1.vm_end as _ } + } + + /// Zap pages in the given page range. + /// + /// This clears page table mappings for the range at the leaf level, leaving all other page + /// tables intact, and freeing any memory referenced by the VMA in this range. That is, + /// anonymous memory is completely freed, file-backed memory has its reference count on page + /// cache folio's dropped, any dirty data will still be written back to disk as usual. + #[inline] + pub fn zap_page_range_single(&self, address: usize, size: usize) { + let (end, did_overflow) = address.overflowing_add(size); + if did_overflow || address < self.start() || self.end() < end { + // TODO: call WARN_ONCE once Rust version of it is added + return; + } + + // SAFETY: By the type invariants, the caller has read access to this VMA, which is + // sufficient for this method call. This method has no requirements on the vma flags. The + // address range is checked to be within the vma. + unsafe { + bindings::zap_page_range_single( + self.as_ptr(), + address as _, + size as _, + core::ptr::null_mut(), + ) + }; + } +} + +/// The integer type used for vma flags. +#[doc(inline)] +pub use bindings::vm_flags_t; + +/// All possible flags for [`VmAreaRef`]. +pub mod flags { + use super::vm_flags_t; + use crate::bindings; + + /// No flags are set. + pub const NONE: vm_flags_t = bindings::VM_NONE as _; + + /// Mapping allows reads. + pub const READ: vm_flags_t = bindings::VM_READ as _; + + /// Mapping allows writes. + pub const WRITE: vm_flags_t = bindings::VM_WRITE as _; + + /// Mapping allows execution. + pub const EXEC: vm_flags_t = bindings::VM_EXEC as _; + + /// Mapping is shared. + pub const SHARED: vm_flags_t = bindings::VM_SHARED as _; + + /// Mapping may be updated to allow reads. + pub const MAYREAD: vm_flags_t = bindings::VM_MAYREAD as _; + + /// Mapping may be updated to allow writes. + pub const MAYWRITE: vm_flags_t = bindings::VM_MAYWRITE as _; + + /// Mapping may be updated to allow execution. + pub const MAYEXEC: vm_flags_t = bindings::VM_MAYEXEC as _; + + /// Mapping may be updated to be shared. + pub const MAYSHARE: vm_flags_t = bindings::VM_MAYSHARE as _; + + /// Page-ranges managed without `struct page`, just pure PFN. + pub const PFNMAP: vm_flags_t = bindings::VM_PFNMAP as _; + + /// Memory mapped I/O or similar. + pub const IO: vm_flags_t = bindings::VM_IO as _; + + /// Do not copy this vma on fork. + pub const DONTCOPY: vm_flags_t = bindings::VM_DONTCOPY as _; + + /// Cannot expand with mremap(). + pub const DONTEXPAND: vm_flags_t = bindings::VM_DONTEXPAND as _; + + /// Lock the pages covered when they are faulted in. + pub const LOCKONFAULT: vm_flags_t = bindings::VM_LOCKONFAULT as _; + + /// Is a VM accounted object. + pub const ACCOUNT: vm_flags_t = bindings::VM_ACCOUNT as _; + + /// Should the VM suppress accounting. + pub const NORESERVE: vm_flags_t = bindings::VM_NORESERVE as _; + + /// Huge TLB Page VM. + pub const HUGETLB: vm_flags_t = bindings::VM_HUGETLB as _; + + /// Synchronous page faults. (DAX-specific) + pub const SYNC: vm_flags_t = bindings::VM_SYNC as _; + + /// Architecture-specific flag. + pub const ARCH_1: vm_flags_t = bindings::VM_ARCH_1 as _; + + /// Wipe VMA contents in child on fork. + pub const WIPEONFORK: vm_flags_t = bindings::VM_WIPEONFORK as _; + + /// Do not include in the core dump. + pub const DONTDUMP: vm_flags_t = bindings::VM_DONTDUMP as _; + + /// Not soft dirty clean area. + pub const SOFTDIRTY: vm_flags_t = bindings::VM_SOFTDIRTY as _; + + /// Can contain `struct page` and pure PFN pages. + pub const MIXEDMAP: vm_flags_t = bindings::VM_MIXEDMAP as _; + + /// MADV_HUGEPAGE marked this vma. + pub const HUGEPAGE: vm_flags_t = bindings::VM_HUGEPAGE as _; + + /// MADV_NOHUGEPAGE marked this vma. + pub const NOHUGEPAGE: vm_flags_t = bindings::VM_NOHUGEPAGE as _; + + /// KSM may merge identical pages. + pub const MERGEABLE: vm_flags_t = bindings::VM_MERGEABLE as _; +} From patchwork Fri Nov 29 16:32:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13888844 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC499D729E9 for ; Fri, 29 Nov 2024 16:33:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C991B6B0098; Fri, 29 Nov 2024 11:33:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C49136B0096; Fri, 29 Nov 2024 11:33:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE9746B0098; Fri, 29 Nov 2024 11:33:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 891256B0095 for ; Fri, 29 Nov 2024 11:33:02 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id F27664119D for ; Fri, 29 Nov 2024 16:33:01 +0000 (UTC) X-FDA: 82839676752.12.539CF68 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf07.hostedemail.com (Postfix) with ESMTP id 6140540007 for ; Fri, 29 Nov 2024 16:32:51 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=nlGyf1RM; spf=pass (imf07.hostedemail.com: domain of 3uuxJZwkKCPsdolfhu1kojrrjoh.frpolqx0-ppnydfn.ruj@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3uuxJZwkKCPsdolfhu1kojrrjoh.frpolqx0-ppnydfn.ruj@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732897976; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=94NYmr5c9ppvPDgAeOeEOikiVfAFQj6mOJl+8qXFwoY=; b=J8+EMM9K4Rog5t3qGQFAimLbKR39nHtxW8Oi7129silhRVvFNA3KXqujtTYDhM34yvLoOX 7pz1SQfXEXxyjHllcWxPOKy1wumlmp7SbTbNk5Zx06EKA9GU6v/2+Y1rr5NPC/X/pXfoh+ JYuNgHbctfN7w8KWezMUMHrgIz1GwB0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732897976; a=rsa-sha256; cv=none; b=YfjV3RIwq8EdtXCVOt0Or0ljJ7K+fGIbMOQRp0YJqRO/06k6xpgfb//MfQiY33JYn707lj 6rWhlZU0s46GuGSxKZs/fxeC9Q1/OKyTqR9CU0fW+/DDov4BnVklRUBZ+SlProgT/HhZ0H xs5WqCHGva15UnywCYVzDpWQAfK9vw8= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=nlGyf1RM; spf=pass (imf07.hostedemail.com: domain of 3uuxJZwkKCPsdolfhu1kojrrjoh.frpolqx0-ppnydfn.ruj@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3uuxJZwkKCPsdolfhu1kojrrjoh.frpolqx0-ppnydfn.ruj@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-434a90febb8so12014765e9.1 for ; Fri, 29 Nov 2024 08:32:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732897978; x=1733502778; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=94NYmr5c9ppvPDgAeOeEOikiVfAFQj6mOJl+8qXFwoY=; b=nlGyf1RMSQwDmU0YU5I8qZOC7ou+EB4w7EXLF5/xK01xNSxr/i3eM92l+y6wm8ag6d ruZ7alT8O29QGyMPZFt9zJTnOTFPz7cTcMo3KgM3YII/Qo1rKDN8JtRIxTcC9IfcTL0a idD408jRo2MqDo9/mo97wyy4lKwKn1AsCprjXk3ROsWYOFH/IdocJUNzOC0iA04/rcPx okMr4iByTWRtK2yWg1pK+wsLP/xTJs9gucG8Yjd6MkvMc/Z42tJ5MoWI39YO/EPmdsWt BVHXm6GZTU8scLIbu2wcguknz5GODAaQoCpS/+kukVfQKuvxQUJobvEXactqUxUBtkZE YYDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732897978; x=1733502778; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=94NYmr5c9ppvPDgAeOeEOikiVfAFQj6mOJl+8qXFwoY=; b=OHl7p+BdOXICvoVaeecuNJ5JH+PLXOeBGEHgB88u78JRnBSXfgiBrfYDdS2tIBk3Iu nRggVgayhlftrwOw22A/LopO1MARD8llPVgGFLxBSeQGHDhtbkFR26vgbsRSWDSk0CY8 +QjqFZSbkj4N0n4kdZPESMprBBsBYn7y3AsFFBlohrMLrXixcpmFLEykMYsH5zrg/wGn Am0y8Fru/ONkC4Yz3AMWX3tmEVn7BA46784Hv29a2O3qD5ZwT+doZcSg3L0+gfNPY5yw dVWs6zGj6YEnMwFQLoKfygTJu4FQ9puuYDBUqqAw86KB5EVExdJ53lrKiMbrrT9ZRFow sjKQ== X-Forwarded-Encrypted: i=1; AJvYcCW4Y4kwFlZdFIW9KfhHnSabnJ3nG9tGtEM2hdB73gmniowaVsZIduEI8cYBk0xTYcAuyl+PVWl4cA==@kvack.org X-Gm-Message-State: AOJu0YxZzdNs+Ibx+s9Br0e7V+5ci6LEvbYwKQHNudGqie/4bw1dF3Tb 9iljt1Kw2ggwrxrhBuxuq0QXLOA2FBKinnESgzCRBEPky/0xjMEhGl0Ym0UBqyFF1VuR5e+/eXb YCgtudjZazBIHdQ== X-Google-Smtp-Source: AGHT+IGrQvjMp3red6jWsR1GGii7AoihEDmHs9z4agdmTgk8GW6+jX/m+iq6oJEN70Cr+c0nxG1Xp0i1oBsbhdE= X-Received: from wmpq23.prod.google.com ([2002:a05:600c:3317:b0:42c:b336:8685]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4750:b0:431:5871:6c5d with SMTP id 5b1f17b1804b1-434a9dbbd1amr138602905e9.3.1732897978665; Fri, 29 Nov 2024 08:32:58 -0800 (PST) Date: Fri, 29 Nov 2024 16:32:36 +0000 In-Reply-To: <20241129-vma-v10-0-4dfff05ba927@google.com> Mime-Version: 1.0 References: <20241129-vma-v10-0-4dfff05ba927@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3694; i=aliceryhl@google.com; h=from:subject:message-id; bh=d5PVsjv5Dk2wDmJvfAYSWwtPELYfnKxOPm05AGfN2t4=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnSeyuYNpPwhGV89bUWs5GRH8aX7HH6P4Ztr1U0 bGuq73pEJ2JAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ0nsrgAKCRAEWL7uWMY5 RqU2EACmE2VsGq4kuy2dH1HczWotSUGlrvZA17ocbWpWcLJh0JQ6N+LZjV0Xu6m6TTovT5IofyS XcN7yaQjwdI9oGmRkOFB1oRLjjAxEUwr46P/sjxC1UgiuiQaJv0auLW8LP5QiGKA+UaNqGQCcdO eGQMKqM4TikGluAykvFlxukfGrYNYTU6sQOVfaZLY2t+/SVMUva/uwFPQh1gpv9vTB9D/Sgi5Wo UCvQFh3JwkBvVaijNJoF6ONbRTdCbLiMDc9dVRNzZfDRw0uR4fQDzMEkSRkJ3v5+fScreYBljBK k0B1DXlJBNp212kQGxXIA0aC32E2UwuKNb9dHMPpBko8yZwn7Rrf3sbMU1fcfiXhneXaFOUprO9 H1v5/MRfiqsXXQpFiPQP1+SRjuHSlQ1wJnCOKaViw++5mXvDilf2gbMZO8HhXVGSB55HaS0a3U/ NjQPkNTFXTBDV8r76dThH5WrcX+Es5A2yJ9gVfTy+Busr0pH9IgPXxO6lorkt4eOoKXvgD1pto7 kfaSTKS+sHsPqpcgAJeI8RikZWbszcU+qzT2Z0y1akLeFVehqY5O7oMjESLS16mMpy4ui+8Dkc5 wDnJuzljFqHjFo7j68LGNQ7CVPHglJvW6OslNF4+CePBEE/j1kEWVjZl4+WoxVFvU+zMzj8t4cw YUSBsw9j8CvU+QQ== X-Mailer: b4 0.13.0 Message-ID: <20241129-vma-v10-3-4dfff05ba927@google.com> Subject: [PATCH v10 3/8] mm: rust: add vm_insert_page From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 6140540007 X-Stat-Signature: msf81gudakk1swm9fntqcynbbmhmppx9 X-Rspam-User: X-HE-Tag: 1732897971-618480 X-HE-Meta: U2FsdGVkX1/CaQYcqE8Pnkv8tv8IlVUi+YcCshQJhSxBl7AUrqMzER0xnDUL7NytWDSomEsyDHDeeyCaJmwUp6Oz2IwqwthznePaWk/HuPxqk5oW0gLStsCfT0hCzJrI5Qfd9/lEPVQBOaaP7cvxztQQp4KSwBNhr44uEHHEJxOoV+rHVJrwR69qS3Olf5RlnTvRlPRHU7zZ8gGNT1gSACw3JWBxVqKZ9LbWcphHpjMD+WazgsbQ3jECNdBi5HMY7JVMMjLEYGAH2caaEBgLLMJKABHQ8ZwFNlqev4x8QyrW4pWIw7XNtAmk4XXIMboZY45y8kXg4Lnx/LUOd4GTEsqsSkzAs5MsBAx0R3hV7nxY1KE4eJpd2p/CX2q8aebvf7Psr/Nq6jA1buchNAT+c2LXsOA443dB2/m+Yz3irfOXQK0ROJk7Lw2LkWuYCQZEQPC4htoXGZvYTeb013VNEn/TKr8yvuWv6LU1HUNP670L3kLarwrnfiyc+V7UQsKLF0mkKAEq+2WZatGbD5UxUighzz9h9GtgLE6RhG+imc374I0tP/S8j07S3hVJ5H3KOY1NuqeaGz+ioKzFsFtPPjtCKC4duMv5y7ZOJHKPFX3QjAs48FozO8cVI/dy/gM75uTKjhw4aHdWA9IWJ5WhQKvLAfjj0N9UVqeIxreOP/C9w91xvdVx7NlbnbvolPbvqSwqNgjS9BlO5jW+5rJx4XsFQDXm7TKIzVTQHVImp/sEeMS6sfr5Onh2sMu61yTfGQsENWHVOxRTTUJnljFYOqhZX+kl2aWYju1wiVfhsjMnScCnSUsATykhdo7KRm1+KkYS2twhnZGhLi6xtDgxEErfsmc7pZY3D4PAb1BxZaN5zHOUX5+cO7dh8N0XVx4/c0Ib0JBtEUlGh2z/i5exjzmaMG8enNi8krYe60dlYwPnQCXYhzAtT/dIsIo/QS2IqeZclcXYmlRnsabJO5F jEVlj6yj WAd4f7WyLLpxFCvBAPSnKNOWrLIbIy5z3Wa0GVM4/3Qw1gJRiXcE0mfJSlTHj19+EjgUw/JpAgD8V2oPV6FxmWdZgRf7ba/Z3f9aVA+KTJMCbv1L4bHiCqwbbbchDd1Co919IswwveQ4Aa/GwnGFbpLNIxdetoA4RzGM4wAneM0HlGb3gnWFYTWYXKh4Xnnaj9laA9AMaCLqHcHJRb7/xSV586uwE1WX3enH6yRUVMy5sgxjG80wC6XaNHxq+KcDbCG176trCRVrtCJFfCUFcnoFPTs4M3CMVz/YMZpwInuDxqZCaW6ggcHLDk/FaSrbzirUbNwCqKIscr3bUnWF6+4+GwN2Ez4AOyYOXv6085opU9StfcvCzIjuTAv7VziY9rdyW2/deGWS1pOmg4+YetHks2igWHaCVef8mGUuQM+AQI7QF9b9GwTFxWsP+LtTx0GByVnppN4jZZtccy1OlBFIMCmDO+OENrVC1vxAoMzzrge0wBje1pA6iqBQ8Ew9OgSVkuV/boIuCBsElBJ478QclXDezEVdYTVo2Tvg74z9EukXn+uQfK1ccvhj4GP2yKhW6PsXfARkBTW9l3ZcgC8FCJ0xn0G9taRLMZkmyw9RSyUkb4XKFFi6Jupx7rotXXxWcRKKVIhByLnSTqgBRVmMxMnPjrHhcJEP39FOcSo7DTLW0jRcMGEjkzNjJVCNp6LzbrAhxRubgRsI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The vm_insert_page method is only usable on vmas with the VM_MIXEDMAP flag, so we introduce a new type to keep track of such vmas. The approach used in this patch assumes that we will not need to encode many flag combinations in the type. I don't think we need to encode more than VM_MIXEDMAP and VM_PFNMAP as things are now. However, if that becomes necessary, using generic parameters in a single type would scale better as the number of flags increases. Acked-by: Lorenzo Stoakes (for mm bits) Signed-off-by: Alice Ryhl --- rust/kernel/mm/virt.rs | 70 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 69 insertions(+), 1 deletion(-) diff --git a/rust/kernel/mm/virt.rs b/rust/kernel/mm/virt.rs index 032eea4d4690..fa1fd6aa703c 100644 --- a/rust/kernel/mm/virt.rs +++ b/rust/kernel/mm/virt.rs @@ -4,7 +4,14 @@ //! Virtual memory. -use crate::{bindings, types::Opaque}; +use crate::{ + bindings, + error::{to_result, Result}, + page::Page, + types::Opaque, +}; + +use core::ops::Deref; /// A wrapper for the kernel's `struct vm_area_struct` with read access. /// @@ -91,6 +98,67 @@ pub fn zap_page_range_single(&self, address: usize, size: usize) { ) }; } + + /// Check whether the `VM_MIXEDMAP` flag is set. + /// + /// This can be used to access methods that require `VM_MIXEDMAP` to be set. + #[inline] + pub fn as_mixedmap_vma(&self) -> Option<&VmAreaMixedMap> { + if self.flags() & flags::MIXEDMAP != 0 { + // SAFETY: We just checked that `VM_MIXEDMAP` is set. All other requirements are + // satisfied by the type invariants of `VmAreaRef`. + Some(unsafe { VmAreaMixedMap::from_raw(self.as_ptr()) }) + } else { + None + } + } +} + +/// A wrapper for the kernel's `struct vm_area_struct` with read access and `VM_MIXEDMAP` set. +/// +/// It represents an area of virtual memory. +/// +/// # Invariants +/// +/// The caller must hold the mmap read lock or the vma read lock. The `VM_MIXEDMAP` flag must be +/// set. +#[repr(transparent)] +pub struct VmAreaMixedMap { + vma: VmAreaRef, +} + +// Make all `VmAreaRef` methods available on `VmAreaMixedMap`. +impl Deref for VmAreaMixedMap { + type Target = VmAreaRef; + + #[inline] + fn deref(&self) -> &VmAreaRef { + &self.vma + } +} + +impl VmAreaMixedMap { + /// Access a virtual memory area given a raw pointer. + /// + /// # Safety + /// + /// Callers must ensure that `vma` is valid for the duration of 'a, and that the mmap read lock + /// (or stronger) is held for at least the duration of 'a. The `VM_MIXEDMAP` flag must be set. + #[inline] + pub unsafe fn from_raw<'a>(vma: *const bindings::vm_area_struct) -> &'a Self { + // SAFETY: The caller ensures that the invariants are satisfied for the duration of 'a. + unsafe { &*vma.cast() } + } + + /// Maps a single page at the given address within the virtual memory area. + /// + /// This operation does not take ownership of the page. + #[inline] + pub fn vm_insert_page(&self, address: usize, page: &Page) -> Result { + // SAFETY: The caller has read access and has verified that `VM_MIXEDMAP` is set. The page + // is order 0. The address is checked on the C side so it can take any value. + to_result(unsafe { bindings::vm_insert_page(self.as_ptr(), address as _, page.as_ptr()) }) + } } /// The integer type used for vma flags. From patchwork Fri Nov 29 16:32:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13888845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1BBAD729E9 for ; Fri, 29 Nov 2024 16:33:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 127E16B0095; Fri, 29 Nov 2024 11:33:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B1D16B009B; Fri, 29 Nov 2024 11:33:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF7666B009A; Fri, 29 Nov 2024 11:33:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id AFB316B0095 for ; Fri, 29 Nov 2024 11:33:04 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4023A140721 for ; Fri, 29 Nov 2024 16:33:04 +0000 (UTC) X-FDA: 82839676878.15.D1EEA87 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf11.hostedemail.com (Postfix) with ESMTP id 6279E40005 for ; Fri, 29 Nov 2024 16:32:53 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MlP9GKC0; spf=pass (imf11.hostedemail.com: domain of 3vOxJZwkKCP0fqnhjw3mqlttlqj.htrqnsz2-rrp0fhp.twl@flex--aliceryhl.bounces.google.com designates 209.85.221.73 as permitted sender) smtp.mailfrom=3vOxJZwkKCP0fqnhjw3mqlttlqj.htrqnsz2-rrp0fhp.twl@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732897978; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XEZERPJwfgVL12T7CsV2RsjLJQFd2+FhD7A3aSN4nWQ=; b=2nq+GQnGWCgG4F/cXHCy7VO4JaMSLKedw/cMhsJTTzaOf+Q/Qx035BrIB6BhvimBw3it79 uCodmauqL/zApxc6I39WrKVfP+DYYzrgIj+YVScdqvri0KBF69iJ/GcFYrzhEnJTcHOOOi DGgxUN4Xf4MIkkMkoflYLG/BQeZvBlQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732897978; a=rsa-sha256; cv=none; b=PzBW4HLE4BVYvlucD99rcnYIsMx6nvC/8IxvGWAtS7eYEE5aF8Yup9zLg3uNejNP6hsFep O6eE0BA61Hu6B5SV3we3sjjlktk63wI6ke/rOtbs5z6Ja+ofW+/aCaD9kpeoAYOKodMCtz DxVlGGTV02eH8QqZtI7JSRtUI5zHPWQ= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MlP9GKC0; spf=pass (imf11.hostedemail.com: domain of 3vOxJZwkKCP0fqnhjw3mqlttlqj.htrqnsz2-rrp0fhp.twl@flex--aliceryhl.bounces.google.com designates 209.85.221.73 as permitted sender) smtp.mailfrom=3vOxJZwkKCP0fqnhjw3mqlttlqj.htrqnsz2-rrp0fhp.twl@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-385d6ee042eso1085306f8f.0 for ; Fri, 29 Nov 2024 08:33:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732897981; x=1733502781; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XEZERPJwfgVL12T7CsV2RsjLJQFd2+FhD7A3aSN4nWQ=; b=MlP9GKC0RX+m5zo5xnHIjD/ryOQ2noszUjkOMl72MK9VeDH/FaTPadtAlG/y2Tq+lR jkWDuI4FfNt/AVmT7EcjALIbBXfDY1qNpYAG1z8lkXj7XuAGY6Mv5wvJ45UpfYh5F3pj 5IEJfsUk7CjkbEK3Ez6LaeQPtfBRQJ9eONGbqyF68iFdz7D86+SgO+NGP4UVrYKqrLif sfqZml6PzasLO8JvJsxGskcyQEEB8jD07gUcP3U6fOhhexqUXKMBY19rB7ZogYhgGEX/ F1N5wCfMz8/CGF8pGX8rw/vOcHGITqXthrhEyL4tRgvu2AgBFe/av69fwRQXI0I0zMnp t37Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732897981; x=1733502781; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XEZERPJwfgVL12T7CsV2RsjLJQFd2+FhD7A3aSN4nWQ=; b=k4eNTS7BxAA7iHX0uHol4JmgzI7rSeK0P7TVTCWR7LhUqRM+HHVhXKca3NokpK4DdA LokQs/ezYggub44rE1rU40xYEVex7prJLslN+fXj3Br5tto54KY2Ft3PTOefmQyXFh5L HLyAjWevKFmNadMMcSwHZNLDUQV1N9booEqo8MvioDj3ZJKMq9IZyabGbJ0WMyPBqQgm Ndb8oTdCMsRGUNcJUr1SK58UgMx83tEDBEMR1GioAqkKdKohe6udX+oevMaOjjcUGVa3 5Xy9Y380iR+CmGfNg8fw4g0bLzTJMfh7XXV3UstNdayVFPGIA/iseewKx+I+Wi0wpvLC ZShQ== X-Forwarded-Encrypted: i=1; AJvYcCUnieisyrcsG97yvBniI99mw+AHuNSS3odjwd+o28J9siElgfFcyLTbjpeh//90NDzBrG5JwbYwKg==@kvack.org X-Gm-Message-State: AOJu0YwLPQARX9YZvEptBlyBa34u2APHa8Yk6gYJlck/t6NVxjN4+ksr kNMhgp3V/ytDHbj8JOwkrH1Po/DCtymSVXn9zvdHrX7CinvwgfMYqXy0H1iJNmddjAHkXrQQa2X NRL6StSrlw4fzwQ== X-Google-Smtp-Source: AGHT+IEtMdEbBKfTi44I0hQxKs9SwxHv8NrQJqJsf3ce8ASa6YkaH62l4mchnBzjtU1EUoOkexS5OiFPJNiqBPU= X-Received: from wrbeh6.prod.google.com ([2002:a05:6000:4106:b0:382:30eb:98f7]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:491a:b0:385:bc42:52e4 with SMTP id ffacd0b85a97d-385c6eba8bemr11919119f8f.24.1732897980729; Fri, 29 Nov 2024 08:33:00 -0800 (PST) Date: Fri, 29 Nov 2024 16:32:37 +0000 In-Reply-To: <20241129-vma-v10-0-4dfff05ba927@google.com> Mime-Version: 1.0 References: <20241129-vma-v10-0-4dfff05ba927@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3946; i=aliceryhl@google.com; h=from:subject:message-id; bh=Vj+fM1HEWgk9zeZ3nCQftraSe6z/pjvOnZ1xL1atMIA=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnSeyve8NrjNvn5/AdaiW267/FN/DvoteK61Oo4 9jv/idgWVyJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ0nsrwAKCRAEWL7uWMY5 RpxAEACz7Q3Z15K9yn6UUnABmC93QguEXyoGn4fILedLcdfZiuIXMtFqpbgt30kuS+yGQJL41+B 1GNgFO57jYNMEHUZv87T8ItmFDJ2mKpQ7kteUR7+MXAXcpwOYI80gz695ogwkssoeMvfrllsf8c N+HTsQlzoRBWcwek+nZnh0RG5ZSR3KEsKtLBZ49/gqJi5p+7SNJnuvR5k6EJC75DGtQw7vYbmxe p+5Hqd2L/mzOLVGo3cBp53uEcawKUUU1NBCBlOI3pPhOssdZb2CETukZ5ZUbOQcACHFLeyOyt1Z MI/FAqyGKDlSINeXZsUn1izy/oxYKx+6bH/ldZ4G9/2+ffHrhgx0Z4vKQ2MCqrONvgsRM3t5U/h 13ldm0DP7VL6NEMeZ1EFquTl9VOWVd48ThVZgEOgK+BMN41A0S4mu2SkL+lOu7hnX+WtI8NWEJS DQ+5Nr1CLoaGxIEDMWfAs2ojsL6eW5NXDoVpmX+EG3dX69N95evSpj4kGm+Op6PKwMSTD8vbmtf 5h4v8zcXVcN5EFehLmSy+UqhnFASNEFy6uSDCZiEN2TZ6cgwWdNkhV+bu7OtvnwUViqGSts+rG2 44LoVmhzRJZhOZbXtPq+CGaRyvxebiBx35ZsGXGC4nQVfZz4wjyrRIfU3H2pHHhAYuG8JkqrETk HrhIta1iCfIzR/Q== X-Mailer: b4 0.13.0 Message-ID: <20241129-vma-v10-4-4dfff05ba927@google.com> Subject: [PATCH v10 4/8] mm: rust: add lock_vma_under_rcu From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 6279E40005 X-Stat-Signature: jp6nzrh38di3147ri5iae7e9dtdroap6 X-Rspam-User: X-HE-Tag: 1732897973-339528 X-HE-Meta: U2FsdGVkX1+k4aX3aeOouWWG5e6zSIvb+3AYYHEQ4OUGANZgFtlUUaRSxkSNig3NfohSH52wZMjLDtR+mboQjE7xTB5kfKmAlP2ZirDvFYpLGhf35iKeELMzCZGj0/fUpvvmr5vLqYNiLMrpps/vdgchsMuMEtCLs3o/4lMORnyeZAoWsyVToTuhfsMcmeXLBhpAzHtn6HSobglt1kR0TrmXbtRFFjyRp5hvHhWDLVvufZyCbHs/to+F+PlVKysP3sohfYYtLWPDWYm8fRcpg69fHpMSUJ+fsqZ91JFHjCgGptyavIHnVWUxMpw9ExcTc79APmzg/UvzsRo5mQ6w2LfjjxPtXtspl1OVnS2dn+rPxoWdRvzmzRVNqnSxWUXa43Dl/SIY2o1dovJG2uThVDMakS9N6DgufPK0wrtLHN7Z7PsV5SYQVvNlZDdufOM+XPzOgJKiqyHTIuT0gpQj8ntUWcAf4A+GcYbcJg/oZ0iD+eXhavmls30px5VMjdoCtn2Tnr1PC/9nbWtZAahDmHhth2FAizJ93Mbb8bwCnAmCrg3h8uJUYrSGOyQl5tZMorYl5DKXcO4DPlhrMO4U9D70B1JpskVTi0o/S+QqLUuQ9PMNjBtxzIxa31X1MGRsYiNaawQZ/3uF2iddjAuA3i3DZ90jN9A3HA0y1dJQDWfGfXYPDReKSeDbEGgtwoGczYNoYUf+PD2Qt5BAOcoyLMm8YJOEd8QoUcaNABJ4/Ed/JY2NHENORFkAcWbaTBg+nX/obmRleBPMgkr2VMQ5MnzEJj7g2/tB7YrcN66Iz4AqjJSvmM0w/85/7DrE16/vaSTL1VOOl8U55lhCXZhqq1MDIFZhCTqO3db5hp6p6Tiz4iZFa8ngvHv3iv1zKP4Ia1erklKhMs2F+54iZLWBV5moUijnvYXwyor1Kih4qwQzlVKd71dbl0JdHwGepoUlyjZVIOU7IZMrCi92bMa dgQ42gYL QY7Q4Eh4WPe65l7DfImj8jHdQ37sm1F9nkjX9Wb8SCPIkFdHPRF4UmfZTSJgcA2MnkRw29vDIblzVVMGecgcstRyOJfuA3eqE3T+n4XbxlF5/h+KeQNMXKNkBptJ3qPQZrY5rTdl2JGw8ezJxoUhEUToyT0R9518UcLcJ2QGDa/h8w08G/+ZY570d6rMWJFUA1B5GsBm3bsRYgR7NNRlaZtbhGtxG/kvzRin1qsxycH/p7qpnEFMrSZUet1IfshxglSrR8hU53k3MnRuzTESNvsY+NLfP5m5GiSnH7z8Lrsb//rEx9aUX6MUmF4c1RzpNQLLOYf+QPjFlqSwwj+LbxIditUG8dq3rNxJi/3MzK6rqCf3Ho/e1pLOhx64cq+QYarry0r2q4e7NM5oFjd1OBHOPcPPiWm22rie6VPelbaemhrhYMZTC7m21IHk9LLAqb19Rvqp1iED0q2lsbjp7l8TaNJUZrQLW5kb68ptllT4AVSAs6AbHbIdhD6R7fwGXisD0j71a+U9xkbhhbtJeR/cQxXFeBkZrxeU8ODw3lRh8XV2j1S4H36HoDBDy0BjZENvC7lGzrc/2t7+8E2553WIzJyS64XYz+Gl1mitEWG/sZDvi/BzrYWF+syiDMjpk6Tt5Bl/hUt+qtRh4SPMTiWmcamfKZLvRBUMai2Kt7cNvrPeYtekv5qB44tF0H6qjOvpfY6zuzQD8NMeEPKTe+QNy4A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the binder driver always uses the mmap lock to make changes to its vma. Because the mmap lock is global to the process, this can involve significant contention. However, the kernel has a feature called per-vma locks, which can significantly reduce contention. For example, you can take a vma lock in parallel with an mmap write lock. This is important because contention on the mmap lock has been a long-term recurring challenge for the Binder driver. This patch introduces support for using `lock_vma_under_rcu` from Rust. The Rust Binder driver will be able to use this to reduce contention on the mmap lock. Acked-by: Lorenzo Stoakes (for mm bits) Reviewed-by: Jann Horn Signed-off-by: Alice Ryhl --- rust/helpers/mm.c | 5 +++++ rust/kernel/mm.rs | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/rust/helpers/mm.c b/rust/helpers/mm.c index 7b72eb065a3e..81b510c96fd2 100644 --- a/rust/helpers/mm.c +++ b/rust/helpers/mm.c @@ -43,3 +43,8 @@ struct vm_area_struct *rust_helper_vma_lookup(struct mm_struct *mm, { return vma_lookup(mm, addr); } + +void rust_helper_vma_end_read(struct vm_area_struct *vma) +{ + vma_end_read(vma); +} diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs index ace8e7d57afe..425b73a9dfe6 100644 --- a/rust/kernel/mm.rs +++ b/rust/kernel/mm.rs @@ -13,6 +13,7 @@ use core::{ops::Deref, ptr::NonNull}; pub mod virt; +use virt::VmAreaRef; /// A wrapper for the kernel's `struct mm_struct`. /// @@ -170,6 +171,32 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser { unsafe { &*ptr.cast() } } + /// Attempt to access a vma using the vma read lock. + /// + /// This is an optimistic trylock operation, so it may fail if there is contention. In that + /// case, you should fall back to taking the mmap read lock. + /// + /// When per-vma locks are disabled, this always returns `None`. + #[inline] + pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option> { + #[cfg(CONFIG_PER_VMA_LOCK)] + { + // SAFETY: Calling `bindings::lock_vma_under_rcu` is always okay given an mm where + // `mm_users` is non-zero. + let vma = unsafe { bindings::lock_vma_under_rcu(self.as_raw(), vma_addr as _) }; + if !vma.is_null() { + return Some(VmaReadGuard { + // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a + // valid vma. The vma is stable for as long as the vma read lock is held. + vma: unsafe { VmAreaRef::from_raw(vma) }, + _nts: NotThreadSafe, + }); + } + } + + None + } + /// Lock the mmap read lock. #[inline] pub fn mmap_read_lock(&self) -> MmapReadGuard<'_> { @@ -238,3 +265,32 @@ fn drop(&mut self) { unsafe { bindings::mmap_read_unlock(self.mm.as_raw()) }; } } + +/// A guard for the vma read lock. +/// +/// # Invariants +/// +/// This `VmaReadGuard` guard owns the vma read lock. +pub struct VmaReadGuard<'a> { + vma: &'a VmAreaRef, + // `vma_end_read` must be called on the same thread as where the lock was taken + _nts: NotThreadSafe, +} + +// Make all `VmAreaRef` methods available on `VmaReadGuard`. +impl Deref for VmaReadGuard<'_> { + type Target = VmAreaRef; + + #[inline] + fn deref(&self) -> &VmAreaRef { + self.vma + } +} + +impl Drop for VmaReadGuard<'_> { + #[inline] + fn drop(&mut self) { + // SAFETY: We hold the read lock by the type invariants. + unsafe { bindings::vma_end_read(self.vma.as_ptr()) }; + } +} From patchwork Fri Nov 29 16:32:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13888849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9064AD729EC for ; Fri, 29 Nov 2024 16:33:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 237B06B00A0; Fri, 29 Nov 2024 11:33:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E9316B00A1; Fri, 29 Nov 2024 11:33:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0138B6B00A2; Fri, 29 Nov 2024 11:33:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D5D0C6B00A0 for ; Fri, 29 Nov 2024 11:33:17 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8586181289 for ; Fri, 29 Nov 2024 16:33:17 +0000 (UTC) X-FDA: 82839677466.05.B98F5E1 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf15.hostedemail.com (Postfix) with ESMTP id 936E0A003E for ; Fri, 29 Nov 2024 16:32:55 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="K+d/E8cJ"; spf=pass (imf15.hostedemail.com: domain of 3vuxJZwkKCAEbmjdfszimhpphmf.dpnmjovy-nnlwbdl.psh@flex--aliceryhl.bounces.google.com designates 209.85.221.73 as permitted sender) smtp.mailfrom=3vuxJZwkKCAEbmjdfszimhpphmf.dpnmjovy-nnlwbdl.psh@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732897974; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NxiAaOQixinTmMpuFlr9Z7Cs/OKNMeW1CE1Bg5Rf1Zc=; b=vG6IHsmz0Mp9hqpg337NfkgR/WFHpUZCU+WZs6vGOo/QLa3/q9JAEE9i0fTiYbm72Wnt+5 6k2ezi2pCPRHoD1uZ/YYxE/8hH84GewEgaADNZRa5xObzeGFQVeHR5z+CUz20G669dloX6 EAK13XSopl6wcCyOQQPC9LsT+kNVQ2c= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="K+d/E8cJ"; spf=pass (imf15.hostedemail.com: domain of 3vuxJZwkKCAEbmjdfszimhpphmf.dpnmjovy-nnlwbdl.psh@flex--aliceryhl.bounces.google.com designates 209.85.221.73 as permitted sender) smtp.mailfrom=3vuxJZwkKCAEbmjdfszimhpphmf.dpnmjovy-nnlwbdl.psh@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732897974; a=rsa-sha256; cv=none; b=05ebRvNq0RUcaOHYjQr9v0dL5Rd3RvpnY2yZ1yZE5Zk5tMBhy9fKRvLg2Ri/BZXVk5WC7E Jed7O+AvECXmgOyH1OkiZ1pcxPuSVdgXK2NE8lDwBVuo02yMpBkxDM7c54HmKp3fZT/vXn mTf1s+nm7tHSftEi0eRNeApy+EL6ORk= Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-381d0582ad3so1370672f8f.0 for ; Fri, 29 Nov 2024 08:33:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732897983; x=1733502783; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NxiAaOQixinTmMpuFlr9Z7Cs/OKNMeW1CE1Bg5Rf1Zc=; b=K+d/E8cJKqv7LpAxKU2Yw5SoGodnQsaoEJe5Kk/FlNMai0AqSYH+z8s38LMZz1y59J 0J5Jz1ok4BNjwperNHpP2DtBx5hi5jmqo7M9QEq1yo5DkXSCpZM1mUXy0dEgZTWZ5Rz5 e+hA7vhoXsl1iERT03+k3CBev2ZP8mPyLwvpF+c8uXoJyCaz+LztgNOpO9m2F5qHO3s7 FJXi8ffxJ10jWR7qtys0dZ5k+PmZo1MBKFo529L9fyS5mLXH0uEPQ3rahqnYH4oRZRAv NJe74TgerZORvrS3FQq7kZtHS/5U1iuwDs/0/vXREds2XK5ET/c3WUAEvoOFYmZciEgr 29qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732897983; x=1733502783; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NxiAaOQixinTmMpuFlr9Z7Cs/OKNMeW1CE1Bg5Rf1Zc=; b=OHvVo1HEZcDsd5y1VYiCICXMU2a9kWSxQIvkm0h6J647I9V8LO06+JHwZaitT55Lq7 sLvZEscD7AqI0RKPrtpWGLhBwFM2GFaZVOPa22s2uiXwbMXkFC+OxwjMuHMQMmvHSSeB 3lOi727GcYfGdA/8jAtCaudp+cjGbVzHygPbPlNH98vgsH+doO49XvVbVBwsOqLlhaFD 42mPCOgPsfXn1IDK2UaeW92FYDRyLXRw1DMn9V5/hJC+3+nYupsjjGhC068yxCyd1J69 mjNa3pzHZvaRymVJ4+3KW1ejQK2zF6sxrmE7yMGCAdFpTRRjGeV6NcoU1+ZueEDawVzI 7GzA== X-Forwarded-Encrypted: i=1; AJvYcCX8uJRCirLWYUroYmXf+ESz2SaZSHzyqi5D1ZXZgk3Fb8Qz0L3yCxnf87URUuZBddGNTG0ljG91WA==@kvack.org X-Gm-Message-State: AOJu0Ywv1HNkwiEWy1ZlgLhW8cwtLhhJxyUInHufRKHF9OYY9uk9eXn8 zYDRYj5US85dLO19Z56qLAvGunYyB1R0Dde7Ig4m2d/rJdt44OHQ1vgz9LoQuCrKD2YduVw2dku ycwfDpr8RRvd5rA== X-Google-Smtp-Source: AGHT+IGYb3T/wZgxg6wvfixNQvKu7OW4dXOoNjGb+FLVFT5jRj0xJ7uK1bZMJYsE5nPmUYdm2iSLsb9tE8N9Qhw= X-Received: from wmbjw2.prod.google.com ([2002:a05:600c:5742:b0:434:a4bc:534f]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:400b:b0:382:486f:1360 with SMTP id ffacd0b85a97d-385c6ed9824mr13201148f8f.44.1732897982795; Fri, 29 Nov 2024 08:33:02 -0800 (PST) Date: Fri, 29 Nov 2024 16:32:38 +0000 In-Reply-To: <20241129-vma-v10-0-4dfff05ba927@google.com> Mime-Version: 1.0 References: <20241129-vma-v10-0-4dfff05ba927@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3090; i=aliceryhl@google.com; h=from:subject:message-id; bh=rBEGLc7KJG1nI+I3PCkS6QqEUHC7WTE77ZFEyztUDIc=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnSeywnZuEoxDobqs4lWzeHdYCGEpayHKpr1Fk9 ltPcXjwVWmJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ0nssAAKCRAEWL7uWMY5 Rn6LD/9y8AHiDmndnB+qvZq4H5F9U2hmnc+pviQiE0r46f7JiYFx/m2fKqG1YdHnt/8lz0N/DOU 3Z11N+ebn5UZJO4QiuZExGBwoDzEonp95mZYKWwSEM5lYWr/tIPZPlrN4jY2tULm1UvEFZ6S/R5 otdlUsOhvEE3MOTA7EjE9pTiZ2vMp+i/hjeTCBqWzywDv3hAML9oTk/16e00eFdrY0mtCvLQxk0 2alwCGptjDvpQOvmUvELwBFKJtlVeTdIZ7RO617X6WtZaKBDvAKXnAxTyCaMPfFyfY6qjVQJgQ+ kbCteUJ0B9ryxuzxddA4Te9UE/wGptVhi8MF39q/huixQByiX9OPDdXnHo4vOoBJA3kahq1AgLl /oH4XHgOxJCX52Cqr9BaqgW7xTfMru7F3cijFzKaoeOzFZwuIamuH0x8a/OdpXML//YwQ80AY73 uz42oob1f8bbvTTPjmtA+wLBZW9ieYSDvbdWno+UOkAR3c8TFWnT/k9kluifCzxlwRN+gJGNkrP 4Ibj7905Fxe4hcKeErmMbzpalPlgTm0K7ERQN6XrD1aasl+GImueaArIqjEKrggKMJxZzl1boRd /CVUyTwmyJOHFEPjEahnBOGSWpVnqa2V18wURAduN8jAF8xde9A7xARVv9ZLP1pmo8kgeukX4f6 sVuweJAEgw6IiCg== X-Mailer: b4 0.13.0 Message-ID: <20241129-vma-v10-5-4dfff05ba927@google.com> Subject: [PATCH v10 5/8] mm: rust: add mmput_async support From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 936E0A003E X-Rspam-User: X-Stat-Signature: fkyygo6qj51a18kyqttzdmcq4d4p7tuh X-HE-Tag: 1732897975-344734 X-HE-Meta: U2FsdGVkX18jIMohxca0ggyGy9QlovjU+734pAmGqpTcqAEXzO7kBAINhkNSXdk+jsLTF8AsybxV3Qq+D0+xkdTKNvwuIpl9bi18DKcGfVj7VK9GcyvYZNotX4btjyiXUAvkdQaaerttmKwq/uTm2pJrxi4EXqyJf8zsnD+QL+6xNqfr4f0QQ0BhLKfpJsldeIVfvwDUmtStKPCKBraRs7FPw6PUwnHQTafNdu+PmFaGa+5MhoY4l+saeDCjQN1hdzMNW1f952l21L3g/nDWZGD6eRNwCv67Ewz9XFkuttVu2UOP+YiQ1H1A7QLspsbMUoXDOQ/rfbIZ2uugL1GM6SrFSnSNiZEKSAV+KwczxuMdkbzYV4WyRqtKfp1DlcD28xE4iq4wREpEydf0ZFYahRQqNHm95/JIemJVPSs48UpmDPp88B8LX3WighKmTIBX7tlE4Qf59cuZf8Ws9gGvCeww5914mr40zrwjuiulaGUqNBzpchr/hjm+4SGz5n3N1xswY1Zcm7aAr86EOpxZ+VRQykPaMNXsBg2BzlzSE3vM1YfaqkzoW37LTM5OKJX5fLdysctaO4NVOk4zuNCRcX5eeJUb1Z3tSVDkXiB/GbZUjfEDQeX/+REmIbxgXzA7iZ9CBkjqW2Rrb1MmfvpTzh6rR98lMf1y3AL6b+z7Fyujew+yf3oZ0g1+7MVHoCKDzBI2+kLJRs415ftq9TB9qbtGG7YvZdoPY0DSte+HQCU9dr/DIlE4fRr2uvmS+i3oBnyoK3lOVh4MVnSWtrgbzSBNdwzTARV2x7ugV+H8HlCUA7Ps5LZfhG0EDz/DNgwqVUhlg+ioqMtB6J8dMtwAQjI/imkk+aOepjoGoo7ShFY9LgqzdkS1kl6mp/E4CDXj2af+yT4mK+7WhCW0xlFtldOQKUSDIy/Sfy94lkHZyE6635mJynktUwCcJmLZbicH8tVPl3k80GeQN2vKBkF mz32xEcR mt1IRAitJUki6vpp73+rB2Yk/0kSCZXC8gnsi/vrZ4TQZi3u2a9uhnWUNH+FtIGXZFrU8/KGPeXiHFQFBFWFbeZvSdc/mnJwyQOLnktPGzO7MSB4P4H7Tg/avBfFjouI+jKSV9r6HaZFrGs42ACDqjHfjV46akgsACQpNKiuB1I7xNYmLKcGAMDdwBJT2Lp6DIQqhD6t6/YnY2YuULkDPsds7wXEh4ZKDHOPTfymQM3RG5xobmjHS5l5w1lw8WSeOY0pny8kX3nEiVMvPN2Du0xx48+O9ohxbydC+D+omx/JI0hDaUKtszuEdbux2LGKbXi6JPjWVj7OiQ0lJWQjN4O5TElbAbn5VXsc7tMK903yRrZWqbOYzo7W+lj6HmDbMpE2EWAWfA6ePWaUdOx2PoYBHJzn2FCCKGDvFIUpcj9Wb9XK339tSKTW/kEwCQftBH09HwA3oUbjTgQsoi9YE9LX53h/qY2vp7uriYJ0/2PP6TzxNhiONJeax3/qce9Fh2eOjwcH3hqz8FUmm2PcbRpZf8qjNzWYTyeiTfAb9Q3QsLQVHjPZWYmoFaDD1TV0osPQ72bcGXWueQGwyTZRtxMG/Ey9VWCD07C84XTnJTl5hC4QTs5aHSdWGhvyy8CyXueKh+JKlNtU+bayxKJ3cPHjWf7Z/NBYBnvgSReyFITUsW4lcTWI7aBKpGO+C3Okp2Dw84CvBKpoWRPg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Adds an MmWithUserAsync type that uses mmput_async when dropped but is otherwise identical to MmWithUser. This has to be done using a separate type because the thing we are changing is the destructor. Rust Binder needs this to avoid a certain deadlock. See commit 9a9ab0d96362 ("binder: fix race between mmput() and do_exit()") for details. It's also needed in the shrinker to avoid cleaning up the mm in the shrinker's context. Acked-by: Lorenzo Stoakes (for mm bits) Signed-off-by: Alice Ryhl --- rust/kernel/mm.rs | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs index 425b73a9dfe6..50f4861ae4b9 100644 --- a/rust/kernel/mm.rs +++ b/rust/kernel/mm.rs @@ -98,6 +98,48 @@ fn deref(&self) -> &Mm { } } +/// A wrapper for the kernel's `struct mm_struct`. +/// +/// This type is identical to `MmWithUser` except that it uses `mmput_async` when dropping a +/// refcount. This means that the destructor of `ARef` is safe to call in atomic +/// context. +/// +/// # Invariants +/// +/// Values of this type are always refcounted using `mmget`. The value of `mm_users` is non-zero. +#[repr(transparent)] +pub struct MmWithUserAsync { + mm: MmWithUser, +} + +// SAFETY: It is safe to call `mmput_async` on another thread than where `mmget` was called. +unsafe impl Send for MmWithUserAsync {} +// SAFETY: All methods on `MmWithUserAsync` can be called in parallel from several threads. +unsafe impl Sync for MmWithUserAsync {} + +// SAFETY: By the type invariants, this type is always refcounted. +unsafe impl AlwaysRefCounted for MmWithUserAsync { + fn inc_ref(&self) { + // SAFETY: The pointer is valid since self is a reference. + unsafe { bindings::mmget(self.as_raw()) }; + } + + unsafe fn dec_ref(obj: NonNull) { + // SAFETY: The caller is giving up their refcount. + unsafe { bindings::mmput_async(obj.cast().as_ptr()) }; + } +} + +// Make all `MmWithUser` methods available on `MmWithUserAsync`. +impl Deref for MmWithUserAsync { + type Target = MmWithUser; + + #[inline] + fn deref(&self) -> &MmWithUser { + &self.mm + } +} + // These methods are safe to call even if `mm_users` is zero. impl Mm { /// Call `mmgrab` on `current.mm`. @@ -171,6 +213,13 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser { unsafe { &*ptr.cast() } } + /// Use `mmput_async` when dropping this refcount. + #[inline] + pub fn into_mmput_async(me: ARef) -> ARef { + // SAFETY: The layouts and invariants are compatible. + unsafe { ARef::from_raw(ARef::into_raw(me).cast()) } + } + /// Attempt to access a vma using the vma read lock. /// /// This is an optimistic trylock operation, so it may fail if there is contention. In that From patchwork Fri Nov 29 16:32:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13888846 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2EDBD729E7 for ; Fri, 29 Nov 2024 16:33:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BAF176B009A; Fri, 29 Nov 2024 11:33:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B0F7F6B009B; Fri, 29 Nov 2024 11:33:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93D2A6B009C; Fri, 29 Nov 2024 11:33:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6F5866B009A for ; Fri, 29 Nov 2024 11:33:08 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 006CF1A12B4 for ; Fri, 29 Nov 2024 16:33:07 +0000 (UTC) X-FDA: 82839677004.24.449997F Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf21.hostedemail.com (Postfix) with ESMTP id 2C4401C0012 for ; Fri, 29 Nov 2024 16:32:49 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="RpWjbO1/"; spf=pass (imf21.hostedemail.com: domain of 3wOxJZwkKCAMdolfhu1kojrrjoh.frpolqx0-ppnydfn.ruj@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3wOxJZwkKCAMdolfhu1kojrrjoh.frpolqx0-ppnydfn.ruj@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732897982; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FQmeIT5gOsUotOB7o6QXCtotwfAnfjr1q7Rp4w0i4JU=; b=mzHlnpBEMljGzLO3Y8iXj1i3qbsDxzQpczVauHHYEPhXPpg2ZH/SwUJ1J30uq/gfDaOV5a kod5Q8xulZDufo87b4T2hoEvGfgroRudO1lSb++/2NOjsJyDkfF1qzB0nFUTdmjUeKehft JKRf6aWJ2R+1YeCEVN6WK/fSGtE7Khg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732897982; a=rsa-sha256; cv=none; b=f5PpZlv6L8RddAxrOMheEM9zoFBquMbeirthnV4lI+tdNHH7LSqM6bLviAfL91LPbCsPti d/H+z+q+MrUK5GFdIDQJapRuB4PoxmhbDhaHKUJs2S3lGYK5CZyGim6WEPvkjsGYo5Z6L1 iVDBhOApge5fXHufZ2PXB+0u5pxpoO8= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="RpWjbO1/"; spf=pass (imf21.hostedemail.com: domain of 3wOxJZwkKCAMdolfhu1kojrrjoh.frpolqx0-ppnydfn.ruj@flex--aliceryhl.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3wOxJZwkKCAMdolfhu1kojrrjoh.frpolqx0-ppnydfn.ruj@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4349c5e58a3so17384715e9.2 for ; Fri, 29 Nov 2024 08:33:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732897985; x=1733502785; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FQmeIT5gOsUotOB7o6QXCtotwfAnfjr1q7Rp4w0i4JU=; b=RpWjbO1/7KfnVcYDeaaMdsJKtjK24gWxp91VaC6iIm5cEFjYV1S6xdmJ5orcANMuBp Rf5fWxNVMn6iHLuAAwevEhzpgrtXPnmwe9ba6S47bVwtNNhv3+dQTGqqOjRB9SMypO9j KoQlKjI3Py5nU/4mn8FX0VVQ6w1o1j0HXhbxLafKzeTfUnXBux4PSUs1afGOD/dh/u3M Ps4P3EfoDAszWUAQEiddRNY/A6bVWf5uEFJ9rRfL39ixYnUkGc6UPcu2OwVAT2w0Cyaq JyaJxtmCM37H6CgHdyRY40hmA1kGFVbhv7o4DBM8rh97GVYoRbWrM6IXSwEB74JsSnfN 7jGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732897985; x=1733502785; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FQmeIT5gOsUotOB7o6QXCtotwfAnfjr1q7Rp4w0i4JU=; b=xTyLEOaXcfMdnAA9M6D6nDJe/mznxKGwG+O/p82BKiSqHdWxPSncCKYYnD7egdlF1N PBm+kld7lB5I4jVL2fhweXeJ07FsqVirr7Vr/AWsidft5/mB9QUpSNr2qq8Qbu98lY0M ysT2HSGBKCgLfwAb0LTdwff77NVsXChX1tPenL66KasvSGbXE/khh8BnqdBF97UPKgbW lj9JP98dtdFq/GEmehQdznwSXfK0+VMxO6o3NCPkyNmFGX39a9r3Y/jqAoiI23cEM32g ywrPF/DS1CfFW07n9F3TpLiNVgrfbW7OE7Al4E2FXQugWFlqD1TMVK8rTnBxiJU1NNYq 3h2w== X-Forwarded-Encrypted: i=1; AJvYcCXQEtdfUVJ0uh4wXWoBszkXgugo1BmbjBCwcqXT9PXwuFy0AJAf1iDIISfvx1TMG1MTZAXTghKu/Q==@kvack.org X-Gm-Message-State: AOJu0Yx9mNhkuO9Bd6I2fGwmJf5gI8i8mxXlLoD/8NOTPzJ362oaU7eD Eegnk09TSQVjTKVCX1vIdk+Fr6YoXksu6dZCkk/hJhTiPu7XPmlv5uxCAvjCjSXCt+DFXY2PrI2 uaP6Xq5DoItGhBQ== X-Google-Smtp-Source: AGHT+IFsZCkak00uYhITnMwKRsDF3s5UKpxdm1Ck5P9OOToSqry0Xi4dvkplER8/UfrYyNysD4wuKqkPHmyCm60= X-Received: from wmbjx1.prod.google.com ([2002:a05:600c:5781:b0:42c:b2f6:e6dd]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:19cc:b0:42c:bb10:7292 with SMTP id 5b1f17b1804b1-434a9dbaee2mr101296295e9.1.1732897984863; Fri, 29 Nov 2024 08:33:04 -0800 (PST) Date: Fri, 29 Nov 2024 16:32:39 +0000 In-Reply-To: <20241129-vma-v10-0-4dfff05ba927@google.com> Mime-Version: 1.0 References: <20241129-vma-v10-0-4dfff05ba927@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=8218; i=aliceryhl@google.com; h=from:subject:message-id; bh=gC7RalGfFyvdhxyetp0qLjWhd8M+19FgWoTk8eiPyqo=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnSeyw5vs6i/9T0252bYqbEn7nkN41XOkM+MXze 3wvucuoTyGJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ0nssAAKCRAEWL7uWMY5 RgTfEACfVuMxx1SnP/BL8wCAsPuxX5JGhXxnfaLz7ZZzUax2XN12bzlMqd5YfYKXzKdwcoYnEwV 94lofBlwvf6N8t5Sz959Mm2XikEzfX1OL3r9waiomQSC9WWgZcMbzlQSTFz64/ZuGS4bvAeRqA/ w4o+9qWe1CE89aDPwR6zabLo4hkqThpqUaSw2ubox3Tw29PgjTJFY6ka5juhsobw1jIEfrFROBl raa4dfyZER0pRuJ1XdHUybuXA/9fi+nPi5/Is8WVM/WYi8CTgPvZvfeyo7cUXZhqS6asQgNcGHx WoXQlaMbC80sDbMtIJp5sRe5Leub6xL0sDvG1mdsi9tUqJE4t/tc9c6BT4cj7UVRgqMkUI/nDUe yFZiNPPdWYxuAQV/uJxvU5ZcX9hyMErXL3I1i9KfGCeHEUAW2KYxFO4POdMd9SP8/P1GLrl70zk LOjgMq4yUyeO8bFwU76m0th2qEkPUF4SawjKMSR8PrenLElrz0JU7e1Dn4V5JWOX16FxM+DBt7f P13GF+rp0/nalQifLMrem7cwLe3VIwclC5uzajUOW6fFdfpbddG1Ae/FfgjrOp4M3a8W7sKFp49 8Vt+43lDn8Fe2EbwTiBlJqyMTRS5OYKfJIcFVvuybRRzWEGSiJQxlukqy2/sZHcTrI5MKdP3tnE gk+7FY2H7FGOs0w== X-Mailer: b4 0.13.0 Message-ID: <20241129-vma-v10-6-4dfff05ba927@google.com> Subject: [PATCH v10 6/8] mm: rust: add VmAreaNew for f_ops->mmap() From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 2C4401C0012 X-Stat-Signature: iw4unggrk7b7hn8iw6quozueki6ucd6j X-Rspam-User: X-HE-Tag: 1732897969-969154 X-HE-Meta: U2FsdGVkX1837UUzfNr7WmwT7VO+EVSP7zskwbCaEe8c7LVoMgwansydF7ny0vqXdU+1EPc/x+35D65vfb8Va5SB6DRyQQYxKnTPi1DFo0QIhTY11tgPH/4Dn/gtK9MZIDZt/38xGD2wlv0Mj8EardMrtbbNvF0JD6Q5SNdm7SGYkrrCmbavN+0X2MPOASjiiRCA3ESG0ih2rbAHKr6tdBhXEjwEU32U8hpnXE8/NaN6X0v/D0tYSI05mxyVhxh0EAY+/1PIXFMJ2eH996T9RSVO0Rn3bCkTZ5T93kp5BdfojqTvfZe+azd71V0JnBo2DeFkF8WgKqWXMh6yjLxWlgWiPzNmcAZm9p03LjXFI57ZpuudDap+DUlZmlXq/cQFcjQNGfy3uAMskAdBoBjxhPLPL5SrBjJ0Q4C8lATVYJkb5vy2U3ykHjyH7Y7fJWtcr0rgRnldu5PzbA/D3ymihQmCjPWU0cu2vJ6iSqq3mTLqL+/7aReuoyDe6gAsUi5Vm826WzgDo0IMIFOhelvZ3GN2t8wEiRzYwsmq4VAeK0BqyNwQWea4OS4YAcyuy01oyoaSB6lWUeb3xzNbi7Q0IoICFxOGGf14q87WBTOCAcYtuUzDt2Dx72WeBPFog//LuP2Fhr6CVAmaghBE4ddLidi9xGHKoM5Cl2lcWHT/jkra8kKnsvX4wpcGySfabGh4RbUdYA4W63Yva0J4MPx9s4qrEcBMBmfYcWiSK0XdjV9YMY82X/4G9J1mFoEVi9SKXGj8NcA8rmrnSaN4/eA/iyycvZ5QuotTw3p42R3R6We6xoS3i/iQKH1g4OFZp6u3CM9IttUKeH4J48/Fj093PszbrMDTzUJnyAOcbMVAL9yEuAeRnTQdnXg2p42ZYaWp03ONFTb7GGvfpTrLQs5pTHEjygjsfVxqmAQqsNs7/Cgpa9uwNq4WLN585uRzhTiKutghSe6x9GOJsPGaVm4 UkrR+Vsu LAsV/BrydiExIp2PAPE9gRpjsU9bJFbC/um8yHAvrGKQHAvVELosReOuTPAC+XW65ij1hIv8KVV42BGefD7uLdWwOszlXL4B4WrddRJBMmUPCHtsF4Ed3Ph3k0l+Hg9i+JrfG+Kdoo19j6VLVpDVbNvmvuc3wd2nDSZkoFOeTD2onaQIsnQfRh+5zN0DdJ6PhKTuQPzDrGT5mHPZvsa0bwEO/1S7kuc4SJZ6m3+O6NsGevUW0T4gQtWmiPrVvzWKRL9x+ztqpn7EEqVOERxcuNu5ujxeCtYZy3Cwbh0ud9hjbvksdeFhtk+W8V/o4FhwUivrJSR0dldAwWaTljoEV5Thg+TOP+oFIADVXoMeIikqhTd5QZweL2ddEug3yEXYuYUlnE+wc8hkcNV1XCyXaXF/bqUOmXFJQzlyFwzov9D30Q6aJ4Tag2QO75hko0EWvCPkaTYNxgkW+7HkOMG1oY0vHuixVzabNWpPFukMr3Uhofcpt4zlmHsPfOaRdgJux3dB4g6jOCKhk0auJiPCdNLmHyk/6nCWe7GOyM+Hq9p3C1CoyZdy0LOgaO98j8guvf6lLNcpPmCI6PK7vKUsDtRItPDn+raQP+UtfUCVXCVJIc42aROvA+U6GUEdBP3l3m5qXtHAw9BCfMiIFCFjMyqo7CQ58Fxy/gwkl/WI3qJbknceRM7Ceb6xsXBfcAfaUuHXL4DfKTgOELkYQv+RTgrp3HA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This type will be used when setting up a new vma in an f_ops->mmap() hook. Using a separate type from VmAreaRef allows us to have a separate set of operations that you are only able to use during the mmap() hook. For example, the VM_MIXEDMAP flag must not be changed after the initial setup that happens during the f_ops->mmap() hook. To avoid setting invalid flag values, the methods for clearing VM_MAYWRITE and similar involve a check of VM_WRITE, and return an error if VM_WRITE is set. Trying to use `try_clear_maywrite` without checking the return value results in a compilation error because the `Result` type is marked #[must_use]. For now, there's only a method for VM_MIXEDMAP and not VM_PFNMAP. When we add a VM_PFNMAP method, we will need some way to prevent you from setting both VM_MIXEDMAP and VM_PFNMAP on the same vma. Acked-by: Lorenzo Stoakes (for mm bits) Reviewed-by: Jann Horn Signed-off-by: Alice Ryhl --- rust/kernel/mm/virt.rs | 181 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 180 insertions(+), 1 deletion(-) diff --git a/rust/kernel/mm/virt.rs b/rust/kernel/mm/virt.rs index fa1fd6aa703c..790b8cb91c27 100644 --- a/rust/kernel/mm/virt.rs +++ b/rust/kernel/mm/virt.rs @@ -6,7 +6,7 @@ use crate::{ bindings, - error::{to_result, Result}, + error::{code::EINVAL, to_result, Result}, page::Page, types::Opaque, }; @@ -161,6 +161,185 @@ pub fn vm_insert_page(&self, address: usize, page: &Page) -> Result { } } +/// A builder for setting up a vma in an `f_ops->mmap()` hook. +/// +/// # Invariants +/// +/// For the duration of 'a, the referenced vma must be undergoing initialization in an +/// `f_ops->mmap()` hook. +pub struct VmAreaNew { + vma: VmAreaRef, +} + +// Make all `VmAreaRef` methods available on `VmAreaNew`. +impl Deref for VmAreaNew { + type Target = VmAreaRef; + + #[inline] + fn deref(&self) -> &VmAreaRef { + &self.vma + } +} + +impl VmAreaNew { + /// Access a virtual memory area given a raw pointer. + /// + /// # Safety + /// + /// Callers must ensure that `vma` is undergoing initial vma setup for the duration of 'a. + #[inline] + pub unsafe fn from_raw<'a>(vma: *const bindings::vm_area_struct) -> &'a Self { + // SAFETY: The caller ensures that the invariants are satisfied for the duration of 'a. + unsafe { &*vma.cast() } + } + + /// Internal method for updating the vma flags. + /// + /// # Safety + /// + /// This must not be used to set the flags to an invalid value. + #[inline] + unsafe fn update_flags(&self, set: vm_flags_t, unset: vm_flags_t) { + let mut flags = self.flags(); + flags |= set; + flags &= !unset; + + // SAFETY: This is not a data race: the vma is undergoing initial setup, so it's not yet + // shared. Additionally, `VmAreaNew` is `!Sync`, so it cannot be used to write in parallel. + // The caller promises that this does not set the flags to an invalid value. + unsafe { (*self.as_ptr()).__bindgen_anon_2.__vm_flags = flags }; + } + + /// Set the `VM_MIXEDMAP` flag on this vma. + /// + /// This enables the vma to contain both `struct page` and pure PFN pages. Returns a reference + /// that can be used to call `vm_insert_page` on the vma. + #[inline] + pub fn set_mixedmap(&self) -> &VmAreaMixedMap { + // SAFETY: We don't yet provide a way to set VM_PFNMAP, so this cannot put the flags in an + // invalid state. + unsafe { self.update_flags(flags::MIXEDMAP, 0) }; + + // SAFETY: We just set `VM_MIXEDMAP` on the vma. + unsafe { VmAreaMixedMap::from_raw(self.vma.as_ptr()) } + } + + /// Set the `VM_IO` flag on this vma. + /// + /// This is used for memory mapped IO and similar. The flag tells other parts of the kernel to + /// avoid looking at the pages. For memory mapped IO this is useful as accesses to the pages + /// could have side effects. + #[inline] + pub fn set_io(&self) { + // SAFETY: Setting the VM_IO flag is always okay. + unsafe { self.update_flags(flags::IO, 0) }; + } + + /// Set the `VM_DONTEXPAND` flag on this vma. + /// + /// This prevents the vma from being expanded with `mremap()`. + #[inline] + pub fn set_dontexpand(&self) { + // SAFETY: Setting the VM_DONTEXPAND flag is always okay. + unsafe { self.update_flags(flags::DONTEXPAND, 0) }; + } + + /// Set the `VM_DONTCOPY` flag on this vma. + /// + /// This prevents the vma from being copied on fork. This option is only permanent if `VM_IO` + /// is set. + #[inline] + pub fn set_dontcopy(&self) { + // SAFETY: Setting the VM_DONTCOPY flag is always okay. + unsafe { self.update_flags(flags::DONTCOPY, 0) }; + } + + /// Set the `VM_DONTDUMP` flag on this vma. + /// + /// This prevents the vma from being included in core dumps. This option is only permanent if + /// `VM_IO` is set. + #[inline] + pub fn set_dontdump(&self) { + // SAFETY: Setting the VM_DONTDUMP flag is always okay. + unsafe { self.update_flags(flags::DONTDUMP, 0) }; + } + + /// Returns whether `VM_READ` is set. + /// + /// This flag indicates whether userspace is mapping this vma as readable. + #[inline] + pub fn get_read(&self) -> bool { + (self.flags() & flags::READ) != 0 + } + + /// Try to clear the `VM_MAYREAD` flag, failing if `VM_READ` is set. + /// + /// This flag indicates whether userspace is allowed to make this vma readable with + /// `mprotect()`. + /// + /// Note that this operation is irreversible. Once `VM_MAYREAD` has been cleared, it can never + /// be set again. + #[inline] + pub fn try_clear_mayread(&self) -> Result { + if self.get_read() { + return Err(EINVAL); + } + // SAFETY: Clearing `VM_MAYREAD` is okay when `VM_READ` is not set. + unsafe { self.update_flags(0, flags::MAYREAD) }; + Ok(()) + } + + /// Returns whether `VM_WRITE` is set. + /// + /// This flag indicates whether userspace is mapping this vma as writable. + #[inline] + pub fn get_write(&self) -> bool { + (self.flags() & flags::WRITE) != 0 + } + + /// Try to clear the `VM_MAYWRITE` flag, failing if `VM_WRITE` is set. + /// + /// This flag indicates whether userspace is allowed to make this vma writable with + /// `mprotect()`. + /// + /// Note that this operation is irreversible. Once `VM_MAYWRITE` has been cleared, it can never + /// be set again. + #[inline] + pub fn try_clear_maywrite(&self) -> Result { + if self.get_write() { + return Err(EINVAL); + } + // SAFETY: Clearing `VM_MAYWRITE` is okay when `VM_WRITE` is not set. + unsafe { self.update_flags(0, flags::MAYWRITE) }; + Ok(()) + } + + /// Returns whether `VM_EXEC` is set. + /// + /// This flag indicates whether userspace is mapping this vma as executable. + #[inline] + pub fn get_exec(&self) -> bool { + (self.flags() & flags::EXEC) != 0 + } + + /// Try to clear the `VM_MAYEXEC` flag, failing if `VM_EXEC` is set. + /// + /// This flag indicates whether userspace is allowed to make this vma executable with + /// `mprotect()`. + /// + /// Note that this operation is irreversible. Once `VM_MAYEXEC` has been cleared, it can never + /// be set again. + #[inline] + pub fn try_clear_mayexec(&self) -> Result { + if self.get_exec() { + return Err(EINVAL); + } + // SAFETY: Clearing `VM_MAYEXEC` is okay when `VM_EXEC` is not set. + unsafe { self.update_flags(0, flags::MAYEXEC) }; + Ok(()) + } +} + /// The integer type used for vma flags. #[doc(inline)] pub use bindings::vm_flags_t; From patchwork Fri Nov 29 16:32:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13888847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 881DFD729EC for ; Fri, 29 Nov 2024 16:33:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C4EB56B009C; Fri, 29 Nov 2024 11:33:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BD6C96B009E; Fri, 29 Nov 2024 11:33:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A272A6B009D; Fri, 29 Nov 2024 11:33:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7504C6B009B for ; Fri, 29 Nov 2024 11:33:10 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 39915C0212 for ; Fri, 29 Nov 2024 16:33:10 +0000 (UTC) X-FDA: 82839677172.11.674134A Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf06.hostedemail.com (Postfix) with ESMTP id 73445180019 for ; Fri, 29 Nov 2024 16:33:02 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=fvJg5PBV; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of 3w-xJZwkKCAYgroikx4nrmuumrk.iusrot03-ssq1giq.uxm@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3w-xJZwkKCAYgroikx4nrmuumrk.iusrot03-ssq1giq.uxm@flex--aliceryhl.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732897984; a=rsa-sha256; cv=none; b=Bx8zNpyttSF3RVsKdsj76IvRTK8s3AZ/jVHy/bf73n/BRT6tEXT29QeM+QGFGZ+jMRkxoI nky1A5Qbc4ly8mpw1aGMUIOEjUPhC14epWh98qZUFPF0BElqHrouDvr+jtB2fpiorToZn0 5eKR1M2nXMMZYHiOJ5erWdir9PmsiCI= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=fvJg5PBV; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of 3w-xJZwkKCAYgroikx4nrmuumrk.iusrot03-ssq1giq.uxm@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3w-xJZwkKCAYgroikx4nrmuumrk.iusrot03-ssq1giq.uxm@flex--aliceryhl.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732897984; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UnhyEorAiAZu+6K7VI1xMGROpYDgIW3AxkwRW5dHmBI=; b=jmjBgWfNXD3kJdfKbXGcKwhhxZuHvfUQ+n1spv1NJEqG5CL7h2q9P+G7oFd4pk+tIFa3e6 QXgyx/OIHvfkoxZnYquOs5nFUawzbM6PuUZc8AiOWdgSk4BQ1IZwIRYbVQ5nwprCZda71S hIusq2zGE+BHm7b+zIg5tWPb6fwPMFE= Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43498af7937so16383005e9.1 for ; Fri, 29 Nov 2024 08:33:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732897987; x=1733502787; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UnhyEorAiAZu+6K7VI1xMGROpYDgIW3AxkwRW5dHmBI=; b=fvJg5PBVpIX3FdF5a0vaeSl41XIdgJOKRlqYiI9qd/MThcZfQZ2zHNAXzw65MgQE4O izH3SyCCY7eeIp5acS1PXDBKBHGa9WQiAiANfqQQlb3esrzY2JMePI7SXrD2pavTeapB aEPkSpcytsx11zjcoYGVyTX+d2eJXKzX5a4qc9xyoRdR+9X/wpEiu5HR7b5cAyjShG4Y VQJpCVpxImxDvuTVgj75T30G2v+g1a8rBsp12WbpXfHjjMf4BhjHz4RTUJdugN6jAdbN rBeuPEIUHkK7Jv1b2xImVHjihEYPyQrWF/PVZSa8/+h3BHFDoEdmnJ1ARbSgIlQsB40o q72A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732897987; x=1733502787; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UnhyEorAiAZu+6K7VI1xMGROpYDgIW3AxkwRW5dHmBI=; b=JGnUgaUyLNm7DdnkBL0iaqsBzTP4MRNiboBdGMtrk8W1wI1vI+rBuJYeDUZUjmwPAO WNf/qX192R5J+iqkL62FgQCSjPWnjwCRLy7niYf4Rk0Oe+UqvMunJf1+B+l84G+xAObp ziS2q7Im2N8YjkX+QaWQ14GhOWpH/EsT42Z1chsMEWgrgGkAU2cAUNpGLgrQ+sDAST11 7xmETEFsRP844YVJ1X9XFWNSuainsQnt0RLngHRX2YOtyiWhJfuqp169rNbdm/9RPsGR gnz1zvHTCAtJPrQ0tXeL6MdhGdC3LmEyL2mnBZhh+Eok2vo0Qcjp6LTJi35XXDL5+qwY fFkg== X-Forwarded-Encrypted: i=1; AJvYcCXf877g5jU/hCNgS3iVqWPAijvCxgdA9w1ZLbxtpRQ17ucAupPPrj70nN+M7vLyMSG0HlPIKxsT2g==@kvack.org X-Gm-Message-State: AOJu0YwnylJ/VpuUVX8u9LSpt/Jea2DWUZL2neRGdc9KXmYD9a7arQrn ywF9DiN2WALXl2i8nP9JolQeLx7MvbTUSNFo9PuyAzAgOhExYfXtKhUrZVdy0Rk04JW6LVLs5Vl dZP5Cxr3eKf0Vig== X-Google-Smtp-Source: AGHT+IEIkhciFv/SoLFFHF6Vxq9iGEQEsrBvyMo5yav6t96an9Wy9B+lJz1jJamwd+Gu4sliNPoz0+xpNLIZ01E= X-Received: from wmlv12.prod.google.com ([2002:a05:600c:214c:b0:434:9de6:413a]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b24:b0:434:a754:661c with SMTP id 5b1f17b1804b1-434a9db83acmr109938695e9.7.1732897987022; Fri, 29 Nov 2024 08:33:07 -0800 (PST) Date: Fri, 29 Nov 2024 16:32:40 +0000 In-Reply-To: <20241129-vma-v10-0-4dfff05ba927@google.com> Mime-Version: 1.0 References: <20241129-vma-v10-0-4dfff05ba927@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=2822; i=aliceryhl@google.com; h=from:subject:message-id; bh=IrZvUF0XzNLQMwbfZ0pN4UXzZJ4vVLLu+S9qv3H24zI=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnSeyx3XZetGy5dEQOShQaSbbMRUwJxGEEMjLw8 9RdWQXDUSaJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ0nssQAKCRAEWL7uWMY5 Ri8ZD/wLIEpXUXaIuGN7hRbJCy7O1J4CbcrlTxrSVyF9E7xofr8DGwiW2NYtspHF6JvEA9vEbKZ VEUdDED+X/G1IXI86Qv4eEGCrNHUL6+idMjPfMaYJw55Rci+8r0Ewi+h9QePeYJNaw53q74kjpT szaSueywBq6/KA0AMbekbkpvM/Mlz3sBAqf0pd4McgQOuj3tp0+ApxmJf6lh4+52AoMeKLT2HdV ceUq8pZjoh+KV4CJIX5olRuJzxOgYXDI3Dtg/eMH/XM8EjyEzWTBUmdFn2REX8ei8h4BqWYrX3A ztvAOHVQjzOO8cHP1ew5/ExWAXiOZGn5/ZYVpWMQngFnorTAyaag3wK17LHrOZUbkx7YffpqrBN +97cJ1Myn4/bRhJoAB1V1MYxrQG+8hsaoddXg0IjCtfQi2p1o1aLqkjw6QO3U3q+q5v3ONXSzV3 K65OLwOalr42aJNfyYeQJafHoTzHvpeJ56/O0bGWqaxqVVd59t/l4NjCAb3vHdZasnVmz4W6Ag1 pcPQWa5L+SzCdE2TxRKYKm5+WXtXUbtC30SMhq+MoPqmFStspXIk/CJ60nyzvxG0hgL8P+ji6Po U7yv2fwSlp1YrDsHbvR2mQe0lh39+/0PLppp8PGqkRDBRuKk8s3ccdhTh6/8QiMsuyTda/Hp5g4 rC86Q3qz2t7SDcA== X-Mailer: b4 0.13.0 Message-ID: <20241129-vma-v10-7-4dfff05ba927@google.com> Subject: [PATCH v10 7/8] rust: miscdevice: add mmap support From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 73445180019 X-Stat-Signature: 3mrrodmiuyh86q6y7g8fseszx56wsa4c X-HE-Tag: 1732897982-266769 X-HE-Meta: U2FsdGVkX19T4Hr5CWm4PJ3v/O0AQg7NKjEIx0oGUNU8XB70OtY5FkxCDimuSQJSCJ4morlAu55M9IjwdjPZTLdmS8oWbLlNSzDrivnF34pJ+6Zgc5Qbj95f3gllyjCX4TVBOy8dHPkU4scvo/8BICMpSeVxhs8P5460WepgD/2dsmgvxAGDayhuxJxvqhmI+ud9XINpx8R9OFSj4O7k16mittrG1QFSpb+sSapJHMishAubuh8NaVkXUp3oGZFPd2RAlPEJc5nNmRLEqO2cPAh0TWFNgfYhRcGxBSDlAz1jRBXpAHVLLjm8k8yKnu9MG0Zk5eDPY1vDGSQCMYeJsYj/zTB2SbTfGMq34U1ozuuEaMv0vFRymHMm0EbIQbvKXHyXL5DE459GttcAQWAsf/s698nflSMRKQI2IdWnE5LbCpPmJsNzpItEe6VcydPUOU9QT+MlU/bTtXJZKkpgVvtLaBY8X0uS1acheFPQyP0U2UsGFJn8Q195UIxUQGgcbGH6JOUsg5SMKjWgDwLqlYCY6IF5JKuW0KTjU0HbP9wL5jGA8gXBMau7eLaTsXdXLYtTdNu3kPYUNLItJM66mBGlUQodrhCTRAWQnRIUcqEo/uRB2uCxBAw1wjtFfT2wDIOUoqvKkR/Kfczp6eArUj+OuYmJFm+fKj8DDeDqp9Reqmi6DjBpDlUeMRxwLYG4508Kcxu90WrhkrC5gO+V216KMaFL1yHiQlFL5BUApx7cH3WesQdTkOKkEH2Kl6Rz+C9HGW7MT7MCTINCwVkd6E9JcGuuBcBrttzGIKreFQOo+IHZIIn/RpM9iAJxI/hPvUigmpndprB31Lltdx3vgbTrWXyLZrf8UWM9ngeJZ9CiH+6MmI9c3yECj+hJevtrpEGLYYdNPKK+iwLlfh+lt93ksMQWk0Wru6kagTp3/IQ07gAHvpQTfhvoddJH/HGMVCg/W97wv8PBwJ+WNsH FqG8/7Gg jyc9IETyw0mVAVDRb8dYrftrelT4Sly7kUVuLlAlAwNmBNQqMZiIOpQqrHbV8qxA8fPTNBGPVBlqxaozskqwLl+13tU46UY8E5wFqIm6Ma1VwTu97TDOTAiEvdkA8kEcK5nfE1tICLJ4CgV4SWkwRNPNX6UBBd1gtEDb+LEEattoIciwjcQjy/tiNOKzEFHSZ+Ljz9XEWoHGatrNSgkYL7nVFP6D5FtyQGrGBpxtcAi02rNLOi8HYnsVnH8lSQfBHBqbbEgldOJ2hiT82INoH20PH6XboLyUjo2xY5p7aSidCD82gUpJW2ZKMxznukXKA4L78Po4qUFY0YeNAbt89f7jWFedtMXz0SeeZPsqMzG2k5N0+Wv/4i0uCW0wEnH0R1cmtWzZ/PSMvWnG5gizYWpGirPDolG6/cczZreMTBFTwl/kX3SNFq1wuh6XqdKHIBxOAJjb/3IvCYWtrf/qQwjkoyLXvEOMLMyePxLmv28ShN66PvMimHKKUTbqkWl6t0KHIDSAEa2Klv1vO0hr+Ei1jGQBrbNKx/H29rR87XCyY8DviXGVYVcFCXHZi2JEPEjczFoHXLLYpFYyf/U+Rmsf22LfTagZEi22BGqslsj1fBULz5oCnL/L4bImW+eG+26jB/4/5yTj72U1O9gMckvhlrgjq48rtuzJW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add the ability to write a file_operations->mmap hook in Rust when using the miscdevice abstraction. The `vma` argument to the `mmap` hook uses the `VmAreaNew` type from the previous commit; this type provides the correct set of operations for a file_operations->mmap hook. Acked-by: Lorenzo Stoakes (for mm bits) Signed-off-by: Alice Ryhl --- rust/kernel/miscdevice.rs | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/rust/kernel/miscdevice.rs b/rust/kernel/miscdevice.rs index 7e2a79b3ae26..99e076338f39 100644 --- a/rust/kernel/miscdevice.rs +++ b/rust/kernel/miscdevice.rs @@ -11,6 +11,7 @@ use crate::{ bindings, error::{to_result, Error, Result, VTABLE_DEFAULT_ERROR}, + mm::virt::VmAreaNew, prelude::*, str::CStr, types::{ForeignOwnable, Opaque}, @@ -110,6 +111,11 @@ fn release(device: Self::Ptr) { drop(device); } + /// Handle for mmap. + fn mmap(_device: ::Borrowed<'_>, _vma: &VmAreaNew) -> Result { + kernel::build_error!(VTABLE_DEFAULT_ERROR) + } + /// Handler for ioctls. /// /// The `cmd` argument is usually manipulated using the utilties in [`kernel::ioctl`]. @@ -156,6 +162,7 @@ impl VtableHelper { const VTABLE: bindings::file_operations = bindings::file_operations { open: Some(fops_open::), release: Some(fops_release::), + mmap: maybe_fn(T::HAS_MMAP, fops_mmap::), unlocked_ioctl: maybe_fn(T::HAS_IOCTL, fops_ioctl::), #[cfg(CONFIG_COMPAT)] compat_ioctl: if T::HAS_COMPAT_IOCTL { @@ -216,6 +223,27 @@ impl VtableHelper { 0 } +/// # Safety +/// +/// `file` must be a valid file that is associated with a `MiscDeviceRegistration`. +/// `vma` must be a vma that is currently being mmap'ed with this file. +unsafe extern "C" fn fops_mmap( + file: *mut bindings::file, + vma: *mut bindings::vm_area_struct, +) -> c_int { + // SAFETY: The mmap call of a file can access the private data. + let private = unsafe { (*file).private_data }; + // SAFETY: Mmap calls can borrow the private data of the file. + let device = unsafe { ::borrow(private) }; + // SAFETY: The caller provides a vma that is undergoing initial VMA setup. + let area = unsafe { VmAreaNew::from_raw(vma) }; + + match T::mmap(device, area) { + Ok(()) => 0, + Err(err) => err.to_errno() as c_int, + } +} + /// # Safety /// /// `file` must be a valid file that is associated with a `MiscDeviceRegistration`. From patchwork Fri Nov 29 16:32:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13888848 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E899D729E7 for ; Fri, 29 Nov 2024 16:33:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34E226B009E; Fri, 29 Nov 2024 11:33:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F9C16B009F; Fri, 29 Nov 2024 11:33:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B1986B00A0; Fri, 29 Nov 2024 11:33:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DA2786B009E for ; Fri, 29 Nov 2024 11:33:12 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A3BBB4119A for ; Fri, 29 Nov 2024 16:33:12 +0000 (UTC) X-FDA: 82839677256.18.BF85BFE Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf20.hostedemail.com (Postfix) with ESMTP id 1101F1C0011 for ; Fri, 29 Nov 2024 16:33:00 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=0CxnnJhg; spf=pass (imf20.hostedemail.com: domain of 3xexJZwkKCAgitqkmz6ptowwotm.kwutqv25-uus3iks.wzo@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3xexJZwkKCAgitqkmz6ptowwotm.kwutqv25-uus3iks.wzo@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732897982; a=rsa-sha256; cv=none; b=Fx+vselkeidWkLseeL1M+cMlH/jeegocipt6K5JAvJNvVkY4yw2dPpccgYKBMZHGb7kq5o w3Rui/Yx1Dp+DhgJHZsokh88Fg7A3eHKV95VxaED5pV5+Z7QkjFPWkL56uuierTQPDoRS9 3Xkzk90LWFXjFbU4Ie6r7vQJsdL7npA= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=0CxnnJhg; spf=pass (imf20.hostedemail.com: domain of 3xexJZwkKCAgitqkmz6ptowwotm.kwutqv25-uus3iks.wzo@flex--aliceryhl.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3xexJZwkKCAgitqkmz6ptowwotm.kwutqv25-uus3iks.wzo@flex--aliceryhl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732897982; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=khaQBcubSs8zPXHk0UASzgDi1+R5RW3CVTJj7OV4mlM=; b=5sgyBQUYad9PYibRnvmXPcK2ml91R234d1Nu7KxQPRIr+pTspexPMpN7T+aIFzDHT6WlNR Q62nXJUTNOg+qTdk80unr+lNB0RD3lM9WkWK4RQO23iTzFmDKN1KaAheo3X8Ee5lN3WtQM 2dlPQvo6OlgXsP24i0XK02U0oX3mimI= Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-434a195814fso15050545e9.3 for ; Fri, 29 Nov 2024 08:33:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732897989; x=1733502789; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=khaQBcubSs8zPXHk0UASzgDi1+R5RW3CVTJj7OV4mlM=; b=0CxnnJhgjf8Q9pPu3VkQgV2stAKZmLKYzrREtUFJN4jdcGVk2HO9Z+/hbEbFvEtqIC P2k617QAJj3agm+Dz0Q7Aoe1oW9Y8yNeNrt7c2xE3UZxpNYROtzrFFsa16MOYV51ZLGE x3Cr/dmA6XGK/2J26SA/uvxIRB8+iWNOeth8nRPxLUXUxpndf1E+oUJQgTz7TiSbz9xO W9s6adtF4+0HzmB+gvtEVVi4lJUQWZNeMXMIrkeOBcGT8cuR+r1YIASBAUprOQ3wI4IU cXxjTryLyQ0bknPSU3ex5v7RRKoFvj10K+MtoSLpWI5Vk/fac75fctiP7hBCvQD9FmRp H6DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732897989; x=1733502789; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=khaQBcubSs8zPXHk0UASzgDi1+R5RW3CVTJj7OV4mlM=; b=Ay1iWHa6odqK6aNJHeQU48Gp1vOlCYprLWXHXxxTH0YharfrVGXDk0i/+PbI+wR9Xy Tbic8IsZ5NCRawdirOYbQd9/uRTfebLqcyi8GcSxfH1AWoBcNWXK+qqt4moyofGIaeE0 yxZxmXZIlJCYKUi60yFlEjjsd2zW0IgwKfLIZaZVQhnF6KTo1PQLp4a8t1X7xHddtN89 dYibBOFeDafFYrKW7eQI0sGtDOjFE0Pz7zLxYrcUSj2Cn5E78O7LQXAjtLkZsCMycKKQ XJoFM5hsacK3x5RlVlFbeCeMyxhPb9hyLpfc3CThmwheBfjP3KuIB6waX5h4bxwZ8WfB SCRQ== X-Forwarded-Encrypted: i=1; AJvYcCWKx6GyHzDv3otFq503v1yzZxqmrMRHxQyDd0twhaziDx4Ifg27CoQ0E84+Wu/mgWul02FbW7Nb9Q==@kvack.org X-Gm-Message-State: AOJu0Yx/KYs51AiNP2dqwxjgoWv9HykmIHg2j+mD+JTrERHXKdiVOqp9 NF7xfWmsSd0jAxigY69wRAUPmYpkVYMNKC7v0uyTKWTNDDZKta/AEID9PNVhFA1FawYFlhE1Koi QmAkfffzGBZCOXA== X-Google-Smtp-Source: AGHT+IEBCg9KJJDJe7VoW4ZdyHwyebrajmoBgyf09TNJjFSpDvZZB+3u8dCYzcG1MDbxDRLvGSfQivBbzvIPAko= X-Received: from wmoi4.prod.google.com ([2002:a05:600c:4804:b0:431:4c36:d336]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:458b:b0:431:5f1c:8359 with SMTP id 5b1f17b1804b1-434a9dcfd5emr120555635e9.15.1732897989312; Fri, 29 Nov 2024 08:33:09 -0800 (PST) Date: Fri, 29 Nov 2024 16:32:41 +0000 In-Reply-To: <20241129-vma-v10-0-4dfff05ba927@google.com> Mime-Version: 1.0 References: <20241129-vma-v10-0-4dfff05ba927@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=22122; i=aliceryhl@google.com; h=from:subject:message-id; bh=UOgti2W3w5pdDKaneEZaDc7ZRGkpusBY1HuaqkaxSYQ=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnSeyys/aPXUMqJz2reACEhC4F6C968B9ndJ6+v TtfUjZpYQCJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZ0nssgAKCRAEWL7uWMY5 RsyPEACl6aaN3iihdjb3Lukz8wuBKhz9hO4mUk4ShWSq1Gpm8ui6yPKbk7FwrE7JRP1d+z4oyqX Ag6s7Vc6MWOSon5OgspWo9ddAWb/LGo+KkA8q7N2mhDrckQKdxIK7roASRVIUCH86nSSGEQdFUW tSv8ZBtT0UbQQ7OpTt/6iCzUEDNxx0UsWXRnkgya3zoeOktB/WFkJ4waQyQqxpQ2rB81nJKIXWm QJNGX4yatpPYp8gQ0QHtpeaArKJOIOCZgX6qeevc9UArg3n4hGcIIf9oZkG8xWeszpoue4XftpD mypcBaFsy12CTFh5Dq36HI3UqSZlZDwQWWCdhRIyBRFhAQ1COjZsxlvTTS180MBmYPREbunV4Lx IAIiOnCg0UOrOZG1PnnTnsPQj/VUBKqs+v0zTGyaKZcBYaKuD2fF7J/VvvmWeP9c0XS6yGHgM2p XEoicgR0hnK1BYVISv1pD7CVyDhYK9A9M40ICECur+GMwggsOCP+iWLd2+dVylN2QGpVQTYySCc BJOkxjWrDU7BHOT++OsYZZ8GtddUuVNZLKkbv1MKEy6dpaHYi3/g5i2uN2agLiaxYPRneEsteLa dX+Upt/nbs3CnGjySWD2cGfEI5e4nCPAeCY1dXAiVt05L0DObOG1EpIyZqC/JtfYdEqv1zapIGX JOeRJO5wY+7nO8A== X-Mailer: b4 0.13.0 Message-ID: <20241129-vma-v10-8-4dfff05ba927@google.com> Subject: [PATCH v10 8/8] task: rust: rework how current is accessed From: Alice Ryhl To: Miguel Ojeda , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , John Hubbard , "Liam R. Howlett" , Andrew Morton , Greg Kroah-Hartman , Arnd Bergmann , Christian Brauner , Jann Horn , Suren Baghdasaryan Cc: Alex Gaynor , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg , Trevor Gross , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rust-for-linux@vger.kernel.org, Alice Ryhl X-Rspamd-Queue-Id: 1101F1C0011 X-Stat-Signature: ycbz1j7cs1t6oqh37ayhkhjewxtxuaqj X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1732897980-636241 X-HE-Meta: U2FsdGVkX19n5R/lBRHnpA/92w9F9kCvM4J6gpyjDGZiwt7YifSA+21YFu+1jGba5HcpFNxlG/LrGA5qLMPDUUxcjh8qRc8aQ3zAdTgiASIea74qDXeJOnOt5Cw1NeeIKqCa5ROXIgrMMfGhk6YZso22XlAm8dg95AcDiUxhmCwZQvB8w2HlHYq4lHQMdbsgapkPB8iRyc7VD1ZSnPgtcHhQjNeYtnT0sncBDPCyfDM03xSUNmqe1oBb60HGW8niHaPgT7k0LqD24ZD85gb44R2KHfOg8oyGR3LgJ0ppvZPTRHFy8MWtL4OnxWDBIOc2Vu6Jlx/tugrARp4m3Mm2ihmAoanLdCbhWRezFd8a01sXd/LcjzR/g/BOWu389MzSk3SMAFVy9JbbqcSEPMvdFWkA1Z9CVokYxmtrHjsyyZMRZCLBHEsm7okdTXe99+YWaaem4R3Gtd/GR/Fja6qCtT/UV/crH3gHKlWmp0XGfAyH1sLndQitDFVBbjrwKMu/L+eumU3c5wUUreSQaYNVCfiwdB/aIoe6Qf6HW4JJIY8zMA6prhqzGhY5UZQA21DS5jBtGesT8eKtZeeUdb83K+xN2NR9xJ16bQyLGbvijcJkdSyFi+rJ8YgFnVbH/ukuzonrE/PqfFwXtOmT8wmLxrQWGWsxwTv/rCs4mMre4YuYJXrcea0uhhLqSWXraE9ZsGwo76E8b7y8WuQDLOHHIvxZpDWmX4V50ufkpSjXAXSGCqgm6GZejrXggwBX+1d1Ed8ddghMjGkzydBqzow7yLdbTrRllqqUG3zfTMva0NUkkEwZDROUEUFOhOpmvQtGPvRiwvFyd/5+pTsX3GyxBdbKcnji66nXEIzvxKr3xCIcxBZ7NYX2bwd2KM6AvreX3aoZoUWsOhbgo4Lrzlz5obrnsvAEIaB3+JrHdA7mDOXFkAZODO5hUpWYPrz0ckeJYaTc1LSBRJVBVZTm1iB Q72z7bPk woTuMQmewOWvISNwJEb7AcIVp0qO2cVLIyzSse24N+ewocZSWpEDUa9y7zGwe1qUJRbAcQ7Bai4SGaD84Jpd+yCNRu93Emk7zY+O1GnjWsVtN5sETqEzPM69ZI6g3o8lPpfbl4UilgUOlmr6NkX7cnEDXg1KzTKSAXocE9yiOx9nIn2yHn4v+ULf/i8+tVrDRpIppUUkilhKVccSgU+aZoMXaR7AQ0t91nfU0+dhqQoZDX6eBbqDMovwwOC9FLaT3jDp76rWvFeDxn1JqC+DMPL8wvBXeHWz5INE7j0IHjyQ2e0heS733rtnlp309h00D8ZBWG+AT1qHkvkPKAs1S0AtDNCdMsSaAIl7bs0RlS0EEuF/i9HtKuu02yhguFR1pN2WaDXlocWev9UJTV/Lta2/1B8sA6mwsErsP9R62k+unu3pvJzVX+sjeDi2gDOVm6vUg311VJyMUClpZvfYo9MbybNN16xJEW3lpMWdSYYsrbVJwxC9Db1wZN7i2BPOTHrb8Kz+JUErwgRifAn9dHct4N6OYxEdzd2EtVUDBgDNT/LyI3o3uhAHedmvROnPLkVo6b8VEdASZsX3dYCxs/y/oxt2lBVqnoBlvhRRv2n99TWawGFGiKjWo6Bf2/zQuhRcOZVdaG+955oyo7S+7KzlIBRY799FiIsOM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a new type called `CurrentTask` that lets you perform various operations that are only safe on the `current` task. Use the new type to provide a way to access the current mm without incrementing its refcount. With this change, you can write stuff such as let vma = current!().mm().lock_vma_under_rcu(addr); without incrementing any refcounts. This replaces the existing abstractions for accessing the current pid namespace. With the old approach, every field access to current involves both a macro and a unsafe helper function. The new approach simplifies that to a single safe function on the `CurrentTask` type. This makes it less heavy-weight to add additional current accessors in the future. That said, creating a `CurrentTask` type like the one in this patch requires that we are careful to ensure that it cannot escape the current task or otherwise access things after they are freed. To do this, I declared that it cannot escape the current "task context" where I defined a "task context" as essentially the region in which `current` remains unchanged. So e.g., release_task() or begin_new_exec() would leave the task context. If a userspace thread returns to userspace and later makes another syscall, then I consider the two syscalls to be different task contexts. This allows values stored in that task to be modified between syscalls, even if they're guaranteed to be immutable during a syscall. Ensuring correctness of `CurrentTask` is slightly tricky if we also want the ability to have a safe `kthread_use_mm()` implementation in Rust. To support that safely, there are two patterns we need to ensure are safe: // Case 1: current!() called inside the scope. let mm; kthread_use_mm(some_mm, || { mm = current!().mm(); }); drop(some_mm); mm.do_something(); // UAF and: // Case 2: current!() called before the scope. let mm; let task = current!(); kthread_use_mm(some_mm, || { mm = task.mm(); }); drop(some_mm); mm.do_something(); // UAF The existing `current!()` abstraction already natively prevents the first case: The `&CurrentTask` would be tied to the inner scope, so the borrow-checker ensures that no reference derived from it can escape the scope. Fixing the second case is a bit more tricky. The solution is to essentially pretend that the contents of the scope execute on an different thread, which means that only thread-safe types can cross the boundary. Since `CurrentTask` is marked `NotThreadSafe`, attempts to move it to another thread will fail, and this includes our fake pretend thread boundary. This has the disadvantage that other types that aren't thread-safe for reasons unrelated to `current` also cannot be moved across the `kthread_use_mm()` boundary. I consider this an acceptable tradeoff. Cc: Christian Brauner Signed-off-by: Alice Ryhl --- rust/kernel/mm.rs | 22 ---- rust/kernel/task.rs | 284 ++++++++++++++++++++++++++++++---------------------- 2 files changed, 167 insertions(+), 139 deletions(-) diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs index 50f4861ae4b9..f7d1079391ef 100644 --- a/rust/kernel/mm.rs +++ b/rust/kernel/mm.rs @@ -142,28 +142,6 @@ fn deref(&self) -> &MmWithUser { // These methods are safe to call even if `mm_users` is zero. impl Mm { - /// Call `mmgrab` on `current.mm`. - #[inline] - pub fn mmgrab_current() -> Option> { - // SAFETY: It's safe to get the `mm` field from current. - let mm = unsafe { - let current = bindings::get_current(); - (*current).mm - }; - - if mm.is_null() { - return None; - } - - // SAFETY: The value of `current->mm` is guaranteed to be null or a valid `mm_struct`. We - // just checked that it's not null. Furthermore, the returned `&Mm` is valid only for the - // duration of this function, and `current->mm` will stay valid for that long. - let mm = unsafe { Mm::from_raw(mm) }; - - // This increments the refcount using `mmgrab`. - Some(ARef::from(mm)) - } - /// Returns a raw pointer to the inner `mm_struct`. #[inline] pub fn as_raw(&self) -> *mut bindings::mm_struct { diff --git a/rust/kernel/task.rs b/rust/kernel/task.rs index 07bc22a7645c..8c1ee46c03eb 100644 --- a/rust/kernel/task.rs +++ b/rust/kernel/task.rs @@ -7,6 +7,7 @@ use crate::{ bindings, ffi::{c_int, c_long, c_uint}, + mm::MmWithUser, pid_namespace::PidNamespace, types::{ARef, NotThreadSafe, Opaque}, }; @@ -31,22 +32,20 @@ #[macro_export] macro_rules! current { () => { - // SAFETY: Deref + addr-of below create a temporary `TaskRef` that cannot outlive the - // caller. + // SAFETY: This expression creates a temporary value that is dropped at the end of the + // caller's scope. The following mechanisms ensure that the resulting `&CurrentTask` cannot + // leave current task context: + // + // * To return to userspace, the caller must leave the current scope. + // * Operations such as `begin_new_exec()` are necessarily unsafe and the caller of + // `begin_new_exec()` is responsible for safety. + // * Rust abstractions for things such as a `kthread_use_mm()` scope must require the + // closure to be `Send`, so the `NotThreadSafe` field of `CurrentTask` ensures that the + // `&CurrentTask` cannot cross the scope in either direction. unsafe { &*$crate::task::Task::current() } }; } -/// Returns the currently running task's pid namespace. -#[macro_export] -macro_rules! current_pid_ns { - () => { - // SAFETY: Deref + addr-of below create a temporary `PidNamespaceRef` that cannot outlive - // the caller. - unsafe { &*$crate::task::Task::current_pid_ns() } - }; -} - /// Wraps the kernel's `struct task_struct`. /// /// # Invariants @@ -105,6 +104,44 @@ unsafe impl Send for Task {} // synchronised by C code (e.g., `signal_pending`). unsafe impl Sync for Task {} +/// Represents the [`Task`] in the `current` global. +/// +/// This type exists to provide more efficient operations that are only valid on the current task. +/// For example, to retrieve the pid-namespace of a task, you must use rcu protection unless it is +/// the current task. +/// +/// # Invariants +/// +/// Each value of this type must only be accessed from the task context it was created within. +/// +/// Of course, every thread is in a different task context, but for the purposes of this invariant, +/// these operations also permanently leave the task context: +/// +/// * Returning to userspace from system call context. +/// * Calling `release_task()`. +/// * Calling `begin_new_exec()` in a binary format loader. +/// +/// Other operations temporarily create a new sub-context: +/// +/// * Calling `kthread_use_mm()` creates a new context, and `kthread_unuse_mm()` returns to the +/// old context. +/// +/// This means that a `CurrentTask` obtained before a `kthread_use_mm()` call may be used again +/// once `kthread_unuse_mm()` is called, but it must not be used between these two calls. +/// Conversely, a `CurrentTask` obtained between a `kthread_use_mm()`/`kthread_unuse_mm()` pair +/// must not be used after `kthread_unuse_mm()`. +#[repr(transparent)] +pub struct CurrentTask(Task, NotThreadSafe); + +// Make all `Task` methods available on `CurrentTask`. +impl Deref for CurrentTask { + type Target = Task; + #[inline] + fn deref(&self) -> &Task { + &self.0 + } +} + /// The type of process identifiers (PIDs). type Pid = bindings::pid_t; @@ -131,119 +168,29 @@ pub fn current_raw() -> *mut bindings::task_struct { /// /// # Safety /// - /// Callers must ensure that the returned object doesn't outlive the current task/thread. - pub unsafe fn current() -> impl Deref { - struct TaskRef<'a> { - task: &'a Task, - _not_send: NotThreadSafe, + /// Callers must ensure that the returned object is only used to access a [`CurrentTask`] + /// within the task context that was active when this function was called. For more details, + /// see the invariants section for [`CurrentTask`]. + pub unsafe fn current() -> impl Deref { + struct TaskRef { + task: *const CurrentTask, } - impl Deref for TaskRef<'_> { - type Target = Task; + impl Deref for TaskRef { + type Target = CurrentTask; fn deref(&self) -> &Self::Target { - self.task + // SAFETY: The returned reference borrows from this `TaskRef`, so it cannot outlive + // the `TaskRef`, which the caller of `Task::current()` has promised will not + // outlive the task/thread for which `self.task` is the `current` pointer. Thus, it + // is okay to return a `CurrentTask` reference here. + unsafe { &*self.task } } } - let current = Task::current_raw(); TaskRef { - // SAFETY: If the current thread is still running, the current task is valid. Given - // that `TaskRef` is not `Send`, we know it cannot be transferred to another thread - // (where it could potentially outlive the caller). - task: unsafe { &*current.cast() }, - _not_send: NotThreadSafe, - } - } - - /// Returns a PidNamespace reference for the currently executing task's/thread's pid namespace. - /// - /// This function can be used to create an unbounded lifetime by e.g., storing the returned - /// PidNamespace in a global variable which would be a bug. So the recommended way to get the - /// current task's/thread's pid namespace is to use the [`current_pid_ns`] macro because it is - /// safe. - /// - /// # Safety - /// - /// Callers must ensure that the returned object doesn't outlive the current task/thread. - pub unsafe fn current_pid_ns() -> impl Deref { - struct PidNamespaceRef<'a> { - task: &'a PidNamespace, - _not_send: NotThreadSafe, - } - - impl Deref for PidNamespaceRef<'_> { - type Target = PidNamespace; - - fn deref(&self) -> &Self::Target { - self.task - } - } - - // The lifetime of `PidNamespace` is bound to `Task` and `struct pid`. - // - // The `PidNamespace` of a `Task` doesn't ever change once the `Task` is alive. A - // `unshare(CLONE_NEWPID)` or `setns(fd_pidns/pidfd, CLONE_NEWPID)` will not have an effect - // on the calling `Task`'s pid namespace. It will only effect the pid namespace of children - // created by the calling `Task`. This invariant guarantees that after having acquired a - // reference to a `Task`'s pid namespace it will remain unchanged. - // - // When a task has exited and been reaped `release_task()` will be called. This will set - // the `PidNamespace` of the task to `NULL`. So retrieving the `PidNamespace` of a task - // that is dead will return `NULL`. Note, that neither holding the RCU lock nor holding a - // referencing count to - // the `Task` will prevent `release_task()` being called. - // - // In order to retrieve the `PidNamespace` of a `Task` the `task_active_pid_ns()` function - // can be used. There are two cases to consider: - // - // (1) retrieving the `PidNamespace` of the `current` task - // (2) retrieving the `PidNamespace` of a non-`current` task - // - // From system call context retrieving the `PidNamespace` for case (1) is always safe and - // requires neither RCU locking nor a reference count to be held. Retrieving the - // `PidNamespace` after `release_task()` for current will return `NULL` but no codepath - // like that is exposed to Rust. - // - // Retrieving the `PidNamespace` from system call context for (2) requires RCU protection. - // Accessing `PidNamespace` outside of RCU protection requires a reference count that - // must've been acquired while holding the RCU lock. Note that accessing a non-`current` - // task means `NULL` can be returned as the non-`current` task could have already passed - // through `release_task()`. - // - // To retrieve (1) the `current_pid_ns!()` macro should be used which ensure that the - // returned `PidNamespace` cannot outlive the calling scope. The associated - // `current_pid_ns()` function should not be called directly as it could be abused to - // created an unbounded lifetime for `PidNamespace`. The `current_pid_ns!()` macro allows - // Rust to handle the common case of accessing `current`'s `PidNamespace` without RCU - // protection and without having to acquire a reference count. - // - // For (2) the `task_get_pid_ns()` method must be used. This will always acquire a - // reference on `PidNamespace` and will return an `Option` to force the caller to - // explicitly handle the case where `PidNamespace` is `None`, something that tends to be - // forgotten when doing the equivalent operation in `C`. Missing RCU primitives make it - // difficult to perform operations that are otherwise safe without holding a reference - // count as long as RCU protection is guaranteed. But it is not important currently. But we - // do want it in the future. - // - // Note for (2) the required RCU protection around calling `task_active_pid_ns()` - // synchronizes against putting the last reference of the associated `struct pid` of - // `task->thread_pid`. The `struct pid` stored in that field is used to retrieve the - // `PidNamespace` of the caller. When `release_task()` is called `task->thread_pid` will be - // `NULL`ed and `put_pid()` on said `struct pid` will be delayed in `free_pid()` via - // `call_rcu()` allowing everyone with an RCU protected access to the `struct pid` acquired - // from `task->thread_pid` to finish. - // - // SAFETY: The current task's pid namespace is valid as long as the current task is running. - let pidns = unsafe { bindings::task_active_pid_ns(Task::current_raw()) }; - PidNamespaceRef { - // SAFETY: If the current thread is still running, the current task and its associated - // pid namespace are valid. `PidNamespaceRef` is not `Send`, so we know it cannot be - // transferred to another thread (where it could potentially outlive the current - // `Task`). The caller needs to ensure that the PidNamespaceRef doesn't outlive the - // current task/thread. - task: unsafe { PidNamespace::from_ptr(pidns) }, - _not_send: NotThreadSafe, + // CAST: The layout of `struct task_struct` and `CurrentTask` is identical. + task: Task::current_raw().cast(), } } @@ -326,6 +273,109 @@ pub fn wake_up(&self) { } } +impl CurrentTask { + /// Access the address space of the current task. + /// + /// This function does not touch the refcount of the mm. + #[inline] + pub fn mm(&self) -> Option<&MmWithUser> { + // SAFETY: The `mm` field of `current` is not modified from other threads, so reading it is + // not a data race. + let mm = unsafe { (*self.as_ptr()).mm }; + + if mm.is_null() { + return None; + } + + // SAFETY: If `current->mm` is non-null, then it references a valid mm with a non-zero + // value of `mm_users`. Furthermore, the returned `&MmWithUser` borrows from this + // `CurrentTask`, so it cannot escape the scope in which the current pointer was obtained. + // + // This is safe even if `kthread_use_mm()`/`kthread_unuse_mm()` are used. There are two + // relevant cases: + // * If the `&CurrentTask` was created before `kthread_use_mm()`, then it cannot be + // accessed during the `kthread_use_mm()`/`kthread_unuse_mm()` scope due to the + // `NotThreadSafe` field of `CurrentTask`. + // * If the `&CurrentTask` was created within a `kthread_use_mm()`/`kthread_unuse_mm()` + // scope, then the `&CurrentTask` cannot escape that scope, so the returned `&MmWithUser` + // also cannot escape that scope. + // In either case, it's not possible to read `current->mm` and keep using it after the + // scope is ended with `kthread_unuse_mm()`. + Some(unsafe { MmWithUser::from_raw(mm) }) + } + + /// Access the pid namespace of the current task. + /// + /// This function does not touch the refcount of the namespace or use RCU protection. + #[doc(alias = "task_active_pid_ns")] + #[inline] + pub fn active_pid_ns(&self) -> Option<&PidNamespace> { + // SAFETY: It is safe to call `task_active_pid_ns` without RCU protection when calling it + // on the current task. + let active_ns = unsafe { bindings::task_active_pid_ns(self.as_ptr()) }; + + if active_ns.is_null() { + return None; + } + + // The lifetime of `PidNamespace` is bound to `Task` and `struct pid`. + // + // The `PidNamespace` of a `Task` doesn't ever change once the `Task` is alive. A + // `unshare(CLONE_NEWPID)` or `setns(fd_pidns/pidfd, CLONE_NEWPID)` will not have an effect + // on the calling `Task`'s pid namespace. It will only effect the pid namespace of children + // created by the calling `Task`. This invariant guarantees that after having acquired a + // reference to a `Task`'s pid namespace it will remain unchanged. + // + // When a task has exited and been reaped `release_task()` will be called. This will set + // the `PidNamespace` of the task to `NULL`. So retrieving the `PidNamespace` of a task + // that is dead will return `NULL`. Note, that neither holding the RCU lock nor holding a + // referencing count to the `Task` will prevent `release_task()` being called. + // + // In order to retrieve the `PidNamespace` of a `Task` the `task_active_pid_ns()` function + // can be used. There are two cases to consider: + // + // (1) retrieving the `PidNamespace` of the `current` task + // (2) retrieving the `PidNamespace` of a non-`current` task + // + // From system call context retrieving the `PidNamespace` for case (1) is always safe and + // requires neither RCU locking nor a reference count to be held. Retrieving the + // `PidNamespace` after `release_task()` for current will return `NULL` but no codepath + // like that is exposed to Rust. + // + // Retrieving the `PidNamespace` from system call context for (2) requires RCU protection. + // Accessing `PidNamespace` outside of RCU protection requires a reference count that + // must've been acquired while holding the RCU lock. Note that accessing a non-`current` + // task means `NULL` can be returned as the non-`current` task could have already passed + // through `release_task()`. + // + // To retrieve (1) the `&CurrentTask` type should be used which ensures that the returned + // `PidNamespace` cannot outlive the current task context. The `CurrentTask::active_pid_ns` + // function allows Rust to handle the common case of accessing `current`'s `PidNamespace` + // without RCU protection and without having to acquire a reference count. + // + // For (2) the `task_get_pid_ns()` method must be used. This will always acquire a + // reference on `PidNamespace` and will return an `Option` to force the caller to + // explicitly handle the case where `PidNamespace` is `None`, something that tends to be + // forgotten when doing the equivalent operation in `C`. Missing RCU primitives make it + // difficult to perform operations that are otherwise safe without holding a reference + // count as long as RCU protection is guaranteed. But it is not important currently. But we + // do want it in the future. + // + // Note for (2) the required RCU protection around calling `task_active_pid_ns()` + // synchronizes against putting the last reference of the associated `struct pid` of + // `task->thread_pid`. The `struct pid` stored in that field is used to retrieve the + // `PidNamespace` of the caller. When `release_task()` is called `task->thread_pid` will be + // `NULL`ed and `put_pid()` on said `struct pid` will be delayed in `free_pid()` via + // `call_rcu()` allowing everyone with an RCU protected access to the `struct pid` acquired + // from `task->thread_pid` to finish. + // + // SAFETY: If `current`'s pid ns is non-null, then it references a valid pid ns. + // Furthermore, the returned `&PidNamespace` borrows from this `CurrentTask`, so it cannot + // escape the scope in which the current pointer was obtained. + Some(unsafe { PidNamespace::from_ptr(active_ns) }) + } +} + // SAFETY: The type invariants guarantee that `Task` is always refcounted. unsafe impl crate::types::AlwaysRefCounted for Task { fn inc_ref(&self) {