From patchwork Fri Mar 22 23:38:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boqun Feng X-Patchwork-Id: 13600493 Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC80382C67; Fri, 22 Mar 2024 23:39:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711150752; cv=none; b=kqQZsrwkT4PELYPf0GbbHNcu9lvd7OaHAlTXUjOk2Aw7HUGyrAJM5eV1iaF5Lw7LW9CgLdmTmeDrzAKWkXLQEumKP6xQthJV28M50cKh8JTAho031ABSd9rwS9pOMyYdvkP0tzXQbmLSarMesiYvHOUkhAPJbwjhA5jiGP+X3oo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711150752; c=relaxed/simple; bh=e6OXAQgc0jxh60UP+AT0xAtl7n5YPlqj9xRF3ydf1Vc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tI9z1mp27vYc9MDfTrjEBkbXCor2ks/jeqDIlpiLVMeD7175F1xNocTBM0f964hXVZ3nhHUqeXcnhvrPMaSJI1xVGmxuBozdlOlN07c+4ht0FI6D4UVgJ/hC4qsk3FhWnJ7QrqeFpuZhB+eVpRNR4U5xdgiQEb7OuWebn9VciQk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AdlOkbtS; arc=none smtp.client-ip=209.85.219.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AdlOkbtS" Received: by mail-qv1-f44.google.com with SMTP id 6a1803df08f44-6928a5e2479so17058406d6.0; Fri, 22 Mar 2024 16:39:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711150749; x=1711755549; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:from:to:cc:subject :date:message-id:reply-to; bh=CMsNwHviwi/fcgZpZ6hEf/r33sW+GCsEsvaN1TfyKyc=; b=AdlOkbtS9FlzFTs+QhsReJzLVn4+/Vb1xWKnOmnQkzYQjKUlggmFFfXzjD8SCSaPSp sF7wGOFtwrirgftUWJguIj+y4szt5Qgp5IGGZIGkG8RokGxGVn8mOvez7Ae9zsd9u8tJ 14LIGDiH7JVpMrnge/4wPmTyKwefn50tuTlWeS94GYokENJwrkXda6CPLAG5vSo6HQ4q oIxmuE4UWln7ij9ZeVTZNPyLoAkmln2ywqQMsVCvE4dRM3nUQot8KrGZIOd8pbHsx31k tqpjzyWMiC1aAVvbj5+DZN1IIUt5I7iHxNr93A5asY/0aIEsZivx4zewi8xhaUlgyTU5 KdEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711150749; x=1711755549; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=CMsNwHviwi/fcgZpZ6hEf/r33sW+GCsEsvaN1TfyKyc=; b=ENtbfylMSZBVSbpdM841TmvmwGjiWjQ4SuQOf87TY2Pk8VmiHVg1XL6XPU7TZLOXqU luoHJGOV+RlXKd9fZCaLhCets7J9G0OwlU1Qxbmqo3dJtejO/Zb520DFB2DGQmGE9RHR OjAe2CLtfLRsFa/3qxi6wTyHnAx8QYl/V2Gs0A9Buapo3KT9/Yl6PkW/szrdmOV1kkBa QydxcS/+088avNoDTUdgVPZtcfRa/kSo18xves0ggqoSb23+PZLU5To8RZ1aC0eTz8BF RF5EhqPKl0sO2sBuVpDab8Rbkjl0own729SYAEqhk/R4J3RRvNBoflnvHUwVR7+SR3vk /mPQ== X-Forwarded-Encrypted: i=1; AJvYcCWf65NoB0W3pF+jfMXDT3EtNlWXukYixo6BNOwKVZp2Msj09T4x12HX8gxV2ojqdbl9YgG7Zlrzikj7MVjqaPCG8pNST93irDGsIl6aAmcV19SaROBnlxC/09sv8biVPRWbTohBizECi5yH7YIlE9PCLopHUxSy7bGMxmIvrBkz2dbDNLtTHSc= X-Gm-Message-State: AOJu0Yzi6vQwbXwsxfR45YqrVa+b9NCXe2LxsMb6FXOh4KoBVCx4BFRJ hKLwKR2sk01u738orOWG1oI44eVWavZZuPdEx+CwGslhrykuXhAg X-Google-Smtp-Source: AGHT+IHZ+NuIYKcs3sHrLWDGnicqM7RhyyK/vWc/KL6bEIvyatErGn/eE74OoTUwRrPjNflkhje49A== X-Received: by 2002:ad4:5ba9:0:b0:696:1ffd:a32c with SMTP id 9-20020ad45ba9000000b006961ffda32cmr714687qvq.31.1711150749485; Fri, 22 Mar 2024 16:39:09 -0700 (PDT) Received: from fauth1-smtp.messagingengine.com (fauth1-smtp.messagingengine.com. [103.168.172.200]) by smtp.gmail.com with ESMTPSA id mx17-20020a0562142e1100b0069679a663e6sm105329qvb.21.2024.03.22.16.39.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Mar 2024 16:39:09 -0700 (PDT) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfauth.nyi.internal (Postfix) with ESMTP id 2EC3C1200032; Fri, 22 Mar 2024 19:39:08 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 22 Mar 2024 19:39:08 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddtfedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepuehoqhhu nhcuhfgvnhhguceosghoqhhunhdrfhgvnhhgsehgmhgrihhlrdgtohhmqeenucggtffrrg htthgvrhhnpeegleejiedthedvheeggfejveefjeejkefgveffieeujefhueeigfegueeh geeggfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gsohhquhhnodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhithihqdeiledvgeehtdei gedqudejjeekheehhedvqdgsohhquhhnrdhfvghngheppehgmhgrihhlrdgtohhmsehfih igmhgvrdhnrghmvg X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 22 Mar 2024 19:39:06 -0400 (EDT) From: Boqun Feng To: rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, llvm@lists.linux.dev Cc: Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , Akira Yokosawa , Daniel Lustig , Joel Fernandes , Nathan Chancellor , Nick Desaulniers , kent.overstreet@gmail.com, Greg Kroah-Hartman , elver@google.com, Mark Rutland , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Catalin Marinas , torvalds@linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [WIP 1/3] rust: Introduce atomic module Date: Fri, 22 Mar 2024 16:38:36 -0700 Message-ID: <20240322233838.868874-2-boqun.feng@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240322233838.868874-1-boqun.feng@gmail.com> References: <20240322233838.868874-1-boqun.feng@gmail.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Although Rust has its own memory ordering model (in the standard C++ memory model), having two models is not wise to start with: it increases the difficulty for correctness reasoning. Since we use Linux Kernel Memory Model for C code in kernel, it makes sense that Rust code also uses LKMM, therefore introduce a module to provide LKMM atomic primitives. Signed-off-by: Boqun Feng --- rust/kernel/sync.rs | 1 + rust/kernel/sync/atomic.rs | 42 ++++++++++++++++++++++++++++ rust/kernel/sync/atomic/arch.rs | 9 ++++++ rust/kernel/sync/atomic/arch/x86.rs | 43 +++++++++++++++++++++++++++++ 4 files changed, 95 insertions(+) create mode 100644 rust/kernel/sync/atomic.rs create mode 100644 rust/kernel/sync/atomic/arch.rs create mode 100644 rust/kernel/sync/atomic/arch/x86.rs diff --git a/rust/kernel/sync.rs b/rust/kernel/sync.rs index c983f63fd56e..dc2d26712f26 100644 --- a/rust/kernel/sync.rs +++ b/rust/kernel/sync.rs @@ -8,6 +8,7 @@ use crate::types::Opaque; mod arc; +pub mod atomic; mod condvar; pub mod lock; mod locked_by; diff --git a/rust/kernel/sync/atomic.rs b/rust/kernel/sync/atomic.rs new file mode 100644 index 000000000000..280040705fb0 --- /dev/null +++ b/rust/kernel/sync/atomic.rs @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Atomic and barrier primitives. +//! +//! These primitives should have the same semantics as their C counterparts, for precise definitions +//! of the semantics, please refer to tools/memory-model. Note that Linux Kernel Memory +//! (Consistency) Model is the only model for Rust development in kernel right now, please avoid to +//! use Rust's own atomics. + +use core::cell::UnsafeCell; + +mod arch; + +/// An atomic `i32`. +pub struct AtomicI32(pub(crate) UnsafeCell); + +impl AtomicI32 { + /// Creates a new atomic value. + pub fn new(v: i32) -> Self { + Self(UnsafeCell::new(v)) + } + + /// Adds `i` to the atomic variable with RELAXED ordering. + /// + /// Returns the old value before the add. + /// + /// # Example + /// + /// ```rust + /// use kernel::sync::atomic::AtomicI32; + /// + /// let a = AtomicI32::new(0); + /// let b = a.fetch_add_relaxed(1); + /// let c = a.fetch_add_relaxed(2); + /// + /// assert_eq!(b, 0); + /// assert_eq!(c, 1); + /// ``` + pub fn fetch_add_relaxed(&self, i: i32) -> i32 { + arch::i32_fetch_add_relaxed(&self.0, i) + } +} diff --git a/rust/kernel/sync/atomic/arch.rs b/rust/kernel/sync/atomic/arch.rs new file mode 100644 index 000000000000..3eb5a103a69a --- /dev/null +++ b/rust/kernel/sync/atomic/arch.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Architectural atomic and barrier primitives. + +#[cfg(CONFIG_X86)] +pub(crate) use x86::*; + +#[cfg(CONFIG_X86)] +pub(crate) mod x86; diff --git a/rust/kernel/sync/atomic/arch/x86.rs b/rust/kernel/sync/atomic/arch/x86.rs new file mode 100644 index 000000000000..2d715f740b22 --- /dev/null +++ b/rust/kernel/sync/atomic/arch/x86.rs @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! x86 implementation for atomic and barrier primitives. + +use core::arch::asm; +use core::cell::UnsafeCell; + +/// Generates an instruction with "lock" prefix. +#[cfg(CONFIG_SMP)] +macro_rules! lock_instr { + ($i:literal) => { concat!("lock; ", $i) } +} + +#[cfg(not(CONFIG_SMP))] +macro_rules! lock_instr { + ($i:literal) => { $i } +} + +/// Atomically exchanges and adds `i` to `*v` in a wrapping way. +/// +/// Return the old value before the addition. +/// +/// # Safety +/// +/// The caller need to make sure `v` points to a valid `i32`. +unsafe fn i32_xadd(v: *mut i32, mut i: i32) -> i32 { + // SAFETY: Per function safety requirement, the address of `v` is valid for "xadd". + unsafe { + asm!( + lock_instr!("xaddl {i:e}, ({v})"), + i = inout(reg) i, + v = in(reg) v, + options(att_syntax, preserves_flags), + ); + } + + i +} + +pub(crate) fn i32_fetch_add_relaxed(v: &UnsafeCell, i: i32) -> i32 { + // SAFETY: `v.get()` points to a valid `i32`. + unsafe { i32_xadd(v.get(), i) } +} From patchwork Fri Mar 22 23:38:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boqun Feng X-Patchwork-Id: 13600494 Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7621082D87; Fri, 22 Mar 2024 23:39:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711150754; cv=none; b=utdqi50FSCY7+zWwLQjpshNnJQgnS/t8DqYz0zSuAHP+Lrd8QYbEG4QElSMtWIa58/go6IqOVD08nrrRS9+oDn2W66y38IBkFmFlYs8Ef7GMkAbIAtCtrXKZOapePeNR8jg1zTHacSEw4V6IUdQGJVpLuNSz0MjMHZ7S38jML6A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711150754; c=relaxed/simple; bh=/NBZoIhK1CtetohxAcHUpmJsAGut/+ZBo1KxCgXiFn0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aRH2mBmkL/SD6/q9w4zT4zTatNm7gc8SVrxuWv2k9uCGBo45QgZVtG4w8cBTyNXjuYLjlQbM2Az+uUd40tYMu1hXJOSFMpwNtxiAD8thoioJBN/4NwB+Kw1q1vPBTBzvve3xRogUEZquF5rj4VJKChpdLHST2C3WfaFZZMxDSxc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CfeeYC1l; arc=none smtp.client-ip=209.85.222.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CfeeYC1l" Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-789e2bf854dso165694885a.3; Fri, 22 Mar 2024 16:39:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711150751; x=1711755551; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:from:to:cc:subject :date:message-id:reply-to; bh=/F96a8YELf+Ur72xFbmW7ZJSVd3YYBPROndGsfciYiA=; b=CfeeYC1ldfisFBhiiBFKBGYcJQ0/SaYb52QkB5VrW3FFBNoe5ZQKWpb+cgVuN9SMpq cbqBCAjNBoI3FwtE6l6YPqS5kn8J8ZoZvIFw5/GyH9Y1H11FmYRrFn1+6buzBOk4/pQO Cn87gJU4kQo3hszZ50B7kkZLp808EJNTXXCCzSbc28WAW7Oz+vq6YodsbAHOifrKftR/ 7OteAVDnIvwrK2iQqflVbZD8y91HU07d0zXX6VkEgTQ9lOl3KVJ87bCt7+kqkIRpu7zg +3bkdFf2ioFf05ycKsn18QLeV4FJJxVEsm/gSFa3JTKeHZxSMcqH+pAVb9kEd7468y/K M41Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711150751; x=1711755551; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=/F96a8YELf+Ur72xFbmW7ZJSVd3YYBPROndGsfciYiA=; b=xQIXwRH73q9uOKtNjh5MU5TzOwaqQiMmeez4E81KeQoVger+sZqxPhiv0ONKC5IiIV wEPzntEalC0Q5vcs6+v5UgUVKPZgEt667h1Kvo8XO2aJRCQMtdSK1eF1WC233GGVOVCI XSHVHA6w1RCVsXE7sfmPX+UvzMf2sccP10aCOs58jxuZBcE0wUiYnRikRsDfV1fGx1nq wf0m4lGXCbI9MQ/sMgDtAMpYZFoiUo0ZsPBEyby+A90j9ox5ZR3eLeZvJgL5LMo2XO2M Bijcbm2a01rcErPgRVPjSwDzd/0DEOOZLLF+lW+GJHn/mdlN/e321nqZfFzavSrFEbCy vs/w== X-Forwarded-Encrypted: i=1; AJvYcCUXnS+JIkTfNrC1askxqs+HFDSs2CKGjQlwhIH51ZGce+/H2syh248ZQFb/MAX+ZSfqnT92RrV955bMuREgIeEVOtDIcHUnOZOPszezCDeYozYYzTZ/23UZEVU+BB48AqaexZzLweJZLKpLFbTQjKKLmDC8KOwbDEOKlpSnVqN5tB9xvF1vvIE= X-Gm-Message-State: AOJu0YwIKj9aRGPO0bHrUjb3dU2TlOcLNYDSOCoxL4VL59knndzBqO5k 2eOPyPucOUx3FadPEHKlJwYY0c9jl1OaIn9BtZpwJWAABNw4UwHl X-Google-Smtp-Source: AGHT+IFnhi7P+k/MkZCWrllad1of00qh69/aWCRn2Ch4hXaX74mXDXtJDyquVhXLLDKOeEFx3KqBoA== X-Received: by 2002:a05:620a:40d3:b0:78a:3fb1:bade with SMTP id g19-20020a05620a40d300b0078a3fb1bademr1295433qko.1.1711150751298; Fri, 22 Mar 2024 16:39:11 -0700 (PDT) Received: from auth1-smtp.messagingengine.com (auth1-smtp.messagingengine.com. [66.111.4.227]) by smtp.gmail.com with ESMTPSA id cz52-20020a05620a36f400b0078a43996ab7sm267608qkb.1.2024.03.22.16.39.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Mar 2024 16:39:10 -0700 (PDT) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailauth.nyi.internal (Postfix) with ESMTP id E694E27C005B; Fri, 22 Mar 2024 19:39:09 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Fri, 22 Mar 2024 19:39:09 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddtfedgudegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepuehoqhhu nhcuhfgvnhhguceosghoqhhunhdrfhgvnhhgsehgmhgrihhlrdgtohhmqeenucggtffrrg htthgvrhhnpeegleejiedthedvheeggfejveefjeejkefgveffieeujefhueeigfegueeh geeggfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gsohhquhhnodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhithihqdeiledvgeehtdei gedqudejjeekheehhedvqdgsohhquhhnrdhfvghngheppehgmhgrihhlrdgtohhmsehfih igmhgvrdhnrghmvg X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 22 Mar 2024 19:39:08 -0400 (EDT) From: Boqun Feng To: rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, llvm@lists.linux.dev Cc: Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , Akira Yokosawa , Daniel Lustig , Joel Fernandes , Nathan Chancellor , Nick Desaulniers , kent.overstreet@gmail.com, Greg Kroah-Hartman , elver@google.com, Mark Rutland , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Catalin Marinas , torvalds@linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [WIP 2/3] rust: atomic: Add ARM64 fetch_add_relaxed() Date: Fri, 22 Mar 2024 16:38:37 -0700 Message-ID: <20240322233838.868874-3-boqun.feng@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240322233838.868874-1-boqun.feng@gmail.com> References: <20240322233838.868874-1-boqun.feng@gmail.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Signed-off-by: Boqun Feng --- rust/kernel/sync/atomic/arch.rs | 6 ++++++ rust/kernel/sync/atomic/arch/arm64.rs | 26 ++++++++++++++++++++++++++ 2 files changed, 32 insertions(+) create mode 100644 rust/kernel/sync/atomic/arch/arm64.rs diff --git a/rust/kernel/sync/atomic/arch.rs b/rust/kernel/sync/atomic/arch.rs index 3eb5a103a69a..fc280f229237 100644 --- a/rust/kernel/sync/atomic/arch.rs +++ b/rust/kernel/sync/atomic/arch.rs @@ -5,5 +5,11 @@ #[cfg(CONFIG_X86)] pub(crate) use x86::*; +#[cfg(CONFIG_ARM64)] +pub(crate) use arm64::*; + #[cfg(CONFIG_X86)] pub(crate) mod x86; + +#[cfg(CONFIG_ARM64)] +pub(crate) mod arm64; diff --git a/rust/kernel/sync/atomic/arch/arm64.rs b/rust/kernel/sync/atomic/arch/arm64.rs new file mode 100644 index 000000000000..438f37cf7df6 --- /dev/null +++ b/rust/kernel/sync/atomic/arch/arm64.rs @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! ARM64 implementation for atomic and barrier primitives. + +use core::arch::asm; +use core::cell::UnsafeCell; + +pub(crate) fn i32_fetch_add_relaxed(v: &UnsafeCell, i: i32) -> i32 { + let mut result; + unsafe { + asm!( + "prfm pstl1strm, [{v}]", + "1: ldxr {result:w}, [{v}]", + "add {val:w}, {result:w}, {i:w}", + "stxr {tmp:w}, {val:w}, [{v}]", + "cbnz {tmp:w}, 1b", + result = out(reg) result, + tmp = out(reg) _, + val = out(reg) _, + v = in(reg) v.get(), + i = in(reg) i, + ) + } + + result +} From patchwork Fri Mar 22 23:38:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boqun Feng X-Patchwork-Id: 13600495 Received: from mail-vk1-f175.google.com (mail-vk1-f175.google.com [209.85.221.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA396839FE; Fri, 22 Mar 2024 23:39:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711150755; cv=none; b=kf4p56SC2vDfAXSMZtTj7YDWxpRp8sLiUyUjz16kVNinKWsqJ6gDsAoiU2HGH/xfF5aD7dbQR3SDY1h62YEockHseXRaXBSa2qN70w57M3PCWow+2WFwenbjjq3mHGY9BY/qs3LP7PMnAD/vgYSV2ELxuqC2j8oQp67ohSnYrUc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711150755; c=relaxed/simple; bh=qQ5oagRUQmpb8GABKAqxfq/Tg/yBhJf6ca7+7tUxq/A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CQhKln5RHcYqZp/e9XQbLsUEwsF/wB7islcXZmlVYgDWejEf8yWR8tt47p2jD01umcUUBwZxf5tR0Ky5myC4Wp3k7RnY4rwElOaolcSrNOLznjN2H1d/+Bv49LGIUV6qIU6Pq7eu5fZm3nsJYwM2/f/k8lO/6cCrWn6UkEAwV9A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dDMwLfMX; arc=none smtp.client-ip=209.85.221.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dDMwLfMX" Received: by mail-vk1-f175.google.com with SMTP id 71dfb90a1353d-4d453ae6af5so1246455e0c.2; Fri, 22 Mar 2024 16:39:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711150753; x=1711755553; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:from:to:cc:subject :date:message-id:reply-to; bh=QZ2rrwnnSPuoyWFM6/wbqSx+XEHGNK8//eAAMbn2SLU=; b=dDMwLfMXrAIntfyxsD7U/i+CyLeVvop5HCkJUXDxZ3E5Ox4MA2QEwrfgbc3hOnaRQ0 8P0NDpDw3HNqDhsCLGLqoemApY2JZ5SkKujRkkNtQYGdbXi2EDoqpz8XqH6vu66H3VXq S/AN1WUsXKkmeupMb34U2zafIg0KFtox6/XDck1h7smrwW8m6hty48gwJJb0Jli9PeB7 cXhIfkeGM6wh1CCk8VfMQcJ991+RSPDSDGvAYHg6FMhfnacQ/5Ni4QLBoeb24gTEp5wt +8nKdEBtEQEQTQ3OUNiN+gv5GEzML0GDpaOH6wpJ9OMLtKugkQOkixyT8xxpS7S6w27u Okqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711150753; x=1711755553; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=QZ2rrwnnSPuoyWFM6/wbqSx+XEHGNK8//eAAMbn2SLU=; b=JNSsPCtmJHpsVFsB4T2qqgIICax3EvoqFll/5Pa8jGhDc3z4tVuItZKm+VS4R81gqd zY86LvZiimkjsouOCALItWaqGwVwodSY2x6+43vqPFoNjKntjPVuOyG0omCnbIMhU8cW wdXZfQo9dB1HNZHzogZv3fzaBlgtSQXz7D3J4n3Be79EtE/M0DI7c7xPu4Jy722RWxVW lpO8oaiVvpv8emYQ/fEGmbqcDCertCU00gPKiO0lE50Z90azATaO3kNH6f4pI6U+8eLH l4JiwlLBKE0UjeikgQKy7AVN7GJ2uSzIUMp3PE2FvJ6I2pIqyV4Rfx5Jafy7mePJ6oCz j1EA== X-Forwarded-Encrypted: i=1; AJvYcCX0jY7H+lU535PTaP5PORM00USAbDk30DNgMhZ+ij6UMdRglItTr2FZHqt68qWe+541Lw5aj5c7zZKu7DJdU1jA/R+vOA5gkhwpdcJpYsYtORDuHCkmunC+ecJ1kHu23BQhvMqZ5dp0CdkbSJMtcDEobp8Vk/4qSX9Jx0IO9nBtT0Lj++3606A= X-Gm-Message-State: AOJu0Yz6CI/ekBopSa/bV0wb4BjPaHfITxfZ8mfcSxTOfjTU1CatVxQS bevk+peHQuMKN4q4AAc/6PAMBBCdMdptjyPZ+4i+K2qb9Ba4HX8S X-Google-Smtp-Source: AGHT+IEaYQaJaeCZmztaSPVyvy8V/Vxeh4oBqEj4kwmvUweyQop95DXCqfqCPm+ZQJ2lS/7MtVUgAg== X-Received: by 2002:a05:6122:c9f:b0:4cb:fc25:7caa with SMTP id ba31-20020a0561220c9f00b004cbfc257caamr1351039vkb.14.1711150752858; Fri, 22 Mar 2024 16:39:12 -0700 (PDT) Received: from fauth1-smtp.messagingengine.com (fauth1-smtp.messagingengine.com. [103.168.172.200]) by smtp.gmail.com with ESMTPSA id dd7-20020ad45807000000b0069675909982sm456994qvb.70.2024.03.22.16.39.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Mar 2024 16:39:12 -0700 (PDT) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfauth.nyi.internal (Postfix) with ESMTP id 98CB91200032; Fri, 22 Mar 2024 19:39:11 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 22 Mar 2024 19:39:11 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddtfedgudegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepuehoqhhu nhcuhfgvnhhguceosghoqhhunhdrfhgvnhhgsehgmhgrihhlrdgtohhmqeenucggtffrrg htthgvrhhnpeegleejiedthedvheeggfejveefjeejkefgveffieeujefhueeigfegueeh geeggfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gsohhquhhnodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhithihqdeiledvgeehtdei gedqudejjeekheehhedvqdgsohhquhhnrdhfvghngheppehgmhgrihhlrdgtohhmsehfih igmhgvrdhnrghmvg X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 22 Mar 2024 19:39:10 -0400 (EDT) From: Boqun Feng To: rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, llvm@lists.linux.dev Cc: Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , Akira Yokosawa , Daniel Lustig , Joel Fernandes , Nathan Chancellor , Nick Desaulniers , kent.overstreet@gmail.com, Greg Kroah-Hartman , elver@google.com, Mark Rutland , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Catalin Marinas , torvalds@linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [WIP 3/3] rust: atomic: Add fetch_sub_release() Date: Fri, 22 Mar 2024 16:38:38 -0700 Message-ID: <20240322233838.868874-4-boqun.feng@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240322233838.868874-1-boqun.feng@gmail.com> References: <20240322233838.868874-1-boqun.feng@gmail.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Signed-off-by: Boqun Feng --- rust/kernel/sync/atomic.rs | 23 +++++++++++++++++++++++ rust/kernel/sync/atomic/arch/arm64.rs | 20 ++++++++++++++++++++ rust/kernel/sync/atomic/arch/x86.rs | 5 +++++ 3 files changed, 48 insertions(+) diff --git a/rust/kernel/sync/atomic.rs b/rust/kernel/sync/atomic.rs index 280040705fb0..c3cae0d25e88 100644 --- a/rust/kernel/sync/atomic.rs +++ b/rust/kernel/sync/atomic.rs @@ -39,4 +39,27 @@ pub fn new(v: i32) -> Self { pub fn fetch_add_relaxed(&self, i: i32) -> i32 { arch::i32_fetch_add_relaxed(&self.0, i) } + + /// Subs `i` to the atomic variable with RELEASE ordering. + /// + /// Returns the old value before the sub. + /// + /// # Example + /// + /// ```rust + /// use kernel::sync::atomic::AtomicI32; + /// + /// let a = AtomicI32::new(1); + /// let b = a.fetch_sub_release(1); + /// let c = a.fetch_sub_release(2); + /// let d = a.fetch_sub_release(3); + /// let e = a.fetch_sub_release(core::i32::MIN); + /// + /// assert_eq!(b, 1); + /// assert_eq!(c, 0); + /// assert_eq!(d, -2); + /// ``` + pub fn fetch_sub_release(&self, i: i32) -> i32 { + arch::i32_fetch_sub_release(&self.0, i) + } } diff --git a/rust/kernel/sync/atomic/arch/arm64.rs b/rust/kernel/sync/atomic/arch/arm64.rs index 438f37cf7df6..beea77ecdb20 100644 --- a/rust/kernel/sync/atomic/arch/arm64.rs +++ b/rust/kernel/sync/atomic/arch/arm64.rs @@ -24,3 +24,23 @@ pub(crate) fn i32_fetch_add_relaxed(v: &UnsafeCell, i: i32) -> i32 { result } + +pub(crate) fn i32_fetch_sub_release(v: &UnsafeCell, i: i32) -> i32 { + let mut result; + unsafe { + asm!( + "prfm pstl1strm, [{v}]", + "1: ldxr {result:w}, [{v}]", + "sub {val:w}, {result:w}, {i:w}", + "stlxr {tmp:w}, {val:w}, [{v}]", + "cbnz {tmp:w}, 1b", + result = out(reg) result, + tmp = out(reg) _, + val = out(reg) _, + v = in(reg) v.get(), + i = in(reg) i, + ) + } + + result +} diff --git a/rust/kernel/sync/atomic/arch/x86.rs b/rust/kernel/sync/atomic/arch/x86.rs index 2d715f740b22..7f764cde4576 100644 --- a/rust/kernel/sync/atomic/arch/x86.rs +++ b/rust/kernel/sync/atomic/arch/x86.rs @@ -41,3 +41,8 @@ pub(crate) fn i32_fetch_add_relaxed(v: &UnsafeCell, i: i32) -> i32 { // SAFETY: `v.get()` points to a valid `i32`. unsafe { i32_xadd(v.get(), i) } } + +pub(crate) fn i32_fetch_sub_release(v: &UnsafeCell, i: i32) -> i32 { + // SAFETY: `v.get()` points to a valid `i32`. + unsafe { i32_xadd(v.get(), i.wrapping_neg()) } +}