From patchwork Wed Oct 30 16:04:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13856821 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D206C2139DB for ; Wed, 30 Oct 2024 16:04:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304301; cv=none; b=Rl81UwHrlR+F38laIkqU3okKqQLvOknyZpT0gzEbDouQci8f4Uuk0OT6AHy89x/hgKffjwP4/9UdN1lmQ/pXJCQ0I8ml3cLy1EQmXbfxyqoPnMt4JCqo6HW9LCVHFXBL4KOWaA2qcsZ+1in47Kia146Gnbgz7xxQWskDd5xgQzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304301; c=relaxed/simple; bh=XID7x+7coWgnGBd5wIl/yqAS/1AzuHdHb0QK8txVMPI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VZ1SxG5ZTq8nGYgK7mPe2eVfItl3P5bG+9Lj8yogdZwv2x/Tb2qXxRNeosLxGcssvDnSEk/TSv2+p28uODFnNNrB9z0MAO8HOBz5D/3DThqYNoG+9w+MYT9FCL/7sS2HdQvdv6bMYRGMsqfq4dqgHNMQ5W/PTejnBV4hsB6csYM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cBdDCRdX; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cBdDCRdX" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43157cff1d1so48814565e9.2 for ; Wed, 30 Oct 2024 09:04:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1730304297; x=1730909097; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uZ5bBsDzzqub9ZaWqauRG4tAucTaa9wcfj1E33xP3xc=; b=cBdDCRdXZxHP8hRCvT71eaRnlgkNgevqUA/Mf26Y68AaLnIGuslkR9rabnvQQzr5Y4 SA9XWwtaXAc/ARVQ6d7I4xbXuQ1okAeormt0/ysJyZd+cr+C1ChHwg4qEzWnNKxh/v19 nfRp2umk9K9b6lTVWP3BZjL7Oz+zkFtvlvv62D+FRw8cWZPTcNEUeflx3mOscNlKXi6o aIkH6bcO7PgEThnbToQvBRijBaKtXDlhfWn+cUbr7k1FpDWfbvSIQvLJ8o3bfVvDkNr4 VZu0cFUStb7PkdDnxBaa9fS0sDbDTpPRF6PkZfhxxe7lWShL9PDQRAFgZxYHH7LwVsyR g/2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730304297; x=1730909097; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uZ5bBsDzzqub9ZaWqauRG4tAucTaa9wcfj1E33xP3xc=; b=G1WUhA0IGZJxPbiXmf40MeZsSas3Yhi6Fs/qmWTrr6If8QSiLFx6UXzt5A6pCEEA16 E0E9PEulmwT72VS4D+GJk7jzAGFQC6/8cVeTGyZuC4hEqzijtV6RRCkIGV2/X5l2aweA rjWpin+Cs9If50e9VHcYx9FylCFySqvNZEYY5w3ISzdcI7tNQ4r336jwsh2v5xRs2KzX INvnCHxLWhFawhRUKPgG5Dz+z5fGNWsETuTOxhY+HBNUkvHOHUx8Fba6M+F8R7gJ9JFS DZSwti3P+bULDtzisoqw1qwsh7OZWmFSnsoxStXknrXOQNllXwaOaeEQjdHZI354q/eI O3Cw== X-Gm-Message-State: AOJu0Yz2qbvAyKUzZf6NAeji9DkneARxpQYgzQ4e2Zl0x+4F1r2On7E2 +WUIBNz0urGEdPsWFVXjDidbMHwTgxIqEJ9kRHtsv5jLHd0q+6+AfwlssMRgeA3vGgPjhd59Hzt XZbs9c73OdvSQLQ== X-Google-Smtp-Source: AGHT+IFVYJyJpHqXosREWKNk401dPodMtz/gzxt4mu2tfZOK6kHkRJP3u08JeJC4s0ELkJy7XaI8OGX/S3j3MuY= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a05:600c:6a99:b0:431:4509:696a with SMTP id 5b1f17b1804b1-4327b6f8619mr645e9.2.1730304297194; Wed, 30 Oct 2024 09:04:57 -0700 (PDT) Date: Wed, 30 Oct 2024 16:04:24 +0000 In-Reply-To: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3944; i=aliceryhl@google.com; h=from:subject:message-id; bh=XID7x+7coWgnGBd5wIl/yqAS/1AzuHdHb0QK8txVMPI=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnIlkhPSPbRePqgmcZkeV4PnRFaF06ORWPyb4Hd ZssKOLJjQqJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZyJZIQAKCRAEWL7uWMY5 RqSUD/9jfyehgB5qzKfR0aAlCXJYAKb9ZzLWaNn1EmqHjp8saPikwqUESI5omMnEt6W8h+4ok42 pV4kX4p2S2qQqcKff9S5dttdXhdH2JZOmIXfwdu5yXJPN7oj5BpzCol3VvJux1Nx3JaScViKnG4 4e0udKPwtlAyubQDD/+0Uk/9ayGweHYNvknT+8hoyjxsNDfCl5mTdR5pa3yXcSQMfXN4Ck+MOQh VD91Vo5mSBpn2DItkLSUPSKgIP9aK7q4yXIaKLf9WgPwcsdzNxuJ/Ak1+IWPdqzEjp0hiq67KkE hmiSAknb+A1aOfNv9gGpdRM+AuuoXsLOnhD5/yv+0utf5lP32ktyNBPq4IoJMANrT0kqkJ+nAIu gs5Eut6gmmuj9FgL+wjElrMboABY6QE/SENYeDZe4xLaDnxQKSqfvem36mOcMTP7j/02/GHoSYB RyLsXv/rPzX0s467Y++J+vePxJuIrQFgS8PVcnA/8dnKfeoU+kE4buLiFk6G/HoS4dxGyqiYbF3 HMltLodkWAYZfs+nfTY3uUa5C7nDIZ+/QXJMyQ2VmpXgulpY7tPclpC1MxbFNAiPxW9jO1qVqOS GF2ERKm5fU5AwBn6KmQYgLsii6hIO/c+lfx1ndbl1K1zCZ0cC906aD5HA9HZrjMJ1EsMzzSVauC RVr59LvwW7YPQNA== X-Mailer: b4 0.13.0 Message-ID: <20241030-tracepoint-v12-1-eec7f0f8ad22@google.com> Subject: [PATCH v12 1/5] rust: add static_branch_unlikely for static_key_false From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl Add just enough support for static key so that we can use it from tracepoints. Tracepoints rely on `static_branch_unlikely` with a `struct static_key_false`, so we add the same functionality to Rust. This patch only provides a generic implementation without code patching (matching the one used when CONFIG_JUMP_LABEL is disabled). Later patches add support for inline asm implementations that use runtime patching. When CONFIG_JUMP_LABEL is unset, `static_key_count` is a static inline function, so a Rust helper is defined for `static_key_count` in this case. If Rust is compiled with LTO, this call should get inlined. The helper can be eliminated once we have the necessary inline asm to make atomic operations from Rust. Reviewed-by: Boqun Feng Reviewed-by: Gary Guo Signed-off-by: Alice Ryhl --- rust/bindings/bindings_helper.h | 1 + rust/helpers/helpers.c | 1 + rust/helpers/jump_label.c | 14 ++++++++++++++ rust/kernel/jump_label.rs | 30 ++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 1 + 5 files changed, 47 insertions(+) diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index ae82e9c941af..e0846e7e93e6 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c index 30f40149f3a9..17e1b60d178f 100644 --- a/rust/helpers/helpers.c +++ b/rust/helpers/helpers.c @@ -12,6 +12,7 @@ #include "build_assert.c" #include "build_bug.c" #include "err.c" +#include "jump_label.c" #include "kunit.c" #include "mutex.c" #include "page.c" diff --git a/rust/helpers/jump_label.c b/rust/helpers/jump_label.c new file mode 100644 index 000000000000..fc1f1e0df08e --- /dev/null +++ b/rust/helpers/jump_label.c @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright (C) 2024 Google LLC. + */ + +#include + +#ifndef CONFIG_JUMP_LABEL +int rust_helper_static_key_count(struct static_key *key) +{ + return static_key_count(key); +} +#endif diff --git a/rust/kernel/jump_label.rs b/rust/kernel/jump_label.rs new file mode 100644 index 000000000000..4b7655b2a022 --- /dev/null +++ b/rust/kernel/jump_label.rs @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2024 Google LLC. + +//! Logic for static keys. +//! +//! C header: [`include/linux/jump_label.h`](srctree/include/linux/jump_label.h). + +/// Branch based on a static key. +/// +/// Takes three arguments: +/// +/// * `key` - the path to the static variable containing the `static_key`. +/// * `keytyp` - the type of `key`. +/// * `field` - the name of the field of `key` that contains the `static_key`. +/// +/// # Safety +/// +/// The macro must be used with a real static key defined by C. +#[macro_export] +macro_rules! static_branch_unlikely { + ($key:path, $keytyp:ty, $field:ident) => {{ + let _key: *const $keytyp = ::core::ptr::addr_of!($key); + let _key: *const $crate::bindings::static_key_false = ::core::ptr::addr_of!((*_key).$field); + let _key: *const $crate::bindings::static_key = _key.cast(); + + $crate::bindings::static_key_count(_key.cast_mut()) > 0 + }}; +} +pub use static_branch_unlikely; diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index b5f4b3ce6b48..708ff817ccc3 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -36,6 +36,7 @@ pub mod firmware; pub mod init; pub mod ioctl; +pub mod jump_label; #[cfg(CONFIG_KUNIT)] pub mod kunit; pub mod list; From patchwork Wed Oct 30 16:04:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13856822 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A31BC2141DA for ; Wed, 30 Oct 2024 16:05:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304303; cv=none; b=hm6GqZXSpmUrKPrtFOKDzXXKnLAnvk2Iq/SO+7MysxFFim3FxZa8apmcILkwbkO+c+Me35BJ68IxQ5SxXYQ4e4aWtIQCjZKzJdf7yHKz6cWrxafJyTZGLl+skAeXQZvl2lQ95cEGxhT0O3sapNgQekoF6tF6kMbrnz3ullomT3I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304303; c=relaxed/simple; bh=7megp7qpFwj2d67QvZOgzlG5lXxVSDo6vxXF2gHW/aA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=upWJF4GybH64lMRCoe2m9Z8KbreMPozQQgTaHATJjBlceS1suuWIaW3UJrmrDWrm71pcAcdehNZX3fpIl3rUSv/kxqF/iOeFPhffEFVAVcX1MrUbCidedBY93wbGUqLb0fUrz46M923JbgxNutlE2sBiENqbz8UzRHvaHkpQpfU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VXyA4rLi; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VXyA4rLi" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e9e897852fso80330547b3.3 for ; Wed, 30 Oct 2024 09:05:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1730304300; x=1730909100; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KfKLqMwW2mmYwxXsPR323ofDuLblSEcxc3pN1/E4KgU=; b=VXyA4rLie/nq/cDL2oAkxp4iBtcHbddf1mDmVcugjLqa72+gWNhuxB/8q7mQoUoXeX Yyh1pe9pPuqlBJ8HMpHVT66h3xymFTY1Y7KTYaYnrR4l13zHz9nJQb86EPY2/piqPpcM 1SKTbDmNGh/Bq9MCUkiVv4YRZ5Sk9TAcJtVecQfyuwfCiE0QvcCAeyqG8aXpnykZovaZ JlIw4ZLJUCmw2cYqKQhuxB6jD95CV8HyVHPPK1eU/63YLBOa5qxdY3hjOMjsbVqiQ4Dn liaFqovm45/X51mAbe7hmMRak37Cvxp8KpF3jotIm+MDsS9nRkBPBhQtFGvy+aAqYHJ/ Dg/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730304300; x=1730909100; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KfKLqMwW2mmYwxXsPR323ofDuLblSEcxc3pN1/E4KgU=; b=LFCdZxpAt53h8oFHkZzK+0eoej/iSLKP81A2CUtYLVe0ibUGWRLDMmIrH3lBt2xt6z 94UuZlZo0gbjTBnjIMIEWcO3t3//L7sfqpW4hBRvvGbJuubQx2mPWaa7JPlpCwr74C9a fHqU7ZGYyDvJ6dqP8fqlot9be0jbk36oPeEnG9IdLXn0YHXLQikYTu9MGEfAyhGB0hIt J1t9x1ICaI7slZ/WqnElagAdiDYHpxHgAs1yVdZ3/DNxNxkeiQXYRIpKDmrRS+oFuLkN ZZHox/UaozkhrQlV97smQzsEGUPLsI//XxtN+8Bs2Dk24Wh136Z3JQIphC0I/Z4MDKRz UJmg== X-Gm-Message-State: AOJu0YymqZCP4Cr7eCvcGohbtIs3zIjxyZLuvRcEEFtDJuxag7tviegL knoEbWGwDUvVWtBnS58MorgvQqmlCj4pJQAGp45L2voC0zOJQq/ge4DK7gn1ILfJmXiAM5qYn0e LNCeNSZzLcMsDFg== X-Google-Smtp-Source: AGHT+IE5NJJ4RdK4KlubbNmWyDqdAz8dOCEwpQRHV+yTd+D1PrNhQg4C5iPY+ZfTTSN1uOUaaXXUzkM/8COhXTM= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a05:690c:3346:b0:6e3:cb96:f577 with SMTP id 00721157ae682-6e9d8ba6bd1mr12229487b3.8.1730304299692; Wed, 30 Oct 2024 09:04:59 -0700 (PDT) Date: Wed, 30 Oct 2024 16:04:25 +0000 In-Reply-To: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=7831; i=aliceryhl@google.com; h=from:subject:message-id; bh=7megp7qpFwj2d67QvZOgzlG5lXxVSDo6vxXF2gHW/aA=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnIlkhVNvtEovD+MGZCJxXXKKvp3nFPpUZr7FiV lKRDqa90P2JAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZyJZIQAKCRAEWL7uWMY5 RllJEACw9+sb/u2gZcS8irGwia+aClYAmOnQq5I0nZUxrWli5nGm22r4jxHyUqgfe65+CfcspQR z8BE3C5rmYa6bc47VChBgpExV98+Dk4qVlREciDdlY6buZjaLOH/L5K+OKQIXTMy1adbJnczeq6 ZuKrVfXBPW1Ycl9/25rtjVmaaLGZnGnj2HX+2gNFqnMBZLr/bfQ2xEYsPwpWAvm7LKar691neRI SneNpcO5YmQREJMa9vPLMCg0NfmvPpqAZc+YserPmkkWcMOg4S5F6o+1OeMwyGzBgEhEdpelpJ2 LcYGKBvHlW5CmLsKdiXdzVb9WTm+sU6SzGceWH0ubAJbRF0RXku7/4EdBLdkZqPU/vzHjdh/5VA b4cePqjzKFgE+J9xjRPoj8s85bWQy+DZnkfXtKyqat7U3nemh51HoaEP0NtI72pLMC7GaVg4loL 0j6YFaBX1jM3VG2UvErEskKrbqtHu3LuLozpYFNbYsc+QZGp1IlZgLnnOHfKAVfRil9J2FRi8wG 3oifoWiqVDbjSGCnu98DOeie5yzb1ddv1x8AcgoOHzEzMgU+BczUJE+km0AKkppqzw6JnhwKFP2 lcbOY/0hQKekXGwHeuzlJ4nu3JrVgjfzHN/dR5s/vIcvQXQhl1e6sGYYb2+A+4YoHDL9VX4nr87 G44dDUGYghc+zWg== X-Mailer: b4 0.13.0 Message-ID: <20241030-tracepoint-v12-2-eec7f0f8ad22@google.com> Subject: [PATCH v12 2/5] rust: add tracepoint support From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl , Carlos Llamas Make it possible to have Rust code call into tracepoints defined by C code. It is still required that the tracepoint is declared in a C header, and that this header is included in the input to bindgen. Instead of calling __DO_TRACE directly, the exported rust_do_trace_ function calls an inline helper function. This is because the `cond` argument does not exist at the callsite of DEFINE_RUST_DO_TRACE. __DECLARE_TRACE always emits an inline static and an extern declaration that is only used when CREATE_RUST_TRACE_POINTS is set. These should not end up in the final binary so it is not a problem that they sometimes are emitted without a user. Reviewed-by: Carlos Llamas Reviewed-by: Gary Guo Reviewed-by: Boqun Feng Signed-off-by: Alice Ryhl --- include/linux/tracepoint.h | 28 ++++++++++++++++++++++- include/trace/define_trace.h | 12 ++++++++++ rust/bindings/bindings_helper.h | 1 + rust/kernel/lib.rs | 1 + rust/kernel/tracepoint.rs | 49 +++++++++++++++++++++++++++++++++++++++++ 5 files changed, 90 insertions(+), 1 deletion(-) diff --git a/include/linux/tracepoint.h b/include/linux/tracepoint.h index 0dc67fad706c..84c4924e499f 100644 --- a/include/linux/tracepoint.h +++ b/include/linux/tracepoint.h @@ -225,6 +225,18 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) preempt_enable_notrace(); \ } while (0) +/* + * Declare an exported function that Rust code can call to trigger this + * tracepoint. This function does not include the static branch; that is done + * in Rust to avoid a function call when the tracepoint is disabled. + */ +#define DEFINE_RUST_DO_TRACE(name, proto, args) +#define __DEFINE_RUST_DO_TRACE(name, proto, args) \ + notrace void rust_do_trace_##name(proto) \ + { \ + __rust_do_trace_##name(args); \ + } + /* * Make sure the alignment of the structure in the __tracepoints section will * not add unwanted padding between the beginning of the section and the @@ -240,6 +252,7 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) extern int __traceiter_##name(data_proto); \ DECLARE_STATIC_CALL(tp_func_##name, __traceiter_##name); \ extern struct tracepoint __tracepoint_##name; \ + extern void rust_do_trace_##name(proto); \ static inline int \ register_trace_##name(void (*probe)(data_proto), void *data) \ { \ @@ -271,6 +284,12 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) #define __DECLARE_TRACE(name, proto, args, cond, data_proto) \ __DECLARE_TRACE_COMMON(name, PARAMS(proto), PARAMS(args), cond, PARAMS(data_proto)) \ + static inline void __rust_do_trace_##name(proto) \ + { \ + __DO_TRACE(name, \ + TP_ARGS(args), \ + TP_CONDITION(cond), 0); \ + } \ static inline void trace_##name(proto) \ { \ if (static_branch_unlikely(&__tracepoint_##name.key)) \ @@ -285,6 +304,12 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) #define __DECLARE_TRACE_SYSCALL(name, proto, args, cond, data_proto) \ __DECLARE_TRACE_COMMON(name, PARAMS(proto), PARAMS(args), cond, PARAMS(data_proto)) \ + static inline void __rust_do_trace_##name(proto) \ + { \ + __DO_TRACE(name, \ + TP_ARGS(args), \ + TP_CONDITION(cond), 1); \ + } \ static inline void trace_##name(proto) \ { \ if (static_branch_unlikely(&__tracepoint_##name.key)) \ @@ -339,7 +364,8 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) void __probestub_##_name(void *__data, proto) \ { \ } \ - DEFINE_STATIC_CALL(tp_func_##_name, __traceiter_##_name); + DEFINE_STATIC_CALL(tp_func_##_name, __traceiter_##_name); \ + DEFINE_RUST_DO_TRACE(_name, TP_PROTO(proto), TP_ARGS(args)) #define DEFINE_TRACE(name, proto, args) \ DEFINE_TRACE_FN(name, NULL, NULL, PARAMS(proto), PARAMS(args)); diff --git a/include/trace/define_trace.h b/include/trace/define_trace.h index ff5fa17a6259..0557626b6f6a 100644 --- a/include/trace/define_trace.h +++ b/include/trace/define_trace.h @@ -76,6 +76,13 @@ #define DECLARE_TRACE(name, proto, args) \ DEFINE_TRACE(name, PARAMS(proto), PARAMS(args)) +/* If requested, create helpers for calling these tracepoints from Rust. */ +#ifdef CREATE_RUST_TRACE_POINTS +#undef DEFINE_RUST_DO_TRACE +#define DEFINE_RUST_DO_TRACE(name, proto, args) \ + __DEFINE_RUST_DO_TRACE(name, PARAMS(proto), PARAMS(args)) +#endif + #undef TRACE_INCLUDE #undef __TRACE_INCLUDE @@ -134,6 +141,11 @@ # undef UNDEF_TRACE_INCLUDE_PATH #endif +#ifdef CREATE_RUST_TRACE_POINTS +# undef DEFINE_RUST_DO_TRACE +# define DEFINE_RUST_DO_TRACE(name, proto, args) +#endif + /* We may be processing more files */ #define CREATE_TRACE_POINTS diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index e0846e7e93e6..752572e638a6 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -20,6 +20,7 @@ #include #include #include +#include #include #include diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 708ff817ccc3..55f81f49024e 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -54,6 +54,7 @@ pub mod sync; pub mod task; pub mod time; +pub mod tracepoint; pub mod types; pub mod uaccess; pub mod workqueue; diff --git a/rust/kernel/tracepoint.rs b/rust/kernel/tracepoint.rs new file mode 100644 index 000000000000..c6e80aa99e8e --- /dev/null +++ b/rust/kernel/tracepoint.rs @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2024 Google LLC. + +//! Logic for tracepoints. + +/// Declare the Rust entry point for a tracepoint. +/// +/// This macro generates an unsafe function that calls into C, and its safety requirements will be +/// whatever the relevant C code requires. To document these safety requirements, you may add +/// doc-comments when invoking the macro. +#[macro_export] +macro_rules! declare_trace { + ($($(#[$attr:meta])* $pub:vis unsafe fn $name:ident($($argname:ident : $argtyp:ty),* $(,)?);)*) => {$( + $( #[$attr] )* + #[inline(always)] + $pub unsafe fn $name($($argname : $argtyp),*) { + #[cfg(CONFIG_TRACEPOINTS)] + { + // SAFETY: It's always okay to query the static key for a tracepoint. + let should_trace = unsafe { + $crate::macros::paste! { + $crate::jump_label::static_branch_unlikely!( + $crate::bindings::[< __tracepoint_ $name >], + $crate::bindings::tracepoint, + key + ) + } + }; + + if should_trace { + $crate::macros::paste! { + // SAFETY: The caller guarantees that it is okay to call this tracepoint. + unsafe { $crate::bindings::[< rust_do_trace_ $name >]($($argname),*) }; + } + } + } + + #[cfg(not(CONFIG_TRACEPOINTS))] + { + // If tracepoints are disabled, insert a trivial use of each argument + // to avoid unused argument warnings. + $( let _unused = $argname; )* + } + } + )*} +} + +pub use declare_trace; From patchwork Wed Oct 30 16:04:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13856823 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F1842144CA for ; Wed, 30 Oct 2024 16:05:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304305; cv=none; b=JWLdmjxNUMjdZo6/3aOKaLZRcuESEleGcNIFD9+59QvcT4xoGQ6dXITPbjTCMojG9VFdzB24peklpmsvQ1RPjRHTBRPJh71Ni0jCJr7XELl3XOmOHfsGUErL3rcoapCrTmSlSbN485ghuTFI43pygs1myfvGkk2QQD0pVJ/ORe4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304305; c=relaxed/simple; bh=6E1WLLql+lELSHgkVX0zcOXTthdmIBZjrDBlYLHV8VE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rE/EG48byIz+9viGZPIblAaOyO+urlydSNXMAGF4HBlaQrelOY/Yxh2JiCUB6VYhww0i5HGsBz2aoC5ZGvPjK6LREIbA2RfCqYNSYJL5B5++g1ogxyyVj+xOfMXX1PLgSV1ViTA5mmCsOuMToxOJKlemegrh4MjNV8rB5PU4gY4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eCsKG9et; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eCsKG9et" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e2b9f2c6559so10501569276.2 for ; Wed, 30 Oct 2024 09:05:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1730304302; x=1730909102; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=54C00wXILAP4wDE2WJebdXFCv98gErGHm0LgpmOmMWo=; b=eCsKG9ete77tTiTYmXqcbBPaJNZBL/UuBoWZtQybVxok1AtcgHkPmi9dWTvq5pDT5k ez6WLwmqUeG716InDy1tEHgFN8lBbdqHjoKY6qGKzgAX8SuzPDW7y5GOpF6GrErYRfBZ dfQX/yvnqXQG+jXn+2mUU+lBo6vf53lO1ZD+eA+iar4kA5wxM1oXPqETG06dFfY8meX3 TlHyJ/sYe3ScSHuvl4ELGNXVF12cGJI2C4b56mSMNzUR2lgAPTPBAffGdYKv/y9ADsQw fAM5F9KdMQci0PmTQxNx4dczadK3+N1DneeukMZne9qil2A2tFJFeFvJtLzO5Op3UeQL SMcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730304302; x=1730909102; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=54C00wXILAP4wDE2WJebdXFCv98gErGHm0LgpmOmMWo=; b=Y2PpH1stna9uMEWBFQlBicQLYYnrULH2sxSo1QEmlPuqlBmnXc/W9Qt8WDCMowBS6p 0+/QDrCIQWW5w3/dEApfTt/Ll8N0LL+hB1TzRPxh7FPH+9oEtEQgGQWYfOqZkWcKZuhu C/n/VtMOSd5LtUkEGPbSbLWD4qEKbfTfnpg8m21v34TDdzev3+Zuc5uAm+wm9TDXJwYS SQHx6ffCqcNXnIcOkZjorhK8HK4sW+w3Ens4Hz1S2nG8HruuiIvkbSMrGtWyZbAXeEn3 dKJ0LpL33YpIuyAlGlxWYIHBB748ChicyJDJDzmyFMim7qAE8IgxGQ8jKK6ULJbffluM KYaA== X-Gm-Message-State: AOJu0YzVhf2o30v5ZdXUGN/kAYEpM8bU0JT33kObQ2yFZHbCobuPTGil 5shfwr+cQEsEi+c2SWOBLmFbZ/OVsjC4t0OPNDah9D6dkv5Pp1TdUgnKj1uGo/vOzbpPH4JmSC0 gXhoJ900elTpmzA== X-Google-Smtp-Source: AGHT+IH02CpxvtSPol89CK83WsKKG+/9Mwolu1CUUBZ+Whn8Inwxg2VMatkn1IVi7GLj+0ruemL1utAOuG8/YqM= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a25:7481:0:b0:e2e:3031:3f0c with SMTP id 3f1490d57ef6-e3087bc8ca5mr10454276.7.1730304302410; Wed, 30 Oct 2024 09:05:02 -0700 (PDT) Date: Wed, 30 Oct 2024 16:04:26 +0000 In-Reply-To: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=4317; i=aliceryhl@google.com; h=from:subject:message-id; bh=6E1WLLql+lELSHgkVX0zcOXTthdmIBZjrDBlYLHV8VE=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnIlkij9ArugE0ISxftxYGap6R5sPzZ34MTem3X YAfJWcND5GJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZyJZIgAKCRAEWL7uWMY5 RtyHEACM0N0SSCrGkN7dCmCzwyY3cCJ5+/ZS8EiisT++meOQqOellWnDP/PWSwMYNbKLQj01avU DJklGMUi2rgWkgWLWUw3GrMRK/18oqPSId9Yq7MRtYctCEBNv4faTk2F5MMNKzh2HAbp896wyn/ +OUeotpRorcYeqS+fhvGRpZUsOt1I4hRqeCODknNXG7YJfnveRdyeX19Nqoc5Nmrr/Uxd/dfeSt +izMRDn9xRkm4BNbNX1yq7vxHnxw03+3+w2BuTdWsH+PLbA+jLAlxlYckKuVq3d81QH7/gRF5yo 7Bmw48EA4n5UMfHLSMXW6eu6yyNK/4rXD98UpVyaC0tmhhzevjQCifkvST3x0kK43p1ik/cpkal MSJn+2cTR1oSj+CjlBikD10FZecb3FGmhyvrlttp71kJqGbsuhJeKmPlg+/CLdw3SVqr8fxyI1U 8AokMAWfRmWN/BaLCLAHONLfPeoaVdg79U5+cIJeaAipIbx41O8Zan7uEfKAwyK8BTa/ialGsSi UvFlK8LwyS8au0VyyKicDxE7Tswji9RMwtfM4BmxSauM4ltVnXcRWuSQpV3V2W7tJkLqtZu77EN Kyr2CONG7Heet1+fPXd8k35lT0KGZoZdqTppbnog7dJLj1eDulTzDBYyiVEbnrBibgMGBS/dBZA Bfq3ja4N4Z6u9OQ== X-Mailer: b4 0.13.0 Message-ID: <20241030-tracepoint-v12-3-eec7f0f8ad22@google.com> Subject: [PATCH v12 3/5] rust: samples: add tracepoint to Rust sample From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl This updates the Rust printing sample to invoke a tracepoint. This ensures that we have a user in-tree from the get-go even though the patch is being merged before its real user. Reviewed-by: Boqun Feng Signed-off-by: Alice Ryhl --- MAINTAINERS | 1 + include/trace/events/rust_sample.h | 31 +++++++++++++++++++++++++++++++ rust/bindings/bindings_helper.h | 1 + samples/rust/Makefile | 3 ++- samples/rust/rust_print.rs | 18 ++++++++++++++++++ samples/rust/rust_print_events.c | 8 ++++++++ 6 files changed, 61 insertions(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index a097afd76ded..a9b71411d77a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -20223,6 +20223,7 @@ C: zulip://rust-for-linux.zulipchat.com P: https://rust-for-linux.com/contributing T: git https://github.com/Rust-for-Linux/linux.git rust-next F: Documentation/rust/ +F: include/trace/events/rust_sample.h F: rust/ F: samples/rust/ F: scripts/*rust* diff --git a/include/trace/events/rust_sample.h b/include/trace/events/rust_sample.h new file mode 100644 index 000000000000..dbc80ca2e465 --- /dev/null +++ b/include/trace/events/rust_sample.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Tracepoints for `samples/rust/rust_print.rs`. + * + * Copyright (C) 2024 Google, Inc. + */ + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM rust_sample + +#if !defined(_RUST_SAMPLE_TRACE_H) || defined(TRACE_HEADER_MULTI_READ) +#define _RUST_SAMPLE_TRACE_H + +#include + +TRACE_EVENT(rust_sample_loaded, + TP_PROTO(int magic_number), + TP_ARGS(magic_number), + TP_STRUCT__entry( + __field(int, magic_number) + ), + TP_fast_assign( + __entry->magic_number = magic_number; + ), + TP_printk("magic=%d", __entry->magic_number) +); + +#endif /* _RUST_SAMPLE_TRACE_H */ + +/* This part must be outside protection */ +#include diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index 752572e638a6..b072c197ce9e 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -23,6 +23,7 @@ #include #include #include +#include /* `bindgen` gets confused at certain things. */ const size_t RUST_CONST_HELPER_ARCH_SLAB_MINALIGN = ARCH_SLAB_MINALIGN; diff --git a/samples/rust/Makefile b/samples/rust/Makefile index 03086dabbea4..f29280ec4820 100644 --- a/samples/rust/Makefile +++ b/samples/rust/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 +ccflags-y += -I$(src) # needed for trace events obj-$(CONFIG_SAMPLE_RUST_MINIMAL) += rust_minimal.o -obj-$(CONFIG_SAMPLE_RUST_PRINT) += rust_print.o +obj-$(CONFIG_SAMPLE_RUST_PRINT) += rust_print.o rust_print_events.o subdir-$(CONFIG_SAMPLE_RUST_HOSTPROGS) += hostprogs diff --git a/samples/rust/rust_print.rs b/samples/rust/rust_print.rs index 6eabb0d79ea3..6d14b08cac1c 100644 --- a/samples/rust/rust_print.rs +++ b/samples/rust/rust_print.rs @@ -69,6 +69,8 @@ fn init(_module: &'static ThisModule) -> Result { arc_print()?; + trace::trace_rust_sample_loaded(42); + Ok(RustPrint) } } @@ -78,3 +80,19 @@ fn drop(&mut self) { pr_info!("Rust printing macros sample (exit)\n"); } } + +mod trace { + use core::ffi::c_int; + + kernel::declare_trace! { + /// # Safety + /// + /// Always safe to call. + unsafe fn rust_sample_loaded(magic: c_int); + } + + pub(crate) fn trace_rust_sample_loaded(magic: i32) { + // SAFETY: Always safe to call. + unsafe { rust_sample_loaded(magic as c_int) } + } +} diff --git a/samples/rust/rust_print_events.c b/samples/rust/rust_print_events.c new file mode 100644 index 000000000000..a9169ff0edf1 --- /dev/null +++ b/samples/rust/rust_print_events.c @@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright 2024 Google LLC + */ + +#define CREATE_TRACE_POINTS +#define CREATE_RUST_TRACE_POINTS +#include From patchwork Wed Oct 30 16:04:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13856824 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCEAC215030 for ; Wed, 30 Oct 2024 16:05:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304310; cv=none; b=CsOsHWyM5VnWQCfcwZFRhphQqB76MG9hNlsVCRzw9DSbQd3naJNUAnnHUajQimT6hwdnQbZs+d/kj+HngxpGgJNv3gzYmhlFlyFL5FraQADeOkEyMIoPaMGjOhH4tBxdMsQAJ7ze+Bf0jiGSS42AnKxWy6tdyfJSy3Kz4kfTVLA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304310; c=relaxed/simple; bh=vyyYul6wy7jKaMGtrgdBnkT/egAvdduh9xveUZb63Jc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ramVpc+RwssYcYJ2/Y9M81ETE8VSJekj6D7U7DYASHYhi5f3cJzboNLtnoE4GCDbE34+eTMqPRdnD0eJflBNeF06VhSjG4JF3U15CHukFyEf1pieEi4fTsPLit60Men5NUZgJ9tNCm2OzQTab5vf6BFk98dqsb7mn6YgAY9nSlc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gNsrGXOf; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gNsrGXOf" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43164f21063so45372855e9.2 for ; Wed, 30 Oct 2024 09:05:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1730304305; x=1730909105; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=aPKiYiMZSv/wKH/IY2uWJCz4bmXKvljXWgWXOs9047A=; b=gNsrGXOfduomfYQJyWyhSv1GgWkjn8a1zcw/sHUUa31KRgLIbCteyJ16niAwdH0OKp bvRvhcuCC21Acw0HhcNsYTNLdrxOfozYOXVkizh+6f5TH+YaMy9g424Y4h/Ep5I0VPE6 eeLSt/3LpVheG256Hr+GTQ4b4DB7a/SLcHNFw/2pvXyARYOLmJhEVKfVsvt2tSDKPZRw nVQLvf69Z79fCK47hdsQugUDDfDuCjxBEX2g5j1sZN2+ZQKNw0+/r7GgzvwU6HXZt+Lm JpTXQ8QZ0hPzRDnPWHKfMPWs7hDUy6MP4Hcj3A9NNcUWO10Hk1nUhyR7IzwYnHBkscga rp1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730304305; x=1730909105; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=aPKiYiMZSv/wKH/IY2uWJCz4bmXKvljXWgWXOs9047A=; b=JYFx9XQd8zMQie+2fhVgOJv4n9Ngk6A0GOets7TKs0OGYNFFMIcQtt6I3dQtijS0dQ cQWyQi/V7g9PEs8Y4uqHkFYjXImCg2iOYKXZmuImJTjBYMu61xurNmALW7Z239mus2pX ZyLfrNb7uQkmj+Y3DxRUpHbUMxTQxM3TDIaM9FQRIhDf84LaxfvlriQ+j2qnvxjX9fyn 9zu0S9n8qA0hj2oYQULrqR85jTv3bezp/Qz4b38VmWbWUU3I2CTieYzwiI+OzHif+WlU HMmF57JP5P+JruVIemwcMMHxWVUrnSoSMEOVCg36pggg/pl2PBQ3wLRGCfSnDum2FeV+ UR2g== X-Gm-Message-State: AOJu0Yyk8yMAScvm5tbBndz5drrBqA/ZIbuy8LPF+rnaGtO2vAbqS0pS gB7V3aWPFC0tyNPi06u9zNCQ2NB8DlGmbPutfFOY+NF+lFuhkrMswPy4ORk+no5idY0XzPsq6a7 08mVoVW9RLLtnSw== X-Google-Smtp-Source: AGHT+IG/v66X5/sNRf0N46LwSlUWGFX5ZXleN+ktqgGoUyfaArdYbWLHHKWwb2NH1SWWJLhii4RCDid8Ewha+T0= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a05:600c:499a:b0:431:47fc:92ab with SMTP id 5b1f17b1804b1-431bb9778d9mr40575e9.1.1730304304891; Wed, 30 Oct 2024 09:05:04 -0700 (PDT) Date: Wed, 30 Oct 2024 16:04:27 +0000 In-Reply-To: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=9457; i=aliceryhl@google.com; h=from:subject:message-id; bh=vyyYul6wy7jKaMGtrgdBnkT/egAvdduh9xveUZb63Jc=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnIlkjbodLH5bNx0hTlA6MfMQ91BcxuqHvIYjmG IvSEx9d2tGJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZyJZIwAKCRAEWL7uWMY5 RsXDD/0bATrvOML+C3wKiReydF6ngt8WuAWiWyT/iX2SO6bmWxevBm8AmGLMMK1yvo2x4CjezNB /8LN7KPzVQHI8KFi1l9i6mNZjoL506Hafo6/tHAGaPX4Mox/7sIpkhyxsV9z67YAfvGMwgM0Ovn uKcayN1MqVxUGirVSwkep6HjDlohBXcGmRESXdTFLLMajgsi3WVdYvc0uotTlCEWykt9TcsG0uA +KmN0G0TT3+/a5FNvLjnkFnOvuBtHMngQ2fAy8v8I8OJzlGECjdqkkSt37Ix9y4ZzjpKrQGT7QH Uckq6N6MWMIQ2DYO+5yW5Tsd/YfBItVvq9nJKscaalKNrzbYRQzc1LXgxnl/2dueWpalDamhzvu 4YOvfSZaX6KiaEbZD5ErPYzWTL6NRA+rgmtiCGSyJuCRdoGzDH8SXfZlOfQaWcmcg0FTq0TEDgh S1nGCgLav4ydlteSpC9w1H1ts9tMiajjxTXbIwoJDO0CxyaB3dUSnl6Cb0KvTw4/j5FDYKa8Ys+ +nbC5RO0huA81UyPZ87wPm/Qcxq1wa9Gm/F5rGm+j7utiSpKDB0T9XYr98aZu2EdSAPbWVvch8w 2GU4Zq620iSxN+26x4YiLGQFAEcSYlh80Ksbz9r6fzIHirOXIJ+0CQ/YXrtdnf+8h2bgGBVtl5y Sp1we2VNV3c67nQ== X-Mailer: b4 0.13.0 Message-ID: <20241030-tracepoint-v12-4-eec7f0f8ad22@google.com> Subject: [PATCH v12 4/5] jump_label: adjust inline asm to be consistent From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl , Palmer Dabbelt To avoid duplication of inline asm between C and Rust, we need to import the inline asm from the relevant `jump_label.h` header into Rust. To make that easier, this patch updates the header files to expose the inline asm via a new ARCH_STATIC_BRANCH_ASM macro. The header files are all updated to define a ARCH_STATIC_BRANCH_ASM that takes the same arguments in a consistent order so that Rust can use the same logic for every architecture. Suggested-by: Peter Zijlstra (Intel) Co-developed-by: Miguel Ojeda Signed-off-by: Miguel Ojeda Acked-by: Peter Zijlstra (Intel) Acked-by: Catalin Marinas Acked-by: Palmer Dabbelt # RISC-V Signed-off-by: Alice Ryhl --- arch/arm/include/asm/jump_label.h | 14 +++++---- arch/arm64/include/asm/jump_label.h | 20 ++++++++----- arch/loongarch/include/asm/jump_label.h | 16 +++++++---- arch/riscv/include/asm/jump_label.h | 50 ++++++++++++++++++--------------- arch/x86/include/asm/jump_label.h | 35 +++++++++-------------- 5 files changed, 73 insertions(+), 62 deletions(-) diff --git a/arch/arm/include/asm/jump_label.h b/arch/arm/include/asm/jump_label.h index e4eb54f6cd9f..a35aba7f548c 100644 --- a/arch/arm/include/asm/jump_label.h +++ b/arch/arm/include/asm/jump_label.h @@ -9,13 +9,17 @@ #define JUMP_LABEL_NOP_SIZE 4 +/* This macro is also expanded on the Rust side. */ +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1:\n\t" \ + WASM(nop) "\n\t" \ + ".pushsection __jump_table, \"aw\"\n\t" \ + ".word 1b, " label ", " key "\n\t" \ + ".popsection\n\t" \ + static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { - asm goto("1:\n\t" - WASM(nop) "\n\t" - ".pushsection __jump_table, \"aw\"\n\t" - ".word 1b, %l[l_yes], %c0\n\t" - ".popsection\n\t" + asm goto(ARCH_STATIC_BRANCH_ASM("%c0", "%l[l_yes]") : : "i" (&((char *)key)[branch]) : : l_yes); return false; diff --git a/arch/arm64/include/asm/jump_label.h b/arch/arm64/include/asm/jump_label.h index a0a5bbae7229..424ed421cd97 100644 --- a/arch/arm64/include/asm/jump_label.h +++ b/arch/arm64/include/asm/jump_label.h @@ -19,10 +19,14 @@ #define JUMP_TABLE_ENTRY(key, label) \ ".pushsection __jump_table, \"aw\"\n\t" \ ".align 3\n\t" \ - ".long 1b - ., %l["#label"] - .\n\t" \ - ".quad %c0 - .\n\t" \ - ".popsection\n\t" \ - : : "i"(key) : : label + ".long 1b - ., " label " - .\n\t" \ + ".quad " key " - .\n\t" \ + ".popsection\n\t" + +/* This macro is also expanded on the Rust side. */ +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1: nop\n\t" \ + JUMP_TABLE_ENTRY(key, label) static __always_inline bool arch_static_branch(struct static_key * const key, const bool branch) @@ -30,8 +34,8 @@ static __always_inline bool arch_static_branch(struct static_key * const key, char *k = &((char *)key)[branch]; asm goto( - "1: nop \n\t" - JUMP_TABLE_ENTRY(k, l_yes) + ARCH_STATIC_BRANCH_ASM("%c0", "%l[l_yes]") + : : "i"(k) : : l_yes ); return false; @@ -43,9 +47,11 @@ static __always_inline bool arch_static_branch_jump(struct static_key * const ke const bool branch) { char *k = &((char *)key)[branch]; + asm goto( "1: b %l[l_yes] \n\t" - JUMP_TABLE_ENTRY(k, l_yes) + JUMP_TABLE_ENTRY("%c0", "%l[l_yes]") + : : "i"(k) : : l_yes ); return false; l_yes: diff --git a/arch/loongarch/include/asm/jump_label.h b/arch/loongarch/include/asm/jump_label.h index 29acfe3de3fa..8a924bd69d19 100644 --- a/arch/loongarch/include/asm/jump_label.h +++ b/arch/loongarch/include/asm/jump_label.h @@ -13,18 +13,22 @@ #define JUMP_LABEL_NOP_SIZE 4 -#define JUMP_TABLE_ENTRY \ +/* This macro is also expanded on the Rust side. */ +#define JUMP_TABLE_ENTRY(key, label) \ ".pushsection __jump_table, \"aw\" \n\t" \ ".align 3 \n\t" \ - ".long 1b - ., %l[l_yes] - . \n\t" \ - ".quad %0 - . \n\t" \ + ".long 1b - ., " label " - . \n\t" \ + ".quad " key " - . \n\t" \ ".popsection \n\t" +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1: nop \n\t" \ + JUMP_TABLE_ENTRY(key, label) + static __always_inline bool arch_static_branch(struct static_key * const key, const bool branch) { asm goto( - "1: nop \n\t" - JUMP_TABLE_ENTRY + ARCH_STATIC_BRANCH_ASM("%0", "%l[l_yes]") : : "i"(&((char *)key)[branch]) : : l_yes); return false; @@ -37,7 +41,7 @@ static __always_inline bool arch_static_branch_jump(struct static_key * const ke { asm goto( "1: b %l[l_yes] \n\t" - JUMP_TABLE_ENTRY + JUMP_TABLE_ENTRY("%0", "%l[l_yes]") : : "i"(&((char *)key)[branch]) : : l_yes); return false; diff --git a/arch/riscv/include/asm/jump_label.h b/arch/riscv/include/asm/jump_label.h index 1c768d02bd0c..87a71cc6d146 100644 --- a/arch/riscv/include/asm/jump_label.h +++ b/arch/riscv/include/asm/jump_label.h @@ -16,21 +16,28 @@ #define JUMP_LABEL_NOP_SIZE 4 +#define JUMP_TABLE_ENTRY(key, label) \ + ".pushsection __jump_table, \"aw\" \n\t" \ + ".align " RISCV_LGPTR " \n\t" \ + ".long 1b - ., " label " - . \n\t" \ + "" RISCV_PTR " " key " - . \n\t" \ + ".popsection \n\t" + +/* This macro is also expanded on the Rust side. */ +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + " .align 2 \n\t" \ + " .option push \n\t" \ + " .option norelax \n\t" \ + " .option norvc \n\t" \ + "1: nop \n\t" \ + " .option pop \n\t" \ + JUMP_TABLE_ENTRY(key, label) + static __always_inline bool arch_static_branch(struct static_key * const key, const bool branch) { asm goto( - " .align 2 \n\t" - " .option push \n\t" - " .option norelax \n\t" - " .option norvc \n\t" - "1: nop \n\t" - " .option pop \n\t" - " .pushsection __jump_table, \"aw\" \n\t" - " .align " RISCV_LGPTR " \n\t" - " .long 1b - ., %l[label] - . \n\t" - " " RISCV_PTR " %0 - . \n\t" - " .popsection \n\t" + ARCH_STATIC_BRANCH_ASM("%0", "%l[label]") : : "i"(&((char *)key)[branch]) : : label); return false; @@ -38,21 +45,20 @@ static __always_inline bool arch_static_branch(struct static_key * const key, return true; } +#define ARCH_STATIC_BRANCH_JUMP_ASM(key, label) \ + " .align 2 \n\t" \ + " .option push \n\t" \ + " .option norelax \n\t" \ + " .option norvc \n\t" \ + "1: j " label " \n\t" \ + " .option pop \n\t" \ + JUMP_TABLE_ENTRY(key, label) + static __always_inline bool arch_static_branch_jump(struct static_key * const key, const bool branch) { asm goto( - " .align 2 \n\t" - " .option push \n\t" - " .option norelax \n\t" - " .option norvc \n\t" - "1: j %l[label] \n\t" - " .option pop \n\t" - " .pushsection __jump_table, \"aw\" \n\t" - " .align " RISCV_LGPTR " \n\t" - " .long 1b - ., %l[label] - . \n\t" - " " RISCV_PTR " %0 - . \n\t" - " .popsection \n\t" + ARCH_STATIC_BRANCH_JUMP_ASM("%0", "%l[label]") : : "i"(&((char *)key)[branch]) : : label); return false; diff --git a/arch/x86/include/asm/jump_label.h b/arch/x86/include/asm/jump_label.h index cbbef32517f0..3f1c1d6c0da1 100644 --- a/arch/x86/include/asm/jump_label.h +++ b/arch/x86/include/asm/jump_label.h @@ -12,35 +12,28 @@ #include #include -#define JUMP_TABLE_ENTRY \ +#define JUMP_TABLE_ENTRY(key, label) \ ".pushsection __jump_table, \"aw\" \n\t" \ _ASM_ALIGN "\n\t" \ ".long 1b - . \n\t" \ - ".long %l[l_yes] - . \n\t" \ - _ASM_PTR "%c0 + %c1 - .\n\t" \ + ".long " label " - . \n\t" \ + _ASM_PTR " " key " - . \n\t" \ ".popsection \n\t" +/* This macro is also expanded on the Rust side. */ #ifdef CONFIG_HAVE_JUMP_LABEL_HACK - -static __always_inline bool arch_static_branch(struct static_key *key, bool branch) -{ - asm goto("1:" - "jmp %l[l_yes] # objtool NOPs this \n\t" - JUMP_TABLE_ENTRY - : : "i" (key), "i" (2 | branch) : : l_yes); - - return false; -l_yes: - return true; -} - +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1: jmp " label " # objtool NOPs this \n\t" \ + JUMP_TABLE_ENTRY(key " + 2", label) #else /* !CONFIG_HAVE_JUMP_LABEL_HACK */ +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1: .byte " __stringify(BYTES_NOP5) "\n\t" \ + JUMP_TABLE_ENTRY(key, label) +#endif /* CONFIG_HAVE_JUMP_LABEL_HACK */ static __always_inline bool arch_static_branch(struct static_key * const key, const bool branch) { - asm goto("1:" - ".byte " __stringify(BYTES_NOP5) "\n\t" - JUMP_TABLE_ENTRY + asm goto(ARCH_STATIC_BRANCH_ASM("%c0 + %c1", "%l[l_yes]") : : "i" (key), "i" (branch) : : l_yes); return false; @@ -48,13 +41,11 @@ static __always_inline bool arch_static_branch(struct static_key * const key, co return true; } -#endif /* CONFIG_HAVE_JUMP_LABEL_HACK */ - static __always_inline bool arch_static_branch_jump(struct static_key * const key, const bool branch) { asm goto("1:" "jmp %l[l_yes]\n\t" - JUMP_TABLE_ENTRY + JUMP_TABLE_ENTRY("%c0 + %c1", "%l[l_yes]") : : "i" (key), "i" (branch) : : l_yes); return false; From patchwork Wed Oct 30 16:04:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13856825 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15E2B215C59 for ; Wed, 30 Oct 2024 16:05:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304311; cv=none; b=lUtvNGQ7hxwE/NuFNOJymYTTELr7ORMAqxWjT8AMQ7bkwB7cljbrbxrqDcoCw2LzSacwlcqnx2Tq1KjlgrjMpB0gAGjKDWZFEgFt6f3IcpBDMeI9PPbrlrovtlieP52raYI4iNoEOApNzHMbQ77v4sebsY7WwW+Rkyi2jKdSpdI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730304311; c=relaxed/simple; bh=tEriRl1s5aMKzJQzUep/R+6Ah6Myzoilu5Gt/biRO6w=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GUQEOQqfRFyjngLplUy09OEx7/GJnRQazF1aj44AzjUU/+xMkTG4DAT/dtJ8qaWoCnHTOCQZ+blCe/iw5SrLrV392QFPeW74TSpGyDcaKeVy78Og63CZJjdrEPAqbBHWnQUGt5uhaqjwqMvXX798BfAhkkL4AWCwg1BKKPyc7Ns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mmZJWuPm; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mmZJWuPm" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e28fc8902e6so12240712276.0 for ; Wed, 30 Oct 2024 09:05:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1730304308; x=1730909108; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=JqrmG66m6B1UCkhHvZIk418M17bGfpbVQcwN/mZmV/M=; b=mmZJWuPmy0eZ3G5XAH7HhCgrqwGdjbFehO38Ynnok2hQJUIAT+0xbWmrYFG3SldMZi 4FfWOvJXZevf6zrB816MafunbmaMQemOkY/lMFJBiCVEwgxB4jBbl+hWpPdGgo7dRw7d KLw4a65AfgVLkfOnKPRMTjh7Mi4+QmwtSUkcNUPx4CV11Kaz2PqS/ziwtm8m/3udiZ7u fJl+c7oyqtO1RopSyFMHtcOFpsL/Wy8nTS8oIHU1MQ8ZiaWEopJvOMONXSLkIwa0ybAp zvtkOoaN5/yXcTmbNmpKrR2MUDaSHdk2vW/bnxGnCI30H+nx6MyA65XwPdU1KNT0tHeb zQBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730304308; x=1730909108; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JqrmG66m6B1UCkhHvZIk418M17bGfpbVQcwN/mZmV/M=; b=Ya8e5z51RfqIdhR3QsnKJacuidFH/cjTdx4BXE1JpjQGzBTil2msrSb4TkwmpnJeMR 8K+qfYS6TjwenfEDosgqyZs1jYqWGb01Cp6jSZ6wy5298piv78xkEQlXJFPDrfB+hVVl tX5fgI6DuqNbkNBkVVw1HGOVZK6YE7lXrC/VuS4vEPgcR3CK65owz28U4K7G0IyXBvzE 0VcEeeF0FEcg8Y33rXsJdvgV7NCCaeTzl/N03kG9Y7ElAKog0DTmWDfTvnDOGCUcrt93 5GDaJEpqV2wwBFnkq4x4pbp9kEMbEY7kDMwHOvy6hXKEa94wPD1sb0bAlO8bp4yoVf3A ScaA== X-Gm-Message-State: AOJu0YwKnGZF5cjjCnh3rW6owURlPioc6s7nF/zAMGudWOLXSOp4jBLj BABWWK4aNNik3guKQAr8PPG0d5Sr7/D2tFD2TzEa63S+R+h+hjoUALWq7CygdhxQSstq+4bXMq2 Hdc6Yeyia095gHg== X-Google-Smtp-Source: AGHT+IFi/aR3eZmVoAL9Wz4p5EtnW6YunurLPIJuDUlAryYR3kFyV0YKLO6vsDSY51xhlGqCS1gC7bq434rMmfY= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a25:ed07:0:b0:e1d:912e:9350 with SMTP id 3f1490d57ef6-e3087bc765dmr51968276.6.1730304307760; Wed, 30 Oct 2024 09:05:07 -0700 (PDT) Date: Wed, 30 Oct 2024 16:04:28 +0000 In-Reply-To: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241030-tracepoint-v12-0-eec7f0f8ad22@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=7761; i=aliceryhl@google.com; h=from:subject:message-id; bh=tEriRl1s5aMKzJQzUep/R+6Ah6Myzoilu5Gt/biRO6w=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBnIlkj+zi/ZeCsvvjAjU6Nb9xOSl6DubI5Y/Iwy 1JbUL9MEGmJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZyJZIwAKCRAEWL7uWMY5 RnMrD/9GOSq+2yFRe/8k8SIbCJosPtOngpcRFceZEKecE5pRUpUhsuGpdIulecGorZlT7tcfTF7 yrDZPb5ueqeo9wHnC6xEW3CZNx87ladgXT8SQj1lNkcnP6KpD/alvOkTHSD/BnedRWQxGRVK9Li KHP0TINKG2hcDqPbkYkjlwdgZF7Ta5ggrweahQAMEnVTa7FcVsR1ZEVdVt+txPh8Zb9IJIsdph4 STpTD9SIC/TLDzSC+1hjY48M72ZHuuabI5ePkc8rstSiDkDlkFgq/FUVeMEhCSlC2irnl91KtQT 6WDkAKn7sVk30GREem94CI7KAcp9b6i7T1vMhm4XEaru6w9wPARl1Gy/6IgJ3JCGGZU+edYb1u2 POCxtK/YUwn1oGWe7DeODIBNXcD3mtVoqOLtwiVSI6pRIRsJALNDpWY3Mvj5uMsBDtXsL8GG1nD QxMhI85t3tzOpDcPvQhG8+6026MCkv1TRefxTfga0067dSW1G1owYNYkV02PndF26FrhYGnXCSb GrG/OGiQzLTKvkgCaePQiB9/qbWor/pEW3K2kT8dTceXB70qm1u9ENXA0wkUkMF72UB0+C3Jo9n aAvVHTvLeIEQgUuSurrni4jzoRe+diXRQH6uSIB8H38CHj2nIVPEDbQ0y3VyGQomV5QcHr7cMuq bDnNS7IMkNetjhw== X-Mailer: b4 0.13.0 Message-ID: <20241030-tracepoint-v12-5-eec7f0f8ad22@google.com> Subject: [PATCH v12 5/5] rust: add arch_static_branch From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl To allow the Rust implementation of static_key_false to use runtime code patching instead of the generic implementation, pull in the relevant inline assembly from the jump_label.h header by running the C preprocessor on a .rs.S file. Build rules are added for .rs.S files. Since the relevant inline asm has been adjusted to export the inline asm via the ARCH_STATIC_BRANCH_ASM macro in a consistent way, the Rust side does not need architecture specific code to pull in the asm. It is not possible to use the existing C implementation of arch_static_branch via a Rust helper because it passes the argument `key` to inline assembly as an 'i' parameter. Any attempt to add a C helper for this function will fail to compile because the value of `key` must be known at compile-time. Suggested-by: Peter Zijlstra (Intel) Co-developed-by: Miguel Ojeda Signed-off-by: Miguel Ojeda Reviewed-by: Boqun Feng Signed-off-by: Alice Ryhl --- rust/Makefile | 6 +++++ rust/kernel/.gitignore | 3 +++ rust/kernel/arch_static_branch_asm.rs.S | 7 +++++ rust/kernel/jump_label.rs | 46 ++++++++++++++++++++++++++++++++- rust/kernel/lib.rs | 35 +++++++++++++++++++++++++ scripts/Makefile.build | 9 ++++++- 6 files changed, 104 insertions(+), 2 deletions(-) diff --git a/rust/Makefile b/rust/Makefile index b5e0a73b78f3..bc2a9071dd29 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -36,6 +36,8 @@ always-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.c obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated.o obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.o +always-$(subst y,$(CONFIG_RUST),$(CONFIG_JUMP_LABEL)) += kernel/arch_static_branch_asm.rs + # Avoids running `$(RUSTC)` for the sysroot when it may not be available. ifdef CONFIG_RUST @@ -424,4 +426,8 @@ $(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \ $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE +$(call if_changed_rule,rustc_library) +ifdef CONFIG_JUMP_LABEL +$(obj)/kernel.o: $(obj)/kernel/arch_static_branch_asm.rs +endif + endif # CONFIG_RUST diff --git a/rust/kernel/.gitignore b/rust/kernel/.gitignore new file mode 100644 index 000000000000..d082731007c6 --- /dev/null +++ b/rust/kernel/.gitignore @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +/arch_static_branch_asm.rs diff --git a/rust/kernel/arch_static_branch_asm.rs.S b/rust/kernel/arch_static_branch_asm.rs.S new file mode 100644 index 000000000000..2afb638708db --- /dev/null +++ b/rust/kernel/arch_static_branch_asm.rs.S @@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include + +// Cut here. + +::kernel::concat_literals!(ARCH_STATIC_BRANCH_ASM("{symb} + {off} + {branch}", "{l_yes}")) diff --git a/rust/kernel/jump_label.rs b/rust/kernel/jump_label.rs index 4b7655b2a022..2f2df03a3275 100644 --- a/rust/kernel/jump_label.rs +++ b/rust/kernel/jump_label.rs @@ -24,7 +24,51 @@ macro_rules! static_branch_unlikely { let _key: *const $crate::bindings::static_key_false = ::core::ptr::addr_of!((*_key).$field); let _key: *const $crate::bindings::static_key = _key.cast(); - $crate::bindings::static_key_count(_key.cast_mut()) > 0 + #[cfg(not(CONFIG_JUMP_LABEL))] + { + $crate::bindings::static_key_count(_key) > 0 + } + + #[cfg(CONFIG_JUMP_LABEL)] + $crate::jump_label::arch_static_branch! { $key, $keytyp, $field, false } }}; } pub use static_branch_unlikely; + +/// Assert that the assembly block evaluates to a string literal. +#[cfg(CONFIG_JUMP_LABEL)] +const _: &str = include!(concat!( + env!("OBJTREE"), + "/rust/kernel/arch_static_branch_asm.rs" +)); + +#[macro_export] +#[doc(hidden)] +#[cfg(CONFIG_JUMP_LABEL)] +macro_rules! arch_static_branch { + ($key:path, $keytyp:ty, $field:ident, $branch:expr) => {'my_label: { + $crate::asm!( + include!(concat!(env!("OBJTREE"), "/rust/kernel/arch_static_branch_asm.rs")); + l_yes = label { + break 'my_label true; + }, + symb = sym $key, + off = const ::core::mem::offset_of!($keytyp, $field), + branch = const $crate::jump_label::bool_to_int($branch), + ); + + break 'my_label false; + }}; +} + +#[cfg(CONFIG_JUMP_LABEL)] +pub use arch_static_branch; + +/// A helper used by inline assembly to pass a boolean to as a `const` parameter. +/// +/// Using this function instead of a cast lets you assert that the input is a boolean, and not some +/// other type that can also be cast to an integer. +#[doc(hidden)] +pub const fn bool_to_int(b: bool) -> i32 { + b as i32 +} diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 55f81f49024e..97286b99270e 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -148,3 +148,38 @@ macro_rules! container_of { ptr.sub(offset) as *const $type }} } + +/// Helper for `.rs.S` files. +#[doc(hidden)] +#[macro_export] +macro_rules! concat_literals { + ($( $asm:literal )* ) => { + ::core::concat!($($asm),*) + }; +} + +/// Wrapper around `asm!` configured for use in the kernel. +/// +/// Uses a semicolon to avoid parsing ambiguities, even though this does not match native `asm!` +/// syntax. +// For x86, `asm!` uses intel syntax by default, but we want to use at&t syntax in the kernel. +#[cfg(any(target_arch = "x86", target_arch = "x86_64"))] +#[macro_export] +macro_rules! asm { + ($($asm:expr),* ; $($rest:tt)*) => { + ::core::arch::asm!( $($asm)*, options(att_syntax), $($rest)* ) + }; +} + +/// Wrapper around `asm!` configured for use in the kernel. +/// +/// Uses a semicolon to avoid parsing ambiguities, even though this does not match native `asm!` +/// syntax. +// For non-x86 arches we just pass through to `asm!`. +#[cfg(not(any(target_arch = "x86", target_arch = "x86_64")))] +#[macro_export] +macro_rules! asm { + ($($asm:expr),* ; $($rest:tt)*) => { + ::core::arch::asm!( $($asm)*, $($rest)* ) + }; +} diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 8f423a1faf50..03ee558fcd4d 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -248,12 +248,13 @@ $(obj)/%.lst: $(obj)/%.c FORCE # Compile Rust sources (.rs) # --------------------------------------------------------------------------- -rust_allowed_features := new_uninit +rust_allowed_features := asm_const,asm_goto,new_uninit # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. rust_common_cmd = \ + OBJTREE=$(abspath $(objtree)) \ RUST_MODFILE=$(modfile) $(RUSTC_OR_CLIPPY) $(rust_flags) \ -Zallow-features=$(rust_allowed_features) \ -Zcrate-attr=no_std \ @@ -303,6 +304,12 @@ quiet_cmd_rustc_ll_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ $(obj)/%.ll: $(obj)/%.rs FORCE +$(call if_changed_dep,rustc_ll_rs) +quiet_cmd_rustc_rs_rs_S = RSCPP $(quiet_modtag) $@ + cmd_rustc_rs_rs_S = $(CPP) $(c_flags) -xc -C -P $< | sed '1,/^\/\/ Cut here.$$/d' >$@ + +$(obj)/%.rs: $(obj)/%.rs.S FORCE + +$(call if_changed_dep,rustc_rs_rs_S) + # Compile assembler sources (.S) # ---------------------------------------------------------------------------