From patchwork Sun Mar 24 21:15:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Puranjay Mohan X-Patchwork-Id: 13600946 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6F247C47DD9 for ; Sun, 24 Mar 2024 21:15:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=5N+xmXqWMlCqC+z2RH07jVF9PMnHqENaH1bPhMlez1o=; b=iT+w8kXUmcNHc7 kfv+1KV2HhC3Asr2Xq47ctyTJBueTr0BNLfcVUrYH5AEamb3xOB3mXTf5LZhk/kN9o/n8veldrRrX eSF5KUcFOXELZi448P5LYdHSc2iGCBHODXxVMOEWt9EBjnl7GHfdL6VDldLlYn5CFZUIvDYj1t3is 0AzTpd4XJklIu2mZXTdqqFVh0LWPpmsv37Z6zF+UKnQZfRjm1Y1PIveeqsbh+n523eY5d83pavqw1 +wMH1HUCMnJg5vcRVLYISCVUT59XD8ssx2hPS2RiOAdg330ptgNmd5P5Zd3vZUsqWvQoqlnVKAgGE i+WwUmH+pqAnX7DverAQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1roVBi-0000000DTes-23Ee; Sun, 24 Mar 2024 21:15:30 +0000 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1roVBd-0000000DTdx-2rlB for linux-arm-kernel@lists.infradead.org; Sun, 24 Mar 2024 21:15:27 +0000 Received: by mail-wr1-x42c.google.com with SMTP id ffacd0b85a97d-3417a3151c4so3410132f8f.3 for ; Sun, 24 Mar 2024 14:15:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711314923; x=1711919723; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=xJs/8rrZFlSm5BVLSgyDRWbw0+owUKGrLUv/BH3DuaU=; b=hBec2ZW5vvmpAnmg6IMMK8/q3CVSqv4PLMCfegEA92tMN3piWSO+3s0hg73niCu4B2 kPrevBMce4x4rY0KOFiWw0WrVhFUEPUJ6bbEMhpE69kb/dM3lsWPpO4b6GEM9g2k1kvF PMVZIqP7ZXcDSUdkjA94lUvfGRMzP92cDc8X6ugBO5+BT+hVbi6jG7vOJIiN71n+l+KU YGj7I1wYWHZcGzRtSvLSlCUBpEVRyJdM2FS8zlLG4ChCXcf5WaO/ly9/CjXzSZu+Sob6 XQiUAyN3Szw/VsoaNcCPNl4UF+O00ztre47DJAJVDiLnNpZ+jF5KWaaqSx6A959jW8Bs EGWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711314923; x=1711919723; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xJs/8rrZFlSm5BVLSgyDRWbw0+owUKGrLUv/BH3DuaU=; b=jUrcalAhUXgjBO1rlUGI8I3BQ+e6lfVqsJwcvxn43Z/crzZsKQa6kX2nWTLjRnyzI6 ptZHxoHR9+Xx41fodfDIGXM74l/npLKZohvemEIYaASFF+SjojnPD6Om2p7UhnimAX7J fpSIkcILDac85LsKwAHXE/5e0v+6A2QHgYNzLcuCjmvhVVX3u28/41p1CZD2ZqzqnZsl pXSa8ila+G4N6kD5FVKWDsUtMmPH02r3wOMf7qF0cUNZUKoAfDKbIwh3KxT4IRKW8K0C azipbAnFdxFol4PIj3AvlSW90crgeLaA2KezX9lcbQejfT/SBuxcLYNNOP+zMkKj0kZv ocgw== X-Forwarded-Encrypted: i=1; AJvYcCUb14bWTmFEEOHwo4Sv1bSfwpXllj02IfBjI9yxavho1yHQN8Lji3WkIEh2LaLVfDM5zcgTTrT8FXHSlNq/Y67RWDx6CrWgVd7ECBAF1dGp4hXwGRA= X-Gm-Message-State: AOJu0YzzHqmqZOmT2SaThqe/uw/kvUIKym9cFaJtJtxuxLmQC/nueFZN f6bJOp4aCnbQE6bj6sLbWJJPaI1C0UAKwkbz3S7v3uAADDEcLKLY X-Google-Smtp-Source: AGHT+IGO+UBAljYUKl4PAtKDUE8xi/oE31T+TEtE/VSp6UjAj09qYWQSrtRtW8zw0s6szgb97QpKFA== X-Received: by 2002:adf:e0d1:0:b0:33e:c91a:127e with SMTP id m17-20020adfe0d1000000b0033ec91a127emr4308274wri.63.1711314923080; Sun, 24 Mar 2024 14:15:23 -0700 (PDT) Received: from localhost (54-240-197-231.amazon.com. [54.240.197.231]) by smtp.gmail.com with ESMTPSA id m9-20020a5d4a09000000b00341bdecdae3sm5420388wrq.117.2024.03.24.14.15.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 24 Mar 2024 14:15:22 -0700 (PDT) From: Puranjay Mohan To: Catalin Marinas , Will Deacon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Zi Shen Lim , Mark Rutland , Suzuki K Poulose , Mark Brown , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org (open list), bpf@vger.kernel.org (open list:BPF [GENERAL] (Safe Dynamic Programs and Tools)), Josh Poimboeuf Cc: puranjay12@gmail.com Subject: [PATCH bpf-next v2 0/1] Support kCFI + BPF on arm64 Date: Sun, 24 Mar 2024 21:15:17 +0000 Message-Id: <20240324211518.93892-1-puranjay12@gmail.com> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240324_141525_843146_EE484AD6 X-CRM114-Status: GOOD ( 18.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Changes in V2: V1: https://lore.kernel.org/bpf/20240227151115.4623-1-puranjay12@gmail.com/ - Rebased on latest bpf-next/master On ARM64 with CONFIG_CFI_CLANG, CFI warnings can be triggered by running the bpf selftests. This is because the JIT doesn't emit proper CFI prologues for BPF programs, callbacks, and struct_ops trampolines. Example Warning: CFI failure at bpf_rbtree_add_impl+0x120/0x1d4 (target: bpf_prog_fb8b097ab47d164a_less+0x0/0x98; expected type: 0x9e4709a9) WARNING: CPU: 0 PID: 1488 at bpf_rbtree_add_impl+0x120/0x1d4 Modules linked in: bpf_testmod(OE) virtio_net net_failover failover aes_ce_blk aes_ce_cipher ghash_ce sha2_ce sha256_arm64 sha1_ce virtio_mmio uio_pdrv_genirq uio dm_mod dax configfs [last unloaded: bpf_testmod(OE)] CPU: 0 PID: 1488 Comm: new_name Tainted: P OE 6.8.0-rc1+ #1 Hardware name: linux,dummy-virt (DT) pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : bpf_rbtree_add_impl+0x120/0x1d4 lr : bpf_prog_234260f1d6227155_rbtree_first_and_remove+0x218/0x438 sp : ffff80008444bb10 x29: ffff80008444bb10 x28: ffff80008444bbf0 x27: ffff80008444bb60 x26: 0000000000000000 x25: 0000000000000010 x24: 0000000000000008 x23: 0000000000000001 x22: ffff00000ab71658 x21: ffff8000843dd5fc x20: ffff00000ab459f0 x19: ffff00000ab71358 x18: 0000000000000000 x17: 000000009e4709a9 x16: 00000000d4202000 x15: 0000aaaadf15e420 x14: 0000000000004007 x13: ffff800084448000 x12: 0000000000000000 x11: dead00000000eb9f x10: ffff00000ab71370 x9 : 0000000000000000 x8 : ffff00000ab71658 x7 : 0000000000000000 x6 : 0000000000000000 x5 : 0000000000000001 x4 : 0000000000000000 x3 : 0000000000000000 x2 : 0000000000000000 x1 : ffff00000ab71658 x0 : ffff00000ab71358 Call trace: bpf_rbtree_add_impl+0x120/0x1d4 bpf_prog_234260f1d6227155_rbtree_first_and_remove+0x218/0x438 bpf_test_run+0x190/0x358 bpf_prog_test_run_skb+0x354/0x460 bpf_prog_test_run+0x128/0x164 __sys_bpf+0x364/0x428 __arm64_sys_bpf+0x30/0x44 invoke_syscall+0x64/0x128 el0_svc_common+0xb4/0xe8 do_el0_svc+0x28/0x34 el0_svc+0x58/0x108 el0t_64_sync_handler+0x90/0xfc el0t_64_sync+0x1a8/0x1ac irq event stamp: 35493817 hardirqs last enabled at (35493816): [] unit_alloc+0x110/0x1b0 hardirqs last disabled at (35493817): [] bpf_spin_lock+0x2c/0xec softirqs last enabled at (35493688): [] bpf_ksym_add+0x164/0x184 softirqs last disabled at (35493810): [] local_bh_disable+0x4/0x30 ---[ end trace 0000000000000000 ]--- This patch fixes the prologue and trampoline generation code to emit the KCFI hash before the expected branch targets. The KCFI hashes are generated at compile time and are unique to function prototypes. To allow the JIT to find these hashes at runtime, the following behaviour of the compiler is used: Two function prototypes are declared, one for BPF programs and another for callbacks: extern unsigned int __bpf_prog_runX(const void *ctx, const struct bpf_insn *insn); extern u64 __bpf_callback_fn(u64, u64, u64, u64, u64); We force a reference to these external symbols: __ADDRESSABLE(__bpf_prog_runX); __ADDRESSABLE(__bpf_callback_fn); This makes the compiler add the following two symbols with the hashes in the symbol table: 00000000d9421881 0 NOTYPE WEAK DEFAULT ABS __kcfi_typeid___bpf_prog_runX 000000009e4709a9 0 NOTYPE WEAK DEFAULT ABS __kcfi_typeid___bpf_callback_fn The JIT can now use the above symbols to emit the hashes in the prologues of the programs and callbacks. For struct_ops trampoline, the bpf_struct_ops_prepare_trampoline() function receives a stub function that would have the hash at (function - 4). The bpf_struct_ops_prepare_trampoline() sets `flags = BPF_TRAMP_F_INDIRECT;` which tells prepare_trampoline() to find the hash before the stub function and emit it in the struct_ops trampoline. Running the selftests causes no CFI warnings: --------------------------------------------- test_progs: Summary: 454/3613 PASSED, 62 SKIPPED, 74 FAILED test_tag: OK (40945 tests) test_verifier: Summary: 789 PASSED, 0 SKIPPED, 0 FAILED ARM64 Doesn't support DYNAMIC_FTRACE_WITH_CALL_OPS when CFI_CLANG is enabled. This causes all tests that attach fentry to kernel functions to fail. While running the selftests, I saw some CFI warnings which were related to static calls. Josh Poimboeuf had sent a patch series[1] last year that includes a patch to fix this issue. Applying this patch and [1] fixes all kCFI issues. [1] https://lore.kernel.org/all/cover.1679456900.git.jpoimboe@kernel.org/ Puranjay Mohan (1): arm64/cfi,bpf: Support kCFI + BPF on arm64 arch/arm64/include/asm/cfi.h | 23 ++++++++++++++ arch/arm64/kernel/alternative.c | 54 +++++++++++++++++++++++++++++++++ arch/arm64/net/bpf_jit_comp.c | 28 +++++++++++++---- 3 files changed, 99 insertions(+), 6 deletions(-) create mode 100644 arch/arm64/include/asm/cfi.h