From patchwork Mon Jul 15 23:01:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733939 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E761C13E88B for ; Mon, 15 Jul 2024 23:02:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084550; cv=none; b=JlI1GGFu/JLvhdhuVuqXtiJKJOpebhnOUTC3OGyp4P4DAVBFxiOXf/POiJvnig3feffpOZmaghdt7hYZGeI3CWQXFYepjQfmsDqJXSuwb72ssKSbm//tIlDY0NKnJunwu3uc14MQ6jj6zRjK6H2dbBeNnDzxpFSRanE/2Jn2Mh4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084550; c=relaxed/simple; bh=7prInfI/5GZApFnoJnQaZKAb4LIudu4TCYFYsFK4BQg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hNaJ91b+NavOni/h3t9AMJXgU76f+FozB1/yj/lTAjpOG4XrsNiwkwBtxDIK2yTs+CBH4N2TErACCjUocqSViet4b/kkLPmfZBuTG6j7mRcePlR/JCKd+6URMvX0eFm1BCe3QFJ0CDXNapq8agx9fp7jkRlnESfkR+M0RFPs0/w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=htWefzjA; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="htWefzjA" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-70b1808dee9so3194848b3a.2 for ; Mon, 15 Jul 2024 16:02:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084548; x=1721689348; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AZu2FQzE3OujfDryDYsk+2bhemJiXfipmm1gVXp59Ag=; b=htWefzjALZ0Mpx3RuDkecwshdmy/E/saVBwYnai3y08szEpNBw+R3V6ox+l+zvYo1f sv7Y5PsCrD3Eo4O0OWpHrjYkanHRkYOT6weMEDKIPWMb+nGiWW7FFQLTQvTmokoNm/m7 znsrPmMq2uGYR09CgoUNw3TKZLIGQnTKKRIhjcp+2reJtCuK8AfQw4O4oofX/NtArO1m uM9JIeScg9m948AV4cSvBB6oP6B3Ye0FL2r5MoO8pik2BQwXXF/2q5MsNpQAOx3y2Ize FZvR47M+za7+BHWHjk8hxuc+eTxGSQazZEs5iJaeZA5LOHPdWXUgvXDiPif1Tje0MrOI XNeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084548; x=1721689348; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AZu2FQzE3OujfDryDYsk+2bhemJiXfipmm1gVXp59Ag=; b=fEkf3VzZ3UNQlKvxOM2gPMeTVCfHkG0RlpZ1GunYDJRmbkcPkbhDwe9U5vkR4QqUeD XRbyvhUPpCC871ednZeB/jlMOdqN+pAEzktihc9D7OSILBfgv/S4b0w96e5n7zlEvr9r grtmGA/1824mKYTBxyd2wQ5QWir2brG+1F2+R1b7/5ndYJ8sjoWBlcyFyLj9Ea58TwCx H19NqDdFtsjTOjUpX4fPkpRgPjFKx7hj0y/mrlt55l08YBmfpPyrZdGLlVbeFVCgpcPp IQAoqBw5dLb3nxZXqaB7F6JelzE7S1VhjXCc+HyUsKpfSUOuz6OhuxMHzdIona7ZOQMv LItA== X-Gm-Message-State: AOJu0Yw+R3CCm9qwDTk69jhrOoBmID7CFONJqSBoum8cToEeBV/KVL+h fxH5pFxdQKKBuCaShZwsbt1N1AtmKvnXAiRF4oPEZcg7FIqE3ZSRwTCz4Q== X-Google-Smtp-Source: AGHT+IGR4N5fTD8OkV6sxz3/zUw7tX8BzszVIGvZAU/XeUocVHv9KZPH4JTX1iDJSEV6lqxapdlzKg== X-Received: by 2002:a05:6a00:b45:b0:704:209a:c59e with SMTP id d2e1a72fcca58-70c1fba0bb1mr634579b3a.9.1721084547695; Mon, 15 Jul 2024 16:02:27 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:27 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 01/12] bpf: add a get_helper_proto() utility function Date: Mon, 15 Jul 2024 16:01:50 -0700 Message-ID: <20240715230201.3901423-2-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Extract the part of check_helper_call() as a utility function allowing to query 'struct bpf_func_proto' for a specific helper function id. Acked-by: Andrii Nakryiko Signed-off-by: Eduard Zingerman --- kernel/bpf/verifier.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index c0263fb5ca4b..3cef46134a51 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10265,6 +10265,19 @@ static void update_loop_inline_state(struct bpf_verifier_env *env, u32 subprogno state->callback_subprogno == subprogno); } +static int get_helper_proto(struct bpf_verifier_env *env, int func_id, + const struct bpf_func_proto **ptr) +{ + if (func_id < 0 || func_id >= __BPF_FUNC_MAX_ID) + return -ERANGE; + + if (!env->ops->get_func_proto) + return -EINVAL; + + *ptr = env->ops->get_func_proto(func_id, env->prog); + return *ptr ? 0 : -EINVAL; +} + static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int *insn_idx_p) { @@ -10281,18 +10294,16 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn /* find function prototype */ func_id = insn->imm; - if (func_id < 0 || func_id >= __BPF_FUNC_MAX_ID) { - verbose(env, "invalid func %s#%d\n", func_id_name(func_id), - func_id); + err = get_helper_proto(env, insn->imm, &fn); + if (err == -ERANGE) { + verbose(env, "invalid func %s#%d\n", func_id_name(func_id), func_id); return -EINVAL; } - if (env->ops->get_func_proto) - fn = env->ops->get_func_proto(func_id, env->prog); - if (!fn) { + if (err) { verbose(env, "program of this type cannot use helper %s#%d\n", func_id_name(func_id), func_id); - return -EINVAL; + return err; } /* eBPF programs must be GPL compatible to use GPL-ed functions */ From patchwork Mon Jul 15 23:01:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733940 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E3611420B0 for ; Mon, 15 Jul 2024 23:02:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084552; cv=none; b=ReCyL5ZzHikKjUkf/bo8gywSUEAOYpk/E0jn8DxSSOWXEF9YLsVfObN6hWAEmt9FhZCXA7yzn+AfqbLVPh23epQO3YzvZX19FQPumupQm6FKKnpEccv7s9/mIlWkQ+DEz8pBr084Jj1qq0+TA4rT0rw69IaaX54vbXinO4Trkv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084552; c=relaxed/simple; bh=DNq1A1XhLcRBq2Hk+xAHQneXK96/mjVLG9ys41XvuSM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i5sQH6A5ioQqHlxEgaTDD/LWj9nKucqufIMxmmf8eiLalQaupoNtCvKZPNU/d1aolwY/E1rLjLhn9GsZTe9JmGcqEKZzQY74n7iZ3TP0MdBjFM0fGI0BrQrvSI3ZFlncW8kQMzS9Ybv3VpEVONThecgZuB2s4/GivF9k7bkMP/o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lE1AbKkY; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lE1AbKkY" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-706a1711ee5so3259047b3a.0 for ; Mon, 15 Jul 2024 16:02:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084549; x=1721689349; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2Fx3F7dHXRV9BUDQp9vJnfzcDopad10RLedxh0oQuOo=; b=lE1AbKkYTyamZdIc5z4m8IIaaw9uZESTTONRlxsJztSDRxGFQcnhhg1SeOGjnlMWY2 hPJWzlQ5PEHHcYOhVRltV6MFWw7flAr8pbkSV0+KVQ116YteJrF8F2hYH56Il8c29VAE jkXuXFC6GGzxqfhBAX/lxV5FwnEd1hhkrxEKFDJJit9zFogdJq/n1uH6y25u5UPMhYRe 0syewwqYZ3y1N4+HRUoLWflua7OsNf4N0soZFpNABUW7ysNcuDYrT+G734HfB8qscTFk fi6muZkEguLs3gSRp1ColDBSMwHBi0R/8qh8ALvHy0a8jUhC9rCjsq6Rip5jUw2a0jDO Pnog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084549; x=1721689349; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2Fx3F7dHXRV9BUDQp9vJnfzcDopad10RLedxh0oQuOo=; b=nG0JbGr4AGyPfNx+m4yawAmgvQprfH4ubbrysquBkqBkYMjQVwDcHnqN48p+Q0e+5+ IKCKlamnnMT9IYkcFsAcClyGy45D9UXL8JU7hY6DRZr5LBrI9nYpFwoQBxOqD4pEE7b2 ILf9h277aouavF6SCT69rUqBGjrVE+o92RQwkRv5lVIQeOzMnf+K1mbjct05uHhqjcI1 CGITz1zX08RWGrC5aOML4Eqysnlw/AIpSvNPW12zbQzw6S4uri3hrI/d3qPIFOP6jzOv 3EslwKhpHxwXaOiafQTzLVW3SxYJ730+ivhCifLCVadlI8IR8+joz9KaYlaw/u5Y0NSV xwGw== X-Gm-Message-State: AOJu0YyLzn4yvlDa+1aoNR7791/QLhQUWX/NXcPbE+F3IRfBstH0MXE+ F12GtWP6dBXBjgrTBVcjQIv8jnR5fAm732/F+Eel402TI5A5ZWZ/ZgwQ6Q== X-Google-Smtp-Source: AGHT+IHndPZScSN7PyCPmz/ym02bVfYUvk5v5TAymRrs5O5ID0Os98Qc+NzN8ijCMenRFYkBeo/Ugg== X-Received: by 2002:a05:6a00:4f85:b0:705:e5da:8293 with SMTP id d2e1a72fcca58-70c1fba11c6mr639052b3a.12.1721084548938; Mon, 15 Jul 2024 16:02:28 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:28 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman , Alexei Starovoitov Subject: [bpf-next v3 02/12] bpf: no_caller_saved_registers attribute for helper calls Date: Mon, 15 Jul 2024 16:01:51 -0700 Message-ID: <20240715230201.3901423-3-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net GCC and LLVM define a no_caller_saved_registers function attribute. This attribute means that function scratches only some of the caller saved registers defined by ABI. For BPF the set of such registers could be defined as follows: - R0 is scratched only if function is non-void; - R1-R5 are scratched only if corresponding parameter type is defined in the function prototype. This commit introduces flag bpf_func_prot->allow_nocsr. If this flag is set for some helper function, verifier assumes that it follows no_caller_saved_registers calling convention. The contract between kernel and clang allows to simultaneously use such functions and maintain backwards compatibility with old kernels that don't understand no_caller_saved_registers calls (nocsr for short): - clang generates a simple pattern for nocsr calls, e.g.: r1 = 1; r2 = 2; *(u64 *)(r10 - 8) = r1; *(u64 *)(r10 - 16) = r2; call %[to_be_inlined] r2 = *(u64 *)(r10 - 16); r1 = *(u64 *)(r10 - 8); r0 = r1; r0 += r2; exit; - kernel removes unnecessary spills and fills, if called function is inlined by verifier or current JIT (with assumption that patch inserted by verifier or JIT honors nocsr contract, e.g. does not scratch r3-r5 for the example above), e.g. the code above would be transformed to: r1 = 1; r2 = 2; call %[to_be_inlined] r0 = r1; r0 += r2; exit; Technically, the transformation is split into the following phases: - function mark_nocsr_patterns(), called from bpf_check() searches and marks potential patterns in instruction auxiliary data; - upon stack read or write access, function check_nocsr_stack_contract() is used to verify if stack offsets, presumably reserved for nocsr patterns, are used only from those patterns; - function remove_nocsr_spills_fills(), called from bpf_check(), applies the rewrite for valid patterns. See comment in mark_nocsr_pattern_for_call() for more details. Suggested-by: Alexei Starovoitov Signed-off-by: Eduard Zingerman --- include/linux/bpf.h | 6 + include/linux/bpf_verifier.h | 14 ++ kernel/bpf/verifier.c | 295 ++++++++++++++++++++++++++++++++++- 3 files changed, 314 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 4f1d4a97b9d1..7640ab047188 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -807,6 +807,12 @@ struct bpf_func_proto { bool gpl_only; bool pkt_access; bool might_sleep; + /* set to true if helper follows contract for gcc/llvm + * attribute no_caller_saved_registers: + * - void functions do not scratch r0 + * - functions taking N arguments scratch only registers r1-rN + */ + bool allow_nocsr; enum bpf_return_type ret_type; union { struct { diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index e98ba5a5cf79..1c0543c64ccd 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -572,6 +572,14 @@ struct bpf_insn_aux_data { bool is_iter_next; /* bpf_iter__next() kfunc call */ bool call_with_percpu_alloc_ptr; /* {this,per}_cpu_ptr() with prog percpu alloc */ u8 alu_state; /* used in combination with alu_limit */ + /* true if STX or LDX instruction is a part of a spill/fill + * pattern for a no_caller_saved_registers call. + */ + u8 nocsr_pattern:1; + /* for CALL instructions, a number of spill/fill pairs in the + * no_caller_saved_registers pattern. + */ + u8 nocsr_spills_num:3; /* below fields are initialized once */ unsigned int orig_idx; /* original instruction index */ @@ -641,6 +649,10 @@ struct bpf_subprog_info { u32 linfo_idx; /* The idx to the main_prog->aux->linfo */ u16 stack_depth; /* max. stack depth used by this function */ u16 stack_extra; + /* offsets in range [stack_depth .. nocsr_stack_off) + * are used for no_caller_saved_registers spills and fills. + */ + s16 nocsr_stack_off; bool has_tail_call: 1; bool tail_call_reachable: 1; bool has_ld_abs: 1; @@ -648,6 +660,8 @@ struct bpf_subprog_info { bool is_async_cb: 1; bool is_exception_cb: 1; bool args_cached: 1; + /* true if nocsr stack region is used by functions that can't be inlined */ + bool keep_nocsr_stack: 1; u8 arg_cnt; struct bpf_subprog_arg_info args[MAX_BPF_FUNC_REG_ARGS]; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 3cef46134a51..163b6b0f2fa7 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4502,6 +4502,23 @@ static int get_reg_width(struct bpf_reg_state *reg) return fls64(reg->umax_value); } +/* See comment for mark_nocsr_pattern_for_call() */ +static void check_nocsr_stack_contract(struct bpf_verifier_env *env, struct bpf_func_state *state, + int insn_idx, int off) +{ + struct bpf_subprog_info *subprog = &env->subprog_info[state->subprogno]; + struct bpf_insn_aux_data *aux = &env->insn_aux_data[insn_idx]; + + if (subprog->nocsr_stack_off <= off || aux->nocsr_pattern) + return; + /* access to the region [max_stack_depth .. nocsr_stack_off) + * from something that is not a part of the nocsr pattern, + * disable nocsr rewrites for current subprogram by setting + * nocsr_stack_off to a value smaller than any possible offset. + */ + subprog->nocsr_stack_off = S16_MIN; +} + /* check_stack_{read,write}_fixed_off functions track spill/fill of registers, * stack boundary and alignment are checked in check_mem_access() */ @@ -4550,6 +4567,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env, if (err) return err; + check_nocsr_stack_contract(env, state, insn_idx, off); mark_stack_slot_scratched(env, spi); if (reg && !(off % BPF_REG_SIZE) && reg->type == SCALAR_VALUE && env->bpf_capable) { bool reg_value_fits; @@ -4684,6 +4702,7 @@ static int check_stack_write_var_off(struct bpf_verifier_env *env, return err; } + check_nocsr_stack_contract(env, state, insn_idx, min_off); /* Variable offset writes destroy any spilled pointers in range. */ for (i = min_off; i < max_off; i++) { u8 new_type, *stype; @@ -4822,6 +4841,7 @@ static int check_stack_read_fixed_off(struct bpf_verifier_env *env, reg = ®_state->stack[spi].spilled_ptr; mark_stack_slot_scratched(env, spi); + check_nocsr_stack_contract(env, state, env->insn_idx, off); if (is_spilled_reg(®_state->stack[spi])) { u8 spill_size = 1; @@ -4982,6 +5002,7 @@ static int check_stack_read_var_off(struct bpf_verifier_env *env, min_off = reg->smin_value + off; max_off = reg->smax_value + off; mark_reg_stack_read(env, ptr_state, min_off, max_off + size, dst_regno); + check_nocsr_stack_contract(env, ptr_state, env->insn_idx, min_off); return 0; } @@ -15963,6 +15984,232 @@ static int visit_func_call_insn(int t, struct bpf_insn *insns, return ret; } +/* Bitmask with 1s for all caller saved registers */ +#define ALL_CALLER_SAVED_REGS ((1u << CALLER_SAVED_REGS) - 1) + +/* Return a bitmask specifying which caller saved registers are + * clobbered by a call to a helper *as if* this helper follows + * no_caller_saved_registers contract: + * - includes R0 if function is non-void; + * - includes R1-R5 if corresponding parameter has is described + * in the function prototype. + */ +static u32 helper_nocsr_clobber_mask(const struct bpf_func_proto *fn) +{ + u8 mask; + int i; + + mask = 0; + if (fn->ret_type != RET_VOID) + mask |= BIT(BPF_REG_0); + for (i = 0; i < ARRAY_SIZE(fn->arg_type); ++i) + if (fn->arg_type[i] != ARG_DONTCARE) + mask |= BIT(BPF_REG_1 + i); + return mask; +} + +/* True if do_misc_fixups() replaces calls to helper number 'imm', + * replacement patch is presumed to follow no_caller_saved_registers contract + * (see mark_nocsr_pattern_for_call() below). + */ +static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) +{ + return false; +} + +/* GCC and LLVM define a no_caller_saved_registers function attribute. + * This attribute means that function scratches only some of + * the caller saved registers defined by ABI. + * For BPF the set of such registers could be defined as follows: + * - R0 is scratched only if function is non-void; + * - R1-R5 are scratched only if corresponding parameter type is defined + * in the function prototype. + * + * The contract between kernel and clang allows to simultaneously use + * such functions and maintain backwards compatibility with old + * kernels that don't understand no_caller_saved_registers calls + * (nocsr for short): + * + * - for nocsr calls clang allocates registers as-if relevant r0-r5 + * registers are not scratched by the call; + * + * - as a post-processing step, clang visits each nocsr call and adds + * spill/fill for every live r0-r5; + * + * - stack offsets used for the spill/fill are allocated as lowest + * stack offsets in whole function and are not used for any other + * purposes; + * + * - when kernel loads a program, it looks for such patterns + * (nocsr function surrounded by spills/fills) and checks if + * spill/fill stack offsets are used exclusively in nocsr patterns; + * + * - if so, and if verifier or current JIT inlines the call to the + * nocsr function (e.g. a helper call), kernel removes unnecessary + * spill/fill pairs; + * + * - when old kernel loads a program, presence of spill/fill pairs + * keeps BPF program valid, albeit slightly less efficient. + * + * For example: + * + * r1 = 1; + * r2 = 2; + * *(u64 *)(r10 - 8) = r1; r1 = 1; + * *(u64 *)(r10 - 16) = r2; r2 = 2; + * call %[to_be_inlined] --> call %[to_be_inlined] + * r2 = *(u64 *)(r10 - 16); r0 = r1; + * r1 = *(u64 *)(r10 - 8); r0 += r2; + * r0 = r1; exit; + * r0 += r2; + * exit; + * + * The purpose of mark_nocsr_pattern_for_call is to: + * - look for such patterns; + * - mark spill and fill instructions in env->insn_aux_data[*].nocsr_pattern; + * - mark set env->insn_aux_data[*].nocsr_spills_num for call instruction; + * - update env->subprog_info[*]->nocsr_stack_off to find an offset + * at which nocsr spill/fill stack slots start; + * - update env->subprog_info[*]->keep_nocsr_stack. + * + * The .nocsr_pattern and .nocsr_stack_off are used by + * check_nocsr_stack_contract() to check if every stack access to + * nocsr spill/fill stack slot originates from spill/fill + * instructions, members of nocsr patterns. + * + * If such condition holds true for a subprogram, nocsr patterns could + * be rewritten by remove_nocsr_spills_fills(). + * Otherwise nocsr patterns are not changed in the subprogram + * (code, presumably, generated by an older clang version). + * + * For example, it is *not* safe to remove spill/fill below: + * + * r1 = 1; + * *(u64 *)(r10 - 8) = r1; r1 = 1; + * call %[to_be_inlined] --> call %[to_be_inlined] + * r1 = *(u64 *)(r10 - 8); r0 = *(u64 *)(r10 - 8); <---- wrong !!! + * r0 = *(u64 *)(r10 - 8); r0 += r1; + * r0 += r1; exit; + * exit; + */ +static void mark_nocsr_pattern_for_call(struct bpf_verifier_env *env, + struct bpf_subprog_info *subprog, + int insn_idx, s16 lowest_off) +{ + struct bpf_insn *insns = env->prog->insnsi, *stx, *ldx; + struct bpf_insn *call = &env->prog->insnsi[insn_idx]; + const struct bpf_func_proto *fn; + u32 clobbered_regs_mask = ALL_CALLER_SAVED_REGS; + u32 expected_regs_mask; + bool can_be_inlined = false; + s16 off; + int i; + + if (bpf_helper_call(call)) { + if (get_helper_proto(env, call->imm, &fn) < 0) + /* error would be reported later */ + return; + clobbered_regs_mask = helper_nocsr_clobber_mask(fn); + can_be_inlined = fn->allow_nocsr && + (verifier_inlines_helper_call(env, call->imm) || + bpf_jit_inlines_helper_call(call->imm)); + } + + if (clobbered_regs_mask == ALL_CALLER_SAVED_REGS) + return; + + /* e.g. if helper call clobbers r{0,1}, expect r{2,3,4,5} in the pattern */ + expected_regs_mask = ~clobbered_regs_mask & ALL_CALLER_SAVED_REGS; + + /* match pairs of form: + * + * *(u64 *)(r10 - Y) = rX (where Y % 8 == 0) + * ... + * call %[to_be_inlined] + * ... + * rX = *(u64 *)(r10 - Y) + */ + for (i = 1, off = lowest_off; i <= ARRAY_SIZE(caller_saved); ++i, off += BPF_REG_SIZE) { + if (insn_idx - i < 0 || insn_idx + i >= env->prog->len) + break; + stx = &insns[insn_idx - i]; + ldx = &insns[insn_idx + i]; + /* must be a stack spill/fill pair */ + if (stx->code != (BPF_STX | BPF_MEM | BPF_DW) || + ldx->code != (BPF_LDX | BPF_MEM | BPF_DW) || + stx->dst_reg != BPF_REG_10 || + ldx->src_reg != BPF_REG_10) + break; + /* must be a spill/fill for the same reg */ + if (stx->src_reg != ldx->dst_reg) + break; + /* must be one of the previously unseen registers */ + if ((BIT(stx->src_reg) & expected_regs_mask) == 0) + break; + /* must be a spill/fill for the same expected offset, + * no need to check offset alignment, BPF_DW stack access + * is always 8-byte aligned. + */ + if (stx->off != off || ldx->off != off) + break; + expected_regs_mask &= ~BIT(stx->src_reg); + env->insn_aux_data[insn_idx - i].nocsr_pattern = 1; + env->insn_aux_data[insn_idx + i].nocsr_pattern = 1; + } + if (i == 1) + return; + + /* Conditionally set 'nocsr_spills_num' to allow forward + * compatibility when more helper functions are marked as + * nocsr at compile time than current kernel supports, e.g: + * + * 1: *(u64 *)(r10 - 8) = r1 + * 2: call A ;; assume A is nocsr for current kernel + * 3: r1 = *(u64 *)(r10 - 8) + * 4: *(u64 *)(r10 - 8) = r1 + * 5: call B ;; assume B is not nocsr for current kernel + * 6: r1 = *(u64 *)(r10 - 8) + * + * There is no need to block nocsr rewrite for such program. + * Set 'nocsr_pattern' for both calls to keep check_nocsr_stack_contract() happy, + * don't set 'nocsr_spills_num' for call B so that remove_nocsr_spills_fills() + * does not remove spill/fill pair {4,6}. + */ + if (can_be_inlined) + env->insn_aux_data[insn_idx].nocsr_spills_num = i - 1; + else + subprog->keep_nocsr_stack = 1; + subprog->nocsr_stack_off = min(subprog->nocsr_stack_off, off); +} + +static int mark_nocsr_patterns(struct bpf_verifier_env *env) +{ + struct bpf_subprog_info *subprog = env->subprog_info; + struct bpf_insn *insn; + s16 lowest_off; + int s, i; + + for (s = 0; s < env->subprog_cnt; ++s, ++subprog) { + /* find lowest stack spill offset used in this subprog */ + lowest_off = 0; + for (i = subprog->start; i < (subprog + 1)->start; ++i) { + insn = env->prog->insnsi + i; + if (insn->code != (BPF_STX | BPF_MEM | BPF_DW) || + insn->dst_reg != BPF_REG_10) + continue; + lowest_off = min(lowest_off, insn->off); + } + /* use this offset to find nocsr patterns */ + for (i = subprog->start; i < (subprog + 1)->start; ++i) { + insn = env->prog->insnsi + i; + if (insn->code != (BPF_JMP | BPF_CALL)) + continue; + mark_nocsr_pattern_for_call(env, subprog, i, lowest_off); + } + } + return 0; +} + /* Visits the instruction at index t and returns one of the following: * < 0 - an error occurred * DONE_EXPLORING - the instruction was fully explored @@ -19109,9 +19356,11 @@ static int opt_remove_dead_code(struct bpf_verifier_env *env) return 0; } +static const struct bpf_insn NOP = BPF_JMP_IMM(BPF_JA, 0, 0, 0); + static int opt_remove_nops(struct bpf_verifier_env *env) { - const struct bpf_insn ja = BPF_JMP_IMM(BPF_JA, 0, 0, 0); + const struct bpf_insn ja = NOP; struct bpf_insn *insn = env->prog->insnsi; int insn_cnt = env->prog->len; int i, err; @@ -20857,6 +21106,40 @@ static int optimize_bpf_loop(struct bpf_verifier_env *env) return 0; } +/* Remove unnecessary spill/fill pairs, members of nocsr pattern, + * adjust subprograms stack depth when possible. + */ +static int remove_nocsr_spills_fills(struct bpf_verifier_env *env) +{ + struct bpf_subprog_info *subprog = env->subprog_info; + struct bpf_insn_aux_data *aux = env->insn_aux_data; + struct bpf_insn *insn = env->prog->insnsi; + int insn_cnt = env->prog->len; + u32 spills_num; + bool modified = false; + int i, j; + + for (i = 0; i < insn_cnt; i++, insn++) { + if (aux[i].nocsr_spills_num > 0 && subprog->nocsr_stack_off > S16_MIN) { + spills_num = aux[i].nocsr_spills_num; + /* NOPs would be removed by opt_remove_nops() */ + for (j = 1; j <= spills_num; ++j) { + *(insn - j) = NOP; + *(insn + j) = NOP; + } + modified = true; + } + if ((subprog + 1)->start == i + 1) { + if (modified && !subprog->keep_nocsr_stack) + subprog->stack_depth = -subprog->nocsr_stack_off; + subprog++; + modified = false; + } + } + + return 0; +} + static void free_states(struct bpf_verifier_env *env) { struct bpf_verifier_state_list *sl, *sln; @@ -21759,6 +22042,10 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3 if (ret < 0) goto skip_full_check; + ret = mark_nocsr_patterns(env); + if (ret < 0) + goto skip_full_check; + ret = do_check_main(env); ret = ret ?: do_check_subprogs(env); @@ -21771,6 +22058,12 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3 if (ret == 0) ret = check_max_stack_depth(env); + /* might decrease stack depth, keep it before passes that + * allocate additional slots. + */ + if (ret == 0) + ret = remove_nocsr_spills_fills(env); + /* instruction rewrites happen after this point */ if (ret == 0) ret = optimize_bpf_loop(env); From patchwork Mon Jul 15 23:01:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733941 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-oa1-f50.google.com (mail-oa1-f50.google.com [209.85.160.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 735B9143892 for ; Mon, 15 Jul 2024 23:02:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084552; cv=none; b=XrthHTUuDd4RKRJSIqe5dSxQ2hUTLXJiDEwrn3A21uzK9F5ag8tLxKEJbqpmAF5OA0TarhC9VoYQjt/ITzGd2Dl4f0CvWitZ31Q+xZeQIH7rSXF6Bb/8uzTR0rRu5ULuAjIXZpgQHOpQhHeAPyYaJqifyEPjazdfLBQjiv5fvD4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084552; c=relaxed/simple; bh=CAnlWEKZ3l2x0UK3l9LyFi6XfFtVv2r33jVfpRuutK0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=N0ZfQjma3bB6BcJ5wDhwYDPIxG90I81CGLXyugjYULDiLUQyW81m7hsNzlwxhJ70k27tbzyTP035/M4OTVuQRaF7h6pboWN5RGvE/5HImAsc99DAPZbu5en4d+4bfLhM/WR5307EEKOw9J0bO8eklzzpQoN4STtyFz4Ayw4F88U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=LTAhmS2i; arc=none smtp.client-ip=209.85.160.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LTAhmS2i" Received: by mail-oa1-f50.google.com with SMTP id 586e51a60fabf-25e2cc76becso2281641fac.3 for ; Mon, 15 Jul 2024 16:02:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084550; x=1721689350; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8u0f4ksXPNvwIFN8EtLn7J6PYIR0lZP9WxJGDTeEJ0o=; b=LTAhmS2iAzXOGgwCH8gNWsjqSEjwZtqPD71Clszso96jMEaF0zTWsrFEjefTAeE3U9 0hi56BTmN7Sf720GhQcEw3vq0QaKmM/zT4vCqSARmzCkLGHY/WLSFAJGgZxKwPE72IiD meNWL5UjorxjMlo1b0W/i59EURdpm1tGZDUXgSh6pACnhbI++MT3mnThbUvci3+E0uMF fU0B7MMENpAtoVbapzu9BDY1tLlzYVYtLsSkY2I5uetkrb5Lu0Ci+Gw+w4Y39c65r1tY H5ZKv+jaDvxQjNwMsghyjEYiTq76PNgcuUKgG1Nm7Y+v+I+U9zZIOEAMrD1qxeyhspw2 zAOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084550; x=1721689350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8u0f4ksXPNvwIFN8EtLn7J6PYIR0lZP9WxJGDTeEJ0o=; b=ZFxUwb+LwJ0cxzL+HQolOcWyHlyKK7nVc/+4Oj5yvf7eMCD3/SqeMSgyXqmHYx9SpW V0QXrecJD5FhZQfup3Erg5TCAtuE0ZFqQLGwtdcUgE5YZnuQSQgQeQGQSON9gCKhMGUc cVOo/gRYeUHERx9BeDiba2s6CvFhEq73KX9SpgvxDm1jArFeH+vjrEYKhlOUBzttOWFs EEHYjatMwIqXbx29EKArVam8eNdIywAgTHZOVkLnos+zYl+fcfb1pNNTf97FbvyHDOs3 rv0X2RA3ILiu2kb60TjSG0hEHhvAKQXhyhdUfr1/QL52noVW69KXGT5Fv+ls6hD2W1it oAYA== X-Gm-Message-State: AOJu0YwRY5c+JkvjJVzUJPg3QKyKw1v4AX27ztyb08PS4sqWj/z03Xnn PFl6KLH5rRUSo6BJxRBc9UDeQxO5EsS36Y3D7dFgediVmykYSh6uBJLaCQ== X-Google-Smtp-Source: AGHT+IH1meRgy2NORA38EcaoJ/g2uyvTAraCmVGxLE3bOAq8G6r666QcX816dEXZMDqcEu827ktA2g== X-Received: by 2002:a05:6870:50:b0:25e:b8a5:7b02 with SMTP id 586e51a60fabf-260bdddd8f1mr153142fac.29.1721084550120; Mon, 15 Jul 2024 16:02:30 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:29 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 03/12] bpf, x86, riscv, arm: no_caller_saved_registers for bpf_get_smp_processor_id() Date: Mon, 15 Jul 2024 16:01:52 -0700 Message-ID: <20240715230201.3901423-4-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The function bpf_get_smp_processor_id() is processed in a different way, depending on the arch: - on x86 verifier replaces call to bpf_get_smp_processor_id() with a sequence of instructions that modify only r0; - on riscv64 jit replaces call to bpf_get_smp_processor_id() with a sequence of instructions that modify only r0; - on arm64 jit replaces call to bpf_get_smp_processor_id() with a sequence of instructions that modify only r0 and tmp registers. These rewrites satisfy attribute no_caller_saved_registers contract. Allow rewrite of no_caller_saved_registers patterns for bpf_get_smp_processor_id() in order to use this function as a canary for no_caller_saved_registers tests. Signed-off-by: Eduard Zingerman --- kernel/bpf/helpers.c | 1 + kernel/bpf/verifier.c | 11 +++++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 5241ba671c5a..e7b4c059ebaf 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -158,6 +158,7 @@ const struct bpf_func_proto bpf_get_smp_processor_id_proto = { .func = bpf_get_smp_processor_id, .gpl_only = false, .ret_type = RET_INTEGER, + .allow_nocsr = true, }; BPF_CALL_0(bpf_get_numa_node_id) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 163b6b0f2fa7..438daf36a694 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -16014,7 +16014,14 @@ static u32 helper_nocsr_clobber_mask(const struct bpf_func_proto *fn) */ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) { - return false; + switch (imm) { +#ifdef CONFIG_X86_64 + case BPF_FUNC_get_smp_processor_id: + return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); +#endif + default: + return false; + } } /* GCC and LLVM define a no_caller_saved_registers function attribute. @@ -20716,7 +20723,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env) #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) /* Implement bpf_get_smp_processor_id() inline. */ if (insn->imm == BPF_FUNC_get_smp_processor_id && - prog->jit_requested && bpf_jit_supports_percpu_insn()) { + verifier_inlines_helper_call(env, insn->imm)) { /* BPF_FUNC_get_smp_processor_id inlining is an * optimization, so if pcpu_hot.cpu_number is ever * changed in some incompatible and hard to support From patchwork Mon Jul 15 23:01:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733942 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F9F8143C49 for ; Mon, 15 Jul 2024 23:02:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084553; cv=none; b=otH3mixhUPcN+gafwehPjsbbaNTXqf0ekg/7PFYIZA+FQrdryr19nesQWIxQTSukE6vQ7GGdswTkBYJI/uTyDf4b9iOq3uscy18jlQM35Tdf39ZY6JJI5ZuHcC7CfC1ENWrnIeIE5rEhqxcX+bzEI97qWLGLsKSe43XcQJldkUA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084553; c=relaxed/simple; bh=qEWSHaY6YxLBfqtxrK2EEvtoM/35ocsIieGV3/yigcs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PM6aXYGeSVy5EXqWR669BrXytNQs1hkVsVFqJkPie8gScGZEMNfj80RvY25qrXSB8ORLPLOSilEPetNehR2nB1tCYW0+KvGOhF6bf4jrPyDNdCt5sj3HAwQI5tqSD+TufLrPaKhHY1jsU3yXjfZRzqoZT0YXaCpbuiu6pnIKnmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cBfaixrp; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cBfaixrp" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-706a1711ee5so3259065b3a.0 for ; Mon, 15 Jul 2024 16:02:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084551; x=1721689351; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6OS8NdMPZNHGoTxtKNDKNnwmxe8ypUk8+Q4SCQfNAw4=; b=cBfaixrpkFY264zZp1QR7Q3OVMsb6TvQLy4S9blEFsG9LmCGiqrinKI7FZjPyaAWIi OepDkvAMgEPnzdctoHlEkCt2UVJitrB6Y+Awaa93s8asXLhLFzrqhhUTgLAirOiRNt4w 2EMOuxfwXgVk8VC0X/ZeXpCRFPgZjVXb9xYYi66zDGyZcbaAyhN7vQ7Lpp4lVFpoI23n FKC99flC/3VeRz9tV0rYN9iYXOFfg5R7f/k2oaTyWXLYTNezJVW9S82lA6UD+6JAZoLy OBu+BBV9NaktqDWuy4X5MKCKy5xRuxtJZyCYHVCebnlSN7pAFePP0abZ1V5pQAPtxcgB WvHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084551; x=1721689351; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6OS8NdMPZNHGoTxtKNDKNnwmxe8ypUk8+Q4SCQfNAw4=; b=GFUVdFyDhjUvT2j9jcKItkSCfKovn6odU5799ftMLs8owJ7zWVTh+KObUyoz8QHd+3 /VyixC+//oOZoj2feEQJk1ASwBfgUv1pZiacLOFfIBgBwu8c+NfB/YM618kGBEbi1NPi 0q/W08ovffD5OP4XwPlCI8/zKF5+tgKjsLUOTw/d6cGlL5A/jFLAmHZKWlQNI5hnbgsO cD/UJFTSZqqAseDIMXPR8l7Qbk5Hrwk9ufODbqMBTYicoE/KFdY0pTfCWz40DT9QqW2U pUfOAOIYzVkZLVHJTDegWx7+YN1UMBrQG+yAQMD2PKhpgRUTFrhCNPTHY2Vxlc+1wtdm 70lQ== X-Gm-Message-State: AOJu0YzupvXsRJDnCP0w3kFvy+YW5nMF1xCjWqPLL9HcHBVYWoGpUB98 MRAaKpodLO5KsQKqngjgxB2rNMfcwhxvPvN7Sw1D53N6kVwnulfpfmci0w== X-Google-Smtp-Source: AGHT+IEDGjLtoxEje4Dh6ZDX+LrG9smbQY6/qt4QvvdfB06GmUS5pKrsvuXNeqRcp9D+Fjkd4CymTw== X-Received: by 2002:a05:6a00:4f85:b0:705:e5da:8293 with SMTP id d2e1a72fcca58-70c1fba11c6mr639165b3a.12.1721084551019; Mon, 15 Jul 2024 16:02:31 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:30 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 04/12] selftests/bpf: extract utility function for BPF disassembly Date: Mon, 15 Jul 2024 16:01:53 -0700 Message-ID: <20240715230201.3901423-5-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net struct bpf_insn *disasm_insn(struct bpf_insn *insn, char *buf, size_t buf_sz); Disassembles instruction 'insn' to a text buffer 'buf'. Removes insn->code hex prefix added by kernel disassembly routine. Returns a pointer to the next instruction (increments insn by either 1 or 2). Acked-by: Andrii Nakryiko Signed-off-by: Eduard Zingerman --- tools/testing/selftests/bpf/Makefile | 1 + tools/testing/selftests/bpf/disasm_helpers.c | 51 +++++++++++++ tools/testing/selftests/bpf/disasm_helpers.h | 12 +++ .../selftests/bpf/prog_tests/ctx_rewrite.c | 74 +++---------------- tools/testing/selftests/bpf/testing_helpers.c | 1 + 5 files changed, 75 insertions(+), 64 deletions(-) create mode 100644 tools/testing/selftests/bpf/disasm_helpers.c create mode 100644 tools/testing/selftests/bpf/disasm_helpers.h diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index dd49c1d23a60..fcacc693ed8a 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -636,6 +636,7 @@ TRUNNER_EXTRA_SOURCES := test_progs.c \ test_loader.c \ xsk.c \ disasm.c \ + disasm_helpers.c \ json_writer.c \ flow_dissector_load.h \ ip_check_defrag_frags.h diff --git a/tools/testing/selftests/bpf/disasm_helpers.c b/tools/testing/selftests/bpf/disasm_helpers.c new file mode 100644 index 000000000000..96b1f2ffe438 --- /dev/null +++ b/tools/testing/selftests/bpf/disasm_helpers.c @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) + +#include +#include "disasm.h" + +struct print_insn_context { + char *buf; + size_t sz; +}; + +static void print_insn_cb(void *private_data, const char *fmt, ...) +{ + struct print_insn_context *ctx = private_data; + va_list args; + + va_start(args, fmt); + vsnprintf(ctx->buf, ctx->sz, fmt, args); + va_end(args); +} + +struct bpf_insn *disasm_insn(struct bpf_insn *insn, char *buf, size_t buf_sz) +{ + struct print_insn_context ctx = { + .buf = buf, + .sz = buf_sz, + }; + struct bpf_insn_cbs cbs = { + .cb_print = print_insn_cb, + .private_data = &ctx, + }; + char *tmp, *pfx_end, *sfx_start; + bool double_insn; + int len; + + print_bpf_insn(&cbs, insn, true); + /* We share code with kernel BPF disassembler, it adds '(FF) ' prefix + * for each instruction (FF stands for instruction `code` byte). + * Remove the prefix inplace, and also simplify call instructions. + * E.g.: "(85) call foo#10" -> "call foo". + * Also remove newline in the end (the 'max(strlen(buf) - 1, 0)' thing). + */ + pfx_end = buf + 5; + sfx_start = buf + max((int)strlen(buf) - 1, 0); + if (strncmp(pfx_end, "call ", 5) == 0 && (tmp = strrchr(buf, '#'))) + sfx_start = tmp; + len = sfx_start - pfx_end; + memmove(buf, pfx_end, len); + buf[len] = 0; + double_insn = insn->code == (BPF_LD | BPF_IMM | BPF_DW); + return insn + (double_insn ? 2 : 1); +} diff --git a/tools/testing/selftests/bpf/disasm_helpers.h b/tools/testing/selftests/bpf/disasm_helpers.h new file mode 100644 index 000000000000..7b26cab70099 --- /dev/null +++ b/tools/testing/selftests/bpf/disasm_helpers.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ + +#ifndef __DISASM_HELPERS_H +#define __DISASM_HELPERS_H + +#include + +struct bpf_insn; + +struct bpf_insn *disasm_insn(struct bpf_insn *insn, char *buf, size_t buf_sz); + +#endif /* __DISASM_HELPERS_H */ diff --git a/tools/testing/selftests/bpf/prog_tests/ctx_rewrite.c b/tools/testing/selftests/bpf/prog_tests/ctx_rewrite.c index 08b6391f2f56..dd75ccb03770 100644 --- a/tools/testing/selftests/bpf/prog_tests/ctx_rewrite.c +++ b/tools/testing/selftests/bpf/prog_tests/ctx_rewrite.c @@ -10,7 +10,8 @@ #include "bpf/btf.h" #include "bpf_util.h" #include "linux/filter.h" -#include "disasm.h" +#include "linux/kernel.h" +#include "disasm_helpers.h" #define MAX_PROG_TEXT_SZ (32 * 1024) @@ -628,63 +629,6 @@ static bool match_pattern(struct btf *btf, char *pattern, char *text, char *reg_ return false; } -static void print_insn(void *private_data, const char *fmt, ...) -{ - va_list args; - - va_start(args, fmt); - vfprintf((FILE *)private_data, fmt, args); - va_end(args); -} - -/* Disassemble instructions to a stream */ -static void print_xlated(FILE *out, struct bpf_insn *insn, __u32 len) -{ - const struct bpf_insn_cbs cbs = { - .cb_print = print_insn, - .cb_call = NULL, - .cb_imm = NULL, - .private_data = out, - }; - bool double_insn = false; - int i; - - for (i = 0; i < len; i++) { - if (double_insn) { - double_insn = false; - continue; - } - - double_insn = insn[i].code == (BPF_LD | BPF_IMM | BPF_DW); - print_bpf_insn(&cbs, insn + i, true); - } -} - -/* We share code with kernel BPF disassembler, it adds '(FF) ' prefix - * for each instruction (FF stands for instruction `code` byte). - * This function removes the prefix inplace for each line in `str`. - */ -static void remove_insn_prefix(char *str, int size) -{ - const int prefix_size = 5; - - int write_pos = 0, read_pos = prefix_size; - int len = strlen(str); - char c; - - size = min(size, len); - - while (read_pos < size) { - c = str[read_pos++]; - if (c == 0) - break; - str[write_pos++] = c; - if (c == '\n') - read_pos += prefix_size; - } - str[write_pos] = 0; -} - struct prog_info { char *prog_kind; enum bpf_prog_type prog_type; @@ -699,9 +643,10 @@ static void match_program(struct btf *btf, char *reg_map[][2], bool skip_first_insn) { - struct bpf_insn *buf = NULL; + struct bpf_insn *buf = NULL, *insn, *insn_end; int err = 0, prog_fd = 0; FILE *prog_out = NULL; + char insn_buf[64]; char *text = NULL; __u32 cnt = 0; @@ -739,12 +684,13 @@ static void match_program(struct btf *btf, PRINT_FAIL("Can't open memory stream\n"); goto out; } - if (skip_first_insn) - print_xlated(prog_out, buf + 1, cnt - 1); - else - print_xlated(prog_out, buf, cnt); + insn_end = buf + cnt; + insn = buf + (skip_first_insn ? 1 : 0); + while (insn < insn_end) { + insn = disasm_insn(insn, insn_buf, sizeof(insn_buf)); + fprintf(prog_out, "%s\n", insn_buf); + } fclose(prog_out); - remove_insn_prefix(text, MAX_PROG_TEXT_SZ); ASSERT_TRUE(match_pattern(btf, pattern, text, reg_map), pinfo->prog_kind); diff --git a/tools/testing/selftests/bpf/testing_helpers.c b/tools/testing/selftests/bpf/testing_helpers.c index d5379a0e6da8..ac7c66f4fc7b 100644 --- a/tools/testing/selftests/bpf/testing_helpers.c +++ b/tools/testing/selftests/bpf/testing_helpers.c @@ -7,6 +7,7 @@ #include #include #include +#include "disasm.h" #include "test_progs.h" #include "testing_helpers.h" #include From patchwork Mon Jul 15 23:01:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733943 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-io1-f48.google.com (mail-io1-f48.google.com [209.85.166.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77B981442E8 for ; Mon, 15 Jul 2024 23:02:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084554; cv=none; b=rf+Eru7sRkYMu7umX/p2nctQrHUVWK3BdEEhYCHTORgEf3MjEA2YgFHkWmqaoOpaEaJyAUu5L6zJ9wjLmU6+2wOBB/LQCSwptjG6Q/dXSSZegWd1H0VEZ3FG0J/VMxy2hrd7+tAf6HrrG7RLU04jFxENmAJJJNFt0UOi3qd6V1A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084554; c=relaxed/simple; bh=ldMPcgcQGMpOFdiM8FOHsMKJSftJhX+5BRjhpcImCPI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nAjmY2cqogeYBO80fMyrQdr4y7bHaDiyxfa+i7jRsmAkjuKDnxsGJn3E8n/vQcBc92dErYNoaoienXp2/jwk9hzsjbREBtMUNaIaE73s5TmsDbT0DF5Ag0FfAqyHOeKaK+6xVCsrIpW61/UDl7Mu9O0nLqCGC7qpwwZZCufqR7Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YOm+gGxb; arc=none smtp.client-ip=209.85.166.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YOm+gGxb" Received: by mail-io1-f48.google.com with SMTP id ca18e2360f4ac-7f6e9662880so220675739f.2 for ; Mon, 15 Jul 2024 16:02:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084552; x=1721689352; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t+36kxACO33x+PbYNxrBVc9gb54uCltMFFp3q5I6q3Q=; b=YOm+gGxbWXTxzLm8uRWKqEzRnxfFFfyVGOGfbE0WXZx8p7nUn0YD7u6hnb/gFrA8Ya gEeX7WDJ80H8BO3DCrSWyvWy73CpXVeIhI1oWqZLxfxegqShDtTvtFChKmOYwdlvVues sKHBzWvYsk9P8iFqAvfsbMmyDBPhURINTrYcSyJ0vEfvWkOolCWutXWrH0PxmSgX+jbd osGG2T5zRB4uQfXBOYXUi5RDnNA8mKve2DTOem5H3OJ86PvbMIsE/9BsRbIzWLvonNjF mbMfpan4OIcuKpgRNqTtUlqCQb8SQDvX76ibUfzFdpPiRJ+xtNpmNsGjWBe9jCpLGc+A jlGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084552; x=1721689352; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t+36kxACO33x+PbYNxrBVc9gb54uCltMFFp3q5I6q3Q=; b=DD1yx8lxCwMLl6wHx0EM8z20ucy0Y4snFX88wYopj2Z51KJfHYVngoF1aZ3wQ8RGmn yaCT5t/bLMt9WmKY27z81ejThxJ9tkBF2sQmCwJr6gz5lHTJc8Uuk5Js0xgb31nW2CVY FWTURCgsSAzvwDcKqf2MT31DDRFhq2noxG7aEFJI1/Vc4vjr+RrTl4G9HVhp283zCviK 5TUWIV5QGeIipmMV0urvb+xtVibOEfp0J9p1Vny6lhEVCVz9PFoMWOEMrcA0Z4NusuQT gQaIZEfs/MG/GJsbXOgBF4346WajXJ1ou9nj8/iqhbxY+XE21q9VTBgV68TqxxZJvz96 +FKA== X-Gm-Message-State: AOJu0Yx471+9ehUvBWpTn/DcI+Fu6/m9r7zgyJ/DQkSjKkUkhD2OTGPc u4MwikOmTJRsu2mRfEZbA2B6l+EvEwKFeW7WoDrFpUZ0RWyUrFW8LbIZag== X-Google-Smtp-Source: AGHT+IHJCKCUF3y3mcnuK1ASfaSwaKroyMAG9O+zth3nJJnn3h61YwxJCSAkm+KNnNu7NphLDogiNg== X-Received: by 2002:a05:6602:1693:b0:80f:81f5:b46c with SMTP id ca18e2360f4ac-8157670971amr67652239f.15.1721084552111; Mon, 15 Jul 2024 16:02:32 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:31 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 05/12] selftests/bpf: print correct offset for pseudo calls in disasm_insn() Date: Mon, 15 Jul 2024 16:01:54 -0700 Message-ID: <20240715230201.3901423-6-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adjust disasm_helpers.c:disasm_insn() to account for the following part of the verifier.c:jit_subprogs: for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) { /* ... */ if (!bpf_pseudo_call(insn)) continue; insn->off = env->insn_aux_data[i].call_imm; subprog = find_subprog(env, i + insn->off + 1); insn->imm = subprog; } Where verifier moves offset of the subprogram to the insn->off field. Signed-off-by: Eduard Zingerman --- tools/testing/selftests/bpf/disasm_helpers.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tools/testing/selftests/bpf/disasm_helpers.c b/tools/testing/selftests/bpf/disasm_helpers.c index 96b1f2ffe438..f529f1c8c171 100644 --- a/tools/testing/selftests/bpf/disasm_helpers.c +++ b/tools/testing/selftests/bpf/disasm_helpers.c @@ -4,6 +4,7 @@ #include "disasm.h" struct print_insn_context { + char scratch[16]; char *buf; size_t sz; }; @@ -18,6 +19,22 @@ static void print_insn_cb(void *private_data, const char *fmt, ...) va_end(args); } +static const char *print_call_cb(void *private_data, const struct bpf_insn *insn) +{ + struct print_insn_context *ctx = private_data; + + /* For pseudo calls verifier.c:jit_subprogs() hides original + * imm to insn->off and changes insn->imm to be an index of + * the subprog instead. + */ + if (insn->src_reg == BPF_PSEUDO_CALL) { + snprintf(ctx->scratch, sizeof(ctx->scratch), "%+d", insn->off); + return ctx->scratch; + } + + return NULL; +} + struct bpf_insn *disasm_insn(struct bpf_insn *insn, char *buf, size_t buf_sz) { struct print_insn_context ctx = { @@ -26,6 +43,7 @@ struct bpf_insn *disasm_insn(struct bpf_insn *insn, char *buf, size_t buf_sz) }; struct bpf_insn_cbs cbs = { .cb_print = print_insn_cb, + .cb_call = print_call_cb, .private_data = &ctx, }; char *tmp, *pfx_end, *sfx_start; From patchwork Mon Jul 15 23:01:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733944 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31B23144306 for ; Mon, 15 Jul 2024 23:02:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084555; cv=none; b=YKTdrWGPkhNXKsst4PSqehcBm0TP4I4nFkGW9rZSs46OSzRkyKpP+k7AqpVd3PXJLUIGhCmoa2SH1Jo+5xg4/vdwLoX50mKEXDYUM+Q4aKg1xMa74L3ZqDZfTYbq+vFcTgmPnPVxm6Dv5qqTCpnb0G49/x5eRRa4JuouzWmkAGQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084555; c=relaxed/simple; bh=Ls5NTw26GGRfzQzeToAl7wGRmkGtN892goc869jj6MQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XO0MAPC8AcQjGg+X/pt9PV+kucdoVDzL1pDLzk1AO3RTAyD0xzpnuue1p9kq+Y7CE18HN4lgWNjQ/3xEnKvKceGHyF3+M7Ls6YzEuPsGNbcL8mbWvwVZnhO8rF0UHEaT+GdO8LrYEn3HGXvh8kx6+V3i5Cc5dPa/xPZCQMARS6w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hlJdF6Ww; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hlJdF6Ww" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-70af22a9c19so3646585b3a.2 for ; Mon, 15 Jul 2024 16:02:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084553; x=1721689353; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=H5ICedsF6VbzId1MCj49chfpgGgDYXegk6kOCiPafJU=; b=hlJdF6Ww0J1mYgGyMYSYdT/c3bQ48aEtTXc+Sujn7dLL1kMUl5VpGctrZ97JQYw1Dj 88YdMfyU0gSZg+4qrpo9ub9y9rtCFaOevz/AsNDXs3x3gh1CSaTLMVMbhpCzt/+ewDup 6VHVx8rvOXp3LCrbwgQAgBcycwRYbPu+Ybt4xZDIaayF4rAoXLPRi9RGKiiHqcNBSjSr 62XYah14PyZKQ+INMCfe9PsCY4ZwQq6Z+nDi0z/EhMIjoUPUn+M+w8QatxLPw2AqeYUi lZ+xg0t6XDZyt9/VFdScgspY4iNXm/9vPeS8mx1S+pnftzWlUgIT8IOOpGG1RIMiey/W 1LwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084553; x=1721689353; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H5ICedsF6VbzId1MCj49chfpgGgDYXegk6kOCiPafJU=; b=ko+ovtiiyvCEtQeun5+qvgEfuQ9/bFP/cGkxfgcqpbJulhF4EMP/DFyidoYJQpIKZr oLj0ZOjI4xkxi5+9+Lv/DqCX8+5BJBoqc3Yok3Kr85Llz/wFXJ3J3cpAIXvuM8hnwRr3 BvBzdV94cfND9FVpt+HKMR0mmMtxlfuaO96IpopsI65w84OSMD6AT+Y2SFJAwybsrQRT +Wd3/xItBUGR+ERGWF2QFC5/2HbhvqAhkJm8CwlVLHQqGfg9VEbXD6C+h7jgFAyUQjZX mllnW6iHZk05whJ8xXC9ILYeDOF2rgXLYTXc9a5skbgQmb2CK1x1OISWUc9BMHFof+hd 6rSA== X-Gm-Message-State: AOJu0YyFyZngQ5wjodAigZ2AaMipXWNp5zOLEhxWuYDy8HWDcieDE8r6 gzd/xuQxhp+xaaYsoOMUdK5VjXbf5GZwv9kN7ZfP7toVrLM+QZZU0eFe/w== X-Google-Smtp-Source: AGHT+IHAd4gM0kyr6sja+HOI766JiOb4G/G2IyRlk4tCFxI0turuHrER9CaBMEdEwFrNfhsBirtY6Q== X-Received: by 2002:a05:6a00:2304:b0:706:251d:d98 with SMTP id d2e1a72fcca58-70c1fb673fbmr608586b3a.4.1721084553383; Mon, 15 Jul 2024 16:02:33 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:32 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 06/12] selftests/bpf: no need to track next_match_pos in struct test_loader Date: Mon, 15 Jul 2024 16:01:55 -0700 Message-ID: <20240715230201.3901423-7-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The call stack for validate_case() function looks as follows: - test_loader__run_subtests() - process_subtest() - run_subtest() - prepare_case(), which does 'tester->next_match_pos = 0'; - validate_case(), which increments tester->next_match_pos. Hence, each subtest is run with next_match_pos freshly set to zero. Meaning that there is no need to persist this variable in the struct test_loader, use local variable instead. Acked-by: Andrii Nakryiko Signed-off-by: Eduard Zingerman --- tools/testing/selftests/bpf/test_loader.c | 19 ++++++++----------- tools/testing/selftests/bpf/test_progs.h | 1 - 2 files changed, 8 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c index f14e10b0de96..47508cf66e89 100644 --- a/tools/testing/selftests/bpf/test_loader.c +++ b/tools/testing/selftests/bpf/test_loader.c @@ -434,7 +434,6 @@ static void prepare_case(struct test_loader *tester, bpf_program__set_flags(prog, prog_flags | spec->prog_flags); tester->log_buf[0] = '\0'; - tester->next_match_pos = 0; } static void emit_verifier_log(const char *log_buf, bool force) @@ -450,25 +449,23 @@ static void validate_case(struct test_loader *tester, struct bpf_program *prog, int load_err) { - int i, j, err; - char *match; regmatch_t reg_match[1]; + const char *log = tester->log_buf; + int i, j, err; for (i = 0; i < subspec->expect_msg_cnt; i++) { struct expect_msg *msg = &subspec->expect_msgs[i]; + const char *match = NULL; if (msg->substr) { - match = strstr(tester->log_buf + tester->next_match_pos, msg->substr); + match = strstr(log, msg->substr); if (match) - tester->next_match_pos = match - tester->log_buf + strlen(msg->substr); + log += strlen(msg->substr); } else { - err = regexec(&msg->regex, - tester->log_buf + tester->next_match_pos, 1, reg_match, 0); + err = regexec(&msg->regex, log, 1, reg_match, 0); if (err == 0) { - match = tester->log_buf + tester->next_match_pos + reg_match[0].rm_so; - tester->next_match_pos += reg_match[0].rm_eo; - } else { - match = NULL; + match = log + reg_match[0].rm_so; + log += reg_match[0].rm_eo; } } diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h index 51341d50213b..b1e949fb16cf 100644 --- a/tools/testing/selftests/bpf/test_progs.h +++ b/tools/testing/selftests/bpf/test_progs.h @@ -447,7 +447,6 @@ typedef int (*pre_execution_cb)(struct bpf_object *obj); struct test_loader { char *log_buf; size_t log_buf_sz; - size_t next_match_pos; pre_execution_cb pre_execution_cb; struct bpf_object *obj; From patchwork Mon Jul 15 23:01:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733945 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8818C13FD66 for ; Mon, 15 Jul 2024 23:02:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084557; cv=none; b=Hbjg7dEWtsckIGIXtZzzP5C8xeWjoM2jxGiN6B29VRq5OscDWKTFPus94U22DOVuLsTxRgUVnY6gV0sqlSIGv7WpGTGTuwBGENqv1Q1XLxooHP1zQ7/QxP+Ry4GVnTSqpe2UzstY2JDRZ9JzvFaWgqXbuAm1UV2wsKwCrQhQejU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084557; c=relaxed/simple; bh=zHssEFSew6/7/I5d3v96ywaAl8Z3NeUArp5+VNQy/C0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lOHrsT6B91v2xyjOG/beqcvvEYvaFC7pctgSxrVK9KzyzT0YMl7u7hH18mB9ebggjB8M1cXYTtWVzJNXTZ4VyS/dgg4OFVFNeRbWLBNKKBIdxAxyW1rvCFvy/LjnSB6ltZWPgeT3vXxXqFHvflRKoNjAMZ/1eGNdw7qKemClakA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QYTjmnJ0; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QYTjmnJ0" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-70afe18837cso3020979b3a.3 for ; Mon, 15 Jul 2024 16:02:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084554; x=1721689354; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Skpti20qxoE6XuXbBaxdLW/TvxJxIRzVhmXOURRx+sc=; b=QYTjmnJ0NoH4F0Xzs2rlKEeTkH28uKXQO3jLQi8pECQ4zotwPE57mdEhEu99PltH34 s0LHteGtf3Q6G9J6SyuYnpBCsnCzhzNzb6wO7yhytV3XUMK/k62AketqNNaD9OekAs7K S2Nj3E8PCSKHQAAJfEF4lxOli5ii+KIuTJsmIf7raecWrXZxUlcQZ5Lr33N9qSOcGMae gQ3SM2Wj8B1Z5+2r+F2BESbf/vTZ9OngdHPBDDhMZoHIW9ANtymeCHhzy+EA+3FQg4vz Nb7e1pZsDRYbWGv7H18YSLpPA0n0jY4QGNv4d4ZCUdgdchAMWS2+Tc9IsZuMqczm8sHe NNmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084554; x=1721689354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Skpti20qxoE6XuXbBaxdLW/TvxJxIRzVhmXOURRx+sc=; b=tgaNJHyQe94Sk7SD44zmF5lNx2PNzHVvl4DPhnOKlWKUUqWFe3WlyJCV2sHmrjQ57W gpwNesn+oJDf/5S/ZQY2dKorvrng4OxAGouly7F4DNQVtbdGxRjVsm7HpBiZUVmoVpoL weBTACN5d4Srm6wvSfz3vzYR/7M6GQjJDllmO0AXvVVkBj9Yp6RLdX2VHgV2FbJtuSI+ uZ+ZNmehPMfJQRnHubnlnWIQVYFAEh7Rj9/RL21pEItPL+EDCxAHV2lV/GcVoTfdjC8g xRyJD2ysYwVUliSpmrSkaG5ffgTnSaBVn/rdue/uZdhRfrVoJzce7Gx6jDWaOyAb7oLc fM4g== X-Gm-Message-State: AOJu0YxdHOnW+isnrMMX1ZIiOTPIF908q8bhEoVCIylVKq1ZW1u6uJrY RuYkw6uAo9Jg9hn1RJiWKS3Gr5eW9Bi04UzuheGDvelD98GuaH+wANItgw== X-Google-Smtp-Source: AGHT+IGDJly7JQmW4Sume9xsYbBZyPdgaRNEsb2bNnzYj/cwNFv78GL6tiOs70TN31XMIWz8NGGDvQ== X-Received: by 2002:a05:6a21:e8a:b0:1c0:f648:8574 with SMTP id adf61e73a8af0-1c3f123d9e7mr428547637.29.1721084554363; Mon, 15 Jul 2024 16:02:34 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:33 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 07/12] selftests/bpf: extract test_loader->expect_msgs as a data structure Date: Mon, 15 Jul 2024 16:01:56 -0700 Message-ID: <20240715230201.3901423-8-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Non-functional change: use a separate data structure to represented expected messages in test_loader. This would allow to use the same functionality for expected set of disassembled instructions in the follow-up commit. Acked-by: Andrii Nakryiko Signed-off-by: Eduard Zingerman --- tools/testing/selftests/bpf/test_loader.c | 81 ++++++++++++----------- 1 file changed, 41 insertions(+), 40 deletions(-) diff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c index 47508cf66e89..3f84903558dd 100644 --- a/tools/testing/selftests/bpf/test_loader.c +++ b/tools/testing/selftests/bpf/test_loader.c @@ -55,11 +55,15 @@ struct expect_msg { regex_t regex; }; +struct expected_msgs { + struct expect_msg *patterns; + size_t cnt; +}; + struct test_subspec { char *name; bool expect_failure; - struct expect_msg *expect_msgs; - size_t expect_msg_cnt; + struct expected_msgs expect_msgs; int retval; bool execute; }; @@ -96,44 +100,45 @@ void test_loader_fini(struct test_loader *tester) free(tester->log_buf); } -static void free_test_spec(struct test_spec *spec) +static void free_msgs(struct expected_msgs *msgs) { int i; + for (i = 0; i < msgs->cnt; i++) + if (msgs->patterns[i].regex_str) + regfree(&msgs->patterns[i].regex); + free(msgs->patterns); + msgs->patterns = NULL; + msgs->cnt = 0; +} + +static void free_test_spec(struct test_spec *spec) +{ /* Deallocate expect_msgs arrays. */ - for (i = 0; i < spec->priv.expect_msg_cnt; i++) - if (spec->priv.expect_msgs[i].regex_str) - regfree(&spec->priv.expect_msgs[i].regex); - for (i = 0; i < spec->unpriv.expect_msg_cnt; i++) - if (spec->unpriv.expect_msgs[i].regex_str) - regfree(&spec->unpriv.expect_msgs[i].regex); + free_msgs(&spec->priv.expect_msgs); + free_msgs(&spec->unpriv.expect_msgs); free(spec->priv.name); free(spec->unpriv.name); - free(spec->priv.expect_msgs); - free(spec->unpriv.expect_msgs); - spec->priv.name = NULL; spec->unpriv.name = NULL; - spec->priv.expect_msgs = NULL; - spec->unpriv.expect_msgs = NULL; } -static int push_msg(const char *substr, const char *regex_str, struct test_subspec *subspec) +static int push_msg(const char *substr, const char *regex_str, struct expected_msgs *msgs) { void *tmp; int regcomp_res; char error_msg[100]; struct expect_msg *msg; - tmp = realloc(subspec->expect_msgs, - (1 + subspec->expect_msg_cnt) * sizeof(struct expect_msg)); + tmp = realloc(msgs->patterns, + (1 + msgs->cnt) * sizeof(struct expect_msg)); if (!tmp) { ASSERT_FAIL("failed to realloc memory for messages\n"); return -ENOMEM; } - subspec->expect_msgs = tmp; - msg = &subspec->expect_msgs[subspec->expect_msg_cnt]; + msgs->patterns = tmp; + msg = &msgs->patterns[msgs->cnt]; if (substr) { msg->substr = substr; @@ -150,7 +155,7 @@ static int push_msg(const char *substr, const char *regex_str, struct test_subsp } } - subspec->expect_msg_cnt += 1; + msgs->cnt += 1; return 0; } @@ -272,25 +277,25 @@ static int parse_test_spec(struct test_loader *tester, spec->mode_mask |= UNPRIV; } else if (str_has_pfx(s, TEST_TAG_EXPECT_MSG_PFX)) { msg = s + sizeof(TEST_TAG_EXPECT_MSG_PFX) - 1; - err = push_msg(msg, NULL, &spec->priv); + err = push_msg(msg, NULL, &spec->priv.expect_msgs); if (err) goto cleanup; spec->mode_mask |= PRIV; } else if (str_has_pfx(s, TEST_TAG_EXPECT_MSG_PFX_UNPRIV)) { msg = s + sizeof(TEST_TAG_EXPECT_MSG_PFX_UNPRIV) - 1; - err = push_msg(msg, NULL, &spec->unpriv); + err = push_msg(msg, NULL, &spec->unpriv.expect_msgs); if (err) goto cleanup; spec->mode_mask |= UNPRIV; } else if (str_has_pfx(s, TEST_TAG_EXPECT_REGEX_PFX)) { msg = s + sizeof(TEST_TAG_EXPECT_REGEX_PFX) - 1; - err = push_msg(NULL, msg, &spec->priv); + err = push_msg(NULL, msg, &spec->priv.expect_msgs); if (err) goto cleanup; spec->mode_mask |= PRIV; } else if (str_has_pfx(s, TEST_TAG_EXPECT_REGEX_PFX_UNPRIV)) { msg = s + sizeof(TEST_TAG_EXPECT_REGEX_PFX_UNPRIV) - 1; - err = push_msg(NULL, msg, &spec->unpriv); + err = push_msg(NULL, msg, &spec->unpriv.expect_msgs); if (err) goto cleanup; spec->mode_mask |= UNPRIV; @@ -387,11 +392,12 @@ static int parse_test_spec(struct test_loader *tester, spec->unpriv.execute = spec->priv.execute; } - if (!spec->unpriv.expect_msgs) { - for (i = 0; i < spec->priv.expect_msg_cnt; i++) { - struct expect_msg *msg = &spec->priv.expect_msgs[i]; + if (spec->unpriv.expect_msgs.cnt == 0) { + for (i = 0; i < spec->priv.expect_msgs.cnt; i++) { + struct expect_msg *msg = &spec->priv.expect_msgs.patterns[i]; - err = push_msg(msg->substr, msg->regex_str, &spec->unpriv); + err = push_msg(msg->substr, msg->regex_str, + &spec->unpriv.expect_msgs); if (err) goto cleanup; } @@ -443,18 +449,14 @@ static void emit_verifier_log(const char *log_buf, bool force) fprintf(stdout, "VERIFIER LOG:\n=============\n%s=============\n", log_buf); } -static void validate_case(struct test_loader *tester, - struct test_subspec *subspec, - struct bpf_object *obj, - struct bpf_program *prog, - int load_err) +static void validate_msgs(char *log_buf, struct expected_msgs *msgs) { regmatch_t reg_match[1]; - const char *log = tester->log_buf; + const char *log = log_buf; int i, j, err; - for (i = 0; i < subspec->expect_msg_cnt; i++) { - struct expect_msg *msg = &subspec->expect_msgs[i]; + for (i = 0; i < msgs->cnt; i++) { + struct expect_msg *msg = &msgs->patterns[i]; const char *match = NULL; if (msg->substr) { @@ -471,9 +473,9 @@ static void validate_case(struct test_loader *tester, if (!ASSERT_OK_PTR(match, "expect_msg")) { if (env.verbosity == VERBOSE_NONE) - emit_verifier_log(tester->log_buf, true /*force*/); + emit_verifier_log(log_buf, true /*force*/); for (j = 0; j <= i; j++) { - msg = &subspec->expect_msgs[j]; + msg = &msgs->patterns[j]; fprintf(stderr, "%s %s: '%s'\n", j < i ? "MATCHED " : "EXPECTED", msg->substr ? "SUBSTR" : " REGEX", @@ -692,9 +694,8 @@ void run_subtest(struct test_loader *tester, goto tobj_cleanup; } } - emit_verifier_log(tester->log_buf, false /*force*/); - validate_case(tester, subspec, tobj, tprog, err); + validate_msgs(tester->log_buf, &subspec->expect_msgs); if (should_do_test_run(spec, subspec)) { /* For some reason test_verifier executes programs From patchwork Mon Jul 15 23:01:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733947 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9BD813FD66 for ; Mon, 15 Jul 2024 23:02:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084559; cv=none; b=as8fb2lIt8PlGyztj+EB5Mb4Px8DNBFbwSLs2eAoOYo9eizNlLAkk5WA6/TDqsov/1PxoON3BsKJC8YsN1jJgOKGJ8AWQBjMQ0EuJPbv1SuRDjAwK7WKm23miLtAfNRae3kNgCkapiYUlNXRlzrB1xdjdavM+7V9fFJIdkZSDAI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084559; c=relaxed/simple; bh=NBnUPa5kXeybE0bH0innWyI3Q4aBa3Uz3jPNvmzZ6f0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H/k07MPvKxLs2GOK8Zb8H6I/UyOLwERVLFSPvgGyYbuOwgVYxcEg11/Ml/nWlQxEHCQk7WJGvJClZ/EvLn0sYkAPpcw8CcY+qb2Uh4R3loT4i3IFUg/FPgN4ljA26V1E4e1d4Yg/1EgF2eoL8SE/QMFwBn9AwLPqfjltLm06oOY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GgcNwAbA; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GgcNwAbA" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-70b0e7f6f8bso4224717b3a.3 for ; Mon, 15 Jul 2024 16:02:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084557; x=1721689357; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dhxKVSVlOw2OBGqZZRk30w4flqSDWzJlPRFfdg9zOrs=; b=GgcNwAbAXqgBsaz9iQmb9OT7PsFJxt+pOC4DS5j6uQnho8ZUSmnfuG0AtPdRNtTOmj VAmTeCVWOvEPU2Fb7RqTmjqXlls1r21fbDFDyPuU+PwqId/SqrukxMZ6PWns2SlS5AqO rYkW7obDblDcQBN89ZzFK+E0e7K1JPjpI4JEz6XkhuMGJ8m2XIZxWB7AV7pjlHsnLcOU Y1FY+jF3xbT/tr4dBi0j2A5L2TM2Dvbr/De1Cf6QN5LU1NB6ViT+rQ0oju3RFiLmrg2V M1teeVNS3XX6THyaIuMM8jpZgq8Va23ITiuBjFztdYC2W+BEU6iJj6KELMqHy6KMBZW0 y8cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084557; x=1721689357; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dhxKVSVlOw2OBGqZZRk30w4flqSDWzJlPRFfdg9zOrs=; b=BVaGJgqNu2cThiYIzbZ5gLxP2R8FmWgp18EqC/0W+hTt2+RxcsSXZfM5R9WPfeaiHF +C/RK19gxFQUOpSE85tGbSdgJW1XF/yaylmeV/uWkFEHpacPh3waIjueP7GH+DoyLz1K 7QdDWB/h+1kxON7Ex87MbBNB5D7hwyE3UKZP3D23sS6fzujLT4QOrD/RtG4qIFpQzt3Z +KBl7QRvI58sG3MRrCP0AQYGoVdePZuD05LhQ5tyw1ualg92iAWqBi4Bf9bzHZyZMowe 5AV9W9urup0HQrWrisYj4xtZP/4jjEIO4Tue60CE71F6hXoKUh9w38au8dTGn+xYieJV 2JFw== X-Gm-Message-State: AOJu0YxpbDT70h+K0yaSqP+FTzFL9HXaGROZGlLvLjZO0iPrg5WO41Nx vV8ae6Fu6ntwiL5c9lZLjjSSFKco6FgHHXkwV2MsR4fAPnBpAwfjp4Wceg== X-Google-Smtp-Source: AGHT+IE/Bg5Eep+q4jkxG7yitzrE/1RWm8GqyfcFlA2JRr3mqGAjyIedxRgY1qRrP83wXShz+V23Tg== X-Received: by 2002:a05:6a00:140d:b0:70b:5b21:6c2 with SMTP id d2e1a72fcca58-70c2ea11d0bmr602141b3a.31.1721084555410; Mon, 15 Jul 2024 16:02:35 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:34 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 08/12] selftests/bpf: allow checking xlated programs in verifier_* tests Date: Mon, 15 Jul 2024 16:01:57 -0700 Message-ID: <20240715230201.3901423-9-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add a macro __xlated("...") for use with test_loader tests. When such annotations are present for the test case: - bpf_prog_get_info_by_fd() is used to get BPF program after all rewrites are applied by verifier. - the program is disassembled and patterns specified in __xlated are searched for in the disassembly text. __xlated matching follows the same mechanics as __msg: each subsequent pattern is matched from the point where previous pattern ended. This allows to write tests like below, where the goal is to verify the behavior of one of the of the transformations applied by verifier: SEC("raw_tp") __xlated("1: w0 = ") __xlated("2: r0 = &(void __percpu *)(r0)") __xlated("3: r0 = *(u32 *)(r0 +0)") __xlated("4: exit") __success __naked void simple(void) { asm volatile ( "call %[bpf_get_smp_processor_id];" "exit;" : : __imm(bpf_get_smp_processor_id) : __clobber_all); } Acked-by: Andrii Nakryiko Signed-off-by: Eduard Zingerman --- tools/testing/selftests/bpf/progs/bpf_misc.h | 5 ++ tools/testing/selftests/bpf/test_loader.c | 82 +++++++++++++++++++- 2 files changed, 84 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/bpf/progs/bpf_misc.h b/tools/testing/selftests/bpf/progs/bpf_misc.h index 81097a3f15eb..a70939c7bc26 100644 --- a/tools/testing/selftests/bpf/progs/bpf_misc.h +++ b/tools/testing/selftests/bpf/progs/bpf_misc.h @@ -26,6 +26,9 @@ * * __regex Same as __msg, but using a regular expression. * __regex_unpriv Same as __msg_unpriv but using a regular expression. + * __xlated Expect a line in a disassembly log after verifier applies rewrites. + * Multiple __xlated attributes could be specified. + * __xlated_unpriv Same as __xlated but for unprivileged mode. * * __success Expect program load success in privileged mode. * __success_unpriv Expect program load success in unprivileged mode. @@ -63,11 +66,13 @@ */ #define __msg(msg) __attribute__((btf_decl_tag("comment:test_expect_msg=" msg))) #define __regex(regex) __attribute__((btf_decl_tag("comment:test_expect_regex=" regex))) +#define __xlated(msg) __attribute__((btf_decl_tag("comment:test_expect_xlated=" msg))) #define __failure __attribute__((btf_decl_tag("comment:test_expect_failure"))) #define __success __attribute__((btf_decl_tag("comment:test_expect_success"))) #define __description(desc) __attribute__((btf_decl_tag("comment:test_description=" desc))) #define __msg_unpriv(msg) __attribute__((btf_decl_tag("comment:test_expect_msg_unpriv=" msg))) #define __regex_unpriv(regex) __attribute__((btf_decl_tag("comment:test_expect_regex_unpriv=" regex))) +#define __xlated_unpriv(msg) __attribute__((btf_decl_tag("comment:test_expect_xlated_unpriv=" msg))) #define __failure_unpriv __attribute__((btf_decl_tag("comment:test_expect_failure_unpriv"))) #define __success_unpriv __attribute__((btf_decl_tag("comment:test_expect_success_unpriv"))) #define __log_level(lvl) __attribute__((btf_decl_tag("comment:test_log_level="#lvl))) diff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c index 3f84903558dd..b44b6a2fc82c 100644 --- a/tools/testing/selftests/bpf/test_loader.c +++ b/tools/testing/selftests/bpf/test_loader.c @@ -7,6 +7,7 @@ #include #include "autoconf_helper.h" +#include "disasm_helpers.h" #include "unpriv_helpers.h" #include "cap_helpers.h" @@ -19,10 +20,12 @@ #define TEST_TAG_EXPECT_SUCCESS "comment:test_expect_success" #define TEST_TAG_EXPECT_MSG_PFX "comment:test_expect_msg=" #define TEST_TAG_EXPECT_REGEX_PFX "comment:test_expect_regex=" +#define TEST_TAG_EXPECT_XLATED_PFX "comment:test_expect_xlated=" #define TEST_TAG_EXPECT_FAILURE_UNPRIV "comment:test_expect_failure_unpriv" #define TEST_TAG_EXPECT_SUCCESS_UNPRIV "comment:test_expect_success_unpriv" #define TEST_TAG_EXPECT_MSG_PFX_UNPRIV "comment:test_expect_msg_unpriv=" #define TEST_TAG_EXPECT_REGEX_PFX_UNPRIV "comment:test_expect_regex_unpriv=" +#define TEST_TAG_EXPECT_XLATED_PFX_UNPRIV "comment:test_expect_xlated_unpriv=" #define TEST_TAG_LOG_LEVEL_PFX "comment:test_log_level=" #define TEST_TAG_PROG_FLAGS_PFX "comment:test_prog_flags=" #define TEST_TAG_DESCRIPTION_PFX "comment:test_description=" @@ -64,6 +67,7 @@ struct test_subspec { char *name; bool expect_failure; struct expected_msgs expect_msgs; + struct expected_msgs expect_xlated; int retval; bool execute; }; @@ -117,6 +121,8 @@ static void free_test_spec(struct test_spec *spec) /* Deallocate expect_msgs arrays. */ free_msgs(&spec->priv.expect_msgs); free_msgs(&spec->unpriv.expect_msgs); + free_msgs(&spec->priv.expect_xlated); + free_msgs(&spec->unpriv.expect_xlated); free(spec->priv.name); free(spec->unpriv.name); @@ -299,6 +305,18 @@ static int parse_test_spec(struct test_loader *tester, if (err) goto cleanup; spec->mode_mask |= UNPRIV; + } else if (str_has_pfx(s, TEST_TAG_EXPECT_XLATED_PFX)) { + msg = s + sizeof(TEST_TAG_EXPECT_XLATED_PFX) - 1; + err = push_msg(msg, NULL, &spec->priv.expect_xlated); + if (err) + goto cleanup; + spec->mode_mask |= PRIV; + } else if (str_has_pfx(s, TEST_TAG_EXPECT_XLATED_PFX_UNPRIV)) { + msg = s + sizeof(TEST_TAG_EXPECT_XLATED_PFX_UNPRIV) - 1; + err = push_msg(msg, NULL, &spec->unpriv.expect_xlated); + if (err) + goto cleanup; + spec->mode_mask |= UNPRIV; } else if (str_has_pfx(s, TEST_TAG_RETVAL_PFX)) { val = s + sizeof(TEST_TAG_RETVAL_PFX) - 1; err = parse_retval(val, &spec->priv.retval, "__retval"); @@ -402,6 +420,16 @@ static int parse_test_spec(struct test_loader *tester, goto cleanup; } } + if (spec->unpriv.expect_xlated.cnt == 0) { + for (i = 0; i < spec->priv.expect_xlated.cnt; i++) { + struct expect_msg *msg = &spec->priv.expect_xlated.patterns[i]; + + err = push_msg(msg->substr, msg->regex_str, + &spec->unpriv.expect_xlated); + if (err) + goto cleanup; + } + } } spec->valid = true; @@ -449,7 +477,15 @@ static void emit_verifier_log(const char *log_buf, bool force) fprintf(stdout, "VERIFIER LOG:\n=============\n%s=============\n", log_buf); } -static void validate_msgs(char *log_buf, struct expected_msgs *msgs) +static void emit_xlated(const char *xlated, bool force) +{ + if (!force && env.verbosity == VERBOSE_NONE) + return; + fprintf(stdout, "XLATED:\n=============\n%s=============\n", xlated); +} + +static void validate_msgs(char *log_buf, struct expected_msgs *msgs, + void (*emit_fn)(const char *buf, bool force)) { regmatch_t reg_match[1]; const char *log = log_buf; @@ -473,7 +509,7 @@ static void validate_msgs(char *log_buf, struct expected_msgs *msgs) if (!ASSERT_OK_PTR(match, "expect_msg")) { if (env.verbosity == VERBOSE_NONE) - emit_verifier_log(log_buf, true /*force*/); + emit_fn(log_buf, true /*force*/); for (j = 0; j <= i; j++) { msg = &msgs->patterns[j]; fprintf(stderr, "%s %s: '%s'\n", @@ -610,6 +646,37 @@ static bool should_do_test_run(struct test_spec *spec, struct test_subspec *subs return true; } +/* Get a disassembly of BPF program after verifier applies all rewrites */ +static int get_xlated_program_text(int prog_fd, char *text, size_t text_sz) +{ + struct bpf_insn *insn_start = NULL, *insn, *insn_end; + __u32 insns_cnt = 0, i; + char buf[64]; + FILE *out = NULL; + int err; + + err = get_xlated_program(prog_fd, &insn_start, &insns_cnt); + if (!ASSERT_OK(err, "get_xlated_program")) + goto out; + out = fmemopen(text, text_sz, "w"); + if (!ASSERT_OK_PTR(out, "open_memstream")) + goto out; + insn_end = insn_start + insns_cnt; + insn = insn_start; + while (insn < insn_end) { + i = insn - insn_start; + insn = disasm_insn(insn, buf, sizeof(buf)); + fprintf(out, "%d: %s\n", i, buf); + } + fflush(out); + +out: + free(insn_start); + if (out) + fclose(out); + return err; +} + /* this function is forced noinline and has short generic name to look better * in test_progs output (in case of a failure) */ @@ -695,7 +762,16 @@ void run_subtest(struct test_loader *tester, } } emit_verifier_log(tester->log_buf, false /*force*/); - validate_msgs(tester->log_buf, &subspec->expect_msgs); + validate_msgs(tester->log_buf, &subspec->expect_msgs, emit_verifier_log); + + if (subspec->expect_xlated.cnt) { + err = get_xlated_program_text(bpf_program__fd(tprog), + tester->log_buf, tester->log_buf_sz); + if (err) + goto tobj_cleanup; + emit_xlated(tester->log_buf, false /*force*/); + validate_msgs(tester->log_buf, &subspec->expect_xlated, emit_xlated); + } if (should_do_test_run(spec, subspec)) { /* For some reason test_verifier executes programs From patchwork Mon Jul 15 23:01:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733946 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-oa1-f46.google.com (mail-oa1-f46.google.com [209.85.160.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0F3E1448C7 for ; Mon, 15 Jul 2024 23:02:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084559; cv=none; b=IdqkIg7G+DbVvkISQdlRiOvHV9Uyex37Rhd5w0HcTHm7eAv8cpRc7mIyQUVp8JV0yfBuruqGJeBDxjXewYflgXhSjicgSqjCixUnZE4gyrFqJ0KbknJT6EyP1y0c8QiDnz/PJyKABWr76qwhfzOiVpigcUtkhmT2p+ft1BHGZ04= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084559; c=relaxed/simple; bh=nSRQgVwzqE9EibMQ+6b2kM1It2Gm6S06QoRB9b5WnkU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Bme/A28dnRZIISHG0ChbxGQR/p9ozft70lEiI404k+2m67700SaSMDAtJ62NoAl7zY6lWPVggXDZkTOqMPVsQRWO5qJKmh/0x8wR6Ja3azU0I6a9u3ZUtJWl9MyMb1bN5cSsmlbOUokoT7YmSLq2/C1NfPfPWaMVi+XcPnjRZMA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ECm6nONU; arc=none smtp.client-ip=209.85.160.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ECm6nONU" Received: by mail-oa1-f46.google.com with SMTP id 586e51a60fabf-250ca14422aso2725303fac.0 for ; Mon, 15 Jul 2024 16:02:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084556; x=1721689356; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zEEKOl8JhrhEjS/sgijGjfbzU9z19n/Dr0hU6V4VsDA=; b=ECm6nONUg60EuQ3cvTYqcMH/I1sl7Zr0ck+592X9KFxjbspzsvex0e+9o/QHBTfkFh vFitO42NukhrmXoRJpVUoJo8gUZCz4Cxms2h4wRjnMC3krTjcuwNJ/xvynhbDXEEfqHX pcm/H1XXkcNpwUN2SEtFuz95JJ6Lw7Y1mlKpThW/FzF0trG5lR2GXoCjIpUD+4gc5JTU KUofy34s5ZPLhE0DZEUUD+/maFFm4ykc7Fznzt//mga9FIZACfIYMC7E9MIdODolEK+M 2JUmObltHpXfuSL5x8DzxEbsM+7W4lWeSBdj9o94UaxhnKYJL9e59JAVBIerpDtZvLvN zzWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084556; x=1721689356; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zEEKOl8JhrhEjS/sgijGjfbzU9z19n/Dr0hU6V4VsDA=; b=AKieLSyu3ApdZeAf2eWigR2kNSai3A4ia+9BcIgfSFE328A4KvjmAOsqzM4lhOTJ+N pSKROjd4PWAfpj5Ha0OJ/d/U2kn+KWpcEtt+qLBPNWFuTT21QnW3eQg3EBnXuU5t6RMj rhoM3X5X19p/LLjmZnOTNkKpbpIZOmjNFf6v0qxj/w1bb+zQLqbQ53VU6Q7GQeR+EGu2 Ma7tZwREgHLxHhSiPguN4EH03wQ9pCpPOJxjh6qqklDxkZaAJLt0n6q4B1+JVr8k3aZs bbFRGhGwTzWalBsdgs5xseHMMW5JFx7rbvdwBlVsaBvlVmCKSERtPkwd+fRo/S7OkVCt wuFQ== X-Gm-Message-State: AOJu0Yz6zBFPfEUTAkLrbrrg3+6QVEQkCs7n2QxZhMMqZE2zT+Ifn2jF 7eLtTgfUDx5d2WcolTXQhhstzuexLvdueD6pDqogCQ2zJqrLEZbq5xsZrA== X-Google-Smtp-Source: AGHT+IF0BpfRmdLoZlEPYSc6mDHHRhL7/Ozz0nAINmHk88sv++YJRe0uyXwFXi+/2VnQ8qJwK7n1xQ== X-Received: by 2002:a05:6870:d614:b0:254:963e:cc3f with SMTP id 586e51a60fabf-260bd51fd74mr262006fac.1.1721084556366; Mon, 15 Jul 2024 16:02:36 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:35 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 09/12] selftests/bpf: __arch_* macro to limit test cases to specific archs Date: Mon, 15 Jul 2024 16:01:58 -0700 Message-ID: <20240715230201.3901423-10-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add annotations __arch_x86_64, __arch_arm64, __arch_riscv64 to specify on which architecture the test case should be tested. Several __arch_* annotations could be specified at once. When test case is not run on current arch it is marked as skipped. For example, the following would be tested only on arm64 and riscv64: SEC("raw_tp") __arch_arm64 __arch_riscv64 __xlated("1: *(u64 *)(r10 - 16) = r1") __xlated("2: call") __xlated("3: r1 = *(u64 *)(r10 - 16);") __success __naked void canary_arm64_riscv64(void) { asm volatile ( "r1 = 1;" "*(u64 *)(r10 - 16) = r1;" "call %[bpf_get_smp_processor_id];" "r1 = *(u64 *)(r10 - 16);" "exit;" : : __imm(bpf_get_smp_processor_id) : __clobber_all); } On x86 it would be skipped: #467/2 verifier_nocsr/canary_arm64_riscv64:SKIP Acked-by: Andrii Nakryiko Signed-off-by: Eduard Zingerman --- tools/testing/selftests/bpf/progs/bpf_misc.h | 8 ++++ tools/testing/selftests/bpf/test_loader.c | 43 ++++++++++++++++++++ 2 files changed, 51 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/bpf_misc.h b/tools/testing/selftests/bpf/progs/bpf_misc.h index a70939c7bc26..a225cd87897c 100644 --- a/tools/testing/selftests/bpf/progs/bpf_misc.h +++ b/tools/testing/selftests/bpf/progs/bpf_misc.h @@ -63,6 +63,10 @@ * __auxiliary Annotated program is not a separate test, but used as auxiliary * for some other test cases and should always be loaded. * __auxiliary_unpriv Same, but load program in unprivileged mode. + * + * __arch_* Specify on which architecture the test case should be tested. + * Several __arch_* annotations could be specified at once. + * When test case is not run on current arch it is marked as skipped. */ #define __msg(msg) __attribute__((btf_decl_tag("comment:test_expect_msg=" msg))) #define __regex(regex) __attribute__((btf_decl_tag("comment:test_expect_regex=" regex))) @@ -82,6 +86,10 @@ #define __auxiliary __attribute__((btf_decl_tag("comment:test_auxiliary"))) #define __auxiliary_unpriv __attribute__((btf_decl_tag("comment:test_auxiliary_unpriv"))) #define __btf_path(path) __attribute__((btf_decl_tag("comment:test_btf_path=" path))) +#define __arch(arch) __attribute__((btf_decl_tag("comment:test_arch=" arch))) +#define __arch_x86_64 __arch("X86_64") +#define __arch_arm64 __arch("ARM64") +#define __arch_riscv64 __arch("RISCV64") /* Convenience macro for use with 'asm volatile' blocks */ #define __naked __attribute__((naked)) diff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c index b44b6a2fc82c..12b0c41e8d64 100644 --- a/tools/testing/selftests/bpf/test_loader.c +++ b/tools/testing/selftests/bpf/test_loader.c @@ -34,6 +34,7 @@ #define TEST_TAG_AUXILIARY "comment:test_auxiliary" #define TEST_TAG_AUXILIARY_UNPRIV "comment:test_auxiliary_unpriv" #define TEST_BTF_PATH "comment:test_btf_path=" +#define TEST_TAG_ARCH "comment:test_arch=" /* Warning: duplicated in bpf_misc.h */ #define POINTER_VALUE 0xcafe4all @@ -80,6 +81,7 @@ struct test_spec { int log_level; int prog_flags; int mode_mask; + int arch_mask; bool auxiliary; bool valid; }; @@ -213,6 +215,12 @@ static void update_flags(int *flags, int flag, bool clear) *flags |= flag; } +enum arch { + ARCH_X86_64 = 0x1, + ARCH_ARM64 = 0x2, + ARCH_RISCV64 = 0x4, +}; + /* Uses btf_decl_tag attributes to describe the expected test * behavior, see bpf_misc.h for detailed description of each attribute * and attribute combinations. @@ -226,6 +234,7 @@ static int parse_test_spec(struct test_loader *tester, bool has_unpriv_result = false; bool has_unpriv_retval = false; int func_id, i, err = 0; + u32 arch_mask = 0; struct btf *btf; memset(spec, 0, sizeof(*spec)); @@ -364,11 +373,26 @@ static int parse_test_spec(struct test_loader *tester, goto cleanup; update_flags(&spec->prog_flags, flags, clear); } + } else if (str_has_pfx(s, TEST_TAG_ARCH)) { + val = s + sizeof(TEST_TAG_ARCH) - 1; + if (strcmp(val, "X86_64") == 0) { + arch_mask |= ARCH_X86_64; + } else if (strcmp(val, "ARM64") == 0) { + arch_mask |= ARCH_ARM64; + } else if (strcmp(val, "RISCV64") == 0) { + arch_mask |= ARCH_RISCV64; + } else { + PRINT_FAIL("bad arch spec: '%s'", val); + err = -EINVAL; + goto cleanup; + } } else if (str_has_pfx(s, TEST_BTF_PATH)) { spec->btf_custom_path = s + sizeof(TEST_BTF_PATH) - 1; } } + spec->arch_mask = arch_mask; + if (spec->mode_mask == 0) spec->mode_mask = PRIV; @@ -677,6 +701,20 @@ static int get_xlated_program_text(int prog_fd, char *text, size_t text_sz) return err; } +static bool run_on_current_arch(int arch_mask) +{ + if (arch_mask == 0) + return true; +#if defined(__x86_64__) + return arch_mask & ARCH_X86_64; +#elif defined(__aarch64__) + return arch_mask & ARCH_ARM64; +#elif defined(__riscv) && __riscv_xlen == 64 + return arch_mask & ARCH_RISCV64; +#endif + return false; +} + /* this function is forced noinline and has short generic name to look better * in test_progs output (in case of a failure) */ @@ -701,6 +739,11 @@ void run_subtest(struct test_loader *tester, if (!test__start_subtest(subspec->name)) return; + if (!run_on_current_arch(spec->arch_mask)) { + test__skip(); + return; + } + if (unpriv) { if (!can_execute_unpriv(tester, spec)) { test__skip(); From patchwork Mon Jul 15 23:01:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733949 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F2DD1422D3 for ; Mon, 15 Jul 2024 23:02:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084561; cv=none; b=IjfhspGsDvODYA7a1Z+X024Lll+PFEkLgY+k2RPauTD8bZxX39rI82Omq9FVcdOIwyoB1wyUihqKS+u44zH9dccEBxRtUoJaZKv5dfuDcMxUw1ueVLcRzuzLLFc5lpE7tbbuc13B0z17S6o/NwRAWnA1rXbdOT+7MhLG77CZML8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084561; c=relaxed/simple; bh=0iWg+tYvHL2dCU0y1pJ4QIvpScCPjyGmlehpZuNGxrc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fjMOOntN/6uw1JQAY8ttVa+T0VMpljWWCYtZVvmUEiKBfd+kOs+5O7fE9RGUYCDQSiRbvfnd9kWydcKuHSAgdvC9y8cxaifK07Xa9by/eG/J/7Ne0FqPr1Fgnl/o3vw3AgYuvgOq525wEGt5X7wBzFbfZHS24xH1VQh0estSRRo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hXFFUQjj; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hXFFUQjj" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-70af5fbf0d5so3003524b3a.1 for ; Mon, 15 Jul 2024 16:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084558; x=1721689358; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cK9Fcv9Y9f1g5kz2MxIGQULTaGe4m0CKztDOyOlIP1A=; b=hXFFUQjjC+aQS/RDGlMP4cGVdmXxT+fARoOE22eRYFaWh224nvG0DuZOAvukkX3naq czc220iwdzCxERoKJtxIObzQSS58He8EzchunK5R0SR74Qo4gl0XFI+2UwjxqqMTGKCu Mn+DrIU/OJZhp+GXBDCwnhzxM/iCx6uuFGNIcJT7RvCxHOiD0zVrfJVWhHPPuq2yBbGB cwKeFay30yK0Y6ULQKl1El2NOs4tjF7vR8bN5AxV/jd9l/NLcvFAFD2z8w9OkeJJmbZS rUR1Kpbt/7aWkMuXQhCEXAiKsYb4JUnIEVG05Sk16I/huGDhQXNPNaVPkq8w2EbIg19W Cu0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084558; x=1721689358; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cK9Fcv9Y9f1g5kz2MxIGQULTaGe4m0CKztDOyOlIP1A=; b=gI7egL/Q3rdvxyNf+PgxKIyTi3xHbrpz+usSbiWUTWIKZ73x3dCP/KnAm/ZsCjXhIW Vw7QFGe7dsI15vzMERPiy1rP/71ROTceN/ouTl5slpgoKeTio5dHnPfYe/+b27r9QXzv fuM9nimbXedh/M5q48rqL3iTg3bK1K373jfr32F3QbfDziOfDss+mwMOlep9Z042GQ5g iXcmQUZmrEpE7KeJSqjYrOlE8FA5WjTFILnXNytR/pbq7OXDOXb8gLx+io52XhT55hC7 N91RK0PMkI9UT7lTcHfxDPTHptTMpmWTRBypxhyaZ3VSm+22KnHesetqXBTcmD5DNwer glXg== X-Gm-Message-State: AOJu0Yz286iA3PdJfWkYKGEeQCDgbIC7uIvCPwBLGfIodDKHTtrIWmxt JRrPqhaU8CGtF8qg64MRAm4nzJvSeMaOPEzZYPByGfQwYm9uAq12/jxCyQ== X-Google-Smtp-Source: AGHT+IGKORA9DzF4eiDROf0ZM7ic///CrbhHoeB6GSXjsU9lGiygWNdWUhEJEsNkpCnLK6HLWnhHqg== X-Received: by 2002:aa7:8887:0:b0:706:938a:5d49 with SMTP id d2e1a72fcca58-70cd8433c1bmr117671b3a.14.1721084557705; Mon, 15 Jul 2024 16:02:37 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:37 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 10/12] selftests/bpf: test no_caller_saved_registers spill/fill removal Date: Mon, 15 Jul 2024 16:01:59 -0700 Message-ID: <20240715230201.3901423-11-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Tests for no_caller_saved_registers processing logic (see verifier.c:match_and_mark_nocsr_pattern()): - a canary positive test case; - a canary test case for arm64 and riscv64; - various tests with broken patterns; - tests with read/write fixed/varying stack access that violate nocsr stack access contract; - tests with multiple subprograms; - tests using nocsr in combination with may_goto/bpf_loop, as all of these features affect stack depth. Signed-off-by: Eduard Zingerman --- .../selftests/bpf/prog_tests/verifier.c | 2 + .../selftests/bpf/progs/verifier_nocsr.c | 705 ++++++++++++++++++ 2 files changed, 707 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/verifier_nocsr.c diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c index 9dc3687bc406..a3c2c5da3e0e 100644 --- a/tools/testing/selftests/bpf/prog_tests/verifier.c +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c @@ -53,6 +53,7 @@ #include "verifier_movsx.skel.h" #include "verifier_netfilter_ctx.skel.h" #include "verifier_netfilter_retcode.skel.h" +#include "verifier_nocsr.skel.h" #include "verifier_or_jmp32_k.skel.h" #include "verifier_precision.skel.h" #include "verifier_prevent_map_lookup.skel.h" @@ -172,6 +173,7 @@ void test_verifier_meta_access(void) { RUN(verifier_meta_access); } void test_verifier_movsx(void) { RUN(verifier_movsx); } void test_verifier_netfilter_ctx(void) { RUN(verifier_netfilter_ctx); } void test_verifier_netfilter_retcode(void) { RUN(verifier_netfilter_retcode); } +void test_verifier_nocsr(void) { RUN(verifier_nocsr); } void test_verifier_or_jmp32_k(void) { RUN(verifier_or_jmp32_k); } void test_verifier_precision(void) { RUN(verifier_precision); } void test_verifier_prevent_map_lookup(void) { RUN(verifier_prevent_map_lookup); } diff --git a/tools/testing/selftests/bpf/progs/verifier_nocsr.c b/tools/testing/selftests/bpf/progs/verifier_nocsr.c new file mode 100644 index 000000000000..84f76f850e9a --- /dev/null +++ b/tools/testing/selftests/bpf/progs/verifier_nocsr.c @@ -0,0 +1,705 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include "../../../include/linux/filter.h" +#include "bpf_misc.h" + +SEC("raw_tp") +__arch_x86_64 +__log_level(4) __msg("stack depth 8") +__xlated("4: r5 = 5") +__xlated("5: w0 = ") +__xlated("6: r0 = &(void __percpu *)(r0)") +__xlated("7: r0 = *(u32 *)(r0 +0)") +__xlated("8: exit") +__success +__naked void simple(void) +{ + asm volatile ( + "r1 = 1;" + "r2 = 2;" + "r3 = 3;" + "r4 = 4;" + "r5 = 5;" + "*(u64 *)(r10 - 16) = r1;" + "*(u64 *)(r10 - 24) = r2;" + "*(u64 *)(r10 - 32) = r3;" + "*(u64 *)(r10 - 40) = r4;" + "*(u64 *)(r10 - 48) = r5;" + "call %[bpf_get_smp_processor_id];" + "r5 = *(u64 *)(r10 - 48);" + "r4 = *(u64 *)(r10 - 40);" + "r3 = *(u64 *)(r10 - 32);" + "r2 = *(u64 *)(r10 - 24);" + "r1 = *(u64 *)(r10 - 16);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +/* The logic for detecting and verifying nocsr pattern is the same for + * any arch, however x86 differs from arm64 or riscv64 in a way + * bpf_get_smp_processor_id is rewritten: + * - on x86 it is done by verifier + * - on arm64 and riscv64 it is done by jit + * + * Which leads to different xlated patterns for different archs: + * - on x86 the call is expanded as 3 instructions + * - on arm64 and riscv64 the call remains as is + * (but spills/fills are still removed) + * + * It is really desirable to check instruction indexes in the xlated + * patterns, so add this canary test to check that function rewrite by + * jit is correctly processed by nocsr logic, keep the rest of the + * tests as x86. + */ +SEC("raw_tp") +__arch_arm64 +__arch_riscv64 +__xlated("0: r1 = 1") +__xlated("1: call bpf_get_smp_processor_id") +__xlated("2: exit") +__success +__naked void canary_arm64_riscv64(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: r0 = &(void __percpu *)(r0)") +__xlated("3: exit") +__success +__naked void canary_zero_spills(void) +{ + asm volatile ( + "call %[bpf_get_smp_processor_id];" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__log_level(4) __msg("stack depth 16") +__xlated("1: *(u64 *)(r10 -16) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r2 = *(u64 *)(r10 -16)") +__success +__naked void wrong_reg_in_pattern1(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r2 = *(u64 *)(r10 - 16);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u64 *)(r10 -16) = r6") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r6 = *(u64 *)(r10 -16)") +__success +__naked void wrong_reg_in_pattern2(void) +{ + asm volatile ( + "r6 = 1;" + "*(u64 *)(r10 - 16) = r6;" + "call %[bpf_get_smp_processor_id];" + "r6 = *(u64 *)(r10 - 16);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u64 *)(r10 -16) = r0") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r0 = *(u64 *)(r10 -16)") +__success +__naked void wrong_reg_in_pattern3(void) +{ + asm volatile ( + "r0 = 1;" + "*(u64 *)(r10 - 16) = r0;" + "call %[bpf_get_smp_processor_id];" + "r0 = *(u64 *)(r10 - 16);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("2: *(u64 *)(r2 -16) = r1") +__xlated("4: r0 = &(void __percpu *)(r0)") +__xlated("6: r1 = *(u64 *)(r10 -16)") +__success +__naked void wrong_base_in_pattern(void) +{ + asm volatile ( + "r1 = 1;" + "r2 = r10;" + "*(u64 *)(r2 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u64 *)(r10 -16) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r2 = 1") +__success +__naked void wrong_insn_in_pattern(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r2 = 1;" + "r1 = *(u64 *)(r10 - 16);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("2: *(u64 *)(r10 -16) = r1") +__xlated("4: r0 = &(void __percpu *)(r0)") +__xlated("6: r1 = *(u64 *)(r10 -8)") +__success +__naked void wrong_off_in_pattern1(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 8) = r1;" + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 8);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u32 *)(r10 -4) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r1 = *(u32 *)(r10 -4)") +__success +__naked void wrong_off_in_pattern2(void) +{ + asm volatile ( + "r1 = 1;" + "*(u32 *)(r10 - 4) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u32 *)(r10 - 4);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u32 *)(r10 -16) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r1 = *(u32 *)(r10 -16)") +__success +__naked void wrong_size_in_pattern(void) +{ + asm volatile ( + "r1 = 1;" + "*(u32 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u32 *)(r10 - 16);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("2: *(u32 *)(r10 -8) = r1") +__xlated("4: r0 = &(void __percpu *)(r0)") +__xlated("6: r1 = *(u32 *)(r10 -8)") +__success +__naked void partial_pattern(void) +{ + asm volatile ( + "r1 = 1;" + "r2 = 2;" + "*(u32 *)(r10 - 8) = r1;" + "*(u64 *)(r10 - 16) = r2;" + "call %[bpf_get_smp_processor_id];" + "r2 = *(u64 *)(r10 - 16);" + "r1 = *(u32 *)(r10 - 8);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("0: r1 = 1") +__xlated("1: r2 = 2") +/* not patched, spills for -8, -16 not removed */ +__xlated("2: *(u64 *)(r10 -8) = r1") +__xlated("3: *(u64 *)(r10 -16) = r2") +__xlated("5: r0 = &(void __percpu *)(r0)") +__xlated("7: r2 = *(u64 *)(r10 -16)") +__xlated("8: r1 = *(u64 *)(r10 -8)") +/* patched, spills for -24, -32 removed */ +__xlated("10: r0 = &(void __percpu *)(r0)") +__xlated("12: exit") +__success +__naked void min_stack_offset(void) +{ + asm volatile ( + "r1 = 1;" + "r2 = 2;" + /* this call won't be patched */ + "*(u64 *)(r10 - 8) = r1;" + "*(u64 *)(r10 - 16) = r2;" + "call %[bpf_get_smp_processor_id];" + "r2 = *(u64 *)(r10 - 16);" + "r1 = *(u64 *)(r10 - 8);" + /* this call would be patched */ + "*(u64 *)(r10 - 24) = r1;" + "*(u64 *)(r10 - 32) = r2;" + "call %[bpf_get_smp_processor_id];" + "r2 = *(u64 *)(r10 - 32);" + "r1 = *(u64 *)(r10 - 24);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u64 *)(r10 -8) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r1 = *(u64 *)(r10 -8)") +__success +__naked void bad_fixed_read(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 8) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 8);" + "r1 = r10;" + "r1 += -8;" + "r1 = *(u64 *)(r1 - 0);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u64 *)(r10 -8) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r1 = *(u64 *)(r10 -8)") +__success +__naked void bad_fixed_write(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 8) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 8);" + "r1 = r10;" + "r1 += -8;" + "*(u64 *)(r1 - 0) = r1;" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("6: *(u64 *)(r10 -16) = r1") +__xlated("8: r0 = &(void __percpu *)(r0)") +__xlated("10: r1 = *(u64 *)(r10 -16)") +__success +__naked void bad_varying_read(void) +{ + asm volatile ( + "r6 = *(u64 *)(r1 + 0);" /* random scalar value */ + "r6 &= 0x7;" /* r6 range [0..7] */ + "r6 += 0x2;" /* r6 range [2..9] */ + "r7 = 0;" + "r7 -= r6;" /* r7 range [-9..-2] */ + "r1 = 1;" + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + "r1 = r10;" + "r1 += r7;" + "r1 = *(u8 *)(r1 - 0);" /* touches slot [-16..-9] where spills are stored */ + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("6: *(u64 *)(r10 -16) = r1") +__xlated("8: r0 = &(void __percpu *)(r0)") +__xlated("10: r1 = *(u64 *)(r10 -16)") +__success +__naked void bad_varying_write(void) +{ + asm volatile ( + "r6 = *(u64 *)(r1 + 0);" /* random scalar value */ + "r6 &= 0x7;" /* r6 range [0..7] */ + "r6 += 0x2;" /* r6 range [2..9] */ + "r7 = 0;" + "r7 -= r6;" /* r7 range [-9..-2] */ + "r1 = 1;" + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + "r1 = r10;" + "r1 += r7;" + "*(u8 *)(r1 - 0) = r7;" /* touches slot [-16..-9] where spills are stored */ + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u64 *)(r10 -8) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r1 = *(u64 *)(r10 -8)") +__success +__naked void bad_write_in_subprog(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 8) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 8);" + "r1 = r10;" + "r1 += -8;" + "call bad_write_in_subprog_aux;" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +__used +__naked static void bad_write_in_subprog_aux(void) +{ + asm volatile ( + "r0 = 1;" + "*(u64 *)(r1 - 0) = r0;" /* invalidates nocsr contract for caller: */ + "exit;" /* caller stack at -8 used outside of the pattern */ + ::: __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u64 *)(r10 -8) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r1 = *(u64 *)(r10 -8)") +__success +__naked void bad_helper_write(void) +{ + asm volatile ( + "r1 = 1;" + /* nocsr pattern with stack offset -8 */ + "*(u64 *)(r10 - 8) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 8);" + "r1 = r10;" + "r1 += -8;" + "r2 = 1;" + "r3 = 42;" + /* read dst is fp[-8], thus nocsr rewrite not applied */ + "call %[bpf_probe_read_kernel];" + "exit;" + : + : __imm(bpf_get_smp_processor_id), + __imm(bpf_probe_read_kernel) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +/* main, not patched */ +__xlated("1: *(u64 *)(r10 -8) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r1 = *(u64 *)(r10 -8)") +__xlated("9: call pc+1") +__xlated("10: exit") +/* subprogram, patched */ +__xlated("11: r1 = 1") +__xlated("13: r0 = &(void __percpu *)(r0)") +__xlated("15: exit") +__success +__naked void invalidate_one_subprog(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 8) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 8);" + "r1 = r10;" + "r1 += -8;" + "r1 = *(u64 *)(r1 - 0);" + "call invalidate_one_subprog_aux;" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +__used +__naked static void invalidate_one_subprog_aux(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 8) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 8);" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +/* main */ +__xlated("0: r1 = 1") +__xlated("2: r0 = &(void __percpu *)(r0)") +__xlated("4: call pc+1") +__xlated("5: exit") +/* subprogram */ +__xlated("6: r1 = 1") +__xlated("8: r0 = &(void __percpu *)(r0)") +__xlated("10: *(u64 *)(r10 -16) = r1") +__xlated("11: exit") +__success +__naked void subprogs_use_independent_offsets(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + "call subprogs_use_independent_offsets_aux;" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +__used +__naked static void subprogs_use_independent_offsets_aux(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 24) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 24);" + "*(u64 *)(r10 - 16) = r1;" + "exit;" + : + : __imm(bpf_get_smp_processor_id) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__log_level(4) __msg("stack depth 8") +__xlated("2: r0 = &(void __percpu *)(r0)") +__success +__naked void helper_call_does_not_prevent_nocsr(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 8) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 8);" + "*(u64 *)(r10 - 8) = r1;" + "call %[bpf_get_prandom_u32];" + "r1 = *(u64 *)(r10 - 8);" + "exit;" + : + : __imm(bpf_get_smp_processor_id), + __imm(bpf_get_prandom_u32) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__log_level(4) __msg("stack depth 16") +/* may_goto counter at -16 */ +__xlated("0: *(u64 *)(r10 -16) =") +__xlated("1: r1 = 1") +__xlated("3: r0 = &(void __percpu *)(r0)") +/* may_goto expansion starts */ +__xlated("5: r11 = *(u64 *)(r10 -16)") +__xlated("6: if r11 == 0x0 goto pc+3") +__xlated("7: r11 -= 1") +__xlated("8: *(u64 *)(r10 -16) = r11") +/* may_goto expansion ends */ +__xlated("9: *(u64 *)(r10 -8) = r1") +__xlated("10: exit") +__success +__naked void may_goto_interaction(void) +{ + asm volatile ( + "r1 = 1;" + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + ".8byte %[may_goto];" + /* just touch some stack at -8 */ + "*(u64 *)(r10 - 8) = r1;" + "exit;" + : + : __imm(bpf_get_smp_processor_id), + __imm_insn(may_goto, BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, +1 /* offset */, 0)) + : __clobber_all); +} + +__used +__naked static void dummy_loop_callback(void) +{ + asm volatile ( + "r0 = 0;" + "exit;" + ::: __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__log_level(4) __msg("stack depth 32+0") +__xlated("2: r1 = 1") +__xlated("3: w0 =") +__xlated("4: r0 = &(void __percpu *)(r0)") +__xlated("5: r0 = *(u32 *)(r0 +0)") +/* bpf_loop params setup */ +__xlated("6: r2 =") +__xlated("7: r3 = 0") +__xlated("8: r4 = 0") +/* ... part of the inlined bpf_loop */ +__xlated("12: *(u64 *)(r10 -32) = r6") +__xlated("13: *(u64 *)(r10 -24) = r7") +__xlated("14: *(u64 *)(r10 -16) = r8") +/* ... */ +__xlated("21: call pc+8") /* dummy_loop_callback */ +/* ... last insns of the bpf_loop_interaction1 */ +__xlated("28: r0 = 0") +__xlated("29: exit") +/* dummy_loop_callback */ +__xlated("30: r0 = 0") +__xlated("31: exit") +__success +__naked int bpf_loop_interaction1(void) +{ + asm volatile ( + "r1 = 1;" + /* nocsr stack region at -16, but could be removed */ + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + "r2 = %[dummy_loop_callback];" + "r3 = 0;" + "r4 = 0;" + "call %[bpf_loop];" + "r0 = 0;" + "exit;" + : + : __imm_ptr(dummy_loop_callback), + __imm(bpf_get_smp_processor_id), + __imm(bpf_loop) + : __clobber_common + ); +} + +SEC("raw_tp") +__arch_x86_64 +__log_level(4) __msg("stack depth 40+0") +/* call bpf_get_smp_processor_id */ +__xlated("2: r1 = 42") +__xlated("3: w0 =") +__xlated("4: r0 = &(void __percpu *)(r0)") +__xlated("5: r0 = *(u32 *)(r0 +0)") +/* call bpf_get_prandom_u32 */ +__xlated("6: *(u64 *)(r10 -16) = r1") +__xlated("7: call") +__xlated("8: r1 = *(u64 *)(r10 -16)") +/* ... */ +/* ... part of the inlined bpf_loop */ +__xlated("15: *(u64 *)(r10 -40) = r6") +__xlated("16: *(u64 *)(r10 -32) = r7") +__xlated("17: *(u64 *)(r10 -24) = r8") +__success +__naked int bpf_loop_interaction2(void) +{ + asm volatile ( + "r1 = 42;" + /* nocsr stack region at -16, cannot be removed */ + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_prandom_u32];" + "r1 = *(u64 *)(r10 - 16);" + "r2 = %[dummy_loop_callback];" + "r3 = 0;" + "r4 = 0;" + "call %[bpf_loop];" + "r0 = 0;" + "exit;" + : + : __imm_ptr(dummy_loop_callback), + __imm(bpf_get_smp_processor_id), + __imm(bpf_get_prandom_u32), + __imm(bpf_loop) + : __clobber_common + ); +} + +char _license[] SEC("license") = "GPL"; From patchwork Mon Jul 15 23:02:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733948 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC1491448E0 for ; Mon, 15 Jul 2024 23:02:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084561; cv=none; b=D0CL3BnLevFQrrxPE7P2f3LgPPKDm1ctCPyydNYiHNwwvKbmXhNx98eHTP9KmVv4hQIAkuoibLVZgn7JSoC2GBIbMy/nBPnoAeKpSn/BxLuxrctR/w9a2XX8wEG8fFARuaM0A5XciDYE+xfaJJFwUfmxnJXdHH/doqwLcmR1YnU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084561; c=relaxed/simple; bh=6rR8nvi0lIDNBcvG9RjGUZ9FH0QpcOFS3RD/q60L57k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pz/vew6tIYha6RfDEiYrLknJiEizZZJzJT+UJIPQDruyj9smubPkDh3D/WZLpb0eIzFTnhDCPR8euZ64xDlo3V2pKZWAJiDNV+eQB02sD2plAPCbkbglPbz7XLtRWehx/zOQ/JMpUzTCb1OCxadUXz1ZeFA7KQ/JEDoWVa79QNo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XQJzvQ6J; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XQJzvQ6J" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-70b42250526so3686506b3a.1 for ; Mon, 15 Jul 2024 16:02:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084559; x=1721689359; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sSvSc3lEG3d5pVzLc2yz77lX3sNbF6SElydtummTNs0=; b=XQJzvQ6JZOzE7xY71fbWa1SOEQ91l595gWfah19G68HODYA6lUYT5DEqCJif7/7dlm zsqR8AMlAA1+10RL2Lz9LQ+9NtY1qZnP0k6FnMcTbnT2VimxOKGlCkiQ5uFX5VvwJfRD Nz6qsIv1ISl3uZcD146Nuzx8tTsUKsk7jDqOeNh+KplwkIbbqBX76srWPZlrprovt79Q DkJNFb11c/ZkHoeP54zj3aDHakrCa7bhZUPx5d6BKsgIbKbL5UjK2MAqCLQCdFcvtcjd 7l1E6JAaN+hrAXgUv9m4Lawa9mp2YHwx1VTKJH/HgK85F2m2LiyPg0P8GRT0HGOLFDNS 52rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084559; x=1721689359; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sSvSc3lEG3d5pVzLc2yz77lX3sNbF6SElydtummTNs0=; b=ToYHrH7joU1TivsLeKi43CFnkuEoUSixVzGYY2jWK6MeQsCdPfWUuRO9OT6DfULeKj 0TXdcDvN2PWLgsxWqqGhNnhimxghXniyUc+qpiUsYE6oor50se6hWgIKAxTh7uQev82S EhNw70tNG0fPcXRialKbEnaFlT1zH+VpEiLZAsIDptcjQT3fyYQ5QPp8RBfEp18ojfR7 HXkOKCB6cW/yk6Vh6tyHQGXSxw0ha5vQcnKmL3+cP8KfsgUrBjvkouOlxQn3QR1NdDSa oFN9lSCaTXEiA1xxioH+wbja1SXc6iPkH5C2LMpabIsJOAHICQZEbdAaJdXeOl5z3Ysq vo0g== X-Gm-Message-State: AOJu0YzX4o5WPtHZZICHFxh9m54kogp94qcc5/fPcxz+pxiGy6ddR6XI t8pRxZskeGnY0yi4C2UpWh4YHkv/KSnOYHSR+SYVeVSIYaDCh1P+3dKO4w== X-Google-Smtp-Source: AGHT+IEopLU7SYBCSGvl2t/gbTCw0+1tSTdnYUn6M3uHSrKcqPlxtZsr4yazjcbViJab+86pUAo/6g== X-Received: by 2002:a05:6a00:2381:b0:705:b6d3:4f15 with SMTP id d2e1a72fcca58-70c2e9b5544mr606985b3a.25.1721084558680; Mon, 15 Jul 2024 16:02:38 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:38 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 11/12] bpf: do check_nocsr_stack_contract() for ARG_ANYTHING helper params Date: Mon, 15 Jul 2024 16:02:00 -0700 Message-ID: <20240715230201.3901423-12-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net There is a number of BPF helper functions that use ARG_ANYTHING to mark parameters that are used as memory addresses. An address of BPF stack slot could be passed as such parameter for two such helper functions: - bpf_probe_read_kernel - bpf_probe_read_kernel_str This might lead to a surprising behavior in combination with nocsr rewrites, e.g. consider the program below: 1: r1 = 1; /* nocsr pattern with stack offset -16 */ 2: *(u64 *)(r10 - 16) = r1; 3: call %[bpf_get_smp_processor_id]; 4: r1 = *(u64 *)(r10 - 16); 5: r1 = r10; 6: r1 += -8; 7: r2 = 1; 8: r3 = r10; 9: r3 += -16; /* bpf_probe_read_kernel(dst: &fp[-8], size: 1, src: &fp[-16]) */ 10: call %[bpf_probe_read_kernel]; 11: exit; Here nocsr rewrite logic would remove instructions (2) and (4). However, (2) writes a value that is later read by a call at (10). Function check_func_arg() is called from check_helper_call() and is responsible for memory access checks, when helper argument type is declared as ARG_PTR_TO_... . However, for ARG_ANYTHING this function returns early and does not call check_stack_{read,write}(). This patch opts to add a check_nocsr_stack_contract() to the ARG_ANYTHING return path if passed parameter is a pointer to stack. Signed-off-by: Eduard Zingerman --- kernel/bpf/verifier.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 438daf36a694..77affc563a64 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -8684,11 +8684,15 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, return err; if (arg_type == ARG_ANYTHING) { + /* return value depends on env->allow_ptr_leaks */ if (is_pointer_value(env, regno)) { verbose(env, "R%d leaks addr into helper function\n", regno); return -EACCES; } + if (reg->type == PTR_TO_STACK) + check_nocsr_stack_contract(env, cur_func(env), insn_idx, + reg->smin_value + reg->off); return 0; } From patchwork Mon Jul 15 23:02:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13733950 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B268813AA45 for ; Mon, 15 Jul 2024 23:02:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084562; cv=none; b=ul/ZontTUu2Rwt/nQ6vm8GLcHqv4drnwg8t7OduQKHkGoU5KtULmqlsEIf2QFsBvSDRI0IuJAqbAZoraDn1SO/A3p6+slarHdDMOkydjcHAy1iZr/SjIfDuSn/qIW2kpNai3uh+OrZvCiPD5mzycVwuLFD90X2IVdFknVPIqx24= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721084562; c=relaxed/simple; bh=APg9O6emCXAGGjuLk1KItMYvtBnc/1l5qtVb5mqamqA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EARmiANlndT8mzBEFqRLCfh2FqjXH+ZjydMtg579hUQP5vQiu/3CUvthsHA3d78oYSyEvTaA/5LRoXhIy3rnaTytwaqUDhXqQHKFL706EtJBHM50E5Wl7jDOIwIJGy6OEteuJmay9WIYZZpT6YbqNZS2Jfw+rv58rG9+Ly/NVRY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iCmPOIb5; arc=none smtp.client-ip=209.85.215.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iCmPOIb5" Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-78135be2d46so3612524a12.0 for ; Mon, 15 Jul 2024 16:02:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721084560; x=1721689360; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6kOsKjJfIvHIToI5g7d4J3FIacvTfIMk8phPLkMiI5A=; b=iCmPOIb5qFYrvMLk+RtwfN6/EdfPXGTgUCoWtO0Pb9mm1s8PQuCFsq7v4w9SI1dW1T ckC5L1UnWlH71UlAt492qSPprA8Hx48QpI5cRiQBOSF1dBnrMa4EFVrVK10+n5Hz0eQE rIVkW3hUjf1MnDmFX7zN5LxBtqp/yfmbWr5svwRMKjdbMRMhjGXOx5rlGXpCufvo0LMd ZRu6Lg5jNtk8ugGsCCXavfhi1fN4HPfRXlsQr5h803a7zOWT6SRSaJVFNleYrjHepFNS AVdy6T9R3gWAPVvERoVdE7P16+Wj36DOaMqhowIPeJF7M3fea5ejTvzsjU27FyJ+gotV M6pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721084560; x=1721689360; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6kOsKjJfIvHIToI5g7d4J3FIacvTfIMk8phPLkMiI5A=; b=My5KizFoRc/AAh+vajC+/i3AAUxhQa6IVkV4rL+L7J7C+9jBhXbDpBXeMHr+6YgdZS zyeDd7SZTRKDU7onhtnpJw9EMlrDhiViPEqHcodynIuBD3Dm1T8Eig7uDFEJIzBb4n7+ bBXLwce5IUgSpPSpmxEjJ0g5rPFIjBBceweECZRVuzv7/8WYmDU59zbp60GQnUVcHTmT b2ghgrlGdlRxqm8IslI9Uw02sVPMV2GWNiukvy8/4PiUK3nA9AJaqRIDf0q8HXWpsjV1 WX0C8w8b5vFhn229ROzQW7+A0ILrUe9TBuUAhYgP3XlkUYu8uqEwNKnKHaRlJ8rn23WS ML7Q== X-Gm-Message-State: AOJu0YwpDQIva5nMXs4jNhidGQJpgPyqfG5DU2ele4YimCT9eLk+8aEM WAfvG+i61Vb/tUeS+s6U7CenyFDr2+rWbybiFHP44HZwTftMUFPF1Zq2dA== X-Google-Smtp-Source: AGHT+IFp06INSDsZd/BcHaIYy2vMn+Vsqkt42YNWODmyiKxrT1LQtgeiVN8AexzOa+p6dJxJmPhgvA== X-Received: by 2002:a05:6a20:4388:b0:1c3:b144:2ee1 with SMTP id adf61e73a8af0-1c3f12978f7mr487307637.44.1721084559672; Mon, 15 Jul 2024 16:02:39 -0700 (PDT) Received: from badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b7ecc9d36sm4915344b3a.205.2024.07.15.16.02.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 16:02:39 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, Eduard Zingerman Subject: [bpf-next v3 12/12] selftests/bpf: check nocsr contract for bpf_probe_read_kernel() Date: Mon, 15 Jul 2024 16:02:01 -0700 Message-ID: <20240715230201.3901423-13-eddyz87@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240715230201.3901423-1-eddyz87@gmail.com> References: <20240715230201.3901423-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Check that nocsr contract is enforced for bpf_probe_read_kernel() and bpf_probe_read_kernel_str(). These functions access memory via parameter with type ARG_ANYTHING. Signed-off-by: Eduard Zingerman --- .../selftests/bpf/progs/verifier_nocsr.c | 89 +++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/verifier_nocsr.c b/tools/testing/selftests/bpf/progs/verifier_nocsr.c index 84f76f850e9a..8b789f56c9e4 100644 --- a/tools/testing/selftests/bpf/progs/verifier_nocsr.c +++ b/tools/testing/selftests/bpf/progs/verifier_nocsr.c @@ -462,6 +462,95 @@ __naked void bad_helper_write(void) : __clobber_all); } +SEC("raw_tp") +__arch_x86_64 +__xlated("1: *(u64 *)(r10 -16) = r1") +__xlated("3: r0 = &(void __percpu *)(r0)") +__xlated("5: r1 = *(u64 *)(r10 -16)") +__success +__naked void bad_probe_read_kernel_fixed_off(void) +{ + asm volatile ( + "r1 = 1;" + /* nocsr pattern with stack offset -24 */ + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + "r1 = r10;" + "r1 += -8;" + "r2 = 1;" + "r3 = r10;" + "r3 += -16;" + /* read src is fp[-16], thus nocsr rewrite not applied */ + "call %[bpf_probe_read_kernel];" + "exit;" + : + : __imm(bpf_get_smp_processor_id), + __imm(bpf_probe_read_kernel) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("2: r0 = &(void __percpu *)(r0)") +__success +__naked void good_probe_read_kernel_fixed_off(void) +{ + asm volatile ( + "r1 = 1;" + /* nocsr pattern with stack offset -24 */ + "*(u64 *)(r10 - 24) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 24);" + "r1 = r10;" + "r1 += -8;" + "r2 = 1;" + "r3 = r10;" + "r3 += -16;" + /* read src is fp[-16], nocsr rewrite should be ok */ + "call %[bpf_probe_read_kernel];" + "exit;" + : + : __imm(bpf_get_smp_processor_id), + __imm(bpf_probe_read_kernel) + : __clobber_all); +} + +SEC("raw_tp") +__arch_x86_64 +__xlated("6: *(u64 *)(r10 -16) = r1") +__xlated("8: r0 = &(void __percpu *)(r0)") +__xlated("10: r1 = *(u64 *)(r10 -16)") +__success +__naked void bad_probe_read_kernel_var_off(void) +{ + asm volatile ( + "r6 = *(u64 *)(r1 + 0);" /* random scalar value */ + "r6 &= 0x7;" /* r6 range [0..7] */ + "r6 += 0x2;" /* r6 range [2..9] */ + "r7 = 0;" + "r7 -= r6;" /* r7 range [-9..-2] */ + "r1 = 1;" + /* nocsr pattern with stack offset -24 */ + "*(u64 *)(r10 - 16) = r1;" + "call %[bpf_get_smp_processor_id];" + "r1 = *(u64 *)(r10 - 16);" + "r1 = r10;" + "r1 += -8;" + "r2 = 1;" + "r3 = r10;" + "r3 += r7;" + /* read src is fp[-9..-2], + * which touches range [-16..-9] reserved for nocsr rewrite + */ + "call %[bpf_probe_read_kernel_str];" + "exit;" + : + : __imm(bpf_get_smp_processor_id), + __imm(bpf_probe_read_kernel_str) + : __clobber_all); +} + SEC("raw_tp") __arch_x86_64 /* main, not patched */