From patchwork Tue Nov 12 11:09:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13872096 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9B472123DB for ; Tue, 12 Nov 2024 11:09:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731409769; cv=none; b=Qdn73hONEJwNQjKUHGx/5ZSYOx2uLlmXluVR/wd9k6t9Oe9GTpiPzIFwVLIWnGCmCTo6p2PxG1a/ZUTgyqRKvwpfeFch+0ePC4XoIpIUhQYdLB+EZEJORFY5eg5laSpgcuMzRckIobCI+Ncl95kWf80/f9SQwLZVvsjzSHgPyrY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731409769; c=relaxed/simple; bh=FawqOUxNkJhlsFIJ/iUBmcyzM8iCx84JZqAfuSGvpw4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=afmRvb5uVQi1NDcfL/tezQrUvNnm649YwWQTzy9+iq9/BbZZO9NCcimZYO8ZqPX7nts82nqVU0lhi88dJQqI8ADjc2Uf5rwdnX8X96dccjQUe+pYlp+0FxD/95HOz88WGhqJH1reDJFmtfLsncNH4XzKqsn1WcfP8vQTwT5yEUU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Xs9zImDL; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Xs9zImDL" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-20c805a0753so52747215ad.0 for ; Tue, 12 Nov 2024 03:09:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731409767; x=1732014567; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ipw+/tOoWpZ+cqXhj6lp+I84AFWlc4yAcESwPamR9RU=; b=Xs9zImDLh5QvgS82ayijAEDKM4oBbjuXN5ozwTqap0kvGa8IKRPmu5YYrhdB40Jg1l CDs9tLMTXFzf7USL6FYZtio7GYm2CbXimWYlpuXn3GwpVMmwHc0xHIbj+ecJNw8CVtZp zs5jw49i1/3LilWOB7dm5m74xRoq8BPx2jiJ2OQ3KA6L/0euyU6bYDD7+5UTQsFmQisf f2bmZyUUjSoxTu+IijRTPSLf21p8fSKxKBczoIaUGzFu4IVf9b6tpIHEdnpL0LgCMqeL 0q5RncCFX1TdFojLGxvfDIemRx2aJMtb9DypLePOZGQmUGp6InTydBrS7kB4xY60GCWJ QxZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731409767; x=1732014567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ipw+/tOoWpZ+cqXhj6lp+I84AFWlc4yAcESwPamR9RU=; b=edVjXfqR36suq3KhwcPisYwbUDeuTILkkcKSWP/OcRC+RdCIDHYdPhtpotQS4sz7jz zwxalzqeWAgm25O1lF/HzdOa51CMKwO+X4ReBVp7cjQ8lx63Wk4uXv760lrF0V9eMmcy t1MEKy9x5f5hiL6tooSZ+HbFo3y7AKOhqURmqvMh0mnk5hEPCErK9XIZdjVIOcdFVJ0L RV2CCCFhybVP5gZV/G64bIZVvY7pEsY/VEDhMwwn9O1jfD8H8QiQ5LbcXeP7xgPDVX31 kwsdZbtjoKpzXmyISujqir9Op5dBbqWznL9AByMJl8ztweVZH8oDL46aHjcsCamV5S2G /1lQ== X-Gm-Message-State: AOJu0YyfhToMD8yvJ5fcMVELhAWkr2t3prxC7iELJ714I4oiMOdd5G4i dOI8j0Qg269LTsf7T0tNahsYt0y5z776gIC6o7NiqRSCFopna7AruME+Hw== X-Google-Smtp-Source: AGHT+IF2MOgE20xqGs7n4O9wjiPghxWyjrRdhRClFxYOxh1cjxaWq2EM4tJ3z14WyktHuM4nC75gGQ== X-Received: by 2002:a17:903:234c:b0:20b:6188:fc5e with SMTP id d9443c01a7336-21183d66a42mr224974075ad.28.1731409766516; Tue, 12 Nov 2024 03:09:26 -0800 (PST) Received: from honey-badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e45eabsm91789135ad.114.2024.11.12.03.09.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 03:09:25 -0800 (PST) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, Eduard Zingerman Subject: [bpf-next 1/4] selftests/bpf: watchdog timer for test_progs Date: Tue, 12 Nov 2024 03:09:03 -0800 Message-ID: <20241112110906.3045278-2-eddyz87@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241112110906.3045278-1-eddyz87@gmail.com> References: <20241112110906.3045278-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net This commit provides a watchdog timer that sets a limit of how long a single sub-test could run: - if sub-test runs for 10 seconds, the name of the test is printed (currently the name of the test is printed only after it finishes); - if sub-test runs for 120 seconds, the running thread is terminated with SIGSEGV (to trigger crash_handler() and get a stack trace). Specifically: - the timer is armed on each call to run_one_test(); - re-armed at each call to test__start_subtest(); - is stopped when exiting run_one_test(). Default timeout could be overridden using '-w' or '--watchdog-timeout' options. Value 0 can be used to turn the timer off. Here is an example execution: $ ./ssh-exec.sh ./test_progs -w 5 -t \ send_signal/send_signal_perf_thread_remote,send_signal/send_signal_nmi_thread_remote WATCHDOG: test case send_signal/send_signal_nmi_thread_remote executes for 5 seconds, terminating with SIGSEGV Caught signal #11! Stack trace: ./test_progs(crash_handler+0x1f)[0x9049ef] /lib64/libc.so.6(+0x40d00)[0x7f1f1184fd00] /lib64/libc.so.6(read+0x4a)[0x7f1f1191cc4a] ./test_progs[0x720dd3] ./test_progs[0x71ef7a] ./test_progs(test_send_signal+0x1db)[0x71edeb] ./test_progs[0x9066c5] ./test_progs(main+0x5ed)[0x9054ad] /lib64/libc.so.6(+0x2a088)[0x7f1f11839088] /lib64/libc.so.6(__libc_start_main+0x8b)[0x7f1f1183914b] ./test_progs(_start+0x25)[0x527385] #292 send_signal:FAIL test_send_signal_common:PASS:reading pipe 0 nsec test_send_signal_common:PASS:reading pipe error: size 0 0 nsec test_send_signal_common:PASS:incorrect result 0 nsec test_send_signal_common:PASS:pipe_write 0 nsec test_send_signal_common:PASS:setpriority 0 nsec Timer is implemented using timer_{create,start} librt API. Internally librt uses pthreads for SIGEV_THREAD timers, so this change adds a background timer thread to the test process. Because of this a few checks in tests 'bpf_iter' and 'iters' need an update to account for an extra thread. For parallelized scenario the watchdog is also created for each worker fork. If one of the workers gets stuck, it would be terminated by a watchdog. In theory, this might lead to a scenario when all worker threads are exhausted, however this should not be a problem for server_main(), as it would exit with some of the tests not run. Signed-off-by: Eduard Zingerman --- .../selftests/bpf/prog_tests/bpf_iter.c | 8 +- .../testing/selftests/bpf/prog_tests/iters.c | 4 +- tools/testing/selftests/bpf/test_progs.c | 104 ++++++++++++++++++ tools/testing/selftests/bpf/test_progs.h | 6 + 4 files changed, 116 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c index b8e1224cfd19..6f1bfacd7375 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c @@ -265,10 +265,10 @@ static void *run_test_task_tid(void *arg) linfo.task.tid = 0; linfo.task.pid = getpid(); - /* This includes the parent thread, this thread, + /* This includes the parent thread, this thread, watchdog timer thread * and the do_nothing_wait thread */ - test_task_common(&opts, 2, 1); + test_task_common(&opts, 3, 1); test_task_common_nocheck(NULL, &num_unknown_tid, &num_known_tid); ASSERT_GT(num_unknown_tid, 2, "check_num_unknown_tid"); @@ -297,7 +297,7 @@ static void test_task_pid(void) opts.link_info = &linfo; opts.link_info_len = sizeof(linfo); - test_task_common(&opts, 1, 1); + test_task_common(&opts, 2, 1); } static void test_task_pidfd(void) @@ -315,7 +315,7 @@ static void test_task_pidfd(void) opts.link_info = &linfo; opts.link_info_len = sizeof(linfo); - test_task_common(&opts, 1, 1); + test_task_common(&opts, 2, 1); close(pidfd); } diff --git a/tools/testing/selftests/bpf/prog_tests/iters.c b/tools/testing/selftests/bpf/prog_tests/iters.c index 89ff23c4a8bc..3cea71f9c500 100644 --- a/tools/testing/selftests/bpf/prog_tests/iters.c +++ b/tools/testing/selftests/bpf/prog_tests/iters.c @@ -192,8 +192,8 @@ static void subtest_task_iters(void) syscall(SYS_getpgid); iters_task__detach(skel); ASSERT_EQ(skel->bss->procs_cnt, 1, "procs_cnt"); - ASSERT_EQ(skel->bss->threads_cnt, thread_num + 1, "threads_cnt"); - ASSERT_EQ(skel->bss->proc_threads_cnt, thread_num + 1, "proc_threads_cnt"); + ASSERT_EQ(skel->bss->threads_cnt, thread_num + 2, "threads_cnt"); + ASSERT_EQ(skel->bss->proc_threads_cnt, thread_num + 2, "proc_threads_cnt"); ASSERT_EQ(skel->bss->invalid_cnt, 0, "invalid_cnt"); pthread_mutex_unlock(&do_nothing_mutex); for (int i = 0; i < thread_num; i++) diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index 7421874380c2..6088d8222d59 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -16,6 +16,7 @@ #include #include #include +#include #include "json_writer.h" #include "network_helpers.h" @@ -179,6 +180,88 @@ int usleep(useconds_t usec) return syscall(__NR_nanosleep, &ts, NULL); } +/* Watchdog timer is started by watchdog_start() and stopped by watchdog_stop(). + * If timer is active for longer than env.secs_till_notify, + * it prints the name of the current test to the stderr. + * If timer is active for longer than env.secs_till_kill, + * it kills the thread executing the test by sending a SIGSEGV signal to it. + */ +static void watchdog_timer_func(union sigval sigval) +{ + struct itimerspec timeout = {}; + char test_name[256]; + int err; + + if (env.subtest_state) + snprintf(test_name, sizeof(test_name), "%s/%s", + env.test->test_name, env.subtest_state->name); + else + snprintf(test_name, sizeof(test_name), "%s", + env.test->test_name); + + switch (env.watchdog_state) { + case WD_NOTIFY: + fprintf(env.stderr_saved, "WATCHDOG: test case %s executes for %d seconds...\n", + test_name, env.secs_till_notify); + timeout.it_value.tv_sec = env.secs_till_kill - env.secs_till_notify; + env.watchdog_state = WD_KILL; + err = timer_settime(env.watchdog, 0, &timeout, NULL); + if (err) + fprintf(env.stderr_saved, "Failed to arm watchdog timer\n"); + break; + case WD_KILL: + fprintf(env.stderr_saved, + "WATCHDOG: test case %s executes for %d seconds, terminating with SIGSEGV\n", + test_name, env.secs_till_kill); + pthread_kill(env.main_thread, SIGSEGV); + break; + } +} + +static void watchdog_start(void) +{ + struct itimerspec timeout = {}; + int err; + + if (env.secs_till_kill == 0) + return; + if (env.secs_till_notify > 0) { + env.watchdog_state = WD_NOTIFY; + timeout.it_value.tv_sec = env.secs_till_notify; + } else { + env.watchdog_state = WD_KILL; + timeout.it_value.tv_sec = env.secs_till_kill; + } + err = timer_settime(env.watchdog, 0, &timeout, NULL); + if (err) + fprintf(env.stderr_saved, "Failed to start watchdog timer\n"); +} + +static void watchdog_stop(void) +{ + struct itimerspec timeout = {}; + int err; + + env.watchdog_state = WD_NOTIFY; + err = timer_settime(env.watchdog, 0, &timeout, NULL); + if (err) + fprintf(env.stderr_saved, "Failed to stop watchdog timer\n"); +} + +static void watchdog_init(void) +{ + struct sigevent watchdog_sev = { + .sigev_notify = SIGEV_THREAD, + .sigev_notify_function = watchdog_timer_func, + }; + int err; + + env.main_thread = pthread_self(); + err = timer_create(CLOCK_MONOTONIC, &watchdog_sev, &env.watchdog); + if (err) + fprintf(stderr, "Failed to initialize watchdog timer\n"); +} + static bool should_run(struct test_selector *sel, int num, const char *name) { int i; @@ -515,6 +598,7 @@ bool test__start_subtest(const char *subtest_name) env.subtest_state = subtest_state; stdio_hijack_init(&subtest_state->log_buf, &subtest_state->log_cnt); + watchdog_start(); return true; } @@ -780,6 +864,7 @@ enum ARG_KEYS { ARG_DEBUG = -1, ARG_JSON_SUMMARY = 'J', ARG_TRAFFIC_MONITOR = 'm', + ARG_WATCHDOG_TIMEOUT = 'w', }; static const struct argp_option opts[] = { @@ -810,6 +895,8 @@ static const struct argp_option opts[] = { { "traffic-monitor", ARG_TRAFFIC_MONITOR, "NAMES", 0, "Monitor network traffic of tests with name matching the pattern (supports '*' wildcard)." }, #endif + { "watchdog-timeout", ARG_WATCHDOG_TIMEOUT, "SECONDS", 0, + "Kill the process if tests are not making progress for specified number of seconds." }, {}, }; @@ -1035,6 +1122,16 @@ static error_t parse_arg(int key, char *arg, struct argp_state *state) true); break; #endif + case ARG_WATCHDOG_TIMEOUT: + env->secs_till_kill = atoi(arg); + if (env->secs_till_kill < 0) { + fprintf(stderr, "Invalid watchdog timeout: %s.\n", arg); + return -EINVAL; + } + if (env->secs_till_kill < env->secs_till_notify) { + env->secs_till_notify = 0; + } + break; default: return ARGP_ERR_UNKNOWN; } @@ -1263,10 +1360,12 @@ static void run_one_test(int test_num) stdio_hijack(&state->log_buf, &state->log_cnt); + watchdog_start(); if (test->run_test) test->run_test(); else if (test->run_serial_test) test->run_serial_test(); + watchdog_stop(); /* ensure last sub-test is finalized properly */ if (env.subtest_state) @@ -1707,6 +1806,7 @@ static int worker_main_send_subtests(int sock, struct test_state *state) static int worker_main(int sock) { save_netns(); + watchdog_init(); while (true) { /* receive command */ @@ -1816,6 +1916,8 @@ int main(int argc, char **argv) sigaction(SIGSEGV, &sigact, NULL); + env.secs_till_notify = 10; + env.secs_till_kill = 120; err = argp_parse(&argp, argc, argv, 0, NULL, &env); if (err) return err; @@ -1824,6 +1926,8 @@ int main(int argc, char **argv) if (err) return err; + watchdog_init(); + /* Use libbpf 1.0 API mode */ libbpf_set_strict_mode(LIBBPF_STRICT_ALL); libbpf_set_print(libbpf_print_fn); diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h index 7a58895867c3..74de33ae37e5 100644 --- a/tools/testing/selftests/bpf/test_progs.h +++ b/tools/testing/selftests/bpf/test_progs.h @@ -131,6 +131,12 @@ struct test_env { pid_t *worker_pids; /* array of worker pids */ int *worker_socks; /* array of worker socks */ int *worker_current_test; /* array of current running test for each worker */ + + pthread_t main_thread; + int secs_till_notify; + int secs_till_kill; + timer_t watchdog; /* watch for stalled tests/subtests */ + enum { WD_NOTIFY, WD_KILL } watchdog_state; }; #define MAX_LOG_TRUNK_SIZE 8192 From patchwork Tue Nov 12 11:09:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13872097 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96EC9213125 for ; Tue, 12 Nov 2024 11:09:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731409770; cv=none; b=X5r8UrZpCE9dH6VML1D/wW6zZ96yWo6Tel2Xc9JfQpomb10J7pWKDjoad8pb60PMvk1LC41NYiFnaEo4b6SntWgzWc5QcKI69QSQRxSFAWZDzg/F2SCAdl4aAZz2xRa5/3VCE5ZZLd/q+rx87nkUof3arxlImx0Bcy8Q0hwMLzA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731409770; c=relaxed/simple; bh=4oZ0Fb/3Gvl9wYrbwx6xGDbjYJ3uyaY8dPZkefpr8z8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Yg0txyMAXBDLwIEqSPXmn3+sal/Ysr7HV/FJEBkFeYNTwmLzGpmkUyantmmMuP1D6EwdLZBgUuU+woZcH90tOMq7oVkhpZpm8XWd3U+o8Tn/oJtQHojTJ6y3J3Car0MA0wnNi5Dw6RO2BvUheU4kakhPLml+wzuekCYTMGXUAP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=c6Uqjfh6; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="c6Uqjfh6" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-211a4682fcaso12208915ad.2 for ; Tue, 12 Nov 2024 03:09:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731409768; x=1732014568; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AWgwsdZ5ATw7w3LmjmIMGPEKMBsYBLsMaihOkWXYU+U=; b=c6Uqjfh65tczvQHSgYffnGztwhRVxfKXv/SeWdB/EdQdXsKODpv0lgDQFL/l2tRDXj 1rYrM08awuWjG2+XYYdn/yf1kINOTpzn6tXaUEbErOCfaJ6PxlweudXts7ro4Kg7+H4A yKocOcs/krob1oBnko8JfV4fxuZCgjk8IgsGCQd2us9dhyEm4XfGBPTZxZved3Mj0QqP smUCunhBWmPfD0g0t+kl80ywoe6vKOpyuFRLk2gW2ZFtPmVxZGHmB5uWbEIUmgJpenUX 3EU5+C/3VYL8WB5lb7jzK/9Zd7chmLnAQh3Upc8JcVhAk3C4ia2yQXyebc1+jEHuzP4w Qa0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731409768; x=1732014568; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AWgwsdZ5ATw7w3LmjmIMGPEKMBsYBLsMaihOkWXYU+U=; b=wvcIEdHl1kojN3jdsktIb66G3NR+H72woVSlUiZCurd9AmvxwwewFsfV5RgXH74iUG v4dRAS5ky8OKb404T8fysgdPyOWZo/565CsL9o2uEyvI3cbyfXiM9aWXx5w3Ifv7pxKX olUjnWNZcDh8hxSH1zWq//WGlX2nzF3BO8YpkH8eJq0aBRGYMWHy3I0ZOVvfzbTkwafP EyyboqDdDWSUlQpyEYJqvEl/mEz2S8M4H8GD0kNNkTUHwxavoWY+D9fc0/2JcVPKJ7NH 5t7D+BAHWjLA6wpQiYPySeD3NwWSPYaceYPs9QQlVPV8MnNR6lhV8PmKveLQ6Z5vvtvh 8n9g== X-Gm-Message-State: AOJu0YyqGJysx4Az/DaCzYUks8iBaT6jbGn/DSYwumT6DuNkk/aZ8s9S JlVxIPBXRWm8HQ5qIE6o5IeXiqvhX7aWiGomNka8MJ0jsO9+bHzsmoa7Vg== X-Google-Smtp-Source: AGHT+IGlPRVoVQ96upJX+wZY6hhrhgFyXDEPoilKP2ZZhpD/DyXcP0oIYBP47Q+WMLmD/Do2BWpIZw== X-Received: by 2002:a17:902:ecc6:b0:20b:8776:4902 with SMTP id d9443c01a7336-2118359cd7bmr213261895ad.38.1731409767569; Tue, 12 Nov 2024 03:09:27 -0800 (PST) Received: from honey-badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e45eabsm91789135ad.114.2024.11.12.03.09.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 03:09:26 -0800 (PST) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, Eduard Zingerman Subject: [bpf-next 2/4] selftests/bpf: add read_with_timeout() utility function Date: Tue, 12 Nov 2024 03:09:04 -0800 Message-ID: <20241112110906.3045278-3-eddyz87@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241112110906.3045278-1-eddyz87@gmail.com> References: <20241112110906.3045278-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net int read_with_timeout(int fd, char *buf, size_t count, long usec) As a regular read(2), but allows to specify a timeout in micro-seconds. Returns -EAGAIN on timeout. Implemented using select(). Signed-off-by: Eduard Zingerman --- tools/testing/selftests/bpf/Makefile | 1 + tools/testing/selftests/bpf/io_helpers.c | 21 +++++++++++++++++++++ tools/testing/selftests/bpf/io_helpers.h | 7 +++++++ 3 files changed, 29 insertions(+) create mode 100644 tools/testing/selftests/bpf/io_helpers.c create mode 100644 tools/testing/selftests/bpf/io_helpers.h diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index edef5df08cb2..b1080284522d 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -742,6 +742,7 @@ TRUNNER_EXTRA_SOURCES := test_progs.c \ unpriv_helpers.c \ netlink_helpers.c \ jit_disasm_helpers.c \ + io_helpers.c \ test_loader.c \ xsk.c \ disasm.c \ diff --git a/tools/testing/selftests/bpf/io_helpers.c b/tools/testing/selftests/bpf/io_helpers.c new file mode 100644 index 000000000000..4ada0a74aa1f --- /dev/null +++ b/tools/testing/selftests/bpf/io_helpers.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +int read_with_timeout(int fd, char *buf, size_t count, long usec) +{ + const long M = 1000 * 1000; + struct timeval tv = { usec / M, usec % M }; + fd_set fds; + int err; + + FD_ZERO(&fds); + FD_SET(fd, &fds); + err = select(fd + 1, &fds, NULL, NULL, &tv); + if (err < 0) + return err; + if (FD_ISSET(fd, &fds)) + return read(fd, buf, count); + return -EAGAIN; +} diff --git a/tools/testing/selftests/bpf/io_helpers.h b/tools/testing/selftests/bpf/io_helpers.h new file mode 100644 index 000000000000..21e1134cd3ce --- /dev/null +++ b/tools/testing/selftests/bpf/io_helpers.h @@ -0,0 +1,7 @@ +// SPDX-License-Identifier: GPL-2.0 +#include + +/* As a regular read(2), but allows to specify a timeout in micro-seconds. + * Returns -EAGAIN on timeout. + */ +int read_with_timeout(int fd, char *buf, size_t count, long usec); From patchwork Tue Nov 12 11:09:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13872098 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A05C213129 for ; Tue, 12 Nov 2024 11:09:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731409771; cv=none; b=EQ2Kl+Sf4jAI038C0p6ekEOl2kObVMPJonmsq8wPY/wtl9GAGlMScKIPGoboxsRD+q+fcbTnjt7SCm9GD8pSF1hoWsuR84YHNx6Z/4B4Bx8yuaEYqdRbE+fCdp0GHLch2nU3B+GxGZ1/4xH+j9aXR5SfTHnieAhQ+FlLFsiwFfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731409771; c=relaxed/simple; bh=6DrMSe8zGJGPSiGwV4J0C5ZfwYoLp2Kc7MGmyGcVGEE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=k3dq+WgbpAcpHKp5nab4E9QdF0E2wdtAnP95odQQVW79k2oXfdh5VHSv3CXAnuIXun3YhJ6rrzEFE2nOnpxRrQV9hk27x2tqtHAaAW9hGtD8Wqg69U6anOus+dk7M03/s5bDIviyrPEQGbLTW4GcdTiDfB7ZkPuai7hzV84Q3CY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QtAhiPce; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QtAhiPce" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2114214c63eso46168545ad.3 for ; Tue, 12 Nov 2024 03:09:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731409768; x=1732014568; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rhEC4u2dafLUPU8BzHjryYcyOG5IYjpEXHjCRLNA+8U=; b=QtAhiPceqRrCzihmPwmBNRzJw5KS52rVESBA9L98YBpiOSHFUoJgG+tTmrBtfh/+vu JMZn9+SpcuA6tVx/3LLqF7ZuYP+9mUl8c1T8Ni364fmJb9yF2cMyEMCLDI1LpBIHcCx8 GS2a4Cj9lpvYe4CpIY5w1j0AV3aoC5qes/L8Vm1JxP9D6ZVqsbdod0jACJROqtH5pSdv rBJQdDdb/fiUNwF4Tn/NMdKCtK9iJ1iKc5fyDsY+PBV52qDsCoZSMl7B6n0wNbsOOjGc EuMNB1vyO59lLB++7c4YiF1MDUooZWCBe+fjEOjGVAmDFRl7LwMf0KPhMm28eSQqKC1F O9mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731409768; x=1732014568; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rhEC4u2dafLUPU8BzHjryYcyOG5IYjpEXHjCRLNA+8U=; b=tqsEWZkqCUpYMoyHrZDHLWRiaf/dMT5qY1Dq/4fuTy2BInCNNKdUxKZcQI83ZkKD9n NCwQEDW0AU9qXO6zfhXPBeMdOQZvE2nyPyPNz5fXXM/krW6hfVyV6NTphJju8RaD6Cm3 P/8pXZb9xaD93h9ASP9SLSrb/UH9RFzfl3XA4bCc+TilTs2r3AcbafMs/hVVUTY60+dA DUTNQSGij54+4rT4h9ZRJD5PWCLI78FQg/4wYYcZsTJaYlBjlSmdWOf5oJ1y/Eu8adR0 wGT/V82L8wBr5EMkrbdgc/3cSDFVwI2Z4IfJ+mCI0pAcqUucnnA5A90R2v1ABfE4Qx/c WBNw== X-Gm-Message-State: AOJu0YwdvZH8w++ap7NYaRb6Xt7tjb7+AciqToxyhomWn//iMMBSiFYi PMx5z6xb96tKO7kELf8rNPTnRoPGApqQUbYqWuMeW9UdRxWCp55Bati3wA== X-Google-Smtp-Source: AGHT+IGVpI6XnjTWGQtRt4W2lVFqOE1z7GiXlvwbP3bgbAvAkdCpS5hYi8e3NY+aEfRPhb3/Sy1G/g== X-Received: by 2002:a17:903:11c9:b0:20c:cccd:17a3 with SMTP id d9443c01a7336-2118359c11amr220845465ad.46.1731409768537; Tue, 12 Nov 2024 03:09:28 -0800 (PST) Received: from honey-badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e45eabsm91789135ad.114.2024.11.12.03.09.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 03:09:28 -0800 (PST) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, Eduard Zingerman Subject: [bpf-next 3/4] selftests/bpf: allow send_signal test to timeout Date: Tue, 12 Nov 2024 03:09:05 -0800 Message-ID: <20241112110906.3045278-4-eddyz87@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241112110906.3045278-1-eddyz87@gmail.com> References: <20241112110906.3045278-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The following invocation: $ t1=send_signal/send_signal_perf_thread_remote \ t2=send_signal/send_signal_nmi_thread_remote \ ./test_progs -t $t1,$t2 Leads to send_signal_nmi_thread_remote to be stuck on a line 180: /* wait for result */ err = read(pipe_c2p[0], buf, 1); In this test case: - perf event PERF_COUNT_HW_CPU_CYCLES is created for parent process; - BPF program is attached to perf event, and sends a signal to child process when event occurs; - parent program burns some CPU in busy loop and calls read() to get notification from child that it received a signal. The perf event is declared with .sample_period = 1. This forces perf to throttle events, and under some unclear conditions the event does not always occur while parent is in busy loop. After parent enters read() system call CPU cycles event won't be generated for parent anymore. Thus, if perf event had not occurred already the test is stuck. This commit updates the parent to wait for notification with a timeout, doing several iterations of busy loop + read_with_timeout(). Signed-off-by: Eduard Zingerman --- .../selftests/bpf/prog_tests/send_signal.c | 32 +++++++++++-------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/send_signal.c b/tools/testing/selftests/bpf/prog_tests/send_signal.c index 1aed94ec14ef..4e03d7a4c6f7 100644 --- a/tools/testing/selftests/bpf/prog_tests/send_signal.c +++ b/tools/testing/selftests/bpf/prog_tests/send_signal.c @@ -3,6 +3,7 @@ #include #include #include "test_send_signal_kern.skel.h" +#include "io_helpers.h" static int sigusr1_received; @@ -24,6 +25,7 @@ static void test_send_signal_common(struct perf_event_attr *attr, int pipe_c2p[2], pipe_p2c[2]; int err = -1, pmu_fd = -1; volatile int j = 0; + int retry_count; char buf[256]; pid_t pid; int old_prio; @@ -163,21 +165,25 @@ static void test_send_signal_common(struct perf_event_attr *attr, /* notify child that bpf program can send_signal now */ ASSERT_EQ(write(pipe_p2c[1], buf, 1), 1, "pipe_write"); - /* For the remote test, the BPF program is triggered from this - * process but the other process/thread is signaled. - */ - if (remote) { - if (!attr) { - for (int i = 0; i < 10; i++) - usleep(1); - } else { - for (int i = 0; i < 100000000; i++) - j /= i + 1; + for (retry_count = 0;;) { + /* For the remote test, the BPF program is triggered from this + * process but the other process/thread is signaled. + */ + if (remote) { + if (!attr) { + for (int i = 0; i < 10; i++) + usleep(1); + } else { + for (int i = 0; i < 100000000; i++) + j /= i + 1; + } } + /* wait for result */ + err = read_with_timeout(pipe_c2p[0], buf, 1, 100); + if (err == -EAGAIN && retry_count++ < 10000) + continue; + break; } - - /* wait for result */ - err = read(pipe_c2p[0], buf, 1); if (!ASSERT_GE(err, 0, "reading pipe")) goto disable_pmu; if (!ASSERT_GT(err, 0, "reading pipe error: size 0")) { From patchwork Tue Nov 12 11:09:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13872099 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6395E213EEE for ; Tue, 12 Nov 2024 11:09:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731409771; cv=none; b=eI30K6vuazneUgsl+Rn+o5lp0vqb0qZ2ceeDei/2I6/vverG4TYlD1MHfDA3QVq3HvfOzXyKzTjuHVkg1rHir4tM7qA3+gUhPy2OtX+EnuQb3dCrIK9XEYVOpIEdlRlaS+7qq+T8RKpjbv8KG1V60ZiLIxBsIj4B917zWeBvhKg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731409771; c=relaxed/simple; bh=m1iiAY5TdZ31BVQfwgfETCvVT0vzokJv88Br9nUkAOQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=akVuNESYPDjt+v7sCA2DbmIo+VZhfErIYMSkTVb3+DAWOwBW/AqIew1BkkV2WLIMmiR5rCECzizww+BUmiHUbWLw4s8U5gS8uS3KrpOyGHtvb5JBYo1BKT8rNU3MFOuzhko9OApvZOBzVU0fditEUnHpP90SKLP+EDMtq113qBM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WK9ZVnGO; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WK9ZVnGO" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-20cdda5cfb6so54550485ad.3 for ; Tue, 12 Nov 2024 03:09:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731409769; x=1732014569; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wbh+Ilos50BuQt+XEpYGq60PcPdNUFcP1RucqNZOLIo=; b=WK9ZVnGOvuPoXBM2+Y4YdvtYNVBwsLHdK+LXfOIuSuq/gM03wW+6WcnFDJ239QKDvC i63uyazNtBXPVw03jQhLt3UqwJIdG0UqZLCIwhzV3OS9Zns3E16IKum3pST2ZQenKdQk Tjfx8XVvWJSZlfrRSjbx1IaPFgnIUyBw2LQMUyH6xNl5KHx/4LivqTYS8u0pEd+BKeE9 KnmQawIEtZoCp4esnZDKZdpb5knLToEWEbsgI61E3ytLDUbj+4yinPW3D8ovbnmAAAK1 3ptZ2ZvevDNkqyWnzsKLn6lSMFVVDipUpbpkxzYIcnDgvF0m2ur6Dni7A+HzbZrhX32b EC1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731409769; x=1732014569; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wbh+Ilos50BuQt+XEpYGq60PcPdNUFcP1RucqNZOLIo=; b=NZt7Ch1GGsOUcA1Kd+4PmvnKsa1DWt5pkF+sFG3XDuWnQjhXTETiktx2ORfjIbj5iY 68g+s2jEGGiyY17soy8gu7xfTxmDihoNI+tgXBaztVbxXvMDk9PkmqJTbSLsFSc14qta Fg/wcyZTs8p1jtUKHnZ+UY75dubBX6D2FIyYEN7hDGkh2DDhBQ+puWYLF5iW0gAapiFj 5TAI6GziMJVgtfEDQRN6PoAwssYkA484HTv/FlCkvUZZjWTeqMDM71qOmZrU9XZAEeKy Lq8ENXunsDoJasZ+gc87i6b4oXu+ky6AGOwv9FJEfyxqDB/Qjoba9kWscJN8ErN566l1 wtlg== X-Gm-Message-State: AOJu0YwrYp4wWd1Nv7TibE0clyFBe7LjflAsovJORlsrbfU4b6Jzrj+b xdIK9VISXEXZTe3elCfQdm0XYj3yuwVfRUrm40QbkFMptRaVqMwLa6VucQ== X-Google-Smtp-Source: AGHT+IGOvnO0SpXcbXoWvMQhV33bX9mHFKrzZASzUuGa1v0OBhQeVIz/UNLYNLUWkgBtUb4Fhyfnyg== X-Received: by 2002:a17:902:ce8b:b0:20c:dd71:c94f with SMTP id d9443c01a7336-2118359aea5mr187400235ad.41.1731409769498; Tue, 12 Nov 2024 03:09:29 -0800 (PST) Received: from honey-badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e45eabsm91789135ad.114.2024.11.12.03.09.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 03:09:29 -0800 (PST) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, Eduard Zingerman Subject: [bpf-next 4/4] selftests/bpf: update send_signal to lower perf evemts frequency Date: Tue, 12 Nov 2024 03:09:06 -0800 Message-ID: <20241112110906.3045278-5-eddyz87@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241112110906.3045278-1-eddyz87@gmail.com> References: <20241112110906.3045278-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Similar to commit [1] sample perf events less often in test_send_signal_nmi(). This should reduce perf events throttling. [1] 7015843afcaf ("selftests/bpf: Fix send_signal test with nested CONFIG_PARAVIRT") Signed-off-by: Eduard Zingerman --- tools/testing/selftests/bpf/prog_tests/send_signal.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/prog_tests/send_signal.c b/tools/testing/selftests/bpf/prog_tests/send_signal.c index 4e03d7a4c6f7..1702aa592c2c 100644 --- a/tools/testing/selftests/bpf/prog_tests/send_signal.c +++ b/tools/testing/selftests/bpf/prog_tests/send_signal.c @@ -229,7 +229,8 @@ static void test_send_signal_perf(bool signal_thread, bool remote) static void test_send_signal_nmi(bool signal_thread, bool remote) { struct perf_event_attr attr = { - .sample_period = 1, + .freq = 1, + .sample_freq = 1000, .type = PERF_TYPE_HARDWARE, .config = PERF_COUNT_HW_CPU_CYCLES, };