| Message ID | 847a5b798f24e81b9dec4e8d9eb3eb1e602a909e.1721903630.git.tony.ambardar@gmail.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | selftests/bpf: Improve libc portability / musl support (part 2) | expand |
On Thu, Jul 25, 2024 at 3:39 AM Tony Ambardar <tony.ambardar@gmail.com> wrote: > > From: Tony Ambardar <tony.ambardar@gmail.com> > > Typically stdin, stdout, stderr are treated as reserved identifiers under > ISO/ANSI C, and a libc implementation is free to define these as macros. Ok, wow that. Do you have a pointer to where in the standard it is said that stdin/stdout/stderr is some sort of reserved identifier that can't be used as a field name? I really don't like these underscored field names. If we have to rename, I'd prefer something like env.saved_stdout instead of env._stdout. But I'd prefer even more if musl wasn't doing this macro definition, of course... > This is the case in musl libc and results in compile errors when these > names are reused as struct fields, as with 'struct test_env' and related > usage in test_progs.[ch] and reg_bounds.c. > > Rename the fields to _stdout and _stderr to avoid many errors seen building > against musl, e.g.: > > In file included from test_progs.h:6, > from test_progs.c:5: > test_progs.c: In function 'print_test_result': > test_progs.c:237:21: error: expected identifier before '(' token > 237 | fprintf(env.stdout, "#%-*d %s:", TEST_NUM_WIDTH, test->test_num, test->test_name); > | ^~~~~~ > test_progs.c:237:9: error: too few arguments to function 'fprintf' > 237 | fprintf(env.stdout, "#%-*d %s:", TEST_NUM_WIDTH, test->test_num, test->test_name); > | ^~~~~~~ > > Signed-off-by: Tony Ambardar <tony.ambardar@gmail.com> > --- > .../selftests/bpf/prog_tests/reg_bounds.c | 2 +- > tools/testing/selftests/bpf/test_progs.c | 66 +++++++++---------- > tools/testing/selftests/bpf/test_progs.h | 8 +-- > 3 files changed, 38 insertions(+), 38 deletions(-) > [...]
On Thu, Jul 25, 2024 at 01:27:03PM -0700, Andrii Nakryiko wrote: > On Thu, Jul 25, 2024 at 3:39 AM Tony Ambardar <tony.ambardar@gmail.com> wrote: > > > > From: Tony Ambardar <tony.ambardar@gmail.com> > > > > Typically stdin, stdout, stderr are treated as reserved identifiers under > > ISO/ANSI C, and a libc implementation is free to define these as macros. > > Ok, wow that. Do you have a pointer to where in the standard it is > said that stdin/stdout/stderr is some sort of reserved identifier that > can't be used as a field name? > I'll need to dig around to share some references. The short answer IIRC is there's enough potential variation in their definitions that their use requires care (or better avoidance). > > I really don't like these underscored field names. If we have to > rename, I'd prefer something like env.saved_stdout instead of > env._stdout. But I'd prefer even more if musl wasn't doing this macro > definition, of course... OK, I'll use clearer names for a v2. I believe the macro definitions are quite common and old, but "how" makes a difference: specifically, using parenthesis happens to break our .stdxxx field names. In glibc <stdio.h> we have for example: ... /* Standard streams. */ extern FILE *stdin; /* Standard input stream. */ extern FILE *stdout; /* Standard output stream. */ extern FILE *stderr; /* Standard error output stream. */ /* C89/C99 say they're macros. Make them happy. */ #define stdin stdin #define stdout stdout #define stderr stderr ... while in musl <stdio.h> we have: ... extern FILE *const stdin; extern FILE *const stdout; extern FILE *const stderr; #define stdin (stdin) #define stdout (stdout) #define stderr (stderr) ... which borks code in test_progs.c: ... env.stderr = stderr; env.stdout = stdout; ... > > > This is the case in musl libc and results in compile errors when these > > names are reused as struct fields, as with 'struct test_env' and related > > usage in test_progs.[ch] and reg_bounds.c. > > > > Rename the fields to _stdout and _stderr to avoid many errors seen building > > against musl, e.g.: > > > > In file included from test_progs.h:6, > > from test_progs.c:5: > > test_progs.c: In function 'print_test_result': > > test_progs.c:237:21: error: expected identifier before '(' token > > 237 | fprintf(env.stdout, "#%-*d %s:", TEST_NUM_WIDTH, test->test_num, test->test_name); > > | ^~~~~~ > > test_progs.c:237:9: error: too few arguments to function 'fprintf' > > 237 | fprintf(env.stdout, "#%-*d %s:", TEST_NUM_WIDTH, test->test_num, test->test_name); > > | ^~~~~~~ > > > > Signed-off-by: Tony Ambardar <tony.ambardar@gmail.com> > > --- > > .../selftests/bpf/prog_tests/reg_bounds.c | 2 +- > > tools/testing/selftests/bpf/test_progs.c | 66 +++++++++---------- > > tools/testing/selftests/bpf/test_progs.h | 8 +-- > > 3 files changed, 38 insertions(+), 38 deletions(-) > > > > [...]
On Fri, Jul 26, 2024 at 09:22:38PM -0700, Tony Ambardar wrote: > On Thu, Jul 25, 2024 at 01:27:03PM -0700, Andrii Nakryiko wrote: > > On Thu, Jul 25, 2024 at 3:39 AM Tony Ambardar <tony.ambardar@gmail.com> wrote: > > > > > > From: Tony Ambardar <tony.ambardar@gmail.com> > > > > > > Typically stdin, stdout, stderr are treated as reserved identifiers under > > > ISO/ANSI C, and a libc implementation is free to define these as macros. > > > > Ok, wow that. Do you have a pointer to where in the standard it is > > said that stdin/stdout/stderr is some sort of reserved identifier that > > can't be used as a field name? > > > > I'll need to dig around to share some references. The short answer IIRC > is there's enough potential variation in their definitions that their > use requires care (or better avoidance). > Hi Andrii, Following up on your request for pointers, some excerpts from a quasi-draft C17 ISO doc located here: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2310.pdf 7.1.2 Standard headers (2) The standard headers are ... <stdio.h> ... (5) Any definition of an object-like macro ... shall expand to code that is fully protected by parentheses ... 7.1.3 Reserved identifiers (1) ... Each macro name in any of the following subclauses ... is reserved for use as specified if any of its associated headers is included ... 7.21.1 Input/output <stdio.h>, Introduction (1) The header <stdio.h> defines several macros ... (3) The macros are ... stderr stdin stdout which are expressions of type "pointer to FILE" ... 7.21.5.4 The freopen function (2) (Footnote 278) The primary use of the freopen function is to change the file associated with a standard text stream (stderr, stdin, or stdout), as those identifiers need not be modifiable lvalues ... So we have reserved idents (IANALL so not sure of field names), macros, parentheses, and potentially unassignable stdout/stderr that might break the output redirection hack in test_progs.c. More than enough to tread carefully I think... Cheers, Tony
diff --git a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c index 0da4225749bd..ff4ebc9eaf3f 100644 --- a/tools/testing/selftests/bpf/prog_tests/reg_bounds.c +++ b/tools/testing/selftests/bpf/prog_tests/reg_bounds.c @@ -1487,7 +1487,7 @@ static int verify_case_opt(struct ctx *ctx, enum num_t init_t, enum num_t cond_t u64 elapsed_ns = get_time_ns() - ctx->start_ns; double remain_ns = elapsed_ns / progress * (1 - progress); - fprintf(env.stderr, "PROGRESS (%s): %d/%d (%.2lf%%), " + fprintf(env._stderr, "PROGRESS (%s): %d/%d (%.2lf%%), " "elapsed %llu mins (%.2lf hrs), " "ETA %.0lf mins (%.2lf hrs)\n", ctx->progress_ctx, diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index f6cfc6a8e8f0..091b49bf671a 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -44,15 +44,15 @@ static void stdio_hijack_init(char **log_buf, size_t *log_cnt) stdout = open_memstream(log_buf, log_cnt); if (!stdout) { - stdout = env.stdout; + stdout = env._stdout; perror("open_memstream"); return; } if (env.subtest_state) - env.subtest_state->stdout = stdout; + env.subtest_state->_stdout = stdout; else - env.test_state->stdout = stdout; + env.test_state->_stdout = stdout; stderr = stdout; #endif @@ -66,8 +66,8 @@ static void stdio_hijack(char **log_buf, size_t *log_cnt) return; } - env.stdout = stdout; - env.stderr = stderr; + env._stdout = stdout; + env._stderr = stderr; stdio_hijack_init(log_buf, log_cnt); #endif @@ -84,13 +84,13 @@ static void stdio_restore_cleanup(void) fflush(stdout); if (env.subtest_state) { - fclose(env.subtest_state->stdout); - env.subtest_state->stdout = NULL; - stdout = env.test_state->stdout; - stderr = env.test_state->stdout; + fclose(env.subtest_state->_stdout); + env.subtest_state->_stdout = NULL; + stdout = env.test_state->_stdout; + stderr = env.test_state->_stdout; } else { - fclose(env.test_state->stdout); - env.test_state->stdout = NULL; + fclose(env.test_state->_stdout); + env.test_state->_stdout = NULL; } #endif } @@ -103,13 +103,13 @@ static void stdio_restore(void) return; } - if (stdout == env.stdout) + if (stdout == env._stdout) return; stdio_restore_cleanup(); - stdout = env.stdout; - stderr = env.stderr; + stdout = env._stdout; + stderr = env._stderr; #endif } @@ -237,25 +237,25 @@ static void print_test_result(const struct prog_test_def *test, const struct tes int skipped_cnt = test_state->skip_cnt; int subtests_cnt = test_state->subtest_num; - fprintf(env.stdout, "#%-*d %s:", TEST_NUM_WIDTH, test->test_num, test->test_name); + fprintf(env._stdout, "#%-*d %s:", TEST_NUM_WIDTH, test->test_num, test->test_name); if (test_state->error_cnt) - fprintf(env.stdout, "FAIL"); + fprintf(env._stdout, "FAIL"); else if (!skipped_cnt) - fprintf(env.stdout, "OK"); + fprintf(env._stdout, "OK"); else if (skipped_cnt == subtests_cnt || !subtests_cnt) - fprintf(env.stdout, "SKIP"); + fprintf(env._stdout, "SKIP"); else - fprintf(env.stdout, "OK (SKIP: %d/%d)", skipped_cnt, subtests_cnt); + fprintf(env._stdout, "OK (SKIP: %d/%d)", skipped_cnt, subtests_cnt); - fprintf(env.stdout, "\n"); + fprintf(env._stdout, "\n"); } static void print_test_log(char *log_buf, size_t log_cnt) { log_buf[log_cnt] = '\0'; - fprintf(env.stdout, "%s", log_buf); + fprintf(env._stdout, "%s", log_buf); if (log_buf[log_cnt - 1] != '\n') - fprintf(env.stdout, "\n"); + fprintf(env._stdout, "\n"); } static void print_subtest_name(int test_num, int subtest_num, @@ -266,14 +266,14 @@ static void print_subtest_name(int test_num, int subtest_num, snprintf(test_num_str, sizeof(test_num_str), "%d/%d", test_num, subtest_num); - fprintf(env.stdout, "#%-*s %s/%s", + fprintf(env._stdout, "#%-*s %s/%s", TEST_NUM_WIDTH, test_num_str, test_name, subtest_name); if (result) - fprintf(env.stdout, ":%s", result); + fprintf(env._stdout, ":%s", result); - fprintf(env.stdout, "\n"); + fprintf(env._stdout, "\n"); } static void jsonw_write_log_message(json_writer_t *w, char *log_buf, size_t log_cnt) @@ -458,7 +458,7 @@ bool test__start_subtest(const char *subtest_name) memset(subtest_state, 0, sub_state_size); if (!subtest_name || !subtest_name[0]) { - fprintf(env.stderr, + fprintf(env._stderr, "Subtest #%d didn't provide sub-test name!\n", state->subtest_num); return false; @@ -466,7 +466,7 @@ bool test__start_subtest(const char *subtest_name) subtest_state->name = strdup(subtest_name); if (!subtest_state->name) { - fprintf(env.stderr, + fprintf(env._stderr, "Subtest #%d: failed to copy subtest name!\n", state->subtest_num); return false; @@ -1036,7 +1036,7 @@ void crash_handler(int signum) sz = backtrace(bt, ARRAY_SIZE(bt)); - if (env.stdout) + if (env._stdout) stdio_restore(); if (env.test) { env.test_state->error_cnt++; @@ -1352,7 +1352,7 @@ static void calculate_summary_and_print_errors(struct test_env *env) if (env->json) { w = jsonw_new(env->json); if (!w) - fprintf(env->stderr, "Failed to create new JSON stream."); + fprintf(env->_stderr, "Failed to create new JSON stream."); } if (w) { @@ -1701,8 +1701,8 @@ int main(int argc, char **argv) return -1; } - env.stdout = stdout; - env.stderr = stderr; + env._stdout = stdout; + env._stderr = stderr; env.has_testmod = true; if (!env.list_test_names) { @@ -1710,7 +1710,7 @@ int main(int argc, char **argv) unload_bpf_testmod(verbose()); if (load_bpf_testmod(verbose())) { - fprintf(env.stderr, "WARNING! Selftests relying on bpf_testmod.ko will be skipped.\n"); + fprintf(env._stderr, "WARNING! Selftests relying on bpf_testmod.ko will be skipped.\n"); env.has_testmod = false; } } @@ -1788,7 +1788,7 @@ int main(int argc, char **argv) } if (env.list_test_names) { - fprintf(env.stdout, "%s\n", test->test_name); + fprintf(env._stdout, "%s\n", test->test_name); env.succ_cnt++; continue; } diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h index b1e949fb16cf..f42f1ae59c6e 100644 --- a/tools/testing/selftests/bpf/test_progs.h +++ b/tools/testing/selftests/bpf/test_progs.h @@ -75,7 +75,7 @@ struct subtest_state { bool skipped; bool filtered; - FILE *stdout; + FILE *_stdout; }; struct test_state { @@ -92,7 +92,7 @@ struct test_state { size_t log_cnt; char *log_buf; - FILE *stdout; + FILE *_stdout; }; struct test_env { @@ -111,8 +111,8 @@ struct test_env { struct test_state *test_state; /* current running test state */ struct subtest_state *subtest_state; /* current running subtest state */ - FILE *stdout; - FILE *stderr; + FILE *_stdout; + FILE *_stderr; int nr_cpus; FILE *json;