| Message ID | 20240902-net-next-mptcp-ksft-subtest-time-v1-0-f1ed499a11b1@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | selftests: mptcp: add time per subtests in TAP output | expand |
On Mon, 02 Sep 2024 13:13:03 +0200 Matthieu Baerts (NGI0) wrote: > Patches here add 'time=<N>ms' in the diagnostic data of the TAP output, > e.g. > > ok 1 - pm_netlink: defaults addr list # time=9ms Looking closer, this: # ok 3 - mptcp[...] MPTCP # time=7184ms # ok 4 - mptcp[...] TCP # time=6458ms Makes NIPA unhappy. The match results for regexps look like this: (None, '4', ' -', 'mptcp[...] MPTCP', ' # ', 'time=6173ms') (None, '4', ' -', 'mptcp[...] TC', None, 'P # time=6173ms') IOW the first one is neat, second one gepooped. The regex really wants there to be no more than a single space before the #. KTAP definition doesn't say that description must not have trailing white space. Best I could come up with is: diff --git a/contest/remote/vmksft-p.py b/contest/remote/vmksft-p.py index fe9e87abdb5c..a37245bd5b30 100755 --- a/contest/remote/vmksft-p.py +++ b/contest/remote/vmksft-p.py @@ -73,7 +73,7 @@ group3 testV skip tests = [] nested_tests = False - result_re = re.compile(r"(not )?ok (\d+)( -)? ([^#]*[^ ])( # )?([^ ].*)?$") + result_re = re.compile(r"(not )?ok (\d+)( -)? ([^#]*[^ ])( +# )?([^ ].*)?$") time_re = re.compile(r"time=(\d+)ms") for line in full_run.split('\n'): Thoughts?
On Tue, 3 Sep 2024 16:22:17 -0700 Jakub Kicinski wrote: > (None, '4', ' -', 'mptcp[...] MPTCP', ' # ', 'time=6173ms') > (None, '4', ' -', 'mptcp[...] TC', None, 'P # time=6173ms') Sorry I fumbled copy/pasting the results, ignore the exact time values in the last group, but the mis-parsing stands.
Hi Jakub, On 04/09/2024 01:22, Jakub Kicinski wrote: > On Mon, 02 Sep 2024 13:13:03 +0200 Matthieu Baerts (NGI0) wrote: >> Patches here add 'time=<N>ms' in the diagnostic data of the TAP output, >> e.g. >> >> ok 1 - pm_netlink: defaults addr list # time=9ms > > Looking closer, this: > > # ok 3 - mptcp[...] MPTCP # time=7184ms > # ok 4 - mptcp[...] TCP # time=6458ms > > Makes NIPA unhappy. The match results for regexps look like this: > > (None, '4', ' -', 'mptcp[...] MPTCP', ' # ', 'time=6173ms') > (None, '4', ' -', 'mptcp[...] TC', None, 'P # time=6173ms') > > IOW the first one is neat, second one gepooped. The regex really wants > there to be no more than a single space before the #. Good catch! > KTAP definition > doesn't say that description must not have trailing white space. Indeed. Same for TAP 13. (TAP 14 is clearer about that and allows multiple spaces) > Best I could come up with is: > > diff --git a/contest/remote/vmksft-p.py b/contest/remote/vmksft-p.py > index fe9e87abdb5c..a37245bd5b30 100755 > --- a/contest/remote/vmksft-p.py > +++ b/contest/remote/vmksft-p.py > @@ -73,7 +73,7 @@ group3 testV skip > tests = [] > nested_tests = False > > - result_re = re.compile(r"(not )?ok (\d+)( -)? ([^#]*[^ ])( # )?([^ ].*)?$") > + result_re = re.compile(r"(not )?ok (\d+)( -)? ([^#]*[^ ])( +# )?([^ ].*)?$") Looks good to me. While at it, we can add a '+' for the spaces after the '#': ( +# +) I see you didn't commit the previous modification. I can open a PR if it helps. > time_re = re.compile(r"time=(\d+)ms") > > for line in full_run.split('\n'): > > Thoughts? In my v2, I will also strip these trailing whitespaces in the selftests, they don't need to be there. Cheers, Matt
On Wed, 4 Sep 2024 18:15:09 +0200 Matthieu Baerts wrote: > > Best I could come up with is: > > > > diff --git a/contest/remote/vmksft-p.py b/contest/remote/vmksft-p.py > > index fe9e87abdb5c..a37245bd5b30 100755 > > --- a/contest/remote/vmksft-p.py > > +++ b/contest/remote/vmksft-p.py > > @@ -73,7 +73,7 @@ group3 testV skip > > tests = [] > > nested_tests = False > > > > - result_re = re.compile(r"(not )?ok (\d+)( -)? ([^#]*[^ ])( # )?([^ ].*)?$") > > + result_re = re.compile(r"(not )?ok (\d+)( -)? ([^#]*[^ ])( +# )?([^ ].*)?$") > > Looks good to me. While at it, we can add a '+' for the spaces after the > '#': > > ( +# +)
Hi Jakub, Thank you for your reply! On 04/09/2024 21:40, Jakub Kicinski wrote: > On Wed, 4 Sep 2024 18:15:09 +0200 Matthieu Baerts wrote: >>> Best I could come up with is: >>> >>> diff --git a/contest/remote/vmksft-p.py b/contest/remote/vmksft-p.py >>> index fe9e87abdb5c..a37245bd5b30 100755 >>> --- a/contest/remote/vmksft-p.py >>> +++ b/contest/remote/vmksft-p.py >>> @@ -73,7 +73,7 @@ group3 testV skip >>> tests = [] >>> nested_tests = False >>> >>> - result_re = re.compile(r"(not )?ok (\d+)( -)? ([^#]*[^ ])( # )?([^ ].*)?$") >>> + result_re = re.compile(r"(not )?ok (\d+)( -)? ([^#]*[^ ])( +# )?([^ ].*)?$") >> >> Looks good to me. While at it, we can add a '+' for the spaces after the >> '#': >> >> ( +# +) > >
Patches here add 'time=<N>ms' in the diagnostic data of the TAP output, e.g. ok 1 - pm_netlink: defaults addr list # time=9ms This addition is useful to quickly identify which subtests are taking a longer time than the others, or more than expected. Note that there are no specific formats to follow to show this time according to the TAP 13, TAP 14 and KTAP specifications, but we follow the format being parsed by NIPA [1]. Patch 1 modifies mptcp_lib.sh to add this support to all MPTCP selftests. Patch 2 removes the now duplicated info in mptcp_connect.sh Patch 3 slightly improves the precision of the first subtests in all MPTCP subtests. Link: https://github.com/linux-netdev/nipa/pull/36 Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org> --- Matthieu Baerts (NGI0) (3): selftests: mptcp: lib: add time per subtests in TAP output sefltests: mptcp: connect: remote time in TAP output selftests: mptcp: reset the last TS before the first test tools/testing/selftests/net/mptcp/mptcp_connect.sh | 3 ++- tools/testing/selftests/net/mptcp/mptcp_join.sh | 3 ++- tools/testing/selftests/net/mptcp/mptcp_lib.sh | 17 ++++++++++++++++- tools/testing/selftests/net/mptcp/mptcp_sockopt.sh | 1 + tools/testing/selftests/net/mptcp/pm_netlink.sh | 2 ++ tools/testing/selftests/net/mptcp/simult_flows.sh | 1 + tools/testing/selftests/net/mptcp/userspace_pm.sh | 1 + 7 files changed, 25 insertions(+), 3 deletions(-) --- base-commit: 221f9cce949ac8042f65b71ed1fde13b99073256 change-id: 20240902-net-next-mptcp-ksft-subtest-time-a83cec43d894 Best regards,