| Message ID | 29e811befea5e751f938e3bf46ca870ec214d53d.1679569719.git.dcaratti@redhat.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | net/sched: act_tunnel_key: add support for TUNNEL_DONT_FRAGMENT | expand |
On 23/03/2023 10:34, Davide Caratti wrote: > currently, users can skip individual test cases by means of writing > > "skip": "yes" > > in the scenario file. Extend this functionality by allowing the execution > of a command, written in the "skip" property for a specific test case. If > such property is present, tdc executes that command and skips the test if > the return value is non-zero. > > Signed-off-by: Davide Caratti <dcaratti@redhat.com> I saw the use case in patch 3 but I didn't understand how it can happen. Shouldn't iproute2 at least match the kernel version? I know it's not a hard requirement for 99% of use cases, but when running tdc I would argue it's the minimum expected.
hello Pedro, thanks for looking at this! On Thu, Mar 23, 2023 at 11:01:53AM -0300, Pedro Tammela wrote: > On 23/03/2023 10:34, Davide Caratti wrote: > > currently, users can skip individual test cases by means of writing > > > > "skip": "yes" > > > > in the scenario file. Extend this functionality by allowing the execution > > of a command, written in the "skip" property for a specific test case. If > > such property is present, tdc executes that command and skips the test if > > the return value is non-zero. > > > > Signed-off-by: Davide Caratti <dcaratti@redhat.com> > > > I saw the use case in patch 3 but I didn't understand how it can happen. > Shouldn't iproute2 at least match the kernel version? I know it's not a hard > requirement for 99% of use cases, but when running tdc I would argue it's > the minimum expected. sure, but there are distributions where patches are backported: on these ones, the kernel/iproute version is not so meaningful. Instead of posting kselftest after the iproute2 support code is merged, I think it's preferrable to just skip those kselftests that can't run because they lack userspace bits; and by the way I see we are already taking this approach elsewhere [1] [2].
On 27/03/2023 07:02, Davide Caratti wrote: > hello Pedro, thanks for looking at this! > > On Thu, Mar 23, 2023 at 11:01:53AM -0300, Pedro Tammela wrote: >> On 23/03/2023 10:34, Davide Caratti wrote: >>> currently, users can skip individual test cases by means of writing >>> >>> "skip": "yes" >>> >>> in the scenario file. Extend this functionality by allowing the execution >>> of a command, written in the "skip" property for a specific test case. If >>> such property is present, tdc executes that command and skips the test if >>> the return value is non-zero. >>> >>> Signed-off-by: Davide Caratti <dcaratti@redhat.com> >> >> >> I saw the use case in patch 3 but I didn't understand how it can happen. >> Shouldn't iproute2 at least match the kernel version? I know it's not a hard >> requirement for 99% of use cases, but when running tdc I would argue it's >> the minimum expected. > > sure, but there are distributions where patches are backported: on these > ones, the kernel/iproute version is not so meaningful. Oh, of course! > Instead of posting kselftest after the iproute2 support code is merged, I > think it's preferrable to just skip those kselftests that can't run because > they lack userspace bits; and by the way I see we are already taking this > approach elsewhere [1] [2]. > I see, so it makes distribution lives easier. Wouldn't it be more clear then to have a separate property called "depends_on" or something similar? If someone adds a new feature that depends on iproute2, then it would be more natural to just add a "depends_on" property. Pedro
diff --git a/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt b/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt index a28571aff0e1..130c49ef8576 100644 --- a/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt +++ b/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt @@ -37,7 +37,9 @@ skip: A completely optional key, if the corresponding value is "yes" then tdc will not execute the test case in question. However, this test case will still appear in the results output but marked as skipped. This key can be placed anywhere inside the - test case at the top level. + test case at the top level. It's possible to specify a command + in the value of "skip": in this case, the test is skipped when + the return value is not zero. category: A list of single-word descriptions covering what the command under test is testing. Example: filter, actions, u32, gact, etc. setup: The list of commands required to ensure the command under test diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py index 7bd94f8e490a..cc355ead1ff0 100755 --- a/tools/testing/selftests/tc-testing/tdc.py +++ b/tools/testing/selftests/tc-testing/tdc.py @@ -361,13 +361,22 @@ def run_one_test(pm, args, index, tidx): print("Test " + tidx["id"] + ": " + tidx["name"]) if 'skip' in tidx: + if (args.verbose > 0): + print('probe command for test skip') if tidx['skip'] == 'yes': - res = TestResult(tidx['id'], tidx['name']) - res.set_result(ResultState.skip) - res.set_errormsg('Test case designated as skipped.') - pm.call_pre_case(tidx, test_skip=True) - pm.call_post_execute() - return res + # 'yes' would block forever: preserve existing skipped test + # replacing 'yes' with 'false' + (p, procout) = exec_cmd(args, pm, 'execute', '/bin/false') + else: + (p, procout) = exec_cmd(args, pm, 'execute', tidx['skip']) + if p: + if (p.returncode != 0): + res = TestResult(tidx['id'], tidx['name']) + res.set_result(ResultState.skip) + res.set_errormsg('probe command failed: test skipped.') + pm.call_pre_case(tidx, test_skip=True) + pm.call_post_execute() + return res # populate NAMES with TESTID for this test NAMES['TESTID'] = tidx['id']
currently, users can skip individual test cases by means of writing "skip": "yes" in the scenario file. Extend this functionality by allowing the execution of a command, written in the "skip" property for a specific test case. If such property is present, tdc executes that command and skips the test if the return value is non-zero. Signed-off-by: Davide Caratti <dcaratti@redhat.com> --- .../creating-testcases/AddingTestCases.txt | 4 +++- tools/testing/selftests/tc-testing/tdc.py | 21 +++++++++++++------ 2 files changed, 18 insertions(+), 7 deletions(-)