From patchwork Thu Dec 7 21:34:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rae Moar X-Patchwork-Id: 13484397 X-Patchwork-Delegate: brendanhiggins@google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="V7RL33fG" Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8AEA13599 for ; Thu, 7 Dec 2023 13:34:18 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-db401df7735so1870915276.3 for ; Thu, 07 Dec 2023 13:34:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701984857; x=1702589657; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=nVvef+BcVgEroNYpsUvlaSbQE1KPpVwrXco8mT3u6R8=; b=V7RL33fGpxz5liHmJfJ19Gv4HVRjyW7C5bIK8yhkT5aETZHjatGv6RZLmeyU17g/qd vUTREXJ9Xjm5AB00QlBhK7xhq9Hyxx0G6EyI+OOJxQj7vjRBVk1qfYYhcO68grMXz0M0 HxGgbwQ9jJgdsPyMk55EzgVCwtYkihg/FzGe2CxZjoJO6JFqGMgwwCIgEo5gNRozsWpU b07gISkkQ87nGOJt120mzrQaI0R99yOZHufCBm4oekrJ3wow8eVhL9JFJ4oJzSlgMYUY P8I7hbY2pTFevjMnQhrROLNafdK9Yy169h+h74QyhJxtHtxmcly4Y1LBJDEe5OHptN5Z 0ReA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701984857; x=1702589657; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=nVvef+BcVgEroNYpsUvlaSbQE1KPpVwrXco8mT3u6R8=; b=uSweBw6wOByl9glnozU40isPn7+KR0Rhvl5Zp+tUw0DXK64LU3Baw0ROZvcrL1lABE yZ8RI41hkN1bKLeoWwak47ur8Y6dPFiVWnevOsFOqahlljqLZ1ZrVTrXLwv5j7pbvMLQ VJeUngCOU9W1EWS7jlzZ3VIEBGYvhX9CXhwtXBOq1G5de+bacje1az4/P46b4w8Mkhn5 VymJusdDvB3NGN9nt0NRN83o8uEf70LgFpZqrQPjyhsZV9zoVn5JkVAXXygYwVUQZzNV EvdCaQHmFtolcYPNk0ETOCaP3RKXOHrjFmJOKEXEtm52hUYejg4mL9MeRRRLHIQ4Aljz /djA== X-Gm-Message-State: AOJu0YwLKM9yapGxvl71wTOO5QMYwYTdNPjYje7cFl8VMCDxYLc8FdSo tjPbAr+k8vNFh7jIN59xSEj6HMvMrg== X-Google-Smtp-Source: AGHT+IFcnTmpIxE1whdzzseWwxE7PYCbkOE0lyizRjIw5inFfT6N6kmdP3S7jk7PUBfYbA1A3Fdf7qTeBA== X-Received: from rmoar-specialist.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:45d3]) (user=rmoar job=sendgmr) by 2002:a25:d7d1:0:b0:db5:4b1d:c63c with SMTP id o200-20020a25d7d1000000b00db54b1dc63cmr35989ybg.11.1701984857742; Thu, 07 Dec 2023 13:34:17 -0800 (PST) Date: Thu, 7 Dec 2023 21:34:09 +0000 Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231207213410.417564-1-rmoar@google.com> Subject: [PATCH v2 1/2] kunit: tool: fix parsing of test attributes From: Rae Moar To: shuah@kernel.org, davidgow@google.com, dlatypov@google.com, brendan.higgins@linux.dev Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, Rae Moar Add parsing of attributes as diagnostic data. Fixes issue with test plan being parsed incorrectly as diagnostic data when located after suite-level attributes. Note that if there does not exist a test plan line, the diagnostic lines between the suite header and the first result will be saved in the suite log rather than the first test case log. Signed-off-by: Rae Moar Reviewed-by: David Gow --- tools/testing/kunit/kunit_parser.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86 diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 79d8832c862a..ce34be15c929 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -450,7 +450,7 @@ def parse_diagnostic(lines: LineStream) -> List[str]: Log of diagnostic lines """ log = [] # type: List[str] - non_diagnostic_lines = [TEST_RESULT, TEST_HEADER, KTAP_START, TAP_START] + non_diagnostic_lines = [TEST_RESULT, TEST_HEADER, KTAP_START, TAP_START, TEST_PLAN] while lines and not any(re.match(lines.peek()) for re in non_diagnostic_lines): log.append(lines.pop()) @@ -726,6 +726,7 @@ def parse_test(lines: LineStream, expected_num: int, log: List[str], is_subtest: # test plan test.name = "main" ktap_line = parse_ktap_header(lines, test) + test.log.extend(parse_diagnostic(lines)) parse_test_plan(lines, test) parent_test = True else: @@ -737,6 +738,7 @@ def parse_test(lines: LineStream, expected_num: int, log: List[str], is_subtest: if parent_test: # If KTAP version line and/or subtest header is found, attempt # to parse test plan and print test header + test.log.extend(parse_diagnostic(lines)) parse_test_plan(lines, test) print_test_header(test) expected_count = test.expected_count From patchwork Thu Dec 7 21:34:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rae Moar X-Patchwork-Id: 13484398 X-Patchwork-Delegate: brendanhiggins@google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="HzZ30J4s" Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C3774784 for ; Thu, 7 Dec 2023 13:34:26 -0800 (PST) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5d67fc68a82so16338167b3.2 for ; Thu, 07 Dec 2023 13:34:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701984865; x=1702589665; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WHU5YaTcNBQsG81gTEGmTL6T2FG7lJRntfdwIVI0JxE=; b=HzZ30J4sxVPo/p8CNdxpezWdqNjSqnjl9rCLm0XyFiDERXJvGlQQB//JvHlDNaftxF 6XwslLMyzJCYdaIc48hKaYeHmLbP/XUW8+z83ay95bj1h9PsXgtVKndT+8zKIMTlI3sp bx3VSCPY+lgGxp7c+zV69q13OiBvJUQ/o7Z/FbjXK5xPfoyGnNloiHH5TAalUaeCNdn5 1VISW6agL5Tyv7wc5qFgbxiHfR6D29maCXuup4+gpx+VRgrMCICJmpoCexywMKDCVtNE bDd+rNQSVka/4HRQz4lBDc/knHyEstKrK9xDLxxz6Xusym+6RQxOrnhbzkjrYcWV+iaw gcPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701984865; x=1702589665; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WHU5YaTcNBQsG81gTEGmTL6T2FG7lJRntfdwIVI0JxE=; b=YcZKJfmTzLlO6Ho3/FKsY2cOctb7zcY1Yvkm7+CcfYMoD+zSAtPqfiWnNY5vSr+KR0 1SigQI7+KvQcTzSPmNCx5gE1+goQaQsrq2RES4AoglKlJ6a7HEboSAGCrhUxZG9Hp1CC E7jmCAonI6oUrboinVuGjQrqsK1fJ3xMa4b/kMhlmL7YQL3Xmv6NZRc2PT+500/DK7br 8aphFhEyGNg0MeG0/Elt8FP+YMlJntrmNf8MwuR6KFJhzK2F7o3P69qPGX5tqjTTXpfb pShk09sOgDc1MPHiViAmXW1caz3CtzUq0+7ekrWJ5xxYss4kMGCluWHZ942WcoXcUsdz F7ag== X-Gm-Message-State: AOJu0YwqVRW8vie/X1avJRkcloj5fE4YUzPlXuhXis66yVprvi034ZFg CHOm0mjG3KrLYnlX73RXpatysAA+0w== X-Google-Smtp-Source: AGHT+IEZKCrsH1xkOF2RQlJeghzypFLltAvtQadwuDk7oRPwDM5B3HQNivo6VPZzxQJSpsr6jXtRX+1MSg== X-Received: from rmoar-specialist.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:45d3]) (user=rmoar job=sendgmr) by 2002:a05:690c:4687:b0:5d7:7d30:595c with SMTP id gx7-20020a05690c468700b005d77d30595cmr41357ywb.9.1701984865678; Thu, 07 Dec 2023 13:34:25 -0800 (PST) Date: Thu, 7 Dec 2023 21:34:10 +0000 In-Reply-To: <20231207213410.417564-1-rmoar@google.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231207213410.417564-1-rmoar@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231207213410.417564-2-rmoar@google.com> Subject: [PATCH v2 2/2] kunit: tool: add test for parsing attributes From: Rae Moar To: shuah@kernel.org, davidgow@google.com, dlatypov@google.com, brendan.higgins@linux.dev Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, Rae Moar Add test for parsing attributes to kunit_tool_test.py. Test checks attributes are parsed and saved in the test logs. This test also checks that the attributes have not interfered with the parsing of other test information, specifically the suite header as the test plan was being incorrectely parsed. Signed-off-by: Rae Moar Reviewed-by: David Gow --- tools/testing/kunit/kunit_tool_test.py | 16 ++++++++++++++++ .../kunit/test_data/test_parse_attributes.log | 9 +++++++++ 2 files changed, 25 insertions(+) create mode 100644 tools/testing/kunit/test_data/test_parse_attributes.log diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index b28c1510be2e..2beb7327e53f 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -331,6 +331,22 @@ class KUnitParserTest(unittest.TestCase): kunit_parser.parse_run_tests(file.readlines()) self.print_mock.assert_any_call(StrContains('suite (1 subtest)')) + def test_parse_attributes(self): + ktap_log = test_data_path('test_parse_attributes.log') + with open(ktap_log) as file: + result = kunit_parser.parse_run_tests(file.readlines()) + + # Test should pass with no errors + self.assertEqual(result.counts, kunit_parser.TestCounts(passed=1, errors=0)) + self.assertEqual(kunit_parser.TestStatus.SUCCESS, result.status) + + # Ensure suite header is parsed correctly + self.print_mock.assert_any_call(StrContains('suite (1 subtest)')) + + # Ensure attributes in correct test log + self.assertContains('# module: example', result.subtests[0].log) + self.assertContains('# test.speed: slow', result.subtests[0].subtests[0].log) + def test_show_test_output_on_failure(self): output = """ KTAP version 1 diff --git a/tools/testing/kunit/test_data/test_parse_attributes.log b/tools/testing/kunit/test_data/test_parse_attributes.log new file mode 100644 index 000000000000..1a13c371fe9d --- /dev/null +++ b/tools/testing/kunit/test_data/test_parse_attributes.log @@ -0,0 +1,9 @@ +KTAP version 1 +1..1 + KTAP version 1 + # Subtest: suite + # module: example + 1..1 + # test.speed: slow + ok 1 test +ok 1 suite \ No newline at end of file