From patchwork Fri Feb 14 01:28:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 13974269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 53A60C021A0 for ; Fri, 14 Feb 2025 01:28:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.888258.1297660 (Exim 4.92) (envelope-from ) id 1tikVT-0005eh-6O; Fri, 14 Feb 2025 01:28:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 888258.1297660; Fri, 14 Feb 2025 01:28:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tikVT-0005e5-23; Fri, 14 Feb 2025 01:28:39 +0000 Received: by outflank-mailman (input) for mailman id 888258; Fri, 14 Feb 2025 01:28:37 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tikVR-0005Nm-06 for xen-devel@lists.xenproject.org; Fri, 14 Feb 2025 01:28:37 +0000 Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 01d02365-ea73-11ef-9896-31a8f345e629; Fri, 14 Feb 2025 02:28:35 +0100 (CET) Received: from phl-compute-12.internal (phl-compute-12.phl.internal [10.202.2.52]) by mailfout.phl.internal (Postfix) with ESMTP id 477201380289; Thu, 13 Feb 2025 20:28:34 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-12.internal (MEProxy); Thu, 13 Feb 2025 20:28:34 -0500 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 13 Feb 2025 20:28:32 -0500 (EST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 01d02365-ea73-11ef-9896-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1739496514; x=1739582914; bh=DC6A2BjUjZ pkR7anGZPjWglSZCWFDKHyJlMNEw2MOAA=; b=gAPuXBZxamzzHozU4vT3HTRAMw k645th90YnJVPR48p1Ge9LyTSnVn5AjbMhvLyWc+7gIgjs0sQXnXFcq0V3peVpQ2 5IoOIzLTJM+xq2HbzasbGvyYcq+qmvIsQUEeoIO4/Sm16DLLt3PZ2GnYSUF4Mswj jMPiZZwrEXDrMfYe8rxGE04GNMiWKLKLIgQH/zg3AJ1686LzIMUbdMCpIUccr940 qYhZMRneQHiImVIyvHUmo5VsPkh/0Bm7yJHsnJmGTTKwrEHzIYDYn1I8X/S0rNMf EocRwqHHeCbSd/nLkOeHpAoDw7RrJj29muJ5WsqqezUvAROeLaNzzrSeKPNQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1739496514; x= 1739582914; bh=DC6A2BjUjZpkR7anGZPjWglSZCWFDKHyJlMNEw2MOAA=; b=u j3ZWJtpSbMVROpEeu6Np+UE56GWs50dRHcAvxdrwQ6iHY+0td6ySWdqPgotQG5xa eoMNj41NF+3+jCPtW7vJGeOst7IRAVSE1jz0rkLI+K1aFt/fW7Zdv+wPdVahFQCz GTNyCekKcOZfmJj7PQ2i9389xx8L7ECuAlCbHbDtnGsDWVEu2LM/mFIdeesKVpic t/SU8ljFkE0MzSlrwakHJnz5M524JN1UojuzzCW3/oxmSRgzQMxP4J+gGyyyUD1Q BYwjhwWKiUEh0nCOAI3ii0bdoIWHUe7hf7M5tvpaIlQ5LELTW9LzEBJxrfX8Jkmd WveTvHI5CvCKoeXT9KsqA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgdegkeeffecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufffkffojghfgggtgfesthekredtredt jeenucfhrhhomhepofgrrhgvkhcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuc eomhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecu ggftrfgrthhtvghrnhepgfeuudehgfdvfeehhedujeehfeduveeugefhkefhheelgeevud etueeiudfggfffnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhf rhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh dpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepgigv nhdquggvvhgvlheslhhishhtshdrgigvnhhprhhojhgvtghtrdhorhhgpdhrtghpthhtoh epmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomhdprhgt phhtthhopegtrghrughovgestggrrhguohgvrdgtohhmpdhrtghpthhtohepshhsthgrsg gvlhhlihhniheskhgvrhhnvghlrdhorhhg X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Doug Goldstein , Stefano Stabellini Subject: [PATCH v2 2/4] automation: add jobs running tests from tools/tests/* Date: Fri, 14 Feb 2025 02:28:08 +0100 Message-ID: X-Mailer: git-send-email 2.48.0 In-Reply-To: References: MIME-Version: 1.0 There are a bunch of tests in tools/tests/, let them run in CI. For each subdirectory expect "make run" will run the test, and observe its exit code. This way, adding new tests is easy, and they will be automatically picked up. For better visibility, log test output to junit xml format, and let gitlab ingest it. Set SUT_ADDR variable with name/address of the system under test, so a network can be used to extract the file. The actual address is set using DHCP. And for the test internal network, still add the 192.168.0.1 IP (but don't replace the DHCP-provided one). Signed-off-by: Marek Marczykowski-Górecki Reviewed-by: Stefano Stabellini --- Changes in v2: - use bash shebang - clarify skipped message - cleanup extra printf params - limit calling DHCP in dom0 to only tests that need it --- automation/gitlab-ci/test.yaml | 23 +++++++++++++++- automation/scripts/build | 1 +- automation/scripts/qubes-x86-64.sh | 28 ++++++++++++++++++- automation/scripts/run-tools-tests | 47 +++++++++++++++++++++++++++++++- 4 files changed, 99 insertions(+) create mode 100755 automation/scripts/run-tools-tests diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml index 1822e3ea5fd7..c21a37933881 100644 --- a/automation/gitlab-ci/test.yaml +++ b/automation/gitlab-ci/test.yaml @@ -130,6 +130,7 @@ PCIDEV: "03:00.0" PCIDEV_INTR: "MSI-X" CONSOLE_OPTS: "console=com1 com1=115200,8n1" + SUT_ADDR: test-2.testnet artifacts: paths: - smoke.serial @@ -263,6 +264,28 @@ adl-pvshim-x86-64-gcc-debug: - *x86-64-test-needs - alpine-3.18-gcc-debug +adl-tools-tests-pv-x86-64-gcc-debug: + extends: .adl-x86-64 + script: + - ./automation/scripts/qubes-x86-64.sh tools-tests-pv 2>&1 | tee ${LOGFILE} + artifacts: + reports: + junit: tests-junit.xml + needs: + - *x86-64-test-needs + - alpine-3.18-gcc-debug + +adl-tools-tests-pvh-x86-64-gcc-debug: + extends: .adl-x86-64 + script: + - ./automation/scripts/qubes-x86-64.sh tools-tests-pvh 2>&1 | tee ${LOGFILE} + artifacts: + reports: + junit: tests-junit.xml + needs: + - *x86-64-test-needs + - alpine-3.18-gcc-debug + zen3p-smoke-x86-64-gcc-debug: extends: .zen3p-x86-64 script: diff --git a/automation/scripts/build b/automation/scripts/build index 952599cc25c2..522efe774ef3 100755 --- a/automation/scripts/build +++ b/automation/scripts/build @@ -109,5 +109,6 @@ else # even though dist/ contains everything, while some containers don't even # build Xen cp -r dist binaries/ + cp -r tools/tests binaries/ collect_xen_artefacts fi diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh index 7eb3ce1bf703..7c80e0c23318 100755 --- a/automation/scripts/qubes-x86-64.sh +++ b/automation/scripts/qubes-x86-64.sh @@ -10,6 +10,8 @@ set -ex # - pci-pv PV dom0, PV domU + PCI Passthrough # - pvshim PV dom0, PVSHIM domU # - s3 PV dom0, S3 suspend/resume +# - tools-tests-pv PV dom0, run tests from tools/tests/* +# - tools-tests-pvh PVH dom0, run tests from tools/tests/* test_variant=$1 ### defaults @@ -19,6 +21,7 @@ timeout=120 domU_type="pvh" domU_vif="'bridge=xenbr0'," domU_extra_config= +retrieve_xml= case "${test_variant}" in ### test: smoke test & smoke test PVH & smoke test HVM & smoke test PVSHIM @@ -126,6 +129,21 @@ done " ;; + ### tests: tools-tests-pv, tools-tests-pvh + "tools-tests-pv"|"tools-tests-pvh") + retrieve_xml=1 + passed="test passed" + domU_check="" + dom0_check=" +/tests/run-tools-tests /tests /tmp/tests-junit.xml && echo \"${passed}\" +nc -l -p 8080 < /tmp/tests-junit.xml >/dev/null & +" + if [ "${test_variant}" = "tools-tests-pvh" ]; then + extra_xen_opts="dom0=pvh" + fi + + ;; + *) echo "Unrecognised test_variant '${test_variant}'" >&2 exit 1 @@ -178,6 +196,8 @@ mkdir srv mkdir sys rm var/run cp -ar ../binaries/dist/install/* . +cp -ar ../binaries/tests . +cp -a ../automation/scripts/run-tools-tests tests/ echo "#!/bin/bash @@ -192,6 +212,10 @@ ifconfig xenbr0 192.168.0.1 " > etc/local.d/xen.start +if [ -n "$retrieve_xml" ]; then + echo "timeout 30s udhcpc -i xenbr0" >> etc/local.d/xen.start +fi + if [ -n "$domU_check" ]; then echo " # get domU console content into test log @@ -272,6 +296,10 @@ if [ $timeout -le 0 ]; then exit 1 fi +if [ -n "$retrieve_xml" ]; then + nc -w 10 "$SUT_ADDR" 8080 > tests-junit.xml \n' > "$xml_out" +printf '\n' >> "$xml_out" +printf ' \n' >> "$xml_out" +failed= +for dir in "$1"/*; do + [ -d "$dir" ] || continue + echo "Running test in $dir" + printf ' \n' "$dir" >> "$xml_out" + ret= + for f in "$dir"/*; do + [ -f "$f" ] || continue + [ -x "$f" ] || continue + "$f" 2>&1 | tee /tmp/out + ret=$? + if [ "$ret" -ne 0 ]; then + echo "FAILED: $ret" + failed+=" $dir" + printf ' \n' "$f" "$ret" >> "$xml_out" + # TODO: could use xml escaping... but current tests seems to + # produce sane output + cat /tmp/out >> "$xml_out" + printf ' \n' >> "$xml_out" + else + echo "PASSED" + fi + done + if [ -z "$ret" ]; then + printf ' \n' "$dir" >> "$xml_out" + fi + printf ' \n' >> "$xml_out" +done +printf ' \n' >> "$xml_out" +printf '\n' >> "$xml_out" + +if [ -n "$failed" ]; then + exit 1 +fi