From patchwork Mon Jun 19 06:56:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangjin Wu X-Patchwork-Id: 13284030 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B489EB64DA for ; Mon, 19 Jun 2023 06:56:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230109AbjFSG4h (ORCPT ); Mon, 19 Jun 2023 02:56:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51592 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230095AbjFSG4g (ORCPT ); Mon, 19 Jun 2023 02:56:36 -0400 Received: from bg4.exmail.qq.com (bg4.exmail.qq.com [43.154.54.12]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8AC013D; Sun, 18 Jun 2023 23:56:34 -0700 (PDT) X-QQ-mid: bizesmtp69t1687157784tkcpo91n Received: from linux-lab-host.localdomain ( [116.30.126.60]) by bizesmtp.qq.com (ESMTP) with id ; Mon, 19 Jun 2023 14:56:23 +0800 (CST) X-QQ-SSF: 01200000000000D0V000000A0000000 X-QQ-FEAT: 5q30pvLz2ifOL7+eKkGV+l+xNjgAcXo7/um90KtQZbwA/x9c6K+kVT3CD9yTU 3Ob2lb03K+MuRZrzURo0i1xFpf1vU1LTevUHfFqRpCwWRGD0eIuTYiXqF/P3Ks3VQdyMgtJ c2pjERcLylH93SwqS70SlZ6Ab+BArIU2bblM2vTaW8mRG2mnPDGk+8T75SMT2mPK02g4aJr y7B9MuJaMsWw6q1mgOJl9sV0I+DySYEFkkkazGClZb6piKGGkHKuJRb36h76tOSq1TKbInd CJqO7AkYKMkbaW5cPKe4y76m4ZhAQuTiQc6yTX23eKND+7aFAQMWABXvEf2KA7ZoF9dKewk DSn942ARQjxWtCABXREZdc1QQXI0f52WOYL6k4Ex2Y5IQC4RRc7+wtlpylCCHxwVwe3HLti X-QQ-GoodBg: 0 X-BIZMAIL-ID: 14674773114149374089 From: Zhangjin Wu To: w@1wt.eu Cc: arnd@arndb.de, falcon@tinylab.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, thomas@t-8ch.de Subject: [PATCH v2 1/3] selftests/nolibc: add a standalone test report macro Date: Mon, 19 Jun 2023 14:56:22 +0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:tinylab.org:qybglogicsvrsz:qybglogicsvrsz3a-3 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org The run-user, run and rerun targets use the same test report script, let's add a standalone test report macro for them. This shrinks code lines and simplify the future maintainability. Suggested-by: Willy Tarreau Link: https://lore.kernel.org/lkml/ZIB792FtG6ibOudp@1wt.eu/ Signed-off-by: Zhangjin Wu --- tools/testing/selftests/nolibc/Makefile | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index 1b7b3c82f8ad..262a9f21d1b4 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -84,6 +84,10 @@ CFLAGS ?= -Os -fno-ident -fno-asynchronous-unwind-tables -std=c89 \ $(CFLAGS_$(ARCH)) $(CFLAGS_STACKPROTECTOR) LDFLAGS := -s +REPORT ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++} /\[SKIPPED\][\r]*$$/{s++} \ + END{ printf("%d test(s) passed, %d skipped, %d failed.", p, s, f); \ + if (s+f > 0) printf(" See all results in %s\n", ARGV[1]); else print; }' + help: @echo "Supported targets under selftests/nolibc:" @echo " all call the \"run\" target below" @@ -131,10 +135,7 @@ libc-test: nolibc-test.c # qemu user-land test run-user: nolibc-test $(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || : - $(Q)awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++} /\[SKIPPED\][\r]*$$/{s++} \ - END{ printf("%d test(s) passed, %d skipped, %d failed.", p, s, f); \ - if (s+f > 0) printf(" See all results in %s\n", ARGV[1]); else print; }' \ - $(CURDIR)/run.out + $(Q)$(REPORT) $(CURDIR)/run.out initramfs: nolibc-test $(QUIET_MKDIR)mkdir -p initramfs @@ -150,18 +151,12 @@ kernel: initramfs # run the tests after building the kernel run: kernel $(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out" - $(Q)awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++} /\[SKIPPED\][\r]*$$/{s++} \ - END{ printf("%d test(s) passed, %d skipped, %d failed.", p, s, f); \ - if (s+f > 0) printf(" See all results in %s\n", ARGV[1]); else print; }' \ - $(CURDIR)/run.out + $(Q)$(REPORT) $(CURDIR)/run.out # re-run the tests from an existing kernel rerun: $(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out" - $(Q)awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++} /\[SKIPPED\][\r]*$$/{s++} \ - END{ printf("%d test(s) passed, %d skipped, %d failed.", p, s, f); \ - if (s+f > 0) printf(" See all results in %s\n", ARGV[1]); else print; }' \ - $(CURDIR)/run.out + $(Q)$(REPORT) $(CURDIR)/run.out clean: $(call QUIET_CLEAN, sysroot) From patchwork Mon Jun 19 06:57:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangjin Wu X-Patchwork-Id: 13284031 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F77AEB64D9 for ; Mon, 19 Jun 2023 06:58:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229865AbjFSG6A (ORCPT ); Mon, 19 Jun 2023 02:58:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229456AbjFSG57 (ORCPT ); Mon, 19 Jun 2023 02:57:59 -0400 Received: from bg4.exmail.qq.com (bg4.exmail.qq.com [43.154.54.12]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FA04199; Sun, 18 Jun 2023 23:57:57 -0700 (PDT) X-QQ-mid: bizesmtp67t1687157865t5q4c2gj Received: from linux-lab-host.localdomain ( [116.30.126.60]) by bizesmtp.qq.com (ESMTP) with id ; Mon, 19 Jun 2023 14:57:40 +0800 (CST) X-QQ-SSF: 01200000000000D0V000000A0000000 X-QQ-FEAT: WrDav1T4GStU8H3DZGX1RXmzi+Ch0f4jZiwIz58gPwD/4di9t+5GipiQLAdrC zFPU5hnJEUMGBND3UktoJWcJl2Roazt8JAP3X41k7+DDekVIZZzFrPLjYAzZIXLzHxLJ4Y0 Gego4TJeRVlaDUVBpBKNV1wG2l2JiVD6inr1lSEU0lbPvpdO+d3a4ODSactXWsF9UBIdNQ8 j1+OMKiMSiQ/vmdKZeuUIQEsswocpqG/TC654WGieHVD1HcpYXKthAeg1YeJulZjmTl1F26 drtD1VCACOclpOZ6VzSHKNglidM8bwW2xcNQWHWqL1UDGoBu96J7q5dA96S+377O3vCDSeg WPafxgOkpqc6yT2HDQpekSD/G9O+YnH5jpi0ZeLWzZdI1snDUDIHar9UnZFtw== X-QQ-GoodBg: 0 X-BIZMAIL-ID: 15193459100436199528 From: Zhangjin Wu To: w@1wt.eu Cc: arnd@arndb.de, falcon@tinylab.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, thomas@t-8ch.de Subject: [PATCH v2 2/3] selftests/nolibc: always print the path to test log file Date: Mon, 19 Jun 2023 14:57:34 +0800 Message-Id: <72617f46112e2124c2fc6509c206b51c5a1f6e0a.1687156560.git.falcon@tinylab.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:tinylab.org:qybglogicsvrsz:qybglogicsvrsz3a-3 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Even when there is no failure, developers may be still interested in the test log file, especially, string alignment, duplicated print, kernel message and so forth, so, always print the path to test log file. A new line is added for such a print to avoid annoying people who don't care about it when the test pass completely. Suggested-by: Willy Tarreau Link: https://lore.kernel.org/lkml/ZIB792FtG6ibOudp@1wt.eu/ Signed-off-by: Zhangjin Wu --- tools/testing/selftests/nolibc/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index 262a9f21d1b4..2a0c3f4fa204 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -85,8 +85,8 @@ CFLAGS ?= -Os -fno-ident -fno-asynchronous-unwind-tables -std=c89 \ LDFLAGS := -s REPORT ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++} /\[SKIPPED\][\r]*$$/{s++} \ - END{ printf("%d test(s) passed, %d skipped, %d failed.", p, s, f); \ - if (s+f > 0) printf(" See all results in %s\n", ARGV[1]); else print; }' + END{ printf("%d test(s) passed, %d skipped, %d failed.\n", p, s, f); \ + printf("See all results in %s\n", ARGV[1]); }' help: @echo "Supported targets under selftests/nolibc:" From patchwork Mon Jun 19 07:01:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangjin Wu X-Patchwork-Id: 13284035 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67EDCEB64D9 for ; Mon, 19 Jun 2023 07:02:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229679AbjFSHB7 (ORCPT ); Mon, 19 Jun 2023 03:01:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229481AbjFSHB6 (ORCPT ); Mon, 19 Jun 2023 03:01:58 -0400 Received: from bg4.exmail.qq.com (bg4.exmail.qq.com [43.155.65.254]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C935199; Mon, 19 Jun 2023 00:01:56 -0700 (PDT) X-QQ-mid: bizesmtp72t1687158107t5cb9jfm Received: from linux-lab-host.localdomain ( [116.30.126.60]) by bizesmtp.qq.com (ESMTP) with id ; Mon, 19 Jun 2023 15:01:46 +0800 (CST) X-QQ-SSF: 01200000000000D0V000000A0000000 X-QQ-FEAT: CR3LFp2JE4mpAf5y1pCIlLMASGgjd5QRvS1f5Cg7IOlRx33kDFXmn3SGmbr8F R0QnHMLBWrZfc3TUdNvkA1afJtX+UfDS7aJyC8WmWi2tAJAhcxy3n54BhSJ/8MUDbXR/LDQ FP0D/l2lQeg6Lw3JvVTdaZFG/291CdvS3v9x2brgznApOdTqVLFtPTt53mV2PoFnDzQZYBZ LHfbdqgtObhhnPGY4uFTmrLG/4GGfArk1fMRX3fqRsAfIhMlpyw31fvj4JQJ2GFxKWu/Iqo 6m3jWiIjnU5+GxeaaJWz1CnCBEAPowXv6+mqU2ONi905qP6wHKf+8ndeeY/j/yQzKqXJxTz 5A0qz9BCIOZEtXmKYuvjTJUzalh9bVP8WUO6cWAPmx4Fm/+LAo= X-QQ-GoodBg: 0 X-BIZMAIL-ID: 7841859380190966851 From: Zhangjin Wu To: w@1wt.eu Cc: arnd@arndb.de, falcon@tinylab.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, thomas@t-8ch.de Subject: [PATCH v2 3/3] selftests/nolibc: restore the failed tests print Date: Mon, 19 Jun 2023 15:01:43 +0800 Message-Id: <2308942c812f5d45035457d2bf6b388b4736c9f5.1687156560.git.falcon@tinylab.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:tinylab.org:qybglogicsvrsz:qybglogicsvrsz3a-3 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org The commit fa0df56a804b ("selftests/nolibc: also count skipped and failed tests in output") added counting for the skipped and failed tests, but also removed the 'FAIL' results print, let's restore it for it really allow users to learn the failed details without opening the log file. Signed-off-by: Zhangjin Wu --- tools/testing/selftests/nolibc/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index 2a0c3f4fa204..000621f21adc 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -84,7 +84,7 @@ CFLAGS ?= -Os -fno-ident -fno-asynchronous-unwind-tables -std=c89 \ $(CFLAGS_$(ARCH)) $(CFLAGS_STACKPROTECTOR) LDFLAGS := -s -REPORT ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++} /\[SKIPPED\][\r]*$$/{s++} \ +REPORT ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++;print} /\[SKIPPED\][\r]*$$/{s++} \ END{ printf("%d test(s) passed, %d skipped, %d failed.\n", p, s, f); \ printf("See all results in %s\n", ARGV[1]); }'