From patchwork Fri Feb 28 01:20:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 11411345 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E64614B4 for ; Fri, 28 Feb 2020 01:21:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1A552246B5 for ; Fri, 28 Feb 2020 01:21:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VXCVjRFV" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730528AbgB1BU7 (ORCPT ); Thu, 27 Feb 2020 20:20:59 -0500 Received: from mail-pl1-f202.google.com ([209.85.214.202]:35328 "EHLO mail-pl1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730464AbgB1BU7 (ORCPT ); Thu, 27 Feb 2020 20:20:59 -0500 Received: by mail-pl1-f202.google.com with SMTP id v24so741466plo.2 for ; Thu, 27 Feb 2020 17:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=gfPIObcjKoc/X97Hku0RqtasQR2kVC34gv6DKj0xOtc=; b=VXCVjRFV1YqUz/8wn7l00lvSHg2xQqUpVRtpFdlra5OC8OF0q0/3b0I8oE3nzS0B4c tupmvt08U9uuorUzoZ1eYDtNg1U63jqDJ0ibGa1taWf7raFaAzoV4BPv3Tg2m4KnmZov V6tpQRa0TMh7UERsCUCTWUWO8SYLoaBeDSCw/OQ+60WXxiMfaBmIcZRNfSDrrnjURMK6 m5775Jw2akrEorxQ6H656cKfbZOHEhm9xqa5dDLNBIKMPYG/okbS3dxAbLcir1Asizeo EmLpGy2DdoKfflEyZLFp6qL14aA0ICqAugWB/JNN5PCwqLQlbpj/EvCmMHCfEdUzY9iN cWlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=gfPIObcjKoc/X97Hku0RqtasQR2kVC34gv6DKj0xOtc=; b=VFOk2FdDNrPYSoQIpCpBMhr3vHxPj0T6bryudx03/yhvt8J9Kaw8AD7dqiX2He+hMB dYanatFOoLATQ35BqY13BVTn3/lxCoaQopcP7HBJxo73LkVKatu2zftrvQGx7HmYh1Je i8mt5nguRFuyvD0V6AjuUNu2k+DDctK/7/HQc29vGMtlQlEW/UxPsLZ1q0k3oE0G2Nvu zB18BzwAVhnIWhENRF2miuqog8Lb1+qq3HT1t6h/pvbW88yijUdihMERxG9GXYGuYzxd tinsCUZD4YlnTe//53S6eEULrkoHrdFhBuzldkEeSH5LzR02nblXpYo6oyojoc1dJ7Xr nEiQ== X-Gm-Message-State: APjAAAVy1xXqI3j67vACmOv7g4VRcRFmObhhFTOmxn5xUgyTEs6ooSGZ GcaZ0BRHOisB6PnVc6bW85Y9ekps/3IZLBAcuzpAMA== X-Google-Smtp-Source: APXvYqypNGxK7wTo1FTf1mIFjhr0aXSAgo4WiptbW+IYOxodMeIGcqOB36P6zCIg73RrYOnWUywb6RzPCsqA6rNjpwfVnA== X-Received: by 2002:a63:7f17:: with SMTP id a23mr2129453pgd.47.1582852857820; Thu, 27 Feb 2020 17:20:57 -0800 (PST) Date: Thu, 27 Feb 2020 17:20:30 -0800 In-Reply-To: <20200228012036.15682-1-brendanhiggins@google.com> Message-Id: <20200228012036.15682-2-brendanhiggins@google.com> Mime-Version: 1.0 References: <20200228012036.15682-1-brendanhiggins@google.com> X-Mailer: git-send-email 2.25.1.481.gfbce0eb801-goog Subject: [PATCH v3 1/7] vmlinux.lds.h: add linker section for KUnit test suites From: Brendan Higgins To: jdike@addtoit.com, richard@nod.at, anton.ivanov@cambridgegreys.com, arnd@arndb.de, keescook@chromium.org, skhan@linuxfoundation.org, alan.maguire@oracle.com, yzaikin@google.com, davidgow@google.com, akpm@linux-foundation.org, rppt@linux.ibm.com, frowand.list@gmail.com Cc: gregkh@linuxfoundation.org, sboyd@kernel.org, logang@deltatee.com, mcgrof@kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Brendan Higgins Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Add a linker section where KUnit can put references to its test suites. This patch is the first step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate late_initcall. Co-developed-by: Iurii Zaikin Signed-off-by: Iurii Zaikin Signed-off-by: Brendan Higgins Reviewed-by: Stephen Boyd --- include/asm-generic/vmlinux.lds.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index e00f41aa8ec4f..99a866f49cb3d 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -856,6 +856,13 @@ KEEP(*(.con_initcall.init)) \ __con_initcall_end = .; +/* Alignment must be consistent with (kunit_suite *) in include/kunit/test.h */ +#define KUNIT_TEST_SUITES \ + . = ALIGN(8); \ + __kunit_suites_start = .; \ + KEEP(*(.kunit_test_suites)) \ + __kunit_suites_end = .; + #ifdef CONFIG_BLK_DEV_INITRD #define INIT_RAM_FS \ . = ALIGN(4); \ @@ -1024,6 +1031,7 @@ INIT_CALLS \ CON_INITCALL \ INIT_RAM_FS \ + KUNIT_TEST_SUITES \ } #define BSS_SECTION(sbss_align, bss_align, stop_align) \ From patchwork Fri Feb 28 01:20:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 11411357 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 766B017E0 for ; Fri, 28 Feb 2020 01:21:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 556CA246BA for ; Fri, 28 Feb 2020 01:21:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KfovGAA3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730561AbgB1BVC (ORCPT ); Thu, 27 Feb 2020 20:21:02 -0500 Received: from mail-pj1-f74.google.com ([209.85.216.74]:40861 "EHLO mail-pj1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730570AbgB1BVC (ORCPT ); Thu, 27 Feb 2020 20:21:02 -0500 Received: by mail-pj1-f74.google.com with SMTP id ev1so739579pjb.5 for ; Thu, 27 Feb 2020 17:21:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ojKrxl1PCfCfQcXP0dlp5ruxQqqYc6a397dYZVnYqp0=; b=KfovGAA3hYngusOc4O801/OpozFq5bk3pBM09Fm0f+xMWm65rYFn4FPokdCHh6Jzut AwQcPV1GQbZQD7YtUQoxvGCQfdMILr/M4f2QE8tKCBkg22l22HKEkPDNmbkuME12auqg VdZRBRlAz9i29ZkqcHN3n9KttVbxKzJodRc0++/lm2FzahO+rKn2n3UmH7on8WoaWvRz Wzbp77tnPo92vJ86YgtIYW1gOwC/AuuB/hNb2KcuQPq5G7ssUIsDyt90aUWsJ4mhLbke /KvsVTb3oL7y2QS9JDSB96JlcRWzWVCOmiNmDItvhmZT/zebT29KmKHbQRgIKCKBsBgw vLbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ojKrxl1PCfCfQcXP0dlp5ruxQqqYc6a397dYZVnYqp0=; b=cuCGmpwkc1uJzNqhd0HZPx3G74jpAQOBX0YUkkRz+nisJ8Pz960Y0nCYaOnfg/NCT7 rr7lqSOZQ/sHXEI+TtuJgAA3a5dndCRqgh3B1P9Ctu4kGwVtJGLZlfPUm6tasq9v4GQt vZ8MVaz/c+kzeTS9eqzfoSznWLtf2NHDg4zrSctf9jomMOoWib4yAYCYE/UE3wmvMKNE Aj4Zp76jgoBMc393UpXMKFBY9D4k5nUeLi02ZIVwP74xRR7mZ8FIBipfHRrzfO7uk3eE ySyaHSl5wSdzE3QxuTU8cNpeeWmQZcjdQvL2Rb4Mw/pKQ45AVwSEHRPQblYhkkLjxMQN V7OQ== X-Gm-Message-State: APjAAAVspMm3sRVlErv2otIgm+IBQNyQzOVjOLGLaYX/JzkvDAp3SA6O M4vNNQEzZiPZRdH+Y9TnZ0zBqgV5kiG8fiU8W1r5sA== X-Google-Smtp-Source: APXvYqwkRljuIrlXDjEYNBekD9KkQQTbA5MI2lo52Sc84mhjN4+wkeOgkPRnjdKBCR1laBgPCE3wql+OCRfkujQNefCpvQ== X-Received: by 2002:a63:ba59:: with SMTP id l25mr2147590pgu.68.1582852860531; Thu, 27 Feb 2020 17:21:00 -0800 (PST) Date: Thu, 27 Feb 2020 17:20:31 -0800 In-Reply-To: <20200228012036.15682-1-brendanhiggins@google.com> Message-Id: <20200228012036.15682-3-brendanhiggins@google.com> Mime-Version: 1.0 References: <20200228012036.15682-1-brendanhiggins@google.com> X-Mailer: git-send-email 2.25.1.481.gfbce0eb801-goog Subject: [PATCH v3 2/7] arch: um: add linker section for KUnit test suites From: Brendan Higgins To: jdike@addtoit.com, richard@nod.at, anton.ivanov@cambridgegreys.com, arnd@arndb.de, keescook@chromium.org, skhan@linuxfoundation.org, alan.maguire@oracle.com, yzaikin@google.com, davidgow@google.com, akpm@linux-foundation.org, rppt@linux.ibm.com, frowand.list@gmail.com Cc: gregkh@linuxfoundation.org, sboyd@kernel.org, logang@deltatee.com, mcgrof@kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Brendan Higgins Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Add a linker section to UML where KUnit can put references to its test suites. This patch is an early step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate late_initcall. Signed-off-by: Brendan Higgins Reviewed-by: Stephen Boyd --- arch/um/include/asm/common.lds.S | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S index eca6c452a41bd..9a9c97f45694c 100644 --- a/arch/um/include/asm/common.lds.S +++ b/arch/um/include/asm/common.lds.S @@ -52,6 +52,10 @@ CON_INITCALL } + .kunit_test_suites : { + KUNIT_TEST_SUITES + } + .exitcall : { __exitcall_begin = .; *(.exitcall.exit) From patchwork Fri Feb 28 01:20:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 11411347 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 94EA814B4 for ; Fri, 28 Feb 2020 01:21:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6B3C2246A5 for ; Fri, 28 Feb 2020 01:21:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SsFE3blu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730569AbgB1BVE (ORCPT ); Thu, 27 Feb 2020 20:21:04 -0500 Received: from mail-pf1-f202.google.com ([209.85.210.202]:34453 "EHLO mail-pf1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730607AbgB1BVE (ORCPT ); Thu, 27 Feb 2020 20:21:04 -0500 Received: by mail-pf1-f202.google.com with SMTP id q5so860030pfh.1 for ; Thu, 27 Feb 2020 17:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=cCVLhSS60CzKiYZGYxwI2VbhU8XZU4Hf40vQtWu8iAk=; b=SsFE3blufo8LdYviSELXvsTvM0T6Qql+JfE+gkcmFyVuWc/HK2V3Z7XsGX4KEB/xGu xF59/eGpLUZQOdfWvYMqc3ccggH/2yYaHyf3g/sXX/diSyppIas/GkB3hMmyZhjECcu+ 8aUNyxXfrV/9RSYvHyTO5T4UPHAGKQnrWxy8MBPrRR9UVhhj0DNQZaJ90yR5w0gw5m5o LNyNouDqONnB2LSjpO+g5d5cqKl2bVrTr4HJJKVhqrkY6J/N2hr56T5+3t9oH1Lz08Bf qjNRhmQKl4xKp+AkDQOdtnpIe9TlrjN1rXVeeLvhm0MbwfnJzPwTfUNRQhkumyPabYOy OfRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=cCVLhSS60CzKiYZGYxwI2VbhU8XZU4Hf40vQtWu8iAk=; b=i8tJnr6MWimwvC8f71U2kiLEeXa6NErr77TY82+4XDXp/mOA0Ix5gPzwDY93jUV1vi fY9+Oo14ON3c4QSefXh0TBAv1LIE5UwU0WTWnegsoZ1upw1lnZd/ftwVd4+QqXesMtDw yMk+NDsuhvY5uTpdz9ZpegFGzS9ygSe3qIZNk8get9m7/ZvvhfulssYvKPVMU+bus8A+ 5fJD7+ycOnETVIzvUKsGCJfo4GPPNrou7of9npTwTxswhQXPNgn4/y+RdcFyt091TL4F xwo9rU5niL3/Vx/OBebEVQtjkphJ14gmZl/lN8TUDnJWgeW4uo6604t9yaehadmV6SC/ RnbQ== X-Gm-Message-State: APjAAAWc1yixYdAAcD/k629dx6pltCCwBEf8YlXIQto3j/TtVWP8RamZ Ob1uPQo/aJ6nSI3LgM9gl79FpOuw+Fw7zxSH0TEitQ== X-Google-Smtp-Source: APXvYqxhn7x3I6Lzn0KIR8NV/0MZifW5WyJ1YKNwC3WuljjsuRvky6R0M5dahP7sVjviK1tTsWoEXisU7yBpZUSY2pYwyg== X-Received: by 2002:a63:2a06:: with SMTP id q6mr2058006pgq.329.1582852863040; Thu, 27 Feb 2020 17:21:03 -0800 (PST) Date: Thu, 27 Feb 2020 17:20:32 -0800 In-Reply-To: <20200228012036.15682-1-brendanhiggins@google.com> Message-Id: <20200228012036.15682-4-brendanhiggins@google.com> Mime-Version: 1.0 References: <20200228012036.15682-1-brendanhiggins@google.com> X-Mailer: git-send-email 2.25.1.481.gfbce0eb801-goog Subject: [PATCH v3 3/7] kunit: test: create a single centralized executor for all tests From: Brendan Higgins To: jdike@addtoit.com, richard@nod.at, anton.ivanov@cambridgegreys.com, arnd@arndb.de, keescook@chromium.org, skhan@linuxfoundation.org, alan.maguire@oracle.com, yzaikin@google.com, davidgow@google.com, akpm@linux-foundation.org, rppt@linux.ibm.com, frowand.list@gmail.com Cc: gregkh@linuxfoundation.org, sboyd@kernel.org, logang@deltatee.com, mcgrof@kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Brendan Higgins Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org From: Alan Maguire Add a centralized executor to dispatch tests rather than relying on late_initcall to schedule each test suite separately. Centralized execution is for built-in tests only; modules will execute tests when loaded. Signed-off-by: Alan Maguire Co-developed-by: Iurii Zaikin Signed-off-by: Iurii Zaikin Co-developed-by: Brendan Higgins Signed-off-by: Brendan Higgins Reviewed-by: Stephen Boyd --- include/kunit/test.h | 73 +++++++++++++++++++++++++++----------------- lib/kunit/Makefile | 3 +- lib/kunit/executor.c | 36 ++++++++++++++++++++++ 3 files changed, 83 insertions(+), 29 deletions(-) create mode 100644 lib/kunit/executor.c diff --git a/include/kunit/test.h b/include/kunit/test.h index 2dfb550c6723a..8a02f93a6b505 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -197,46 +197,63 @@ void kunit_init_test(struct kunit *test, const char *name); int kunit_run_tests(struct kunit_suite *suite); -/** - * kunit_test_suites() - used to register one or more &struct kunit_suite - * with KUnit. - * - * @suites: a statically allocated list of &struct kunit_suite. - * - * Registers @suites with the test framework. See &struct kunit_suite for - * more information. - * - * When builtin, KUnit tests are all run as late_initcalls; this means - * that they cannot test anything where tests must run at a different init - * phase. One significant restriction resulting from this is that KUnit - * cannot reliably test anything that is initialize in the late_init phase; - * another is that KUnit is useless to test things that need to be run in - * an earlier init phase. - * - * An alternative is to build the tests as a module. Because modules - * do not support multiple late_initcall()s, we need to initialize an - * array of suites for a module. - * - * TODO(brendanhiggins@google.com): Don't run all KUnit tests as - * late_initcalls. I have some future work planned to dispatch all KUnit - * tests from the same place, and at the very least to do so after - * everything else is definitely initialized. +/* + * If a test suite is built-in, module_init() gets translated into + * an initcall which we don't want as the idea is that for builtins + * the executor will manage execution. So ensure we do not define + * module_{init|exit} functions for the builtin case when registering + * suites via kunit_test_suites() below. */ -#define kunit_test_suites(...) \ - static struct kunit_suite *suites[] = { __VA_ARGS__, NULL}; \ - static int kunit_test_suites_init(void) \ +#ifdef MODULE +#define kunit_test_suites_for_module(__suites) \ + static int __init kunit_test_suites_init(void) \ { \ + struct kunit_suite *suites[] = (__suites); \ unsigned int i; \ + \ for (i = 0; suites[i] != NULL; i++) \ kunit_run_tests(suites[i]); \ return 0; \ } \ - late_initcall(kunit_test_suites_init); \ + module_init(kunit_test_suites_init); \ + \ static void __exit kunit_test_suites_exit(void) \ { \ return; \ } \ module_exit(kunit_test_suites_exit) +#else +#define kunit_test_suites_for_module(__suites) +#endif /* MODULE */ + +#define __kunit_test_suites(unique_array, unique_suites, ...) \ + static struct kunit_suite *unique_array[] = { __VA_ARGS__, NULL }; \ + kunit_test_suites_for_module(unique_array); \ + static struct kunit_suite **unique_suites \ + __used __aligned(8) __section(.kunit_test_suites) = unique_array + +/** + * kunit_test_suites() - used to register one or more &struct kunit_suite + * with KUnit. + * + * @suites: a statically allocated list of &struct kunit_suite. + * + * Registers @suites with the test framework. See &struct kunit_suite for + * more information. + * + * When builtin, KUnit tests are all run via executor; this is done + * by placing the array of struct kunit_suite * in the .kunit_test_suites + * ELF section. + * + * An alternative is to build the tests as a module. Because modules do not + * support multiple initcall()s, we need to initialize an array of suites for a + * module. + * + */ +#define kunit_test_suites(...) \ + __kunit_test_suites(__UNIQUE_ID(array), \ + __UNIQUE_ID(suites), \ + __VA_ARGS__) #define kunit_test_suite(suite) kunit_test_suites(&suite) diff --git a/lib/kunit/Makefile b/lib/kunit/Makefile index fab55649b69a5..c282f02ca066b 100644 --- a/lib/kunit/Makefile +++ b/lib/kunit/Makefile @@ -3,7 +3,8 @@ obj-$(CONFIG_KUNIT) += kunit.o kunit-objs += test.o \ string-stream.o \ assert.o \ - try-catch.o + try-catch.o \ + executor.o obj-$(CONFIG_KUNIT_TEST) += kunit-test.o diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c new file mode 100644 index 0000000000000..6429927d598a5 --- /dev/null +++ b/lib/kunit/executor.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include + +/* + * These symbols point to the .kunit_test_suites section and are defined in + * include/asm-generic/vmlinux.lds.h, and consequently must be extern. + */ +extern struct kunit_suite * const * const __kunit_suites_start[]; +extern struct kunit_suite * const * const __kunit_suites_end[]; + +#if IS_BUILTIN(CONFIG_KUNIT) + +static int kunit_run_all_tests(void) +{ + struct kunit_suite * const * const *suites, * const *subsuite; + bool has_test_failed = false; + + for (suites = __kunit_suites_start; + suites < __kunit_suites_end; + suites++) { + for (subsuite = *suites; *subsuite != NULL; subsuite++) { + if (kunit_run_tests(*subsuite)) + has_test_failed = true; + } + } + + if (has_test_failed) + return -EFAULT; + + return 0; +} + +late_initcall(kunit_run_all_tests); + +#endif /* IS_BUILTIN(CONFIG_KUNIT) */ From patchwork Fri Feb 28 01:20:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 11411349 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3203992A for ; Fri, 28 Feb 2020 01:21:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 10E96246A5 for ; Fri, 28 Feb 2020 01:21:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Bty/Q2oV" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730617AbgB1BVH (ORCPT ); Thu, 27 Feb 2020 20:21:07 -0500 Received: from mail-pl1-f202.google.com ([209.85.214.202]:50894 "EHLO mail-pl1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730632AbgB1BVG (ORCPT ); Thu, 27 Feb 2020 20:21:06 -0500 Received: by mail-pl1-f202.google.com with SMTP id g5so716328plq.17 for ; Thu, 27 Feb 2020 17:21:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=8lHd0YZkwZG2tzVOdxbzliRdb15FJbxx1Rjn1yB5+70=; b=Bty/Q2oVTN47foIGrI34kAsL9y3zp5FC92Tuh2vIv90lA66BfzLsxq5CRGdQbuTU/V n9fsZay5lgeSnvTqrjeQFbuDLt8lGR8C8QILxlmjfijeVL9WgNaefmZfmv/doN7k5/S8 KDIo01iFyP9kibL/VroPQ24eUBVXE2M+B5HjmVxniWizwDLZv8WhkeLTRcTVx2Dtg/pA ASEMv/Lpre04igKn6rhqAcPph95bj91sQcBUU9tgeRwUFrU2REdGiZETCN6W1ZkhBUr4 XN7JrUYawuanT9NktwcCxIrp78dZRMo5IxQc8AD4YNi8wC1H11QUDW5GjX6zndj+Wze/ 4MGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=8lHd0YZkwZG2tzVOdxbzliRdb15FJbxx1Rjn1yB5+70=; b=gf0FLWYwpMvPQGL553pwLVq+ybqS7sKGS4jDO5wvOaWjrsWG8H2qPx8xL+9IIFzrmX DfGDYYEuZiMPAyjIqb14qoq4n8VbkKWs73doc8ofMd+kL1noTOSH1ZjnJA/S/lgTazww DmnVCyAqPaxfnBxdxzXs9yi+lCUlBpgClEcQgiVBlTrWUjEL0aXd0qqu1nS4tZqbrXlJ UfzCjY1LRVUCacprr3t4MAfX3GE6OADIWY0Ygm9LAktBYL0dLPeEAe705uwFnuVisBYh u8ctj17cvrNoXJNaFmZrxBEaIYBoKzgfz4gnyI9e7xbUXGKVqj3qfKmVhKDj1VXzxNaa Cfvw== X-Gm-Message-State: APjAAAX6zC7m+a6RNxQy9VQDz/knvoLw1aalSBiH0N9VwQVns+tTPRUi P0Ar3G7c5SykKelHIUMvYNFRGBuHJOpN2tlwGnHtzQ== X-Google-Smtp-Source: APXvYqwGEoxXjZAGlPCODTL6b1fAO5IM9ZcV9N9/Stf3e2FL1lhptdTQ9Z2499r3LVL2+l69sq+uvpIK6dT577YqJbUzyg== X-Received: by 2002:a63:b949:: with SMTP id v9mr2010452pgo.336.1582852865540; Thu, 27 Feb 2020 17:21:05 -0800 (PST) Date: Thu, 27 Feb 2020 17:20:33 -0800 In-Reply-To: <20200228012036.15682-1-brendanhiggins@google.com> Message-Id: <20200228012036.15682-5-brendanhiggins@google.com> Mime-Version: 1.0 References: <20200228012036.15682-1-brendanhiggins@google.com> X-Mailer: git-send-email 2.25.1.481.gfbce0eb801-goog Subject: [PATCH v3 4/7] init: main: add KUnit to kernel init From: Brendan Higgins To: jdike@addtoit.com, richard@nod.at, anton.ivanov@cambridgegreys.com, arnd@arndb.de, keescook@chromium.org, skhan@linuxfoundation.org, alan.maguire@oracle.com, yzaikin@google.com, davidgow@google.com, akpm@linux-foundation.org, rppt@linux.ibm.com, frowand.list@gmail.com Cc: gregkh@linuxfoundation.org, sboyd@kernel.org, logang@deltatee.com, mcgrof@kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Brendan Higgins Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Remove KUnit from init calls entirely, instead call directly from kernel_init(). Co-developed-by: Alan Maguire Signed-off-by: Alan Maguire Signed-off-by: Brendan Higgins Reviewed-by: Stephen Boyd --- include/kunit/test.h | 9 +++++++++ init/main.c | 4 ++++ lib/kunit/executor.c | 4 +--- 3 files changed, 14 insertions(+), 3 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 8a02f93a6b505..8689dd1459844 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -197,6 +197,15 @@ void kunit_init_test(struct kunit *test, const char *name); int kunit_run_tests(struct kunit_suite *suite); +#if IS_BUILTIN(CONFIG_KUNIT) +int kunit_run_all_tests(void); +#else +static inline int kunit_run_all_tests(void) +{ + return 0; +} +#endif /* IS_BUILTIN(CONFIG_KUNIT) */ + /* * If a test suite is built-in, module_init() gets translated into * an initcall which we don't want as the idea is that for builtins diff --git a/init/main.c b/init/main.c index ee4947af823f3..7875a5c486dc4 100644 --- a/init/main.c +++ b/init/main.c @@ -104,6 +104,8 @@ #define CREATE_TRACE_POINTS #include +#include + static int kernel_init(void *); extern void init_IRQ(void); @@ -1444,6 +1446,8 @@ static noinline void __init kernel_init_freeable(void) do_basic_setup(); + kunit_run_all_tests(); + console_on_rootfs(); /* diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 6429927d598a5..b75a46c560847 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -11,7 +11,7 @@ extern struct kunit_suite * const * const __kunit_suites_end[]; #if IS_BUILTIN(CONFIG_KUNIT) -static int kunit_run_all_tests(void) +int kunit_run_all_tests(void) { struct kunit_suite * const * const *suites, * const *subsuite; bool has_test_failed = false; @@ -31,6 +31,4 @@ static int kunit_run_all_tests(void) return 0; } -late_initcall(kunit_run_all_tests); - #endif /* IS_BUILTIN(CONFIG_KUNIT) */ From patchwork Fri Feb 28 01:20:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 11411353 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A8F917E0 for ; Fri, 28 Feb 2020 01:21:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0100D246AE for ; Fri, 28 Feb 2020 01:21:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kMCnPefE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730668AbgB1BVO (ORCPT ); Thu, 27 Feb 2020 20:21:14 -0500 Received: from mail-yw1-f74.google.com ([209.85.161.74]:34507 "EHLO mail-yw1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730662AbgB1BVL (ORCPT ); Thu, 27 Feb 2020 20:21:11 -0500 Received: by mail-yw1-f74.google.com with SMTP id o1so2595284ywl.1 for ; Thu, 27 Feb 2020 17:21:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PLTBxyerPi61VOCjyv+TPa3h4yiHHMfUoC17UPRMggY=; b=kMCnPefEIl8HdfN8MA1lHtN+Ng5kiZzhHT+ke4XvEl9H8Ygf9GJcTal2XIamCTFTbS wFsa0lQR8cTwJIPlOEeT++5OJHQ7xhpiDP966QIjrWA9fntiRg47uHs9Vg6knR4prMj/ qtqXgOGcpCVTs2QoqM+S5WbbFjuTvAn8ucrCErkYLpXLJrQjEjHL28GLm35KoIn8UKGb uNFOItiShsCq1Yx4+1HqpsgZW/AyY7hs7UeuypvLyzzTHLwlvPBCjFX1fVjI2hkTJhSY 3lyFd1U7IRIlnucQ/rDVMJFVy35ra0IGepGEN56ZPTviRTpxIZNtLFQf8oWTttRwHyCM aJag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PLTBxyerPi61VOCjyv+TPa3h4yiHHMfUoC17UPRMggY=; b=K4czt7zMjGc7LuGNW0J1ggwt9fjQCcKQIptmLjGdPbvKA4aCOAHcsNoYxPgrzl1nQ/ Pbd1L6hwFXUH/gafDN/szfxdHPp+R7SCEspIRzkLB/KjJjzYaz17Y9572l6d9iEnJNaA /HDoL/5232yq4XDgasGkYuOdFPoUkytWHG3oos2zZOysnQHVrtSIgyQsYIsmW7pppCco YXgwEKd/IITM69ETz69rQpQLOIvws5MewMBXx3MG4Xg2Cjh/Eo3H1lMhA/ryLLtvGd9O Q0z8h/Lmg0dr3VhhPiNmKtPwDHqBTwuEIAWFxsVCji9PnJd6cp6MFJ4Ip8zCrkYSGXTW myvQ== X-Gm-Message-State: APjAAAUJxskTbGG+OYQd99qklfHbDvPV5JyXZL55YFJskecsx6t5gQJb L6MaAQqNcdEpskaJw7ctRnKNglMsz9blf47vYaYQ1A== X-Google-Smtp-Source: APXvYqyTcd0alDtSswfI7dznn3gjEf5mlQrgZgLLcbRCPc6A3iDO+DtHytetlICWtdtynHPFuTB9cwf5wBQAb8llZeiE3A== X-Received: by 2002:a0d:cad3:: with SMTP id m202mr2353504ywd.264.1582852868267; Thu, 27 Feb 2020 17:21:08 -0800 (PST) Date: Thu, 27 Feb 2020 17:20:34 -0800 In-Reply-To: <20200228012036.15682-1-brendanhiggins@google.com> Message-Id: <20200228012036.15682-6-brendanhiggins@google.com> Mime-Version: 1.0 References: <20200228012036.15682-1-brendanhiggins@google.com> X-Mailer: git-send-email 2.25.1.481.gfbce0eb801-goog Subject: [PATCH v3 5/7] kunit: test: add test plan to KUnit TAP format From: Brendan Higgins To: jdike@addtoit.com, richard@nod.at, anton.ivanov@cambridgegreys.com, arnd@arndb.de, keescook@chromium.org, skhan@linuxfoundation.org, alan.maguire@oracle.com, yzaikin@google.com, davidgow@google.com, akpm@linux-foundation.org, rppt@linux.ibm.com, frowand.list@gmail.com Cc: gregkh@linuxfoundation.org, sboyd@kernel.org, logang@deltatee.com, mcgrof@kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Brendan Higgins Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org TAP 14 allows an optional test plan to be emitted before the start of the start of testing[1]; this is valuable because it makes it possible for a test harness to detect whether the number of tests run matches the number of tests expected to be run, ensuring that no tests silently failed. Link[1]: https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md#the-plan Signed-off-by: Brendan Higgins Reviewed-by: Stephen Boyd --- lib/kunit/executor.c | 17 +++++ lib/kunit/test.c | 11 --- tools/testing/kunit/kunit_parser.py | 74 ++++++++++++++++--- .../test_is_test_passed-all_passed.log | 1 + .../test_data/test_is_test_passed-crash.log | 1 + .../test_data/test_is_test_passed-failure.log | 1 + 6 files changed, 82 insertions(+), 23 deletions(-) diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index b75a46c560847..7fd16feff157e 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -11,11 +11,28 @@ extern struct kunit_suite * const * const __kunit_suites_end[]; #if IS_BUILTIN(CONFIG_KUNIT) +static void kunit_print_tap_header(void) +{ + struct kunit_suite * const * const *suites, * const *subsuite; + int num_of_suites = 0; + + for (suites = __kunit_suites_start; + suites < __kunit_suites_end; + suites++) + for (subsuite = *suites; *subsuite != NULL; subsuite++) + num_of_suites++; + + pr_info("TAP version 14\n"); + pr_info("1..%d\n", num_of_suites); +} + int kunit_run_all_tests(void) { struct kunit_suite * const * const *suites, * const *subsuite; bool has_test_failed = false; + kunit_print_tap_header(); + for (suites = __kunit_suites_start; suites < __kunit_suites_end; suites++) { diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 9242f932896c7..da56b94261b43 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -18,16 +18,6 @@ static void kunit_set_failure(struct kunit *test) WRITE_ONCE(test->success, false); } -static void kunit_print_tap_version(void) -{ - static bool kunit_has_printed_tap_version; - - if (!kunit_has_printed_tap_version) { - pr_info("TAP version 14\n"); - kunit_has_printed_tap_version = true; - } -} - static size_t kunit_test_cases_len(struct kunit_case *test_cases) { struct kunit_case *test_case; @@ -41,7 +31,6 @@ static size_t kunit_test_cases_len(struct kunit_case *test_cases) static void kunit_print_subtest_start(struct kunit_suite *suite) { - kunit_print_tap_version(); pr_info("\t# Subtest: %s\n", suite->name); pr_info("\t1..%zd\n", kunit_test_cases_len(suite->test_cases)); } diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 4ffbae0f67325..78b3bdd03b1e4 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -45,6 +45,7 @@ class TestStatus(Enum): FAILURE = auto() TEST_CRASHED = auto() NO_TESTS = auto() + FAILURE_TO_PARSE_TESTS = auto() kunit_start_re = re.compile(r'^TAP version [0-9]+$') kunit_end_re = re.compile('List of all partitions:') @@ -106,7 +107,7 @@ OkNotOkResult = namedtuple('OkNotOkResult', ['is_ok','description', 'text']) OK_NOT_OK_SUBTEST = re.compile(r'^\t(ok|not ok) [0-9]+ - (.*)$') -OK_NOT_OK_MODULE = re.compile(r'^(ok|not ok) [0-9]+ - (.*)$') +OK_NOT_OK_MODULE = re.compile(r'^(ok|not ok) ([0-9]+) - (.*)$') def parse_ok_not_ok_test_case(lines: List[str], test_case: TestCase, @@ -196,7 +197,9 @@ def max_status(left: TestStatus, right: TestStatus) -> TestStatus: else: return TestStatus.SUCCESS -def parse_ok_not_ok_test_suite(lines: List[str], test_suite: TestSuite) -> bool: +def parse_ok_not_ok_test_suite(lines: List[str], + test_suite: TestSuite, + expected_suite_index: int) -> bool: consume_non_diagnositic(lines) if not lines: test_suite.status = TestStatus.TEST_CRASHED @@ -209,6 +212,12 @@ def parse_ok_not_ok_test_suite(lines: List[str], test_suite: TestSuite) -> bool: test_suite.status = TestStatus.SUCCESS else: test_suite.status = TestStatus.FAILURE + suite_index = int(match.group(2)) + if suite_index != expected_suite_index: + print_with_timestamp( + red('[ERROR] ') + 'expected_suite_index ' + + str(expected_suite_index) + ', but got ' + + str(suite_index)) return True else: return False @@ -221,7 +230,7 @@ def bubble_up_test_case_errors(test_suite: TestSuite) -> TestStatus: max_test_case_status = bubble_up_errors(lambda x: x.status, test_suite.cases) return max_status(max_test_case_status, test_suite.status) -def parse_test_suite(lines: List[str]) -> TestSuite: +def parse_test_suite(lines: List[str], expected_suite_index: int) -> TestSuite: if not lines: return None consume_non_diagnositic(lines) @@ -240,7 +249,7 @@ def parse_test_suite(lines: List[str]) -> TestSuite: test_suite.cases.append(test_case) test_case = parse_test_case(lines, expected_test_case_num > 0) expected_test_case_num -= 1 - if parse_ok_not_ok_test_suite(lines, test_suite): + if parse_ok_not_ok_test_suite(lines, test_suite, expected_suite_index): test_suite.status = bubble_up_test_case_errors(test_suite) return test_suite elif not lines: @@ -260,6 +269,17 @@ def parse_tap_header(lines: List[str]) -> bool: else: return False +TEST_PLAN = re.compile(r'[0-9]+\.\.([0-9]+)') + +def parse_test_plan(lines: List[str]) -> int: + consume_non_diagnositic(lines) + match = TEST_PLAN.match(lines[0]) + if match: + lines.pop(0) + return int(match.group(1)) + else: + return None + def bubble_up_suite_errors(test_suite_list: List[TestSuite]) -> TestStatus: return bubble_up_errors(lambda x: x.status, test_suite_list) @@ -268,19 +288,34 @@ def parse_test_result(lines: List[str]) -> TestResult: return TestResult(TestStatus.NO_TESTS, [], lines) consume_non_diagnositic(lines) if not parse_tap_header(lines): - return None + return TestResult(TestStatus.NO_TESTS, [], lines) + expected_test_suite_num = parse_test_plan(lines) + if not expected_test_suite_num: + return TestResult(TestStatus.FAILURE_TO_PARSE_TESTS, [], lines) test_suites = [] - test_suite = parse_test_suite(lines) - while test_suite: - test_suites.append(test_suite) - test_suite = parse_test_suite(lines) - return TestResult(bubble_up_suite_errors(test_suites), test_suites, lines) + for i in range(1, expected_test_suite_num + 1): + test_suite = parse_test_suite(lines, i) + if test_suite: + test_suites.append(test_suite) + else: + print_with_timestamp( + red('[ERROR] ') + ' expected ' + + str(expected_test_suite_num) + + ' test suites, but got ' + str(i - 2)) + break + test_suite = parse_test_suite(lines, -1) + if test_suite: + print_with_timestamp(red('[ERROR] ') + + 'got unexpected test suite: ' + test_suite.name) + if test_suites: + return TestResult(bubble_up_suite_errors(test_suites), test_suites, lines) + else: + return TestResult(TestStatus.NO_TESTS, [], lines) -def parse_run_tests(kernel_output) -> TestResult: +def print_and_count_results(test_result: TestResult) -> None: total_tests = 0 failed_tests = 0 crashed_tests = 0 - test_result = parse_test_result(list(isolate_kunit_output(kernel_output))) for test_suite in test_result.suites: if test_suite.status == TestStatus.SUCCESS: print_suite_divider(green('[PASSED] ') + test_suite.name) @@ -302,6 +337,21 @@ def parse_run_tests(kernel_output) -> TestResult: print_with_timestamp(red('[FAILED] ') + test_case.name) print_log(map(yellow, test_case.log)) print_with_timestamp('') + return total_tests, failed_tests, crashed_tests + +def parse_run_tests(kernel_output) -> TestResult: + total_tests = 0 + failed_tests = 0 + crashed_tests = 0 + test_result = parse_test_result(list(isolate_kunit_output(kernel_output))) + if test_result.status == TestStatus.NO_TESTS: + print(red('[ERROR] ') + yellow('no tests run!')) + elif test_result.status == TestStatus.FAILURE_TO_PARSE_TESTS: + print(red('[ERROR] ') + yellow('could not parse test results!')) + else: + (total_tests, + failed_tests, + crashed_tests) = print_and_count_results(test_result) print_with_timestamp(DIVIDER) fmt = green if test_result.status == TestStatus.SUCCESS else red print_with_timestamp( diff --git a/tools/testing/kunit/test_data/test_is_test_passed-all_passed.log b/tools/testing/kunit/test_data/test_is_test_passed-all_passed.log index 62ebc0288355c..bc0dc8fe35b76 100644 --- a/tools/testing/kunit/test_data/test_is_test_passed-all_passed.log +++ b/tools/testing/kunit/test_data/test_is_test_passed-all_passed.log @@ -1,4 +1,5 @@ TAP version 14 +1..2 # Subtest: sysctl_test 1..8 # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed diff --git a/tools/testing/kunit/test_data/test_is_test_passed-crash.log b/tools/testing/kunit/test_data/test_is_test_passed-crash.log index 0b249870c8be4..4d97f6708c4a5 100644 --- a/tools/testing/kunit/test_data/test_is_test_passed-crash.log +++ b/tools/testing/kunit/test_data/test_is_test_passed-crash.log @@ -1,6 +1,7 @@ printk: console [tty0] enabled printk: console [mc-1] enabled TAP version 14 +1..2 # Subtest: sysctl_test 1..8 # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed diff --git a/tools/testing/kunit/test_data/test_is_test_passed-failure.log b/tools/testing/kunit/test_data/test_is_test_passed-failure.log index 9e89d32d5667a..7a416497e3bec 100644 --- a/tools/testing/kunit/test_data/test_is_test_passed-failure.log +++ b/tools/testing/kunit/test_data/test_is_test_passed-failure.log @@ -1,4 +1,5 @@ TAP version 14 +1..2 # Subtest: sysctl_test 1..8 # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed From patchwork Fri Feb 28 01:20:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 11411351 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBD3314B4 for ; Fri, 28 Feb 2020 01:21:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CB755246AE for ; Fri, 28 Feb 2020 01:21:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="aVEeGKe2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730665AbgB1BVO (ORCPT ); Thu, 27 Feb 2020 20:21:14 -0500 Received: from mail-pj1-f73.google.com ([209.85.216.73]:39818 "EHLO mail-pj1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730667AbgB1BVM (ORCPT ); Thu, 27 Feb 2020 20:21:12 -0500 Received: by mail-pj1-f73.google.com with SMTP id kx17so741153pjb.4 for ; Thu, 27 Feb 2020 17:21:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=+RQb0O9/ANDgn7O27sRp5ZtZTpbReSUGwniCuF+Cbp4=; b=aVEeGKe22nuT+AY9XSYnyJt98FnHF4PfCk5p8sv4bJiC+R4AJxjBf/mXc0e0toX5Hu jvDqabQJ/NjX4KfjQQouS3ID09TUMXxfXSIlk5W3/evKbc4aCMG8eunt9sQsa7CKsz0Y jkm/o0h1C3+GMSMQzkbp6YSsrcwLYplOvfDnnDH/ex3TN6z/nKoaJUw5wJDpAPDPci1s q+RSfm7HbtJz4tOLR4sxBuird496AqZ6I8yrJx8BCEztWD//3DR2CZz85nDg5CK40aMW Sq7XNgHjZcqYet6A3zQF7GGH8LNmFN7REhzxNiJ0UKoWyWo+VIoizh9Ukk3X+1mp0T3c eENg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=+RQb0O9/ANDgn7O27sRp5ZtZTpbReSUGwniCuF+Cbp4=; b=fMcnPcCQhcFFk5Xui8tio2T0zW0HbVYTM5+DMnJhR+hzp4Ri1Yy0hXYOEOmJXZXaaI 3CKKS59RsnI5zH3G+6HG5WEe9hR7nPLgIR01kC9UpsmLHKg2BjfR/0vK9bKmmsaqjdEA i9PfdNWU3FYMgb5mHH1ZLSqW9WDjCItPZPm92G8Lk9aDA/WCyY30Sv5MCXccPnnv+gZK op9C1CRBDiu5Nmb4IOzytVYQLD34Eao827xBUPkGKETPAdo2iCzQ0ZDu271wb2khqxP4 1u1ye14UaZ5JUjAtpMdB1pgB40OkW57YLgiucMCP+oN7D4e8WqJgE6dofpYjFhLm1eBb u0BQ== X-Gm-Message-State: APjAAAVuNIoncVMcc0ww2NPXXgfuxe2Tfz1etZsyDGOIvZw4uCGSinTX iBZwL/ZlSRnvHKV+Fs5lQvSPQTu/sFNVd85zIEJRkA== X-Google-Smtp-Source: APXvYqxQ1rOCCg5Gw3xbVsq9larNndZOz8N2rPlyFxHiMX0CbG1fYPY5L0MBjR5rVPDuE7TzH4FU4VdnMiw912h55MWbkQ== X-Received: by 2002:a63:104a:: with SMTP id 10mr2052540pgq.276.1582852870640; Thu, 27 Feb 2020 17:21:10 -0800 (PST) Date: Thu, 27 Feb 2020 17:20:35 -0800 In-Reply-To: <20200228012036.15682-1-brendanhiggins@google.com> Message-Id: <20200228012036.15682-7-brendanhiggins@google.com> Mime-Version: 1.0 References: <20200228012036.15682-1-brendanhiggins@google.com> X-Mailer: git-send-email 2.25.1.481.gfbce0eb801-goog Subject: [PATCH v3 6/7] kunit: Add 'kunit_shutdown' option From: Brendan Higgins To: jdike@addtoit.com, richard@nod.at, anton.ivanov@cambridgegreys.com, arnd@arndb.de, keescook@chromium.org, skhan@linuxfoundation.org, alan.maguire@oracle.com, yzaikin@google.com, davidgow@google.com, akpm@linux-foundation.org, rppt@linux.ibm.com, frowand.list@gmail.com Cc: gregkh@linuxfoundation.org, sboyd@kernel.org, logang@deltatee.com, mcgrof@kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Brendan Higgins Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org From: David Gow Add a new kernel command-line option, 'kunit_shutdown', which allows the user to specify that the kernel poweroff, halt, or reboot after completing all KUnit tests; this is very handy for running KUnit tests on UML or a VM so that the UML/VM process exits cleanly immediately after running all tests without needing a special initramfs. Signed-off-by: David Gow Signed-off-by: Brendan Higgins Reviewed-by: Stephen Boyd --- lib/kunit/executor.c | 20 ++++++++++++++++++++ tools/testing/kunit/kunit_kernel.py | 2 +- tools/testing/kunit/kunit_parser.py | 2 +- 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 7fd16feff157e..a93821116ccec 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 +#include #include /* @@ -11,6 +12,23 @@ extern struct kunit_suite * const * const __kunit_suites_end[]; #if IS_BUILTIN(CONFIG_KUNIT) +static char *kunit_shutdown; +core_param(kunit_shutdown, kunit_shutdown, charp, 0644); + +static void kunit_handle_shutdown(void) +{ + if (!kunit_shutdown) + return; + + if (!strcmp(kunit_shutdown, "poweroff")) + kernel_power_off(); + else if (!strcmp(kunit_shutdown, "halt")) + kernel_halt(); + else if (!strcmp(kunit_shutdown, "reboot")) + kernel_restart(NULL); + +} + static void kunit_print_tap_header(void) { struct kunit_suite * const * const *suites, * const *subsuite; @@ -42,6 +60,8 @@ int kunit_run_all_tests(void) } } + kunit_handle_shutdown(); + if (has_test_failed) return -EFAULT; diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index d99ae75ef72fa..6cf0697c788b6 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -145,7 +145,7 @@ class LinuxSourceTree(object): return self.validate_config(build_dir) def run_kernel(self, args=[], timeout=None, build_dir=''): - args.extend(['mem=256M']) + args.extend(['mem=256M', 'kunit_shutdown=halt']) process = self._ops.linux_bin(args, timeout, build_dir) with open(os.path.join(build_dir, 'test.log'), 'w') as f: for line in process.stdout: diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 78b3bdd03b1e4..633811dd9bce8 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -48,7 +48,7 @@ class TestStatus(Enum): FAILURE_TO_PARSE_TESTS = auto() kunit_start_re = re.compile(r'^TAP version [0-9]+$') -kunit_end_re = re.compile('List of all partitions:') +kunit_end_re = re.compile(r'reboot: System halted') def isolate_kunit_output(kernel_output): started = False From patchwork Fri Feb 28 01:20:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 11411355 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 30F7A92A for ; Fri, 28 Feb 2020 01:21:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 11163246A1 for ; Fri, 28 Feb 2020 01:21:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IH1npzZ+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730714AbgB1BVU (ORCPT ); Thu, 27 Feb 2020 20:21:20 -0500 Received: from mail-yw1-f73.google.com ([209.85.161.73]:34511 "EHLO mail-yw1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730642AbgB1BVO (ORCPT ); Thu, 27 Feb 2020 20:21:14 -0500 Received: by mail-yw1-f73.google.com with SMTP id o1so2595456ywl.1 for ; Thu, 27 Feb 2020 17:21:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MkaS0G1bk+Q5OhBTj99Zlxgrs85NSvS6F5H6SOsXVyU=; b=IH1npzZ+v8JTFD9rsz35zQYEul6R/W7yA2D6y+OHVbHvXqTHNvw94iwhkAAP2DDbVi GBl13bzJPRjEuu19+D47ufplr6qg9wjANWrvanrNpgcqF3wYeITyoFt3EBjPEKAChMee DeaGGyUBsY44m410q88ZWKqK3we31TbcCkm2I/9FYqcTsjtfZrtU59w7sW8UgKR1PAJe CAKr9zLHEDRGuRNAQVHoLYwNfztD6WsUdwPciC+p1QF6ZslfNc/x8Hy7CBCjDGP7F1O7 OMmtpISW28bF/hQBE0TciN5jdd+8PFqiw4ehn8+83YXO6s8MOtXeeqCaCT+kJgk3y0WI ockg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MkaS0G1bk+Q5OhBTj99Zlxgrs85NSvS6F5H6SOsXVyU=; b=FkkUSdW+HiadNcRruBoQ+tgosbOj/isjN8W+lSDlkWYuFZJl4ef6GcYUxnRxxCj/X7 9Rd3FjIpJxJ3+qZQhHe5/5Y+TcV3cgrirwuKgoRqVWibyoQo3SqNepkl7vP5i46DIGfO D3zmhr8fTgD5H5J5w0+AV6fHzrWo1va+U1dwAAsSsL3YEdSHzFSpvpPzVPTxB1fsdHqZ j7IMMhDgb6NpnDmTkLboDCLQ0l/sY1QcIz78nEVifC/IYTPGD7fSEm9kS1EqbGhFHt7Z r7s5C+FnhWYGciT2F3Wgv+843WkA2E/O3gkxTmmZKIyB9s7PpqGbGSD6HPy8a6Z+tqA5 sChw== X-Gm-Message-State: APjAAAV03FKyQBSz4E00kZIHYPESHs8/9fHybFWIEmJ3MLMm2KNeQ6YP 8bf7CDm2Q9oVqxXGgGdFkpx+oe1QZcr7PJilh/6U0Q== X-Google-Smtp-Source: APXvYqw/2+N2t2lLq1K+89TJ7bx2qoTvJJopuhhqt7f6cbBPG3uSnhVLpCVqqF2LFAFntCjNNsq3zJIjfvM4Xn6GSUj6sQ== X-Received: by 2002:a0d:c584:: with SMTP id h126mr2210425ywd.173.1582852873384; Thu, 27 Feb 2020 17:21:13 -0800 (PST) Date: Thu, 27 Feb 2020 17:20:36 -0800 In-Reply-To: <20200228012036.15682-1-brendanhiggins@google.com> Message-Id: <20200228012036.15682-8-brendanhiggins@google.com> Mime-Version: 1.0 References: <20200228012036.15682-1-brendanhiggins@google.com> X-Mailer: git-send-email 2.25.1.481.gfbce0eb801-goog Subject: [PATCH v3 7/7] Documentation: Add kunit_shutdown to kernel-parameters.txt From: Brendan Higgins To: jdike@addtoit.com, richard@nod.at, anton.ivanov@cambridgegreys.com, arnd@arndb.de, keescook@chromium.org, skhan@linuxfoundation.org, alan.maguire@oracle.com, yzaikin@google.com, davidgow@google.com, akpm@linux-foundation.org, rppt@linux.ibm.com, frowand.list@gmail.com Cc: gregkh@linuxfoundation.org, sboyd@kernel.org, logang@deltatee.com, mcgrof@kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Brendan Higgins Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Add kunit_shutdown, an option to specify that the kernel shutsdown after running KUnit tests, to the kernel-parameters.txt documentation. Signed-off-by: Brendan Higgins Reviewed-by: Stephen Boyd --- Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index dbc22d6846275..6ad63e98916f9 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2099,6 +2099,14 @@ 0: force disabled 1: force enabled + kunit_shutdown [KERNEL UNIT TESTING FRAMEWORK] Shutdown kernel after + running built-in tests. Tests configured as modules will + not be run. + Default: (flag not present) don't shutdown + poweroff: poweroff the kernel after running tests + halt: halt the kernel after running tests + reboot: reboot the kernel after running tests + kvm.ignore_msrs=[KVM] Ignore guest accesses to unhandled MSRs. Default is 0 (don't ignore, but inject #GP)