From patchwork Fri Apr 8 05:08:22 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas Meneghel Rodrigues X-Patchwork-Id: 693981 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter1.kernel.org (8.14.4/8.14.3) with ESMTP id p388EDtZ003611 for ; Fri, 8 Apr 2011 08:15:21 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752377Ab1DHFIp (ORCPT ); Fri, 8 Apr 2011 01:08:45 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49942 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751988Ab1DHFIf (ORCPT ); Fri, 8 Apr 2011 01:08:35 -0400 Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p3858XnS031227 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 8 Apr 2011 01:08:33 -0400 Received: from freedom.redhat.com (vpn-8-92.rdu.redhat.com [10.11.8.92]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p3858QKf008833; Fri, 8 Apr 2011 01:08:31 -0400 From: Lucas Meneghel Rodrigues To: autotest@test.kernel.org Cc: kvm@vger.kernel.org, crosa@redhat.com, ehabkost@redhat.com, mgoldish@redhat.com, jadmanski@google.com, Lucas Meneghel Rodrigues Subject: [PATCH 1/4] client job: Introduce a run_test_detail method Date: Fri, 8 Apr 2011 02:08:22 -0300 Message-Id: <1302239305-15786-2-git-send-email-lmr@redhat.com> In-Reply-To: <1302239305-15786-1-git-send-email-lmr@redhat.com> References: <1302239305-15786-1-git-send-email-lmr@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter1.kernel.org [140.211.167.41]); Fri, 08 Apr 2011 08:15:21 +0000 (UTC) For the KVM test, we have for a long time a system of 'test dependencies': If a dependency test, such as guest installation has failed, it is not possible to run subsequent tests for obvious reasons. There are minor failures though, that don't necessarily block other tests, such as some specific types of image problems found on a guest after the routine check. However, job.run_test() returns a boolean, making it not possible to fail a test, so we are not ignoring problems, and at the same time, execute dependent tests, because the failure was deemed to be non-fatal. Therefore, introduce and additional method, job.run_test_detail(), that returns the test status (see all possible statuses on client/common_lib/error.py), rather than a boolean. With this finer grained control, we can implement the kvm subtest dependency system in a better way. Strategy: refactor the run_test method into _run_test_base and creating the 2 implementations, run_test, and run_test_detail in terms of _run_test_base. Risk: High (touches a very very visible test writer API) Tests executed: run_test continues to work, run_test_detail provides the correct return codes and change does not break job_unittest. Signed-off-by: Lucas Meneghel Rodrigues --- client/bin/job.py | 54 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 50 insertions(+), 4 deletions(-) diff --git a/client/bin/job.py b/client/bin/job.py index 97412c0..9effcdb 100644 --- a/client/bin/job.py +++ b/client/bin/job.py @@ -539,10 +539,9 @@ class base_client_job(base_job.base_job): raise error.UnhandledTestError(e) - @_run_test_complete_on_exit - def run_test(self, url, *args, **dargs): + def _run_test_base(self, url, *args, **dargs): """ - Summon a test object and run it. + Prepares arguments and run functions to run_test and run_test_detail. @param url A url that identifies the test to run. @param tag An optional keyword argument that will be added to the @@ -550,7 +549,11 @@ class base_client_job(base_job.base_job): @param subdir_tag An optional keyword argument that will be added to the subdir name. - @returns True if the test passes, False otherwise. + @returns: + subdir: Test subdirectory + testname: Test name + group_func: Actual test run function + timeout: Test timeout """ group, testname = self.pkgmgr.get_package_name(url, 'test') testname, subdir, tag = self._build_tagged_test_name(testname, dargs) @@ -573,6 +576,25 @@ class base_client_job(base_job.base_job): else: self.record('GOOD', subdir, testname, 'completed successfully') + return (subdir, testname, group_func, timeout) + + + @_run_test_complete_on_exit + def run_test(self, url, *args, **dargs): + """ + Summon a test object and run it. + + @param url A url that identifies the test to run. + @param tag An optional keyword argument that will be added to the + test and subdir name. + @param subdir_tag An optional keyword argument that will be added + to the subdir name. + + @returns True if the test passes, False otherwise. + """ + (subdir, testname, group_func, timeout) = self._run_test_base(url, + *args, + **dargs) try: self._rungroup(subdir, testname, group_func, timeout) return True @@ -585,6 +607,30 @@ class base_client_job(base_job.base_job): # UnhandledTestError that is caught above. + @_run_test_complete_on_exit + def run_test_detail(self, url, *args, **dargs): + """ + Summon a test object and run it, returning test status. + + @param url A url that identifies the test to run. + @param tag An optional keyword argument that will be added to the + test and subdir name. + @param subdir_tag An optional keyword argument that will be added + to the subdir name. + + @returns Test status + @see: client/common_lib/error.py, exit_status + """ + (subdir, testname, group_func, timeout) = self._run_test_base(url, + *args, + **dargs) + try: + self._rungroup(subdir, testname, group_func, timeout) + return 'GOOD' + except error.TestBaseException, detail: + return detail.exit_status + + def _rungroup(self, subdir, testname, function, timeout, *args, **dargs): """\ subdir: