From patchwork Fri Sep 30 13:25:31 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= X-Patchwork-Id: 9358289 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7B23B6075E for ; Fri, 30 Sep 2016 13:26:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C8B62A025 for ; Fri, 30 Sep 2016 13:26:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 602232A029; Fri, 30 Sep 2016 13:26:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C6E1B2A025 for ; Fri, 30 Sep 2016 13:26:06 +0000 (UTC) Received: from localhost ([::1]:44531 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bpxpF-00046p-BC for patchwork-qemu-devel@patchwork.kernel.org; Fri, 30 Sep 2016 09:26:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58423) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bpxoy-00046U-3N for qemu-devel@nongnu.org; Fri, 30 Sep 2016 09:25:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bpxov-0006mU-B3 for qemu-devel@nongnu.org; Fri, 30 Sep 2016 09:25:47 -0400 Received: from mail-lf0-x22d.google.com ([2a00:1450:4010:c07::22d]:36143) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bpxou-0006ly-SA for qemu-devel@nongnu.org; Fri, 30 Sep 2016 09:25:45 -0400 Received: by mail-lf0-x22d.google.com with SMTP id g62so105772088lfe.3 for ; Fri, 30 Sep 2016 06:25:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=BIuTWPnOvTyom2hwjHIUdLn0VfzjRwFwVuC/5sUj5Mw=; b=FsIIg98Ib5ZpoMJIg/Vv1WGqpnlDOeK0UsCIO/CQuaFhwGQBQoc2j/WSkI/Q0ir/xN MlJJJFSk+6IyGD7n0GW6LMsbHXMlKSf6DsoBXsbEoEDRmvZ08TpkEhA7IhJSs06j3Ly1 l9CH/kfJ0B8SLDcxmq3DBiIAG0PfNHInMfTQuwwj3v0x8XaP54StkrErRhyIC2VjfQ00 MWZGrC2OKvUoNYk7NTRT/xbUVFDMroXyuXOdmscQWXyG141Z5Y59gMw2/Ve+t1nI3C8y nva6WWzMcwx7vZtLMjiyUqhsA7NW29nvSVQtBUVpVFo7NPLDMyt1t4wJWO8vm92klLZF hu9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BIuTWPnOvTyom2hwjHIUdLn0VfzjRwFwVuC/5sUj5Mw=; b=Rxc9NCCsV5/RvlOkYeIqUUbwCGkMQWyqCK7JjHtr4rnR0rgkADOYHzdK5ic6k0C6ou Nf1PgXvEhyXhd5rtusChFxmWgDWOeNZqkgs1vOY2tPxr1x3958O5OpD5aidgMSl6oak6 0IPCrt1S9wsRDHQC4RlpBrEWtwlU1egxLno/UtGs1OXscG6Mjc+pfUyiyEI9Dru0CV38 8B6TaEIpm64wp2BDMQ1iLQnRCSpF6l12uHp0RxjbAx2uzAuB+oQ5oTTWaKfqTeNMeV8F gQ/wtYuheq+PVrX1hYqQbeBsBlQCqy2S+nIcbIe9gbFyao6m6phNv6CwVI6wUDmd6xt3 Wesg== X-Gm-Message-State: AA6/9Rk5bp2KmJfHvv6wmZlHOdQipumyKZECpoMiBFKeVHtPamDJ8TxK/o92c6m+DbbJ1rsl9p0AFM29GVOg8g== X-Received: by 10.25.15.229 with SMTP id 98mr2547911lfp.24.1475241942853; Fri, 30 Sep 2016 06:25:42 -0700 (PDT) MIME-Version: 1.0 References: <1475136691-920-1-git-send-email-famz@redhat.com> In-Reply-To: <1475136691-920-1-git-send-email-famz@redhat.com> From: =?UTF-8?B?TWFyYy1BbmRyw6kgTHVyZWF1?= Date: Fri, 30 Sep 2016 13:25:31 +0000 Message-ID: To: Fam Zheng , qemu-devel@nongnu.org X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:4010:c07::22d X-Content-Filtered-By: Mailman/MimeDel 2.1.21 Subject: Re: [Qemu-devel] [PATCH v2] tests: Run qtest cases in parallel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Hi On Thu, Sep 29, 2016 at 12:13 PM Fam Zheng wrote: > Previously all qtest cases in a category, such as check-qtest-y, are > executed in a single long gtester command. This patch separates each > test into its own make target to allow better parallism, unless gcov is > configured. > > Slightly reorganize the gcov enabled case too, but the execution > sequence is kept. > > This saves >50% of the time (see below), which means a lot for patchew > automatic testing given how 'make check' takes part in testings. > > On my machine: > > before: > > $ make docker-test-quick@fedora J=8 &>/dev/null && \ > time make docker-test-quick@fedora J=8 >/dev/null; > > real 1m47.090s > user 0m1.599s > sys 0m0.258s > > after: > > real 0m54.067s > user 0m1.447s > sys 0m0.253s > > Signed-off-by: Fam Zheng > > --- > > v2: Fix gcov case by not changing it. [Daniel] > --- > tests/Makefile.include | 48 > ++++++++++++++++++++++++++++++++++++------------ > 1 file changed, 36 insertions(+), 12 deletions(-) > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index 8162f6f..575030a 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -729,27 +729,51 @@ GCOV_OPTIONS = -n $(if $(V),-f,) > # gtester tests, possibly with verbose output > > .PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS)) > -$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: > $(check-qtest-y) > - $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda > */*/*/*.gcda,) > - $(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \ > + > +run-tests = $(call quiet-command,\ > + $(if $(QTEST_TARGET), \ > + > QTEST_QEMU_BINARY=$(QTEST_TARGET)-softmmu/qemu-system-$(QTEST_TARGET)) \ > QTEST_QEMU_IMG=qemu-img$(EXESUF) \ > MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 + > 1))} \ > - gtester $(GTESTER_OPTIONS) -m=$(SPEED) $(check-qtest-$*-y) > $(check-qtest-generic-y),"GTESTER $@") > - $(if $(CONFIG_GCOV),@for f in $(gcov-files-$*-y) > $(gcov-files-generic-y); do \ > + gtester $1 $(GTESTER_OPTIONS) -m=$(SPEED),"GTESTER $2") > + > +ifneq ($(CONFIG_GCOV),) > + > +check-qtest-%: QTEST_TARGET=$* > +$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: > $(check-qtest-y) > + @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda > + $(call run-tests, $(check-qtest-$*-y), $@) > + @for f in $(gcov-files-$*-y) $(gcov-files-generic-y); do \ > echo Gcov report for $$f:;\ > $(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \ > - done,) > + done > > .PHONY: $(patsubst %, check-%, $(check-unit-y)) > $(patsubst %, check-%, $(check-unit-y)): check-%: % > - $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda > */*/*/*.gcda,) > - $(call quiet-command, \ > - MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 + > 1))} \ > - gtester $(GTESTER_OPTIONS) -m=$(SPEED) $*,"GTESTER $*") > - $(if $(CONFIG_GCOV),@for f in $(gcov-files-$(subst tests/,,$*)-y) > $(gcov-files-generic-y); do \ > + @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda > + $(call run-tests, $*, $*) > + @for f in $(gcov-files-$(subst tests/,,$*)-y) > $(gcov-files-generic-y); do \ > echo Gcov report for $$f:;\ > $(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \ > - done,) > + done > + > +else > + > +run-test-%: tests/% > + $(call run-tests, $<, $<) > + > +$(foreach target, $(QTEST_TARGETS), \ > + $(eval check-qtest-$(target): QTEST_TARGET := $(target)) \ > + $(eval check-qtest-$(target): $(patsubst tests/%, run-test-%, \ > + $(check-qtest-y) \ > + $(check-qtest-$(target)-y) \ > + $(check-qtest-generic-y))) \ > +) > +$(patsubst %, check-%, $(check-unit-y)): check-tests/%: run-test-% > + > +endif > + > +.PHONY: $(patsubst %, check-%, $(check-unit-y)) > > it's quite difficult to read, but it looks correct to me. Something I wish we had that we can perhaps add as well here (add dep on -softmmu build): $(call run-tests, $(check-qtest-$*-y), $@) @@ -767,6 +768,7 @@ run-test-%: tests/% $(foreach target, $(QTEST_TARGETS), \ $(eval check-qtest-$(target): QTEST_TARGET := $(target)) \ + $(eval check-qtest-$(target): subdir-$(target)-softmmu) \ $(eval check-qtest-$(target): $(patsubst tests/%, run-test-%, \ $(check-qtest-y) \ $(check-qtest-$(target)-y) \ I can post a seperate patch later though, Similarly, qga test: -tests/test-qga: tests/test-qga.o $(qtest-obj-y) +tests/test-qga: tests/test-qga.o $(qtest-obj-y) qemu-ga$(EXESUF) In any case Reviewed-by: Marc-André Lureau > # gtester tests with XML output > > -- > 2.7.4 > > > -- Marc-André Lureau diff --git a/tests/Makefile.include b/tests/Makefile.include index f5491c7..8e5dc72 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -743,6 +743,7 @@ run-tests = $(call quiet-command,\ ifneq ($(CONFIG_GCOV),) check-qtest-%: QTEST_TARGET=$* +check-qtest-%: subdir-$*-softmmu $(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y) @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda