From patchwork Fri Feb 5 09:24:12 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fam Zheng X-Patchwork-Id: 8232761 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 39F90BEEE5 for ; Fri, 5 Feb 2016 09:24:59 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 51AE2201B9 for ; Fri, 5 Feb 2016 09:24:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 48837200E9 for ; Fri, 5 Feb 2016 09:24:57 +0000 (UTC) Received: from localhost ([::1]:47005 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRcdM-0004Bs-Lr for patchwork-qemu-devel@patchwork.kernel.org; Fri, 05 Feb 2016 04:24:56 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55403) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRcd7-0003xw-Ul for qemu-devel@nongnu.org; Fri, 05 Feb 2016 04:24:43 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aRcd6-0007p3-KD for qemu-devel@nongnu.org; Fri, 05 Feb 2016 04:24:41 -0500 Received: from mx1.redhat.com ([209.132.183.28]:49571) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRcd6-0007ot-A0 for qemu-devel@nongnu.org; Fri, 05 Feb 2016 04:24:40 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (Postfix) with ESMTPS id F38A3A0B4D; Fri, 5 Feb 2016 09:24:39 +0000 (UTC) Received: from fam-t430.nay.redhat.com (dhcp-15-42.nay.redhat.com [10.66.15.42]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u159OODQ022411; Fri, 5 Feb 2016 04:24:35 -0500 From: Fam Zheng To: qemu-devel@nongnu.org Date: Fri, 5 Feb 2016 17:24:12 +0800 Message-Id: <1454664263-25969-2-git-send-email-famz@redhat.com> In-Reply-To: <1454664263-25969-1-git-send-email-famz@redhat.com> References: <1454664263-25969-1-git-send-email-famz@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: kwolf@redhat.com, peter.maydell@linaro.org, jsnow@redhat.com, stefanha@redhat.com, sw@weilnetz.de, Paolo Bonzini , =?UTF-8?q?Alex=20Benn=C3=A9e?= , david@gibson.dropbear.id.au Subject: [Qemu-devel] [PATCH 01/12] tests: Add utilities for docker testing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP docker_run: A wrapper for "docker run" (or "sudo -n docker run" if necessary), which takes care of killing and removing the running container at SIGINT. docker_clean: A tool to tear down all the containers including inactive ones that are started by docker_run. docker_build: A tool to compare an image from given dockerfile and rebuild it if they're different. Signed-off-by: Fam Zheng --- tests/docker/docker.py | 108 ++++++++++++++++++++++++++++++++++++++++++++++ tests/docker/docker_build | 42 ++++++++++++++++++ tests/docker/docker_clean | 22 ++++++++++ tests/docker/docker_run | 28 ++++++++++++ 4 files changed, 200 insertions(+) create mode 100755 tests/docker/docker.py create mode 100755 tests/docker/docker_build create mode 100755 tests/docker/docker_clean create mode 100755 tests/docker/docker_run diff --git a/tests/docker/docker.py b/tests/docker/docker.py new file mode 100755 index 0000000..e513da0 --- /dev/null +++ b/tests/docker/docker.py @@ -0,0 +1,108 @@ +#!/usr/bin/env python2 -B +# +# Docker controlling module +# +# Copyright (c) 2016 Red Hat Inc. +# +# Authors: +# Fam Zheng +# +# This work is licensed under the terms of the GNU GPL, version 2 +# or (at your option) any later version. See the COPYING file in +# the top-level directory. + +import os +import subprocess +import json +import hashlib +import atexit +import time +import uuid + +class ContainerTerminated(Exception): + pass + +class Docker(object): + def __init__(self): + self._command = self._guess_command() + self._instances = [] + atexit.register(self._kill_instances) + + def _do(self, cmd, quiet=True, **kwargs): + if quiet: + kwargs["stdout"] = subprocess.PIPE + return subprocess.call(self._command + cmd, **kwargs) + + def _do_kill_instances(self, only_known, only_active=True): + cmd = ["ps", "-q"] + if not only_active: + cmd.append("-a") + for i in self._output(cmd).split(): + r = self._output(["inspect", i]) + labels = json.loads(r)[0]["Config"]["Labels"] + active = json.loads(r)[0]["State"]["Running"] + if not labels: + continue + u = labels.get("com.qemu.instance.uuid", None) + if not u: + continue + if only_known and u not in self._instances: + continue + print "Terminating", i + if active: + self._do(["kill", i]) + self._do(["rm", i]) + + def clean(self): + self._do_kill_instances(False, False) + return 0 + + def _kill_instances(self): + return self._do_kill_instances(True) + + def _output(self, cmd, **kwargs): + return subprocess.check_output(self._command + cmd, + stderr=subprocess.STDOUT, + **kwargs) + + def _guess_command(self): + for c in [["docker"], ["sudo", "-n", "docker"]]: + if subprocess.call(c + ["images"], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) == 0: + return c + raise Exception("Cannot find working docker command") + + def get_image_dockerfile_checksum(self, tag): + resp = self._output(["inspect", tag]) + t = json.loads(resp)[0] + return t["Config"].get("Labels", {}).get("com.qemu.dockerfile-checksum", "") + + def checksum(self, text): + return hashlib.sha1(text).hexdigest() + + def build_image(self, tag, dockerfile, df, quiet=True): + tmp = dockerfile + "\n" + \ + "LABEL com.qemu.dockerfile-checksum=%s" % self.checksum(dockerfile) + tmp_df = df + ".tmp" + f = open(tmp_df, "wb") + f.write(tmp) + f.close() + self._do(["build", "-t", tag, "-f", tmp_df, os.path.dirname(df)], + quiet=quiet) + os.unlink(tmp_df) + + def image_matches_dockerfile(self, tag, dockerfile): + try: + a = self.get_image_dockerfile_checksum(tag) + except: + return False + return a == self.checksum(dockerfile) + + def run(self, cmd, quiet, **kwargs): + label = uuid.uuid1().hex + self._instances.append(label) + r = self._do(["run", "--label", "com.qemu.instance.uuid=" + label] + cmd, quiet=quiet) + self._instances.remove(label) + return r + diff --git a/tests/docker/docker_build b/tests/docker/docker_build new file mode 100755 index 0000000..b4f0dec --- /dev/null +++ b/tests/docker/docker_build @@ -0,0 +1,42 @@ +#!/usr/bin/env python2 +# +# Compare to Dockerfile and rebuild a docker image if necessary. +# +# Copyright (c) 2016 Red Hat Inc. +# +# Authors: +# Fam Zheng +# +# This work is licensed under the terms of the GNU GPL, version 2 +# or (at your option) any later version. See the COPYING file in +# the top-level directory. + +import sys +import docker +import argparse + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("tag", + help="Image Tag") + parser.add_argument("dockerfile", + help="Dockerfile name") + parser.add_argument("--verbose", "-v", action="store_true", + help="Print verbose information") + args = parser.parse_args() + + dockerfile = open(args.dockerfile, "rb").read() + tag = args.tag + + d = docker.Docker() + if d.image_matches_dockerfile(tag, dockerfile): + if args.verbose: + print "Image is up to date." + return 0 + + quiet = not args.verbose + d.build_image(tag, dockerfile, args.dockerfile, quiet=quiet) + return 0 + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/docker/docker_clean b/tests/docker/docker_clean new file mode 100755 index 0000000..88cdba6 --- /dev/null +++ b/tests/docker/docker_clean @@ -0,0 +1,22 @@ +#!/usr/bin/env python2 +# +# Clean up uselsee containers. +# +# Copyright (c) 2016 Red Hat Inc. +# +# Authors: +# Fam Zheng +# +# This work is licensed under the terms of the GNU GPL, version 2 +# or (at your option) any later version. See the COPYING file in +# the top-level directory. + +import sys +import docker + +def main(): + docker.Docker().clean() + return 0 + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/docker/docker_run b/tests/docker/docker_run new file mode 100755 index 0000000..5cf9d04 --- /dev/null +++ b/tests/docker/docker_run @@ -0,0 +1,28 @@ +#!/usr/bin/env python2 +# +# Wrapper for "docker run" with automatical clean up if the execution is +# iterrupted. +# +# Copyright (c) 2016 Red Hat Inc. +# +# Authors: +# Fam Zheng +# +# This work is licensed under the terms of the GNU GPL, version 2 +# or (at your option) any later version. See the COPYING file in +# the top-level directory. + +import os +import sys +import argparse +import docker + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--quiet", action="store_true", + help="Run quietly unless an error occured") + args, argv = parser.parse_known_args() + return docker.Docker().run(argv, quiet=args.quiet) + +if __name__ == "__main__": + sys.exit(main())