From patchwork Wed Jun 30 01:26:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 12350981 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49F76C11F67 for ; Wed, 30 Jun 2021 01:29:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB80961D85 for ; Wed, 30 Jun 2021 01:29:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB80961D85 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53328 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lyP33-0002lV-Vo for qemu-devel@archiver.kernel.org; Tue, 29 Jun 2021 21:29:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44874) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyOzt-000445-Jk for qemu-devel@nongnu.org; Tue, 29 Jun 2021 21:26:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24971) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyOzq-0004R8-SY for qemu-devel@nongnu.org; Tue, 29 Jun 2021 21:26:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625016393; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OZq84VMmhn6ncrqesCbVbOTRL5Po4i8CKJW1QiAhH0Y=; b=fOoJbjw7utek3UeBoYoUXpjdxz9x64AOZG/yQCtjBtr9ya+V05NsxmtYSnJ2u8YVo6qhYF 4da4rWrSmmc6Gx4gf11o6Y/6wB5NZZHlXDxpNJlyjoowq+hxSFKxYXuIrUbZWyuBD1mXE7 q7C6im6ExJVwBtSoyYI44GfUVIgDHlA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-246-Bh2QwCX3MhmaChwf9tUipA-1; Tue, 29 Jun 2021 21:26:29 -0400 X-MC-Unique: Bh2QwCX3MhmaChwf9tUipA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8B245802C80; Wed, 30 Jun 2021 01:26:28 +0000 (UTC) Received: from fedora.hsd1.nh.comcast.net (ovpn-114-132.rdu2.redhat.com [10.10.114.132]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D6D6F5C230; Wed, 30 Jun 2021 01:26:26 +0000 (UTC) From: Cleber Rosa To: =?utf-8?q?Alex_Benn=C3=A9e?= , Peter Maydell , qemu-devel@nongnu.org Subject: [PATCH v7 1/4] Jobs based on custom runners: documentation and configuration placeholder Date: Tue, 29 Jun 2021 21:26:16 -0400 Message-Id: <20210630012619.115262-2-crosa@redhat.com> In-Reply-To: <20210630012619.115262-1-crosa@redhat.com> References: <20210630012619.115262-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.435, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?utf-8?q?Da?= =?utf-8?q?niel_P_=2E_Berrang=C3=A9?= , Eduardo Habkost , Erik Skultety , Stefan Hajnoczi , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Wainer dos Santos Moschetta , Willian Rampazzo , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Beraldo Leal Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" As described in the included documentation, the "custom runner" jobs extend the GitLab CI jobs already in place. One of their primary goals of catching and preventing regressions on a wider number of host systems than the ones provided by GitLab's shared runners. This sets the stage in which other community members can add their own machine configuration documentation/scripts, and accompanying job definitions. As a general rule, those newly added contributed jobs should run as "non-gating", until their reliability is verified (AKA "allow_failure: true"). Signed-off-by: Cleber Rosa Reviewed-by: Alex Bennée Reviewed-by: Thomas Huth Reviewed-by: Willian Rampazzo Reviewed-by: Wainer dos Santos Moschetta --- .gitlab-ci.d/custom-runners.yml | 14 ++++++++++++++ .gitlab-ci.d/qemu-project.yml | 1 + docs/devel/ci.rst | 32 ++++++++++++++++++++++++++++++++ docs/devel/index.rst | 1 + 4 files changed, 48 insertions(+) create mode 100644 .gitlab-ci.d/custom-runners.yml create mode 100644 docs/devel/ci.rst diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml new file mode 100644 index 0000000000..a07b27384c --- /dev/null +++ b/.gitlab-ci.d/custom-runners.yml @@ -0,0 +1,14 @@ +# The CI jobs defined here require GitLab runners installed and +# registered on machines that match their operating system names, +# versions and architectures. This is in contrast to the other CI +# jobs that are intended to run on GitLab's "shared" runners. + +# Different than the default approach on "shared" runners, based on +# containers, the custom runners have no such *requirement*, as those +# jobs should be capable of running on operating systems with no +# compatible container implementation, or no support from +# gitlab-runner. To avoid problems that gitlab-runner can cause while +# reusing the GIT repository, let's enable the clone strategy, which +# guarantees a fresh repository on each job run. +variables: + GIT_STRATEGY: clone diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml index 64cb2ba1da..dde8270301 100644 --- a/.gitlab-ci.d/qemu-project.yml +++ b/.gitlab-ci.d/qemu-project.yml @@ -9,3 +9,4 @@ include: - local: '/.gitlab-ci.d/crossbuilds.yml' - local: '/.gitlab-ci.d/buildtest.yml' - local: '/.gitlab-ci.d/static_checks.yml' + - local: '/.gitlab-ci.d/custom-runners.yml' diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst new file mode 100644 index 0000000000..064ffa9988 --- /dev/null +++ b/docs/devel/ci.rst @@ -0,0 +1,32 @@ +== +CI +== + +QEMU has configurations enabled for a number of different CI services. +The most up to date information about them and their status can be +found at:: + + https://wiki.qemu.org/Testing/CI + +Jobs on Custom Runners +====================== + +Besides the jobs run under the various CI systems listed before, there +are a number additional jobs that will run before an actual merge. +These use the same GitLab CI's service/framework already used for all +other GitLab based CI jobs, but rely on additional systems, not the +ones provided by GitLab as "shared runners". + +The architecture of GitLab's CI service allows different machines to +be set up with GitLab's "agent", called gitlab-runner, which will take +care of running jobs created by events such as a push to a branch. +Here, the combination of a machine, properly configured with GitLab's +gitlab-runner, is called a "custom runner". + +The GitLab CI jobs definition for the custom runners are located under:: + + .gitlab-ci.d/custom-runners.yml + +Custom runners entail custom machines. To see a list of the machines +currently deployed in the QEMU GitLab CI and their maintainers, please +refer to the QEMU `wiki `__. diff --git a/docs/devel/index.rst b/docs/devel/index.rst index 977c3893bd..da15273786 100644 --- a/docs/devel/index.rst +++ b/docs/devel/index.rst @@ -27,6 +27,7 @@ Contents: migration atomics stable-process + ci qtest decodetree secure-coding-practices From patchwork Wed Jun 30 01:26:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 12350973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8B29C11F67 for ; Wed, 30 Jun 2021 01:28:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3525E61D85 for ; Wed, 30 Jun 2021 01:28:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3525E61D85 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:46580 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lyP1E-0006f8-3V for qemu-devel@archiver.kernel.org; Tue, 29 Jun 2021 21:28:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44876) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyOzt-000446-Js for qemu-devel@nongnu.org; Tue, 29 Jun 2021 21:26:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:47286) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyOzr-0004Sd-1l for qemu-devel@nongnu.org; Tue, 29 Jun 2021 21:26:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625016394; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HrJCGM14fHiP9HcI+Z6zLBDhZYy0tpp6iFQA9M+7wYs=; b=GNei/BTmhMccl2yZVQaV4XKamWAu/CfdjZS7hqck+iJO25+037KgtyMaS85H34RE2mJR// YBSrcvWM7GZWqQV1Mu5ZS79tc1HeNBw4henuhqZitA383NNO75v3H7TWI8oxyc55/Owwrr M82JkaaaWMTdAbwlpZ0uUmvSqZ3ggbk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-492-ymWKA1-bN_uPW8xxGfTqng-1; Tue, 29 Jun 2021 21:26:31 -0400 X-MC-Unique: ymWKA1-bN_uPW8xxGfTqng-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 223BD18414A0; Wed, 30 Jun 2021 01:26:30 +0000 (UTC) Received: from fedora.hsd1.nh.comcast.net (ovpn-114-132.rdu2.redhat.com [10.10.114.132]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AF7795C1D1; Wed, 30 Jun 2021 01:26:28 +0000 (UTC) From: Cleber Rosa To: =?utf-8?q?Alex_Benn=C3=A9e?= , Peter Maydell , qemu-devel@nongnu.org Subject: [PATCH v7 2/4] Jobs based on custom runners: build environment docs and playbook Date: Tue, 29 Jun 2021 21:26:17 -0400 Message-Id: <20210630012619.115262-3-crosa@redhat.com> In-Reply-To: <20210630012619.115262-1-crosa@redhat.com> References: <20210630012619.115262-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.435, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?utf-8?q?Da?= =?utf-8?q?niel_P_=2E_Berrang=C3=A9?= , Eduardo Habkost , Erik Skultety , Stefan Hajnoczi , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Wainer dos Santos Moschetta , Willian Rampazzo , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Beraldo Leal Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" To run basic jobs on custom runners, the environment needs to be properly set up. The most common requirement is having the right packages installed. The playbook introduced here covers the QEMU's project s390x and aarch64 machines. At the time this is being proposed, those machines have already had this playbook applied to them. Signed-off-by: Cleber Rosa Tested-by: Alex Bennée Reviewed-by: Alex Bennée Reviewed-by: Willian Rampazzo Tested-by: Willian Rampazzo Reviewed-by: Wainer dos Santos Moschetta --- docs/devel/ci.rst | 40 +++++++++ scripts/ci/setup/.gitignore | 2 + scripts/ci/setup/build-environment.yml | 116 +++++++++++++++++++++++++ scripts/ci/setup/inventory.template | 1 + 4 files changed, 159 insertions(+) create mode 100644 scripts/ci/setup/.gitignore create mode 100644 scripts/ci/setup/build-environment.yml create mode 100644 scripts/ci/setup/inventory.template diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst index 064ffa9988..bfedbb1025 100644 --- a/docs/devel/ci.rst +++ b/docs/devel/ci.rst @@ -30,3 +30,43 @@ The GitLab CI jobs definition for the custom runners are located under:: Custom runners entail custom machines. To see a list of the machines currently deployed in the QEMU GitLab CI and their maintainers, please refer to the QEMU `wiki `__. + +Machine Setup Howto +------------------- + +For all Linux based systems, the setup can be mostly automated by the +execution of two Ansible playbooks. Create an ``inventory`` file +under ``scripts/ci/setup``, such as this:: + + fully.qualified.domain + other.machine.hostname + +You may need to set some variables in the inventory file itself. One +very common need is to tell Ansible to use a Python 3 interpreter on +those hosts. This would look like:: + + fully.qualified.domain ansible_python_interpreter=/usr/bin/python3 + other.machine.hostname ansible_python_interpreter=/usr/bin/python3 + +Build environment +~~~~~~~~~~~~~~~~~ + +The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will +set up machines with the environment needed to perform builds and run +QEMU tests. This playbook consists on the installation of various +required packages (and a general package update while at it). It +currently covers a number of different Linux distributions, but it can +be expanded to cover other systems. + +The minimum required version of Ansible successfully tested in this +playbook is 2.8.0 (a version check is embedded within the playbook +itself). To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory build-environment.yml + +Please note that most of the tasks in the playbook require superuser +privileges, such as those from the ``root`` account or those obtained +by ``sudo``. If necessary, please refer to ``ansible-playbook`` +options such as ``--become``, ``--become-method``, ``--become-user`` +and ``--ask-become-pass``. diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore new file mode 100644 index 0000000000..ee088604d1 --- /dev/null +++ b/scripts/ci/setup/.gitignore @@ -0,0 +1,2 @@ +inventory + diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml new file mode 100644 index 0000000000..581c1c75d1 --- /dev/null +++ b/scripts/ci/setup/build-environment.yml @@ -0,0 +1,116 @@ +# Copyright (c) 2021 Red Hat, Inc. +# +# Author: +# Cleber Rosa +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. +# +# This is an ansible playbook file. Run it to set up systems with the +# environment needed to build QEMU. +--- +- name: Installation of basic packages to build QEMU + hosts: all + tasks: + - name: Check for suitable ansible version + delegate_to: localhost + assert: + that: + - '((ansible_version.major == 2) and (ansible_version.minor >= 8)) or (ansible_version.major >= 3)' + msg: "Unsuitable ansible version, please use version 2.8.0 or later" + + - name: Update apt cache / upgrade packages via apt + apt: + update_cache: yes + upgrade: yes + when: + - ansible_facts['distribution'] == 'Ubuntu' + + - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04 + package: + name: + # Originally from tests/docker/dockerfiles/ubuntu1804.docker + - ccache + - gcc + - gettext + - git + - glusterfs-common + - libaio-dev + - libattr1-dev + - libbrlapi-dev + - libbz2-dev + - libcacard-dev + - libcap-ng-dev + - libcurl4-gnutls-dev + - libdrm-dev + - libepoxy-dev + - libfdt-dev + - libgbm-dev + - libgtk-3-dev + - libibverbs-dev + - libiscsi-dev + - libjemalloc-dev + - libjpeg-turbo8-dev + - liblzo2-dev + - libncurses5-dev + - libncursesw5-dev + - libnfs-dev + - libnss3-dev + - libnuma-dev + - libpixman-1-dev + - librados-dev + - librbd-dev + - librdmacm-dev + - libsasl2-dev + - libsdl2-dev + - libseccomp-dev + - libsnappy-dev + - libspice-protocol-dev + - libssh-dev + - libusb-1.0-0-dev + - libusbredirhost-dev + - libvdeplug-dev + - libvte-2.91-dev + - libzstd-dev + - make + - python3-yaml + - python3-sphinx + - python3-sphinx-rtd-theme + - ninja-build + - sparse + - xfslibs-dev + state: present + when: + - ansible_facts['distribution'] == 'Ubuntu' + + - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x + package: + name: + - libspice-server-dev + - libxen-dev + state: present + when: + - ansible_facts['distribution'] == 'Ubuntu' + - ansible_facts['architecture'] != 's390x' + + - name: Install basic packages to build QEMU on Ubuntu 18.04 + package: + name: + # Originally from tests/docker/dockerfiles/ubuntu1804.docker + - clang + when: + - ansible_facts['distribution'] == 'Ubuntu' + - ansible_facts['distribution_version'] == '18.04' + + - name: Install basic packages to build QEMU on Ubuntu 20.04 + package: + name: + # Originally from tests/docker/dockerfiles/ubuntu2004.docker + - clang-10 + - genisoimage + - liblttng-ust-dev + - libslirp-dev + - netcat-openbsd + when: + - ansible_facts['distribution'] == 'Ubuntu' + - ansible_facts['distribution_version'] == '20.04' diff --git a/scripts/ci/setup/inventory.template b/scripts/ci/setup/inventory.template new file mode 100644 index 0000000000..2fbb50c4a8 --- /dev/null +++ b/scripts/ci/setup/inventory.template @@ -0,0 +1 @@ +localhost From patchwork Wed Jun 30 01:26:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 12350977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11960C11F6A for ; Wed, 30 Jun 2021 01:28:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5F04C61D8A for ; Wed, 30 Jun 2021 01:28:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F04C61D8A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:46618 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lyP1E-0006gp-EC for qemu-devel@archiver.kernel.org; Tue, 29 Jun 2021 21:28:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44886) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyOzu-00044S-6n for qemu-devel@nongnu.org; Tue, 29 Jun 2021 21:26:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48995) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyOzq-0004SW-TT for qemu-devel@nongnu.org; Tue, 29 Jun 2021 21:26:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625016394; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V5LSU8c3OLKtY67vIj/JEM+PehhQRCpmnLk6zUXYfN4=; b=CNf9pkeluZX2bcKi/7Q7GvksFm+iZUg3Xs2pHINIwrSOct6aoshkT6vYTpMPzf/6WEeEQW 2KXCjOC+wBjCAwye0rbnDj8MJuEjwfTM0yb9yGmCgEY6RCdNdEJrC3+T/x/2hdjFKo3LZr 0m9oqpvLYzYk61+rJKEKZSEdbLtb4Q0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-420-cKgHrZY1My2flrZ335hu_A-1; Tue, 29 Jun 2021 21:26:32 -0400 X-MC-Unique: cKgHrZY1My2flrZ335hu_A-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A7CDB18414A1; Wed, 30 Jun 2021 01:26:31 +0000 (UTC) Received: from fedora.hsd1.nh.comcast.net (ovpn-114-132.rdu2.redhat.com [10.10.114.132]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 41D0C5C1D1; Wed, 30 Jun 2021 01:26:30 +0000 (UTC) From: Cleber Rosa To: =?utf-8?q?Alex_Benn=C3=A9e?= , Peter Maydell , qemu-devel@nongnu.org Subject: [PATCH v7 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook Date: Tue, 29 Jun 2021 21:26:18 -0400 Message-Id: <20210630012619.115262-4-crosa@redhat.com> In-Reply-To: <20210630012619.115262-1-crosa@redhat.com> References: <20210630012619.115262-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.435, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?utf-8?q?Da?= =?utf-8?q?niel_P_=2E_Berrang=C3=A9?= , Eduardo Habkost , Erik Skultety , Stefan Hajnoczi , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Wainer dos Santos Moschetta , Willian Rampazzo , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Beraldo Leal Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" To have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps. The playbook introduced here covers the Linux distributions and has been primarily tested on OS/machines that the QEMU project has available to act as runners, namely: * Ubuntu 20.04 on aarch64 * Ubuntu 18.04 on s390x But, it should work on all other Linux distributions. Earlier versions were tested on FreeBSD too, so chances of success are high. Signed-off-by: Cleber Rosa Reviewed-by: Willian Rampazzo Tested-by: Willian Rampazzo Reviewed-by: Wainer dos Santos Moschetta Reviewed-by: Alex Bennée Tested-by: Alex Bennée --- docs/devel/ci.rst | 55 +++++++++++++++++++++++ scripts/ci/setup/.gitignore | 2 +- scripts/ci/setup/gitlab-runner.yml | 71 ++++++++++++++++++++++++++++++ scripts/ci/setup/vars.yml.template | 12 +++++ 4 files changed, 139 insertions(+), 1 deletion(-) create mode 100644 scripts/ci/setup/gitlab-runner.yml create mode 100644 scripts/ci/setup/vars.yml.template diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst index bfedbb1025..b3bf3ef615 100644 --- a/docs/devel/ci.rst +++ b/docs/devel/ci.rst @@ -70,3 +70,58 @@ privileges, such as those from the ``root`` account or those obtained by ``sudo``. If necessary, please refer to ``ansible-playbook`` options such as ``--become``, ``--become-method``, ``--become-user`` and ``--ask-become-pass``. + +gitlab-runner setup and registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The gitlab-runner agent needs to be installed on each machine that +will run jobs. The association between a machine and a GitLab project +happens with a registration token. To find the registration token for +your repository/project, navigate on GitLab's web UI to: + + * Settings (the gears-like icon at the bottom of the left hand side + vertical toolbar), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * Under "Set up a specific Runner manually", look for the value under + "And this registration token:" + +Copy the ``scripts/ci/setup/vars.yml.template`` file to +``scripts/ci/setup/vars.yml``. Then, set the +``gitlab_runner_registration_token`` variable to the value obtained +earlier. + +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory gitlab-runner.yml + +Following the registration, it's necessary to configure the runner tags, +and optionally other configurations on the GitLab UI. Navigate to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * "Runners activated for this project", then + * Click on the "Edit" icon (next to the "Lock" Icon) + +Tags are very important as they are used to route specific jobs to +specific types of runners, so it's a good idea to double check that +the automatically created tags are consistent with the OS and +architecture. For instance, an Ubuntu 20.04 aarch64 system should +have tags set as:: + + ubuntu_20.04,aarch64 + +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` +would contain:: + + ubuntu-20.04-aarch64-all: + tags: + - ubuntu_20.04 + - aarch64 + +It's also recommended to: + + * increase the "Maximum job timeout" to something like ``2h`` + * give it a better Description diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore index ee088604d1..f4a6183f1f 100644 --- a/scripts/ci/setup/.gitignore +++ b/scripts/ci/setup/.gitignore @@ -1,2 +1,2 @@ inventory - +vars.yml diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml new file mode 100644 index 0000000000..1127db516f --- /dev/null +++ b/scripts/ci/setup/gitlab-runner.yml @@ -0,0 +1,71 @@ +# Copyright (c) 2021 Red Hat, Inc. +# +# Author: +# Cleber Rosa +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. +# +# This is an ansible playbook file. Run it to set up systems with the +# gitlab-runner agent. +--- +- name: Installation of gitlab-runner + hosts: all + vars_files: + - vars.yml + tasks: + - debug: + msg: 'Checking for a valid GitLab registration token' + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" + + - name: Create a group for the gitlab-runner service + group: + name: gitlab-runner + + - name: Create a user for the gitlab-runner service + user: + user: gitlab-runner + group: gitlab-runner + comment: GitLab Runner + home: /home/gitlab-runner + shell: /bin/bash + + - name: Remove the .bash_logout file when on Ubuntu systems + file: + path: /home/gitlab-runner/.bash_logout + state: absent + when: "ansible_facts['distribution'] == 'Ubuntu'" + + - name: Set the Operating System for gitlab-runner + set_fact: + gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}" + - debug: + msg: gitlab-runner OS is {{ gitlab_runner_os }} + + - name: Set the architecture for gitlab-runner + set_fact: + gitlab_runner_arch: "{{ ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}" + - debug: + msg: gitlab-runner arch is {{ gitlab_runner_arch }} + + - name: Download the matching gitlab-runner + get_url: + dest: /usr/local/bin/gitlab-runner + url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" + owner: gitlab-runner + group: gitlab-runner + mode: u=rwx,g=rwx,o=rx + + - name: Register the gitlab-runner + command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" + + - name: Install the gitlab-runner service using its own functionality + command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner + register: gitlab_runner_install_service_result + failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" + + - name: Enable the gitlab-runner service + service: + name: gitlab-runner + state: started + enabled: yes diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template new file mode 100644 index 0000000000..e48089761f --- /dev/null +++ b/scripts/ci/setup/vars.yml.template @@ -0,0 +1,12 @@ +# The version of the gitlab-runner to use +gitlab_runner_version: 13.12.0 +# The URL of the gitlab server to use, usually https://gitlab.com unless you're +# using a private GitLab instance +gitlab_runner_server_url: https://gitlab.com +# A mapping of the ansible to gitlab architecture nomenclature +ansible_to_gitlab_arch: + x86_64: amd64 + aarch64: arm64 + s390x: s390x +# A unique token made available by GitLab to your project for registering runners +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN From patchwork Wed Jun 30 01:26:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 12350975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DB14C11F69 for ; Wed, 30 Jun 2021 01:28:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9F7EC61D85 for ; Wed, 30 Jun 2021 01:28:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F7EC61D85 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:46850 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lyP1F-0006qX-Q5 for qemu-devel@archiver.kernel.org; Tue, 29 Jun 2021 21:28:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44898) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyOzw-000476-4b for qemu-devel@nongnu.org; Tue, 29 Jun 2021 21:26:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:38685) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyOzs-0004Ua-Ls for qemu-devel@nongnu.org; Tue, 29 Jun 2021 21:26:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625016395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hhhKaXg4xVWVkGgP+8LO1sCXiVIGAsi0mHlXv8Wh9ds=; b=CpwZGrWiYFRTAsQfN2UvnUykcWvJGw81vG7M0HjzBSwwZzLTVexgelyPAIQnktaBx/Fa7W 0kZrAdOR7hlUZ23EH3Et0n6lB9VqsH/XtWVE3PGpDhNJrAlWlE5Z71383FhrC/blGkxjAX fdWU5FDJdruwVUgs+DKsIuJP8ng9O8M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-444-VK5ZxAaVPveyzKxCe9NkcQ-1; Tue, 29 Jun 2021 21:26:34 -0400 X-MC-Unique: VK5ZxAaVPveyzKxCe9NkcQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3BC3E1084F54; Wed, 30 Jun 2021 01:26:33 +0000 (UTC) Received: from fedora.hsd1.nh.comcast.net (ovpn-114-132.rdu2.redhat.com [10.10.114.132]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C9D2C5C1D1; Wed, 30 Jun 2021 01:26:31 +0000 (UTC) From: Cleber Rosa To: =?utf-8?q?Alex_Benn=C3=A9e?= , Peter Maydell , qemu-devel@nongnu.org Subject: [PATCH v7 4/4] Jobs based on custom runners: add job definitions for QEMU's machines Date: Tue, 29 Jun 2021 21:26:19 -0400 Message-Id: <20210630012619.115262-5-crosa@redhat.com> In-Reply-To: <20210630012619.115262-1-crosa@redhat.com> References: <20210630012619.115262-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.435, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?utf-8?q?Da?= =?utf-8?q?niel_P_=2E_Berrang=C3=A9?= , Eduardo Habkost , Erik Skultety , Stefan Hajnoczi , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Wainer dos Santos Moschetta , Willian Rampazzo , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Beraldo Leal Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The QEMU project has two machines (aarch64 and s390x) that can be used for jobs that do build and run tests. This introduces those jobs, which are a mapping of custom scripts used for the same purpose. Signed-off-by: Cleber Rosa Reviewed-by: Willian Rampazzo Reviewed-by: Wainer dos Santos Moschetta --- .gitlab-ci.d/custom-runners.yml | 208 ++++++++++++++++++++++++++++++++ 1 file changed, 208 insertions(+) diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml index a07b27384c..061d3cdfed 100644 --- a/.gitlab-ci.d/custom-runners.yml +++ b/.gitlab-ci.d/custom-runners.yml @@ -12,3 +12,211 @@ # guarantees a fresh repository on each job run. variables: GIT_STRATEGY: clone + +# All ubuntu-18.04 jobs should run successfully in an environment +# setup by the scripts/ci/setup/build-environment.yml task +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04" +ubuntu-18.04-s390x-all-linux-static: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763 + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages + - mkdir build + - cd build + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + - make --output-sync -j`nproc` check-tcg V=1 + +ubuntu-18.04-s390x-all: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-18.04-s390x-alldbg: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --enable-debug --disable-libssh + - make clean + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-18.04-s390x-clang: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + when: manual + script: + - mkdir build + - cd build + - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-18.04-s390x-tci: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --enable-tcg-interpreter + - make --output-sync -j`nproc` + +ubuntu-18.04-s390x-notcg: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + when: manual + script: + - mkdir build + - cd build + - ../configure --disable-libssh --disable-tcg + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +# All ubuntu-20.04 jobs should run successfully in an environment +# setup by the scripts/ci/setup/qemu/build-environment.yml task +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04" +ubuntu-20.04-aarch64-all-linux-static: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763 + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages + - mkdir build + - cd build + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + - make --output-sync -j`nproc` check-tcg V=1 + +ubuntu-20.04-aarch64-all: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-20.04-aarch64-alldbg: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --enable-debug --disable-libssh + - make clean + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-20.04-aarch64-clang: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + when: manual + script: + - mkdir build + - cd build + - ../configure --disable-libssh --cc=clang-10 --cxx=clang++-10 --enable-sanitizers + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-20.04-aarch64-tci: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --enable-tcg-interpreter + - make --output-sync -j`nproc` + +ubuntu-20.04-aarch64-notcg: + allow_failure: true + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + when: manual + script: + - mkdir build + - cd build + - ../configure --disable-libssh --disable-tcg + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1