From patchwork Fri Feb 19 21:58:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 12096249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91318C433E6 for ; Fri, 19 Feb 2021 22:00:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E80C564E24 for ; Fri, 19 Feb 2021 22:00:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E80C564E24 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34014 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lDDof-0002LR-PQ for qemu-devel@archiver.kernel.org; Fri, 19 Feb 2021 17:00:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42682) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lDDnh-00015R-V7 for qemu-devel@nongnu.org; Fri, 19 Feb 2021 16:59:03 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:32493) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lDDnX-0004m2-UR for qemu-devel@nongnu.org; Fri, 19 Feb 2021 16:59:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1613771931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7oaHANp0svNorVrNpEGNCcyUWOrGQfZAV8MSaKvs3pU=; b=b5JWKGYc06g5zNl6Ee8AkAcr2iuFvjyQ9AGteuqvt4hmDnabQxwm2w3hhSltR3eMHQhaOH p8ier4H6IIKtX/mVqKhQpGS7dIw18wPeUWUWSqDU+HEE8htFfTLrn0cbc+JVHeFnaZkmRG iWAQ5JpvNlcl6US1vKXSbKEtLAZAetw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-411-SnKYtp2kO1SzUjwPAf5OJg-1; Fri, 19 Feb 2021 16:58:49 -0500 X-MC-Unique: SnKYtp2kO1SzUjwPAf5OJg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 532AB1020C22; Fri, 19 Feb 2021 21:58:48 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-114-28.rdu2.redhat.com [10.10.114.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CC8BE708E2; Fri, 19 Feb 2021 21:58:46 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org, =?utf-8?q?Alex_Benn=C3=A9e?= , Peter Maydell Subject: [PATCH v5 2/4] Jobs based on custom runners: build environment docs and playbook Date: Fri, 19 Feb 2021 16:58:36 -0500 Message-Id: <20210219215838.752547-3-crosa@redhat.com> In-Reply-To: <20210219215838.752547-1-crosa@redhat.com> References: <20210219215838.752547-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?utf-8?q?Da?= =?utf-8?q?niel_P_=2E_Berrang=C3=A9?= , Eduardo Habkost , Erik Skultety , Stefan Hajnoczi , Andrea Bolognani , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Beraldo Leal Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" To run basic jobs on custom runners, the environment needs to be properly set up. The most common requirement is having the right packages installed. The playbook introduced here covers the QEMU's project s390x and aarch64 machines. At the time this is being proposed, those machines have already had this playbook applied to them. Signed-off-by: Cleber Rosa --- docs/devel/ci.rst | 30 ++++++++++ scripts/ci/setup/build-environment.yml | 76 ++++++++++++++++++++++++++ scripts/ci/setup/inventory | 1 + 3 files changed, 107 insertions(+) create mode 100644 scripts/ci/setup/build-environment.yml create mode 100644 scripts/ci/setup/inventory diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst index 585b7bf4b8..a556558435 100644 --- a/docs/devel/ci.rst +++ b/docs/devel/ci.rst @@ -26,3 +26,33 @@ gitlab-runner, is called a "custom runner". The GitLab CI jobs definition for the custom runners are located under:: .gitlab-ci.d/custom-runners.yml + +Machine Setup Howto +------------------- + +For all Linux based systems, the setup can be mostly automated by the +execution of two Ansible playbooks. Start by adding your machines to +the ``inventory`` file under ``scripts/ci/setup``, such as this:: + + fully.qualified.domain + other.machine.hostname + +You may need to set some variables in the inventory file itself. One +very common need is to tell Ansible to use a Python 3 interpreter on +those hosts. This would look like:: + + fully.qualified.domain ansible_python_interpreter=/usr/bin/python3 + other.machine.hostname ansible_python_interpreter=/usr/bin/python3 + +Build environment +~~~~~~~~~~~~~~~~~ + +The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will +set up machines with the environment needed to perform builds and run +QEMU tests. It covers a number of different Linux distributions and +FreeBSD. + +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory build-environment.yml diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml new file mode 100644 index 0000000000..0197e0a48b --- /dev/null +++ b/scripts/ci/setup/build-environment.yml @@ -0,0 +1,76 @@ +--- +- name: Installation of basic packages to build QEMU + hosts: all + tasks: + - name: Update apt cache + apt: + update_cache: yes + when: + - ansible_facts['distribution'] == 'Ubuntu' + + - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04 + package: + name: + # Originally from tests/docker/dockerfiles/ubuntu1804.docker + - ccache + - clang + - gcc + - gettext + - git + - glusterfs-common + - libaio-dev + - libattr1-dev + - libbrlapi-dev + - libbz2-dev + - libcacard-dev + - libcap-ng-dev + - libcurl4-gnutls-dev + - libdrm-dev + - libepoxy-dev + - libfdt-dev + - libgbm-dev + - libgtk-3-dev + - libibverbs-dev + - libiscsi-dev + - libjemalloc-dev + - libjpeg-turbo8-dev + - liblzo2-dev + - libncurses5-dev + - libncursesw5-dev + - libnfs-dev + - libnss3-dev + - libnuma-dev + - libpixman-1-dev + - librados-dev + - librbd-dev + - librdmacm-dev + - libsasl2-dev + - libsdl2-dev + - libseccomp-dev + - libsnappy-dev + - libspice-protocol-dev + - libssh-dev + - libusb-1.0-0-dev + - libusbredirhost-dev + - libvdeplug-dev + - libvte-2.91-dev + - libzstd-dev + - make + - ninja-build + - python3-yaml + - python3-sphinx + - sparse + - xfslibs-dev + state: present + when: + - ansible_facts['distribution'] == 'Ubuntu' + + - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x + package: + name: + - libspice-server-dev + - libxen-dev + state: present + when: + - ansible_facts['distribution'] == 'Ubuntu' + - ansible_facts['architecture'] != 's390x' diff --git a/scripts/ci/setup/inventory b/scripts/ci/setup/inventory new file mode 100644 index 0000000000..2fbb50c4a8 --- /dev/null +++ b/scripts/ci/setup/inventory @@ -0,0 +1 @@ +localhost