From patchwork Mon Oct 31 17:52:31 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony PERARD X-Patchwork-Id: 9406177 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 91E7260721 for ; Mon, 31 Oct 2016 17:55:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8716F2934E for ; Mon, 31 Oct 2016 17:55:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7B8AD29381; Mon, 31 Oct 2016 17:55:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BD8742934E for ; Mon, 31 Oct 2016 17:55:31 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c1GlN-0001nL-Cx; Mon, 31 Oct 2016 17:52:49 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c1GlM-0001mn-5e for xen-devel@lists.xen.org; Mon, 31 Oct 2016 17:52:48 +0000 Received: from [193.109.254.147] by server-6.bemta-6.messagelabs.com id 8F/61-12520-FE487185; Mon, 31 Oct 2016 17:52:47 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrAIsWRWlGSWpSXmKPExsXitHRDpO67FvE Igwlz9SyWfFzM4sDocXT3b6YAxijWzLyk/IoE1oye5XUFE2Mr1m28wt7A+Ma5i5GDQ0LAX+LL DIcuRk4ONgEDiRXTrzKC2CICshKru+awg9jMAhESL26sYgOxhQViJR4vOsYEYrMIqEr0n/3BD GLzCthJPF7eAVYjISAvsfD8EbAaTgF7iReX77OC2EJANfOvQcSFBNQkbixcxgLRKyhxcuYTFo hdEhIHX7xghpjDLXH79FTmCYx8s5CUzUJStoCRaRWjenFqUVlqka6FXlJRZnpGSW5iZo6uoYG ZXm5qcXFiempOYlKxXnJ+7iZGYDgxAMEOxtmX/Q8xSnIwKYnyfg0TjxDiS8pPqcxILM6ILyrN SS0+xCjDwaEkwXu9GSgnWJSanlqRlpkDDGyYtAQHj5II70mQNG9xQWJucWY6ROoUozHHnlmvH jBxdCz69IBJiCUvPy9VShyiVACkNKM0D24QLOIuMcpKCfMyAp0mxFOQWpSbWYIq/4pRnINRSZ jXExi/QjyZeSVw+14BncIEdEpaoQjIKSWJCCmpBsYOwbfWLEdSFDP61yW9tTi9kGf3GlGRyZ2 zrjJ+qnx24+IWox3v4/7s5ApxVPrI1Mb4edZ7ib2ONTxhu00OlO+xsejh37hewG8V51G3wu0M jef/zozhvFy3RvZaS3iywcVdQs9tv7xP89bbNYl9YmqJuUT5WfMfzbtWfFx2Y8N8Qx1G4+TAv 21KLMUZiYZazEXFiQC0cx0bswIAAA== X-Env-Sender: prvs=105888baf=anthony.perard@citrix.com X-Msg-Ref: server-2.tower-27.messagelabs.com!1477936364!16278293!2 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.0.13; banners=-,-,- X-VirusChecked: Checked Received: (qmail 34871 invoked from network); 31 Oct 2016 17:52:46 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-2.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 31 Oct 2016 17:52:46 -0000 X-IronPort-AV: E=Sophos;i="5.31,428,1473120000"; d="scan'208";a="387012118" From: Anthony PERARD To: Date: Mon, 31 Oct 2016 17:52:31 +0000 Message-ID: <20161031175233.32320-2-anthony.perard@citrix.com> X-Mailer: git-send-email 2.10.2 In-Reply-To: <20161031175233.32320-1-anthony.perard@citrix.com> References: <20161031175233.32320-1-anthony.perard@citrix.com> MIME-Version: 1.0 X-DLP: MIA2 Cc: Anthony PERARD , Ian Jackson Subject: [Xen-devel] [OSSTEST PATCH v6 1/3] ts-openstack-deploy: Deploy OpenStack on a host with devstack X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This script installs any necessary packages and clones all of the OpenStack trees which are used by devstack to deploy OpenStack. Signed-off-by: Anthony PERARD --- Changes in V6: - rebased - fix issues due to new debian and newer devstack: - add missing libvirt group - switch back to old nova-network instead of neutron - have devstack use 'service' instead of 'systemctl' to restart services Only change in V5: - edit stackrc from devstack file to change the hardcoded path DEST No change in V4: - acked Change in V3: - Use host as argument to run the job. - Use selectjob() and get rid of the unused $gho. - Use target_jobdir() instead of builddirsprops(). - Remove GIT_BASE from devstack config. - Rename the script to ts-openstack-deploy (from ts-openstack-devstack). --- sg-run-job | 5 + ts-openstack-deploy | 338 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 343 insertions(+) create mode 100755 ts-openstack-deploy diff --git a/sg-run-job b/sg-run-job index 9f8d003..5146dd1 100755 --- a/sg-run-job +++ b/sg-run-job @@ -474,6 +474,11 @@ proc run-job/test-rumprun {} { ts-guest-destroy-hard host $g + } +proc need-hosts/test-devstack {} { return host } +proc run-job/test-devstack {} { + run-ts . = ts-openstack-deploy host +} + if {[file exists sg-run-job-adhoc]} { source sg-run-job-adhoc } diff --git a/ts-openstack-deploy b/ts-openstack-deploy new file mode 100755 index 0000000..5758f82 --- /dev/null +++ b/ts-openstack-deploy @@ -0,0 +1,338 @@ +#!/usr/bin/perl +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2015 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see . + +use strict qw(vars); +use Osstest; +use Osstest::TestSupport; +use Osstest::BuildSupport; + +tsreadconfig(); +our ($whhost) = @ARGV; +$whhost ||= 'host'; +our $ho = selecthost($whhost); +our $builddir = target_jobdir($ho); + +sub tgt_init (); + +sub packages () { + # Install open-iscsi ahead of devstack ... + target_install_packages($ho, qw(git sudo open-iscsi)); + # ... and start open-iscsi to have /etc/iscsi/initiatorname.iscsi + # generated. This is done on install on Ubuntu. + target_cmd_root($ho, 'service open-iscsi start'); +} + +sub checkout () { + prepbuilddirs(); + build_clone($ho, 'cinder', $builddir, 'cinder'); + build_clone($ho, 'devstack', $builddir, 'devstack'); + build_clone($ho, 'glance', $builddir, 'glance'); + build_clone($ho, 'keystone', $builddir, 'keystone'); + build_clone($ho, 'nova', $builddir, 'nova'); + build_clone($ho, 'requirements', $builddir, 'requirements'); + build_clone($ho, 'tempest', $builddir, 'tempest'); + + my $vg = target_choose_vg($ho, 10*1024); # 10GB + target_putfilecontents_stash($ho, 60, <) { + if (m/^DEST=\/opt\/stack$/) { + s/DEST=\/opt\/stack/DEST=$builddir/; + } + print EO or die $!; + } + }); + + # libvirt is already installed, but not as a package, so avoid installation of + # the libvirt package with devstack + target_editfile($ho, "$builddir/devstack/files/debs/nova", sub { + while () { + next if m/.*libvirt.*/; + print EO or die $!; + } + }); + target_editfile($ho, "$builddir/devstack/lib/nova_plugins/functions-libvirt", sub { + while () { + next if m/install_package.*libvirt.*/; + print EO or die $!; + } + }); + + # devstack blindly assume that systemd is used if systemctl is present + target_editfile($ho, "$builddir/devstack/functions-common", sub { + while () { + if (m%\[ -x /bin/systemctl%) { + s%\[ -x /bin/systemctl \]%false% + } + print EO or die $!; + } + }); + + # OpenStack needs access to libvirt from a user. + target_cmd_root($ho, </dev/null; then + groupadd libvirt + fi + cat >> /etc/libvirt/libvirtd.conf </dev/null + RETVAL=$? + if [ "$RETVAL" -ne 0 ] ; then + log_end_msg 1 + exit 1 + else + log_end_msg 0 + fi + # Put tgtd into "offline" state until all the targets are configured. + # We don't want initiators to (re)connect and fail the connection + # if it's not ready. + tgtadm --op update --mode sys --name State -v offline + # Configure the targets. + tgt-admin -e -c $TGTD_CONFIG + # Put tgtd into "ready" state. + tgtadm --op update --mode sys --name State -v ready +} + +stop() +{ + if [ "$RUNLEVEL" == 0 -o "$RUNLEVEL" == 6 ] ; then + forcedstop + fi + log_daemon_msg "Stopping $DESC" "$NAME" + # Remove all targets. It only removes targets which are not in use. + tgt-admin --update ALL -c /dev/null &>/dev/null + # tgtd will exit if all targets were removed + tgtadm --op delete --mode system &>/dev/null + RETVAL=$? + if [ "$RETVAL" -eq 107 ] ; then + if [ "$TASK" != "restart" ] ; then + log_end_msg 1 + exit 1 + else + log_end_msg 0 + fi + elif [ "$RETVAL" -ne 0 ] ; then + log_end_msg 1 + echo "Some initiators are still connected - could not stop tgtd" + exit 2 + else + log_end_msg 0 + fi + echo -n +} + +forcedstop() +{ + # NOTE: Forced shutdown of the iscsi target may cause data corruption + # for initiators that are connected. + echo "Force-stopping target framework daemon" + # Offline everything first. May be needed if we're rebooting, but + # expect the initiators to reconnect cleanly when we boot again + # (i.e. we don't want them to reconnect to a tgtd which is still + # working, but the target is gone). + tgtadm --op update --mode sys --name State -v offline &>/dev/null + RETVAL=$? + if [ "$RETVAL" -eq 107 ] ; then + echo "tgtd is not running" + [ "$TASK" != "restart" ] && exit 1 + else + tgt-admin --offline ALL + # Remove all targets, even if they are still in use. + tgt-admin --update ALL -c /dev/null -f + # It will shut down tgtd only after all targets were removed. + tgtadm --op delete --mode system + RETVAL=$? + if [ "$RETVAL" -ne 0 ] ; then + echo "Failed to shutdown tgtd" + exit 1 + fi + fi + echo -n +} + +reload() +{ + log_daemon_msg "Reloading configuration of $DESC" "$NAME" + # Update configuration for targets. Only targets which + # are not in use will be updated. + tgt-admin --update ALL -c $TGTD_CONFIG &>/dev/null + RETVAL=$? + if [ "$RETVAL" -eq 107 ] ; then + log_end_msg 1 + echo "tgtd is not running" + exit 1 + fi + log_end_msg 0 +} + +forcedreload() +{ + log_daemon_msg "Forced-reload configuration of $DESC" "$NAME" + # Update configuration for targets, even those in use. + tgt-admin --update ALL -f -c $TGTD_CONFIG &>/dev/null + RETVAL=$? + if [ "$RETVAL" -eq 107 ] ; then + log_end_msg 1 + echo "tgtd is not running" + exit 1 + else + log_end_msg 0 + fi +} + +status() +{ + # Don't name this script "tgtd"... + TGTD_PROC=$(ps -C tgtd | grep -c tgtd) + if [ "$TGTD_PROC" -eq 2 ] ; then + echo "tgtd is running. Run 'tgt-admin -s' to see detailed target info." + else + echo "tgtd is NOT running." + fi +} + +case $1 in + start) + start + ;; + stop) + stop + ;; + forcedstop) + forcedstop + ;; + restart) + TASK=restart + stop && start + ;; + forcedrestart) + TASK=restart + forcedstop && start + ;; + reload) + reload + ;; + force-reload) + forcedreload + ;; + status) + status + ;; + *) + echo "Usage: $0 {start|stop|forcedstop|restart|forcedrestart|reload|force-reload|status}" + exit 2 + ;; +esac +END +}