| Message ID | 20210608031425.833536-4-crosa@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | GitLab Custom Runners and Jobs (was: QEMU Gating CI) | expand |
Hi, On 6/8/21 12:14 AM, Cleber Rosa wrote: > To have the jobs dispatched to custom runners, gitlab-runner must > be installed, active as a service and properly configured. The > variables file and playbook introduced here should help with those > steps. > > The playbook introduced here covers the Linux distributions and > has been primarily tested on OS/machines that the QEMU project > has available to act as runners, namely: > > * Ubuntu 20.04 on aarch64 > * Ubuntu 18.04 on s390x > > But, it should work on all other Linux distributions. Earlier > versions were tested on FreeBSD too, so chances of success are > high. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > --- > docs/devel/ci.rst | 57 ++++++++++++++++++++++++++++ > scripts/ci/setup/.gitignore | 1 + > scripts/ci/setup/gitlab-runner.yml | 61 ++++++++++++++++++++++++++++++ > scripts/ci/setup/vars.yml.template | 12 ++++++ > 4 files changed, 131 insertions(+) > create mode 100644 scripts/ci/setup/.gitignore > create mode 100644 scripts/ci/setup/gitlab-runner.yml > create mode 100644 scripts/ci/setup/vars.yml.template > > diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst > index 35c6b5e269..bbd89e54d7 100644 > --- a/docs/devel/ci.rst > +++ b/docs/devel/ci.rst > @@ -56,3 +56,60 @@ To run the playbook, execute:: > > cd scripts/ci/setup > ansible-playbook -i inventory build-environment.yml > + > +gitlab-runner setup and registration > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The gitlab-runner agent needs to be installed on each machine that > +will run jobs. The association between a machine and a GitLab project > +happens with a registration token. To find the registration token for > +your repository/project, navigate on GitLab's web UI to: > + > + * Settings (the gears like icon), then > + * CI/CD, then > + * Runners, and click on the "Expand" button, then > + * Under "Set up a specific Runner manually", look for the value under > + "Use the following registration token during setup" > + > +Copy the ``scripts/ci/setup/vars.yml.template`` file to > +``scripts/ci/setup/vars.yml``. Then, set the > +``gitlab_runner_registration_token`` variable to the value obtained > +earlier. > + > +.. note:: gitlab-runner is not available from the standard location > + for all OS and architectures combinations. For some systems, > + a custom build may be necessary. Some builds are avaiable > + at https://cleber.fedorapeople.org/gitlab-runner/ and this > + URI may be used as a value on ``vars.yml`` I think you can remove the information about the gitlab-running being not available for some systems. > + > +To run the playbook, execute:: > + > + cd scripts/ci/setup > + ansible-playbook -i inventory gitlab-runner.yml > + > +Following the registration, it's necessary to configure the runner tags, > +and optionally other configurations on the GitLab UI. Navigate to: > + > + * Settings (the gears like icon), then > + * CI/CD, then > + * Runners, and click on the "Expand" button, then > + * "Runners activated for this project", then > + * Click on the "Edit" icon (next to the "Lock" Icon) > + > +Under tags, add values matching the jobs a runner should run. For a > +Ubuntu 20.04 aarch64 system, the tags should be set as:: > + > + ubuntu_20.04,aarch64 Also users no longer need manually create the tags. Remaining of the file looks good to me. > + > +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` > +would contain:: > + > + ubuntu-20.04-aarch64-all: > + tags: > + - ubuntu_20.04 > + - aarch64 > + > +It's also recommended to: > + > + * increase the "Maximum job timeout" to something like ``2h`` > + * give it a better Description > diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore > new file mode 100644 > index 0000000000..f112d05dd0 > --- /dev/null > +++ b/scripts/ci/setup/.gitignore > @@ -0,0 +1 @@ > +vars.yml > \ No newline at end of file > diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml > new file mode 100644 > index 0000000000..98dab92bb5 > --- /dev/null > +++ b/scripts/ci/setup/gitlab-runner.yml > @@ -0,0 +1,61 @@ > +--- > +- name: Installation of gitlab-runner > + hosts: all > + vars_files: > + - vars.yml > + tasks: > + - debug: > + msg: 'Checking for a valid GitLab registration token' > + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" > + > + - name: Create a group for the gitlab-runner service > + group: > + name: gitlab-runner > + > + - name: Create a user for the gitlab-runner service > + user: > + user: gitlab-runner > + group: gitlab-runner > + comment: GitLab Runner > + home: /home/gitlab-runner > + shell: /bin/bash > + > + - name: Remove the .bash_logout file when on Ubuntu systems > + file: > + path: /home/gitlab-runner/.bash_logout > + state: absent > + when: "ansible_facts['distribution'] == 'Ubuntu'" > + > + - name: Set the Operating System for gitlab-runner > + set_fact: > + gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}" > + - debug: > + msg: gitlab-runner OS is {{ gitlab_runner_os }} > + > + - name: Set the architecture for gitlab-runner > + set_fact: > + gitlab_runner_arch: "{{ ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}" > + - debug: > + msg: gitlab-runner arch is {{ gitlab_runner_arch }} > + > + - name: Download the matching gitlab-runner > + get_url: > + dest: /usr/local/bin/gitlab-runner > + url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" > + owner: gitlab-runner > + group: gitlab-runner > + mode: u=rwx,g=rwx,o=rx > + > + - name: Register the gitlab-runner > + command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" > + > + - name: Install the gitlab-runner service using its own functionality > + command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner > + register: gitlab_runner_install_service_result > + failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" > + > + - name: Enable the gitlab-runner service > + service: > + name: gitlab-runner > + state: started > + enabled: yes > diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template > new file mode 100644 > index 0000000000..e48089761f > --- /dev/null > +++ b/scripts/ci/setup/vars.yml.template > @@ -0,0 +1,12 @@ > +# The version of the gitlab-runner to use > +gitlab_runner_version: 13.12.0 > +# The URL of the gitlab server to use, usually https://gitlab.com unless you're > +# using a private GitLab instance > +gitlab_runner_server_url: https://gitlab.com > +# A mapping of the ansible to gitlab architecture nomenclature > +ansible_to_gitlab_arch: > + x86_64: amd64 > + aarch64: arm64 > + s390x: s390x > +# A unique token made available by GitLab to your project for registering runners > +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN
On Tue, Jun 8, 2021 at 12:14 AM Cleber Rosa <crosa@redhat.com> wrote: > > To have the jobs dispatched to custom runners, gitlab-runner must > be installed, active as a service and properly configured. The > variables file and playbook introduced here should help with those > steps. > > The playbook introduced here covers the Linux distributions and > has been primarily tested on OS/machines that the QEMU project > has available to act as runners, namely: > > * Ubuntu 20.04 on aarch64 > * Ubuntu 18.04 on s390x > > But, it should work on all other Linux distributions. Earlier > versions were tested on FreeBSD too, so chances of success are > high. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > --- > docs/devel/ci.rst | 57 ++++++++++++++++++++++++++++ > scripts/ci/setup/.gitignore | 1 + > scripts/ci/setup/gitlab-runner.yml | 61 ++++++++++++++++++++++++++++++ > scripts/ci/setup/vars.yml.template | 12 ++++++ > 4 files changed, 131 insertions(+) > create mode 100644 scripts/ci/setup/.gitignore > create mode 100644 scripts/ci/setup/gitlab-runner.yml > create mode 100644 scripts/ci/setup/vars.yml.template > > diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst > index 35c6b5e269..bbd89e54d7 100644 > --- a/docs/devel/ci.rst > +++ b/docs/devel/ci.rst > @@ -56,3 +56,60 @@ To run the playbook, execute:: > > cd scripts/ci/setup > ansible-playbook -i inventory build-environment.yml > + > +gitlab-runner setup and registration > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The gitlab-runner agent needs to be installed on each machine that > +will run jobs. The association between a machine and a GitLab project > +happens with a registration token. To find the registration token for > +your repository/project, navigate on GitLab's web UI to: > + > + * Settings (the gears like icon), then * Settings (the gears like icon in the end of the left menu), then I took some time to find it as it was hidden at the end of the scrolling. > + * CI/CD, then > + * Runners, and click on the "Expand" button, then > + * Under "Set up a specific Runner manually", look for the value under > + "Use the following registration token during setup" For me, it shows: "And this registration token:" > + > +Copy the ``scripts/ci/setup/vars.yml.template`` file to > +``scripts/ci/setup/vars.yml``. Then, set the > +``gitlab_runner_registration_token`` variable to the value obtained > +earlier. > + > +.. note:: gitlab-runner is not available from the standard location > + for all OS and architectures combinations. For some systems, > + a custom build may be necessary. Some builds are avaiable If you keep this block (see comment from Wainer), s/avaiable/available/ > + at https://cleber.fedorapeople.org/gitlab-runner/ and this > + URI may be used as a value on ``vars.yml`` > + > +To run the playbook, execute:: > + > + cd scripts/ci/setup > + ansible-playbook -i inventory gitlab-runner.yml > + > +Following the registration, it's necessary to configure the runner tags, > +and optionally other configurations on the GitLab UI. Navigate to: > + > + * Settings (the gears like icon), then > + * CI/CD, then > + * Runners, and click on the "Expand" button, then > + * "Runners activated for this project", then > + * Click on the "Edit" icon (next to the "Lock" Icon) > + > +Under tags, add values matching the jobs a runner should run. For a > +Ubuntu 20.04 aarch64 system, the tags should be set as:: > + > + ubuntu_20.04,aarch64 > + > +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` > +would contain:: > + > + ubuntu-20.04-aarch64-all: > + tags: > + - ubuntu_20.04 > + - aarch64 > + > +It's also recommended to: > + > + * increase the "Maximum job timeout" to something like ``2h`` > + * give it a better Description > diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore > new file mode 100644 > index 0000000000..f112d05dd0 > --- /dev/null > +++ b/scripts/ci/setup/.gitignore > @@ -0,0 +1 @@ > +vars.yml > \ No newline at end of file > diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml > new file mode 100644 > index 0000000000..98dab92bb5 > --- /dev/null > +++ b/scripts/ci/setup/gitlab-runner.yml > @@ -0,0 +1,61 @@ > +--- > +- name: Installation of gitlab-runner > + hosts: all > + vars_files: > + - vars.yml > + tasks: > + - debug: > + msg: 'Checking for a valid GitLab registration token' > + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" > + > + - name: Create a group for the gitlab-runner service > + group: > + name: gitlab-runner > + > + - name: Create a user for the gitlab-runner service > + user: > + user: gitlab-runner > + group: gitlab-runner > + comment: GitLab Runner > + home: /home/gitlab-runner > + shell: /bin/bash > + > + - name: Remove the .bash_logout file when on Ubuntu systems > + file: > + path: /home/gitlab-runner/.bash_logout > + state: absent > + when: "ansible_facts['distribution'] == 'Ubuntu'" > + > + - name: Set the Operating System for gitlab-runner > + set_fact: > + gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}" > + - debug: > + msg: gitlab-runner OS is {{ gitlab_runner_os }} > + > + - name: Set the architecture for gitlab-runner > + set_fact: > + gitlab_runner_arch: "{{ ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}" > + - debug: > + msg: gitlab-runner arch is {{ gitlab_runner_arch }} > + > + - name: Download the matching gitlab-runner > + get_url: > + dest: /usr/local/bin/gitlab-runner > + url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" > + owner: gitlab-runner > + group: gitlab-runner > + mode: u=rwx,g=rwx,o=rx > + > + - name: Register the gitlab-runner > + command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" > + > + - name: Install the gitlab-runner service using its own functionality > + command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner > + register: gitlab_runner_install_service_result > + failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" > + > + - name: Enable the gitlab-runner service > + service: > + name: gitlab-runner > + state: started > + enabled: yes > diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template > new file mode 100644 > index 0000000000..e48089761f > --- /dev/null > +++ b/scripts/ci/setup/vars.yml.template > @@ -0,0 +1,12 @@ > +# The version of the gitlab-runner to use > +gitlab_runner_version: 13.12.0 > +# The URL of the gitlab server to use, usually https://gitlab.com unless you're > +# using a private GitLab instance > +gitlab_runner_server_url: https://gitlab.com > +# A mapping of the ansible to gitlab architecture nomenclature > +ansible_to_gitlab_arch: > + x86_64: amd64 > + aarch64: arm64 > + s390x: s390x > +# A unique token made available by GitLab to your project for registering runners > +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN > -- > 2.25.4 > With my suggestions and Wainer's: Reviewed-by: Willian Rampazzo <willianr@redhat.com> Tested-by: Willian Rampazzo <willianr@redhat.com>
On 08/06/2021 05.14, Cleber Rosa wrote: > To have the jobs dispatched to custom runners, gitlab-runner must > be installed, active as a service and properly configured. The > variables file and playbook introduced here should help with those > steps. > > The playbook introduced here covers the Linux distributions and > has been primarily tested on OS/machines that the QEMU project > has available to act as runners, namely: > > * Ubuntu 20.04 on aarch64 > * Ubuntu 18.04 on s390x > > But, it should work on all other Linux distributions. Earlier > versions were tested on FreeBSD too, so chances of success are > high. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > --- > docs/devel/ci.rst | 57 ++++++++++++++++++++++++++++ > scripts/ci/setup/.gitignore | 1 + > scripts/ci/setup/gitlab-runner.yml | 61 ++++++++++++++++++++++++++++++ > scripts/ci/setup/vars.yml.template | 12 ++++++ > 4 files changed, 131 insertions(+) > create mode 100644 scripts/ci/setup/.gitignore > create mode 100644 scripts/ci/setup/gitlab-runner.yml > create mode 100644 scripts/ci/setup/vars.yml.template [...] > diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore > new file mode 100644 > index 0000000000..f112d05dd0 > --- /dev/null > +++ b/scripts/ci/setup/.gitignore > @@ -0,0 +1 @@ > +vars.yml > \ No newline at end of file Add a newline, please. > diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml > new file mode 100644 > index 0000000000..98dab92bb5 > --- /dev/null > +++ b/scripts/ci/setup/gitlab-runner.yml > @@ -0,0 +1,61 @@ > +--- > +- name: Installation of gitlab-runner > + hosts: all > + vars_files: > + - vars.yml > + tasks: > + - debug: > + msg: 'Checking for a valid GitLab registration token' > + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" Could you please add a comment at the top of the file or name it differently so that it is clear from a quick glance that this is an ansible playbook? Poeple might later wonder otherwise... Thomas
On Tue, Jun 8, 2021 at 3:04 PM Wainer dos Santos Moschetta <wainersm@redhat.com> wrote: > > Hi, > > On 6/8/21 12:14 AM, Cleber Rosa wrote: > > To have the jobs dispatched to custom runners, gitlab-runner must > > be installed, active as a service and properly configured. The > > variables file and playbook introduced here should help with those > > steps. > > > > The playbook introduced here covers the Linux distributions and > > has been primarily tested on OS/machines that the QEMU project > > has available to act as runners, namely: > > > > * Ubuntu 20.04 on aarch64 > > * Ubuntu 18.04 on s390x > > > > But, it should work on all other Linux distributions. Earlier > > versions were tested on FreeBSD too, so chances of success are > > high. > > > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > > --- > > docs/devel/ci.rst | 57 ++++++++++++++++++++++++++++ > > scripts/ci/setup/.gitignore | 1 + > > scripts/ci/setup/gitlab-runner.yml | 61 ++++++++++++++++++++++++++++++ > > scripts/ci/setup/vars.yml.template | 12 ++++++ > > 4 files changed, 131 insertions(+) > > create mode 100644 scripts/ci/setup/.gitignore > > create mode 100644 scripts/ci/setup/gitlab-runner.yml > > create mode 100644 scripts/ci/setup/vars.yml.template > > > > diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst > > index 35c6b5e269..bbd89e54d7 100644 > > --- a/docs/devel/ci.rst > > +++ b/docs/devel/ci.rst > > @@ -56,3 +56,60 @@ To run the playbook, execute:: > > > > cd scripts/ci/setup > > ansible-playbook -i inventory build-environment.yml > > + > > +gitlab-runner setup and registration > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > > +The gitlab-runner agent needs to be installed on each machine that > > +will run jobs. The association between a machine and a GitLab project > > +happens with a registration token. To find the registration token for > > +your repository/project, navigate on GitLab's web UI to: > > + > > + * Settings (the gears like icon), then > > + * CI/CD, then > > + * Runners, and click on the "Expand" button, then > > + * Under "Set up a specific Runner manually", look for the value under > > + "Use the following registration token during setup" > > + > > +Copy the ``scripts/ci/setup/vars.yml.template`` file to > > +``scripts/ci/setup/vars.yml``. Then, set the > > +``gitlab_runner_registration_token`` variable to the value obtained > > +earlier. > > + > > +.. note:: gitlab-runner is not available from the standard location > > + for all OS and architectures combinations. For some systems, > > + a custom build may be necessary. Some builds are avaiable > > + at https://cleber.fedorapeople.org/gitlab-runner/ and this > > + URI may be used as a value on ``vars.yml`` > I think you can remove the information about the gitlab-running being > not available for some systems. Good catch! > > + > > +To run the playbook, execute:: > > + > > + cd scripts/ci/setup > > + ansible-playbook -i inventory gitlab-runner.yml > > + > > +Following the registration, it's necessary to configure the runner tags, > > +and optionally other configurations on the GitLab UI. Navigate to: > > + > > + * Settings (the gears like icon), then > > + * CI/CD, then > > + * Runners, and click on the "Expand" button, then > > + * "Runners activated for this project", then > > + * Click on the "Edit" icon (next to the "Lock" Icon) > > + > > +Under tags, add values matching the jobs a runner should run. For a > > +Ubuntu 20.04 aarch64 system, the tags should be set as:: > > + > > + ubuntu_20.04,aarch64 > > Also users no longer need manually create the tags. > Of course. But, given that this is a general rule, and there may be OS/architectures for which Ansible may produce tag names one would not expect, I'll ask the user to double check the tags. The text I'm proposing on v7 is: Tags are very important as they are used to route specific jobs to specific types of runners, so it's a good idea to double check that the automatically created tags are consistent with the OS and architecture. For instance, an Ubuntu 20.04 aarch64 system should have tags set as:: ubuntu_20.04,aarch64 > Remaining of the file looks good to me. > Thanks a lot! - Cleber.
On Wed, Jun 9, 2021 at 1:46 PM Willian Rampazzo <wrampazz@redhat.com> wrote: > > On Tue, Jun 8, 2021 at 12:14 AM Cleber Rosa <crosa@redhat.com> wrote: > > > > To have the jobs dispatched to custom runners, gitlab-runner must > > be installed, active as a service and properly configured. The > > variables file and playbook introduced here should help with those > > steps. > > > > The playbook introduced here covers the Linux distributions and > > has been primarily tested on OS/machines that the QEMU project > > has available to act as runners, namely: > > > > * Ubuntu 20.04 on aarch64 > > * Ubuntu 18.04 on s390x > > > > But, it should work on all other Linux distributions. Earlier > > versions were tested on FreeBSD too, so chances of success are > > high. > > > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > > --- > > docs/devel/ci.rst | 57 ++++++++++++++++++++++++++++ > > scripts/ci/setup/.gitignore | 1 + > > scripts/ci/setup/gitlab-runner.yml | 61 ++++++++++++++++++++++++++++++ > > scripts/ci/setup/vars.yml.template | 12 ++++++ > > 4 files changed, 131 insertions(+) > > create mode 100644 scripts/ci/setup/.gitignore > > create mode 100644 scripts/ci/setup/gitlab-runner.yml > > create mode 100644 scripts/ci/setup/vars.yml.template > > > > diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst > > index 35c6b5e269..bbd89e54d7 100644 > > --- a/docs/devel/ci.rst > > +++ b/docs/devel/ci.rst > > @@ -56,3 +56,60 @@ To run the playbook, execute:: > > > > cd scripts/ci/setup > > ansible-playbook -i inventory build-environment.yml > > + > > +gitlab-runner setup and registration > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > > +The gitlab-runner agent needs to be installed on each machine that > > +will run jobs. The association between a machine and a GitLab project > > +happens with a registration token. To find the registration token for > > +your repository/project, navigate on GitLab's web UI to: > > + > > + * Settings (the gears like icon), then > > * Settings (the gears like icon in the end of the left menu), then > ACK. What about: " * Settings (the gears-like icon at the bottom of the left hand side vertical toolbar), then" > I took some time to find it as it was hidden at the end of the scrolling. > > > + * CI/CD, then > > + * Runners, and click on the "Expand" button, then > > + * Under "Set up a specific Runner manually", look for the value under > > + "Use the following registration token during setup" > > For me, it shows: "And this registration token:" > Right, it must have changed. Updating it on v7. > > + > > +Copy the ``scripts/ci/setup/vars.yml.template`` file to > > +``scripts/ci/setup/vars.yml``. Then, set the > > +``gitlab_runner_registration_token`` variable to the value obtained > > +earlier. > > + > > +.. note:: gitlab-runner is not available from the standard location > > + for all OS and architectures combinations. For some systems, > > + a custom build may be necessary. Some builds are avaiable > > If you keep this block (see comment from Wainer), s/avaiable/available/ > I'd rather remove it, but thanks for spotting the typo. > > + at https://cleber.fedorapeople.org/gitlab-runner/ and this > > + URI may be used as a value on ``vars.yml`` > > + > > +To run the playbook, execute:: > > + > > + cd scripts/ci/setup > > + ansible-playbook -i inventory gitlab-runner.yml > > + > > +Following the registration, it's necessary to configure the runner tags, > > +and optionally other configurations on the GitLab UI. Navigate to: > > + > > + * Settings (the gears like icon), then > > + * CI/CD, then > > + * Runners, and click on the "Expand" button, then > > + * "Runners activated for this project", then > > + * Click on the "Edit" icon (next to the "Lock" Icon) > > + > > +Under tags, add values matching the jobs a runner should run. For a > > +Ubuntu 20.04 aarch64 system, the tags should be set as:: > > + > > + ubuntu_20.04,aarch64 > > + > > +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` > > +would contain:: > > + > > + ubuntu-20.04-aarch64-all: > > + tags: > > + - ubuntu_20.04 > > + - aarch64 > > + > > +It's also recommended to: > > + > > + * increase the "Maximum job timeout" to something like ``2h`` > > + * give it a better Description > > diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore > > new file mode 100644 > > index 0000000000..f112d05dd0 > > --- /dev/null > > +++ b/scripts/ci/setup/.gitignore > > @@ -0,0 +1 @@ > > +vars.yml > > \ No newline at end of file > > diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml > > new file mode 100644 > > index 0000000000..98dab92bb5 > > --- /dev/null > > +++ b/scripts/ci/setup/gitlab-runner.yml > > @@ -0,0 +1,61 @@ > > +--- > > +- name: Installation of gitlab-runner > > + hosts: all > > + vars_files: > > + - vars.yml > > + tasks: > > + - debug: > > + msg: 'Checking for a valid GitLab registration token' > > + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" > > + > > + - name: Create a group for the gitlab-runner service > > + group: > > + name: gitlab-runner > > + > > + - name: Create a user for the gitlab-runner service > > + user: > > + user: gitlab-runner > > + group: gitlab-runner > > + comment: GitLab Runner > > + home: /home/gitlab-runner > > + shell: /bin/bash > > + > > + - name: Remove the .bash_logout file when on Ubuntu systems > > + file: > > + path: /home/gitlab-runner/.bash_logout > > + state: absent > > + when: "ansible_facts['distribution'] == 'Ubuntu'" > > + > > + - name: Set the Operating System for gitlab-runner > > + set_fact: > > + gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}" > > + - debug: > > + msg: gitlab-runner OS is {{ gitlab_runner_os }} > > + > > + - name: Set the architecture for gitlab-runner > > + set_fact: > > + gitlab_runner_arch: "{{ ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}" > > + - debug: > > + msg: gitlab-runner arch is {{ gitlab_runner_arch }} > > + > > + - name: Download the matching gitlab-runner > > + get_url: > > + dest: /usr/local/bin/gitlab-runner > > + url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" > > + owner: gitlab-runner > > + group: gitlab-runner > > + mode: u=rwx,g=rwx,o=rx > > + > > + - name: Register the gitlab-runner > > + command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" > > + > > + - name: Install the gitlab-runner service using its own functionality > > + command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner > > + register: gitlab_runner_install_service_result > > + failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" > > + > > + - name: Enable the gitlab-runner service > > + service: > > + name: gitlab-runner > > + state: started > > + enabled: yes > > diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template > > new file mode 100644 > > index 0000000000..e48089761f > > --- /dev/null > > +++ b/scripts/ci/setup/vars.yml.template > > @@ -0,0 +1,12 @@ > > +# The version of the gitlab-runner to use > > +gitlab_runner_version: 13.12.0 > > +# The URL of the gitlab server to use, usually https://gitlab.com unless you're > > +# using a private GitLab instance > > +gitlab_runner_server_url: https://gitlab.com > > +# A mapping of the ansible to gitlab architecture nomenclature > > +ansible_to_gitlab_arch: > > + x86_64: amd64 > > + aarch64: arm64 > > + s390x: s390x > > +# A unique token made available by GitLab to your project for registering runners > > +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN > > -- > > 2.25.4 > > > > With my suggestions and Wainer's: > > Reviewed-by: Willian Rampazzo <willianr@redhat.com> > Tested-by: Willian Rampazzo <willianr@redhat.com> >
On Thu, Jun 10, 2021 at 2:24 AM Thomas Huth <thuth@redhat.com> wrote: > > On 08/06/2021 05.14, Cleber Rosa wrote: > > To have the jobs dispatched to custom runners, gitlab-runner must > > be installed, active as a service and properly configured. The > > variables file and playbook introduced here should help with those > > steps. > > > > The playbook introduced here covers the Linux distributions and > > has been primarily tested on OS/machines that the QEMU project > > has available to act as runners, namely: > > > > * Ubuntu 20.04 on aarch64 > > * Ubuntu 18.04 on s390x > > > > But, it should work on all other Linux distributions. Earlier > > versions were tested on FreeBSD too, so chances of success are > > high. > > > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > > --- > > docs/devel/ci.rst | 57 ++++++++++++++++++++++++++++ > > scripts/ci/setup/.gitignore | 1 + > > scripts/ci/setup/gitlab-runner.yml | 61 ++++++++++++++++++++++++++++++ > > scripts/ci/setup/vars.yml.template | 12 ++++++ > > 4 files changed, 131 insertions(+) > > create mode 100644 scripts/ci/setup/.gitignore > > create mode 100644 scripts/ci/setup/gitlab-runner.yml > > create mode 100644 scripts/ci/setup/vars.yml.template > [...] > > diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore > > new file mode 100644 > > index 0000000000..f112d05dd0 > > --- /dev/null > > +++ b/scripts/ci/setup/.gitignore > > @@ -0,0 +1 @@ > > +vars.yml > > \ No newline at end of file > > Add a newline, please. > Sure! Thanks for spotting that. > > diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml > > new file mode 100644 > > index 0000000000..98dab92bb5 > > --- /dev/null > > +++ b/scripts/ci/setup/gitlab-runner.yml > > @@ -0,0 +1,61 @@ > > +--- > > +- name: Installation of gitlab-runner > > + hosts: all > > + vars_files: > > + - vars.yml > > + tasks: > > + - debug: > > + msg: 'Checking for a valid GitLab registration token' > > + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" > > Could you please add a comment at the top of the file or name it differently > so that it is clear from a quick glance that this is an ansible playbook? > Poeple might later wonder otherwise... > Good point. I'm adding a proper header with copyright / comment. > Thomas > Thanks for the review. - Cleber.
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst index 35c6b5e269..bbd89e54d7 100644 --- a/docs/devel/ci.rst +++ b/docs/devel/ci.rst @@ -56,3 +56,60 @@ To run the playbook, execute:: cd scripts/ci/setup ansible-playbook -i inventory build-environment.yml + +gitlab-runner setup and registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The gitlab-runner agent needs to be installed on each machine that +will run jobs. The association between a machine and a GitLab project +happens with a registration token. To find the registration token for +your repository/project, navigate on GitLab's web UI to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * Under "Set up a specific Runner manually", look for the value under + "Use the following registration token during setup" + +Copy the ``scripts/ci/setup/vars.yml.template`` file to +``scripts/ci/setup/vars.yml``. Then, set the +``gitlab_runner_registration_token`` variable to the value obtained +earlier. + +.. note:: gitlab-runner is not available from the standard location + for all OS and architectures combinations. For some systems, + a custom build may be necessary. Some builds are avaiable + at https://cleber.fedorapeople.org/gitlab-runner/ and this + URI may be used as a value on ``vars.yml`` + +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory gitlab-runner.yml + +Following the registration, it's necessary to configure the runner tags, +and optionally other configurations on the GitLab UI. Navigate to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * "Runners activated for this project", then + * Click on the "Edit" icon (next to the "Lock" Icon) + +Under tags, add values matching the jobs a runner should run. For a +Ubuntu 20.04 aarch64 system, the tags should be set as:: + + ubuntu_20.04,aarch64 + +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` +would contain:: + + ubuntu-20.04-aarch64-all: + tags: + - ubuntu_20.04 + - aarch64 + +It's also recommended to: + + * increase the "Maximum job timeout" to something like ``2h`` + * give it a better Description diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore new file mode 100644 index 0000000000..f112d05dd0 --- /dev/null +++ b/scripts/ci/setup/.gitignore @@ -0,0 +1 @@ +vars.yml \ No newline at end of file diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml new file mode 100644 index 0000000000..98dab92bb5 --- /dev/null +++ b/scripts/ci/setup/gitlab-runner.yml @@ -0,0 +1,61 @@ +--- +- name: Installation of gitlab-runner + hosts: all + vars_files: + - vars.yml + tasks: + - debug: + msg: 'Checking for a valid GitLab registration token' + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" + + - name: Create a group for the gitlab-runner service + group: + name: gitlab-runner + + - name: Create a user for the gitlab-runner service + user: + user: gitlab-runner + group: gitlab-runner + comment: GitLab Runner + home: /home/gitlab-runner + shell: /bin/bash + + - name: Remove the .bash_logout file when on Ubuntu systems + file: + path: /home/gitlab-runner/.bash_logout + state: absent + when: "ansible_facts['distribution'] == 'Ubuntu'" + + - name: Set the Operating System for gitlab-runner + set_fact: + gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}" + - debug: + msg: gitlab-runner OS is {{ gitlab_runner_os }} + + - name: Set the architecture for gitlab-runner + set_fact: + gitlab_runner_arch: "{{ ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}" + - debug: + msg: gitlab-runner arch is {{ gitlab_runner_arch }} + + - name: Download the matching gitlab-runner + get_url: + dest: /usr/local/bin/gitlab-runner + url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" + owner: gitlab-runner + group: gitlab-runner + mode: u=rwx,g=rwx,o=rx + + - name: Register the gitlab-runner + command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" + + - name: Install the gitlab-runner service using its own functionality + command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner + register: gitlab_runner_install_service_result + failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" + + - name: Enable the gitlab-runner service + service: + name: gitlab-runner + state: started + enabled: yes diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template new file mode 100644 index 0000000000..e48089761f --- /dev/null +++ b/scripts/ci/setup/vars.yml.template @@ -0,0 +1,12 @@ +# The version of the gitlab-runner to use +gitlab_runner_version: 13.12.0 +# The URL of the gitlab server to use, usually https://gitlab.com unless you're +# using a private GitLab instance +gitlab_runner_server_url: https://gitlab.com +# A mapping of the ansible to gitlab architecture nomenclature +ansible_to_gitlab_arch: + x86_64: amd64 + aarch64: arm64 + s390x: s390x +# A unique token made available by GitLab to your project for registering runners +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN
To have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps. The playbook introduced here covers the Linux distributions and has been primarily tested on OS/machines that the QEMU project has available to act as runners, namely: * Ubuntu 20.04 on aarch64 * Ubuntu 18.04 on s390x But, it should work on all other Linux distributions. Earlier versions were tested on FreeBSD too, so chances of success are high. Signed-off-by: Cleber Rosa <crosa@redhat.com> --- docs/devel/ci.rst | 57 ++++++++++++++++++++++++++++ scripts/ci/setup/.gitignore | 1 + scripts/ci/setup/gitlab-runner.yml | 61 ++++++++++++++++++++++++++++++ scripts/ci/setup/vars.yml.template | 12 ++++++ 4 files changed, 131 insertions(+) create mode 100644 scripts/ci/setup/.gitignore create mode 100644 scripts/ci/setup/gitlab-runner.yml create mode 100644 scripts/ci/setup/vars.yml.template