From patchwork Thu Apr 11 09:09:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625632 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D574143C75 for ; Thu, 11 Apr 2024 09:10:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826603; cv=none; b=EU+YbgDD4tp3fwv1PqL4SqW1b94K/QtfW+QiOyJTwJx3PV3eS++4cQ8fp1mq7IqUqjbJT6ygesINDRUiVMy3t6GZkH4kLFVPFudYHjmwsw8+spGDGa7ZZWbRzD5LS+qW/vWINKbTNC2byvzCAo3qQ85N6fT2fb6TvdCkBd6YK04= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826603; c=relaxed/simple; bh=PaOP4IQIuau8Qi2g0rxqkSEYw2aWULuBoNtu+hIzKW0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=dPudmMmQPcAYnID6bHQW8jGnIkKiZLYivEsRbQQNS9TBRBj3u9jrse9rPBVH99m9JwfW5Zxn5PrdkDRCYGxBPxWFulunsAR2PSlYH0t+I/HNPJAhkuJdiHWBAaeeOQKWyY+w9Kd36ks3Sfsb7QsTj2haHpkVqtUzLUU/3xemWgs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=moxqcXOO; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=Yu0HZdTa; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="moxqcXOO"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Yu0HZdTa" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfout.nyi.internal (Postfix) with ESMTP id 3AB0A1380130; Thu, 11 Apr 2024 05:10:01 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 11 Apr 2024 05:10:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826601; x=1712913001; bh=xUM34d+qMf LgaFBi3LMIYtZ4N78vD9LWtj2rI8Vy4tU=; b=moxqcXOO5AKnV+J2ehp6rmiOmB 0ozVBccPz4YUVFniKSuptPqpfaatA+qLsneP3L3gRCnCIA7nM0wVVBr9IhGSOhOK UBVsQCCiCqrtGCxHkU30KWbM5QexIag45V9v6A1gLDO/H+c4B4EDin7jtMzsNS1Z lPrKn0hepBsv2uJUKF6R/92hcnC8CBW2wzayS4KovTtj6a3JFcSmXVMPMGAsT07y SeaohzLFjzbwYGUptuz0oU4V9CV69+6ZT/dGh5Td0HOkuvQP3kl1zwr4fASe1t09 7SioI3l0UZmVn+XmTR4p6wdQdujrMS/5CEgfuRRgieu0KP+0MglH2/dnZ8VA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826601; x=1712913001; bh=xUM34d+qMfLgaFBi3LMIYtZ4N78v D9LWtj2rI8Vy4tU=; b=Yu0HZdTacTZMXBDOc88sJuhkuHPMuDcu1ug7d50zcDXt iyZacZM7wqFztWJRuaxHBTPtifXgO2boPuspAihukZoPaiEIqGJ5ha225KO5ubEJ XkoJNZWQbqu58btpv00GHSQZP3HF/Zl8/NLdyC5vkd3IWpSKondb1NuZ7jku6J2B uKgd3O7zJWJ446zcOlOcgJTPIw00twcC0MBJjL9oQX/wxYmdoWIboFyuYC+TOHdv rGJsB+Hlnw/OmkKuFCz3pd0ap0v6vxV6PSvyKKaKSY8QchEcqlqtrPi6vwY5Vtsj o4A6r6yS/bVnUrpwZxbIlRixIPhbHtypXamvj6+8mg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepieefvdehgfekieejueektdelhfeiheeltddtleekieevieehleektdfgveetkeet necuffhomhgrihhnpehvvggtthhorhdrtggtnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:09:59 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id e4b35702 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:09:47 +0000 (UTC) Date: Thu, 11 Apr 2024 11:09:57 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 01/13] ci: rename "runs_on_pool" to "distro" Message-ID: <46502bbe22e3404d41ab8da152d9498c607470b7.1712825204.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The "runs_on_pool" environment variable is used by our CI scripts to distinguish the different kinds of operating systems. It is quite specific to GitHub Actions though and not really a descriptive name. Rename the variable to "distro" to clarify its intent. Signed-off-by: Patrick Steinhardt --- .github/workflows/main.yml | 2 +- ci/install-dependencies.sh | 2 +- ci/lib.sh | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 3428773b09..684ef5c00d 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -303,7 +303,7 @@ jobs: CC: ${{matrix.vector.cc}} CC_PACKAGE: ${{matrix.vector.cc_package}} jobname: ${{matrix.vector.jobname}} - runs_on_pool: ${{matrix.vector.pool}} + distro: ${{matrix.vector.pool}} runs-on: ${{matrix.vector.pool}} steps: - uses: actions/checkout@v4 diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index b4e22de3cb..7d247b5ef4 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -11,7 +11,7 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" -case "$runs_on_pool" in +case "$distro" in ubuntu-*) sudo apt-get -q update sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ diff --git a/ci/lib.sh b/ci/lib.sh index 0a73fc7bd1..d882250db5 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -279,7 +279,7 @@ then cache_dir="$HOME/none" - runs_on_pool=$(echo "$CI_JOB_IMAGE" | tr : -) + distro=$(echo "$CI_JOB_IMAGE" | tr : -) JOBS=$(nproc) else echo "Could not identify CI type" >&2 @@ -318,7 +318,7 @@ export DEFAULT_TEST_TARGET=prove export GIT_TEST_CLONE_2GB=true export SKIP_DASHED_BUILT_INS=YesPlease -case "$runs_on_pool" in +case "$distro" in ubuntu-*) if test "$jobname" = "linux-gcc-default" then From patchwork Thu Apr 11 09:10:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625633 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0620613C3CA for ; Thu, 11 Apr 2024 09:10:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826609; cv=none; b=U/yxRxS629fXF8OCGF8pyxQPYcGCZWWV+t2TIrlIkxLjjLnAmgoUag8bH5J6GhsqgeVssvcaI4GRHd2dC7vxPR43zKmaDZVQKIyPJBhsXe6fik9f5h4bM9OwwKcfCDIX5Mkvu0UURNcQd+T9p/qo00DM9bSoimvs39CjhXbR/Pw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826609; c=relaxed/simple; bh=3Fl/Xwd3ydRsKJpbSQyDCPG1S72fW+4gbfuCEz3QEdw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=U98Nx1q1IXH9/78bVh8/Sb/7VEdoXEg0WXQ6nNQaoKhTYgFzb67FtM2GjPu7kZxXEBCOyg2AoNWsjpniCAeqYKvO4dpSkBCkuUOIyusMtsIJkKCU/MBYgiR2WxHODDQbRPTRW4NjCt0DqIcaBuqLCLQ0LH/vAkK2bE8q5uG/RAA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=imZOeWza; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=MXgf5u92; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="imZOeWza"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="MXgf5u92" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfout.nyi.internal (Postfix) with ESMTP id 04CCB1380158; Thu, 11 Apr 2024 05:10:07 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 11 Apr 2024 05:10:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826607; x=1712913007; bh=VNk9eX6SCC 34Yl2veA58Ou5H/atOotM4dWMXN3mJ40w=; b=imZOeWzab0Eni2Ac5vTmyba6Go F/mjbExEcyDajMX7P+w2pT2LsPHSs8BlBJDlJhRH7kPtMoluhzAsroNCNW0IAbll vD2PSriSxdKhsn6g9iYMVzPGus4kBQnz78qNdQEP4prrt9eU8qhT9Bwc/cAOXiXZ wf4xPwMQfKu9gSG/+6mxYtZrDofWBFaa1gcEA+xZO+JqJrNawBmp+zjV4TPiuzfM uoEvJG5x6afsRE2cCxF3z/8IKSnht8+kyGIar5t0MXvDY8EXUMSSqyM3DCC9VFdk vtFFhINqft9kFH0ZSaYY5zkNYcd0C12yUG8C1kxzYCAtbvWCDrntbGYBaW3w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826607; x=1712913007; bh=VNk9eX6SCC34Yl2veA58Ou5H/atO otM4dWMXN3mJ40w=; b=MXgf5u92uMAY897jBbrvQQ119wrv758Sm/PK972xuVCc z22hGMEVirj4QyXZhBVYLjUBXpYIS8hEl0LrlXEi7Jiap/+HHSun/yen7VvgywhU Ky4/S676xTtFTRYR/bZAV8ixoC/H/2pz9/wZ0h7GTAQYPqe2Q5rlS2TIWWfuqNAk 63jIE65LtKBK/YytT2iLpQPSl7bMtXK4i7pwnYugRWjqaSXHiqPz4rfSSAgFmged sW+RgNHr43VAIJLCYFJ1UwjtOcA00ynWFJqxSJT9DjlVNU1bSTGpj9dCZMmm54v/ v0nhQiCvrqspGoUcaNT6C7j6zVRC5rONuviBjOqQsQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:10:05 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id a1de307b (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:09:53 +0000 (UTC) Date: Thu, 11 Apr 2024 11:10:03 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 02/13] ci: expose distro name in dockerized GitHub jobs Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Expose a distro name in dockerized jobs. This will be used in a subsequent commit where we merge the installation scripts for dockerized and non-dockerized jobs. Signed-off-by: Patrick Steinhardt --- .github/workflows/main.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 684ef5c00d..71cd4e5486 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -342,12 +342,16 @@ jobs: vector: - jobname: linux-musl image: alpine + distro: alpine-latest - jobname: linux32 image: daald/ubuntu32:xenial + distro: ubuntu32-16.04 - jobname: pedantic image: fedora + distro: fedora-latest env: jobname: ${{matrix.vector.jobname}} + distro: ${{matrix.vector.distro}} runs-on: ubuntu-latest container: ${{matrix.vector.image}} steps: From patchwork Thu Apr 11 09:10:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625634 Received: from fhigh1-smtp.messagingengine.com (fhigh1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C37A3143C72 for ; Thu, 11 Apr 2024 09:10:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826614; cv=none; b=t50oTCXxC2Hw3IIZjG4QFe0ViZCD2j0ujLGoKHBL0s/aASiztdGkJrCMzoRRs008JIjv0s1WxqiLUsUyS8Qjo/uru5yEjw99FUoGYZBQOdJ1XMQXJuHUCzw3BYwk9sQejZERramL4NXe7MY+NWAebzY+GVZlUNQ9jf6LkVcGx/A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826614; c=relaxed/simple; bh=5Li7yGeyUFRozVwQV+D6DmVeWxrgmsIAVKyUwbu1Cxg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=j/h03SjImT0p1EOazQzPJ/yh1L4as8ZgDFCREnk2fLl84m5z3q58cu/US/YwmRAeEDox4RTQU6ynE074g5nvwO4OYGIBd7zUxV8LP+vOCRQcaKcb1wiCYG8nA2SEP2dOp+RTpkboT5wWpNwzYRieWs8arZOIDR79AMxmvgaho5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=B19tnaxW; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=gYs/lBSl; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="B19tnaxW"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="gYs/lBSl" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfhigh.nyi.internal (Postfix) with ESMTP id DAF631140152; Thu, 11 Apr 2024 05:10:11 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 11 Apr 2024 05:10:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826611; x=1712913011; bh=u5XHrr+xQG y2YT8G+mZspVrzX7g3Olq3Rtby0F+pRfw=; b=B19tnaxWkLMlQVRqh6QEP+fV24 rUs7NUtLY+vIyspT7GkOxqUoG24CMrWFtysP/B7KBi+tJo7AzJXJzCqeOGGLcxVx n/z9PPJ3s7Jlm7t39zqAjqUlPo6UKT0j80GlxlYv6V40HeNa0m+XIxh7rJtzAajW upeAr3n+DB3yipi/2OGKK9hbYxiFlnDTNDRo/fKCq8ATnGWL9scrwqCV1IMfjekw SlTeb1+50D7ZHwraGy3UBobV++0O+Z+2YA8TzC+q2LkBHXI4pxcGS2+uIWlHA2L0 /OcM2DSC9AyhObVS5ZdiskI/8PiL1o1kKSpRGxvl+Ub0kz7xR5RduGl4ef4g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826611; x=1712913011; bh=u5XHrr+xQGy2YT8G+mZspVrzX7g3 Olq3Rtby0F+pRfw=; b=gYs/lBSlfFbGFNCUPJuj7QMaFSbjJ8A0iaKas5moOoaz pOngGwvWEFfhO9eDIZsRmDzxSrYfyAT3BLhv6SRaH5C1mBUSQlyF0hc+E1YZZ/Ma GjiC3U/II6fm9es6iUL3NaMAs35gY3J96aMj143RmLdQ1WW6ufVGQ9fOrtBv4leD eTkUWsyEL4dI07wrm1Mzhv38mfwNsuhnFIcEjSTb5WEU2QqFaZ/daE3Czz4Epccr ZAMQTO9nMKH4/3g0chLl5tUET3pZ0aWCOpewyGJIdVfjw3kheRS9SzGiOqkjkzs4 NJmyBH8PS1Pvdd5loK0BYjkprH/6rCeRUV7x6O+xOw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:10:10 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id ca4f3fef (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:09:58 +0000 (UTC) Date: Thu, 11 Apr 2024 11:10:07 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 03/13] ci: skip sudo when we are already root Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Our "install-dependencies.sh" script is executed by non-dockerized jobs to install dependencies. These jobs don't run with "root" permissions, but with a separate user. Consequently, we need to use sudo(8) there to elevate permissions when installing packages. We're about to merge "install-docker-dependencies.sh" into that script though, and our Docker containers do run as "root". Using sudo(8) is thus unnecessary there, even though it would be harmless. On some images like Alpine Linux though there is no sudo(8) available by default, which would consequently break the build. Adapt the script to make "sudo" a no-op when running as "root" user. This allows us to easily reuse the script for our dockerized jobs. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 7d247b5ef4..7dfd3e50ed 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -11,6 +11,17 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" +# Make sudo a no-op and execute the command directly when running as root. +# While using sudo would be fine on most platforms when we are root already, +# some platforms like e.g. Alpine Linux do not have sudo available by default +# and would thus break. +if test "$(id -u)" -eq 0 +then + sudo () { + "$@" + } +fi + case "$distro" in ubuntu-*) sudo apt-get -q update From patchwork Thu Apr 11 09:10:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625635 Received: from fhigh1-smtp.messagingengine.com (fhigh1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56E79144D31 for ; Thu, 11 Apr 2024 09:10:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826620; cv=none; b=RXd8i+zSY6oKLdBnugXQaOuQ4gJHECBUexNaxz1TRlWUCU05eThOWVaHGIptTm/WCS17ihL9B/Bg9WHO4+8VohvZtW81B4xlfpfY2vI6v6DIeOH0IEOuu5RivgINQkuOWZ24sUhd4tAWUoNZn3HO/tIMD2f13+kNkAox/f5ktS4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826620; c=relaxed/simple; bh=nseXqVENMM3XzCwMOSVcx0kVoNDTCBmsh+pvS+q9Sa8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=jFyIIG3k8J1z0lqy9qmqvQ/Co09KK1LuXlP4neQp2D6VcfnsTxkrq3FOl3+P5AaJVpHF07aWQUqI95QuvtBxUSahN9PMCyxMf84ZLwlUfCERgMNaYhHge6ecy18XiRdBwF7gStwOfDxqVgn5kH97r/uHpf7GSouksed/ieJidnQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=Js2DX+Nf; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=V6obZONg; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Js2DX+Nf"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="V6obZONg" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 9EB581140179; Thu, 11 Apr 2024 05:10:17 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 11 Apr 2024 05:10:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826617; x=1712913017; bh=W/z9YvNKef F1G0bbY4msHe69GC0GyoP3ZIwDYjjFLw0=; b=Js2DX+NfdLgjmYXhIBPknJgZ7k tXLIvrjR5AnoVscvDMOxXHFLfJyjxxApILk/vZ2CiQKbbxy84z5WUdcWtVaj98CT 74D3fmmWeVGjgUi8ANJxSDpR8n34RGnLzenmwx0MyHAMR27PIwNFBk3OvLmxFtqQ +3GOzhjvGeZH8ePFRg7klpgI455umc/CFmkFMQJnoT8s2ddIa17K1XB0RD86le6G jQ63G/8SoC3D6/9hd/AFSrBOz/tEgPCj1VfhVfSCw3uF7z/hgPMVqgiZD2ZgfTH7 IDIv02prOeFMnITUuReJBKxgyaEYBjRRkBlNd/9z5TWkh+s2BcJ2GS5ttO2w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826617; x=1712913017; bh=W/z9YvNKefF1G0bbY4msHe69GC0G yoP3ZIwDYjjFLw0=; b=V6obZONgQw2yTeTMj4f/suFM5hJfIhXnfzNlmOrfGS8t QkhTMLWCYQ88aFcbGfah6vuhAh1dqV2j0GoZdL3Xr7IJb+cEbu1n/fbza2oHDS/D Nx3v4b2xJOrq1Z29UA6fKZeWAM+/zbv05YxDpigb8Sca32aJm31Ft9EgnggXcrgX pRKAoNOWJ1XaVqFZgnM7yj1Fh+vpwnOLuDBhzUQ/Flne+MnDxt0PeuRh2qbK2iaD ytC0JV3youtlAW3WEpkT11X7YysQ24Mz1rVMYAEc2YJl2AKsZEIL5+eZNhT9fbz+ IhyIhg5U0tBGX8NQQmpIDxTe4+fmsX31ggY1rnKeNg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepjefffedvgfelhfejkeefteffhfejudfhhfdvtdevjedvhefgvdduhfdugeegkefg necuffhomhgrihhnpehpvghrfhhorhgtvgdrtghomhdpghhithhhuhgsrdgtohhmnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhs rdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:10:16 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 9593c9fb (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:10:03 +0000 (UTC) Date: Thu, 11 Apr 2024 11:10:13 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 04/13] ci: drop duplicate package installation for "linux-gcc-default" Message-ID: <803f5020e04da1ea84691a63e97afbc8b99bd3d4.1712825204.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The "linux-gcc-default" job installs common Ubuntu packages. This is already done in the distro-specific switch, so we basically duplicate the effort here. Drop the duplicate package installations and inline the variable that contains those common packages. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 7dfd3e50ed..fad53aac96 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -7,9 +7,6 @@ P4WHENCE=https://cdist2.perforce.com/perforce/r21.2 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION -UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev - tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl - libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" # Make sudo a no-op and execute the command directly when running as root. # While using sudo would be fine on most platforms when we are root already, @@ -25,8 +22,13 @@ fi case "$distro" in ubuntu-*) sudo apt-get -q update - sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ - $UBUNTU_COMMON_PKGS $CC_PACKAGE $PYTHON_PACKAGE + sudo apt-get -q -y install \ + language-pack-is libsvn-perl apache2 \ + make libssl-dev libcurl4-openssl-dev libexpat-dev \ + tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \ + libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl \ + $CC_PACKAGE $PYTHON_PACKAGE + mkdir --parents "$P4_PATH" pushd "$P4_PATH" wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d" @@ -34,6 +36,7 @@ ubuntu-*) chmod u+x p4d chmod u+x p4 popd + mkdir --parents "$GIT_LFS_PATH" pushd "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" @@ -83,10 +86,6 @@ Documentation) test -n "$ALREADY_HAVE_ASCIIDOCTOR" || sudo gem install --version 1.5.8 asciidoctor ;; -linux-gcc-default) - sudo apt-get -q update - sudo apt-get -q -y install $UBUNTU_COMMON_PKGS - ;; esac if type p4d >/dev/null 2>&1 && type p4 >/dev/null 2>&1 From patchwork Thu Apr 11 09:10:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625636 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47CB26FB5 for ; Thu, 11 Apr 2024 09:10:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826624; cv=none; b=c4o+QZ1QNUc0OCLiY1p2n08lNVUvdTSgN7eYiD49ODHNcYDJA6pzL6wvD5hbcm/codN8lsm5E7a80g0ySdNhKtLIafSwje/0MvAEmcum4kJpp8ct98lkZ6asuTHTHbdI2oKvWf8JAflbNxNp7etIPwJsOKjNaIoVDzeWK40OsFQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826624; c=relaxed/simple; bh=EhJehMy4v/JcBG6bPzCDdLb6/iitYNTgethpzXmJsOg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=qnYof+NhluKCdy1qS0Yftl/lZuAl+fi/qJFQPKSxozQnkkSgtSHeYvlqsDWA24nDyd/aayXYxdJ+8q4yd6OqoFkLcBq4kZ3Uphh8gi+CEMyKI3f3M+eKcanTjrhks2dS8pebuB5rQkUX4/o6aRs9aJFh/P16WSTtGldncnleTdQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=dGFI3ai/; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=FP5sLoLe; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="dGFI3ai/"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="FP5sLoLe" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfout.nyi.internal (Postfix) with ESMTP id 5D05C1380222; Thu, 11 Apr 2024 05:10:22 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 11 Apr 2024 05:10:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826622; x=1712913022; bh=hSjMHjZqGv cL0fgEMEdT1ZJb3eMDkdcteTRcrBxkoEY=; b=dGFI3ai/fX4CrJ+vk8Tihvd/Ns n63F5afyFayk+HnS3GdY8IUrxd+e9bduC55Qal8SV1R0vAWlVzCn80N+oocTMVW5 SW0ipfpj8M27CL6AAGf1XLJo0xvxw5xXZsKtX5cCqx/UmVHlZ9Y5nZODDmk7Ryw7 NPSg5XEtTDggQocz7R/ILayADQw5t5rlhpfTsIb2w18lQqkZXKZydSK9gTe2K6lu b57Tvq5khpE5QdqcudhM7bMuRrCusV3qXz+mP8qzGlMAhqqppoABGHsnASrKSyrP mY8kolv63V42mDgao0UUxQFdr+0w5EU1KL/bkRs7h0mbFE8pWzPLl2ZzpLEw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826622; x=1712913022; bh=hSjMHjZqGvcL0fgEMEdT1ZJb3eMD kdcteTRcrBxkoEY=; b=FP5sLoLebOYpShc5M512Ne9hvV+3w25+vk+41pXUvUc9 o5aUAzeyYFZNO7TsuYqwLAC3DlHvcPDEve9HMTjEBopiVRgEBxfEog8BiFxEGc7J 26YkM5FnwSLh0RrQ5cWDrNWPS1E/CH5CRdEISEmuEh6epnkqqjLUKBFtXULTF9Q3 xWsUi9XVWbAssyUhtFZj9K5bdX4J8HPy6t0PYptN3tI5AOqQjLYMV8k+DxmjkMio GDhakSSxfPDVLOwZH4kEwbiZICv+sfh0QnbQXqHMA7VCXvWlz9pwG7rJfK+tOi87 8+sxzQhbHAS60epiEpIT08ug8jOi3b/stb0txjpmOA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:10:20 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id ab2d78c1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:10:08 +0000 (UTC) Date: Thu, 11 Apr 2024 11:10:18 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 05/13] ci: convert "install-dependencies.sh" to use "/bin/sh" Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're about to merge the "install-docker-dependencies.sh" script into "install-dependencies.sh". This will also move our Alpine-based jobs over to use the latter script. This script uses the Bash shell though, which is not available by default on Alpine Linux. Refactor "install-dependencies.sh" to use "/bin/sh" instead of Bash. This requires us to get rid of the pushd/popd invocations, which are replaced by some more elaborate commands that download or extract executables right to where they are needed. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 29 ++++++++++++----------------- 1 file changed, 12 insertions(+), 17 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index fad53aac96..2d6af876d6 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -1,4 +1,4 @@ -#!/usr/bin/env bash +#!/bin/sh # # Install dependencies required to build and test Git on Linux and macOS # @@ -30,19 +30,15 @@ ubuntu-*) $CC_PACKAGE $PYTHON_PACKAGE mkdir --parents "$P4_PATH" - pushd "$P4_PATH" - wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d" - wget --quiet "$P4WHENCE/bin.linux26x86_64/p4" - chmod u+x p4d - chmod u+x p4 - popd + wget --quiet --directory-prefix="$P4_PATH" \ + "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" + chmod u+x "$P4_PATH/p4d" "$P4_PATH/p4" mkdir --parents "$GIT_LFS_PATH" - pushd "$GIT_LFS_PATH" - wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" - tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" - cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs . - popd + wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ + -C "$GIT_LFS_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" ;; macos-*) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 @@ -53,11 +49,10 @@ macos-*) brew link --force gettext mkdir -p "$P4_PATH" - pushd "$P4_PATH" - wget -q "$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz" && - tar -xf helix-core-server.tgz && - sudo xattr -d com.apple.quarantine p4 p4d 2>/dev/null || true - popd + wget -q "$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz" && + tar -xf helix-core-server.tgz -C "$P4_PATH" p4 p4d && + sudo xattr -d com.apple.quarantine "$P4_PATH/p4" "$P4_PATH/p4d" 2>/dev/null || true + rm helix-core-server.tgz if test -n "$CC_PACKAGE" then From patchwork Thu Apr 11 09:10:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625637 Received: from fhigh1-smtp.messagingengine.com (fhigh1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D7086FB5 for ; Thu, 11 Apr 2024 09:10:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826633; cv=none; b=JRfbH9r1WK4TwQzfNtNW4z7mPjqbxm1IoTSEHQelVn1u1BGzrQRuQZdCMz0gSVmsBDAcn2CVRcGBx6+NzLxCCjjVai+0aJFuHiZ8gpTvKRRvMeel/N26CNAhwj6IziCwK2TtZnWfvBwPm9HQY8o+E0HOO3wbMvnDrwRkNDcfK0s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826633; c=relaxed/simple; bh=o07hlBR34doZ58FGQCHU5vvc/lDxMkeTagg0+WkE75U=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=S709oSUM4C7uSoU6999XaLHkLMl1OWqXVl0sVC75AxMg9+NcqyOZMoiNApYqweamFBYxk+gUR+8Z+mUFj1QOFAQs5CjguKApB3JWVZqzaBuIN9Ck+mOMPDFwsoj40SON/PELKK8exl7c1QPpl8DJ7FPIE13neojGcEh/XEu4B6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=W0+kIkFU; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=rcZcEdlh; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="W0+kIkFU"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="rcZcEdlh" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 674DD1140152; Thu, 11 Apr 2024 05:10:30 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 11 Apr 2024 05:10:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826630; x=1712913030; bh=/Wjh96nBV1 QyjAWT6NqfSNlHTEyrDyUMxIAWUSFuRYw=; b=W0+kIkFU5ugViLUJ418jOroHfL CxCetK+WZ4DuVX8WWWKRH8zguHfMwTeqYkqgQRhVG9ijJqB7+Ci7i8WaTRP/3aIc qb7QfNQy3A1zfStGjDV6bJrZsZ2qimy37ysxtlKP2FkWPcyCKdgKLPM85WaPM0w8 0IOO3mnadlyg54d03qSHbZ1chU0PIXYRsiUIcErgyRIRCNyLJLbW2+Q+6YDwlmAp sIA9j8hO5RxvjHuDGDOBX0WQSkVh3KIef/zQurixSRwo3NpC+2znpEFKNbnyPq52 ArLtvw7PJ40at1Qk0Hb1XWkjwEzJLdMbe0E0wUm1eYxuElUesbKrWdQwB3YA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826630; x=1712913030; bh=/Wjh96nBV1QyjAWT6NqfSNlHTEyr DyUMxIAWUSFuRYw=; b=rcZcEdlhJUFdeO4+2Bz+9ZMeniNEsd7A/oDx1oluhj8M Ao9Hk061Zxid1//YvVKoiVbWBF5TKNauwkwB+zKg5hfnWjBEMEfP0JGRv42PDb/L ZxzaarBKW/j+RZ7EweeBqllRCJEvFKWn1OA+mwwyjDGJtkmO4qi3tLNgovpnNL9t 7tF3PqdrU6MxlqM1s9vjsv2XL2UV/JkhDa9PyQS/TmWmoQUkiqR5FGmuQo8kRdpb 7Mfu0rOp+G6Q2Pr6MUYzDIbxPVqxF9CuxJlquZ/P//y/+fexpt0ST7qNYyELO4zc iGelqZHW/GMWp4/YcTtPyzkNwH8cpRzFLuz/dKzgtg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:10:28 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 6cf832b8 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:10:17 +0000 (UTC) Date: Thu, 11 Apr 2024 11:10:26 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 06/13] ci: merge custom PATH directories Message-ID: <61f108d95463f94b3921669a276472bd01fdc606.1712825204.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're downloading various executables required by our tests. Each of these executables goes into its own directory, which is then appended to the PATH variable. Consequently, whenever we add a new dependency and thus a new directory, we would have to adapt to this change in several places. Refactor this to instead put all binaries into a single directory. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 15 +++++++-------- ci/lib.sh | 10 +++------- 2 files changed, 10 insertions(+), 15 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 2d6af876d6..bafe37f2d1 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -29,15 +29,14 @@ ubuntu-*) libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl \ $CC_PACKAGE $PYTHON_PACKAGE - mkdir --parents "$P4_PATH" - wget --quiet --directory-prefix="$P4_PATH" \ + mkdir --parents "$CUSTOM_PATH" + wget --quiet --directory-prefix="$CUSTOM_PATH" \ "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" - chmod u+x "$P4_PATH/p4d" "$P4_PATH/p4" + chmod u+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" - mkdir --parents "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ - -C "$GIT_LFS_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" ;; macos-*) @@ -48,10 +47,10 @@ macos-*) brew install $BREW_INSTALL_PACKAGES brew link --force gettext - mkdir -p "$P4_PATH" + mkdir -p "$CUSTOM_PATH" wget -q "$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz" && - tar -xf helix-core-server.tgz -C "$P4_PATH" p4 p4d && - sudo xattr -d com.apple.quarantine "$P4_PATH/p4" "$P4_PATH/p4d" 2>/dev/null || true + tar -xf helix-core-server.tgz -C "$CUSTOM_PATH" p4 p4d && + sudo xattr -d com.apple.quarantine "$CUSTOM_PATH/p4" "$CUSTOM_PATH/p4d" 2>/dev/null || true rm helix-core-server.tgz if test -n "$CC_PACKAGE" diff --git a/ci/lib.sh b/ci/lib.sh index d882250db5..4cce854bad 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -340,10 +340,6 @@ ubuntu-*) # image. # Keep that in mind when you encounter a broken OS X build! export LINUX_GIT_LFS_VERSION="1.5.2" - - P4_PATH="$HOME/custom/p4" - GIT_LFS_PATH="$HOME/custom/git-lfs" - export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" ;; macos-*) MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" @@ -351,12 +347,12 @@ macos-*) then MAKEFLAGS="$MAKEFLAGS APPLE_COMMON_CRYPTO_SHA1=Yes" fi - - P4_PATH="$HOME/custom/p4" - export PATH="$P4_PATH:$PATH" ;; esac +CUSTOM_PATH="$HOME/path" +export PATH="$CUSTOM_PATH:$PATH" + case "$jobname" in linux32) CC=gcc From patchwork Thu Apr 11 09:10:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625638 Received: from fhigh1-smtp.messagingengine.com (fhigh1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B6896FB5 for ; Thu, 11 Apr 2024 09:10:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826637; cv=none; b=dQH/o/yilu6WuBUQt3bHJZ1JTwlIAAjN6UL+qexPMdZDgYWHNdx/k+giNbE/8vKzc1dpzJ+X+pZzLX6lMksNQhnzyEGQQ5iaxI7wYEfWj8bvohJ+HcAakRxf8gkuLLw0P0kyiVJZz4dnyIix132fgNR0SwOo41Byg1TOAkJFJHE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826637; c=relaxed/simple; bh=q/8hdGUJzUJjvwLRKYxQDbZ+r2p2ceZmQsj9DhQINE4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=nSxCqBnBOfocf/i6ApEnoIh1HXP0tSMbTDUNEiUWHaig/aei9pSwLU00/d8VFwrqoOPOWY4gfDC0aL30DkU3iJhvS4MepjbuUkW4LBYdSH8ELbjLYxpRXxa7+yRgc+5PJFbLb6sdIyy/ojGfWFJ1rAkPIz/YNDe19GABtRZxOP8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=oc5mTQN4; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=hV5r13TV; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="oc5mTQN4"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="hV5r13TV" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 4B8021140178; Thu, 11 Apr 2024 05:10:35 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 11 Apr 2024 05:10:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826635; x=1712913035; bh=muFuvoCvxD IQlfRh+fbK5XFYE0T0D4cEG72UiDyyRXo=; b=oc5mTQN48ZLZiZ5iLVGGXE97cn Airz4wcNNEJkXmoKLSW4KHjQ06mGq+angul909PwEcDvRXshN84qOQH1Jqr5N83V MVNSANjya2Prom3Vepiig8IK7MB5golrWJmkYSRY0UH2Sp8pXvA+IxLR7zUjBzHi AHIX2BwM0spv+kOz7BYCk5Sj6wpMk2neQJb2mfUXREke2JbhuFZcCTmb0mbWw2La dJgE7BNA71AyWBD5cepvdsgsIGXTPcsjFoXnRUJ5dpy+ikhG8eSt7gI698s+tQVk CYsWIIQYqJQ35mgnxaLin8vTA2MndwIknEuzcf2a/pgGwabuf5z6xqhskn9Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826635; x=1712913035; bh=muFuvoCvxDIQlfRh+fbK5XFYE0T0 D4cEG72UiDyyRXo=; b=hV5r13TVR+6URkAUnq+YEF3+6rik9An2VNddHqEjWlXn 6+7fO/HzPaP2neTV37DVoQBa+u2wBVSmhHvLmNJQ+vx4KB+9MWCzXsKSde42SoS7 MiXrLUcnq/Nhe5QtXPft89Wi5t3mqYQCqPXi7B2XXukMeR/kI0thyPrgm/Pl6k9Q KRoGSRp7v6Tf5i5B/TqawZ+zy8sPvnWqS78VTMwYuHa89HzgZ9avtaL5fBvC7geq 7QyJEp4LqnI7W+dkZHgeCVR/zslc/qOL/nlrCWFsoYi4NDE12GwRys8ZkblVVYd2 XF/nVWaQq5lkhlMmHAEyJYFWyS6cOYTAvtsDlCozvA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:10:33 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 92f5c85e (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:10:21 +0000 (UTC) Date: Thu, 11 Apr 2024 11:10:31 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 07/13] ci: fix setup of custom path for GitLab CI Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Part of "install-dependencies.sh" is to install some binaries required for tests into a custom directory that gets added to the PATH. This directory is located at "$HOME/path" and thus depends on the current user that the script executes as. This creates problems for GitLab CI, which installs dependencies as the root user, but runs tests as a separate, unprivileged user. As their respective home directories are different, we will end up using two different custom path directories. Consequently, the unprivileged user will not be able eto find the binaries that were setu up as root user. Fix this issue by allowing CI to override the custom path, which allows GitLab to set up a constant value that isn't derived from "$HOME". --- .gitlab-ci.yml | 2 ++ ci/lib.sh | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c0fa2fe90b..3a0ef4d4d4 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -9,6 +9,8 @@ workflow: test:linux: image: $image + variables: + CUSTOM_PATH: "/custom" before_script: - ./ci/install-docker-dependencies.sh script: diff --git a/ci/lib.sh b/ci/lib.sh index 4cce854bad..473a2d0348 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -350,7 +350,7 @@ macos-*) ;; esac -CUSTOM_PATH="$HOME/path" +CUSTOM_PATH="${CUSTOM_PATH:-$HOME/path}" export PATH="$CUSTOM_PATH:$PATH" case "$jobname" in From patchwork Thu Apr 11 09:10:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625639 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0207713C3CA for ; Thu, 11 Apr 2024 09:10:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826642; cv=none; b=Ct1rTt8hGEYVbyduNhjrJmCgeA0qX3rrEtiC9+L3Bxlqanz4BSpfVFAixIWpX9xCqtccGY+5us3bfZZPDi11U39otVNSCdNIiEi+Q8YfQrhuc/q8sBDkZkVqhWUhveZnIo/daHjsk989s21niX5yTOrCSWi1BSr3i28br4RMlKs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826642; c=relaxed/simple; bh=gWkHATY15jAUbxA5GiraQ0katyXl3CmLz8sS+ojbshg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=RWfyZEjoXqWfHnq06u8sNMLRUOmx+iPyUZFmPAOQPecwF67HjUeNg9G04P6fD0kScxdUPOcIfCGDD6RxgkPzfD/NG1o+Lnw+Va091RJ4/urXv0yGpSJPBxwX59ZDQSw7NOLkn+AE1wx6LhWyXh7JR2SsDYYa9w5VAObF4bVmLkQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=bfJZ7lJp; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=e+JSryma; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="bfJZ7lJp"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="e+JSryma" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfout.nyi.internal (Postfix) with ESMTP id 184E61380130; Thu, 11 Apr 2024 05:10:40 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 11 Apr 2024 05:10:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826640; x=1712913040; bh=ymqQBVdg1E OLIjEYQ3NUwTkZ8QiWPAiZQd1ldmcSQKE=; b=bfJZ7lJpn6nELpo4KhHrtESLLP BMx6dgdrEaRW0akSOtPLYA7aXemLg6EVvcBz4kXExuJ3dlkoujTMB5E75YaJRE4J CxE/qyMYGgh6n3CxK3Atlyfukdd8tzsPcnU4MyaFBaqBMmipYMH4l8l9AYwF0TKL P9SH0FZIbU6wG1KwtQxLKNRIYn68WccdZkZPl0CRfy8B29Zz7Vv4HGGSzm3ePH8k q9uXsjOqV9ZszoReOeGYk+eEJod5UHSNWTRdXlW6UxcgI0OciulNgDfAOvUDj78N Di/BdI0rJvMclyaqtaWpMiBQd3fN7hnKeohxtPwE7+qb0eSN+BjMCop51qog== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826640; x=1712913040; bh=ymqQBVdg1EOLIjEYQ3NUwTkZ8QiW PAiZQd1ldmcSQKE=; b=e+JSrymaB1xsExXydd4RiqhxBdDDeQb7EfuPHBbkRWzz aE3F9A5sCTS+jS7U/Xs7RGEjrLhjyAWw6aitXOnLNSveIcV7uH1CEJMi0Q4pzWJX 5XYEWzuMIpO6hPxlSRP1Syjbgae1VjhKkEJDbkNBmp6e1eEr6HM3rhusxKRnzB84 jPLLY5ejpSS2qYSBfn2RWOHR8Qsez8JU4BaSO2dCFJPzXLjBIrtTQ2FF9plHLL9y LS62tiFXxxZ7UZySH6QTjEC+EesB2UFaJ3cqOuepIwc/nBqK5GBWU7O2aq47yQh+ uVgr5bGmvp26dow2FKDz9/aA+39sASAoL9foU/UUSg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepveeuvdeujeehueeutdetkefffedtueetudefieeiieekudevudffffekheejfeeu necuffhomhgrihhnpehshhdqrdgtihdpihhnshhtrghllhdquggvphgvnhguvghntghivg hsrdhshhdpphgvrhhfohhrtggvrdgtohhmpdhgihhthhhusgdrtghomhenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:10:38 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 9caa5cef (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:10:26 +0000 (UTC) Date: Thu, 11 Apr 2024 11:10:36 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 08/13] ci: merge scripts which install dependencies Message-ID: <7748f87f8cd23826adbbfbaa7616d8b5be346c42.1712825204.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We have two different scripts which install dependencies, one for dockerized jobs and one for non-dockerized ones. Naturally, these scripts have quite some duplication. Furthermore, either of these scripts is missing some test dependencies that the respective other script has, thus reducing test coverage. Merge those two scripts such that there is a single source of truth for test dependencies, only. Signed-off-by: Patrick Steinhardt --- .github/workflows/main.yml | 2 +- .gitlab-ci.yml | 4 +-- ci/install-dependencies.sh | 32 ++++++++++++++++++--- ci/install-docker-dependencies.sh | 46 ------------------------------- 4 files changed, 31 insertions(+), 53 deletions(-) delete mode 100755 ci/install-docker-dependencies.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 71cd4e5486..5838986895 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -359,7 +359,7 @@ jobs: if: matrix.vector.jobname != 'linux32' - uses: actions/checkout@v1 # cannot be upgraded because Node.js Actions aren't supported in this container if: matrix.vector.jobname == 'linux32' - - run: ci/install-docker-dependencies.sh + - run: ci/install-dependencies.sh - run: ci/run-build-and-tests.sh - name: print test failures if: failure() && env.FAILED_TEST_ARTIFACTS != '' diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 3a0ef4d4d4..ba65f50aac 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -12,7 +12,7 @@ test:linux: variables: CUSTOM_PATH: "/custom" before_script: - - ./ci/install-docker-dependencies.sh + - ./ci/install-dependencies.sh script: - useradd builder --create-home - chown -R builder "${CI_PROJECT_DIR}" @@ -100,7 +100,7 @@ static-analysis: variables: jobname: StaticAnalysis before_script: - - ./ci/install-docker-dependencies.sh + - ./ci/install-dependencies.sh script: - ./ci/run-static-analysis.sh - ./ci/check-directional-formatting.bash diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index bafe37f2d1..e673797115 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -5,6 +5,8 @@ . ${0%/*}/lib.sh +begin_group "Install dependencies" + P4WHENCE=https://cdist2.perforce.com/perforce/r21.2 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION @@ -20,14 +22,27 @@ then fi case "$distro" in +alpine-*) + apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \ + pcre2-dev python3 musl-libintl perl-utils ncurses \ + apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \ + bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null + ;; +fedora-*) + dnf -yq update >/dev/null && + dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null + ;; ubuntu-*) + # Required so that apt doesn't wait for user input on certain packages. + export DEBIAN_FRONTEND=noninteractive + sudo apt-get -q update sudo apt-get -q -y install \ - language-pack-is libsvn-perl apache2 \ - make libssl-dev libcurl4-openssl-dev libexpat-dev \ + language-pack-is libsvn-perl apache2 cvs cvsps git gnupg subversion \ + make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo \ tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \ - libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl \ - $CC_PACKAGE $PYTHON_PACKAGE + libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ + ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE mkdir --parents "$CUSTOM_PATH" wget --quiet --directory-prefix="$CUSTOM_PATH" \ @@ -39,6 +54,13 @@ ubuntu-*) -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" ;; +ubuntu32-*) + sudo linux32 --32bit i386 sh -c ' + apt update >/dev/null && + apt install -y build-essential libcurl4-openssl-dev \ + libssl-dev libexpat-dev gettext python >/dev/null + ' + ;; macos-*) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 # Uncomment this if you want to run perf tests: @@ -98,3 +120,5 @@ then else echo >&2 "WARNING: git-lfs wasn't installed, see above for clues why" fi + +end_group "Install dependencies" diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh deleted file mode 100755 index eb2c9e1eca..0000000000 --- a/ci/install-docker-dependencies.sh +++ /dev/null @@ -1,46 +0,0 @@ -#!/bin/sh -# -# Install dependencies required to build and test Git inside container -# - -. ${0%/*}/lib.sh - -begin_group "Install dependencies" - -case "$jobname" in -linux32) - linux32 --32bit i386 sh -c ' - apt update >/dev/null && - apt install -y build-essential libcurl4-openssl-dev \ - libssl-dev libexpat-dev gettext python >/dev/null - ' - ;; -linux-musl) - apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \ - pcre2-dev python3 musl-libintl perl-utils ncurses \ - apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \ - bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null - ;; -linux-*|StaticAnalysis) - # Required so that apt doesn't wait for user input on certain packages. - export DEBIAN_FRONTEND=noninteractive - - apt update -q && - apt install -q -y sudo git make language-pack-is libsvn-perl apache2 libssl-dev \ - libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev \ - perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl \ - libdbd-sqlite3-perl libio-socket-ssl-perl libnet-smtp-ssl-perl ${CC_PACKAGE:-${CC:-gcc}} \ - apache2 cvs cvsps gnupg libcgi-pm-perl subversion - - if test "$jobname" = StaticAnalysis - then - apt install -q -y coccinelle - fi - ;; -pedantic) - dnf -yq update >/dev/null && - dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null - ;; -esac - -end_group "Install dependencies" From patchwork Thu Apr 11 09:10:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625640 Received: from fhigh1-smtp.messagingengine.com (fhigh1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D25BD1442FA for ; Thu, 11 Apr 2024 09:10:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826655; cv=none; b=YDeISWIXziiQmqtHY/knyyCSTTgDs9fTBZrmaQ0ZWphT37oKMp7x5WPzrzHTaYha34coflORyft0PPprVYMf77yYWLDXp7WUQPrAGtayauhlfuuhKXILeDMbYrRa+MZB3HXBGS6KEGbe+CGfZQHIXwfT3T1Dw8MQ6f423CdSyHk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826655; c=relaxed/simple; bh=zLDMggoUrZWFoaxmQwA8QMbLai7xUZDp7x5xz7iXy2k=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=GoHeO3kZ95vhPE+glJgE79KTKsdut6nMdCcjMOrbqleikHCprGt97fRL2X6RdxTSVj6KKcfqILfnzxp8ZC/M9ou/JUMKMoFVFJg9uV6WkEMLT8OImZSRvRw9xyjyvYHN+V6iZ4nmYdTnAosX0/xRSiCysTG/K6HsPvr0wlckX5I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=iUq70cMS; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=nzFwlfhu; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="iUq70cMS"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="nzFwlfhu" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.nyi.internal (Postfix) with ESMTP id E2D0E1140198; Thu, 11 Apr 2024 05:10:50 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 11 Apr 2024 05:10:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826650; x=1712913050; bh=tjsVx4nQhx ZS45gfmhMJ/ocdUu92HV7bOMCb/JVjw2E=; b=iUq70cMS7omFvmLbGpRSAEYn4G vWDqkiwCEPsJKIqazu1VwaYO69rL0gTo3MY7MqDrq7ebMP5hbnCR8gZ85DffFRGn eNwqi+JJoLd2kJ4GNZDA/wlOI5WMS3r7yeIwRodCLNUsg2YQ0EtXlgp2erO9sgBd CqSnHdAZ5YQrM7LZ181Y1gd+hj5sbicKOmda6gsDZeZZzWtzSLAAB8FpKnpPSTKC rQI6n2W18Cnu8TEY+KfBeaP72TAO0PJz1bO+YywjcFfLEy7bYGED6quQrY/4164L llyU0HyaTESYYudnXFbz/URwdcNyRNcCLeGbzOsL+93WXKUuP8+zgRs86E5g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826650; x=1712913050; bh=tjsVx4nQhxZS45gfmhMJ/ocdUu92 HV7bOMCb/JVjw2E=; b=nzFwlfhuMioec0NjWoqU9RqxDYqKpDFTuEppBDl/TXED T4CLXbw7yH3rTFlomp3v8QLbNJnfNKGPsbOgNSYadYdnvKArCU9yFWJ5bA8yJPYr riqGCkY4RWmN+x0+D+XBTrHkXBj6ZEH3tQWnDEPXtTFrlU/fzlSkikFNfEAHhJOJ xAfN9IXry0LZV06gKVyRaLi7AJNs9i09QaxXHL/01nKRLMP7121OQXdc/HllQqHD WjmQlL6NrPHh0ni+QQTFQsCNWeM8AhOL5GCgbTxMfqddB7ofVUY8hLkimDAzrnMp XlvsvGOIN6lqsgY0BVN0OkL9/ZcNMvBFE0vAbdDlBw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:10:49 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 0ae94a03 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:10:37 +0000 (UTC) Date: Thu, 11 Apr 2024 11:10:47 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 09/13] ci: make Perforce binaries executable for all users Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The Perforce binaries are only made executable for the current user. On GitLab CI though we execute tests as a different user than "root", and thus these binaries may not be executable by that test user at all. This has gone unnoticed so far because those binaries are optional -- in case they don't exist we simply skip over tests requiring them. Fix the setup so that we set the executable bits for all users. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index e673797115..0b9bb686d8 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -47,7 +47,7 @@ ubuntu-*) mkdir --parents "$CUSTOM_PATH" wget --quiet --directory-prefix="$CUSTOM_PATH" \ "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" - chmod u+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" + chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ From patchwork Thu Apr 11 09:10:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625641 Received: from fhigh1-smtp.messagingengine.com (fhigh1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B42B142642 for ; Thu, 11 Apr 2024 09:10:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826661; cv=none; b=F8tTvo04OmmfOhOXMthrNhPFM6QB6j4n2Jw8EJaB+ID8BmUeE/lx/wL2nqAxE0ACnkfDHXr3DblfqGkvREqbBm7yazxo4xNSwm5FYhIC/QQLepgSlo1avJTMWAVpGT4zR5DLjrYnV1dqIjC6b9aAzEzj2p1Q8oyFc/QuVXUuNuI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826661; c=relaxed/simple; bh=JE9ozeDEYqIUn2wB0SBRnXMiAYeu6gv3ILaB00aA4jw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=pOEi7CUWqLevc6tJE/K4WWG3wU3nI094lzxVvYVxDgmCqvedlGu3EbnItZYfkyxY7x1UNpwxS39AMSAVaV5bqtbOowEYym/DAIu9MEFtBtRVVa+SJnGL/guD6E8VQnPGZSDt/qQMFJLJpOFLdQUqLWpklOS1vrDFyzOza/2quYs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=l8DZitej; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=ADaxLXsa; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="l8DZitej"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="ADaxLXsa" Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 2D31A1140177; Thu, 11 Apr 2024 05:10:59 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Thu, 11 Apr 2024 05:10:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826659; x=1712913059; bh=mFiOi/9RE2 f+hS8raagOqiNSDlcylZ/bhaQgwEQsLm8=; b=l8DZiteju2PBGelmCSEYtIpRBY 9fi+nPEOk0QXBMg0MNjXp+lRbTQYX16C9T/U8t0HV5maKYIhjVLc8IU3Prib12F3 +EwCMmnDPtVr1YxfmTUyZdsF9Sdt8wn8omSuPCt6aOkCTblannOS7rhG6bVbxM0x /sj+BsPw8DU7/xEdvTMEPf0ZujMcv/IIgr8W3WXkwn6rrSlE/XEdbNom+KcFozFd 7gja3bbyadrHtiuprkeq47ZFITtN63okch1wR0F0KV9AEUPGUdGsl4ekD88A6/y3 M6+UtHEVhTcLACvfS0bmrAqhGmI0bQgYDN3NCW87JbES3pHY3O9EvbDWnisA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826659; x=1712913059; bh=mFiOi/9RE2f+hS8raagOqiNSDlcy lZ/bhaQgwEQsLm8=; b=ADaxLXsa4uENk4JP1eGFDMtzFO0fF3wgORFqnJ/MrvxR PJNL5rS5Olzw2LN4R+v3E94+p3jBKILxMzEJ1SGGfvSkMm72u+y085xpbtrGrvFB LVNfXVmwhmXgdl6vnOTPTPGKOMvO8mmeDFdp0hXk2bV1TErZJzNnvjMlSmLfFB1M dx6i0DNKNanJjOPhzS3wEmnb4shEM+Hg7KtMjVvAa6OI12ioHr4xPorTeZjW3hxv EEtQPwnkAnxGBbYAtnXW0Kg96orVYU4Gtsk54Cm4IYRgEZumo6rZ/K7d3E4kDjwk cV/ujupE/I9q1UgQqAhy3kumXommohNf289kt6L0sQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhephefgvdeuhfduudffuedufffgffehgedugeeivddvjeeugfejudeikedvgffgteei necuffhomhgrihhnpehpvghrfhhorhgtvgdrtghomhdpghhithhhuhgsrdgtohhmpdgvtg hlihhpshgvrdhorhhgpddvtddvfeduuddvledugeehtddqrhdrshhhnecuvehluhhsthgv rhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:10:57 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 6ba03b42 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:10:45 +0000 (UTC) Date: Thu, 11 Apr 2024 11:10:54 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 10/13] ci: install JGit dependency Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We have some tests in t5310 that use JGit to verify that bitmaps can be read both by Git and by JGit. We do not execute these tests in our CI jobs though because we don't make JGit available there. Consequently, the tests basically bitrot because almost nobody is ever going to have JGit in their path. Install JGit to plug this test gap. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 0b9bb686d8..c196e56762 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -9,6 +9,7 @@ begin_group "Install dependencies" P4WHENCE=https://cdist2.perforce.com/perforce/r21.2 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION +JGITWHENCE=https://repo.eclipse.org/content/groups/releases//org/eclipse/jgit/org.eclipse.jgit.pgm/6.8.0.202311291450-r/org.eclipse.jgit.pgm-6.8.0.202311291450-r.sh # Make sudo a no-op and execute the command directly when running as root. # While using sudo would be fine on most platforms when we are root already, @@ -39,7 +40,7 @@ ubuntu-*) sudo apt-get -q update sudo apt-get -q -y install \ language-pack-is libsvn-perl apache2 cvs cvsps git gnupg subversion \ - make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo \ + make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \ tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \ libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE @@ -53,6 +54,9 @@ ubuntu-*) tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + + wget --quiet "$JGITWHENCE" --output-document="$CUSTOM_PATH/jgit" + chmod a+x "$CUSTOM_PATH/jgit" ;; ubuntu32-*) sudo linux32 --32bit i386 sh -c ' @@ -113,6 +117,7 @@ then else echo >&2 "WARNING: perforce wasn't installed, see above for clues why" fi + if type git-lfs >/dev/null 2>&1 then echo "$(tput setaf 6)Git-LFS Version$(tput sgr0)" @@ -121,4 +126,12 @@ else echo >&2 "WARNING: git-lfs wasn't installed, see above for clues why" fi +if type jgit >/dev/null 2>&1 +then + echo "$(tput setaf 6)JGit Version$(tput sgr0)" + jgit version +else + echo >&2 "WARNING: JGit wasn't installed, see above for clues why" +fi + end_group "Install dependencies" From patchwork Thu Apr 11 09:11:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625642 Received: from fhigh1-smtp.messagingengine.com (fhigh1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4AA3142642 for ; Thu, 11 Apr 2024 09:11:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826667; cv=none; b=WORn13Ak8p9M7QiqUlSO3PaS7YC6DRp0b9Tx8jdNeSZawSU/yuFG+ifN+M9iNRsfd1H416UeZTs4RIoJ2dcX4wAgPiEkqMtdkGWOSpghSNFUJvtJS+BRtUWG3k+mKtei3bEQanw2wnE6iWFqz8A5wdertSLjoMTsXKRbrm8RUAo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826667; c=relaxed/simple; bh=jPc0YsuC4LLQyf6iVtPQq3oStMVa/OHFaVsiqJpgc7g=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=V2mMB0Z5o0Xh5wOWQYD1nUL95e1+m/u2ywvaXbrrW0UrEyv+ANKXvVgpegjAce9ntjxnwR18Idu+85Uy8tTTXlACWKC3jnAYGVHCoO2n45tiK/oYAQJtp5C0psvE5m30FNKfF/TwWFdyORWUKDb1108lEz/TGAi9LyL+ALlN2Qs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=Kok5oDj6; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=d395STEm; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Kok5oDj6"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="d395STEm" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfhigh.nyi.internal (Postfix) with ESMTP id EF4A0114019B; Thu, 11 Apr 2024 05:11:04 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Thu, 11 Apr 2024 05:11:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826664; x=1712913064; bh=AiR2HOz3pZ FFeGTEOdOMw01Mclv/3za2GVTmpUX0R0s=; b=Kok5oDj6akU/W3T5u+ZiislexT HoFvDlBJOXt0S007ok7KRi5CiGgmj9lzKS8flJ0nDxZAEJ48K8Wg5JEsSfsrjase 6wulvrvfFV1iVXNTQTeAeHgu3h1LWcF6pXkJzh6Vu67/SjuF3smhojxGptZ5KBm1 kHoL/cwQJWbJovIcv1g4XvLTPFwyDFtRnq/jdvpOIKksuvyI2Dwt+ClEJ3ul8ydU U7cr9hNi+l6U1kiCviZLLpWxiujDdKeBvQMkRvTpDhpAQpvDJjQMDy+U5OKZIZ4R CUv3TGnaEz+K2/6dfEX4rvVXibtY2LjpaQoaNJ1h1t097yJRYEUqnQxTTEPg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826664; x=1712913064; bh=AiR2HOz3pZFFeGTEOdOMw01Mclv/ 3za2GVTmpUX0R0s=; b=d395STEmZL+zZpCRKhVL7YEKThDlhaspef4hEneyQIO6 FDYaCGBoFbiOP7F2aZkVIE4bJY8HVdy2Y5YagDTq/Y+mJBwTo+tcGoTFhDTRoyG5 lt+35Bnjj/ho/4YudfSHjWPQCSDismrW16FLLq2m9yjrI0Z5cwmFO0YH/eWW3NV7 Rj6RIivBkPbJ0/nWxMwb1LbO5hCY4EqDQrfA/dGLg46JJMkDUMv+MOdFBGIRxSUB 7rR6G2w05gdEn/OL/0pNFM9nq9tPIANDrEYBgjo4uVN0OXKIdFuaSfc6SBRgd1Co OGCdjARP/6FhMpgc5mk5T59X8Z6B8QCufvZO1b7Mkw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:11:03 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 59e0a155 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:10:50 +0000 (UTC) Date: Thu, 11 Apr 2024 11:11:00 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 11/13] t06xx: always execute backend-specific tests Message-ID: <7136c8b6c21457515118c5084a715fed92b427e8.1712825204.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The tests in t06xx exercise specific ref formats. Next to probing some basic functionality, these tests also exercise other low-level details specific to the format. Those tests are only executed though in case `GIT_TEST_DEFAULT_REF_FORMAT` is set to the ref format of the respective backend-under-test. Ideally, we would run the full test matrix for ref formats such that our complete test suite is executed with every supported format on every supported platform. This is quite an expensive undertaking though, and thus we only execute e.g. the "reftable" tests on macOS and Linux. As a result, we basically have no test coverage for the "reftable" format at all on other platforms like Windows. Adapt these tests so that they override `GIT_TEST_DEFAULT_REF_FORMAT`, which means that they'll always execute. This increases test coverage on platforms that don't run the full test matrix, which at least gives us some basic test coverage on those platforms for the "reftable" format. This of course comes at the cost of running those tests multiple times on platforms where we do run the full test matrix. But arguably, this is a good thing because it will also cause us to e.g. run those tests with the address sanitizer and other non-standard parameters. Signed-off-by: Patrick Steinhardt --- t/t0600-reffiles-backend.sh | 8 ++------ t/t0601-reffiles-pack-refs.sh | 9 +++------ t/t0610-reftable-basics.sh | 9 +++------ 3 files changed, 8 insertions(+), 18 deletions(-) diff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh index 64214340e7..a390cffc80 100755 --- a/t/t0600-reffiles-backend.sh +++ b/t/t0600-reffiles-backend.sh @@ -4,16 +4,12 @@ test_description='Test reffiles backend' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=files +export GIT_TEST_DEFAULT_REF_FORMAT TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh -if ! test_have_prereq REFFILES -then - skip_all='skipping reffiles specific tests' - test_done -fi - test_expect_success 'setup' ' git commit --allow-empty -m Initial && C=$(git rev-parse HEAD) && diff --git a/t/t0601-reffiles-pack-refs.sh b/t/t0601-reffiles-pack-refs.sh index 7d4ab0b91a..60a544b8ee 100755 --- a/t/t0601-reffiles-pack-refs.sh +++ b/t/t0601-reffiles-pack-refs.sh @@ -9,18 +9,15 @@ test_description='git pack-refs should not change the branch semantic This test runs git pack-refs and git show-ref and checks that the branch semantic is still the same. ' + GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=files +export GIT_TEST_DEFAULT_REF_FORMAT TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh -if ! test_have_prereq REFFILES -then - skip_all='skipping reffiles specific tests' - test_done -fi - test_expect_success 'enable reflogs' ' git config core.logallrefupdates true ' diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh index 931d888bbb..fd0ddb96ae 100755 --- a/t/t0610-reftable-basics.sh +++ b/t/t0610-reftable-basics.sh @@ -4,17 +4,14 @@ # test_description='reftable basics' + GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=reftable +export GIT_TEST_DEFAULT_REF_FORMAT . ./test-lib.sh -if ! test_have_prereq REFTABLE -then - skip_all='skipping reftable tests; set GIT_TEST_DEFAULT_REF_FORMAT=reftable' - test_done -fi - INVALID_OID=$(test_oid 001) test_expect_success 'init: creates basic reftable structures' ' From patchwork Thu Apr 11 09:11:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625643 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7532142642 for ; Thu, 11 Apr 2024 09:11:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826671; cv=none; b=OBmnZDWAV1N6XpejA+QYAThQMBi0s9NJJ6ghakbWqqRLX6Zo8grLjHNGFBb6UzolVsKb/uat628AdJgSj9QaBFjuDx1JhWxYDggfgvAFt7xSyxN8W4OeC5FH/CEn5ndutEQLc20lIaB7AcEvjEPm6h2WxsoY7cPbFRRh8AiFU2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826671; c=relaxed/simple; bh=uHd+CRm64yrilAsejITRSIG5pxeOjd5S68kSZwpJrD0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=FBfRh2kt0h6GgdUGiFuYRN+J/16G2hXPpe1xnqTC+Rks8HQZr8RLB3JjYl/AAgTKqGWoMRmMu5uiAvf3LdJrl9gOdWtTiV5V5oTxivu6Ob2kiR+uo4Y7oU9vDE0OkFDmwg6MX/LX9XEEzI8BI3r/upSLoGqfnXZ2htMusyklYNo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=CLzedELG; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=j7zzEaRv; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="CLzedELG"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="j7zzEaRv" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfout.nyi.internal (Postfix) with ESMTP id C543A1380249; Thu, 11 Apr 2024 05:11:08 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 11 Apr 2024 05:11:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826668; x=1712913068; bh=p17Hk86N1S Qdz9j034h2CkWHBQhy2eKA2Ri7IFVYdmI=; b=CLzedELGnSUmxvGjvt7S7sFD4K uK5EP8AlFcqh8xeh9aIqGVmRVNkBNn7BKtCUk7T8gX1s6M1a1fDxs8AYxSugvoYv 7lsIwwIcq3H27YDEGsD954nodRD1+6D5YqpFndCVYwJyPwOyHBJTJ4Er9+/MPyy6 zsRvI4k4GssgbC6cbydUBBkLBkdv2clT1wDNsDWndZeJkT8615c9fGHb/F7/6wk0 sSIEvEFQSTtK/8HER4wB7iBCnhNoBx70bBMlSkCB5DCunm0yyvmrUIpC127d/APN IoxCyRMlHREL8t7zE8QS6AsU4MMluO+g9ahyRuBxjqwsiNKQ/PVFJh5y3Huw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826668; x=1712913068; bh=p17Hk86N1SQdz9j034h2CkWHBQhy 2eKA2Ri7IFVYdmI=; b=j7zzEaRv+5weI6xUbvRjojwU/gxqr6SkyZKp7g7G7UiA kuTUBVD1FLWxfWkYlpnCq8Oq8pN9Gjyi3mndO+qi0kr7aGTb0c/xkkXl0LxLt+Cz SIOG1hn/kOeijFbVinubEL+g+AqY8dp39mgRp9UE/TkaY37Cz79Qmr9QGu/5mwXb KMazXvyvCm6Mzdk0G8CxXao6Op5dAy5YAndNtgcSkKh3zQRKNTzbKn9+hF4pR2YH kN7OR4t6F6Tq1qW8RJgfavINyelbbozDtgXPUTd4BagHYmce0npTVI4rO91QGH0x ExfOJdwGNhZ1DeSn4ySBN0hMBX9inu7YY7GBLEpXRQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:11:07 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 9604cf71 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:10:55 +0000 (UTC) Date: Thu, 11 Apr 2024 11:11:05 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 12/13] t0610: fix non-portable variable assignment Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Older versions of the Dash shell fail to parse `local var=val` assignments in some cases when `val` is unquoted. Such failures can be observed e.g. with Ubuntu 20.04 and older, which has a Dash version that still has this bug. Such an assignment has been introduced in t0610. The issue wasn't detected for a while because this test used to only run when the GIT_TEST_DEFAULT_REF_FORMAT environment variable was set to "reftable". We have dropped that requirement now though, meaning that it runs unconditionally, including on jobs which use such older versions of Ubuntu. We have worked around such issues in the past, e.g. in ebee5580ca (parallel-checkout: avoid dash local bug in tests, 2021-06-06), by quoting the `val` side. Apply the same fix to t0610. Signed-off-by: Patrick Steinhardt --- t/t0610-reftable-basics.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh index fd0ddb96ae..b6e67724ce 100755 --- a/t/t0610-reftable-basics.sh +++ b/t/t0610-reftable-basics.sh @@ -78,9 +78,9 @@ test_expect_success 'init: reinitializing reftable with files backend fails' ' ' test_expect_perms () { - local perms="$1" - local file="$2" - local actual=$(ls -l "$file") && + local perms="$1" && + local file="$2" && + local actual="$(ls -l "$file")" && case "$actual" in $perms*) From patchwork Thu Apr 11 09:11:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625644 Received: from fhigh1-smtp.messagingengine.com (fhigh1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56CBF1448C8 for ; Thu, 11 Apr 2024 09:11:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826675; cv=none; b=BD7bjAcxwcKiYA5VvsdIxAEVq+HDHp72yGvVuGcXQBvNJ68LpLcdLAkgpy21LHq6pt/7Asw6JktFaF+MQ5McCdqC2lsslkpdGAkQbg2rO8wIKBHrt/zSW0sj6lqzkLWWAitj9p1NHxpOOd/LgsaqOhEw4sCBR4tSzs0j4x3r/zw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826675; c=relaxed/simple; bh=j4v++WAgSF6wM8d18uCMI5TmhqyXBvJ/veVexaGB4kQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=GYXauXSqshjoeRE0jBYeHGnIn0ChA9n9uAcjjn39obV0FgiGh6WrdDBbKqKRRq0ELZmWNTodZQwco1l9k3YvScBmV0JpAWLTBCgmqhRt/OsPxARyJ6KTLvMAA3W0lCoLRnOkQNEfflI4VxXL+67ETu4EdUeA+cJlX91z8ayEOaE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=KWcCYQrF; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=H0aSJkLF; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="KWcCYQrF"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="H0aSJkLF" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 97CF11140198; Thu, 11 Apr 2024 05:11:13 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Thu, 11 Apr 2024 05:11:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826673; x=1712913073; bh=6aSWjJ12Fu xT9lPiMEBv9PFjB3APa8lRCKefDAMrz88=; b=KWcCYQrFKW1dDTznnzeE1eFMQT 2C8GobMmUilS8w4E5n/LBbKgpyK7Fw1OoBREAAeALgl1hDCVvx4cYQ20yQ+4ajrU cNs0HtWXF2YtebFom7evigrSNgzG/Xht849pOihj+iPvynSc84dCU7gq7ZzdTMBt M/FZ8R1432DunlMGgT9dzjZs5iTedpbqTmvJVhAKO0JGtJ9Q33Zm8fcsiyctlK7d HsP4IpGyMkXf3kO9C67LxePHoc73j+w0PyG+5quwT+UmTqpFCa9vVH9JdVmWoZZl h7L74jmkubVYORiZMv/dvbi1JkNBubGuO9eN/Ei14fNwAq3y8V3GY0YfHFrQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826673; x=1712913073; bh=6aSWjJ12FuxT9lPiMEBv9PFjB3AP a8lRCKefDAMrz88=; b=H0aSJkLFh42IBEZkVDnZmKgi1HVQIVPXqMndVuq/SJMN cQGVL2Z6M4dxm49Hmqwy+GywZcNYQ3j88ycxFMob9YDNja40uitDk3n4B8wCFT1d LrQcJTGRjUCj3yFFdd1+zIROWzmBOcn9T53KM1gQ9xD0ouwkIPEjeKSaOiM3qjIp rDzVv6uesa/yrfPRZAD7cVxZld3MB3NTKnhGaRm9YRgHpQgdIGPoG11kX/L+FNsJ +0czil0UFPxVQ/3Evz+oq2Vj9wTXIj9vfQw8P5iFkHcNZ/ovp/zBY4/32re03JXo BEFXPfVd5FZMtK1/+0LSI/7/lR4okAjWm3eVv0KpCg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:11:11 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id c3240194 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:11:00 +0000 (UTC) Date: Thu, 11 Apr 2024 11:11:09 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 13/13] t0612: add tests to exercise Git/JGit reftable compatibility Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: While the reftable format is a recent introduction in Git, JGit already knows to read and write reftables since 2017. Given the complexity of the format there is a very real risk of incompatibilities between those two implementations, which is something that we really want to avoid. Add some basic tests that verify that reftables written by Git and JGit can be read by the respective other implementation. For now this test suite is rather small, only covering basic functionality. But it serves as a good starting point and can be extended over time. Signed-off-by: Patrick Steinhardt --- t/t0612-reftable-jgit-compatibility.sh | 132 +++++++++++++++++++++++++ 1 file changed, 132 insertions(+) create mode 100755 t/t0612-reftable-jgit-compatibility.sh diff --git a/t/t0612-reftable-jgit-compatibility.sh b/t/t0612-reftable-jgit-compatibility.sh new file mode 100755 index 0000000000..d0d7e80b49 --- /dev/null +++ b/t/t0612-reftable-jgit-compatibility.sh @@ -0,0 +1,132 @@ +#!/bin/sh + +test_description='reftables are compatible with JGit' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=reftable +export GIT_TEST_DEFAULT_REF_FORMAT + +# JGit does not support the 'link' DIRC extension. +GIT_TEST_SPLIT_INDEX=0 +export GIT_TEST_SPLIT_INDEX + +. ./test-lib.sh + +if ! test_have_prereq JGIT +then + skip_all='skipping reftable JGit tests; JGit is not present in PATH' + test_done +fi + +if ! test_have_prereq SHA1 +then + skip_all='skipping reftable JGit tests; JGit does not support SHA256 reftables' + test_done +fi + +test_commit_jgit () { + touch "$1" && + jgit add "$1" && + jgit commit -m "$1" +} + +test_same_refs () { + git show-ref --head >cgit.actual && + jgit show-ref >jgit-tabs.actual && + tr "\t" " " jgit.actual && + test_cmp cgit.actual jgit.actual +} + +test_same_ref () { + git rev-parse "$1" >cgit.actual && + jgit rev-parse "$1" >jgit.actual && + test_cmp cgit.actual jgit.actual +} + +test_same_reflog () { + git reflog "$*" >cgit.actual && + jgit reflog "$*" >jgit-newline.actual && + sed '/^$/d' jgit.actual && + test_cmp cgit.actual jgit.actual +} + +test_expect_success 'CGit repository can be read by JGit' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + test_commit A && + test_same_refs && + test_same_ref HEAD && + test_same_reflog HEAD + ) +' + +test_expect_success 'JGit repository can be read by CGit' ' + test_when_finished "rm -rf repo" && + jgit init repo && + ( + cd repo && + + touch file && + jgit add file && + jgit commit -m "initial commit" && + + # Note that we must convert the ref storage after we have + # written the default branch. Otherwise JGit will end up with + # no HEAD at all. + jgit convert-ref-storage --format=reftable && + + test_same_refs && + test_same_ref HEAD && + # Interestingly, JGit cannot read its own reflog here. CGit can + # though. + printf "%s HEAD@{0}: commit (initial): initial commit" "$(git rev-parse --short HEAD)" >expect && + git reflog HEAD >actual && + test_cmp expect actual + ) +' + +test_expect_success 'mixed writes from JGit and CGit' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + + test_commit A && + test_commit_jgit B && + test_commit C && + test_commit_jgit D && + + test_same_refs && + test_same_ref HEAD && + test_same_reflog HEAD + ) +' + +test_expect_success 'JGit can read multi-level index' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + + test_commit A && + awk " + BEGIN { + print \"start\"; + for (i = 0; i < 10000; i++) + printf \"create refs/heads/branch-%d HEAD\n\", i; + print \"commit\"; + } + " >input && + git update-ref --stdin