From patchwork Fri Apr 12 04:43:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626834 Received: from fhigh5-smtp.messagingengine.com (fhigh5-smtp.messagingengine.com [103.168.172.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAEC21BC23 for ; Fri, 12 Apr 2024 04:44:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.156 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897045; cv=none; b=I6UoIQ9231CCUdyRC6BnFemccYzKHK5QOCJuerRO5XhvI7Dwh/Wk4bLgizIxGbe5f/vuuQjvVOyU49245Klg/kgkOhWM/8Xwh5UeOkZQI1631LwtxqAt2HpWtP4vERBjvChn6IFj0W6CmEQ8dLcfwpiwJncmQYScq2aA74LkucE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897045; c=relaxed/simple; bh=ycbp+YcUsmXszMOkQCbJL8Db/GaF8D/ZugsGC59cX10=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=J9+70lTifAqHThW46BAIYhw+K+91UpNbqKj6QnBB2NoGvv4rXrj7ogjDXiLr/pu7kk/v8zSUreAR344XamKsi3OJRgT306QtkPyg1SGMdxXr880klc4xxfHVFCB65o6U7+FGo6eebwnRma4rQxdGMyZQ8eBsI625i1sr/u7O7dc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=aMRbKAMe; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=kAocKsQD; arc=none smtp.client-ip=103.168.172.156 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="aMRbKAMe"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="kAocKsQD" Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfhigh.nyi.internal (Postfix) with ESMTP id C871011401D1; Fri, 12 Apr 2024 00:44:01 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Fri, 12 Apr 2024 00:44:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897041; x=1712983441; bh=fUFQu0AMvW FG37/QXKbinhWw+OzgpQ2+yU7ty1t79wk=; b=aMRbKAMevgbP0oK+nInPXi3QuZ /H8lzqSNiF9Ys1tSqVltSBDd6VWeO6DzIwzjV9v+64Tcx5/zl5chnIiRrAXFheGr 4Dtr5P0Sysb9GQMiOrPxp53MWUx1xgEAGv5bDrTxiNQVfIgiWlcOCV2nfIY+kxtP UjMxEGHTHMgcI7M9ijLzvPMJYO9RtEAzaQ2u7Mcpr/+h13JExflap7sncKWFZxd9 xR8nCseAkW8RIx0mEdN6WPtvnU+GQS3Ung9iZEwwhA44o+5ufKQsxpcnex+54H27 THYG2MyDEHTClH4DPRAqnPicc0Qq5eC/Q80t1c7eyvaZuWwUTMxuzYMC67HQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897041; x=1712983441; bh=fUFQu0AMvWFG37/QXKbinhWw+Ozg pQ2+yU7ty1t79wk=; b=kAocKsQDzP8ZSqnexSc7ZqqMgRbYORT5YZZWy0QMialM 53xbC49gwsMI2xNV5LoR4qF4+j6yST18gmxjTOxrEkIuzJ1SzkxnyrJfjADEhuU/ c7GWG+ap6hzINx5HC8zAjuSkKpH0f8xrA9zj3gb+3ybdM6F8T20AvAI/aCchgrBt vX3yobt+hvT4r0hlAbcHZtY+4ehrcg0ubYA32U1jFgg4jTsLj/SV3VC8qy6f1bIG rO1s504N9Iesm2oauEFK8a+S/yMx4cFCmw9a3tmTdDN36KZNHoFbvXoahd1Uv4eP IFX4WSEKDKM2tcj/tQj9BHa6lGtEeH3eOeGeOTIqKw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddgheduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepieefvdehgfekieejueektdelhfeiheeltddtleekieevieehleektdfgveetkeet necuffhomhgrihhnpehvvggtthhorhdrtggtnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:00 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id b1588bd1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:43:46 +0000 (UTC) Date: Fri, 12 Apr 2024 06:43:57 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 01/13] ci: rename "runs_on_pool" to "distro" Message-ID: <99b0b60359c1466fd291e890b8403d9cbfd706c8.1712896869.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The "runs_on_pool" environment variable is used by our CI scripts to distinguish the different kinds of operating systems. It is quite specific to GitHub Actions though and not really a descriptive name. Rename the variable to "distro" to clarify its intent. Signed-off-by: Patrick Steinhardt --- .github/workflows/main.yml | 2 +- ci/install-dependencies.sh | 2 +- ci/lib.sh | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 3428773b09..684ef5c00d 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -303,7 +303,7 @@ jobs: CC: ${{matrix.vector.cc}} CC_PACKAGE: ${{matrix.vector.cc_package}} jobname: ${{matrix.vector.jobname}} - runs_on_pool: ${{matrix.vector.pool}} + distro: ${{matrix.vector.pool}} runs-on: ${{matrix.vector.pool}} steps: - uses: actions/checkout@v4 diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index b4e22de3cb..7d247b5ef4 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -11,7 +11,7 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" -case "$runs_on_pool" in +case "$distro" in ubuntu-*) sudo apt-get -q update sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ diff --git a/ci/lib.sh b/ci/lib.sh index 0a73fc7bd1..d882250db5 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -279,7 +279,7 @@ then cache_dir="$HOME/none" - runs_on_pool=$(echo "$CI_JOB_IMAGE" | tr : -) + distro=$(echo "$CI_JOB_IMAGE" | tr : -) JOBS=$(nproc) else echo "Could not identify CI type" >&2 @@ -318,7 +318,7 @@ export DEFAULT_TEST_TARGET=prove export GIT_TEST_CLONE_2GB=true export SKIP_DASHED_BUILT_INS=YesPlease -case "$runs_on_pool" in +case "$distro" in ubuntu-*) if test "$jobname" = "linux-gcc-default" then From patchwork Fri Apr 12 04:44:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626835 Received: from fout1-smtp.messagingengine.com (fout1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C3761C2B3 for ; Fri, 12 Apr 2024 04:44:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897049; cv=none; b=X0R3ZH3v1Cs1DIpLDU4QQ6adOVcEY6k2cwXIbF3mfsx3L/7hwJ+NnnCvpeoLZSO6aJeX5z13IM5+Y90u7+7anMXhzgonx5nO7N9JmG9lM02rYwUuZZgYjMkO6tYc0ss2OM4xlvxi+4umJtPVx3lp3j7AhLuR8HltFeu0Q9c6mAY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897049; c=relaxed/simple; bh=HjiV6nbLoBvFdEXnMaFm8/Z7Zsx9cLEr2PFXf2GGOC0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IFS3+NTKmri1VFe3sJI1Qtf0x7ZFCXkmmFUPH4FXwBjfXKDzsoqxP7cK4j/jEYOL3mXtgj6WOXiSd0XE9xtZWocltfvFsnxOPJiS5gWyC34hjEtIQFjBETF/lTQqPelIXPotQCJMYNjUnCTXZMxLR+rMv/guoAR1CbU0tYot3us= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=LFH4CxTf; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=DFNViOlq; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="LFH4CxTf"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="DFNViOlq" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfout.nyi.internal (Postfix) with ESMTP id 7A71813802F7; Fri, 12 Apr 2024 00:44:06 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 12 Apr 2024 00:44:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897046; x=1712983446; bh=3iiJbImMl6 qe07299cG5yYdsDcYU6AQFCKtMofwCUWg=; b=LFH4CxTfBACtRLlu6ZGkMDlDCa +V74tw4VJXcu/eqnALPahb1KK7Z2zoiOWbe0OQ6J+CF5WyoGwTBIqtBvHeyTOA64 2z/fUDsSaOO8l1n07/RulDiFmFbOucfOyDtAfdqkPZFacMm/dk2BQrn3M94kJzNS Tkoom5HN/CQj/JCWzSV+VMkN7ONyJrfkwSoUt3ba09yt5dx/jMru4YPumjrwOSEv xuP9niumKIZtUxyi0mMF5vRbUqly7gV3z5WX7i/kn3kkGwQP8x22ZWk9H3bpOe8B yCxEQ4AMzBnfo/ASz0WSDbqnj6GGthknnd2zJyvUrITxKSiXTCu/+kmyu9Qw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897046; x=1712983446; bh=3iiJbImMl6qe07299cG5yYdsDcYU 6AQFCKtMofwCUWg=; b=DFNViOlqbUQQyKkllgGp+6Bia/xdpI9Q/jxHSQpo2Yr2 OY6osGuzYlQnEJJWj5nFNkvK5I56nW7mJnorfn4zVPztuKjScNHZnxCtfz982Cl4 p4xoJjM4mri8OPDwdqwX+VyKyIoFJRkTUb1A2fu7sNi6ZCCdFgYra6SlKi3QqOCj DZ4wJHA3xSWVEAwm6F+rnOir8I84RYm64qejxsD2JMIkuK8cjF+kgXG6Vft5wAF2 L5vy1y0taJ6zmcByyZSgvyZvwyfwfJc/1DIGZ4uDQ0YhceA3/V17FwgLTJm5k/4a P1kgKk6P3a4Y5ErKvgyARarDRHZChdY/DSO7uii2rA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:04 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 2090888f (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:43:51 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:02 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 02/13] ci: expose distro name in dockerized GitHub jobs Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Expose a distro name in dockerized jobs. This will be used in a subsequent commit where we merge the installation scripts for dockerized and non-dockerized jobs. Signed-off-by: Patrick Steinhardt --- .github/workflows/main.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 684ef5c00d..71cd4e5486 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -342,12 +342,16 @@ jobs: vector: - jobname: linux-musl image: alpine + distro: alpine-latest - jobname: linux32 image: daald/ubuntu32:xenial + distro: ubuntu32-16.04 - jobname: pedantic image: fedora + distro: fedora-latest env: jobname: ${{matrix.vector.jobname}} + distro: ${{matrix.vector.distro}} runs-on: ubuntu-latest container: ${{matrix.vector.image}} steps: From patchwork Fri Apr 12 04:44:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626836 Received: from fout1-smtp.messagingengine.com (fout1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CC9C1D54B for ; Fri, 12 Apr 2024 04:44:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897053; cv=none; b=RAvnq9UW4Y9WOdoPXL8tZeCpw0qVKbqZ8qotoWqLbMaB6xc3P5K501+g6ZnhzUxmbybNL0f4TJJ5n2dvuGvNHRABPz95H17UsO7Huehonu3V0fOZt5/N8h5ie+EXwCXazWSA6QsLkbWj58ZEqAOUu0u8sNP/mmRwCGNC5XgE1Gg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897053; c=relaxed/simple; bh=pZnXdx3aZZ1cjfNPvV3iayvnukr7aDKMgK/U6+zxFL4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=BrFfLq1Cq5oiAyMoLSvHGvV1O0WdM2a2Iox7gq0DH6thvAJ7IHYxohvrZgX451NVjBKU92rQYvfnkyS9D28/H1o3slr2r54zsXYS6XAvhKyCxtl5XVWF1agsHTSMqzUsqqVW2oae8pKfCMPzvLpcERHDwTzyve5fcqUylmkjRIQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=WWyTeuv3; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=NogiDGkO; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="WWyTeuv3"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="NogiDGkO" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfout.nyi.internal (Postfix) with ESMTP id 2C5FC13802F7; Fri, 12 Apr 2024 00:44:11 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 12 Apr 2024 00:44:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897051; x=1712983451; bh=aihsZgRb3/ 4V16/Gcjo4TKpsma8zxZnR5+5zNslb4V4=; b=WWyTeuv3jUGgF+rhqYyFDPULRt QmSuWuDRzXuN69yTZJcV/2dsONONAXk70Nrpfp7Wybfcsma7lEBXwQo2/+SJEkXB qmO2juPF2Zr5Tsrr50/VJqhwgUouEGwc7pcujD8RnuVy6sHv/GaFq3QgRGGRpmO3 QMFLEc1htCN2o+JJl1pvI9+0tilnap59UsjeG3f8kvh01er64mt9uRhsgbIXWEpD pU7bijaC6u9D1Fu7mEzdIAfoUTeAlKqYXIIvwatarb+ohGkKn2cXLp/DgWYFnX3J 4LsnHCgxZvp2NXMb5z9/mfjUMZzrZHVKENl8lWo/TAEs3SmKGylJ+r61O36w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897051; x=1712983451; bh=aihsZgRb3/4V16/Gcjo4TKpsma8z xZnR5+5zNslb4V4=; b=NogiDGkOMSwWENL00owWrGB1MGgurS0D8Tf2Wgy9eBZF tUYAmoa3Bt61giYf7SBvKeBnxFPLKRYW0Bisbq4g45a4Uqc2Kh7p2JzRXLmiKCyy BpTiiqExbdMGzkvNWwRSlsu2odocezC/KytsrUH0n3ikWwsUjN21CX90Zz50hqNV vu2ayBhYKFY4eB+mx1An0uDZadmIylXhyR6wRh7wJEk3EAeA/j6cRD+IYcORcZ8A 229ykHU8d6eOeow71qpAswpAkQ9rRSR35/mlA2zH/ci43eSbojJgNhP3gPPuD0ZX PXamKD+WVeVg7NmBLLIgJatlINwwkCKHWopjilUPpA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:09 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 97123517 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:43:56 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:07 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 03/13] ci: skip sudo when we are already root Message-ID: <44a0e0760010a585a63e61b922783157227d3fb8.1712896869.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Our "install-dependencies.sh" script is executed by non-dockerized jobs to install dependencies. These jobs don't run with "root" permissions, but with a separate user. Consequently, we need to use sudo(8) there to elevate permissions when installing packages. We're about to merge "install-docker-dependencies.sh" into that script though, and our Docker containers do run as "root". Using sudo(8) is thus unnecessary there, even though it would be harmless. On some images like Alpine Linux though there is no sudo(8) available by default, which would consequently break the build. Adapt the script to make "sudo" a no-op when running as "root" user. This allows us to easily reuse the script for our dockerized jobs. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 7d247b5ef4..7dfd3e50ed 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -11,6 +11,17 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" +# Make sudo a no-op and execute the command directly when running as root. +# While using sudo would be fine on most platforms when we are root already, +# some platforms like e.g. Alpine Linux do not have sudo available by default +# and would thus break. +if test "$(id -u)" -eq 0 +then + sudo () { + "$@" + } +fi + case "$distro" in ubuntu-*) sudo apt-get -q update From patchwork Fri Apr 12 04:44:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626837 Received: from fout1-smtp.messagingengine.com (fout1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD85E79DC for ; Fri, 12 Apr 2024 04:44:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897058; cv=none; b=arrSAanrhk3nIxEbE5EBq545ikY4YsP/MGj7ZdDN/wR2ZPCWDMDSkXwFjvnAgbyNxUKB5VNdW91DneRxBJFSiI1XqhtcBR+Zk/ktAinF1FwjkurEzt0g3JOqDQt2Udi4lEhEwx+NgKNWhbgDBTVS0o6bsTLZa8rPEIC4mqkDMX4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897058; c=relaxed/simple; bh=mB8bRW1DwUVksyvLaoxZ3hHFGrhOayMCXJCeRpbyDCE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=tYki4SzXpmaT0k/d/dlVNo7f1jGfchmnpdv8/3OXD1AySrlfCjib85458QjS0J0e2gEjRWAiRvORAuq02UraKFUC5y1bpBB1Up3xAcNMe0r1HH0Trg4m3f0/RaLrkkvorfuyg/4rD5e0Roisdyols29bvB7Ksf6NHo+gJvwYSpY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=Yqrr4KpK; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=lJ6ysOhN; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Yqrr4KpK"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="lJ6ysOhN" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfout.nyi.internal (Postfix) with ESMTP id C119313802C7; Fri, 12 Apr 2024 00:44:15 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 12 Apr 2024 00:44:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897055; x=1712983455; bh=+NI+xjsE6z BNNMO4GEDFkhk63uWNpUIlEEB2b+S4goU=; b=Yqrr4KpKMjpXILqmzekYjChbS2 H04nJO0nE3T8p9J+4EvEiQNAX6emJqxD4F2IoRnHqPugfRsDGptOiyH3f9yaElpI hrzz2FblUGLw35wYUYqaL/u31pfrJWiFhFsy9e9ioMo7EsRCkgn5KYygUqE9Yf5L b1zXnhHbdd3AJKXxXQEBoq/3hh36ViroemEnxFbqGjd09951twyzsMwSLvbVDj3O li5ewRVGNYg1/1Ub28ERyp7IwxEinpA2puPabXR5yMOFs9vU526wRg5Ef8bOjgR1 6WPlioCGMvJSpQ5BSM3i2Ko5C/+N3ESjR7r7ZbuAWCc3D3zpX7tn6IL8bJmg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897055; x=1712983455; bh=+NI+xjsE6zBNNMO4GEDFkhk63uWN pUIlEEB2b+S4goU=; b=lJ6ysOhN7IAjYc7VADIJXRvmIKoUYTcN9y+aN9cGkeAT ARRm2ge/4D0m7ZMzW5J1JwIcgBbBLWsemuHr1rYldk6QQeWtoufnCBxhSi/OJq6y d/CWoAcHgKinkvB8+pcqiw8KrXRJTmFcUf2F3CC+Zni993G+f1OIpsztVS94srJ8 W+td9CM7HmgDna6IzROSYmsyQIU1E7vtwOffLv4wZzMp6juBJPF0WyHLli0RFa+D y8GF2RyrxAPaEii0GADyLHhjjWRJq6A1Kb5MTs3EZW6NEa/Cy8+9GaIPK63TknOI wvMOjcIpxuqGGBa9Ok0IzUzF+7sehWAEdjGVFhDrdg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepjefffedvgfelhfejkeefteffhfejudfhhfdvtdevjedvhefgvdduhfdugeegkefg necuffhomhgrihhnpehpvghrfhhorhgtvgdrtghomhdpghhithhhuhgsrdgtohhmnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhs rdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:14 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 40208adb (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:00 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:12 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 04/13] ci: drop duplicate package installation for "linux-gcc-default" Message-ID: <8e58ce38d9a8a9465d172d3ef091a7a7cdde3393.1712896869.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The "linux-gcc-default" job installs common Ubuntu packages. This is already done in the distro-specific switch, so we basically duplicate the effort here. Drop the duplicate package installations and inline the variable that contains those common packages. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 7dfd3e50ed..fad53aac96 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -7,9 +7,6 @@ P4WHENCE=https://cdist2.perforce.com/perforce/r21.2 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION -UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev - tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl - libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" # Make sudo a no-op and execute the command directly when running as root. # While using sudo would be fine on most platforms when we are root already, @@ -25,8 +22,13 @@ fi case "$distro" in ubuntu-*) sudo apt-get -q update - sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ - $UBUNTU_COMMON_PKGS $CC_PACKAGE $PYTHON_PACKAGE + sudo apt-get -q -y install \ + language-pack-is libsvn-perl apache2 \ + make libssl-dev libcurl4-openssl-dev libexpat-dev \ + tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \ + libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl \ + $CC_PACKAGE $PYTHON_PACKAGE + mkdir --parents "$P4_PATH" pushd "$P4_PATH" wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d" @@ -34,6 +36,7 @@ ubuntu-*) chmod u+x p4d chmod u+x p4 popd + mkdir --parents "$GIT_LFS_PATH" pushd "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" @@ -83,10 +86,6 @@ Documentation) test -n "$ALREADY_HAVE_ASCIIDOCTOR" || sudo gem install --version 1.5.8 asciidoctor ;; -linux-gcc-default) - sudo apt-get -q update - sudo apt-get -q -y install $UBUNTU_COMMON_PKGS - ;; esac if type p4d >/dev/null 2>&1 && type p4 >/dev/null 2>&1 From patchwork Fri Apr 12 04:44:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626838 Received: from fout1-smtp.messagingengine.com (fout1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 64649171A7 for ; Fri, 12 Apr 2024 04:44:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897064; cv=none; b=RU1QLyun1OPZ/nS9jY6zEgAP7cewBtty31++UgbBU2hcmfk3uYWIv6DIlF11J9Dune1j02k60HQ3uA/qZN9AfS3xzkF6ZuACyAez+gQnJFzIuvM4R4kOZJZ3gIf07psxGj7Fj4RnFzLyzSZmjzHGnMpSnsoND3f11lhYbdYxUqQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897064; c=relaxed/simple; bh=vC4++A7jEZqKU2WtEUpW29k3PSOM+rm1oGDQcBqGOls=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=LTUoEPWlCAgJKKSYjH7OOCpoCb04HRcNnKmiuzPNEiFfAps2WWfumD6gV4zqInqxU9ziQB7c4ec+HE/yFIm8EF03zr/vfynLDRBIaZegCWcOB2Ov3tqt6fhYxKyVO9xdKFJq7tAwuWUzMWsEEroPbCRHEVaUyG5RBZY211XBHf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=dC8PKmkO; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=FlcyeVu+; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="dC8PKmkO"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="FlcyeVu+" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.nyi.internal (Postfix) with ESMTP id 6B65613802C7; Fri, 12 Apr 2024 00:44:21 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Fri, 12 Apr 2024 00:44:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897061; x=1712983461; bh=izbZvmkmR8 5zHHWTj+d+j0a27rjfATGifY+2is3D6QE=; b=dC8PKmkOFB/DxWPZRMDylQai1U efW3o5Sd46ACu6mdghsYZAh1pXuiRpXqF1G9v50mo5FaC6uZSm+iBsJzauxIqsf9 6hsrgDXagQQMoMWeVj9nmYspiuSnLp3tDDXXY43TY6hEBfBucCq7eeniOfee8eqz o4FXpfErKn22WWTUI1w8YQz9BawuBC0Vw9lORInGJI1t+WzROd5hPQ7bbwkwYh+C Igv3LT1Tui8sBsW7Q69ZqXUPAX2lRdn5hpKQEdyCcDPi4sFpdgtTNakWpUH2RKJz BF5lnCRUzTUJZ0cDf9oYQZY9mlIsrXIuH3+kPl1GFyRAZtbzrBSBAy/+aCJg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897061; x=1712983461; bh=izbZvmkmR85zHHWTj+d+j0a27rjf ATGifY+2is3D6QE=; b=FlcyeVu+lRIYnXLElbFQF5IhCQex4VOAZhk6w1SFK5cd XUZ4bpvPhDvmUI9VodBIKn3hFn8+pbwHOpir0EyA1hsl1eNrbJlwR92Axi147vN5 KQxS8ZHmWUxAscJxGTR+3UI9Bm429EKitfiAumM/kLP+46sPYqTn0+LEJh+CBj3Q TuPlKqXzdhVrJIFHhqSI/XCwXOC6mLGXigZH3qWZxIcBnq2XExD3md1Odh7/2z+w UIpw2vNkB4gj8GUWdiJsU00HFV6KwokcoE2cDNxSUm/LyAgqBEH59spq7kyc6/6y cypRCaeNwcvQqLbvIPWg263LzoZliNqKgAMFCDGVRA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddgheduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:19 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 1778a7d1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:06 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:17 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 05/13] ci: convert "install-dependencies.sh" to use "/bin/sh" Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're about to merge the "install-docker-dependencies.sh" script into "install-dependencies.sh". This will also move our Alpine-based jobs over to use the latter script. This script uses the Bash shell though, which is not available by default on Alpine Linux. Refactor "install-dependencies.sh" to use "/bin/sh" instead of Bash. This requires us to get rid of the pushd/popd invocations, which are replaced by some more elaborate commands that download or extract executables right to where they are needed. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 29 ++++++++++++----------------- 1 file changed, 12 insertions(+), 17 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index fad53aac96..2d6af876d6 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -1,4 +1,4 @@ -#!/usr/bin/env bash +#!/bin/sh # # Install dependencies required to build and test Git on Linux and macOS # @@ -30,19 +30,15 @@ ubuntu-*) $CC_PACKAGE $PYTHON_PACKAGE mkdir --parents "$P4_PATH" - pushd "$P4_PATH" - wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d" - wget --quiet "$P4WHENCE/bin.linux26x86_64/p4" - chmod u+x p4d - chmod u+x p4 - popd + wget --quiet --directory-prefix="$P4_PATH" \ + "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" + chmod u+x "$P4_PATH/p4d" "$P4_PATH/p4" mkdir --parents "$GIT_LFS_PATH" - pushd "$GIT_LFS_PATH" - wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" - tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" - cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs . - popd + wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ + -C "$GIT_LFS_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" ;; macos-*) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 @@ -53,11 +49,10 @@ macos-*) brew link --force gettext mkdir -p "$P4_PATH" - pushd "$P4_PATH" - wget -q "$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz" && - tar -xf helix-core-server.tgz && - sudo xattr -d com.apple.quarantine p4 p4d 2>/dev/null || true - popd + wget -q "$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz" && + tar -xf helix-core-server.tgz -C "$P4_PATH" p4 p4d && + sudo xattr -d com.apple.quarantine "$P4_PATH/p4" "$P4_PATH/p4d" 2>/dev/null || true + rm helix-core-server.tgz if test -n "$CC_PACKAGE" then From patchwork Fri Apr 12 04:44:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626839 Received: from fhigh5-smtp.messagingengine.com (fhigh5-smtp.messagingengine.com [103.168.172.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04F471BDDF for ; Fri, 12 Apr 2024 04:44:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.156 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897068; cv=none; b=pyR/ITBlcKlO6IbWn6DrvLZ5mnjuK8FODB4ON4osCdOQ8mbbku9Wqa2tAgLlDjr2e0qd8u1YZXqKt5Ts9tjaFOOmDPrvZKjBGzbEE1RvhJEW5ax5bL79a3E8hl3sMoEGrSjVS6mtwss5rAutm5QaoMx1I+xSJIAoDhwkaDIOaCA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897068; c=relaxed/simple; bh=S/8Xtr2KskHdtdaZU265EFtbpjZdAU1EpuqZzAu0cE0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=gE+8KC65BOoH7LzoN2gihq0ZBAmddDxbWoNOeBE541XvLnXfQYwOxiqxYtSr9wo8WW9jwvIiJ7Tup2XxJCnlWL4A+SW6oLejAS+bBpjrhqde73DVndkWPvab7v3kKZyhWVFGwX1/V5IPpWUGg3GUpHz5yKdBEu+TdvA8FFRORvU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=PNL/ZaRZ; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=RiBymUQ+; arc=none smtp.client-ip=103.168.172.156 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="PNL/ZaRZ"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="RiBymUQ+" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 124D511401DC; Fri, 12 Apr 2024 00:44:26 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 12 Apr 2024 00:44:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897066; x=1712983466; bh=D4Z1RNPwMt EIe75LBeFODaJyQ28maiABfI/8Z3OO9sQ=; b=PNL/ZaRZEArohT3g01/qkdfwZ/ xdDOeoGvCR4Sbm2Q8O2bZjaLBJ6mKN1tl8XDvPVTWXe4R4LD9Ay9Y1WrzbANqFU+ oGBL4EZjiCmWA0hxSpb+kUrDgdlv3M/8djVNOBoZgJBiNSpGVSKaKY+B8pKrX5du 9/nzExR8tOOHnVi7XBxCcLJEPfcE5zXh18mWhmK/PEenGHLMriFhfRvGkhuSxlc/ a+FO+ZCRi/gCeZBDo2RNGJSMfcm3zufZWrEZ4zqY9jI3SKCJqeZktUfkfygoTb1A AkuR3EV5x/V5brWcTcU3bhggKoaXWJpBn38aGHGfBIao98c0sAGMBqDi00qQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897066; x=1712983466; bh=D4Z1RNPwMtEIe75LBeFODaJyQ28m aiABfI/8Z3OO9sQ=; b=RiBymUQ+7PFS6d2jbuZVB870i8vM78fDmyAkW+JaFr2f hSwaJurlJuTn8WBqMeYNsVOMqzKEjd6F4WO9Ve7NANzoS2JNiQxWYVgaBkOZ0sNa sKgc1MKULMmcHD8hTDc2oSrwdqFIguxR2uYsag8pS0SYOZcr0V5wxMsrzMXqRMFX cNsCth1VN1/5I8yvM07gptorOYw2xkwD0QLv7KZDZBrIcwc3KB8KF1sWKCiLZOLv P/nlKh1AXn/9wGVzuZJlyuLpk1zLplBhO7Uj7bMGTlBnbacO/QrTIKdtYzXcfbSW XTuPPEnJsYLrz8ulUiHkRtjQXZ9DxEkQtaewkPA5Rw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepvdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:24 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 36a09873 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:10 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:22 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 06/13] ci: merge custom PATH directories Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're downloading various executables required by our tests. Each of these executables goes into its own directory, which is then appended to the PATH variable. Consequently, whenever we add a new dependency and thus a new directory, we would have to adapt to this change in several places. Refactor this to instead put all binaries into a single directory. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 15 +++++++-------- ci/lib.sh | 10 +++------- 2 files changed, 10 insertions(+), 15 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 2d6af876d6..bafe37f2d1 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -29,15 +29,14 @@ ubuntu-*) libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl \ $CC_PACKAGE $PYTHON_PACKAGE - mkdir --parents "$P4_PATH" - wget --quiet --directory-prefix="$P4_PATH" \ + mkdir --parents "$CUSTOM_PATH" + wget --quiet --directory-prefix="$CUSTOM_PATH" \ "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" - chmod u+x "$P4_PATH/p4d" "$P4_PATH/p4" + chmod u+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" - mkdir --parents "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ - -C "$GIT_LFS_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" ;; macos-*) @@ -48,10 +47,10 @@ macos-*) brew install $BREW_INSTALL_PACKAGES brew link --force gettext - mkdir -p "$P4_PATH" + mkdir -p "$CUSTOM_PATH" wget -q "$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz" && - tar -xf helix-core-server.tgz -C "$P4_PATH" p4 p4d && - sudo xattr -d com.apple.quarantine "$P4_PATH/p4" "$P4_PATH/p4d" 2>/dev/null || true + tar -xf helix-core-server.tgz -C "$CUSTOM_PATH" p4 p4d && + sudo xattr -d com.apple.quarantine "$CUSTOM_PATH/p4" "$CUSTOM_PATH/p4d" 2>/dev/null || true rm helix-core-server.tgz if test -n "$CC_PACKAGE" diff --git a/ci/lib.sh b/ci/lib.sh index d882250db5..4cce854bad 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -340,10 +340,6 @@ ubuntu-*) # image. # Keep that in mind when you encounter a broken OS X build! export LINUX_GIT_LFS_VERSION="1.5.2" - - P4_PATH="$HOME/custom/p4" - GIT_LFS_PATH="$HOME/custom/git-lfs" - export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" ;; macos-*) MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" @@ -351,12 +347,12 @@ macos-*) then MAKEFLAGS="$MAKEFLAGS APPLE_COMMON_CRYPTO_SHA1=Yes" fi - - P4_PATH="$HOME/custom/p4" - export PATH="$P4_PATH:$PATH" ;; esac +CUSTOM_PATH="$HOME/path" +export PATH="$CUSTOM_PATH:$PATH" + case "$jobname" in linux32) CC=gcc From patchwork Fri Apr 12 04:44:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626840 Received: from fhigh5-smtp.messagingengine.com (fhigh5-smtp.messagingengine.com [103.168.172.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9BC941C6A6 for ; Fri, 12 Apr 2024 04:44:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.156 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897073; cv=none; b=sqw/cIf1VIweojfHMxsnkmLQBio94rfJDgFjDUnEG3YPAgX+qGDyocm+wT605nIVhK7kOHVMuPM76dNO/aFjT8d0G+D3FmVIthoNGsKLcdfhB/uvqTB68Kj90oBXzP/OOpi+GA3dsb2mbb3fSJC8WqDOgevuyspP+vVEp1gf910= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897073; c=relaxed/simple; bh=YXb1wbEfkL6YD8EFiigXxpsrVBVhgx7JOWECpuz4/B8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=cgi4l76+GyRc84Ae2j3ExFiosWn1XwhGULWWMl+g7ncuotFn4I3RTogXM0Wb0YyDUl7avWE/08qMoD2mRZss7eZ8v4V3zwuE6LxO5yvQBqZOHqHQC0rsWtklsttx+Y9YbwmehCAufi64pSgeZtnzENim2TrxVssnkgpQonjV/5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=D1n5X46W; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=jZ4maJ1C; arc=none smtp.client-ip=103.168.172.156 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="D1n5X46W"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="jZ4maJ1C" Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfhigh.nyi.internal (Postfix) with ESMTP id AC32011401D7; Fri, 12 Apr 2024 00:44:30 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Fri, 12 Apr 2024 00:44:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897070; x=1712983470; bh=g8YDcIxG0H IRrsZLKSGpb9LIPFoJ/abZgLCbVxni9PE=; b=D1n5X46WNIINCpreN+i1+qI8gP 7O93pM3a5072seVnselyg2S+30kwI7WN71TcyOGJJSvp9KdQuuXK0DcWr/1l19en q7hiU0YIcVkwQYI9rhQzVTejHLRCStD1hPiCVF0sKvWFR3qtcRwFWea2NH1hCoq0 souwJdQ4lnilUbBNALZvPc0Lt2TaUK1JSDcvxUlsuGfPMHvy5LH6TAZwsCKErOq3 CW85dZA5O/mRuBAs658gE9LFVXqgJE0R6r1CmC3D9E15e6yGLK98ggR4gkDGNZyj ED03ktVJaTGHpprZw01EjLXcdK484UDmkH8e9qzNqSq3b/JAo6+aouFzGyRA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897070; x=1712983470; bh=g8YDcIxG0HIRrsZLKSGpb9LIPFoJ /abZgLCbVxni9PE=; b=jZ4maJ1Cmnp42yq0LnHgsre2jHZvgM+r973ZUwN0E5hT sGuKlaw7yovjTT0B8QJHq7kUixS2DZFc+Hfi1A+Mv1F6lzsaoU8AkDXhGcAQdNCJ 1ZKd/Rsu6vapRiHk3CoTgfPZTm8nCRvcJmokFRuc+ko/M1PJgmCkZ/yhAThEKQ4o ZvnHWu34X276CP229AI1P+7qPEBklA6SdQFZNkHUQYd6Urt18rF2JgZg3TF0UOMh Fb78G6/26kX4Dqv++NOY09VDDlK3kumL+TxTh2yXvj+rs3iotajD/1mycE1HX9AH W1vK1a/khFzNyt7LlLH1Zckz+AuoYKgFuABB148siw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddgheduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:29 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 27b59c85 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:15 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:27 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 07/13] ci: fix setup of custom path for GitLab CI Message-ID: <9fc462b578b74ecc34aaeda5a1c8bcf3e0ba66ff.1712896869.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Part of "install-dependencies.sh" is to install some binaries required for tests into a custom directory that gets added to the PATH. This directory is located at "$HOME/path" and thus depends on the current user that the script executes as. This creates problems for GitLab CI, which installs dependencies as the root user, but runs tests as a separate, unprivileged user. As their respective home directories are different, we will end up using two different custom path directories. Consequently, the unprivileged user will not be able to find the binaries that were set up as root user. Fix this issue by allowing CI to override the custom path, which allows GitLab to set up a constant value that isn't derived from "$HOME". Signed-off-by: Patrick Steinhardt --- .gitlab-ci.yml | 2 ++ ci/lib.sh | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c0fa2fe90b..3a0ef4d4d4 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -9,6 +9,8 @@ workflow: test:linux: image: $image + variables: + CUSTOM_PATH: "/custom" before_script: - ./ci/install-docker-dependencies.sh script: diff --git a/ci/lib.sh b/ci/lib.sh index 4cce854bad..473a2d0348 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -350,7 +350,7 @@ macos-*) ;; esac -CUSTOM_PATH="$HOME/path" +CUSTOM_PATH="${CUSTOM_PATH:-$HOME/path}" export PATH="$CUSTOM_PATH:$PATH" case "$jobname" in From patchwork Fri Apr 12 04:44:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626841 Received: from fhigh5-smtp.messagingengine.com (fhigh5-smtp.messagingengine.com [103.168.172.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D6421E491 for ; Fri, 12 Apr 2024 04:44:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.156 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897078; cv=none; b=SjIyc3oeD8LJWd7U2URYXLiWUoetriU7c5DptHF3cnTznLPRcalqEF2LEXE/gB6FgSCOuDH9p2KidMAl0mD0iQ92nS7Yk6UBvX+P5SHB3YUnOsCsRDXy2yjBHW1l4KIsFPRVhO6klpMPHwA5iGKmPGYBdRsKasPyJX3Ls80lrq8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897078; c=relaxed/simple; bh=ZQwfrGygsWfUhffsHxN9TbJ5epiuIsVNcxs2cw0ZAcg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=l8RAz1bDMVFk7iI2jaE+h67k93fcD1WXl68B8Pu/7m5fU0Q7iWqjW2rYjwYNejbxa67phYRd2jFD5J5wFuKjHGKiLLQR6Yy+bT9jUFwwTab+QDiDiS5z4hoerXFk3mnyTsmrzj9Kcw1r4QbBPkJSS+szS6lHM6AJt9msJOOrZl4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=kXbwdEpB; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=cetoBLoZ; arc=none smtp.client-ip=103.168.172.156 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="kXbwdEpB"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="cetoBLoZ" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 4B5AB11401D7; Fri, 12 Apr 2024 00:44:36 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Fri, 12 Apr 2024 00:44:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897076; x=1712983476; bh=z0UVGeyYsF 0BalnhQyzcU3MztgbMquzGlCI10w4i/0A=; b=kXbwdEpB1zU67twTtqsprhlT64 6CLvYxIhij9h73FBKj+B9EgaxqaKRQWRbGKZkDBMBtrp5FPwAAKcrDLU1hKMq+FQ 62B3RLS2fOAWcvQdxiMvl8pAZ3xz18FQuhAJfcBRyTplHcIKqXwws3gljATZecfl US3HY4CBSWTtJDBsY5E5hXXAncCw2pALz39Ag/rBl3Dm/Z7dhk3s33iPHph5EFEh wtm5rCI67U9fbploHyWUjDDiN8m7NOp6bAInvBGmj8dInsyp2ydAAnwaQi4a4Hjt ovyuJL0Y0sh+nyTPS8ECW7pWzqbL52LE07ypUnsnY9UbkHeZ98S+uvqD95fw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897076; x=1712983476; bh=z0UVGeyYsF0BalnhQyzcU3MztgbM quzGlCI10w4i/0A=; b=cetoBLoZOK/lFBKzhMRM6uQUcxA5Qqy0nAZDfy83Kf9W u/UKtqF+aDQLVEDXwWOr1Oss4Ayrj4fTPo36V+xnzdts8lCDdtjkyHiVRMV8UkmB 4VXrvQj7jtMJwrLy3CMMJMnr9x2S7T3f7RrKFKd6MIXS/hxaxBY9/x24ch2jxfDY L3otNL198SIEPqECA0F8jL3jNgOf4AhhJLz81r5RplkzSKVsBtlsYQzCnLpzhd06 COa8WTwfr1pwmOYAXvIbdX7OxesUwNlT2CFzJDI8vd9kSPkhV+uY9Yv+f8xMkBWH NCP2W1cKtEEVhf+vMqSDey37AJVuOmUYXxPZr8C3/g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddgheduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepveeuvdeujeehueeutdetkefffedtueetudefieeiieekudevudffffekheejfeeu necuffhomhgrihhnpehshhdqrdgtihdpihhnshhtrghllhdquggvphgvnhguvghntghivg hsrdhshhdpphgvrhhfohhrtggvrdgtohhmpdhgihhthhhusgdrtghomhenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:34 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id dfb5dd78 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:21 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:32 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 08/13] ci: merge scripts which install dependencies Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We have two different scripts which install dependencies, one for dockerized jobs and one for non-dockerized ones. Naturally, these scripts have quite some duplication. Furthermore, either of these scripts is missing some test dependencies that the respective other script has, thus reducing test coverage. Merge those two scripts such that there is a single source of truth for test dependencies, only. Signed-off-by: Patrick Steinhardt --- .github/workflows/main.yml | 2 +- .gitlab-ci.yml | 4 +-- ci/install-dependencies.sh | 32 ++++++++++++++++++--- ci/install-docker-dependencies.sh | 46 ------------------------------- 4 files changed, 31 insertions(+), 53 deletions(-) delete mode 100755 ci/install-docker-dependencies.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 71cd4e5486..5838986895 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -359,7 +359,7 @@ jobs: if: matrix.vector.jobname != 'linux32' - uses: actions/checkout@v1 # cannot be upgraded because Node.js Actions aren't supported in this container if: matrix.vector.jobname == 'linux32' - - run: ci/install-docker-dependencies.sh + - run: ci/install-dependencies.sh - run: ci/run-build-and-tests.sh - name: print test failures if: failure() && env.FAILED_TEST_ARTIFACTS != '' diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 3a0ef4d4d4..ba65f50aac 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -12,7 +12,7 @@ test:linux: variables: CUSTOM_PATH: "/custom" before_script: - - ./ci/install-docker-dependencies.sh + - ./ci/install-dependencies.sh script: - useradd builder --create-home - chown -R builder "${CI_PROJECT_DIR}" @@ -100,7 +100,7 @@ static-analysis: variables: jobname: StaticAnalysis before_script: - - ./ci/install-docker-dependencies.sh + - ./ci/install-dependencies.sh script: - ./ci/run-static-analysis.sh - ./ci/check-directional-formatting.bash diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index bafe37f2d1..e673797115 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -5,6 +5,8 @@ . ${0%/*}/lib.sh +begin_group "Install dependencies" + P4WHENCE=https://cdist2.perforce.com/perforce/r21.2 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION @@ -20,14 +22,27 @@ then fi case "$distro" in +alpine-*) + apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \ + pcre2-dev python3 musl-libintl perl-utils ncurses \ + apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \ + bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null + ;; +fedora-*) + dnf -yq update >/dev/null && + dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null + ;; ubuntu-*) + # Required so that apt doesn't wait for user input on certain packages. + export DEBIAN_FRONTEND=noninteractive + sudo apt-get -q update sudo apt-get -q -y install \ - language-pack-is libsvn-perl apache2 \ - make libssl-dev libcurl4-openssl-dev libexpat-dev \ + language-pack-is libsvn-perl apache2 cvs cvsps git gnupg subversion \ + make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo \ tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \ - libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl \ - $CC_PACKAGE $PYTHON_PACKAGE + libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ + ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE mkdir --parents "$CUSTOM_PATH" wget --quiet --directory-prefix="$CUSTOM_PATH" \ @@ -39,6 +54,13 @@ ubuntu-*) -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" ;; +ubuntu32-*) + sudo linux32 --32bit i386 sh -c ' + apt update >/dev/null && + apt install -y build-essential libcurl4-openssl-dev \ + libssl-dev libexpat-dev gettext python >/dev/null + ' + ;; macos-*) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 # Uncomment this if you want to run perf tests: @@ -98,3 +120,5 @@ then else echo >&2 "WARNING: git-lfs wasn't installed, see above for clues why" fi + +end_group "Install dependencies" diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh deleted file mode 100755 index eb2c9e1eca..0000000000 --- a/ci/install-docker-dependencies.sh +++ /dev/null @@ -1,46 +0,0 @@ -#!/bin/sh -# -# Install dependencies required to build and test Git inside container -# - -. ${0%/*}/lib.sh - -begin_group "Install dependencies" - -case "$jobname" in -linux32) - linux32 --32bit i386 sh -c ' - apt update >/dev/null && - apt install -y build-essential libcurl4-openssl-dev \ - libssl-dev libexpat-dev gettext python >/dev/null - ' - ;; -linux-musl) - apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \ - pcre2-dev python3 musl-libintl perl-utils ncurses \ - apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \ - bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null - ;; -linux-*|StaticAnalysis) - # Required so that apt doesn't wait for user input on certain packages. - export DEBIAN_FRONTEND=noninteractive - - apt update -q && - apt install -q -y sudo git make language-pack-is libsvn-perl apache2 libssl-dev \ - libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev \ - perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl \ - libdbd-sqlite3-perl libio-socket-ssl-perl libnet-smtp-ssl-perl ${CC_PACKAGE:-${CC:-gcc}} \ - apache2 cvs cvsps gnupg libcgi-pm-perl subversion - - if test "$jobname" = StaticAnalysis - then - apt install -q -y coccinelle - fi - ;; -pedantic) - dnf -yq update >/dev/null && - dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null - ;; -esac - -end_group "Install dependencies" From patchwork Fri Apr 12 04:44:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626842 Received: from fhigh5-smtp.messagingengine.com (fhigh5-smtp.messagingengine.com [103.168.172.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3CB0224DB for ; Fri, 12 Apr 2024 04:44:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.156 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897083; cv=none; b=cY/vTTzr2sDG18KOHjqF+pGfiB6hyS1GkB1levPGLqL5OD0Qme0gYG20F1mhDOTjV+oLxEFyOLOfESQ7AYgE2LB2DiA/ZQ1SYvh5+0F4CMiutRJZxdtBrIjX9cvXiSexIRs04SmJB22zXDcQKoXpigQFBabPi3VKSti2lA0pxsU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897083; c=relaxed/simple; bh=A0QdMs3tr4XG9cji3OBd1FBCqNV2pFkHPGMhAMNnl4Q=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Ld3sQpbFfg9MGN8/AwdDK84GPiB+rRLQ2fzNyXJL0wZGOhHYIOjETOTpxrHyb7TZT1E3h8bxd9GUPHL3mG20+HGSW86xg/kEvz6kip1Ol6Z2FUFUVRXEjASjlW2CI/m+Td+y+oF3MDT1tXphTfx4Tue+891fNBKKKe6P7E3BsDs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=RpPY2sWf; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=l3273I6w; arc=none smtp.client-ip=103.168.172.156 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="RpPY2sWf"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="l3273I6w" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 036FC11401D5; Fri, 12 Apr 2024 00:44:41 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Fri, 12 Apr 2024 00:44:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897081; x=1712983481; bh=1GUUf6HYMI OavK6CIPV38Rh1+jA3RUbsRXSQ8p9tMp8=; b=RpPY2sWfoxoDbYUNRDpqjEcMxU w80gkeU3z1Z7eX5ul4mOpd5jq/8jfPgYvxO86W5UgMOU5rXXeihDFyNCd0xnCLXh aAFvjRRS13ZLqTLSU18gHhU5/hRDtFJryPf+6gfF0M0DbHiTdSevCZHk1R8doJeK +BINTQIJi9MG+rfHHeSt8flTXxKDRclzDpX8Y53XKqXgjyq5D2gD8/Y8EP/I0dQx ZEBD1aZ8RZzQO/Hyr4TYCJ/3OQhA5xG24A1zwrxM8TzRy4/JqWWzyIAipRObi9NP T1iVbdfYC1qkceU3r3FMfizIi+SSxYY7OldEj7H3LaXLtdia2afUUZIGueuA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897081; x=1712983481; bh=1GUUf6HYMIOavK6CIPV38Rh1+jA3 RUbsRXSQ8p9tMp8=; b=l3273I6wH5yz0jAVRQOXDaYiQfO1gLgKyZsQBwIyG1yM w84me7lVNN7zuiJlfeqvpyHLDHV8bEzmm42v8VUOirqqozObcuY3qYzyVxJQs5T9 NoNwp8AUbZ1j8Gx75BDCEAvLUEPFs1WYSAKVqXuB+z857mVEf3H4P7dT0RpW7Ppy kilB3fxpMoiDDZMFSEDoHEXOb+gK8LB10zv87cKfEJDqGzpPVofIMD2kFE8el2md RJm1xvepu2fcqiMc8eQeluPosySKqIlCgLca35rO+4PaCjQCP+gABsIZVHdbsXCh vfmwxRFv4LsYXlKbmIhXDouI1gBYJRug3i4WBouwdA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:39 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id da7b5254 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:26 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:37 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 09/13] ci: make Perforce binaries executable for all users Message-ID: <720d5a46822231ce8cce26a0704d5a3e6ec95b58.1712896869.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The Perforce binaries are only made executable for the current user. On GitLab CI though we execute tests as a different user than "root", and thus these binaries may not be executable by that test user at all. This has gone unnoticed so far because those binaries are optional -- in case they don't exist we simply skip over tests requiring them. Fix the setup so that we set the executable bits for all users. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index e673797115..0b9bb686d8 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -47,7 +47,7 @@ ubuntu-*) mkdir --parents "$CUSTOM_PATH" wget --quiet --directory-prefix="$CUSTOM_PATH" \ "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" - chmod u+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" + chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ From patchwork Fri Apr 12 04:44:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626843 Received: from fout1-smtp.messagingengine.com (fout1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B8292C683 for ; Fri, 12 Apr 2024 04:44:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897089; cv=none; b=VdfcRnxX2bH3+XE6JcetTB1tRenFJa9ztH+xFsuPFxkv9ci6sWvS7VXMd3QS1dv2V9tJhIfMx1MBYEGhZVkXWJWhAmYgvG70CQyjPIeUk4Jp+YUDj11D+SV1jSPt8NTejXsiPetl03TKuBmzJdGNDBHz4NjsrD+/cO7uIDr9rX8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897089; c=relaxed/simple; bh=DljDrVpUABrOe0yHe5XQ0SL53isAlI0KmK6mdOlG4bc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=bQfBbCUi3fcgLXjulEdhJy1H3BYkYYm+46v4Gw7SN7JQwG9z96eAS/iPCGDu6vcARpuAS24c+GT8Qlg89ofl5Xeq94qybHqZNLSCzpzw+L/QajtYXytoTMmraKX6m0L0LqogQH2OsGG+JLNG8N+32IHhU0n4gwjgOga98+pdDa8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=lo1kiRt6; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=vX08ddSy; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="lo1kiRt6"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="vX08ddSy" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfout.nyi.internal (Postfix) with ESMTP id 9728713802F7; Fri, 12 Apr 2024 00:44:47 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 12 Apr 2024 00:44:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897087; x=1712983487; bh=KrPyBTVgUa a4ZoqE4P4uAbirfV+Dfim8WzSQGlhvITo=; b=lo1kiRt6848+9vCB8Zb4loy9u7 xLRKgL0+blSK7fc2DNl/qocjzc/wJjz6hi1FHMMlLLF2PnC3QcD4T03jK8QCBshE m2C2D/KjSFsMV+1KHLM74Ywc+KcV9ZCPq4k7NyHB7saE5o4Jm7Kouq9786oCzPoG sgj0pJbHJ+8uWyFLRhjVdd19HOM2aliTl/Mc0L9cTlO5C8jOiqyxWWm/fcLLLdZK I+bEf8V/dJuTtX8tbQ4Dn4v5Vpg6cA1C1N98LLwS5VrITq4UmW4FL2aWeGSIOQ9G dT/mPbGFuY/TznIiuo/f/3D2/T8fAljTzfvNSzsRXa7XWYVDhz5NZvuyFGHA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897087; x=1712983487; bh=KrPyBTVgUaa4ZoqE4P4uAbirfV+D fim8WzSQGlhvITo=; b=vX08ddSylxPDrd9G20AbamQWtNk6ecELFNwG+zsWvxAZ IZRo53CCvo6wLmuKajAqy2GCamNC7PMBUD2m7ZXw+VQToflbQQ0Ab2M8qUUsIW1a ee3DSDJaLUJe03RQ//NaIHO5Zc0siUdZHf+470wlTCE4hTGSSrXhQK71Xfw5w7Qs oNkbu9kpiDLv4+3hw9C4gpVtKCnytsYZFO3Vo/m40Qg8QGtdFjgv+navX/4b5zUe g6V8RsmgfWudeDfVINbNcrT3pPR0wuFeHmmE2Fu56HYU6GmkuKAUtklhKqGd8gQr OUpXAkswtlyu2A4V5c2e98rE53GZfnqjIBZr+iXB7w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhephefgvdeuhfduudffuedufffgffehgedugeeivddvjeeugfejudeikedvgffgteei necuffhomhgrihhnpehpvghrfhhorhgtvgdrtghomhdpghhithhhuhgsrdgtohhmpdgvtg hlihhpshgvrdhorhhgpddvtddvfeduuddvledugeehtddqrhdrshhhnecuvehluhhsthgv rhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:46 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 545b68bd (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:32 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:43 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 10/13] ci: install JGit dependency Message-ID: <77f6d6ecaac037ad03b8e93ea392e0270bad9e45.1712896869.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We have some tests in t5310 that use JGit to verify that bitmaps can be read both by Git and by JGit. We do not execute these tests in our CI jobs though because we don't make JGit available there. Consequently, the tests basically bitrot because almost nobody is ever going to have JGit in their path. Install JGit to plug this test gap. Signed-off-by: Patrick Steinhardt --- ci/install-dependencies.sh | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 0b9bb686d8..c196e56762 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -9,6 +9,7 @@ begin_group "Install dependencies" P4WHENCE=https://cdist2.perforce.com/perforce/r21.2 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION +JGITWHENCE=https://repo.eclipse.org/content/groups/releases//org/eclipse/jgit/org.eclipse.jgit.pgm/6.8.0.202311291450-r/org.eclipse.jgit.pgm-6.8.0.202311291450-r.sh # Make sudo a no-op and execute the command directly when running as root. # While using sudo would be fine on most platforms when we are root already, @@ -39,7 +40,7 @@ ubuntu-*) sudo apt-get -q update sudo apt-get -q -y install \ language-pack-is libsvn-perl apache2 cvs cvsps git gnupg subversion \ - make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo \ + make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \ tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \ libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE @@ -53,6 +54,9 @@ ubuntu-*) tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + + wget --quiet "$JGITWHENCE" --output-document="$CUSTOM_PATH/jgit" + chmod a+x "$CUSTOM_PATH/jgit" ;; ubuntu32-*) sudo linux32 --32bit i386 sh -c ' @@ -113,6 +117,7 @@ then else echo >&2 "WARNING: perforce wasn't installed, see above for clues why" fi + if type git-lfs >/dev/null 2>&1 then echo "$(tput setaf 6)Git-LFS Version$(tput sgr0)" @@ -121,4 +126,12 @@ else echo >&2 "WARNING: git-lfs wasn't installed, see above for clues why" fi +if type jgit >/dev/null 2>&1 +then + echo "$(tput setaf 6)JGit Version$(tput sgr0)" + jgit version +else + echo >&2 "WARNING: JGit wasn't installed, see above for clues why" +fi + end_group "Install dependencies" From patchwork Fri Apr 12 04:44:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626844 Received: from fout1-smtp.messagingengine.com (fout1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33DD88BF6 for ; Fri, 12 Apr 2024 04:44:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897094; cv=none; b=D9qtF+Qa98Br7DBcvCQjMc5yMd03VCXfSjWR/x5kMo+mwgPAke+r3YSMQ/6EyJgDhMMgzl3rsLCOzwLLVxHiMX9ScIIAXu0BDdVRz6LQAnD4S+qwlE/InGJxaWM4pjen/REkxGqsSp6c0C0RkFeOQa/5lDflbg6QHAIVRi7dKA4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897094; c=relaxed/simple; bh=3wNQEReFgUIepwnKDlnWeKLFTte43BohAN2ahrZeeuM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=XpBynikkF29x/a/6YHCHd23QmNeHM3XuvZ5j07Zli9naYoY2UNgCFfg2ACMCvwuDL66UGXgEOgIBzezknXt8Dit4XkiWO20mJvX8atRs6LDKHy7N2/t3MzgsV7KVkxWlQ6DpbXm9/ZnCYulM6d9iOK68XW/bMyZo7UClvABX3KQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=ilKpcVCz; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=humV5ACd; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="ilKpcVCz"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="humV5ACd" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.nyi.internal (Postfix) with ESMTP id 3AFD71380306; Fri, 12 Apr 2024 00:44:52 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Fri, 12 Apr 2024 00:44:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897092; x=1712983492; bh=iLKJTMTZJ8 ImPIoF+619W3Fj3dH11HgVvneJcS/KolI=; b=ilKpcVCzSVJ0J1MQ3UqLRXqVvs LxtYfKtgHWnIPLqwk2Coz19YpYikrB5osX0HxZX2VuQ5gkg/JySOb0HPDQKzEvfy agwmqbQj3KOcQlegFezOYZCHXqwn42gL/RIuA0qxFb/MauN2/SFKX0U2X3UhB8n9 /T2ot/hi+b0+Ayy2wgpU5YJsFS6Xb2MMvz3Xi+nSGKLD8k7w/s9ag2n6f9V8/3/q Io+cMXy3VZ4Z+g+R897wwaOHtiqR4PHeBs/PMYf2dopzb9/vVzCgBpNslbIsDzT0 lsJbDqpRA+mU1F22MBgj0k9tFPNt93NVtoFYGb2lBH+8ZqNSsq/bcM/e9dHQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897092; x=1712983492; bh=iLKJTMTZJ8ImPIoF+619W3Fj3dH1 1HgVvneJcS/KolI=; b=humV5ACd9/F3x0jFbe8JBpCre6WFumDSgma8Ij2hX8kb B0WbTQyJaHvD5sgLIUegXdBneuqi53Q7st/Mvn9RfgyO1G+vFrGZKxuXTcAeoLSS GfDLzm9arUbIpkeudpha++tDoTokqn4Woj5OEUUoxdNMhVl4B30/YgNfZ67ToAZu mrWP9YS0SrM11/hg6Ha+kqDk+aXDExZ/TvWoZPMo5ajFEJZCxaGaBbfjQI8bVXlQ uhPTTZMGk/1Fk7nXCZP1B2uuuBqnpqRTXeN+h20uST/0urd1KnsuFktOv/k5XGjP KDV++v07tU7vnbtc88HGLsuFX5xjUQii4kBOpMb3uw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:50 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 6b23b115 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:37 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:48 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 11/13] t06xx: always execute backend-specific tests Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The tests in t06xx exercise specific ref formats. Next to probing some basic functionality, these tests also exercise other low-level details specific to the format. Those tests are only executed though in case `GIT_TEST_DEFAULT_REF_FORMAT` is set to the ref format of the respective backend-under-test. Ideally, we would run the full test matrix for ref formats such that our complete test suite is executed with every supported format on every supported platform. This is quite an expensive undertaking though, and thus we only execute e.g. the "reftable" tests on macOS and Linux. As a result, we basically have no test coverage for the "reftable" format at all on other platforms like Windows. Adapt these tests so that they override `GIT_TEST_DEFAULT_REF_FORMAT`, which means that they'll always execute. This increases test coverage on platforms that don't run the full test matrix, which at least gives us some basic test coverage on those platforms for the "reftable" format. This of course comes at the cost of running those tests multiple times on platforms where we do run the full test matrix. But arguably, this is a good thing because it will also cause us to e.g. run those tests with the address sanitizer and other non-standard parameters. Signed-off-by: Patrick Steinhardt --- t/t0600-reffiles-backend.sh | 8 ++------ t/t0601-reffiles-pack-refs.sh | 9 +++------ t/t0610-reftable-basics.sh | 9 +++------ 3 files changed, 8 insertions(+), 18 deletions(-) diff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh index 64214340e7..a390cffc80 100755 --- a/t/t0600-reffiles-backend.sh +++ b/t/t0600-reffiles-backend.sh @@ -4,16 +4,12 @@ test_description='Test reffiles backend' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=files +export GIT_TEST_DEFAULT_REF_FORMAT TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh -if ! test_have_prereq REFFILES -then - skip_all='skipping reffiles specific tests' - test_done -fi - test_expect_success 'setup' ' git commit --allow-empty -m Initial && C=$(git rev-parse HEAD) && diff --git a/t/t0601-reffiles-pack-refs.sh b/t/t0601-reffiles-pack-refs.sh index 7d4ab0b91a..60a544b8ee 100755 --- a/t/t0601-reffiles-pack-refs.sh +++ b/t/t0601-reffiles-pack-refs.sh @@ -9,18 +9,15 @@ test_description='git pack-refs should not change the branch semantic This test runs git pack-refs and git show-ref and checks that the branch semantic is still the same. ' + GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=files +export GIT_TEST_DEFAULT_REF_FORMAT TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh -if ! test_have_prereq REFFILES -then - skip_all='skipping reffiles specific tests' - test_done -fi - test_expect_success 'enable reflogs' ' git config core.logallrefupdates true ' diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh index 931d888bbb..fd0ddb96ae 100755 --- a/t/t0610-reftable-basics.sh +++ b/t/t0610-reftable-basics.sh @@ -4,17 +4,14 @@ # test_description='reftable basics' + GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=reftable +export GIT_TEST_DEFAULT_REF_FORMAT . ./test-lib.sh -if ! test_have_prereq REFTABLE -then - skip_all='skipping reftable tests; set GIT_TEST_DEFAULT_REF_FORMAT=reftable' - test_done -fi - INVALID_OID=$(test_oid 001) test_expect_success 'init: creates basic reftable structures' ' From patchwork Fri Apr 12 04:44:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626845 Received: from fout1-smtp.messagingengine.com (fout1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C40279DC for ; Fri, 12 Apr 2024 04:44:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897099; cv=none; b=epRYanE+CaGkrwb548beeHQAEud/nZJkY/4KtFL1sJdjWiGTj2dyXNZ3gqOaT1EnFoL9ebN1CTco8+mTT9xjIRi0FiGRoQvUiSXDpU7urZi3cJwK7N9isbzV/OnoTyW3Kfp4O9troG3py0Iz7dBCqZao28ndkoGndClYksr6gMA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897099; c=relaxed/simple; bh=NabVWjCW093tNGt84x85NEyBYtHlDAkiJEk56F5IRzo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=O4e5yejNRrSBrw+l8R/pY8bVWQn9SI/xf+EYSssuMl5fteApLbHMwVdNK6Ptv5cB0Dx1jjemuaNopd55YBUMSkEyeHxp2guImN0omLTSY7YjQpVuXsk6KHyMGBH9ut318ufXSxrXI/hP1DqoMyGZAKt0cdAWNpOMw87IO+ZP+I0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=qfp/iGb0; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=Z0wvOahU; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="qfp/iGb0"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Z0wvOahU" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfout.nyi.internal (Postfix) with ESMTP id CBB0413802C9; Fri, 12 Apr 2024 00:44:56 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Fri, 12 Apr 2024 00:44:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897096; x=1712983496; bh=WgPu7k6hps 5GanXDBZLr+JJaZ6JBzHbeq9bPCLRt/Io=; b=qfp/iGb0B8zzuhJXqecWU08mxe oDW9klzyF3AKQyRFyeHZLkuIQLApWGEnHd+T/5+OwUPsQjrdjMgMq60oYOrR637y WtTDDWJOU88ZRRThvx1ZROKdHO9bFuWtmN7l/ARmHgu96nQo1dLcPJ6qtg5YV5KZ wH8AEyE5+okWJI8m7YJSLvkTcT5w9pDx9HAXxqsaCFskY03fhVkrGMBE7+SRac7Q reg6TC5/Ljh/PLMqKtl8w+6fybDbENWZ0tJ59TSLAJYMeScqk2ZQgUaqlMca8Txv MGaZryvZUzrMdQ79xgMZemWOrFTIyck05SwniDOqVWg1NeHPVqkAKRaIGKFw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897096; x=1712983496; bh=WgPu7k6hps5GanXDBZLr+JJaZ6JB zHbeq9bPCLRt/Io=; b=Z0wvOahUE7dH22b0pBxNQWMyZAJxPii0vh1KhadjphIS UcZgk+CIqVq+yg1swcpn4ap+Qt5wst8jiP98J1WRN3RqTjuDoto7a+lkUwAFMLm5 NqLJE8GnLU505u7rXq8xhxm2vHjmYpvJy4cOgqXQxQ/V9mPPyuMqeQGDN3IvG7sI ZpPHhHe26BffYROZInMFs8emqEWhRpyWqldwR5ARJeOyFjgwNdBAAx8HEQ9tkZ7m 5X0QpuBROOXalKZYsXf7UUBbEBP8K2H9hTPdebQmxUYbSOgfQttek3DEDmh4Ot0C XY8ONlhIkMyFj8q5bBp3hSrlxpKeM++0nL6dYeSCmA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:44:55 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id f5ecadd9 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:42 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:53 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 12/13] t0610: fix non-portable variable assignment Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Older versions of the Dash shell fail to parse `local var=val` assignments in some cases when `val` is unquoted. Such failures can be observed e.g. with Ubuntu 20.04 and older, which has a Dash version that still has this bug. Such an assignment has been introduced in t0610. The issue wasn't detected for a while because this test used to only run when the GIT_TEST_DEFAULT_REF_FORMAT environment variable was set to "reftable". We have dropped that requirement now though, meaning that it runs unconditionally, including on jobs which use such older versions of Ubuntu. We have worked around such issues in the past, e.g. in ebee5580ca (parallel-checkout: avoid dash local bug in tests, 2021-06-06), by quoting the `val` side. Apply the same fix to t0610. Signed-off-by: Patrick Steinhardt --- t/t0610-reftable-basics.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh index fd0ddb96ae..b6e67724ce 100755 --- a/t/t0610-reftable-basics.sh +++ b/t/t0610-reftable-basics.sh @@ -78,9 +78,9 @@ test_expect_success 'init: reinitializing reftable with files backend fails' ' ' test_expect_perms () { - local perms="$1" - local file="$2" - local actual=$(ls -l "$file") && + local perms="$1" && + local file="$2" && + local actual="$(ls -l "$file")" && case "$actual" in $perms*) From patchwork Fri Apr 12 04:44:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13626846 Received: from fhigh5-smtp.messagingengine.com (fhigh5-smtp.messagingengine.com [103.168.172.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FA9718EA2 for ; Fri, 12 Apr 2024 04:45:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.156 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897104; cv=none; b=ho0ztqZrXYztMq8JDglo/BJQCrS7zSF3I0bfpSOSHpVk6FcX9JduoivCTp3CI2h3+OBxfOM9ZmuLy7QMmt2G2/rq2B+4RFaA0ES3gSph/4rOVWyfBQcCcF0L79RRT+bzHRuVk3y5R6uMQtgreaTYwC/KIt+VEFtI17y7JkEmPTg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712897104; c=relaxed/simple; bh=lSqZf9aAteGN24eAYu57TtyLvkF99VDzp1D9yg6dPAo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=h1qTQozfJHueN0haZ2fdYAEKo01gcPcJItAAiDQMekCHomMqp4M8IP6Zy67f3KPdKiuZgr/udBL06dhT5Q8m7nMUxp5BDIyVOEFd1flxPIjSb1N37czFsvbLeQ5whym0bmALBbMSKoiGEHURbgOOMGIoyiPomQaxmLfhEtztkeU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=pXu8rmcE; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=HQxZD0o4; arc=none smtp.client-ip=103.168.172.156 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="pXu8rmcE"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="HQxZD0o4" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 7EDFC11401DA; Fri, 12 Apr 2024 00:45:02 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Fri, 12 Apr 2024 00:45:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712897102; x=1712983502; bh=H6+tF51ceJ weVHaz1C6GUTRMhy2yYczLBWou38X9bcE=; b=pXu8rmcELYTTTOjXXvwKJfLOmF dcS2zaaogTsoSqq8wEnB5k3iN0FJi3qKtwB8JJ3LDp9aZSMwzV4TaDXP8VY+o9Sr mm1537x/s+41kHLVPbJwVVjVxVwO9tGYt+Kxt1d7qjehsKRljJdj7KReBQxtIAn+ 2jPP0fGJE+a5SPV1/JtXSGrrkMBhAEEu+T9X1Z4Fs9XoWRXV5HUl3OWuud6EmDEp aNuFe2efQVgZywX/M4FZbusJy558eAPFJ+JAmLAhL7MlyMCHOb1uD8IDoP8CDvRK jggHwoCVQ+O/Vk5XxJW56btLemC+ZmBnelI4d3VCnvMwVQNOR8vrd0raTG4A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712897102; x=1712983502; bh=H6+tF51ceJweVHaz1C6GUTRMhy2y YczLBWou38X9bcE=; b=HQxZD0o4UmvAMk/hXjYj7+3NVEp1YnSq9XiSXPK3LsWk SULKuoFyPIiD83dRbhxMvnX5n6K1GjK8tyc2VqGuxF2tMzZFIig8cS4jLx7dXCO1 I7ZHjKJpDZ2ZrNVv5gf62pk0Y0xq0SkQjdpWaE5tI7wy85gOMYUyPEPq6RypzqqI zDA2ahAquMhGmAk6yTgrwbk6uf6FXEEr+rrwYIbLFOoq9GIVikttDzVuf63Uqnf3 HLxpespa/2P0SD5H29P6ti6XPa02BIcdWyYoVorqHhFOuaWKNBlmjIimawiUfy0R j55RRNB0NhxiAx2iP8U4Z2DLSiLyvA/oqXiSmlmheA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeitddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Apr 2024 00:45:00 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id d5230237 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Apr 2024 04:44:47 +0000 (UTC) Date: Fri, 12 Apr 2024 06:44:58 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v4 13/13] t0612: add tests to exercise Git/JGit reftable compatibility Message-ID: <218c694d2e1230b7b4f231e83feafc945820a26c.1712896869.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: While the reftable format is a recent introduction in Git, JGit already knows to read and write reftables since 2017. Given the complexity of the format there is a very real risk of incompatibilities between those two implementations, which is something that we really want to avoid. Add some basic tests that verify that reftables written by Git and JGit can be read by the respective other implementation. For now this test suite is rather small, only covering basic functionality. But it serves as a good starting point and can be extended over time. Signed-off-by: Patrick Steinhardt --- t/t0612-reftable-jgit-compatibility.sh | 132 +++++++++++++++++++++++++ 1 file changed, 132 insertions(+) create mode 100755 t/t0612-reftable-jgit-compatibility.sh diff --git a/t/t0612-reftable-jgit-compatibility.sh b/t/t0612-reftable-jgit-compatibility.sh new file mode 100755 index 0000000000..d0d7e80b49 --- /dev/null +++ b/t/t0612-reftable-jgit-compatibility.sh @@ -0,0 +1,132 @@ +#!/bin/sh + +test_description='reftables are compatible with JGit' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=reftable +export GIT_TEST_DEFAULT_REF_FORMAT + +# JGit does not support the 'link' DIRC extension. +GIT_TEST_SPLIT_INDEX=0 +export GIT_TEST_SPLIT_INDEX + +. ./test-lib.sh + +if ! test_have_prereq JGIT +then + skip_all='skipping reftable JGit tests; JGit is not present in PATH' + test_done +fi + +if ! test_have_prereq SHA1 +then + skip_all='skipping reftable JGit tests; JGit does not support SHA256 reftables' + test_done +fi + +test_commit_jgit () { + touch "$1" && + jgit add "$1" && + jgit commit -m "$1" +} + +test_same_refs () { + git show-ref --head >cgit.actual && + jgit show-ref >jgit-tabs.actual && + tr "\t" " " jgit.actual && + test_cmp cgit.actual jgit.actual +} + +test_same_ref () { + git rev-parse "$1" >cgit.actual && + jgit rev-parse "$1" >jgit.actual && + test_cmp cgit.actual jgit.actual +} + +test_same_reflog () { + git reflog "$*" >cgit.actual && + jgit reflog "$*" >jgit-newline.actual && + sed '/^$/d' jgit.actual && + test_cmp cgit.actual jgit.actual +} + +test_expect_success 'CGit repository can be read by JGit' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + test_commit A && + test_same_refs && + test_same_ref HEAD && + test_same_reflog HEAD + ) +' + +test_expect_success 'JGit repository can be read by CGit' ' + test_when_finished "rm -rf repo" && + jgit init repo && + ( + cd repo && + + touch file && + jgit add file && + jgit commit -m "initial commit" && + + # Note that we must convert the ref storage after we have + # written the default branch. Otherwise JGit will end up with + # no HEAD at all. + jgit convert-ref-storage --format=reftable && + + test_same_refs && + test_same_ref HEAD && + # Interestingly, JGit cannot read its own reflog here. CGit can + # though. + printf "%s HEAD@{0}: commit (initial): initial commit" "$(git rev-parse --short HEAD)" >expect && + git reflog HEAD >actual && + test_cmp expect actual + ) +' + +test_expect_success 'mixed writes from JGit and CGit' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + + test_commit A && + test_commit_jgit B && + test_commit C && + test_commit_jgit D && + + test_same_refs && + test_same_ref HEAD && + test_same_reflog HEAD + ) +' + +test_expect_success 'JGit can read multi-level index' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + + test_commit A && + awk " + BEGIN { + print \"start\"; + for (i = 0; i < 10000; i++) + printf \"create refs/heads/branch-%d HEAD\n\", i; + print \"commit\"; + } + " >input && + git update-ref --stdin