From patchwork Thu Apr 11 09:09:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13625631 Received: from fhigh1-smtp.messagingengine.com (fhigh1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6166F6FB5 for ; Thu, 11 Apr 2024 09:09:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826601; cv=none; b=MX+VL/aL3VmpNxlwOIP+xZBCaP3VIfUxxrpbD5q4xITsL/A/ge/r+xxf4gl4q8j8L59XISo4qJ5DL8ISrbcdSIPYr2UL0K18n2ZBa6vHiocl/auktiJ0gBpY7y6JhfN0hpVfNwrW/0VoAzDkLSOt6pbCCmT6cqivXqaOE0oIfp0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712826601; c=relaxed/simple; bh=A3I6MDbG9klYW/MSf9vOTe58h1KTLnyaQTL3G8aE+oo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=WiBxGDOgTuldmVsrg3UGleIKLRAgfYg4A+yPILX3Re9yAzTGjqXM+/yba7HFxAuhPkM90alGaftwQag3lIPxyeEW/u0lqjJig1dy9I6bCbwFHqPNhemC5lpP46jzm+indG3dHekPbg0PuuM1pDRgGctoQRFvPAHbSfLlQKuVHxk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=d+WBcaut; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=T25nGJJ1; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="d+WBcaut"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="T25nGJJ1" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 5E8AA1140174; Thu, 11 Apr 2024 05:09:58 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 11 Apr 2024 05:09:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1712826598; x=1712912998; bh=EE17XkUC2l FZveNeY1CAPoovF8CYatLDj05oD5mPXD0=; b=d+WBcautiirhoavhlfUoPXw6a9 4lYHeSwuSrtSeimFBKqhGKXQn5PZx1vTIOcE3R5Dd28Ueb3PvpPx8GTOT7Jz24rH Chber416foGxWhuzXlNGO5MBqMWXsdP1swxX755zK1RKEZytJ+8HUNDllcRr9Ybi W9IhEzR+uBxppoUyRfKFUtc1aiQ3D79Tq2TEHbNH/x1RFoAh/7Hqf4cIrfYg+nV9 ERD4qdQg2UVqYdCgucOmPmZjY9dm3c9DCITd56mQEKCGxMvcbNJcSA0s9daS2W4m umVf2dQO5Ogy827XdJLdDj/YQ8J3/DZF+HHf+uQGL5ZEXFrg8PRSn/nkrn6w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1712826598; x=1712912998; bh=EE17XkUC2lFZveNeY1CAPoovF8CY atLDj05oD5mPXD0=; b=T25nGJJ1ozmTmvTM2p3xH6IVIwfDQpy/OtWS8wymc7l6 DtK2B4ERdIPFpkdwIsrM+SFHQdVmzLr52ZeiJxZFFDRpdHdQHvzUhWb5D2WbA8zy W+IGkq8uHBIrVWkYHCYqaar75x+JpeWrxPSm6k72G9pUzvpR1EPRGX33K/1rV6TS 0rY97rYbgOWF+r6vD+ICDwRB1jxH0cBHZJK0VSpVEMnje8jH+snjhLaoDNKl6UTg jbEjpctMpRV6vwco1O/XFvdJax297BZDGSCIsnzc5rMYH7ZhIQhXjI1XDU4GA37f c/jPP3a6eLQ/S0vPGqFF+X/RmiqxkhRe2Ua0AyVhLA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepvdevledtffevleefffeltdffuedvtedtkeejgeehudekveehleetjeduudegjeei necuffhomhgrihhnpehgihhtlhgrsgdrtghomhdpghhithhhuhgsrdgtohhmpdhshhdqrd gtihdpihhnshhtrghllhdquggvphgvnhguvghntghivghsrdhshhenucevlhhushhtvghr ufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Apr 2024 05:09:56 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 58685c52 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 11 Apr 2024 09:09:43 +0000 (UTC) Date: Thu, 11 Apr 2024 11:09:52 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Josh Steadmon , Luca Milanesio , Eric Sunshine , Toon Claes , Justin Tobler Subject: [PATCH v3 00/13] t: exercise Git/JGit reftable compatibility Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Hi, this is the third version of my patch series that introduces compatibility checks for the CGit/JGit refatble implementations. Changes compared to v2: - Adjusted commit subject for the skipping-sudo patch. - Broken an overly long line in "install-dependencies.sh". - Fixed tests that depend on custom binaries not running on GitLab CI due to the unprivileged-user setup. - Fixed some typos. CI jobs for this version: - https://gitlab.com/gitlab-org/git/-/pipelines/1248875682 - https://github.com/git/git/actions/runs/8643984413 Thanks for all the feedback! Patrick Patrick Steinhardt (13): ci: rename "runs_on_pool" to "distro" ci: expose distro name in dockerized GitHub jobs ci: skip sudo when we are already root ci: drop duplicate package installation for "linux-gcc-default" ci: convert "install-dependencies.sh" to use "/bin/sh" ci: merge custom PATH directories ci: fix setup of custom path for GitLab CI ci: merge scripts which install dependencies ci: make Perforce binaries executable for all users ci: install JGit dependency t06xx: always execute backend-specific tests t0610: fix non-portable variable assignment t0612: add tests to exercise Git/JGit reftable compatibility .github/workflows/main.yml | 8 +- .gitlab-ci.yml | 6 +- ci/install-dependencies.sh | 101 +++++++++++++------ ci/install-docker-dependencies.sh | 46 --------- ci/lib.sh | 14 +-- t/t0600-reffiles-backend.sh | 8 +- t/t0601-reffiles-pack-refs.sh | 9 +- t/t0610-reftable-basics.sh | 15 ++- t/t0612-reftable-jgit-compatibility.sh | 132 +++++++++++++++++++++++++ 9 files changed, 229 insertions(+), 110 deletions(-) delete mode 100755 ci/install-docker-dependencies.sh create mode 100755 t/t0612-reftable-jgit-compatibility.sh Range-diff against v2: 1: 89723b6812 = 1: 46502bbe22 ci: rename "runs_on_pool" to "distro" 2: e60a40bd65 = 2: d076ed9857 ci: expose distro name in dockerized GitHub jobs 3: 16603d40fd ! 3: cc0c29052f ci: allow skipping sudo on dockerized jobs @@ Metadata Author: Patrick Steinhardt ## Commit message ## - ci: allow skipping sudo on dockerized jobs + ci: skip sudo when we are already root Our "install-dependencies.sh" script is executed by non-dockerized jobs to install dependencies. These jobs don't run with "root" permissions, 4: b4f6d6d3bf = 4: 803f5020e0 ci: drop duplicate package installation for "linux-gcc-default" 5: 6abc53bf51 ! 5: d2745e9438 ci: convert "install-dependencies.sh" to use "/bin/sh" @@ ci/install-dependencies.sh: ubuntu-*) - cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs . - popd + wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" -+ tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" -C "$GIT_LFS_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" ++ tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ ++ -C "$GIT_LFS_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" ;; macos-*) 6: d9be4db56f ! 6: 61f108d954 ci: merge custom PATH directories @@ ci/install-dependencies.sh: ubuntu-*) - mkdir --parents "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" -- tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" -C "$GIT_LFS_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" -+ tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ +- -C "$GIT_LFS_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" ++ -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" ;; macos-*) -: ---------- > 7: ef61b578da ci: fix setup of custom path for GitLab CI 7: 4a90c003d1 ! 8: 7748f87f8c ci: merge scripts which install dependencies @@ .github/workflows/main.yml: jobs: if: failure() && env.FAILED_TEST_ARTIFACTS != '' ## .gitlab-ci.yml ## -@@ .gitlab-ci.yml: workflow: - test:linux: - image: $image +@@ .gitlab-ci.yml: test:linux: + variables: + CUSTOM_PATH: "/custom" before_script: - - ./ci/install-docker-dependencies.sh + - ./ci/install-dependencies.sh @@ ci/install-dependencies.sh: then mkdir --parents "$CUSTOM_PATH" wget --quiet --directory-prefix="$CUSTOM_PATH" \ @@ ci/install-dependencies.sh: ubuntu-*) - tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" ;; +ubuntu32-*) 8: 5240046a0f ! 9: f7399382f2 ci: make Perforce binaries executable for all users @@ ci/install-dependencies.sh: ubuntu-*) + chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" - tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ 9: 29ceb623b9 ! 10: b835ff8b78 ci: install JGit dependency @@ ci/install-dependencies.sh: ubuntu-*) libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE @@ ci/install-dependencies.sh: ubuntu-*) - wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" - tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ + -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + + wget --quiet "$JGITWHENCE" --output-document="$CUSTOM_PATH/jgit" 10: fc3472cdf3 = 11: 7136c8b6c2 t06xx: always execute backend-specific tests 11: cedf5929d1 ! 12: cf4ee9c427 t0610: fix non-portable variable assignment @@ Commit message Such an assignment has been introduced in t0610. The issue wasn't detected for a while because this test used to only run when the - GIT_TEST_DEFAULT_REF_FORMAT environment variable was set to "refatble". + GIT_TEST_DEFAULT_REF_FORMAT environment variable was set to "reftable". We have dropped that requirement now though, meaning that it runs - unconditionally, inclluding on jobs which use such older versions of + unconditionally, including on jobs which use such older versions of Ubuntu. We have worked around such issues in the past, e.g. in ebee5580ca 12: 160b026e69 ! 13: a9cd20eebc t0612: add tests to exercise Git/JGit reftable compatibility @@ t/t0612-reftable-jgit-compatibility.sh (new) + +if ! test_have_prereq JGIT +then -+ skip_all='skipping reftable JGit tests' ++ skip_all='skipping reftable JGit tests; JGit is not present in PATH' + test_done +fi +