From patchwork Fri Jul 29 12:28:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12932395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 643C0C00144 for ; Fri, 29 Jul 2022 12:28:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235979AbiG2M2v (ORCPT ); Fri, 29 Jul 2022 08:28:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232085AbiG2M2u (ORCPT ); Fri, 29 Jul 2022 08:28:50 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0AAE924F38 for ; Fri, 29 Jul 2022 05:28:49 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id p10so1609981wru.8 for ; Fri, 29 Jul 2022 05:28:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=dYH/PJWKrAZvl1XQlIdFcunJNpZNVCkruyhnh38JqkU=; b=HQWd6a9Q51nwm37RIYiiE9gBPCoSSOfHKnKSZvhV1HLe5v8RhO5dpWVD7Xt2o88imz 96WTZFjSCufFpVn35l9vT2QSRNtuvmGIn3V2JcWuzFiDxzF77qUILE8eCKsZ8fNAQZPn AV62JF9IRuGZq+jZmilzq0KCPjgLf9mkeHgZejKeBtVx3Z1pzdEg0w9WdQemmaaem/Ic 3CxsvPfkm04I6Mu7cMY4vynOlvQF+yzl7u0yalUDLkS2yN1eZxA/dlSmE646jFrLKx8w WMrYfay5SjgJ+TxO9zSiIXjTQOYtLnn0miUJYXOKyjBi+UgGFXVijhcv44ZmIs/R8YBW 6htw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=dYH/PJWKrAZvl1XQlIdFcunJNpZNVCkruyhnh38JqkU=; b=Z6950Ff7f5h+SB8fUYuSS4PHsKkHhAvCRDnhTICQbGDOUwLms1jdH2fjwqudLBFnD3 szx3fLotK4JMggRQIxzu1SglL0lHTEoOqKfQ4bM66IXqrladhSRPfULuevOUrdRdBGcA dvAh4+u30N7JOSCN+3+/KbWXecf6SYvJ2sjr0EaWwphOyxWh2YWcziQH6Vc9vILolM8L FbVLJHR7lCXcNI8hxM2EHUmGvAi44WemVjUWzgCu5sHTo7+76FIaVGvQbIUMWWRwa3KE nTuEoULY/sc1vwfDBdq0H7fVeyMBFxtnwrOCxP1v+h2S0h3Tqb7kiSiUjBgfJZYlDxLv fT3g== X-Gm-Message-State: ACgBeo1EY+eFWPOEhTI/sE/mGCmS7oI0uW2CZ4lKGlyjNQZ36Gc0k1Za /ydA0/DGr2eP+Z5HGjDXzj2ZxWzbJTs= X-Google-Smtp-Source: AA6agR5EQ3RNeZHMwgUxDheAMGaQi9PSKFWTN33X8e+1Oo+BwC8lNMBxeNZyUBHjhnzs5d5sPEXfsQ== X-Received: by 2002:a5d:5266:0:b0:21f:1280:85f with SMTP id l6-20020a5d5266000000b0021f1280085fmr1515425wrc.412.1659097727096; Fri, 29 Jul 2022 05:28:47 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id bi9-20020a05600c3d8900b003a0375c4f73sm4000640wmb.44.2022.07.29.05.28.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 05:28:46 -0700 (PDT) Message-Id: <501a89aa0b2de396b0c7b82b2e24046b9c98c382.1659097724.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 29 Jul 2022 12:28:43 +0000 Subject: [PATCH 1/2] t5351: avoid relying on `core.fsyncMethod = batch` to be supported Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin On FreeBSD, this mode is not supported. But since 3a251bac0d1a (trace2: only include "fsync" events if we git_fsync(), 2022-07-18) t5351 will fail if this mode is unsupported. Let's address this in the minimal fashion, by detecting that that mode is unsupported and expecting a different count of hardware flushes in that case. This fixes the CI/PR builds on FreeBSD again. Note: A better way would be to test only what is relevant in t5351.6 "unpack big object in stream (core.fsyncmethod=batch)" again instead of blindly comparing the output against some exact text. But that would pretty much revert the idea of above-mentioned commit, and that commit was _just_ accepted into Git's main branch so one must assume that it was intentional. Signed-off-by: Johannes Schindelin --- bulk-checkin.c | 2 ++ t/t5351-unpack-large-objects.sh | 10 ++++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/bulk-checkin.c b/bulk-checkin.c index 98ec8938424..855b68ec23b 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -340,6 +340,8 @@ void fsync_loose_object_bulk_checkin(int fd, const char *filename) */ if (!bulk_fsync_objdir || git_fsync(fd, FSYNC_WRITEOUT_ONLY) < 0) { + if (errno == ENOSYS) + warning(_("core.fsyncMethod = batch is unsupported on this platform")); fsync_or_die(fd, filename); } } diff --git a/t/t5351-unpack-large-objects.sh b/t/t5351-unpack-large-objects.sh index f785cb06173..dd7ebc40072 100755 --- a/t/t5351-unpack-large-objects.sh +++ b/t/t5351-unpack-large-objects.sh @@ -70,9 +70,15 @@ test_expect_success 'unpack big object in stream (core.fsyncmethod=batch)' ' GIT_TRACE2_EVENT="$(pwd)/trace2.txt" \ GIT_TEST_FSYNC=true \ git -C dest.git $BATCH_CONFIGURATION unpack-objects X-Patchwork-Id: 12932396 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA7B2C19F29 for ; Fri, 29 Jul 2022 12:28:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236022AbiG2M2y (ORCPT ); Fri, 29 Jul 2022 08:28:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56958 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235972AbiG2M2v (ORCPT ); Fri, 29 Jul 2022 08:28:51 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30BEFBE3D for ; Fri, 29 Jul 2022 05:28:50 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id b26so5846292wrc.2 for ; Fri, 29 Jul 2022 05:28:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ji5bR1HZYHuEesrRLzUH0myHHYt73mi+kKbxx0xaJs8=; b=amsuNHwlnPwLh5wUv9CKJptVWlBCPf6fcXxtpkD8awN3NMLns1ZZzVH/zLLmABLns/ s4NDhaYjmSBvYfCq39NtignKbk77R1iTC1XqZzNj3HhgIlb75T8QTW0tz6FoK48hXbZG i1Mk5npPudgX+OJz13zUGck1xV2Io2kahsPr1gilSuSEwEuHF7uFSUmppkNWvvcflLkS vNQDfxw5i60+zJRWYI3bYxiGLnTZ8x+wuF6Om6Jrb2CKYO8CgEombBsaXv1J3A0+LD/d hmnLvFH8yxhTrVeqJx5kEN2mP6FdZwhIRKWlIuWa6RCl7cv3jWL4t4EIEy+ItSG/IivZ hbyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ji5bR1HZYHuEesrRLzUH0myHHYt73mi+kKbxx0xaJs8=; b=jVse1IIm5ANT0hegYKNmiyq0XQNMMVSlUpQlxw2Qfq/OnDO6LW/bGF5t2yQJUAgElx 3xdN1XD6/FNYqPzRWCsJWVDdHr911kr8W5kw95oXpQr3sV8EBCOMSd+Q5MXVZv7OtHsK YTFgor274zZLVRkB11d5tkUexK38TZRIeg9HF7Fiul61yG/Nuzs01n1tZmEh7D6YBB1x r6QeuQ9kMxLccCmmiiU2v8sp5GOl9ohpZ7e1JBL2PTxC89gKeUYFBRIyPDJWI7OAUDAB Jf1AIs3ms0Zukqx8UdBelZ72UWaV3xECeEebvthzn+tBUOJvk1LoUnjJQwSvMyKbOJAU 5Qsw== X-Gm-Message-State: ACgBeo36/5FX3vkaknTpMBJeQiTjqPZH+qX2SNhB/2QCtmU9+M4Y24aJ nPOtu9og+5MOz3NM5QofkxXsG42dhVw= X-Google-Smtp-Source: AA6agR6KQxzuQ2SEJaaaVljKnfVfcplWjlipe8U8EoDeMkfuwKYW6mdcz1J525GH3w/MwFhFVSV0dQ== X-Received: by 2002:a5d:4dc8:0:b0:21e:660f:8dfe with SMTP id f8-20020a5d4dc8000000b0021e660f8dfemr2266163wru.468.1659097728244; Fri, 29 Jul 2022 05:28:48 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j17-20020a05600c301100b003a32438c518sm8074622wmh.6.2022.07.29.05.28.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 05:28:47 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Fri, 29 Jul 2022 12:28:44 +0000 Subject: [PATCH 2/2] t5351: avoid using `test_cmp` for binary data Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin The `test_cmp` function is meant to provide nicer output than `cmp` when expected and actual output of Git commands disagree. The implicit assumption is that the output is line-based and human readable. However, aaf81223f48 (unpack-objects: use stream_loose_object() to unpack large objects, 2022-06-11) introduced a call that compares the contents of pack files, which are distinctly not line-based nor human readable. This causes problems because on Windows, we hand off to the Bash function `mingw_test_cmp` that compares the lines while ignoring line ending differences. And this Bash function spends an insane amount of cycles trying to read in that binary pack file, so that it is almost indistinguishable from an infinite loop. For example, t5351 took 1486 seconds in the CI run at https://github.com/git/git/runs/7398490747?check_suite_focus=true#step:5:171, to complete. And yes, that is almost half an hour. Since Git's tests already use `cmp` consistently when comparing pack files, let's change this instance to use `cmp` instead of `test_cmp`, too, and fix that performance problem. Now t5351 takes all of 22 seconds. Signed-off-by: Johannes Schindelin --- t/t5351-unpack-large-objects.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t5351-unpack-large-objects.sh b/t/t5351-unpack-large-objects.sh index dd7ebc40072..e936f91c3ba 100755 --- a/t/t5351-unpack-large-objects.sh +++ b/t/t5351-unpack-large-objects.sh @@ -93,7 +93,7 @@ test_expect_success 'do not unpack existing large objects' ' # The destination came up with the exact same pack... DEST_PACK=$(echo dest.git/objects/pack/pack-*.pack) && - test_cmp pack-$PACK.pack $DEST_PACK && + cmp pack-$PACK.pack $DEST_PACK && # ...and wrote no loose objects test_stdout_line_count = 0 find dest.git/objects -type f ! -name "pack-*"