From patchwork Wed Apr 10 02:13:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10892937 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AC5071515 for ; Wed, 10 Apr 2019 02:13:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 94B4A285A4 for ; Wed, 10 Apr 2019 02:13:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 893F6285AF; Wed, 10 Apr 2019 02:13:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B4EC285A4 for ; Wed, 10 Apr 2019 02:13:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727169AbfDJCNP (ORCPT ); Tue, 9 Apr 2019 22:13:15 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:36378 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726532AbfDJCNP (ORCPT ); Tue, 9 Apr 2019 22:13:15 -0400 Received: by mail-pl1-f195.google.com with SMTP id ck15so390608plb.3 for ; Tue, 09 Apr 2019 19:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=QHITU3saHrz6FKalxeyHDbDq0zATq6MwsUSWFfoZ3Fs=; b=cDCI81w/nKDI+G+nB6sado1p8pBbqTEpofkrsFdk1IDlT/q6sIwHUn6x7CL4FTNqD9 57L12ffkr9wBLnm+l4wjNE4X8UE3et+Cb/LLuEHRjQhwz045TgGdiIhJ8deKt74aK6X5 4SP8i7nHEIv//Gl1WRhylZW31kDe952DyfwRbAqrfJhFzv66w2+asTWp9c0XkrKuNgiq O7+E+fYkhjVMXDSUgUJwqzfdGea8G6mKL8bCVvvQ1OvJwGNHMm+pGoEBEqf7DrXN/5yX n2caZJ3Sh/CAPjsBUej/wifp6AYs6UkF0FrSxDWLPodcDPxXukOFbFHgdaUwze6OA7BO ONLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=QHITU3saHrz6FKalxeyHDbDq0zATq6MwsUSWFfoZ3Fs=; b=izBnrM5Q/UGPuS8oTpMhfuFoUw86HqjvAHUERxG0+ZIzwGwBooGljFE4BUghUrMgJc buQmwMQy2e+V8Xa5sznT7vMj+tTEKiInfr0uqzMuPUV2g7x6DFcabfxioMGkO/UdH4AI J9eEFCNGAC9mpXHy2Qa820FwFGWpEWcOz58Zl2ecHRO284Ty8laYAjIhV5xnWZy/MH4M meHPfFH9sGF+6p7MUeNI07TsMToW2xDc9jvuvQ1ua/ro032jjdERIi92xKfeoW00zKQg ho0bNqcoDG9yUkkMgamO3pSrQjgW5lOR5bW7kVGhb0gmZE5gscEGV34yv8DfkH1xxw5k hGIg== X-Gm-Message-State: APjAAAWlkn5lvgmFPHd1/CIluTuUP3CcrnVTBnyJ6uZl8mq743IxE/Kf yEfp2cGYPzYI/mNrVBKOJLQLVFS1LlrB8dci X-Google-Smtp-Source: APXvYqwYCIWuCnGlDwE1LeOifz7tsMAcJ8Ea2N/0o5Sc2N4ycvvJXIpg4Iv5Vy04rUx1Ta1SFHKCNg== X-Received: by 2002:a17:902:820e:: with SMTP id x14mr40774954pln.207.1554862392830; Tue, 09 Apr 2019 19:13:12 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:c47d:9491:9418:887]) by smtp.gmail.com with ESMTPSA id y37sm48546862pgk.78.2019.04.09.19.13.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2019 19:13:12 -0700 (PDT) Date: Tue, 9 Apr 2019 19:13:11 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com, peff@peff.net, sunshine@sunshineco.com, szeder.dev@gmail.com Subject: [PATCH v2 1/7] t: move 'hex2oct' into test-lib-functions.sh Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The helper 'hex2oct' is used to convert base-16 encoded data into a base-8 binary form, and is useful for preparing data for commands that accept input in a binary format, such as 'git hash-object', via 'printf'. This helper is defined identically in three separate places throughout 't'. Move the definition to test-lib-function.sh, so that it can be used in new test suites, and its definition is not redundant. This will likewise make our job easier in the subsequent commit, which also uses 'hex2oct'. Signed-off-by: Taylor Blau --- t/t1007-hash-object.sh | 4 ---- t/t1450-fsck.sh | 4 ---- t/t5601-clone.sh | 4 ---- t/test-lib-functions.sh | 6 ++++++ 4 files changed, 6 insertions(+), 12 deletions(-) diff --git a/t/t1007-hash-object.sh b/t/t1007-hash-object.sh index a37753047e..7099d33508 100755 --- a/t/t1007-hash-object.sh +++ b/t/t1007-hash-object.sh @@ -199,10 +199,6 @@ test_expect_success 'too-short tree' ' test_i18ngrep "too-short tree object" err ' -hex2oct() { - perl -ne 'printf "\\%03o", hex for /../g' -} - test_expect_success 'malformed mode in tree' ' hex_sha1=$(echo foo | git hash-object --stdin -w) && bin_sha1=$(echo $hex_sha1 | hex2oct) && diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index 49f08d5b9c..0f268a3664 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -256,10 +256,6 @@ test_expect_success 'unparseable tree object' ' test_i18ngrep ! "fatal: empty filename in tree entry" out ' -hex2oct() { - perl -ne 'printf "\\%03o", hex for /../g' -} - test_expect_success 'tree entry with type mismatch' ' test_when_finished "remove_object \$blob" && test_when_finished "remove_object \$tree" && diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh index d6948cbdab..3f49943010 100755 --- a/t/t5601-clone.sh +++ b/t/t5601-clone.sh @@ -611,10 +611,6 @@ test_expect_success 'GIT_TRACE_PACKFILE produces a usable pack' ' git -C replay.git index-pack -v --stdin X-Patchwork-Id: 10892939 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D4E717E1 for ; Wed, 10 Apr 2019 02:13:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75ADD285A4 for ; Wed, 10 Apr 2019 02:13:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A418289C4; Wed, 10 Apr 2019 02:13:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BE7C8285A4 for ; Wed, 10 Apr 2019 02:13:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727178AbfDJCNR (ORCPT ); Tue, 9 Apr 2019 22:13:17 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:32976 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726532AbfDJCNR (ORCPT ); Tue, 9 Apr 2019 22:13:17 -0400 Received: by mail-pl1-f195.google.com with SMTP id t16so399677plo.0 for ; Tue, 09 Apr 2019 19:13:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=184R+gIqcEZHsADEqxOOSCWhDR9lJDz8+T+aZFKddmI=; b=SjSo8ARAs9TycQU62bvePVV2D58RIN+S7i7Ajg3ozrNMLUFB0Zhrzd2V73kBr51XY8 gQIlHnaULotzUMePW4BS1QQtU9qmNSrsmm5CXq6Sqr8y311Vdb9EalOVf+iMDXLZImul p7q2zn2iwUqH9d24h4FS/pSRq9uwqUQWcK0VqcQSTpEN6uBd4Jw5vZF3o6j70+QWfBcP gXLt6TMh+5LMhov6yiiG1s1ACfm7xNd46iJaN5iX371kgEoJt2m47C1oFHNPqLh/R8cA ihJ30tD2QHHFIEqQQY1fyIh25kdbQKiO4cBFpdWgouEspENCHQ1qQMF1K7gxjn5+SUwX PAPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=184R+gIqcEZHsADEqxOOSCWhDR9lJDz8+T+aZFKddmI=; b=jA0G2yGHAd61mhtZsaELJxCZRWMkdY7XL/fm1zXRLoYzo9678X3KEhHn4kBH2gzME/ FAAl8eAteKaYZGTzx7GZfeHFqFfjHngCGIp5ngvzMGzutMe6euBCKgLvcP+huJBFIW/Z zb1gy04Q8lMDIWeaIAsJSu5Qv9uBbehysxschQvtBAJAeQvkJUoIXqQOfil/eief5z/e 2yLfFiCXDwDpkKFxtC+bkV5AkPXNmQLWE7hVfwRqRBGC3XtdP7hrxfLvCzBQdZFAHLX3 W6fNW2EdBZvwhajZ86AH9QmcvlRCOrOjxLmlzsBAGKSZW9vAXwN5u7L8vbNuNsIhavV/ eg0w== X-Gm-Message-State: APjAAAVNyb4PwmBGioN6RqGZ/QJCgD0abCTNssh7kdm7LySDoSmYa3/i wR99TEa9hPa7/7O7cTDl2Fs7X/ngXJTAjZdR X-Google-Smtp-Source: APXvYqwlUG0L6S/Bl1klis8HMrM0wjGbChnWjQLGtdASKAi1/WyzNiLHIKmyIijIGLN998SHvE9jug== X-Received: by 2002:a17:902:31c3:: with SMTP id x61mr39412727plb.143.1554862395886; Tue, 09 Apr 2019 19:13:15 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:c47d:9491:9418:887]) by smtp.gmail.com with ESMTPSA id k10sm22966695pgo.82.2019.04.09.19.13.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2019 19:13:15 -0700 (PDT) Date: Tue, 9 Apr 2019 19:13:14 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com, peff@peff.net, sunshine@sunshineco.com, szeder.dev@gmail.com Subject: [PATCH v2 2/7] t: introduce tests for unexpected object types Message-ID: <137e2df24d80f796675691e57d08948b6eede6e3.1554861974.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Call an object's type "unexpected" when the actual type of an object does not match Git's contextual expectation. For example, a tree entry whose mode differs from the object's actual type, or a commit's parent which is not another commit, and so on. This can manifest itself in various unfortunate ways, including Git SIGSEGV-ing under specific conditions. Consider the following example: Git traverses a blob (say, via `git rev-list`), and then tries to read out a tree-entry which lists that object as something other than a blob. In this case, `lookup_blob()` will return NULL, and the subsequent dereference will result in a SIGSEGV. Introduce tests that present objects of "unexpected" type in the above fashion to 'git rev-list'. Mark as failures the combinations that are already broken (i.e., they exhibit the segfault described above). In the cases that are not broken (i.e., they have NULL-ness checks or similar), mark these as expecting success. We might hit an unexpected type in two different ways (imagine we have a tree entry that claims to be a tree but actually points to a blob): - when we call lookup_tree(), we might find that we've already seen the object referenced as a blob, in which case we'd get NULL. We can exercise this with "git rev-list --objects $blob $tree", which guarantees that the blob will have been parsed before we look in the tree. These tests are marked as "seen" in the test script. - we call lookup_tree() successfully, but when we try to read the object, we find out it's something else. We construct our tests such that $blob is not otherwise mentioned in $tree. These tests are marked as "lone" in the script. We should check that we behave sensibly in both cases (especially because it is easy for a malicious actor to provoke one case or the other). Co-authored-by: Jeff King Signed-off-by: Taylor Blau Signed-off-by: Jeff King --- t/t6102-rev-list-unexpected-objects.sh | 123 +++++++++++++++++++++++++ 1 file changed, 123 insertions(+) create mode 100755 t/t6102-rev-list-unexpected-objects.sh diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh new file mode 100755 index 0000000000..15072ecce3 --- /dev/null +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -0,0 +1,123 @@ +#!/bin/sh + +test_description='git rev-list should handle unexpected object types' + +. ./test-lib.sh + +test_expect_success 'setup well-formed objects' ' + blob="$(printf "foo" | git hash-object -w --stdin)" && + tree="$(printf "100644 blob $blob\tfoo" | git mktree)" && + commit="$(git commit-tree $tree -m "first commit")" && + git cat-file commit $commit >good-commit +' + +test_expect_success 'setup unexpected non-blob entry' ' + printf "100644 foo\0$(echo $tree | hex2oct)" >broken-tree && + broken_tree="$(git hash-object -w --literally -t tree broken-tree)" +' + +test_expect_failure 'traverse unexpected non-blob entry (lone)' ' + test_must_fail git rev-list --objects $broken_tree +' + +test_expect_failure 'traverse unexpected non-blob entry (seen)' ' + test_must_fail git rev-list --objects $tree $broken_tree +' + +test_expect_success 'setup unexpected non-tree entry' ' + printf "40000 foo\0$(echo $blob | hex2oct)" >broken-tree && + broken_tree="$(git hash-object -w --literally -t tree broken-tree)" +' + +test_expect_failure 'traverse unexpected non-tree entry (lone)' ' + test_must_fail git rev-list --objects $broken_tree +' + +test_expect_failure 'traverse unexpected non-tree entry (seen)' ' + test_must_fail git rev-list --objects $blob $broken_tree +' + +test_expect_success 'setup unexpected non-commit parent' ' + sed "/^author/ { h; s/.*/parent $blob/; G; }" broken-commit && + broken_commit="$(git hash-object -w --literally -t commit \ + broken-commit)" +' + +test_expect_success 'traverse unexpected non-commit parent (lone)' ' + test_must_fail git rev-list --objects $broken_commit >output 2>&1 && + test_i18ngrep "not a commit" output +' + +test_expect_success 'traverse unexpected non-commit parent (seen)' ' + test_must_fail git rev-list --objects $commit $broken_commit \ + >output 2>&1 && + test_i18ngrep "not a commit" output +' + +test_expect_success 'setup unexpected non-tree root' ' + sed -e "s/$tree/$blob/" broken-commit && + broken_commit="$(git hash-object -w --literally -t commit \ + broken-commit)" +' + +test_expect_failure 'traverse unexpected non-tree root (lone)' ' + test_must_fail git rev-list --objects $broken_commit +' + +test_expect_failure 'traverse unexpected non-tree root (seen)' ' + test_must_fail git rev-list --objects $blob $broken_commit +' + +test_expect_success 'setup unexpected non-commit tag' ' + git tag -a -m "tagged commit" tag $commit && + git cat-file tag tag >good-tag && + test_when_finished "git tag -d tag" && + sed -e "s/$commit/$blob/" broken-tag && + tag=$(git hash-object -w --literally -t tag broken-tag) +' + +test_expect_success 'traverse unexpected non-commit tag (lone)' ' + test_must_fail git rev-list --objects $tag +' + +test_expect_success 'traverse unexpected non-commit tag (seen)' ' + test_must_fail git rev-list --objects $blob $tag >output 2>&1 && + test_i18ngrep "not a commit" output +' + +test_expect_success 'setup unexpected non-tree tag' ' + git tag -a -m "tagged tree" tag $tree && + git cat-file tag tag >good-tag && + test_when_finished "git tag -d tag" && + sed -e "s/$tree/$blob/" broken-tag && + tag=$(git hash-object -w --literally -t tag broken-tag) +' + +test_expect_success 'traverse unexpected non-tree tag (lone)' ' + test_must_fail git rev-list --objects $tag +' + +test_expect_success 'traverse unexpected non-tree tag (seen)' ' + test_must_fail git rev-list --objects $blob $tag >output 2>&1 && + test_i18ngrep "not a tree" output +' + +test_expect_success 'setup unexpected non-blob tag' ' + git tag -a -m "tagged blob" tag $blob && + git cat-file tag tag >good-tag && + test_when_finished "git tag -d tag" && + sed -e "s/$blob/$commit/" broken-tag && + tag=$(git hash-object -w --literally -t tag broken-tag) +' + +test_expect_failure 'traverse unexpected non-blob tag (lone)' ' + test_must_fail git rev-list --objects $tag +' + +test_expect_success 'traverse unexpected non-blob tag (seen)' ' + test_must_fail git rev-list --objects $commit $tag >output 2>&1 && + test_i18ngrep "not a blob" output +' + +test_done From patchwork Wed Apr 10 02:13:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10892941 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 984BE17E1 for ; Wed, 10 Apr 2019 02:13:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82997285A4 for ; Wed, 10 Apr 2019 02:13:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 76D2E285AF; Wed, 10 Apr 2019 02:13:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 62FDB285A4 for ; Wed, 10 Apr 2019 02:13:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727186AbfDJCNT (ORCPT ); Tue, 9 Apr 2019 22:13:19 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:34089 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726930AbfDJCNT (ORCPT ); Tue, 9 Apr 2019 22:13:19 -0400 Received: by mail-pg1-f193.google.com with SMTP id v12so548581pgq.1 for ; Tue, 09 Apr 2019 19:13:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=iYU1FBmhk3gn/IXklEp8UKZLWeSuZxcJ3xVZ0/pfIYg=; b=RSe7fGh5/pctSraZcxmrRfUU+6ZBv73dFBZ2G1DwWzAGediTKv8hUlEZiY2Y+Zv4gC vSzhyRoQwlZid7ddCH8WpXfvJXeSSZl8fAI5saRFTEWQfB8ODpqRB9hpp2a5ZKgMezT/ LwiEZV2jDGrfKLcUaoG1emqePBvn7C5fDxCsStC2U2A0u8lEEYjm2CL5dFdSUR0/pW0k 3zVDDXcVN3Kr4OTY+U0Cz5MhmPNImBYoMjPwOVbbo6WeanONgsa4hMMxIhTX9lK1tq6e X5gQMtuNDogxbBXfmDnWXkm6CFQYOfF2kStmZLd3OYFfM876Ua8/MLVHb8hyKdnvgmQe 11DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=iYU1FBmhk3gn/IXklEp8UKZLWeSuZxcJ3xVZ0/pfIYg=; b=qO+sQQo/CY8OZY7r98wLeToYNlVu0S8OXFzXZI4Pv/0qvlmeTTNr14TWz7uh/lvDhn NVdDUyCk+41IkEn2UwOEIWjCkt0P4yr9MjoigsrR4ZY0WJcoeta5udq4pccsMYcoce/H 2KpOynWaDRlD9mxhe0DunE+7ANjArpmfzi5UuxCOyYXs3GhU3upIIy6X3tKCPRePmfmR 7zlwCs1rMHsRg3BIM4/A852M4hQ201v+VoJEvn3Ps/qk2wkJFE/kbMXhIam5EXn5BGxN +cKR/x025fHAqnIf0C2FZLcLpZv7QDBX4Mw0G63NM8TGqSOxUFnVMgRC9Ayn+I0R0nqo BXIQ== X-Gm-Message-State: APjAAAVxk/jtmMB0Dsr45EQkazt8pyLszK5UntnphoyvfPABqatk7J5z mVy9qco8GU1uCg+opNCjX4cGD36fGQyKiqLm X-Google-Smtp-Source: APXvYqz1wZdp/kK6fgg5cPPiEvzMz7ABVH1WSDymU6xVHnDV3d22Tmmky8IzIOvvzkhtOsMAax2ZPA== X-Received: by 2002:aa7:928b:: with SMTP id j11mr21196898pfa.200.1554862398277; Tue, 09 Apr 2019 19:13:18 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:c47d:9491:9418:887]) by smtp.gmail.com with ESMTPSA id u17sm47176298pfn.19.2019.04.09.19.13.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2019 19:13:17 -0700 (PDT) Date: Tue, 9 Apr 2019 19:13:17 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com, peff@peff.net, sunshine@sunshineco.com, szeder.dev@gmail.com Subject: [PATCH v2 3/7] list-objects.c: handle unexpected non-blob entries Message-ID: <1d1ac9b7a74fefccb6003f4ab8eb1a6d3660dff7.1554861974.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Fix one of the cases described in the previous commit where a tree-entry that is promised to a blob is in fact a non-blob. When 'lookup_blob()' returns NULL, it is because Git has cached the requested object as a non-blob. In this case, prevent a SIGSEGV by 'die()'-ing immediately before attempting to dereference the result. Signed-off-by: Taylor Blau --- list-objects.c | 5 +++++ t/t6102-rev-list-unexpected-objects.sh | 5 +++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index dc77361e11..ea04bbdee6 100644 --- a/list-objects.c +++ b/list-objects.c @@ -133,6 +133,11 @@ static void process_tree_contents(struct traversal_context *ctx, base, entry.path); else { struct blob *b = lookup_blob(ctx->revs->repo, &entry.oid); + if (!b) { + die(_("entry '%s' in tree %s has blob mode, " + "but is not a blob"), + entry.path, oid_to_hex(&tree->object.oid)); + } b->object.flags |= NOT_USER_GIVEN; process_blob(ctx, b, base, entry.path); } diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh index 15072ecce3..1377c60378 100755 --- a/t/t6102-rev-list-unexpected-objects.sh +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -20,8 +20,9 @@ test_expect_failure 'traverse unexpected non-blob entry (lone)' ' test_must_fail git rev-list --objects $broken_tree ' -test_expect_failure 'traverse unexpected non-blob entry (seen)' ' - test_must_fail git rev-list --objects $tree $broken_tree +test_expect_success 'traverse unexpected non-blob entry (seen)' ' + test_must_fail git rev-list --objects $tree $broken_tree >output 2>&1 && + test_i18ngrep "is not a blob" output ' test_expect_success 'setup unexpected non-tree entry' ' From patchwork Wed Apr 10 02:13:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10892943 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E9901515 for ; Wed, 10 Apr 2019 02:13:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3956A285A4 for ; Wed, 10 Apr 2019 02:13:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2D572285AF; Wed, 10 Apr 2019 02:13:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE08F285A4 for ; Wed, 10 Apr 2019 02:13:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727190AbfDJCNW (ORCPT ); Tue, 9 Apr 2019 22:13:22 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:46389 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726532AbfDJCNV (ORCPT ); Tue, 9 Apr 2019 22:13:21 -0400 Received: by mail-pl1-f195.google.com with SMTP id y6so358816pll.13 for ; Tue, 09 Apr 2019 19:13:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=FUyReao+6GzBUUf9qm/1o6mzN/asMwdmDcEImgML47I=; b=oTvTnDvTV3uoKax3wsKt4bTjT/MaWw3G4OqRsUPYnGzaHAwnXGgeXEzktIc6Te1y9+ KGjflkOt9xGpe5ioLIgOS1jgcDlBVNVuq4R1QeXtTHObS5atMTS63Mdwn1mi5KLIoS0i 051CRksVZydAOZsFXtGTsMKlizIc4X//C62VEy6x7prt9bxHFWQmHBvMWu864PVO5r7g 8rhGCKXoVjz60SYQBtL6Oa5iZsGscubY/yBjjazKBeohGZuBuBG02WkneE//aCwrkQHm l0Na13n9TR2t67uQhRxXrpGrV63XRkj8r5hd9k7DcKHTV54uT6s4E1aWXuR+oHY35NfU I63g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=FUyReao+6GzBUUf9qm/1o6mzN/asMwdmDcEImgML47I=; b=UlfizEbaH8JmGQb2tVPSoBjh6F0jlse0FJBomcWod2p7Fj2HphkQUUwSK+jRA9MZZv dKtGC+05diCN7I2o3ww2hD9ZVeM4uvGk7k19+t2DPYJFemAAabsmcz7359cAiGPVFvFS 9cnGUJYGzo5fUHHKbmC26/5o1VezGJhR3T1LBuEZbYlx9PL9TVdEHCW34thvlXn+hLFM EXlBa+S1enml7EIiVGa9V6DaQpDvEeDEgvEwMVz14sl20kHw0PWlLGnOoNxJJaRq9wTy sz3/BjQRj8koAfNnXR6eTmnyfk3af2n1V1HNtRTqXNH/+2siN7JQWWw/JZjGzbjIDlMZ DdOg== X-Gm-Message-State: APjAAAXH18v10b8wgwiLmOHV+sG448aCc1XFIOzf2Jxw3nnSlSNdPhsF sojO4xWdjTA96Y3075nLiE1Oxy7i3zAArpIr X-Google-Smtp-Source: APXvYqwMAs98mb7kxk297F5cEO/Tp0/Ia5n9Z2tTRRUqAcj4U0SjGgll9yMGPQAx0pdn7noHi2DvaA== X-Received: by 2002:a17:902:8345:: with SMTP id z5mr40989145pln.197.1554862400206; Tue, 09 Apr 2019 19:13:20 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:c47d:9491:9418:887]) by smtp.gmail.com with ESMTPSA id z22sm18223311pgv.23.2019.04.09.19.13.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2019 19:13:19 -0700 (PDT) Date: Tue, 9 Apr 2019 19:13:19 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com, peff@peff.net, sunshine@sunshineco.com, szeder.dev@gmail.com Subject: [PATCH v2 4/7] list-objects.c: handle unexpected non-tree entries Message-ID: <97c7e23ec9c2ac33abb2e110c5425482ecfd1925.1554861974.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Apply similar treatment as the previous commit for non-tree entries, too. Signed-off-by: Taylor Blau --- list-objects.c | 5 +++++ t/t6102-rev-list-unexpected-objects.sh | 5 +++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index ea04bbdee6..bb7e61ef4b 100644 --- a/list-objects.c +++ b/list-objects.c @@ -125,6 +125,11 @@ static void process_tree_contents(struct traversal_context *ctx, if (S_ISDIR(entry.mode)) { struct tree *t = lookup_tree(ctx->revs->repo, &entry.oid); + if (!t) { + die(_("entry '%s' in tree %s has tree mode, " + "but is not a tree"), + entry.path, oid_to_hex(&tree->object.oid)); + } t->object.flags |= NOT_USER_GIVEN; process_tree(ctx, t, base, entry.path); } diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh index 1377c60378..e3ec195532 100755 --- a/t/t6102-rev-list-unexpected-objects.sh +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -34,8 +34,9 @@ test_expect_failure 'traverse unexpected non-tree entry (lone)' ' test_must_fail git rev-list --objects $broken_tree ' -test_expect_failure 'traverse unexpected non-tree entry (seen)' ' - test_must_fail git rev-list --objects $blob $broken_tree +test_expect_success 'traverse unexpected non-tree entry (seen)' ' + test_must_fail git rev-list --objects $blob $broken_tree >output 2>&1 && + test_i18ngrep "is not a tree" output ' test_expect_success 'setup unexpected non-commit parent' ' From patchwork Wed Apr 10 02:13:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10892945 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AC52217E1 for ; Wed, 10 Apr 2019 02:13:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93B50285A4 for ; Wed, 10 Apr 2019 02:13:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8831C285AF; Wed, 10 Apr 2019 02:13:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3360D285A4 for ; Wed, 10 Apr 2019 02:13:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727198AbfDJCNY (ORCPT ); Tue, 9 Apr 2019 22:13:24 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:39107 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726532AbfDJCNX (ORCPT ); Tue, 9 Apr 2019 22:13:23 -0400 Received: by mail-pl1-f196.google.com with SMTP id a96so382412pla.6 for ; Tue, 09 Apr 2019 19:13:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=SrdbbYJLYN1A9NpG7aHq8VQP3FpKbnoaDhrpoWW65iU=; b=Un99nteXuGo+xHLrlm21CjfFNy79eiUTu1CBTsnVBod0ZaTrV3duVTJGyCzIymS8RB y8LxJe9CQhbRFWtgL6x1s6w5M+Lyl7HpymPtlBwJc9YkXlafZm9oSABjm0LirxUyjm28 WaAt9HZEhZuriLyUZvfoAc8qQNzsPP7nwZt0ysz7QXQrYlKeHyHkg26ErMHmqehK508V AawxFbByd0upRZvSym5/IcsRLM2DkSXTwl8HXZvtyxMP8LHHOnnMahEz2bkpx/R/gRer h+Sxnxt+OFuwKsNBA8R+LWzRULuZ7dJjVa3sibviWPRoRS3At69bK76REP0tRmaXd0Js t4gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=SrdbbYJLYN1A9NpG7aHq8VQP3FpKbnoaDhrpoWW65iU=; b=dwtf9E+hup71294YR/Js6Ya/KZnQod0hRkBu4s4+txkcl1LYMxJOUoCTCL/NvTFEmX E6QSEv0fjkW2AE3EVFzB2b+u5TZI+mUqJSdbHYuPYHIlrUTAr62C0Bb6DbyPCKs4lNhH 7EEJPj0uUvILjlPYr4PVl54PLEntKWDgBHRv2DVF803QngqV/tbPEOrFuB2auH21MflC Mm4Fb9UVFoJJTdWuauq2PBWXIAwjpDEdFRYlVE8jrK7CeQ6aEyNX4GlpfEzt+GimCNlH fYgXCXj0dPhYG9qwytU8PgDXaqk68c2hnvJ/mGe8memRbuuo7sMcnm1UUeIDcPSPWFzj Eakg== X-Gm-Message-State: APjAAAUm1CSsfYG+HGXO1Roy/wXhRcX/Ow3c7x4Vbyf2GaRUYHvELlt0 LI6K5SIhTrdsNP/eh+f51OkI2QgXbgKxQh1K X-Google-Smtp-Source: APXvYqw6Z6IvqNICYvXQsM2mdDxljh0XAtekqHXCkiHFZ8s40ngiWk27rX81fNKYDO1HU9ZPZ2QNgQ== X-Received: by 2002:a17:902:e9:: with SMTP id a96mr20611001pla.176.1554862402247; Tue, 09 Apr 2019 19:13:22 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:c47d:9491:9418:887]) by smtp.gmail.com with ESMTPSA id a17sm55984568pfj.123.2019.04.09.19.13.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2019 19:13:21 -0700 (PDT) Date: Tue, 9 Apr 2019 19:13:20 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com, peff@peff.net, sunshine@sunshineco.com, szeder.dev@gmail.com Subject: [PATCH v2 5/7] get_commit_tree(): return NULL for broken tree Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jeff King Return NULL from 'get_commit_tree()' when a commit's root tree is corrupt, doesn't exist, or points to an object which is not a tree. In [1], this situation became a BUG(), but it can certainly occur in cases which are not a bug in Git, for e.g., if a caller manually crafts a commit whose tree is corrupt in any of the above ways. Note that the expect_failure test in t6102 triggers this BUG(), but we can't flip it to expect_success yet. Solving this problem actually reveals a second bug. [1]: 7b8a21dba1 (commit-graph: lazy-load trees for commits, 2018-04-06) Co-authored-by: Taylor Blau Signed-off-by: Taylor Blau Signed-off-by: Jeff King --- commit.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/commit.c b/commit.c index a5333c7ac6..e2cde566a9 100644 --- a/commit.c +++ b/commit.c @@ -345,10 +345,10 @@ struct tree *get_commit_tree(const struct commit *commit) if (commit->maybe_tree || !commit->object.parsed) return commit->maybe_tree; - if (commit->graph_pos == COMMIT_NOT_FROM_GRAPH) - BUG("commit has NULL tree, but was not loaded from commit-graph"); + if (commit->graph_pos != COMMIT_NOT_FROM_GRAPH) + return get_commit_tree_in_graph(the_repository, commit); - return get_commit_tree_in_graph(the_repository, commit); + return NULL; } struct object_id *get_commit_tree_oid(const struct commit *commit) From patchwork Wed Apr 10 02:13:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10892947 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F324A17E1 for ; Wed, 10 Apr 2019 02:13:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB152285A4 for ; Wed, 10 Apr 2019 02:13:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CEC3C285AF; Wed, 10 Apr 2019 02:13:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 53CBB285A4 for ; Wed, 10 Apr 2019 02:13:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727204AbfDJCN0 (ORCPT ); Tue, 9 Apr 2019 22:13:26 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:46394 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726532AbfDJCNZ (ORCPT ); Tue, 9 Apr 2019 22:13:25 -0400 Received: by mail-pl1-f195.google.com with SMTP id y6so358916pll.13 for ; Tue, 09 Apr 2019 19:13:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=J6pvJyAVVkdi2nWzGxuznoVHl8qMv+EJV5RdSVqHEUA=; b=SBuECwnNKr3A+vnUBMMOB5UpfiEIp0gX27WTO8u5ntGRHKcwQIZ3t/XXApRZ3foEbd +u4z30sTx3Fe7qwzYIaFWFdLaxGDsYkkT+d4m0oZUKl5tVQwA9uc4HFC/40R6qw1D8aE NGg3riPzMKY2gd2oFrVnJ/GlKpsVGriT4HZV7xkdFQwbHeGUO+H9etDBH0lef7x/bSoh +Jw1ygWZCLM/+sSU0gn1JOch9lEpxNEoSWJunKTjGMiNucn0wNA8D8iSzNjmbDPGUZAm sEJ+OAfjBBi/nq5bw/u77sL+ZnSdAuc7HrO7vKHTQm6bEj2jDLn72W6+HVw/VTbKC1lR 1zlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=J6pvJyAVVkdi2nWzGxuznoVHl8qMv+EJV5RdSVqHEUA=; b=ms/B378d7sMzNnR6QNMwbjMz9fZuzEou5GLnxrGYIb/GRVTKgFVi4Xxa+XTRV+bnYj ylghfI2IPIEuULzetW/i+V0TEZa8zYuNbu+jie9YpgTRNEkt94Pr4FV1TRgjArLCe4iV 1j21t2JC2eY5H+NrgbqvNyROBSuky/g9D543MsIV1cVoAp4pAopYW6e0sFqNCioUB61H 1ew72ZtLjK5Xq1zfhZhBylF5teLlRcWyt/1JKMqtxBhmpjHAHolGCFcyF7Ss4t6QKeKR L3ZfeiOoaFgmWi2pXWKhvaKAHRp4dEHCUi7Kq8IMK8IdjKp2wBdkTLKLH8dwltMjjWVO AE5g== X-Gm-Message-State: APjAAAXJ7XBVs6rirVYdx0TPZJdplreCDfBHHWyC+vbFykx/y6A1UGhN ZlFSKrpzvam5xmdBQSSTr2wonocmCQBbST/p X-Google-Smtp-Source: APXvYqxdcirh0BXBCOOYycOe/YqYwdhdW78AyAf2wzIU10wB//sLjidL5XDEqWjSrUz9VYTf98EWHg== X-Received: by 2002:a17:902:b948:: with SMTP id h8mr40673218pls.39.1554862404176; Tue, 09 Apr 2019 19:13:24 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:c47d:9491:9418:887]) by smtp.gmail.com with ESMTPSA id g2sm50284156pfd.134.2019.04.09.19.13.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2019 19:13:23 -0700 (PDT) Date: Tue, 9 Apr 2019 19:13:23 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com, peff@peff.net, sunshine@sunshineco.com, szeder.dev@gmail.com Subject: [PATCH v2 6/7] rev-list: let traversal die when --missing is not in use Message-ID: <88ca5dfe68c4cdb558aff6e90a525ec06f15dbd6.1554861974.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jeff King Commit 7c0fe330d5 (rev-list: handle missing tree objects properly, 2018-10-05) taught the traversal machinery used by git-rev-list to ignore missing trees, so that rev-list could handle them itself. However, it does so only by checking via oid_object_info_extended() that the object exists at all. This can miss several classes of errors that were previously detected by rev-list: - type mismatches (e.g., we expected a tree but got a blob) - failure to read the object data (e.g., due to bitrot on disk) This is especially important because we use "rev-list --objects" as our connectivity check to admit new objects to the repository, and it will now miss these cases (though the bitrot one is less important here, because we'd typically have just hashed and stored the object). There are a few options to fix this: 1. we could check these properties in rev-list when we do the existence check. This is probably too expensive in practice (perhaps even for a type check, but definitely for checking the whole content again, which implies loading each object into memory twice). 2. teach the traversal machinery to differentiate between a missing object, and one that could not be loaded as expected. This probably wouldn't be too hard to detect type mismatches, but detecting bitrot versus a truly missing object would require deep changes to the object-loading code. 3. have the traversal machinery communicate the failure to the caller, so that it can decide how to proceed without re-evaluting the object itself. Of those, I think (3) is probably the best path forward. However, this patch does none of them. In the name of expediently fixing the regression to a normal "rev-list --objects" that we use for connectivity checks, this simply restores the pre-7c0fe330d5 behavior of having the traversal die as soon as it fails to load a tree (when --missing is set to MA_ERROR, which is the default). Note that we can't get rid of the object-existence check in finish_object(), because this also handles blobs (which are not otherwise checked at all by the traversal code). Signed-off-by: Jeff King --- builtin/rev-list.c | 4 +++- t/t6102-rev-list-unexpected-objects.sh | 4 ++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 425a5774db..9f31837d30 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -379,7 +379,6 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) repo_init_revisions(the_repository, &revs, prefix); revs.abbrev = DEFAULT_ABBREV; revs.commit_format = CMIT_FMT_UNSPECIFIED; - revs.do_not_die_on_missing_tree = 1; /* * Scan the argument list before invoking setup_revisions(), so that we @@ -409,6 +408,9 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) } } + if (arg_missing_action) + revs.do_not_die_on_missing_tree = 1; + argc = setup_revisions(argc, argv, &revs, &s_r_opt); memset(&info, 0, sizeof(info)); diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh index e3ec195532..28ee1bcb07 100755 --- a/t/t6102-rev-list-unexpected-objects.sh +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -30,7 +30,7 @@ test_expect_success 'setup unexpected non-tree entry' ' broken_tree="$(git hash-object -w --literally -t tree broken-tree)" ' -test_expect_failure 'traverse unexpected non-tree entry (lone)' ' +test_expect_success 'traverse unexpected non-tree entry (lone)' ' test_must_fail git rev-list --objects $broken_tree ' @@ -63,7 +63,7 @@ test_expect_success 'setup unexpected non-tree root' ' broken-commit)" ' -test_expect_failure 'traverse unexpected non-tree root (lone)' ' +test_expect_success 'traverse unexpected non-tree root (lone)' ' test_must_fail git rev-list --objects $broken_commit ' From patchwork Wed Apr 10 02:13:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10892949 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 691EA1515 for ; Wed, 10 Apr 2019 02:13:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 51528285A4 for ; Wed, 10 Apr 2019 02:13:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 44AAC285AF; Wed, 10 Apr 2019 02:13:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DCF9F285A4 for ; Wed, 10 Apr 2019 02:13:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727212AbfDJCN3 (ORCPT ); Tue, 9 Apr 2019 22:13:29 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:43337 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726532AbfDJCN1 (ORCPT ); Tue, 9 Apr 2019 22:13:27 -0400 Received: by mail-pf1-f194.google.com with SMTP id c8so412483pfd.10 for ; Tue, 09 Apr 2019 19:13:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=LLfpLlhpw31KK81zU7lFM9kt119J6dbXq0O/nIm28/8=; b=Rxe2WuuA52DcBt1Zairv9Osr1BlWjGyvCQImBs1tMTSEDYgzpPqSCcVTxjdhs5Cwx/ pfLPyu8AGQCzK3b4EJxtYe/ULrh9wGFaHMnEKSrmQgCZ512qyHSkcip8T47MBZgZjvHM mt4r65AsOiEHQJPVglYr3gs82K4m2XBfWv349Lsdg74vpmMd4Itm+vzT7Mam2N0iDl4F SO5HGT0gHhhg7a/ZeTlgE3lziZyQimiYeSEvfH0xhIjg7fz6ApM9FBzvpxk8ZVdKjIXG gpGE0zShJcdGjy1Hi/HjCA/NtdpBBGL5Dg6iulx1zroz7FEqVVsIcGnS7Whttk8/2vtT 43Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=LLfpLlhpw31KK81zU7lFM9kt119J6dbXq0O/nIm28/8=; b=kvoFQki4+stPMc+ctP61azsG/Pvb+2Bz9UbCQA/0jH8aqjL51crHsVFAMyayqaSphg 88Ahh0M439PRzjLkVCTkw5qHCrj449M93H2/S27KUKc0KKf/Lojctvv87nlmn3tSAA04 8E1SD4UFMcV2kR44rNiGS7QDq31RIw9Wb3/6IFwA/sfLl00PWFruTa81gsfF79bg2vCA D6g2bJzb+MXtM3YrS3v1L26JjGXymIceIWe49CQCoGrC75uMtHY5lduwzZJkBAdfkolh DA0n4fvbbzxAWz7G5EC4q21mPyjy4SUHK3g6JDiyL2y4BmY7mibsgyq9gMuiYdUMqxb7 Z3vA== X-Gm-Message-State: APjAAAXF/rnWKtgIl2jsSTh6SQ87FsKok4AFH4O15VMj1JjPxSZWs7I8 mY8OfXhcrUc/UxihVOkLczWrpv8+kpiuUwLj X-Google-Smtp-Source: APXvYqzutRrWrk1Kt7GPNlJgeKL8awziDsAl2CkZ//oa8sSoK10zDGfupLL3KRdqutmsRpnS6M61oA== X-Received: by 2002:aa7:83d1:: with SMTP id j17mr41084488pfn.78.1554862406390; Tue, 09 Apr 2019 19:13:26 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:c47d:9491:9418:887]) by smtp.gmail.com with ESMTPSA id u26sm46050806pfn.5.2019.04.09.19.13.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2019 19:13:25 -0700 (PDT) Date: Tue, 9 Apr 2019 19:13:25 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com, peff@peff.net, sunshine@sunshineco.com, szeder.dev@gmail.com Subject: [PATCH v2 7/7] rev-list: detect broken root trees Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jeff King When the traversal machinery sees a commit without a root tree, it assumes that the tree was part of a BOUNDARY commit, and quietly ignores the tree. But it could also be caused by a commit whose root tree is broken or missing. Instead, let's die() when we see a NULL root tree. We can differentiate it from the BOUNDARY case by seeing if the commit was actually parsed. This covers that case, plus future-proofs us against any others where we might try to show an unparsed commit. Signed-off-by: Jeff King --- list-objects.c | 3 +++ t/t6102-rev-list-unexpected-objects.sh | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index bb7e61ef4b..b5651ddd5b 100644 --- a/list-objects.c +++ b/list-objects.c @@ -374,6 +374,9 @@ static void do_traverse(struct traversal_context *ctx) struct tree *tree = get_commit_tree(commit); tree->object.flags |= NOT_USER_GIVEN; add_pending_tree(ctx->revs, tree); + } else if (commit->object.parsed) { + die(_("unable to load root tree for commit %s"), + oid_to_hex(&commit->object.oid)); } ctx->show_commit(commit, ctx->show_data); diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh index 28ee1bcb07..28611c978e 100755 --- a/t/t6102-rev-list-unexpected-objects.sh +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -67,8 +67,10 @@ test_expect_success 'traverse unexpected non-tree root (lone)' ' test_must_fail git rev-list --objects $broken_commit ' -test_expect_failure 'traverse unexpected non-tree root (seen)' ' - test_must_fail git rev-list --objects $blob $broken_commit +test_expect_success 'traverse unexpected non-tree root (seen)' ' + test_must_fail git rev-list --objects $blob $broken_commit \ + >output 2>&1 && + test_i18ngrep "not a tree" output ' test_expect_success 'setup unexpected non-commit tag' '