From patchwork Thu Dec 16 16:13:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12681641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E7FEC433F5 for ; Thu, 16 Dec 2021 16:13:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239073AbhLPQNr (ORCPT ); Thu, 16 Dec 2021 11:13:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239011AbhLPQNr (ORCPT ); Thu, 16 Dec 2021 11:13:47 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8413BC061574 for ; Thu, 16 Dec 2021 08:13:46 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id i22so15319478wrb.13 for ; Thu, 16 Dec 2021 08:13:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=SPcmNAFRyeGcydZVsSbRi2Onf9sX9YSSmUTswrDmLZY=; b=n/ZKenjfWfqgN0+2Qw+ahjWv/2cTXjlyVlYEBIcUm1bivjyPbdkzpXQL2uHGXnhIfS KNVRlCs/48f75WBreDxTnXst9Sk9cqMxWwPC490/Yy0gOIqJYq/jcX8Csu0iJKrZrepg KJytqLEMVnHchVAGbWbdxx9gymCaJpCHrfbmJOCgAzoCm7uqWsPYVCi4EiIlGtKCEF1V aVd3kjW4SiXQ161g/3Vj7hZgnJWOumh5rPllyOMa2qDI7CSN5K66dURINmNo24RNvQAx cQaeWNTsc39nz+3v4X1EyNa+AVBwkMyQheJgMBDZdDKpYTcQH4n2PiF+kOnvFmnAx+G6 uUmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=SPcmNAFRyeGcydZVsSbRi2Onf9sX9YSSmUTswrDmLZY=; b=ur0YKUZZvj3Gr9p4sKOu1Oc2qFGJ5c+govpTgyUoc6rvIyXvJplBkBkmXzFHh4Pd5j h58RN+AC+KHr72IyjqVDVlA3wOmeUFUtqW9v8jIHlF5pc/Gp/BACS1B4k/10WNXgyv2H fhrS7WZOBkyRexi+LEFBkDriqBIbcI17yy1iQ1PYJwCZEe6OmTY3akyCG2hfx0y+9UW4 qEZz/Ne8KhcPz7vJF3iGs9piaHUEgKBvx2YC5zBvtEeErpAz4cZcUEVY7xwTQQHsf91o zg+K1lhVgPMW9s4AK5WpLQSL6gIf412S6irs/UDm6GXyKwOxkOIX6jRyNT+zRug1X23Z 1lQA== X-Gm-Message-State: AOAM530tUfSg4bMkTKI1qLFMqc/HTnGTwuHXwinOrHj7aTAPW9aJFsbH LcqGbSimUpQTasEPpU2WX+MyNIzi3og= X-Google-Smtp-Source: ABdhPJzSpe6fXD69XLbOyk2Z2KoJQckcO5T6uGifhJBJpA4MfoCqIySH3n12M15ekrVsVCRHch5NFA== X-Received: by 2002:a5d:4563:: with SMTP id a3mr9939432wrc.130.1639671224882; Thu, 16 Dec 2021 08:13:44 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y7sm4940962wrw.55.2021.12.16.08.13.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Dec 2021 08:13:44 -0800 (PST) Message-Id: <5353c541d9f72aa0a14cd2b02e5ddcc0b6a4dae1.1639671222.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 16 Dec 2021 16:13:40 +0000 Subject: [PATCH v4 1/3] sparse-checkout: fix segfault on malformed patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are used to populate two hashsets that accelerate pattern matching. If the user modifies the sparse-checkout file outside of the 'sparse-checkout' builtin, then strange patterns can happen, triggering some error checks. One of these error checks is possible to hit when some special characters exist in a line. A warning message is correctly written to stderr, but then there is additional logic that attempts to remove the line from the hashset and free the data. This leads to a segfault in the 'git sparse-checkout list' command because it iterates over the contents of the hashset, which is now invalid. The fix here is to stop trying to remove from the hashset. In addition, we disable cone mode sparse-checkout because of the malformed data. This results in the pattern-matching working with a possibly-slower algorithm, but using the patterns as they are in the sparse-checkout file. This also changes the behavior of commands such as 'git sparse-checkout list' because the output patterns will be the contents of the sparse-checkout file instead of the list of directories. This is an existing behavior for other types of bad patterns. Add a test that triggers the segfault without the code change. Reported-by: John Burnett Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee --- dir.c | 4 +--- t/t1091-sparse-checkout-builtin.sh | 21 +++++++++++++++++++++ 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/dir.c b/dir.c index 5aa6fbad0b7..7e72958d51d 100644 --- a/dir.c +++ b/dir.c @@ -819,9 +819,7 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern /* we already included this at the parent level */ warning(_("your sparse-checkout file may have issues: pattern '%s' is repeated"), given->pattern); - hashmap_remove(&pl->parent_hashmap, &translated->ent, &data); - free(data); - free(translated); + goto clear_hashmaps; } return; diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 272ba1b566b..3921ea80138 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -708,4 +708,25 @@ test_expect_success 'cone mode clears ignored subdirectories' ' test_cmp expect out ' +test_expect_success 'malformed cone-mode patterns' ' + git -C repo sparse-checkout init --cone && + mkdir -p repo/foo/bar && + touch repo/foo/bar/x repo/foo/y && + cat >repo/.git/info/sparse-checkout <<-\EOF && + /* + !/*/ + /foo/ + !/foo/*/ + /foo/\*/ + EOF + + # Listing the patterns will notice the duplicate pattern and + # emit a warning. It will list the patterns directly instead + # of using the cone-mode translation to a set of directories. + git -C repo sparse-checkout list >actual 2>err && + test_cmp repo/.git/info/sparse-checkout actual && + grep "warning: your sparse-checkout file may have issues: pattern .* is repeated" err && + grep "warning: disabling cone pattern matching" err +' + test_done From patchwork Thu Dec 16 16:13:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12681643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 738C0C433EF for ; Thu, 16 Dec 2021 16:13:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239143AbhLPQNv (ORCPT ); Thu, 16 Dec 2021 11:13:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239025AbhLPQNr (ORCPT ); Thu, 16 Dec 2021 11:13:47 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C013C061574 for ; Thu, 16 Dec 2021 08:13:47 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id e5so11691758wrc.5 for ; Thu, 16 Dec 2021 08:13:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5sdtls0qkuJiYusI35JI3atU6pS13AMJMmDf+mNHOnk=; b=oUpzBjf9N+6mj7ypeh9I2rUeqUvyDCeQyDagUj/Kh6xavdzeadauI5yBJlPnfl5tuE LzIC/LZqDX+3ojv62PMkhphTA7kIAys/o514cUyOJ4om87XCZdcT1SdVnqJtj2EXu3YS NSjdAphbEaSdEFII6QZxfa7Fj0bjSYqFA5p6PLmzjpY+msR02xFgp0u68w8LCSjBvNc3 wYhLYlGn7cr/BuiUlvZyf9ZM921hZhchgNIw8vpCQ0PJmob64gtiC65rqvr/rUsMygOM rLcTyWxf7+sjIepJLd47f05S+p1pfP0NH3OmNHNO0igRcogWbJKr84SURVKfspza9EnE mTBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5sdtls0qkuJiYusI35JI3atU6pS13AMJMmDf+mNHOnk=; b=YVntO3gW8Ycowf9SdBW0bMIDkuAStLKASNFSrAS+K7iFyhF70qxGnlwuxh2AdKM3/A t5fy4+y1hjhvyJvXTR81jeT4q1+zfbpGa+MpqYHQ7RfAjEtfgqPZ3PY00zH57s6w4RCt JYxhK51uVIsslYxWL7/mWQNMMjUrTeNt1HLIhnclSfe0xoPiLxEPEGcFpR2Fn+XB9rny elSROaqMv+rxcoDSsqmPIHZIj10XNEPxWPcSokTutsKhzwYAkgMU8CKBI/ivcrPhTcDE NiHJ+p8+H5j9K7xvAAXJKM0/mZKeyNVpRrc5CoDR0oOzORFHv+0k2WKQHzqIo3rZe9Je LInA== X-Gm-Message-State: AOAM530suRT98TSUdRcfi9LnxfZhrMgbiUvljEcvaPORsM8DsIyg418S bTYfYubsTzcDSuQgNTrcP+h3kGfgK+s= X-Google-Smtp-Source: ABdhPJw1w6c2KByvA4cu97EP9E21zcXtIDgV4J3yn5Pf/NnGMOmObmoTEk2qSrE5PifhjZmnLsM12A== X-Received: by 2002:a5d:6da2:: with SMTP id u2mr9461102wrs.529.1639671225789; Thu, 16 Dec 2021 08:13:45 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k13sm5123790wri.6.2021.12.16.08.13.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Dec 2021 08:13:45 -0800 (PST) Message-Id: <3fd625290a318f853c7a96cbc10be76308ef2a3d.1639671222.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 16 Dec 2021 16:13:41 +0000 Subject: [PATCH v4 2/3] sparse-checkout: fix OOM error with mixed patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Add a test to t1091-sparse-checkout-builtin.sh that would result in an infinite loop and out-of-memory error before this change. The issue relies on having non-cone-mode patterns while trying to modify the patterns in cone-mode. The fix is simple, allowing us to break from the loop when the input path does not contain a slash, as the "dir" pattern we added does not. This is only a fix to the critical out-of-memory error. A better response to such a strange state will follow in a later change. Reported-by: Calbabreaker Helped-by: Taylor Blau Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 2 +- t/t1091-sparse-checkout-builtin.sh | 11 +++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index d0f5c4702be..9ccdcde9832 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -483,7 +483,7 @@ static void insert_recursive_pattern(struct pattern_list *pl, struct strbuf *pat char *oldpattern = e->pattern; size_t newlen; - if (slash == e->pattern) + if (!slash || slash == e->pattern) break; newlen = slash - e->pattern; diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 3921ea80138..1f877ced0c8 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -103,6 +103,17 @@ test_expect_success 'clone --sparse' ' check_files clone a ' +test_expect_success 'switching to cone mode with non-cone mode patterns' ' + git init bad-patterns && + ( + cd bad-patterns && + git sparse-checkout init && + git sparse-checkout add dir && + git config core.sparseCheckoutCone true && + git sparse-checkout add dir + ) +' + test_expect_success 'interaction with clone --no-checkout (unborn index)' ' git clone --no-checkout "file://$(pwd)/repo" clone_no_checkout && git -C clone_no_checkout sparse-checkout init --cone && From patchwork Thu Dec 16 16:13:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12681645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4134C433FE for ; Thu, 16 Dec 2021 16:13:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239163AbhLPQNw (ORCPT ); Thu, 16 Dec 2021 11:13:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239093AbhLPQNs (ORCPT ); Thu, 16 Dec 2021 11:13:48 -0500 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21F34C06173F for ; Thu, 16 Dec 2021 08:13:48 -0800 (PST) Received: by mail-wr1-x42c.google.com with SMTP id a9so45106150wrr.8 for ; Thu, 16 Dec 2021 08:13:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=BetIyBobJRBgFMtcCsw2ibE8x45ldtmHtv7K3P7JMaY=; b=YJP0MDg+MBVLOgpf+9Yl5GXP0FYZ9z0LS5OmWYI110tQeS8LR0a5qdRppIa1fjGYRT IKqS4+FU5bosj1JSs/MHhk2ZXGhM8puM/0RdoK8DIwi5OvWFpFfXqaSEltyhDouz5nBN z3h4Pgmes2MoAn8ftv+rIC48HFOIfV1UN2NRcxo5nhmKcPADb0lYuxt5RFWdGbAHL4Qa ygVD3oCe7uPEQXdHFUKFwBVLtQGHXLzL38iSerhPQhVbH3ypR7Zh+yshUKBrp66NSVHe oSUA2/V5p5Lerbz7Q7+3/ong+ZdxkIMPU/lzZ2e29AFCFqVcP9CmrCOMOt8bWAn1xiL5 jesA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=BetIyBobJRBgFMtcCsw2ibE8x45ldtmHtv7K3P7JMaY=; b=5gSNnuwP0eT8lhJCZ3KeJsoEyYVr2V5b3XnV6xFwzmtgwH6MvQtE/p76GhaXY6BelN nXzo6fW9fYqBJiLhVgDntsMwOCZo27R/GNv4N9vFHM1k3H4J5tO2UpqjZhaCQov/jxmZ hZ1Qwl9NMUJou4cyyge+DzUeNW/auhu0yACTnPdsrq8YjM5pixVpQe5aRL+atYwave7z t04RqCrb1XYdveXY4TY2wx+rXEiZzj6T/ytMvG7bSmkQDa8isxrBU/nlu9CK2zQafsr4 2n2F9g+cqrNqW6FBSzP0IZ6RoMg9SRG+3rmzK7CTI/fg/R+GuN8++dTdFTbosyuhy1ln s6Qg== X-Gm-Message-State: AOAM533T6ktIsGI8KFZ2s6i0lNiCVpnilyVwv8PnbOxq2zDO0fEcjwT7 HU/Nd6Bzl9SPW+gllEqwm0sjA9MvZxw= X-Google-Smtp-Source: ABdhPJzPBf74/X+B5DiRDpa7voqyV5ZgWlLFt9O5gLMqPI/Op6EdtKL20lbHncqEUBRreC+XUVFPAw== X-Received: by 2002:adf:e4ce:: with SMTP id v14mr1318113wrm.370.1639671226568; Thu, 16 Dec 2021 08:13:46 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d1sm4996108wrz.92.2021.12.16.08.13.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Dec 2021 08:13:46 -0800 (PST) Message-Id: In-Reply-To: References: Date: Thu, 16 Dec 2021 16:13:42 +0000 Subject: [PATCH v4 3/3] sparse-checkout: refuse to add to bad patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When in cone mode sparse-checkout, it is unclear how 'git sparse-checkout add ...' should behave if the existing sparse-checkout file does not match the cone mode patterns. Change the behavior to fail with an error message about the existing patterns. Also, all cone mode patterns start with a '/' character, so add that restriction. This is necessary for our example test 'cone mode: warn on bad pattern', but also requires modifying the example sparse-checkout file we use to test the warnings related to recognizing cone mode patterns. This error checking would cause a failure further down the test script because of a test that adds non-cone mode patterns without cleaning them up. Perform that cleanup as part of the test now. Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 3 +++ dir.c | 2 +- t/t1091-sparse-checkout-builtin.sh | 7 +++++-- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 9ccdcde9832..6580075a631 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -598,6 +598,9 @@ static void add_patterns_cone_mode(int argc, const char **argv, die(_("unable to load existing sparse-checkout patterns")); free(sparse_filename); + if (!existing.use_cone_patterns) + die(_("existing sparse-checkout patterns do not use cone mode")); + hashmap_for_each_entry(&existing.recursive_hashmap, &iter, pe, ent) { if (!hashmap_contains_parent(&pl->recursive_hashmap, pe->pattern, &buffer) || diff --git a/dir.c b/dir.c index 7e72958d51d..eca75e89213 100644 --- a/dir.c +++ b/dir.c @@ -727,7 +727,7 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern } if (given->patternlen < 2 || - *given->pattern == '*' || + *given->pattern != '/' || strstr(given->pattern, "**")) { /* Not a cone pattern. */ warning(_("unrecognized pattern: '%s'"), given->pattern); diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 1f877ced0c8..34b97fd3ee0 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -110,7 +110,8 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' ' git sparse-checkout init && git sparse-checkout add dir && git config core.sparseCheckoutCone true && - git sparse-checkout add dir + test_must_fail git sparse-checkout add dir 2>err && + grep "existing sparse-checkout patterns do not use cone mode" err ) ' @@ -176,12 +177,14 @@ test_expect_success 'set sparse-checkout using --stdin' ' ' test_expect_success 'add to sparse-checkout' ' - cat repo/.git/info/sparse-checkout >expect && + cat repo/.git/info/sparse-checkout >old && + test_when_finished cp old repo/.git/info/sparse-checkout && cat >add <<-\EOF && pattern1 /folder1/ pattern2 EOF + cat old >expect && cat add >>expect && git -C repo sparse-checkout add --stdin actual &&