From patchwork Fri Dec 10 15:18:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12669765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7683BC433F5 for ; Fri, 10 Dec 2021 15:18:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242741AbhLJPV4 (ORCPT ); Fri, 10 Dec 2021 10:21:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242752AbhLJPVt (ORCPT ); Fri, 10 Dec 2021 10:21:49 -0500 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B77B9C061746 for ; Fri, 10 Dec 2021 07:18:13 -0800 (PST) Received: by mail-wm1-x329.google.com with SMTP id 137so7078149wma.1 for ; Fri, 10 Dec 2021 07:18:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=cVnWEd0GVO9mjLJ3Yu+vv33DBI83fSFBcOn2vfL69ZY=; b=Ug8JSuSLBZYcf+oXFDK6HzjQaf8ICoOcn7XSv6ke9pUB26OH/B0ooOHLEcu2YmW3CO +jg8FyYWAnhBGYJNj3WBdEgcnR0EOUzuwYKbGDQ1y78M0ZjunfP8cOoBTExnwo1rEEti BJGa1tjCTYm/c7x6JGrmw2i5xKcux6/MGOZGsHc7KEskiVzdvJ/PAWP7vn/nLhhgW9jS K5U3qwUeOmV7yUmXL0kDT0MXUya3qh+6qsMPlZcuH0JPGVfMN8rYP8xlbJPab9IKEJPs ilzegl8KqY2yXsodguNR//TE/knEofS/xjP0+G2FwgVa/KR19l5xI1J7HnRSF5YGONny TpBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=cVnWEd0GVO9mjLJ3Yu+vv33DBI83fSFBcOn2vfL69ZY=; b=KorNXYvmy4tS9IQ0fkvh51hrfMlHqpB6QlNpv6igEx8GYM9UNMyDjHczI4fxJi1ZGE bJeTefIYc+QJ9qziq7Tm8bgxjXUDCF6MrSNmC49zXx54NXvOvfQjga+uSxOWvOUSKpKH lY6Wsn0ido4286jwkmwSUy+SBoeWCitk3ls5jhv6NoAVYyuLmqxmwwmrOamp1mYqRiKI v1KpQwQXcBtvn39DVe4pl5Z1PbDM3LlHaAkVV+NzNQLMQy0tjOcoQ74OsGrR4vP/HXdu dWKpCxtFKeuudCdJEkLCk9OLIECRgV7jWaVnr5m1lAZgNXHe+JwExNJFKX6ACcnpmb5e 793A== X-Gm-Message-State: AOAM533VwHSYVP/la0Zfi/irn3jQNo4oCZp92Bf/1jKtbTB99BDL/sFQ KQq86iCjGfF0DYWv1ocfvL/1MaXcFRU= X-Google-Smtp-Source: ABdhPJysug577KSBVF9cb8Ry+HUcah1bpUHRmBJd0kaYONcT59dFCjuGs/rMqmUA3SoHtu1d8dj3qQ== X-Received: by 2002:a1c:8:: with SMTP id 8mr17019939wma.106.1639149492040; Fri, 10 Dec 2021 07:18:12 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v2sm2943921wmc.36.2021.12.10.07.18.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 07:18:11 -0800 (PST) Message-Id: In-Reply-To: References: Date: Fri, 10 Dec 2021 15:18:08 +0000 Subject: [PATCH v2 1/4] sparse-checkout: fix segfault on malformed patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are used to populate two hashsets that accelerate pattern matching. If the user modifies the sparse-checkout file outside of the 'sparse-checkout' builtin, then strange patterns can happen, triggering some error checks. One of these error checks is possible to hit when some special characters exist in a line. A warning message is correctly written to stderr, but then there is additional logic that attempts to remove the line from the hashset and free the data. This leads to a segfault in the 'git sparse-checkout list' command because it iterates over the contents of the hashset, which is no invalid. The fix here is to stop trying to remove from the hashset. Better to leave bad data in the sparse-checkout matching logic (with a warning) than to segfault. If we are in this state, then we are already traversing into undefined behavior, so this change to keep the entry in the hashset is no worse than removing it. Add a test that triggers the segfault without the code change. Reported-by: John Burnett Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee --- dir.c | 3 --- t/t1091-sparse-checkout-builtin.sh | 15 +++++++++++++++ 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/dir.c b/dir.c index 5aa6fbad0b7..0693c7cb3ee 100644 --- a/dir.c +++ b/dir.c @@ -819,9 +819,6 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern /* we already included this at the parent level */ warning(_("your sparse-checkout file may have issues: pattern '%s' is repeated"), given->pattern); - hashmap_remove(&pl->parent_hashmap, &translated->ent, &data); - free(data); - free(translated); } return; diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 272ba1b566b..c72b8ee2e7b 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -708,4 +708,19 @@ test_expect_success 'cone mode clears ignored subdirectories' ' test_cmp expect out ' +test_expect_success 'malformed cone-mode patterns' ' + git -C repo sparse-checkout init --cone && + mkdir -p repo/foo/bar && + touch repo/foo/bar/x repo/foo/y && + cat >repo/.git/info/sparse-checkout <<-\EOF && + /* + !/*/ + /foo/ + !/foo/*/ + /foo/\*/ + EOF + cat repo/.git/info/sparse-checkout && + git -C repo sparse-checkout list +' + test_done From patchwork Fri Dec 10 15:18:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12669763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F3C8C433EF for ; Fri, 10 Dec 2021 15:18:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242776AbhLJPVz (ORCPT ); Fri, 10 Dec 2021 10:21:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242756AbhLJPVt (ORCPT ); Fri, 10 Dec 2021 10:21:49 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B7FFC0617A2 for ; Fri, 10 Dec 2021 07:18:14 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id v11so15441214wrw.10 for ; Fri, 10 Dec 2021 07:18:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=N5DKjb3WTYvXkZ27+dhQuJ7yDgaXYUlIgn/9ox8Zc5Y=; b=b8PsBAUM4OlhrQVAvDOy0znGMALe1DXStv6G/JmPuu5ZRtosIiSuJHMS/vL8l/iLkD HwLoOiuEr0iAF1eXCmvfcI0NaKf+a4w1xYaUOIrTODjhfKJUCBY4sMv0Y90v9Xlt2PRn Ewqiz+zGGPO5Qu3atyxcxA62bjLU93g4LWRDAxbDbsmXGVkoOyfRJDm3SYmqNYnUBNMK WyHQ4tTGSM4oUimuxi1mKm4b0rd0T1JADU55MsqnV7mqao8tyquITwoiXD1D0I4q9M8J BrpO3Lbw4J/NUMHDfowOQgc8rmmdg9MiBpgnVxWMWbZz8GWc0xGr5fPt8eK5utDMjad2 NTLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=N5DKjb3WTYvXkZ27+dhQuJ7yDgaXYUlIgn/9ox8Zc5Y=; b=lUlM9G6Srg4pnFbhfCiFS1fWFANWQpQaxNvWSIioxwcpOUydPxD04LLWZ9HLrV+ZCY SgL9JKxZfmlPXeoyCDlxcCTP4nbgWtUs6dp1Ma8ZEQr+X/xgTkufKtXqHLcAZZNfAX/4 0LU09JPNxOLl4i+5pGZ2YJX9hSHJU+TQZ763CQwqhEciQphqK+HN1YPVc+Zkr9lyEUQZ YB7Ff/ebE5LjHvYBGSdZNyQDddn9ubCQc/RCbx0Op3SaCjTVtweGWKyKibUNZrB/h1O9 LoNpg7Yhxrju0PmSWJu5O8CS9rSst4NR7S9g+u1vXBFN+t6GGyDbRhx4pDsBg98h/CwL Y33Q== X-Gm-Message-State: AOAM533aMC9hEYd0RiHt5+y8H8hWLnVL5HQgjySwBN59vlMUufp7wEy4 YIRgLXvtg1IurPf2Lxv+9CP5ZmqMK4k= X-Google-Smtp-Source: ABdhPJznHQEIIkZACikCBZBMpde6k7Y/uPS5MKk6ipgp1twp+I+dYetWcNf8IfDJru12m4vmvAN1pg== X-Received: by 2002:adf:efc6:: with SMTP id i6mr14315413wrp.428.1639149492958; Fri, 10 Dec 2021 07:18:12 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m21sm2763081wrb.2.2021.12.10.07.18.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 07:18:12 -0800 (PST) Message-Id: <86fbf130c039b2c087974b213e12919f7809e05b.1639149490.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 10 Dec 2021 15:18:09 +0000 Subject: [PATCH v2 2/4] sparse-checkout: fix OOM error with mixed patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Add a test to t1091-sparse-checkout-builtin.sh that would result in an infinite loop and out-of-memory error before this change. The issue relies on having non-cone-mode patterns while trying to modify the patterns in cone-mode. The fix is simple, allowing us to break from the loop when the input path does not contain a slash, as the "dir" pattern we added does not. This is only a fix to the critical out-of-memory error. A better response to such a strange state will follow in a later change. Reported-by: Calbabreaker Helped-by: Taylor Blau Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 2 +- t/t1091-sparse-checkout-builtin.sh | 11 +++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index d0f5c4702be..9ccdcde9832 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -483,7 +483,7 @@ static void insert_recursive_pattern(struct pattern_list *pl, struct strbuf *pat char *oldpattern = e->pattern; size_t newlen; - if (slash == e->pattern) + if (!slash || slash == e->pattern) break; newlen = slash - e->pattern; diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index c72b8ee2e7b..67ce24c9c83 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -103,6 +103,17 @@ test_expect_success 'clone --sparse' ' check_files clone a ' +test_expect_success 'switching to cone mode with non-cone mode patterns' ' + git init bad-patterns && + ( + cd bad-patterns && + git sparse-checkout init && + git sparse-checkout add dir && + git config core.sparseCheckoutCone true && + git sparse-checkout add dir + ) +' + test_expect_success 'interaction with clone --no-checkout (unborn index)' ' git clone --no-checkout "file://$(pwd)/repo" clone_no_checkout && git -C clone_no_checkout sparse-checkout init --cone && From patchwork Fri Dec 10 15:18:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12669767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FA5DC433EF for ; Fri, 10 Dec 2021 15:18:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242752AbhLJPV5 (ORCPT ); Fri, 10 Dec 2021 10:21:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39160 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242761AbhLJPVv (ORCPT ); Fri, 10 Dec 2021 10:21:51 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 720B1C061746 for ; Fri, 10 Dec 2021 07:18:15 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id j3so15531802wrp.1 for ; Fri, 10 Dec 2021 07:18:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=hefOraxaOEQ0KhvCVtqg70N18D185JeHbaisQYJ4NRA=; b=GNj8FU3XnqeFnnhxgstKJ9v5RJHW5w1hjyMTLhjwWTgNwMUdXdrL0gG3LCyYXT6U9R RRTa1Sy7DuzZJglIL0Cx1taWkst0Lih58QXzViLtYnGGg2To1GqS0d7kTonLTPbj+zMK XyvDtdPj+VxQ0BT66YsB26JMk029YhZdrAXcUa3VT4l8opkkk2uuGsc5RUTXrbpXeYUK Jjc9n3zTO8wvbwGvgnp9bM8O8pCpt1va6cavwiyRGT+1WPpTmbgTJb2watCTZbSF/Njt c3zDhCZ+XwL7g4twq64hIDv84hEx/A3w1SQuXWfX347ek/btZaZTRaaxVB9bCT9eDM1p /CUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=hefOraxaOEQ0KhvCVtqg70N18D185JeHbaisQYJ4NRA=; b=4R27URDm59JfQ6uFEng8rfF5aib4EkbKwTu3IiGxdFkvsd3eplZ4FxaAPLAmD2Zkzd 4UM9XaniqLqblUY54Ji9IGj806IXzsMy8KKa/GSdOhTpv2yOBlRz9ZJ3iCe70YTRoJ9i up0Of+UxqNY9zuutwmCAl3fa/s45k7l//BqvJgVjBgkTRq+a55NuXS/8mh4UwJgXRmA5 1TYMfnU+nOG6ceeZ/lxnpk25zfo0q4HSljOKyl/Ncyj1g+vPJ2fQSYm5CwaEzPj0oQoq aaMbWQomxiNhTKXOYQjXfY00D3+I+SiU+SmOw2kB6U4LcIqEdUOHGKrKqIp0VHldl4/u N4Kw== X-Gm-Message-State: AOAM531HA8ye6iuI1v3X0gYyXaUEDDvpFQS2XJnyRG5I72I7G8vr28ff N/ji6YX/7E5gsIMUs84f+WuuXuDEkqs= X-Google-Smtp-Source: ABdhPJxtmgtYU31sCrGATx3QQR23udrjkRI1qKd+a2YdAmtpBjj86ErcdBPyJ95cVy+vhWCiTJkv4A== X-Received: by 2002:adf:fbd0:: with SMTP id d16mr15223689wrs.107.1639149493669; Fri, 10 Dec 2021 07:18:13 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e24sm2894005wra.78.2021.12.10.07.18.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 07:18:13 -0800 (PST) Message-Id: <5d096e380a400759f3915568228fa703232cdd18.1639149490.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 10 Dec 2021 15:18:10 +0000 Subject: [PATCH v2 3/4] sparse-checkout: refuse to add to bad patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When in cone mode sparse-checkout, it is unclear how 'git sparse-checkout add ...' should behave if the existing sparse-checkout file does not match the cone mode patterns. Change the behavior to fail with an error message about the existing patterns. Also, all cone mode patterns start with a '/' character, so add that restriction. This is necessary for our example test 'cone mode: warn on bad pattern', but also requires modifying the example sparse-checkout file we use to test the warnings related to recognizing cone mode patterns. This error checking would cause a failure further down the test script because of a test that adds non-cone mode patterns without cleaning them up. Perform that cleanup as part of the test now. Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 3 +++ dir.c | 2 +- t/t1091-sparse-checkout-builtin.sh | 7 +++++-- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 9ccdcde9832..6580075a631 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -598,6 +598,9 @@ static void add_patterns_cone_mode(int argc, const char **argv, die(_("unable to load existing sparse-checkout patterns")); free(sparse_filename); + if (!existing.use_cone_patterns) + die(_("existing sparse-checkout patterns do not use cone mode")); + hashmap_for_each_entry(&existing.recursive_hashmap, &iter, pe, ent) { if (!hashmap_contains_parent(&pl->recursive_hashmap, pe->pattern, &buffer) || diff --git a/dir.c b/dir.c index 0693c7cb3ee..a5dddafa16d 100644 --- a/dir.c +++ b/dir.c @@ -727,7 +727,7 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern } if (given->patternlen < 2 || - *given->pattern == '*' || + *given->pattern != '/' || strstr(given->pattern, "**")) { /* Not a cone pattern. */ warning(_("unrecognized pattern: '%s'"), given->pattern); diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 67ce24c9c83..2ed247d75a5 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -110,7 +110,8 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' ' git sparse-checkout init && git sparse-checkout add dir && git config core.sparseCheckoutCone true && - git sparse-checkout add dir + test_must_fail git sparse-checkout add dir 2>err && + grep "existing sparse-checkout patterns do not use cone mode" err ) ' @@ -176,12 +177,14 @@ test_expect_success 'set sparse-checkout using --stdin' ' ' test_expect_success 'add to sparse-checkout' ' - cat repo/.git/info/sparse-checkout >expect && + cat repo/.git/info/sparse-checkout >old && + test_when_finished cp old repo/.git/info/sparse-checkout && cat >add <<-\EOF && pattern1 /folder1/ pattern2 EOF + cat old >expect && cat add >>expect && git -C repo sparse-checkout add --stdin actual &&