From patchwork Wed Dec 15 13:46:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12678347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF599C433EF for ; Wed, 15 Dec 2021 13:46:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237453AbhLONqM (ORCPT ); Wed, 15 Dec 2021 08:46:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234455AbhLONqM (ORCPT ); Wed, 15 Dec 2021 08:46:12 -0500 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6FADC061574 for ; Wed, 15 Dec 2021 05:46:11 -0800 (PST) Received: by mail-wr1-x42a.google.com with SMTP id t9so38304626wrx.7 for ; Wed, 15 Dec 2021 05:46:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=bkM55JutuPy5Bj2XQ92/MSM2wQIb24IQFA9MxfrcIXc=; b=hOn0B/1ypv79/faXeClZWwx6S5ewgNWC8yBwM9/xZjeA/D4W4ciO9BZqMh9pIMT+a4 YOXKg/JQmeZmDIbLk+SCOP3fS6/eH/bMHafx/gv5cusswQhXnpoKZMQzpOM+EQDPHKYg ECVyOnHgcpBnLsylfDo+KGY+oAULG8h9mAwwWg8GD74/ltxy8UO3dvjOz+//qqLKYN1J sc66T9Y9LlopA7Rsuw+Bw30vJYxHiNvqtdHyDvRsJ2rfE1GgAuV5wGK/bTZk/cZLyayL x8mBcXivStYz9MJIjYj/pntdWrgsF73rFaJWpYgfl9IFDwFnQBEIP6s1R/ujUOUBTR9P q/iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=bkM55JutuPy5Bj2XQ92/MSM2wQIb24IQFA9MxfrcIXc=; b=DRKkYNI+FQSnhGqKzlDofgadfaSEEF6GB7WcfyXWYKxYw/C05s2QVvwafykTZiyZBI K0koTXOEn+YJVNoO6pJoRuT027yidfJsJlpQKu5TOKL61ZQaIYYwuITaY0ooLvgAh/ju KhS5uGL3ldnhdnZU0+DHRa4UoAxQZXZW6BFbqbh9O46KnmXOIMj+HMWfoZm6a0eWzxuA py+79iRO/PFJSlTjrN++hfpjGzEToHz3GOMJx0M0+NKEHSqIpfo5GfZ7SLBV3qXynWvx jpORlSvnvm74Z79xmQez1OwvAUjcPwbACAy0VsvqiOonwHuObpjhx6EC80YAYfa/Y9kB eTtg== X-Gm-Message-State: AOAM532mFqQ65UynGU0D/AICrfKHFowOeLes420TY1ojG1KzGyUPt3ge 4iwOcD4AfIPS5lE+VyIH/s6pUVrEt1QIYw== X-Google-Smtp-Source: ABdhPJwKZGz6xz/Z8aiOqoV05GqHpBfrhik2ESOqtOUQRGykFR6qUvEE5vazor4CloIBFSKJtumb7A== X-Received: by 2002:a5d:630b:: with SMTP id i11mr4397607wru.316.1639575970081; Wed, 15 Dec 2021 05:46:10 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k7sm2113018wrg.105.2021.12.15.05.46.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Dec 2021 05:46:09 -0800 (PST) Message-Id: <1744a26845fbe4d7dbc80f387be1d842b5f8fe94.1639575968.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 15 Dec 2021 13:46:06 +0000 Subject: [PATCH v3 1/3] sparse-checkout: fix segfault on malformed patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are used to populate two hashsets that accelerate pattern matching. If the user modifies the sparse-checkout file outside of the 'sparse-checkout' builtin, then strange patterns can happen, triggering some error checks. One of these error checks is possible to hit when some special characters exist in a line. A warning message is correctly written to stderr, but then there is additional logic that attempts to remove the line from the hashset and free the data. This leads to a segfault in the 'git sparse-checkout list' command because it iterates over the contents of the hashset, which is now invalid. The fix here is to stop trying to remove from the hashset. Better to leave bad data in the sparse-checkout matching logic (with a warning) than to segfault. If we are in this state, then we are already traversing into undefined behavior, so this change to keep the entry in the hashset is no worse than removing it. Add a test that triggers the segfault without the code change. Reported-by: John Burnett Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee --- dir.c | 3 --- t/t1091-sparse-checkout-builtin.sh | 15 +++++++++++++++ 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/dir.c b/dir.c index 5aa6fbad0b7..0693c7cb3ee 100644 --- a/dir.c +++ b/dir.c @@ -819,9 +819,6 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern /* we already included this at the parent level */ warning(_("your sparse-checkout file may have issues: pattern '%s' is repeated"), given->pattern); - hashmap_remove(&pl->parent_hashmap, &translated->ent, &data); - free(data); - free(translated); } return; diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 272ba1b566b..c72b8ee2e7b 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -708,4 +708,19 @@ test_expect_success 'cone mode clears ignored subdirectories' ' test_cmp expect out ' +test_expect_success 'malformed cone-mode patterns' ' + git -C repo sparse-checkout init --cone && + mkdir -p repo/foo/bar && + touch repo/foo/bar/x repo/foo/y && + cat >repo/.git/info/sparse-checkout <<-\EOF && + /* + !/*/ + /foo/ + !/foo/*/ + /foo/\*/ + EOF + cat repo/.git/info/sparse-checkout && + git -C repo sparse-checkout list +' + test_done From patchwork Wed Dec 15 13:46:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12678349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B6F3C4332F for ; Wed, 15 Dec 2021 13:46:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237469AbhLONqO (ORCPT ); Wed, 15 Dec 2021 08:46:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237461AbhLONqN (ORCPT ); Wed, 15 Dec 2021 08:46:13 -0500 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68BA2C061574 for ; Wed, 15 Dec 2021 05:46:12 -0800 (PST) Received: by mail-wm1-x331.google.com with SMTP id z4-20020a1c7e04000000b0032fb900951eso1478182wmc.4 for ; Wed, 15 Dec 2021 05:46:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=N5DKjb3WTYvXkZ27+dhQuJ7yDgaXYUlIgn/9ox8Zc5Y=; b=lj5X/XPO9p6NWcvaqFVPDrIED+TMklorIyczz7eTc/QPmPp56pUsmK8ghi4Dx9UJl6 o/JS5zmraBA5E+YF/tsyPMdba0gW1+goEbvnihs3SvrEV8Qiy5+HYUDu6GamM7FH/ZmF GRbb7LXgg0TKWfmwv2RVgF7aXPJ3J9d//LKT6w6f1SHKV1DP4sIyvOwFz3FAZp4qHnmy opHem7xskxd91Fdb2QAOIBapcOero5/Ph1KEx71U5tqeiOkSTzMqYMlF5dSMg6Q8pZFT sH802vAVRRPKXl7lNxwtXC+uPU9v7sTpsexk0tiVPMtSciBB4Xca6uSrJpIiMSfuwxbk zWYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=N5DKjb3WTYvXkZ27+dhQuJ7yDgaXYUlIgn/9ox8Zc5Y=; b=jiKlare30E4/ahIzeaiOgmftvG9USI57yWYKrk5Xf14h+ui01tM0dl+MD/qktfrTmv 2cIX0qtvoRjE39VgOPYD2Q2UDPnsz8XGv9auVty/KskqujUTd1SXZZ6OB+e+J4DlcBAa 4AQqtFNPSGPHZupJNVo+i3aZrBbY2N9IAFTgwrXoIxd5HKStBxJSByvxXH7sNzZtvS14 1OAWCY0hTZboi4cVobK9BQ8Z/KBiOGhPU3iayMKn6N4VEiwnFdXpgtdwfM19viypYeT5 oSLJDsgIAdKHy6MC9u9tcLigpbNP0JYyvHcsdRtAaDXehH5YDkZ4m0my+/NbiN191Jo8 70RA== X-Gm-Message-State: AOAM532cYIX75wGahnjjMnplQzOdqfJ3u6Qi3GVC5NoCGxiO4LofRiKH Oz1IFvL1IAcfPQIxXutxWVDlnec+EfH35g== X-Google-Smtp-Source: ABdhPJykeLNIDdOVOgWM2TUHPGlnDj6aio3e3zh63pzW4vuITzIt6Z0mTD//CToKLtdOonmIfXu/cw== X-Received: by 2002:a1c:9a89:: with SMTP id c131mr4574481wme.80.1639575970876; Wed, 15 Dec 2021 05:46:10 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r20sm1108745wmd.37.2021.12.15.05.46.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Dec 2021 05:46:10 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 15 Dec 2021 13:46:07 +0000 Subject: [PATCH v3 2/3] sparse-checkout: fix OOM error with mixed patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Add a test to t1091-sparse-checkout-builtin.sh that would result in an infinite loop and out-of-memory error before this change. The issue relies on having non-cone-mode patterns while trying to modify the patterns in cone-mode. The fix is simple, allowing us to break from the loop when the input path does not contain a slash, as the "dir" pattern we added does not. This is only a fix to the critical out-of-memory error. A better response to such a strange state will follow in a later change. Reported-by: Calbabreaker Helped-by: Taylor Blau Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 2 +- t/t1091-sparse-checkout-builtin.sh | 11 +++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index d0f5c4702be..9ccdcde9832 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -483,7 +483,7 @@ static void insert_recursive_pattern(struct pattern_list *pl, struct strbuf *pat char *oldpattern = e->pattern; size_t newlen; - if (slash == e->pattern) + if (!slash || slash == e->pattern) break; newlen = slash - e->pattern; diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index c72b8ee2e7b..67ce24c9c83 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -103,6 +103,17 @@ test_expect_success 'clone --sparse' ' check_files clone a ' +test_expect_success 'switching to cone mode with non-cone mode patterns' ' + git init bad-patterns && + ( + cd bad-patterns && + git sparse-checkout init && + git sparse-checkout add dir && + git config core.sparseCheckoutCone true && + git sparse-checkout add dir + ) +' + test_expect_success 'interaction with clone --no-checkout (unborn index)' ' git clone --no-checkout "file://$(pwd)/repo" clone_no_checkout && git -C clone_no_checkout sparse-checkout init --cone && From patchwork Wed Dec 15 13:46:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12678351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A5FCC433F5 for ; Wed, 15 Dec 2021 13:46:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237461AbhLONqQ (ORCPT ); Wed, 15 Dec 2021 08:46:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40158 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237473AbhLONqO (ORCPT ); Wed, 15 Dec 2021 08:46:14 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53FC8C06173E for ; Wed, 15 Dec 2021 05:46:13 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id p3-20020a05600c1d8300b003334fab53afso18527077wms.3 for ; Wed, 15 Dec 2021 05:46:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=hefOraxaOEQ0KhvCVtqg70N18D185JeHbaisQYJ4NRA=; b=kSAkbW2UC7J7WGVxWkiDuza21SpHs3QSmPbDCiQ0Mmflkfxu67K70Iod/BJa/sd8W9 GJZsYlvhmvJFFK41t2DGgYKOalS9OrFztD8LvMTXD99Fq7ehm5nUlI/PdBL8SqmbbM74 UlfozV9Gr1q6OBhaDx1A2eBqmKThUX6GULKLCUHH2C5pANODvjXGiMNJTc3bB7Kxsr8d 8hjTHEptqe30MWsLZZNmSprtvKeooGmvVlyUePqzzAc5jZfLQVHZctkhr4/0reStaM4U +23k83S65qzNVg/L/eRr8SJLTty4ARMsu/rjA/4/UKUPFI4u9YqqM7k0oI45MyRaKf0E gKcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=hefOraxaOEQ0KhvCVtqg70N18D185JeHbaisQYJ4NRA=; b=3KQKX16OpR7ASf4kKx2d5lm0JVsGibldsTc0XyFam22MvSyF2Sq5DDhzkst6qhkZ/j /rQxpmmRAPBC4iN74OW7tZkhyH1zGSgbnZ4gO5hYVc59q94Mm0IrAOtsrJhZTXNzE+r/ uWQMiu+WZFh2SRMX8HAp+fsFXEl3gxUcqYCCMC5eyjYfqlt5IrtyG14/cBVdcquPADyu r5etkm1D6O+E/oSGh9BM7TLlcL3A/ety1UQYgTEs6zxzV3mFAe9On9ythr++MffFp7oU +gtt0m7BAHPu2kaP+ESJeauK1tPa2Ci2YPAfMDwpPqdzO4IHoHtdgM9xFraLaaM20aaq 5gzQ== X-Gm-Message-State: AOAM532s9GOBd3P1RqUdNdFLnkyeBD032g4qr7md1qpe3E7ZtKagiF9F ab9g6L90Tx/NYm2OZUfo6MBUvSmi5nmmJw== X-Google-Smtp-Source: ABdhPJzXscysyx2Dtp2OobubOTsSx8ATpw1e2muPcDOogBI/j//9lQfGK1Bc4nCccVLGMUkG9KY6SA== X-Received: by 2002:a1c:4d0b:: with SMTP id o11mr4623532wmh.68.1639575971789; Wed, 15 Dec 2021 05:46:11 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y11sm2655249wry.70.2021.12.15.05.46.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Dec 2021 05:46:11 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 15 Dec 2021 13:46:08 +0000 Subject: [PATCH v3 3/3] sparse-checkout: refuse to add to bad patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When in cone mode sparse-checkout, it is unclear how 'git sparse-checkout add ...' should behave if the existing sparse-checkout file does not match the cone mode patterns. Change the behavior to fail with an error message about the existing patterns. Also, all cone mode patterns start with a '/' character, so add that restriction. This is necessary for our example test 'cone mode: warn on bad pattern', but also requires modifying the example sparse-checkout file we use to test the warnings related to recognizing cone mode patterns. This error checking would cause a failure further down the test script because of a test that adds non-cone mode patterns without cleaning them up. Perform that cleanup as part of the test now. Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 3 +++ dir.c | 2 +- t/t1091-sparse-checkout-builtin.sh | 7 +++++-- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 9ccdcde9832..6580075a631 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -598,6 +598,9 @@ static void add_patterns_cone_mode(int argc, const char **argv, die(_("unable to load existing sparse-checkout patterns")); free(sparse_filename); + if (!existing.use_cone_patterns) + die(_("existing sparse-checkout patterns do not use cone mode")); + hashmap_for_each_entry(&existing.recursive_hashmap, &iter, pe, ent) { if (!hashmap_contains_parent(&pl->recursive_hashmap, pe->pattern, &buffer) || diff --git a/dir.c b/dir.c index 0693c7cb3ee..a5dddafa16d 100644 --- a/dir.c +++ b/dir.c @@ -727,7 +727,7 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern } if (given->patternlen < 2 || - *given->pattern == '*' || + *given->pattern != '/' || strstr(given->pattern, "**")) { /* Not a cone pattern. */ warning(_("unrecognized pattern: '%s'"), given->pattern); diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 67ce24c9c83..2ed247d75a5 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -110,7 +110,8 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' ' git sparse-checkout init && git sparse-checkout add dir && git config core.sparseCheckoutCone true && - git sparse-checkout add dir + test_must_fail git sparse-checkout add dir 2>err && + grep "existing sparse-checkout patterns do not use cone mode" err ) ' @@ -176,12 +177,14 @@ test_expect_success 'set sparse-checkout using --stdin' ' ' test_expect_success 'add to sparse-checkout' ' - cat repo/.git/info/sparse-checkout >expect && + cat repo/.git/info/sparse-checkout >old && + test_when_finished cp old repo/.git/info/sparse-checkout && cat >add <<-\EOF && pattern1 /folder1/ pattern2 EOF + cat old >expect && cat add >>expect && git -C repo sparse-checkout add --stdin actual &&