From patchwork Tue Dec 7 20:02:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12662497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6769EC433F5 for ; Tue, 7 Dec 2021 20:02:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240992AbhLGUFx (ORCPT ); Tue, 7 Dec 2021 15:05:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40166 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229631AbhLGUFv (ORCPT ); Tue, 7 Dec 2021 15:05:51 -0500 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AB1AC061574 for ; Tue, 7 Dec 2021 12:02:20 -0800 (PST) Received: by mail-wm1-x336.google.com with SMTP id k37-20020a05600c1ca500b00330cb84834fso2615759wms.2 for ; Tue, 07 Dec 2021 12:02:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=SW7KP2FKXWZD8JoH4kNbs/21W1QaalBtFoxsXrhI4vw=; b=o2h8h24y5txHw4js0XBSHW58TtkNNy9so/WLAsQptRFkvpG2XLNH+xAsN2n7gdSVLh T5oBr0mQbVMmj+lMnkDL0iu1jsY+PPnVVgdDH0ePyknHM/Wbb3qzsqGyqWx9JO8iIU0S DgRIjjTjD6ijGq76mGUrLAYGCVc8mU7skRUiivz51jfpQFCy5oTZ+yIA8jOttgJKv6xn H5eWpQnRXPIzs0yJu53xDb2aWxi4eEpNlS/5sZwOZ/ZxgtNYyzILvp1CofXc77KaTMB7 psviG1DS9tKi77V+43tmhSV/4YM54ZrtMRtSY8HTGdFbO3aIWwkGmWM5fvhDDPK8Ti6M 5I+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=SW7KP2FKXWZD8JoH4kNbs/21W1QaalBtFoxsXrhI4vw=; b=r3fMKo8GVIyhmqmcAvfWUBm1PTezCWiLO4WvSbUMOH6FDVBZnydXWnu36RoYN7Gr4y TAW+KIR3MwoT6TMJJ2VpAzIBDYOxthbVPW7BLurmX2gsY85Ei3ktiA8TL2o/OEYy+PrH LrotWdTvkc074zPxZq4+BbadBXrma0el+allriiWTkjoajaceprO+Y8OoelDZ/TxYZKW hEWE1XDTt2TOXnOar4O4ISVG69OzCCwcORpPx83qVPLLK6N8AviB1PNt7h9oZjeQ/gF6 BqZW65gTxDRvXmFKca1ZqMM7o3rwi9Wk5niGzItp807Jvwm4e0yMQw59fZfADuOIxSDG zkiw== X-Gm-Message-State: AOAM530lfMla38JzLHdXwETGXBYnb1mk6wMUoo8HYKS4auxjEL0HSkMO 2uoIBE2KhvZ3gMgRNb9UxDqjWGZN23g= X-Google-Smtp-Source: ABdhPJwBh/xex07PqjoVmNMC/mR+NW5A6en4YTu7qFrDkwdvhSDI5zmLDuajqQK3iMKrxFS5X4HwKQ== X-Received: by 2002:a7b:cd02:: with SMTP id f2mr9717845wmj.115.1638907338631; Tue, 07 Dec 2021 12:02:18 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h18sm767203wre.46.2021.12.07.12.02.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Dec 2021 12:02:18 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 07 Dec 2021 20:02:14 +0000 Subject: [PATCH 1/3] sparse-checkout: fix segfault on malformed patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are used to populate two hashsets that accelerate pattern matching. If the user modifies the sparse-checkout file outside of the 'sparse-checkout' builtin, then strange patterns can happen, triggering some error checks. One of these error checks is possible to hit when some special characters exist in a line. A warning message is correctly written to stderr, but then there is additional logic that attempts to remove the line from the hashset and free the data. This leads to a segfault in the 'git sparse-checkout list' command because it iterates over the contents of the hashset, which is no invalid. The fix here is to stop trying to remove from the hashset. Better to leave bad data in the sparse-checkout matching logic (with a warning) than to segfault. If we are in this state, then we are already traversing into undefined behavior, so this change to keep the entry in the hashset is no worse than removing it. Add a test that triggers the segfault without the code change. Reported-by: John Burnett Signed-off-by: Derrick Stolee --- dir.c | 3 --- t/t1091-sparse-checkout-builtin.sh | 15 +++++++++++++++ 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/dir.c b/dir.c index 5aa6fbad0b7..0693c7cb3ee 100644 --- a/dir.c +++ b/dir.c @@ -819,9 +819,6 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern /* we already included this at the parent level */ warning(_("your sparse-checkout file may have issues: pattern '%s' is repeated"), given->pattern); - hashmap_remove(&pl->parent_hashmap, &translated->ent, &data); - free(data); - free(translated); } return; diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 272ba1b566b..c72b8ee2e7b 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -708,4 +708,19 @@ test_expect_success 'cone mode clears ignored subdirectories' ' test_cmp expect out ' +test_expect_success 'malformed cone-mode patterns' ' + git -C repo sparse-checkout init --cone && + mkdir -p repo/foo/bar && + touch repo/foo/bar/x repo/foo/y && + cat >repo/.git/info/sparse-checkout <<-\EOF && + /* + !/*/ + /foo/ + !/foo/*/ + /foo/\*/ + EOF + cat repo/.git/info/sparse-checkout && + git -C repo sparse-checkout list +' + test_done From patchwork Tue Dec 7 20:02:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12662499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B87EC433EF for ; Tue, 7 Dec 2021 20:02:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240998AbhLGUFx (ORCPT ); Tue, 7 Dec 2021 15:05:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240986AbhLGUFv (ORCPT ); Tue, 7 Dec 2021 15:05:51 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD51DC061746 for ; Tue, 7 Dec 2021 12:02:20 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id t18so16973wrg.11 for ; Tue, 07 Dec 2021 12:02:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=8n49IrCinbvTdGsbFNvzcILsfe/9BYZoqmc6kar9Ivg=; b=BvJj9qBWdpxBvaBJB73b30gSPfa6zWEHhTPBC+ASW7TdgQs2BlDOi/EkDqLnHgt7iv G9zn/krP3P27RKxDOXFDl3NzHgPEKoFmbaLoR+oNuDsrAzvbDLLvZ5tm2zo3kBdIKuJc uwVjkAHseWKrette0R0xQUlYNYHiy3Ebu7TDepN6CKSrDAFOX4TLsb+9jJm3IClpI6J9 vd8f+rqswh5Hf1f81H6/VtAuDYu40oYFNEmI3/heXkg/pzLSp8KpIrvQRupS/b/J/FZL k2uaf73383oj0zDafOlwpU4akiLMdzhfdRsxFUH/NGMde7GLzn6GYZ6aW/YOXJ3ScPJN 5m3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=8n49IrCinbvTdGsbFNvzcILsfe/9BYZoqmc6kar9Ivg=; b=J1fbmcsJsDYGT+waLdB3my3cZDJsIblSEDdH6dzJDOtMuz73sp551yS3a0ttFDQ7Tu uSoEvA3AwBh1H/XB0hFYw41OsdjIFiQasML0Q1GQNlDF5Y5AfT6lVN/WWdJfHSJRKBad lk7+Bsc7SmzoyXqWowmuu1Cnc43M0Tudh5+VnGFx/EKkv15c0YIsxpuKMVW8+JDfd4I3 STqGP+ZPMjmWlt8KSy6uTv/urJDopTrzRFaefqCsuZo2bfEKuWarYx8JesTxjRV0vK4Y Reoin7skQ7SkYl6TLHpx9ruWKDrwMficn8Oc5nGuljRlL6H6M37hwYW5yExuZ4PPZ/Km PKJg== X-Gm-Message-State: AOAM530QeIpTOrQWr29g9sNKdQm9H2SShiGO2Rb8N45V2fvbLO8TR6rn sX0Nr9MlSjRkbC+OPkfdkeG3AxBDxHw= X-Google-Smtp-Source: ABdhPJx0J2Hsshq1RqGPV0ffL5D8YtmPEqlhnPY3FPNVKkxQJlukKimaBCm3l+e7cNWKahurXKZbcg== X-Received: by 2002:a5d:42cf:: with SMTP id t15mr53461081wrr.444.1638907339373; Tue, 07 Dec 2021 12:02:19 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m1sm549793wme.39.2021.12.07.12.02.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Dec 2021 12:02:19 -0800 (PST) Message-Id: <239bf23eacb7dbf7b0659b483709dc4579d43b1a.1638907336.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 07 Dec 2021 20:02:15 +0000 Subject: [PATCH 2/3] sparse-checkout: fix OOM error with mixed patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Add a test to t1091-sparse-checkout-builtin.sh that would result in an infinite loop and out-of-memory error before this change. The issue relies on having non-cone-mode patterns while trying to modify the patterns in cone-mode. The fix is simple, allowing us to break from the loop when the input path does not contain a slash, as the "dir" pattern we added does not. This is only a fix to the critical out-of-memory error. A better response to such a strange state will follow in a later change. Reported-by: Calbabreaker Helped-by: Taylor Blau Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 2 +- t/t1091-sparse-checkout-builtin.sh | 11 +++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index d0f5c4702be..9ccdcde9832 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -483,7 +483,7 @@ static void insert_recursive_pattern(struct pattern_list *pl, struct strbuf *pat char *oldpattern = e->pattern; size_t newlen; - if (slash == e->pattern) + if (!slash || slash == e->pattern) break; newlen = slash - e->pattern; diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index c72b8ee2e7b..67ce24c9c83 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -103,6 +103,17 @@ test_expect_success 'clone --sparse' ' check_files clone a ' +test_expect_success 'switching to cone mode with non-cone mode patterns' ' + git init bad-patterns && + ( + cd bad-patterns && + git sparse-checkout init && + git sparse-checkout add dir && + git config core.sparseCheckoutCone true && + git sparse-checkout add dir + ) +' + test_expect_success 'interaction with clone --no-checkout (unborn index)' ' git clone --no-checkout "file://$(pwd)/repo" clone_no_checkout && git -C clone_no_checkout sparse-checkout init --cone && From patchwork Tue Dec 7 20:02:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12662501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB286C433FE for ; Tue, 7 Dec 2021 20:02:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241026AbhLGUF5 (ORCPT ); Tue, 7 Dec 2021 15:05:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240987AbhLGUFw (ORCPT ); Tue, 7 Dec 2021 15:05:52 -0500 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9454CC061748 for ; Tue, 7 Dec 2021 12:02:21 -0800 (PST) Received: by mail-wr1-x42e.google.com with SMTP id i5so96621wrb.2 for ; Tue, 07 Dec 2021 12:02:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=nHSBja1nsAH0zardxs26xICqSvhBOQvdPGSC0uUkPM8=; b=LXRI6ldy4p0P5I9E6zbAI5nAV/iU3w6rslPfHH6yAKTyDA5BSQYnYqN3vofwXBU9RD FHpD/fR07gR5xn75ATu172EeVhI+BxiYKO28aLdS4G0wOaSBCThUtLl+g6CIOs1ebuu6 uXQnCMbhTXNnOb5wds+ep0A20OFRb8A+Z09e/OjYzGa/+wuBNfkYO5R29uJNbJiy5gOC ABtaaKRleBZ7yI03qlZ1S8C4oyQ3mCtoGoaUP0CXhyQw3I9By2eq5rW8cClodyqfdaKx 3tCiZUdjwLSCcVETn+IHD/ucKp34LNVcn8uUL/q8xAPAY58X6gmxEwcssIR20SOuIsP1 An6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=nHSBja1nsAH0zardxs26xICqSvhBOQvdPGSC0uUkPM8=; b=0CLyLGUAzdL1CoKT5zlOl9E0I6eSzvh42BK9Eb9pZhkZaVCZLG+HMPyPwdjXQGSyk0 UIqCKsTQcKWs+YFfLmTKTH6VAaktgvyEZU+7BOfHXS1dUvlaPY4IP1Vh/GHgYX3kso5G aD04fApLTiqPSTLjRZm0NAsagsW8iywfZ4EBb4zb5foeat3H38dS73EgktkPxtnzVHFY 65GJhQ5/zWVa/UnVSxgXXTMqWLQTCB6aMNZt5ZslTB3srsRiFpJ15YMrKn4FfDPPBH1n 6qoTR49jVOK8t/2KCbDEbql4yv9rLO/x6BrC5eGxGGxYNQTA3MxT782DeTful1EH/Bc+ AMXw== X-Gm-Message-State: AOAM532pixzf9AeGPcGYJ9GQoIIs2vLJIriev5JBUKvYh/cPgM+U0qbX dXv4iqor++Vx9TV6YzkLzhCM178YsMo= X-Google-Smtp-Source: ABdhPJyvrcsfbrMAp90Mg1KXbQx+WQJvJnFazrJjevR29Ktp2S+gH1o9skEhKEwkR+bcZf7nqJ0ajQ== X-Received: by 2002:a5d:4883:: with SMTP id g3mr53648636wrq.590.1638907340018; Tue, 07 Dec 2021 12:02:20 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q26sm677142wrc.39.2021.12.07.12.02.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Dec 2021 12:02:19 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 07 Dec 2021 20:02:16 +0000 Subject: [PATCH 3/3] sparse-checkout: refuse to add to bad patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, newren@gmail.com, vdye@github.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When in cone mode sparse-checkout, it is unclear how 'git sparse-checkout add ...' should behave if the existing sparse-checkout file does not match the cone mode patterns. Change the behavior to fail with an error message about the existing patterns. Also, all cone mode patterns start with a '/' character, so add that restriction. This is necessary for our example test 'cone mode: warn on bad pattern', but also requires modifying the example sparse-checkout file we use to test the warnings related to recognizing cone mode patterns. This error checking would cause a failure further down the test script because of a test that adds non-cone mode patterns without cleaning them up. Perform that cleanup as part of the test now. Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 3 +++ dir.c | 2 +- t/t1091-sparse-checkout-builtin.sh | 7 +++++-- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 9ccdcde9832..6580075a631 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -598,6 +598,9 @@ static void add_patterns_cone_mode(int argc, const char **argv, die(_("unable to load existing sparse-checkout patterns")); free(sparse_filename); + if (!existing.use_cone_patterns) + die(_("existing sparse-checkout patterns do not use cone mode")); + hashmap_for_each_entry(&existing.recursive_hashmap, &iter, pe, ent) { if (!hashmap_contains_parent(&pl->recursive_hashmap, pe->pattern, &buffer) || diff --git a/dir.c b/dir.c index 0693c7cb3ee..a5dddafa16d 100644 --- a/dir.c +++ b/dir.c @@ -727,7 +727,7 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern } if (given->patternlen < 2 || - *given->pattern == '*' || + *given->pattern != '/' || strstr(given->pattern, "**")) { /* Not a cone pattern. */ warning(_("unrecognized pattern: '%s'"), given->pattern); diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 67ce24c9c83..2ed247d75a5 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -110,7 +110,8 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' ' git sparse-checkout init && git sparse-checkout add dir && git config core.sparseCheckoutCone true && - git sparse-checkout add dir + test_must_fail git sparse-checkout add dir 2>err && + grep "existing sparse-checkout patterns do not use cone mode" err ) ' @@ -176,12 +177,14 @@ test_expect_success 'set sparse-checkout using --stdin' ' ' test_expect_success 'add to sparse-checkout' ' - cat repo/.git/info/sparse-checkout >expect && + cat repo/.git/info/sparse-checkout >old && + test_when_finished cp old repo/.git/info/sparse-checkout && cat >add <<-\EOF && pattern1 /folder1/ pattern2 EOF + cat old >expect && cat add >>expect && git -C repo sparse-checkout add --stdin actual &&