From patchwork Thu Oct 4 23:01:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 10626969 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5C13A15E8 for ; Thu, 4 Oct 2018 23:01:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4CB1229505 for ; Thu, 4 Oct 2018 23:01:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 40E3929509; Thu, 4 Oct 2018 23:01:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D17E429505 for ; Thu, 4 Oct 2018 23:01:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728034AbeJEF51 (ORCPT ); Fri, 5 Oct 2018 01:57:27 -0400 Received: from mail-qk1-f202.google.com ([209.85.222.202]:44811 "EHLO mail-qk1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725998AbeJEF50 (ORCPT ); Fri, 5 Oct 2018 01:57:26 -0400 Received: by mail-qk1-f202.google.com with SMTP id w126-v6so10023907qka.11 for ; Thu, 04 Oct 2018 16:01:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Q3vaXquMBRR1IKS411UnOKKvXcrNh0MtJBwabO6K+oA=; b=MU277sc1fxt8foWYBivxG+Ic2gVSBaCMbniVzNu/cxlYGawwc/71XHQRfaAILiEPLQ F1/gCWDoBod19pZRng8OQh5h/IbIZve/Y+u4V0/Kv8kkjyM3+V5DgpS2WttixZ98Uplj gWZXOPy29xrehULjiYD9sXLo0iu/d1FHNqEswUOcJBOdclb4bHsHSIpFzYKqhrmqldEU uQOzbeiNu1aj6GGqlhfwp3hiMNheo2MxZQ4ymsJ3GD5NcMQvg0fulEEiDj/uuXvGocgv hvwydn2mXAjNqMm4tITk4key7JL64bN/imgLUwcyKcbHGEGePFIRPCvvvva0bZNQU/t9 7rrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Q3vaXquMBRR1IKS411UnOKKvXcrNh0MtJBwabO6K+oA=; b=X83JhJ//SN8n0MqCAGlxFTUj9+cr/LiB/hR3V8VRjLITtf7gUYQZowgEXxCbXBoW8F iiW1uYbCYL6Eb1HneCLBEFvsUE7PKYcYoQWXRGVT5owiPAfFW/jTOq+a4fxu++kcd/PC xVyzneqnlzBWa05duXEIguQb0KPazZRH/aGI8Le5vhLeOwYVZcE8DWQ3zwocoO39cVbf U4o192o1Z9mQiFk7NFqGM3/h15E9gTl1J8NXeoVj5W1o/Pbdx40oceMOYHHHvvFFtix5 wYrva6laJVLlBSKlhgSoijYMUyVkAz4jW+lnpp9hMxUCKm0qV5IhksOq58l4etMfxX/u eaDw== X-Gm-Message-State: ABuFfojtWM/9QpULGFZbDjTn6HlQo1kV09TKPpqZzUKaCosjLY+czM5F kzVRQQipW/MA6R3EysQV+WBNlQc28s3nzZfZK6HBD7Sue7OoKFiA6NuOybVP3mhXJGb/c8uedCJ wW7wPmhpwE6+dK3QWNttp5px+5vB2QU2AWdygwS3awUX26wtSeGpH/BBUMmhncUI= X-Google-Smtp-Source: ACcGV622gJV2xKaPO3h+1SXpvHyZ//g7Idc8O+hgL0L/rVktAAhcK00rXIZxJ5Goqx1xUShdhbzenP03P/xemA== X-Received: by 2002:ac8:160d:: with SMTP id p13-v6mr6213078qtj.62.1538694106424; Thu, 04 Oct 2018 16:01:46 -0700 (PDT) Date: Thu, 4 Oct 2018 16:01:28 -0700 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [RFC PATCH 1/2] fuzz: Add basic fuzz testing target. From: Josh Steadmon To: git@vger.kernel.org Cc: Josh Steadmon Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Josh Steadmon --- .gitignore | 2 ++ Makefile | 30 +++++++++++++++++++++++++++++- fuzz-pack-headers.c | 14 ++++++++++++++ 3 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 fuzz-pack-headers.c diff --git a/.gitignore b/.gitignore index 9d1363a1eb..87a28b3115 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,5 @@ +/fuzz_corpora +/fuzz-pack-headers /GIT-BUILD-OPTIONS /GIT-CFLAGS /GIT-LDFLAGS diff --git a/Makefile b/Makefile index 13e1c52478..10bb82b115 100644 --- a/Makefile +++ b/Makefile @@ -590,6 +590,8 @@ XDIFF_OBJS = VCSSVN_OBJS = GENERATED_H = EXTRA_CPPFLAGS = +FUZZ_OBJS = +FUZZ_PROGRAMS = LIB_OBJS = PROGRAM_OBJS = PROGRAMS = @@ -682,6 +684,10 @@ SCRIPTS = $(SCRIPT_SH_INS) \ ETAGS_TARGET = TAGS +FUZZ_OBJS += fuzz-pack-headers.o + +FUZZ_PROGRAMS += $(patsubst %.o,%,$(FUZZ_OBJS)) + # Empty... EXTRA_PROGRAMS = @@ -2250,6 +2256,7 @@ TEST_OBJS := $(patsubst %$X,%.o,$(TEST_PROGRAMS)) $(patsubst %,t/helper/%,$(TEST OBJECTS := $(LIB_OBJS) $(BUILTIN_OBJS) $(PROGRAM_OBJS) $(TEST_OBJS) \ $(XDIFF_OBJS) \ $(VCSSVN_OBJS) \ + $(FUZZ_OBJS) \ common-main.o \ git.o ifndef NO_CURL @@ -2931,7 +2938,7 @@ profile-clean: cocciclean: $(RM) contrib/coccinelle/*.cocci.patch* -clean: profile-clean coverage-clean cocciclean +clean: profile-clean coverage-clean cocciclean fuzz-clean $(RM) *.res $(RM) $(OBJECTS) $(RM) $(LIB_FILE) $(XDIFF_LIB) $(VCSSVN_LIB) @@ -3061,3 +3068,24 @@ cover_db: coverage-report cover_db_html: cover_db cover -report html -outputdir cover_db_html cover_db + +### Fuzz testing +# +.PHONY: fuzz-clean fuzz-objs fuzz-compile + +FUZZ_CFLAGS = $(CFLAGS) -fsanitize-coverage=trace-pc-guard -fsanitize=address +FUZZ_LDFLAGS = $(FUZZ_CFLAGS) + + +fuzz-clean: + $(RM) $(FUZZ_PROGRAMS) $(FUZZ_OBJS) + +fuzz-objs: $(FUZZ_OBJS) + +fuzz-compile: + $(MAKE) CC=clang LD=clang CFLAGS="$(FUZZ_CFLAGS)" \ + LDFLAGS="$(FUZZ_LDFLAGS)" all fuzz-objs + +$(FUZZ_PROGRAMS): fuzz-compile + clang++ $(FUZZ_LDFLAGS) $(LIB_OBJS) $(BUILTIN_OBJS) $(XDIFF_OBJS) \ + $(EXTLIBS) git.o $@.o /usr/lib/llvm-4.0/lib/libFuzzer.a -o $@ diff --git a/fuzz-pack-headers.c b/fuzz-pack-headers.c new file mode 100644 index 0000000000..99da1d0fd3 --- /dev/null +++ b/fuzz-pack-headers.c @@ -0,0 +1,14 @@ +#include "packfile.h" + +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size); + +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) +{ + enum object_type type; + unsigned long len; + + unpack_object_header_buffer((const unsigned char *)data, + (unsigned long)size, &type, &len); + + return 0; +} From patchwork Thu Oct 4 23:01:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 10626971 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4248A15E8 for ; Thu, 4 Oct 2018 23:01:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 324AA29505 for ; Thu, 4 Oct 2018 23:01:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2674929509; Thu, 4 Oct 2018 23:01:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DB4429505 for ; Thu, 4 Oct 2018 23:01:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728059AbeJEF5a (ORCPT ); Fri, 5 Oct 2018 01:57:30 -0400 Received: from mail-pg1-f201.google.com ([209.85.215.201]:53538 "EHLO mail-pg1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725998AbeJEF5a (ORCPT ); Fri, 5 Oct 2018 01:57:30 -0400 Received: by mail-pg1-f201.google.com with SMTP id z8-v6so5492206pgp.20 for ; Thu, 04 Oct 2018 16:01:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=SBm7rigYT/BwHKx7cs96cvxw21DKmxxnkRFoxQi9oB4=; b=DHOWWiB6N9uLmU/Ib4znndyuwid8/7SYnf4yr7f+CjanseV8fCWQ6RVg2nNBomyR7m k5UIffgCXL/KK4jKK9lysUQnpkAIdFzaMCAlAUpQQGFYFeX1k/d0sPu1szMgEzCx+4Gd j1BahQwtJtI6ZIVnA7WBkZkzfOO0y3lKiqomgIhAMLmR7JqUW454BVnCoa9JZ3NBQGDX jDJQEnxBNa9hrn1iKzJSu1Yge2UgaADLiGr89kamGataorbCkO98s1E/aqH3ejW92H01 VzOBCWzHVy8uy8gbTGOW9GbjcU30al43GXEv9N68HWqFVAv8rzmz9VdepyIXcZU0x4ce hj5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SBm7rigYT/BwHKx7cs96cvxw21DKmxxnkRFoxQi9oB4=; b=CvbUOzq8sgDfCGOlRfeZcltA8mB7ON80X4XrZ53YH+Fu6QTBi4yostXam6UdVR25WV ht7hpDBitAMxEedefNa3poTWMxOZWpvGX9YDWVWgoW36TobZHQWk0iHdkmlvkQwIe4zV 55pqlt3IDMbrJjl0UMrBFqG/e6HEjv1oWP5Qvp0jEqiWEIH0qL1HDbTfyTbPZpwUsalW 4YgMVCI1eCwhwiQdww6OqcC4AUZj2iV56yIqVmmQhOpr5m0hVpZLR4SXr3CyWirb/dKH cQxqcscExXt1PEEjTuZYeMJRqSQOyz0b+qi7OTn3A2ncMMWxCy5JmCnUKwsfpF4Rmugu Wvaw== X-Gm-Message-State: ABuFfogmf+nCi/KI2NJnf5t2lluSqJVjbmARFaM75Epi+urfNyYrNSoa PY7ZDePrN3+0P8aNYHjr+CtfTXdYkYdkk8GWTfC603jiRsb21d5PMt9XrBMPAzc0X5d3did3MR1 6dJBIotoCmT3ehksFnxbMYWKLGzs07E2L1TMNvwbOyrdKzb3uJ+n9/snTOVfaRbc= X-Google-Smtp-Source: ACcGV63B/bYvarCa1kxVIK8tLyMgqneQ9kEtoCetnyXjrJUvkQRv4bwX1QhMJBoFbsC8yZd67VCw9Zkn7IXZ+g== X-Received: by 2002:a62:12ca:: with SMTP id 71-v6mr3911090pfs.55.1538694109911; Thu, 04 Oct 2018 16:01:49 -0700 (PDT) Date: Thu, 4 Oct 2018 16:01:29 -0700 In-Reply-To: Message-Id: <60b9daad5c5132811875b6094abf4d87852e4263.1538693039.git.steadmon@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [RFC PATCH 2/2] fuzz: Add fuzz testing for packfile indices. From: Josh Steadmon To: git@vger.kernel.org Cc: Josh Steadmon Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Josh Steadmon --- .gitignore | 1 + Makefile | 5 ++++- fuzz-pack-idx.c | 13 +++++++++++++ packfile.c | 44 +++++++++++++++++++++++++------------------- packfile.h | 13 +++++++++++++ 5 files changed, 56 insertions(+), 20 deletions(-) create mode 100644 fuzz-pack-idx.c diff --git a/.gitignore b/.gitignore index 87a28b3115..64b3377d40 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ /fuzz_corpora /fuzz-pack-headers +/fuzz-pack-idx /GIT-BUILD-OPTIONS /GIT-CFLAGS /GIT-LDFLAGS diff --git a/Makefile b/Makefile index 10bb82b115..f7e6be8f19 100644 --- a/Makefile +++ b/Makefile @@ -685,6 +685,7 @@ SCRIPTS = $(SCRIPT_SH_INS) \ ETAGS_TARGET = TAGS FUZZ_OBJS += fuzz-pack-headers.o +FUZZ_OBJS += fuzz-pack-idx.o FUZZ_PROGRAMS += $(patsubst %.o,%,$(FUZZ_OBJS)) @@ -3071,7 +3072,7 @@ cover_db_html: cover_db ### Fuzz testing # -.PHONY: fuzz-clean fuzz-objs fuzz-compile +.PHONY: fuzz-clean fuzz-objs fuzz-compile fuzz-all FUZZ_CFLAGS = $(CFLAGS) -fsanitize-coverage=trace-pc-guard -fsanitize=address FUZZ_LDFLAGS = $(FUZZ_CFLAGS) @@ -3089,3 +3090,5 @@ fuzz-compile: $(FUZZ_PROGRAMS): fuzz-compile clang++ $(FUZZ_LDFLAGS) $(LIB_OBJS) $(BUILTIN_OBJS) $(XDIFF_OBJS) \ $(EXTLIBS) git.o $@.o /usr/lib/llvm-4.0/lib/libFuzzer.a -o $@ + +fuzz-all: $(FUZZ_PROGRAMS) diff --git a/fuzz-pack-idx.c b/fuzz-pack-idx.c new file mode 100644 index 0000000000..0c3d777aac --- /dev/null +++ b/fuzz-pack-idx.c @@ -0,0 +1,13 @@ +#include "object-store.h" +#include "packfile.h" + +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size); + +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) +{ + struct packed_git p; + + load_idx("fuzz-input", GIT_SHA1_RAWSZ, (void *)data, size, &p); + + return 0; +} diff --git a/packfile.c b/packfile.c index 841b36182f..86074a76e9 100644 --- a/packfile.c +++ b/packfile.c @@ -80,10 +80,8 @@ void pack_report(void) static int check_packed_git_idx(const char *path, struct packed_git *p) { void *idx_map; - struct pack_idx_header *hdr; size_t idx_size; - uint32_t version, nr, i, *index; - int fd = git_open(path); + int fd = git_open(path), ret; struct stat st; const unsigned int hashsz = the_hash_algo->rawsz; @@ -101,16 +99,32 @@ static int check_packed_git_idx(const char *path, struct packed_git *p) idx_map = xmmap(NULL, idx_size, PROT_READ, MAP_PRIVATE, fd, 0); close(fd); - hdr = idx_map; + ret = load_idx(path, hashsz, idx_map, idx_size, p); + + if (ret) + munmap(idx_map, idx_size); + + return ret; +} + +int load_idx(const char *path, const unsigned int hashsz, void *idx_map, + size_t idx_size, struct packed_git *p) +{ + struct pack_idx_header *hdr = idx_map; + uint32_t version, nr, i, *index; + + if (idx_size < 4 * 256 + hashsz + hashsz) + return error("index file %s is too small", path); + if (idx_map == NULL) + return error("empty data"); + if (hdr->idx_signature == htonl(PACK_IDX_SIGNATURE)) { version = ntohl(hdr->idx_version); - if (version < 2 || version > 2) { - munmap(idx_map, idx_size); + if (version < 2 || version > 2) return error("index file %s is version %"PRIu32 " and is not supported by this binary" " (try upgrading GIT to a newer version)", path, version); - } } else version = 1; @@ -120,10 +134,8 @@ static int check_packed_git_idx(const char *path, struct packed_git *p) index += 2; /* skip index header */ for (i = 0; i < 256; i++) { uint32_t n = ntohl(index[i]); - if (n < nr) { - munmap(idx_map, idx_size); + if (n < nr) return error("non-monotonic index %s", path); - } nr = n; } @@ -135,10 +147,8 @@ static int check_packed_git_idx(const char *path, struct packed_git *p) * - hash of the packfile * - file checksum */ - if (idx_size != 4*256 + nr * (hashsz + 4) + hashsz + hashsz) { - munmap(idx_map, idx_size); + if (idx_size != 4 * 256 + nr * (hashsz + 4) + hashsz + hashsz) return error("wrong index v1 file size in %s", path); - } } else if (version == 2) { /* * Minimum size: @@ -157,20 +167,16 @@ static int check_packed_git_idx(const char *path, struct packed_git *p) unsigned long max_size = min_size; if (nr) max_size += (nr - 1)*8; - if (idx_size < min_size || idx_size > max_size) { - munmap(idx_map, idx_size); + if (idx_size < min_size || idx_size > max_size) return error("wrong index v2 file size in %s", path); - } if (idx_size != min_size && /* * make sure we can deal with large pack offsets. * 31-bit signed offset won't be enough, neither * 32-bit unsigned one will be. */ - (sizeof(off_t) <= 4)) { - munmap(idx_map, idx_size); + (sizeof(off_t) <= 4)) return error("pack too large for current definition of off_t in %s", path); - } } p->index_version = version; diff --git a/packfile.h b/packfile.h index 442625723d..6c4037605d 100644 --- a/packfile.h +++ b/packfile.h @@ -164,4 +164,17 @@ extern int has_pack_index(const unsigned char *sha1); */ extern int is_promisor_object(const struct object_id *oid); +/* + * Expose a function for fuzz testing. + * + * load_idx() parses a block of memory as a packfile index and puts the results + * into a struct packed_git. + * + * This function should not be used directly. It is exposed here only so that we + * have a convenient entry-point for fuzz testing. For real uses, you should + * probably use open_pack_index() or parse_pack_index() instead. + */ +extern int load_idx(const char *path, const unsigned int hashsz, void *idx_map, + size_t idx_size, struct packed_git *p); + #endif