From patchwork Mon Jun 24 13:02:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013025 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 34DFE1580 for ; Mon, 24 Jun 2019 13:02:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25B1028BBD for ; Mon, 24 Jun 2019 13:02:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19FB628BC0; Mon, 24 Jun 2019 13:02:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0074528BBD for ; Mon, 24 Jun 2019 13:02:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729009AbfFXNCw (ORCPT ); Mon, 24 Jun 2019 09:02:52 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:35701 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726801AbfFXNCv (ORCPT ); Mon, 24 Jun 2019 09:02:51 -0400 Received: by mail-pg1-f193.google.com with SMTP id s27so7108996pgl.2 for ; Mon, 24 Jun 2019 06:02:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NxO18jhYr9BpexWn56R/qPBTMSoTD92+oXI5cx3c4bA=; b=Xia1ZcqUfnN12XrX75ePG/oaMI2Q14zO//MlSEiVG6v1uBAtmRoU10km1maO910QjT Rk8U3jhtpSZtfjqtmzsxnSFgf0rHaI7CowFHnFuaiSQ/9w/mN30+3c1JCz63XB9bgMhI iP4bb3iVRK0Bab3M+Ja1S0ueNRgg7V1QEG4sYrz5aXgE/lWbgHW80FQIflI17UaHBg4g PlBiii99X1CQTFBArLsFY1IgbRwN3yfUp7WOYhlKQSTu84zl+97eNh6D4hG0YpjhiyAz sL+UD4Ino97MnrQ+M7rAlJBayos98Y56Vi+Y8RvmvdLBXvFRj93AAQoftYpP26nWI4Rp zdCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NxO18jhYr9BpexWn56R/qPBTMSoTD92+oXI5cx3c4bA=; b=SX4GhP+7E2qb5PkaA6UXzYcrQJO+2s9t2SzcVnTLjZhTF0FiCmgwy2nAZOv8+4Mwgb HEQdkme618FQiE+kMuWHj4VS/O8/z1REg6A8aePT1UXOAv+n3462s7Xn97FKY3fbOibj AqcRVGOBbmGiDVSKquy7gZ87wc9ZTwA4q+J+t3njzJgLOxKGZmuUe8kDb5lRGtenLFNW 5Ef7rCMIbs9IkiEtg1TQ4AFyqz4NgnUPnhhtDTCiXDE+8TkrmT0O7NQ7oRYDew41bCXc yptrcv1D8ZzezFkY8WMvyahKN8qaWSVpK/5VgQuS8Iszk/gvWcA1espYJvAWOJM6C9ED kYHA== X-Gm-Message-State: APjAAAVrocfZixDXdK5HmaekDyTIy2UZeWcHDn55kmbk1/cucvuo/RxX /yCs9EJ6pTnjTYf+JESsr/pRbMM2 X-Google-Smtp-Source: APXvYqxOf/ol2N/24RDkPCs89TQ2uJbBprudwxbJ8Wmm2Ru64IT18MjHQCMeJqosWqOYTn8yEx0y0Q== X-Received: by 2002:a63:d0:: with SMTP id 199mr33044565pga.85.1561381369829; Mon, 24 Jun 2019 06:02:49 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id bo20sm12470311pjb.23.2019.06.24.06.02.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:02:49 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:02:44 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 01/10] ls-files: add --json to dump the index Date: Mon, 24 Jun 2019 20:02:17 +0700 Message-Id: <20190624130226.17293-2-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP So far we don't have a command to basically dump the index file out, with all its glory details. Checking some info, for example, stat time, usually involves either writing new code or firing up "xxd" and decoding values by yourself. This --json is supposed to help that. It dumps the index in a human readable format but also easy to be processed with tools. And it will print almost enough info to reconstruct the index later. In this patch we only dump the main part, not extensions. But at the end of the series, the entire index is dumped. The end result could be very verbose even on a small repository such as git.git. Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git-ls-files.txt | 5 +++ builtin/ls-files.c | 38 +++++++++++++--- cache.h | 2 + json-writer.c | 22 ++++++++++ json-writer.h | 23 ++++++++++ read-cache.c | 72 ++++++++++++++++++++++++++++++- t/t3011-ls-files-json.sh (new +x) | 44 +++++++++++++++++++ t/t3011/basic (new) | 67 ++++++++++++++++++++++++++++ 8 files changed, 265 insertions(+), 8 deletions(-) diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt index 8461c0e83e..fec5cb7170 100644 --- a/Documentation/git-ls-files.txt +++ b/Documentation/git-ls-files.txt @@ -162,6 +162,11 @@ a space) at the start of each line: possible for manual inspection; the exact format may change at any time. +--debug-json:: + Dump the entire index content in JSON format. This is for + debugging purposes. The JSON structure is subject to change. + Note that the strings are not always valid UTF-8. + --eol:: Show and of files. is the file content identification used by Git when diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 7f83c9a6f2..b60cd9ab28 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -8,6 +8,7 @@ #include "cache.h" #include "repository.h" #include "config.h" +#include "json-writer.h" #include "quote.h" #include "dir.h" #include "builtin.h" @@ -31,6 +32,7 @@ static int show_modified; static int show_killed; static int show_valid_bit; static int show_fsmonitor_bit; +static int show_json; static int line_terminator = '\n'; static int debug_mode; static int show_eol; @@ -577,6 +579,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) N_("pretend that paths removed since are still present")), OPT__ABBREV(&abbrev), OPT_BOOL(0, "debug", &debug_mode, N_("show debugging data")), + OPT_BOOL(0, "debug-json", &show_json, + N_("dump index content in JSON format")), OPT_END() }; @@ -632,7 +636,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) "--error-unmatch"); parse_pathspec(&pathspec, 0, - PATHSPEC_PREFER_CWD, + show_json ? PATHSPEC_PREFER_FULL : PATHSPEC_PREFER_CWD, prefix, argv); /* @@ -660,8 +664,18 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) /* With no flags, we default to showing the cached files */ if (!(show_stage || show_deleted || show_others || show_unmerged || - show_killed || show_modified || show_resolve_undo)) + show_killed || show_modified || show_resolve_undo || show_json)) show_cached = 1; + if (show_json && (show_stage || show_deleted || show_others || + show_unmerged || show_killed || show_modified || + show_cached || pathspec.nr)) + die(_("--debug-json cannot be used with other file selection options")); + if (show_json && show_resolve_undo) + die(_("--debug-json cannot be used with %s"), "--resolve-undo"); + if (show_json && with_tree) + die(_("--debug-json cannot be used with %s"), "--with-tree"); + if (show_json && debug_mode) + die(_("--debug-json cannot be used with %s"), "--debug"); if (with_tree) { /* @@ -673,10 +687,22 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) overlay_tree_on_index(the_repository->index, with_tree, max_prefix); } - show_files(the_repository, &dir); - - if (show_resolve_undo) - show_ru_info(the_repository->index); + if (!show_json) { + show_files(the_repository, &dir); + + if (show_resolve_undo) + show_ru_info(the_repository->index); + } else { + struct json_writer jw = JSON_WRITER_INIT; + + discard_index(the_repository->index); + the_repository->index->jw = &jw; + if (repo_read_index(the_repository) < 0) + die("index file corrupt"); + puts(jw.json.buf); + the_repository->index->jw = NULL; + jw_release(&jw); + } if (ps_matched) { int bad; diff --git a/cache.h b/cache.h index bf20337ef4..84d0aeed20 100644 --- a/cache.h +++ b/cache.h @@ -326,6 +326,7 @@ static inline unsigned int canon_mode(unsigned int mode) #define UNTRACKED_CHANGED (1 << 7) #define FSMONITOR_CHANGED (1 << 8) +struct json_writer; struct split_index; struct untracked_cache; @@ -350,6 +351,7 @@ struct index_state { uint64_t fsmonitor_last_update; struct ewah_bitmap *fsmonitor_dirty; struct mem_pool *ce_mem_pool; + struct json_writer *jw; }; /* Name hashing */ diff --git a/json-writer.c b/json-writer.c index aadb9dbddc..0608726512 100644 --- a/json-writer.c +++ b/json-writer.c @@ -202,6 +202,28 @@ void jw_object_null(struct json_writer *jw, const char *key) strbuf_addstr(&jw->json, "null"); } +void jw_object_filemode(struct json_writer *jw, const char *key, mode_t mode) +{ + object_common(jw, key); + strbuf_addf(&jw->json, "\"%06o\"", mode); +} + +void jw_object_stat_data(struct json_writer *jw, const char *name, + const struct stat_data *sd) +{ + jw_object_inline_begin_object(jw, name); + jw_object_intmax(jw, "ctime_sec", sd->sd_ctime.sec); + jw_object_intmax(jw, "ctime_nsec", sd->sd_ctime.nsec); + jw_object_intmax(jw, "mtime_sec", sd->sd_mtime.sec); + jw_object_intmax(jw, "mtime_nsec", sd->sd_mtime.nsec); + jw_object_intmax(jw, "device", sd->sd_dev); + jw_object_intmax(jw, "inode", sd->sd_ino); + jw_object_intmax(jw, "uid", sd->sd_uid); + jw_object_intmax(jw, "gid", sd->sd_gid); + jw_object_intmax(jw, "size", sd->sd_size); + jw_end(jw); +} + static void increase_indent(struct strbuf *sb, const struct json_writer *jw, int indent) diff --git a/json-writer.h b/json-writer.h index 83906b09c1..c48c4cbf33 100644 --- a/json-writer.h +++ b/json-writer.h @@ -42,8 +42,11 @@ * of the given strings. */ +#include "git-compat-util.h" #include "strbuf.h" +struct stat_data; + struct json_writer { /* @@ -81,6 +84,9 @@ void jw_object_true(struct json_writer *jw, const char *key); void jw_object_false(struct json_writer *jw, const char *key); void jw_object_bool(struct json_writer *jw, const char *key, int value); void jw_object_null(struct json_writer *jw, const char *key); +void jw_object_filemode(struct json_writer *jw, const char *key, mode_t value); +void jw_object_stat_data(struct json_writer *jw, const char *key, + const struct stat_data *sd); void jw_object_sub_jw(struct json_writer *jw, const char *key, const struct json_writer *value); @@ -104,4 +110,21 @@ void jw_array_inline_begin_array(struct json_writer *jw); int jw_is_terminated(const struct json_writer *jw); void jw_end(struct json_writer *jw); +/* + * These _gently versions accept NULL json_writer to reduce too much + * branching at the call site. + */ +static inline void jw_object_inline_begin_array_gently(struct json_writer *jw, + const char *name) +{ + if (jw) + jw_object_inline_begin_array(jw, name); +} + +static inline void jw_end_gently(struct json_writer *jw) +{ + if (jw) + jw_end(jw); +} + #endif /* JSON_WRITER_H */ diff --git a/read-cache.c b/read-cache.c index 4dd22f4f6e..db5147d088 100644 --- a/read-cache.c +++ b/read-cache.c @@ -25,6 +25,7 @@ #include "fsmonitor.h" #include "thread-utils.h" #include "progress.h" +#include "json-writer.h" /* Mask for the name length in ce_flags in the on-disk index */ @@ -1952,6 +1953,49 @@ static void *load_index_extensions(void *_data) return NULL; } +static void dump_cache_entry(struct index_state *istate, + int index, + unsigned long offset, + const struct cache_entry *ce) +{ + struct json_writer *jw = istate->jw; + + jw_array_inline_begin_object(jw); + + /* + * this is technically redundant, but it's for easier + * navigation when there hundreds of entries + */ + jw_object_intmax(jw, "id", index); + + jw_object_string(jw, "name", ce->name); + + jw_object_filemode(jw, "mode", ce->ce_mode); + + jw_object_intmax(jw, "flags", ce->ce_flags); + /* + * again redundant info, just so you don't have to decode + * flags values manually + */ + if (ce->ce_flags & CE_EXTENDED) + jw_object_true(jw, "extended_flags"); + if (ce->ce_flags & CE_VALID) + jw_object_true(jw, "assume_unchanged"); + if (ce->ce_flags & CE_INTENT_TO_ADD) + jw_object_true(jw, "intent_to_add"); + if (ce->ce_flags & CE_SKIP_WORKTREE) + jw_object_true(jw, "skip_worktree"); + if (ce_stage(ce)) + jw_object_intmax(jw, "stage", ce_stage(ce)); + + jw_object_string(jw, "oid", oid_to_hex(&ce->oid)); + + jw_object_stat_data(jw, "stat", &ce->ce_stat_data); + jw_object_intmax(jw, "file_offset", offset); + + jw_end(jw); +} + /* * A helper function that will load the specified range of cache entries * from the memory mapped file and add them to the given index. @@ -1972,6 +2016,9 @@ static unsigned long load_cache_entry_block(struct index_state *istate, ce = create_from_disk(ce_mem_pool, istate->version, disk_ce, &consumed, previous_ce); set_index_entry(istate, i, ce); + if (istate->jw) + dump_cache_entry(istate, i, src_offset, ce); + src_offset += consumed; previous_ce = ce; } @@ -1983,6 +2030,8 @@ static unsigned long load_all_cache_entries(struct index_state *istate, { unsigned long consumed; + jw_object_inline_begin_array_gently(istate->jw, "entries"); + if (istate->version == 4) { mem_pool_init(&istate->ce_mem_pool, estimate_cache_size_from_compressed(istate->cache_nr)); @@ -1993,6 +2042,8 @@ static unsigned long load_all_cache_entries(struct index_state *istate, consumed = load_cache_entry_block(istate, istate->ce_mem_pool, 0, istate->cache_nr, mmap, src_offset, NULL); + + jw_end_gently(istate->jw); return consumed; } @@ -2120,6 +2171,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) size_t extension_offset = 0; int nr_threads, cpus; struct index_entry_offset_table *ieot = NULL; + int jw_pretty = 1; if (istate->initialized) return istate->cache_nr; @@ -2154,6 +2206,8 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) istate->cache_nr = ntohl(hdr->hdr_entries); istate->cache_alloc = alloc_nr(istate->cache_nr); istate->cache = xcalloc(istate->cache_alloc, sizeof(*istate->cache)); + istate->timestamp.sec = st.st_mtime; + istate->timestamp.nsec = ST_MTIME_NSEC(st); istate->initialized = 1; p.istate = istate; @@ -2176,6 +2230,20 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) if (!HAVE_THREADS) nr_threads = 1; + if (istate->jw) { + jw_object_begin(istate->jw, jw_pretty); + jw_object_intmax(istate->jw, "version", istate->version); + jw_object_string(istate->jw, "oid", oid_to_hex(&istate->oid)); + jw_object_intmax(istate->jw, "mtime_sec", istate->timestamp.sec); + jw_object_intmax(istate->jw, "mtime_nsec", istate->timestamp.nsec); + + /* + * Threading may mess up json writing. This is for + * debugging only, so performance is not a concern. + */ + nr_threads = 1; + } + if (nr_threads > 1) { extension_offset = read_eoie_extension(mmap, mmap_size); if (extension_offset) { @@ -2204,8 +2272,6 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) src_offset += load_all_cache_entries(istate, mmap, mmap_size, src_offset); } - istate->timestamp.sec = st.st_mtime; - istate->timestamp.nsec = ST_MTIME_NSEC(st); /* if we created a thread, join it otherwise load the extensions on the primary thread */ if (extension_offset) { @@ -2216,6 +2282,8 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) p.src_offset = src_offset; load_index_extensions(&p); } + jw_end_gently(istate->jw); + munmap((void *)mmap, mmap_size); /* diff --git a/t/t3011-ls-files-json.sh b/t/t3011-ls-files-json.sh new file mode 100755 index 0000000000..97bcd814be --- /dev/null +++ b/t/t3011-ls-files-json.sh @@ -0,0 +1,44 @@ +#!/bin/sh + +test_description='ls-files dumping json' + +. ./test-lib.sh + +strip_number() { + for name; do + echo 's/\("'$name'":\) [0-9]\+/\1 /' >>filter.sed + done +} + +strip_string() { + for name; do + echo 's/\("'$name'":\) ".*"/\1 /' >>filter.sed + done +} + +compare_json() { + git ls-files --debug-json >json && + sed -f filter.sed json >filtered && + test_cmp "$TEST_DIRECTORY"/t3011/"$1" filtered +} + +test_expect_success 'setup' ' + mkdir sub && + echo one >one && + git add one && + echo 2 >sub/two && + git add sub/two && + + echo intent-to-add >ita && + git add -N ita && + + strip_number ctime_sec ctime_nsec mtime_sec mtime_nsec && + strip_number device inode uid gid file_offset ext_size && + strip_string oid ident +' + +test_expect_success 'ls-files --json, main entries' ' + compare_json basic +' + +test_done diff --git a/t/t3011/basic b/t/t3011/basic new file mode 100644 index 0000000000..9436445d90 --- /dev/null +++ b/t/t3011/basic @@ -0,0 +1,67 @@ +{ + "version": 3, + "oid": , + "mtime_sec": , + "mtime_nsec": , + "entries": [ + { + "id": 0, + "name": "ita", + "mode": "100644", + "flags": 536887296, + "extended_flags": true, + "intent_to_add": true, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 0 + }, + "file_offset": + }, + { + "id": 1, + "name": "one", + "mode": "100644", + "flags": 0, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 4 + }, + "file_offset": + }, + { + "id": 2, + "name": "sub/two", + "mode": "100644", + "flags": 0, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 2 + }, + "file_offset": + } + ] +} From patchwork Mon Jun 24 13:02:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013027 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1B361580 for ; Mon, 24 Jun 2019 13:02:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A317328BB1 for ; Mon, 24 Jun 2019 13:02:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9760120223; Mon, 24 Jun 2019 13:02:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F30620223 for ; Mon, 24 Jun 2019 13:02:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729106AbfFXNC5 (ORCPT ); Mon, 24 Jun 2019 09:02:57 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:37998 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726801AbfFXNC5 (ORCPT ); Mon, 24 Jun 2019 09:02:57 -0400 Received: by mail-pg1-f193.google.com with SMTP id z75so4490272pgz.5 for ; Mon, 24 Jun 2019 06:02:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6pND3A6UIfQ+2CQ8yMVh4PK9XHBF8wkl1DDOK5kObOw=; b=beO71RVoKQij7JjZ4z4qDqsPOmoy23+5ZPTIFsupn/ASpbX1pRJF/1ioosKKCXrNZ6 7sorBJCE8HHxmBuFwyX1yhSUCvmC27B+qwPplsjOGBJImk8XaOIcebWeiPwkLZ+egCzt Pnice7QebllDMMEiq+kv/bNXK+oRjWGBEkQj+hebiuKZGyOsYqS2ZwPa6Avza2Ep18cL ZWJq7w+62q3BQKcy75iXzgILeyQJQrJw7Q8w6vnMG79TGSvpDTtNogr0g9CoOn4YfSEa tQakSe1/wbv029fo/Vwl+cf2z8b5/KAcZ4LH5fvsJ/YN0ZI+vjrsXHjE9KTNAfvXXywk 5lUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6pND3A6UIfQ+2CQ8yMVh4PK9XHBF8wkl1DDOK5kObOw=; b=hEPCYNBBPPKlkJdwiF1IzbQPq+AR0RIRbLDcj+ziJj41Pt0+EkzFp/P9bS9uzYF0Ef 2Rd9YHBpeT01TtU/Nyhd3KsiC6zokJvJd5J3xQ1B/90gg9iaskxzivIGYMn6tuxxppOu fNNclg6gNoEwB5u2QhD2VPcg8tH9MIuACgACoESlz+gv3UMYTg17SrFePX3I3aQqomfY HiHd6qkTkZtkvtPmHNgkyt6oTgIFeyotzjdoyMtOfccDqMG4vZJbAGJVpjgwnXL00tXf N7lm6VmRNiWrOt+2NmsJDiUi/a3486Xa7Mx7wbSRmNuYVRf+/QzD5SEUMDzgHJbbGLe5 7Bbw== X-Gm-Message-State: APjAAAWp8vPGW3nIntJ99rTreHdIAsmmH4DWje41j+KViVe8vHlt/SGD Kt42FStaOHkQKEY8mSBYY/xqwYuU X-Google-Smtp-Source: APXvYqwBmobABB4vMbjg9PasjnFKwrDttSrsPaLjAI8NCGVVuZ0H1olljFocKoA54IJYRovJg7/1eQ== X-Received: by 2002:a17:90a:2567:: with SMTP id j94mr24874816pje.121.1561381376256; Mon, 24 Jun 2019 06:02:56 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id g2sm14061136pfb.95.2019.06.24.06.02.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:02:55 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:02:50 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 02/10] read-cache.c: dump common extension info in json Date: Mon, 24 Jun 2019 20:02:18 +0700 Message-Id: <20190624130226.17293-3-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 49 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 14 deletions(-) diff --git a/read-cache.c b/read-cache.c index db5147d088..4accd8bb08 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1694,8 +1694,26 @@ static int verify_hdr(const struct cache_header *hdr, unsigned long size) } static int read_index_extension(struct index_state *istate, - const char *ext, const char *data, unsigned long sz) + const char *map, + unsigned long *offset) { + int ret = 0; + const char *ext = map + *offset; + uint32_t sz = get_be32(ext + 4); + const char *data = ext + 8; + + if (istate->jw) { + char buf[5]; + + memcpy(buf, ext, 4); + buf[4] = '\0'; + jw_object_inline_begin_object(istate->jw, buf); + + jw_object_intmax(istate->jw, "file_offset", *offset); + jw_object_intmax(istate->jw, "ext_size", sz); + } + *offset += sz + 8; + switch (CACHE_EXT(ext)) { case CACHE_EXT_TREE: istate->cache_tree = cache_tree_read(data, sz); @@ -1704,8 +1722,7 @@ static int read_index_extension(struct index_state *istate, istate->resolve_undo = resolve_undo_read(data, sz); break; case CACHE_EXT_LINK: - if (read_link_extension(istate, data, sz)) - return -1; + ret = read_link_extension(istate, data, sz); break; case CACHE_EXT_UNTRACKED: istate->untracked = read_untracked_extension(data, sz); @@ -1719,12 +1736,14 @@ static int read_index_extension(struct index_state *istate, break; default: if (*ext < 'A' || 'Z' < *ext) - return error(_("index uses %.4s extension, which we do not understand"), + ret = error(_("index uses %.4s extension, which we do not understand"), ext); - fprintf_ln(stderr, _("ignoring %.4s extension"), ext); + else + fprintf_ln(stderr, _("ignoring %.4s extension"), ext); break; } - return 0; + jw_end_gently(istate->jw); + return ret; } static struct cache_entry *create_from_disk(struct mem_pool *ce_mem_pool, @@ -1930,25 +1949,27 @@ static void *load_index_extensions(void *_data) { struct load_index_extensions *p = _data; unsigned long src_offset = p->src_offset; + int dump_json = 0; while (src_offset <= p->mmap_size - the_hash_algo->rawsz - 8) { - /* After an array of active_nr index entries, + if (p->istate->jw && !dump_json) { + jw_object_inline_begin_object(p->istate->jw, "extensions"); + dump_json = 1; + } + /* + * After an array of active_nr index entries, * there can be arbitrary number of extended * sections, each of which is prefixed with * extension name (4-byte) and section length * in 4-byte network byte order. */ - uint32_t extsize = get_be32(p->mmap + src_offset + 4); - if (read_index_extension(p->istate, - p->mmap + src_offset, - p->mmap + src_offset + 8, - extsize) < 0) { + if (read_index_extension(p->istate, p->mmap, &src_offset) < 0) { munmap((void *)p->mmap, p->mmap_size); die(_("index file corrupt")); } - src_offset += 8; - src_offset += extsize; } + if (dump_json) + jw_end(p->istate->jw); return NULL; } From patchwork Mon Jun 24 13:02:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013035 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB30176 for ; Mon, 24 Jun 2019 13:03:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB804287A1 for ; Mon, 24 Jun 2019 13:03:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AFEA428AD6; Mon, 24 Jun 2019 13:03:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25031288BD for ; Mon, 24 Jun 2019 13:03:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730476AbfFXNDE (ORCPT ); Mon, 24 Jun 2019 09:03:04 -0400 Received: from mail-pl1-f193.google.com ([209.85.214.193]:39396 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729130AbfFXNDD (ORCPT ); Mon, 24 Jun 2019 09:03:03 -0400 Received: by mail-pl1-f193.google.com with SMTP id b7so6875195pls.6 for ; Mon, 24 Jun 2019 06:03:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hS84pD7SLtkTGLA9zIhMVzeq4ScnuuDvladmMWsLCRw=; b=uw9t6Ggi+ddbwgHQFIy4sh+jT/RYg6m/XlrhXhpXz3ywTCbDGwCxXlHLp2naIvW5tn ZaGFkBrM1eAGHsboe/0nYlfo0CZ9YvlHGpSs29tvUGr7AOIodZ80a3SX2XGetbcsqPbK gQqqITpvJpPXwhk3J43dHmGYqkiXJ/3fdXWbmyZuJq/4EPDDDBxCdWp92fBWQZeYgQhr Cr9f/TJrjP8cG2XiYtXxUYIVAgKRS3Ue/mVXUIjwpBLISC8uwTUuWGCnu6VHiQck3+g9 YDHr78gNRLIXgT3qFQb7a/McFPXqDo7a9j5umRj4BewMiiaq5Xyzi+QLyqovwHt9kjDE 2qGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hS84pD7SLtkTGLA9zIhMVzeq4ScnuuDvladmMWsLCRw=; b=b5SzWvX0xRQEEaLH4jq30uQiwgZBJAvwko/wUy+ly9/f5BQ/UOH8mQhRWB09BrcKIs JS4zU9Q09MBEfEmv6ArS5SrOQEq+lFselfKF9u/FfhbzfDDFNF4gu8mOeif76h95cUDb EbXsg0BRiXo+g8tP4TyIuNXKR2rThgyYTLbecp2m+GRgg6UlAsNlNqDUOH5bO0pD1TfJ Dc40iXe61xkD2nybi0XVvccuKdDYIsRJqWSBx2Sm+rzUTkV59+KPU7gdNB7JWTAifh4a kmKULjfRD7utc4B2WzIpunKkHVkBFJEPlMqzOoPmsfl/lxuRMFDX1DyL6rdH0/Fo7cS8 Q+ew== X-Gm-Message-State: APjAAAUfqpze+QdTbnae3S4uzPpnbROJXWHQ4Zt74B4zHtviHQG8CH0G iEclCxci70djgiCc8mo3ZQbgHzpa X-Google-Smtp-Source: APXvYqxkFugBs8DAISh9mLGKwongXU2yPKUSLjip8tf2PLomV5cVaMIo+2J0GhSTCqdr7GuiNeFM2w== X-Received: by 2002:a17:902:20b:: with SMTP id 11mr15259622plc.78.1561381382809; Mon, 24 Jun 2019 06:03:02 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id f7sm12067386pfd.43.2019.06.24.06.02.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:03:02 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:02:56 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 03/10] cache-tree.c: dump "TREE" extension as json Date: Mon, 24 Jun 2019 20:02:19 +0700 Message-Id: <20190624130226.17293-4-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- cache-tree.c | 36 +++++++++++++++++++++++++++++++----- cache-tree.h | 5 ++++- read-cache.c | 2 +- t/t3011-ls-files-json.sh | 4 +++- t/t3011/basic | 20 +++++++++++++++++++- 5 files changed, 58 insertions(+), 9 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index b13bfaf71e..b6a233307e 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -3,6 +3,7 @@ #include "tree.h" #include "tree-walk.h" #include "cache-tree.h" +#include "json-writer.h" #include "object-store.h" #include "replace-object.h" @@ -492,7 +493,8 @@ void cache_tree_write(struct strbuf *sb, struct cache_tree *root) write_one(sb, root, "", 0); } -static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) +static struct cache_tree *read_one(const char **buffer, unsigned long *size_p, + struct json_writer *jw) { const char *buf = *buffer; unsigned long size = *size_p; @@ -546,6 +548,15 @@ static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) *buffer, subtree_nr); #endif + if (jw) { + if (it->entry_count >= 0) { + jw_object_string(jw, "oid", oid_to_hex(&it->oid)); + jw_object_intmax(jw, "entry_count", it->entry_count); + } else { + jw_object_null(jw, "oid"); + } + jw_object_inline_begin_array(jw, "subdirs"); + } /* * Just a heuristic -- we do not add directories that often but * we do not want to have to extend it immediately when we do, @@ -559,12 +570,18 @@ static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) struct cache_tree_sub *subtree; const char *name = buf; - sub = read_one(&buf, &size); + if (jw) { + jw_array_inline_begin_object(jw); + jw_object_string(jw, "name", name); + } + sub = read_one(&buf, &size, jw); + jw_end_gently(jw); if (!sub) goto free_return; subtree = cache_tree_sub(it, name); subtree->cache_tree = sub; } + jw_end_gently(jw); if (subtree_nr != it->subtree_nr) die("cache-tree: internal error"); *buffer = buf; @@ -576,11 +593,20 @@ static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) return NULL; } -struct cache_tree *cache_tree_read(const char *buffer, unsigned long size) +struct cache_tree *cache_tree_read(const char *buffer, unsigned long size, + struct json_writer *jw) { + struct cache_tree *ret; + + if (jw) { + jw_object_inline_begin_object(jw, "root"); + } if (buffer[0]) - return NULL; /* not the whole tree */ - return read_one(&buffer, &size); + ret = NULL; /* not the whole tree */ + else + ret = read_one(&buffer, &size, jw); + jw_end_gently(jw); + return ret; } static struct cache_tree *cache_tree_find(struct cache_tree *it, const char *path) diff --git a/cache-tree.h b/cache-tree.h index 757bbc48bc..fc3c73284b 100644 --- a/cache-tree.h +++ b/cache-tree.h @@ -6,6 +6,8 @@ #include "tree-walk.h" struct cache_tree; +struct json_writer; + struct cache_tree_sub { struct cache_tree *cache_tree; int count; /* internally used by update_one() */ @@ -28,7 +30,8 @@ void cache_tree_invalidate_path(struct index_state *, const char *); struct cache_tree_sub *cache_tree_sub(struct cache_tree *, const char *); void cache_tree_write(struct strbuf *, struct cache_tree *root); -struct cache_tree *cache_tree_read(const char *buffer, unsigned long size); +struct cache_tree *cache_tree_read(const char *buffer, unsigned long size, + struct json_writer *jw); int cache_tree_fully_valid(struct cache_tree *); int cache_tree_update(struct index_state *, int); diff --git a/read-cache.c b/read-cache.c index 4accd8bb08..d09ce42b9a 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1716,7 +1716,7 @@ static int read_index_extension(struct index_state *istate, switch (CACHE_EXT(ext)) { case CACHE_EXT_TREE: - istate->cache_tree = cache_tree_read(data, sz); + istate->cache_tree = cache_tree_read(data, sz, istate->jw); break; case CACHE_EXT_RESOLVE_UNDO: istate->resolve_undo = resolve_undo_read(data, sz); diff --git a/t/t3011-ls-files-json.sh b/t/t3011-ls-files-json.sh index 97bcd814be..fc313f2c9a 100755 --- a/t/t3011-ls-files-json.sh +++ b/t/t3011-ls-files-json.sh @@ -29,6 +29,8 @@ test_expect_success 'setup' ' echo 2 >sub/two && git add sub/two && + git commit -m first && + echo intent-to-add >ita && git add -N ita && @@ -37,7 +39,7 @@ test_expect_success 'setup' ' strip_string oid ident ' -test_expect_success 'ls-files --json, main entries' ' +test_expect_success 'ls-files --json, main entries and TREE' ' compare_json basic ' diff --git a/t/t3011/basic b/t/t3011/basic index 9436445d90..e27f5be5ff 100644 --- a/t/t3011/basic +++ b/t/t3011/basic @@ -63,5 +63,23 @@ }, "file_offset": } - ] + ], + "extensions": { + "TREE": { + "file_offset": , + "ext_size": , + "root": { + "oid": null, + "subdirs": [ + { + "name": "sub", + "oid": , + "entry_count": 1, + "subdirs": [ + ] + } + ] + } + } + } } From patchwork Mon Jun 24 13:02:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1628A76 for ; Mon, 24 Jun 2019 13:03:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 081432842E for ; Mon, 24 Jun 2019 13:03:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F05FB28779; Mon, 24 Jun 2019 13:03:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4EF652842E for ; Mon, 24 Jun 2019 13:03:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730506AbfFXNDJ (ORCPT ); Mon, 24 Jun 2019 09:03:09 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:45278 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729130AbfFXNDJ (ORCPT ); Mon, 24 Jun 2019 09:03:09 -0400 Received: by mail-pl1-f196.google.com with SMTP id bi6so6856061plb.12 for ; Mon, 24 Jun 2019 06:03:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mQmITDvZW2CXGzoSuENvM0CTFqjQaG5lFvaTu8HDWV4=; b=YFif0D9rGIipysT4x72Qzo4H5kdOtugqUwPMEREorx1/sNGIbIq16LXP1JBi1ELO2m e2ih6kwUAqHvuvqnt6QDU73Enw+6xgaa+8zSr1n6Kc4u+eYWjQXtUYK8fleo2778FZ8J XtrHuqhM6Twr54lxqKoOr4fIxrL2sCa0lWRdck0bSVRUWpDD7tOEUKsAGSltFXIeZb/6 AMlFSKxBv0/KoDW2+Q/XSSXuqsOHsc+ke9+oHmwv1zWMeecpn7q4y33+UEM9wgSHW257 U9AIkYogNKzbgBRtViDOQdvptI01dCiCJVRb9mkuFmmmgeHMnm2v1u5OCw45xfA7cM1Q J0Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mQmITDvZW2CXGzoSuENvM0CTFqjQaG5lFvaTu8HDWV4=; b=HSdqLC4U3GRgZThHJjcEcOpCRQKeorpW+vO7qtmCFPHKeadhziEeUw3BYNWyqyElud swErijWttYC3j+v5VicKcsWFLj9DW9uP+0fiaCIPv6vPbajaBfbgjPQB6ogrujb3rt1X kJfpVRbswS2IM/8ygAHBOwavFL7XrC9wOzKFso6njDDDm19T/JZh1gSPJeSkOo7Zy9X6 4vk0xr89ZDig9ohcS/EtDrYQhXphNdNOr5LXpV00cB3J7FXnh+8CTPHED7w31782VZKS vFa7lOU3RS8CmG+td5dSekpzfYz/i5NdY15Z5OorQekLMqo/wFgRFzFvzalBOMQkTaxg /33A== X-Gm-Message-State: APjAAAVSZ/ZsydztbiUa/0CB9RVrzLkKdqr0QN2XRpJZvTf4/SqsqgEF w3u4ifW7K7Yf4do25fxODlq39LMX X-Google-Smtp-Source: APXvYqxnbd/nLSKMBtMJHSuEtpykls2+AxfPnSKXaeU6GINAX2p4SNJ9gVlHkzM/e3B745yKyX1Bwg== X-Received: by 2002:a17:902:e65:: with SMTP id 92mr141065909plw.13.1561381388222; Mon, 24 Jun 2019 06:03:08 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id x3sm12312646pja.7.2019.06.24.06.03.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:03:07 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:03:03 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 04/10] dir.c: dump "UNTR" extension as json Date: Mon, 24 Jun 2019 20:02:20 +0700 Message-Id: <20190624130226.17293-5-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The big part of UNTR extension is dumped at the end instead of dumping as soon as we read it, because we actually "patch" some fields in untracked_cache_dir with EWAH bitmaps at the end. Signed-off-by: Nguyễn Thái Ngọc Duy --- dir.c | 57 +++++++++++++++++++++++++++++++++++++++- dir.h | 4 ++- json-writer.h | 6 +++++ read-cache.c | 2 +- t/t3011-ls-files-json.sh | 3 ++- t/t3011/basic | 39 +++++++++++++++++++++++++++ 6 files changed, 107 insertions(+), 4 deletions(-) diff --git a/dir.c b/dir.c index ba4a51c296..8808577ea3 100644 --- a/dir.c +++ b/dir.c @@ -19,6 +19,7 @@ #include "varint.h" #include "ewah/ewok.h" #include "fsmonitor.h" +#include "json-writer.h" #include "submodule-config.h" /* @@ -2826,7 +2827,42 @@ static void load_oid_stat(struct oid_stat *oid_stat, const unsigned char *data, oid_stat->valid = 1; } -struct untracked_cache *read_untracked_extension(const void *data, unsigned long sz) +static void jw_object_oid_stat(struct json_writer *jw, const char *key, + const struct oid_stat *oid_stat) +{ + jw_object_inline_begin_object(jw, key); + jw_object_bool(jw, "valid", oid_stat->valid); + jw_object_string(jw, "oid", oid_to_hex(&oid_stat->oid)); + jw_object_stat_data(jw, "stat", &oid_stat->stat); + jw_end(jw); +} + +static void jw_object_untracked_cache_dir(struct json_writer *jw, + const struct untracked_cache_dir *ucd) +{ + int i; + + jw_object_bool(jw, "valid", ucd->valid); + jw_object_bool(jw, "check-only", ucd->check_only); + jw_object_stat_data(jw, "stat", &ucd->stat_data); + jw_object_string(jw, "exclude-oid", oid_to_hex(&ucd->exclude_oid)); + jw_object_inline_begin_array(jw, "untracked"); + for (i = 0; i < ucd->untracked_nr; i++) + jw_array_string(jw, ucd->untracked[i]); + jw_end(jw); + + jw_object_inline_begin_object(jw, "dirs"); + for (i = 0; i < ucd->dirs_nr; i++) { + jw_object_inline_begin_object(jw, ucd->dirs[i]->name); + jw_object_untracked_cache_dir(jw, ucd->dirs[i]); + jw_end(jw); + } + jw_end(jw); +} + +struct untracked_cache *read_untracked_extension(const void *data, + unsigned long sz, + struct json_writer *jw) { struct untracked_cache *uc; struct read_data rd; @@ -2864,6 +2900,19 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long uc->dir_flags = get_be32(next + ouc_offset(dir_flags)); exclude_per_dir = (const char *)next + exclude_per_dir_offset; uc->exclude_per_dir = xstrdup(exclude_per_dir); + + if (jw) { + jw_object_string(jw, "ident", ident); + jw_object_oid_stat(jw, "info_exclude", &uc->ss_info_exclude); + jw_object_oid_stat(jw, "excludes_file", &uc->ss_excludes_file); + jw_object_intmax(jw, "flags", uc->dir_flags); + if (uc->dir_flags & DIR_SHOW_OTHER_DIRECTORIES) + jw_object_bool(jw, "show_other_directories", 1); + if (uc->dir_flags & DIR_HIDE_EMPTY_DIRECTORIES) + jw_object_bool(jw, "hide_empty_directories", 1); + jw_object_string(jw, "excludes_per_dir", uc->exclude_per_dir); + } + /* NUL after exclude_per_dir is covered by sizeof(*ouc) */ next += exclude_per_dir_offset + strlen(exclude_per_dir) + 1; if (next >= end) @@ -2905,6 +2954,12 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long ewah_each_bit(rd.sha1_valid, read_oid, &rd); next = rd.data; + if (jw) { + jw_object_inline_begin_object(jw, "root"); + jw_object_untracked_cache_dir(jw, uc->root); + jw_end(jw); + } + done: free(rd.ucd); ewah_free(rd.valid); diff --git a/dir.h b/dir.h index 680079bbe3..80efdd05c4 100644 --- a/dir.h +++ b/dir.h @@ -6,6 +6,8 @@ #include "cache.h" #include "strbuf.h" +struct json_writer; + struct dir_entry { unsigned int len; char name[FLEX_ARRAY]; /* more */ @@ -362,7 +364,7 @@ void untracked_cache_remove_from_index(struct index_state *, const char *); void untracked_cache_add_to_index(struct index_state *, const char *); void free_untracked_cache(struct untracked_cache *); -struct untracked_cache *read_untracked_extension(const void *data, unsigned long sz); +struct untracked_cache *read_untracked_extension(const void *data, unsigned long sz, struct json_writer *jw); void write_untracked_extension(struct strbuf *out, struct untracked_cache *untracked); void add_untracked_cache(struct index_state *istate); void remove_untracked_cache(struct index_state *istate); diff --git a/json-writer.h b/json-writer.h index c48c4cbf33..c3d0fbd1ef 100644 --- a/json-writer.h +++ b/json-writer.h @@ -121,6 +121,12 @@ static inline void jw_object_inline_begin_array_gently(struct json_writer *jw, jw_object_inline_begin_array(jw, name); } +static inline void jw_array_inline_begin_object_gently(struct json_writer *jw) +{ + if (jw) + jw_array_inline_begin_object(jw); +} + static inline void jw_end_gently(struct json_writer *jw) { if (jw) diff --git a/read-cache.c b/read-cache.c index d09ce42b9a..a70df4b0a5 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1725,7 +1725,7 @@ static int read_index_extension(struct index_state *istate, ret = read_link_extension(istate, data, sz); break; case CACHE_EXT_UNTRACKED: - istate->untracked = read_untracked_extension(data, sz); + istate->untracked = read_untracked_extension(data, sz, istate->jw); break; case CACHE_EXT_FSMONITOR: read_fsmonitor_extension(istate, data, sz); diff --git a/t/t3011-ls-files-json.sh b/t/t3011-ls-files-json.sh index fc313f2c9a..082fe8e966 100755 --- a/t/t3011-ls-files-json.sh +++ b/t/t3011-ls-files-json.sh @@ -30,6 +30,7 @@ test_expect_success 'setup' ' git add sub/two && git commit -m first && + git update-index --untracked-cache && echo intent-to-add >ita && git add -N ita && @@ -39,7 +40,7 @@ test_expect_success 'setup' ' strip_string oid ident ' -test_expect_success 'ls-files --json, main entries and TREE' ' +test_expect_success 'ls-files --json, main entries, UNTR and TREE' ' compare_json basic ' diff --git a/t/t3011/basic b/t/t3011/basic index e27f5be5ff..8e049f5350 100644 --- a/t/t3011/basic +++ b/t/t3011/basic @@ -80,6 +80,45 @@ } ] } + }, + "UNTR": { + "file_offset": , + "ext_size": , + "ident": , + "info_exclude": { + "valid": true, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 0 + } + }, + "excludes_file": { + "valid": true, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 0 + } + }, + "flags": 6, + "show_other_directories": true, + "hide_empty_directories": true, + "excludes_per_dir": ".gitignore" } } } From patchwork Mon Jun 24 13:02:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013043 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1CE4176 for ; Mon, 24 Jun 2019 13:03:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D44028BB1 for ; Mon, 24 Jun 2019 13:03:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 010FA28BB3; Mon, 24 Jun 2019 13:03:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75F2928852 for ; Mon, 24 Jun 2019 13:03:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730566AbfFXNDP (ORCPT ); Mon, 24 Jun 2019 09:03:15 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:42860 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728686AbfFXNDO (ORCPT ); Mon, 24 Jun 2019 09:03:14 -0400 Received: by mail-pg1-f193.google.com with SMTP id k13so925295pgq.9 for ; Mon, 24 Jun 2019 06:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5ilEyhx7a3dmSnJVVCND1Jd0Y+WKmiJh5t+FI6c13Jg=; b=AAZt3xgNaOHS5ruROcwMEi//m+LZnKmTMkH3dshNRslcO1AODwKTgiOFHbTQLy8XUg XCcpW/ftnoHRy3rB7F+zNuUWGoSWg5HtM/hhhY7dDj7E8P6mbnwq4lizeURa0fEt1XkB xe8bTNlKhKLH23HPyVymLNsosPKWSTp3pSr1CFnn9LHscWAt4CSD/dnrU+zCBOxWIufr Ow6DO5VEGeVr+cmWiMypswjzmn0R8is+ELzSR5Cued9m3m+ZnEuJl5JP2r88LAeM/ddV tEGtAvOypo87dvLCepiR9Mb9ybR1U+iCJo6WehCq9nbNzSHUG/gKPlUVrQXXQjsLsQh5 JEpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5ilEyhx7a3dmSnJVVCND1Jd0Y+WKmiJh5t+FI6c13Jg=; b=sTuV93ZHjbDmA5Ix4ZR2RSm2UnYmzusBDWFiZ+SsOnHfI9wiavrilE9GxeN+EGJzt7 LjTW2P5UmajizBZs19K/oXNVAKvD1S6nCvzv70xhUd205DtxkJ38gmqNvzf0kODnPG4H T40s3Y8GIJx8PoUr2eOsCh6L8cdWdNvqCxq/qwPwF6fbpLX+sQqB6VrTix2QaDcCvxcz OYSZ4x7D7Y+AV8ktMYo7G3MzGpQkoaCPf8BVLyCNfopmA1IG3/aiDZCnHP9UZb+GudWh OWfahfMlhFMnP18VuZW0KEzEEayMJGznaUnsosuzxowynmJdy8weo6f7lzgaEPL0w5az IOaw== X-Gm-Message-State: APjAAAXQb8yEuYKST28IwB45sBLxEquh5Ule3Ouh1FnyZiHf0ifWWUUa eGHm5WaQQk9P7XRtezB7p3OBmlpt X-Google-Smtp-Source: APXvYqxULM3dIksRhyvAuLTZ7/MrZ6FOKEUbxOB9BSVHhxRa+hm4j459RgIQ6WWLb51Qy6ZqVPmb7Q== X-Received: by 2002:a63:6b46:: with SMTP id g67mr25169313pgc.45.1561381393608; Mon, 24 Jun 2019 06:03:13 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id y5sm10998397pgv.12.2019.06.24.06.03.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:03:12 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:03:08 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 05/10] split-index.c: dump "link" extension as json Date: Mon, 24 Jun 2019 20:02:21 +0700 Message-Id: <20190624130226.17293-6-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- json-writer.c | 14 ++++++++++++++ json-writer.h | 3 +++ split-index.c | 9 ++++++++- t/t3011-ls-files-json.sh | 14 ++++++++++++++ t/t3011/split-index (new) | 39 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 78 insertions(+), 1 deletion(-) diff --git a/json-writer.c b/json-writer.c index 0608726512..c0bd302e4e 100644 --- a/json-writer.c +++ b/json-writer.c @@ -1,4 +1,5 @@ #include "cache.h" +#include "ewah/ewok.h" #include "json-writer.h" void jw_init(struct json_writer *jw) @@ -224,6 +225,19 @@ void jw_object_stat_data(struct json_writer *jw, const char *name, jw_end(jw); } +static void dump_ewah_one(size_t pos, void *jw) +{ + jw_array_intmax(jw, pos); +} + +void jw_object_ewah(struct json_writer *jw, const char *key, + struct ewah_bitmap *ewah) +{ + jw_object_inline_begin_array(jw, key); + ewah_each_bit(ewah, dump_ewah_one, jw); + jw_end(jw); +} + static void increase_indent(struct strbuf *sb, const struct json_writer *jw, int indent) diff --git a/json-writer.h b/json-writer.h index c3d0fbd1ef..07d841d52a 100644 --- a/json-writer.h +++ b/json-writer.h @@ -45,6 +45,7 @@ #include "git-compat-util.h" #include "strbuf.h" +struct ewah_bitmap; struct stat_data; struct json_writer @@ -87,6 +88,8 @@ void jw_object_null(struct json_writer *jw, const char *key); void jw_object_filemode(struct json_writer *jw, const char *key, mode_t value); void jw_object_stat_data(struct json_writer *jw, const char *key, const struct stat_data *sd); +void jw_object_ewah(struct json_writer *jw, const char *key, + struct ewah_bitmap *ewah); void jw_object_sub_jw(struct json_writer *jw, const char *key, const struct json_writer *value); diff --git a/split-index.c b/split-index.c index e6154e4ea9..41552bf771 100644 --- a/split-index.c +++ b/split-index.c @@ -1,4 +1,5 @@ #include "cache.h" +#include "json-writer.h" #include "split-index.h" #include "ewah/ewok.h" @@ -25,7 +26,7 @@ int read_link_extension(struct index_state *istate, data += the_hash_algo->rawsz; sz -= the_hash_algo->rawsz; if (!sz) - return 0; + goto done; si->delete_bitmap = ewah_new(); ret = ewah_read_mmap(si->delete_bitmap, data, sz); if (ret < 0) @@ -38,6 +39,12 @@ int read_link_extension(struct index_state *istate, return error("corrupt replace bitmap in link extension"); if (ret != sz) return error("garbage at the end of link extension"); +done: + if (istate->jw) { + jw_object_string(istate->jw, "oid", oid_to_hex(&si->base_oid)); + jw_object_ewah(istate->jw, "delete_bitmap", si->delete_bitmap); + jw_object_ewah(istate->jw, "replace_bitmap", si->replace_bitmap); + } return 0; } diff --git a/t/t3011-ls-files-json.sh b/t/t3011-ls-files-json.sh index 082fe8e966..dbb572ce9d 100755 --- a/t/t3011-ls-files-json.sh +++ b/t/t3011-ls-files-json.sh @@ -44,4 +44,18 @@ test_expect_success 'ls-files --json, main entries, UNTR and TREE' ' compare_json basic ' +test_expect_success 'ls-files --json, split index' ' + git init split && + ( + cd split && + echo one >one && + git add one && + git update-index --split-index && + echo updated >>one && + test_must_fail git -c splitIndex.maxPercentChange=100 update-index --refresh && + cp ../filter.sed . && + compare_json split-index + ) +' + test_done diff --git a/t/t3011/split-index b/t/t3011/split-index new file mode 100644 index 0000000000..cdcc4ddded --- /dev/null +++ b/t/t3011/split-index @@ -0,0 +1,39 @@ +{ + "version": 2, + "oid": , + "mtime_sec": , + "mtime_nsec": , + "entries": [ + { + "id": 0, + "name": "", + "mode": "100644", + "flags": 0, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 4 + }, + "file_offset": + } + ], + "extensions": { + "link": { + "file_offset": , + "ext_size": , + "oid": , + "delete_bitmap": [ + ], + "replace_bitmap": [ + 0 + ] + } + } +} From patchwork Mon Jun 24 13:02:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013047 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E08AD1580 for ; Mon, 24 Jun 2019 13:03:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF0D4288BD for ; Mon, 24 Jun 2019 13:03:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C127F2842E; Mon, 24 Jun 2019 13:03:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 62C42288BD for ; Mon, 24 Jun 2019 13:03:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730569AbfFXNDU (ORCPT ); Mon, 24 Jun 2019 09:03:20 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:33862 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728425AbfFXNDT (ORCPT ); Mon, 24 Jun 2019 09:03:19 -0400 Received: by mail-pg1-f194.google.com with SMTP id p10so7114864pgn.1 for ; Mon, 24 Jun 2019 06:03:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mrQ8p9NRDRcPK8HvPOBRujSFKyHRUH2QCCFUlkr783A=; b=ft96zKdfwzd8LrID+JrRdMkFw7iMSkQiLYBAe3xxtHdK3iRVgfL6MhLScgJ5439VWD afl3zZoYPRG7zfLD+uX4Gq3b5qUCb30lvozSfEoqFb+jC+QEwA8R5nFLZHjohG82CCp6 W1bOGAWNNnSyN1NeoWkz3v7fQev7ZOKPQmRwVBjd9HD5rDIsjZekTtvP0fVbPumxAOia tj+0aD3FULdUw/luIakCx4ATR/H5jKN+K+V4nW1G2z3Yp02pRtKq4HTKFKrTZlezRyyR Y6ozH2DfAf1PFpgsZaXVn9NlsY08/G3CglZ6u7706uqh9yffCJ/Zu4ZyJdbOqlyRzNmK UM1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mrQ8p9NRDRcPK8HvPOBRujSFKyHRUH2QCCFUlkr783A=; b=X4pXyZNV4qT2Hboh/IGYFc/5vVbih3WweCyPvCeKMNCTJ5sde6kSsiQVRlnd6MKe2N FaarUzD2h7gC6eQPsH0/711f47Sb5hjoRVFTeWLPsrClxT70ELn2sd0/aFui/9NU521t bdj3JW8KUykhLGKS1UmhCeEB1SWKl2FYW5lGSO3aIFO6Smx6gvw7Xen4p8U53V9VqHm4 aH2hbkqGCbjyPUWOraT0UqMXJhZxBtN1SMKLN0w2mAlEuJNMUeg6IdGCAyW9CMM/stgL l7S7dGO7GpR9dE4r6MpviV6gzkPRpvfUN2It/X0/K1il5pvCW1fjnrqEyPi3EzbCTKcG oImQ== X-Gm-Message-State: APjAAAXdwfPq4zzmJl/+3/5BCM3bS7TGUf9Nrwt2q9XL9Oz4BLcyubaP 2m0uqgZR4V2JC1WHN1h8NvPYxHHT X-Google-Smtp-Source: APXvYqzochGVQStd3OBmAS0GfuqGJ9NMTIxCbwekSWT5/bjLP2G/5xJOonmw/b60J4REhp1gx1VSMQ== X-Received: by 2002:a63:a506:: with SMTP id n6mr27996632pgf.161.1561381398889; Mon, 24 Jun 2019 06:03:18 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id r7sm15781722pfl.134.2019.06.24.06.03.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:03:18 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:03:13 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 06/10] fsmonitor.c: dump "FSMN" extension as json Date: Mon, 24 Jun 2019 20:02:22 +0700 Message-Id: <20190624130226.17293-7-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- fsmonitor.c | 6 ++++++ t/t3011-ls-files-json.sh | 14 +++++++++++++- t/t3011/fsmonitor (new) | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+), 1 deletion(-) diff --git a/fsmonitor.c b/fsmonitor.c index 1dee0aded1..5ed55ad176 100644 --- a/fsmonitor.c +++ b/fsmonitor.c @@ -3,6 +3,7 @@ #include "dir.h" #include "ewah/ewok.h" #include "fsmonitor.h" +#include "json-writer.h" #include "run-command.h" #include "strbuf.h" @@ -50,6 +51,11 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data, } istate->fsmonitor_dirty = fsmonitor_dirty; + if (istate->jw) { + jw_object_intmax(istate->jw, "version", hdr_version); + jw_object_intmax(istate->jw, "last_update", istate->fsmonitor_last_update); + jw_object_ewah(istate->jw, "dirty", fsmonitor_dirty); + } trace_printf_key(&trace_fsmonitor, "read fsmonitor extension successful"); return 0; } diff --git a/t/t3011-ls-files-json.sh b/t/t3011-ls-files-json.sh index dbb572ce9d..25215f83ae 100755 --- a/t/t3011-ls-files-json.sh +++ b/t/t3011-ls-files-json.sh @@ -36,7 +36,7 @@ test_expect_success 'setup' ' git add -N ita && strip_number ctime_sec ctime_nsec mtime_sec mtime_nsec && - strip_number device inode uid gid file_offset ext_size && + strip_number device inode uid gid file_offset ext_size last_update && strip_string oid ident ' @@ -58,4 +58,16 @@ test_expect_success 'ls-files --json, split index' ' ) ' +test_expect_success 'ls-files --json, fsmonitor extension ' ' + git init fsmonitor && + ( + cd fsmonitor && + echo one >one && + git add one && + git update-index --fsmonitor && + cp ../filter.sed . && + compare_json fsmonitor + ) +' + test_done diff --git a/t/t3011/fsmonitor b/t/t3011/fsmonitor new file mode 100644 index 0000000000..17f2d4a664 --- /dev/null +++ b/t/t3011/fsmonitor @@ -0,0 +1,38 @@ +{ + "version": 2, + "oid": , + "mtime_sec": , + "mtime_nsec": , + "entries": [ + { + "id": 0, + "name": "one", + "mode": "100644", + "flags": 0, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 4 + }, + "file_offset": + } + ], + "extensions": { + "FSMN": { + "file_offset": , + "ext_size": , + "version": 1, + "last_update": , + "dirty": [ + 0 + ] + } + } +} From patchwork Mon Jun 24 13:02:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013049 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E31601580 for ; Mon, 24 Jun 2019 13:03:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D5713287A1 for ; Mon, 24 Jun 2019 13:03:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C9C45288E1; Mon, 24 Jun 2019 13:03:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4000328BB8 for ; Mon, 24 Jun 2019 13:03:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729357AbfFXNDZ (ORCPT ); Mon, 24 Jun 2019 09:03:25 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:39440 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728375AbfFXNDZ (ORCPT ); Mon, 24 Jun 2019 09:03:25 -0400 Received: by mail-pl1-f196.google.com with SMTP id b7so6875755pls.6 for ; Mon, 24 Jun 2019 06:03:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PecPMyJROs8IrDFUONUHNmYQPhRzwtbSdECZHEdJ1kg=; b=fzLGc79wddlha+wAtuM4SeQ4EXZqVWg+26FHFXigsuvj+VrftHOlHlBBO3jtzLnNCM OQ06jKKROtwLiB7Zn6f5xORafnNmdktkuQ5VihuvUm9sWy8AostPPbsmWpmO8HVZTj0I c9LMO5UlCcEiUjz3cUe0jSuwg2Ssa1JCGKTH1cGJtv+i6OY1/k209HYxEnfeKkTXPfOu EI1sK05DXwgI8ltFlcd3lEltDk7KYYN9sm9PSDPHDtlgUoxAgWNUfTGYuXi26wntlMlD HD8DVuCpk7Ri9L9EXCL4x5223U2mcGqWVisDIS38cbC9FVCvxhOg69P/8Kzebfexkqoj mJ2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PecPMyJROs8IrDFUONUHNmYQPhRzwtbSdECZHEdJ1kg=; b=ovvwtfcCTnSLugFZMZnfcHyVifhbLdK3nCfzat7lYfep25R5tNvs7jQbC0iCuQH9nm bGgXKJEI+QusH9eCCSt85lRF33oITTClusdTajy5gwluPH375leuGbqRkX9AopaBp4qW qI2ctgUDbcFysEg/W3C5TwF1CxobZ/4N0aADJIXtOQ5KRiEk9JKTK9U7u1CV7sWKZ0cs 88qJibjvxMX6n5uoXzIXkg1j8mTsPPSHTb5ENkxUXIuxn8jnpR7Jnnpza4I8kTBAM+TF m+t0FZKzTUZ3iedvwiUVsSSD/McKnzE2cEEqakZK7K2T9oSLzg+Q0ycorPJklQusVpum Dqqw== X-Gm-Message-State: APjAAAVrtbwjxknSy3LWizT86HX5N5GFVkw5ke1kclKKmeKDm5cjmuje AXrd34BzCK7Fxj/Pjufd+gC8Pw+8 X-Google-Smtp-Source: APXvYqzuIYgcJB45h6knmNKBxoYT3IAhjYHZUYQiQi05FCoT+GBrsKxHiP8SNbFg/6s1zx0fEydu3A== X-Received: by 2002:a17:902:8c83:: with SMTP id t3mr120534378plo.93.1561381404243; Mon, 24 Jun 2019 06:03:24 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id f15sm13292106pje.17.2019.06.24.06.03.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:03:23 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:03:19 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 07/10] resolve-undo.c: dump "REUC" extension as json Date: Mon, 24 Jun 2019 20:02:23 +0700 Message-Id: <20190624130226.17293-8-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 2 +- resolve-undo.c | 30 +++++++++++++++++- resolve-undo.h | 4 ++- t/t3011-ls-files-json.sh | 20 ++++++++++++ t/t3011/rerere (new) | 66 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 119 insertions(+), 3 deletions(-) diff --git a/read-cache.c b/read-cache.c index a70df4b0a5..e5183636fc 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1719,7 +1719,7 @@ static int read_index_extension(struct index_state *istate, istate->cache_tree = cache_tree_read(data, sz, istate->jw); break; case CACHE_EXT_RESOLVE_UNDO: - istate->resolve_undo = resolve_undo_read(data, sz); + istate->resolve_undo = resolve_undo_read(data, sz, istate->jw); break; case CACHE_EXT_LINK: ret = read_link_extension(istate, data, sz); diff --git a/resolve-undo.c b/resolve-undo.c index 236320f179..68921e3dfe 100644 --- a/resolve-undo.c +++ b/resolve-undo.c @@ -1,5 +1,6 @@ #include "cache.h" #include "dir.h" +#include "json-writer.h" #include "resolve-undo.h" #include "string-list.h" @@ -49,7 +50,30 @@ void resolve_undo_write(struct strbuf *sb, struct string_list *resolve_undo) } } -struct string_list *resolve_undo_read(const char *data, unsigned long size) +static void dump_resolve_undo(struct json_writer *jw, + const char *path, + const struct resolve_undo_info *ui) +{ + int i; + + if (!jw) + return; + + jw_array_inline_begin_object(jw); + jw_object_string(jw, "path", path); + + jw_object_inline_begin_array(jw, "stages"); + for (i = 0; i < 3; i++) { + jw_array_inline_begin_object(jw); + jw_object_filemode(jw, "mode", ui->mode[i]); + jw_object_string(jw, "oid", oid_to_hex(&ui->oid[i])); + jw_end(jw); + } + jw_end(jw); +} + +struct string_list *resolve_undo_read(const char *data, unsigned long size, + struct json_writer *jw) { struct string_list *resolve_undo; size_t len; @@ -59,6 +83,7 @@ struct string_list *resolve_undo_read(const char *data, unsigned long size) resolve_undo = xcalloc(1, sizeof(*resolve_undo)); resolve_undo->strdup_strings = 1; + jw_object_inline_begin_array_gently(jw, "entries"); while (size) { struct string_list_item *lost; @@ -94,7 +119,10 @@ struct string_list *resolve_undo_read(const char *data, unsigned long size) size -= rawsz; data += rawsz; } + + dump_resolve_undo(jw, lost->string, ui); } + jw_end_gently(jw); return resolve_undo; error: diff --git a/resolve-undo.h b/resolve-undo.h index 2b3f0f901e..46b4e93a7e 100644 --- a/resolve-undo.h +++ b/resolve-undo.h @@ -3,6 +3,8 @@ #include "cache.h" +struct json_writer; + struct resolve_undo_info { unsigned int mode[3]; struct object_id oid[3]; @@ -10,7 +12,7 @@ struct resolve_undo_info { void record_resolve_undo(struct index_state *, struct cache_entry *); void resolve_undo_write(struct strbuf *, struct string_list *); -struct string_list *resolve_undo_read(const char *, unsigned long); +struct string_list *resolve_undo_read(const char *, unsigned long, struct json_writer *); void resolve_undo_clear_index(struct index_state *); int unmerge_index_entry_at(struct index_state *, int); void unmerge_index(struct index_state *, const struct pathspec *); diff --git a/t/t3011-ls-files-json.sh b/t/t3011-ls-files-json.sh index 25215f83ae..dc57138f5b 100755 --- a/t/t3011-ls-files-json.sh +++ b/t/t3011-ls-files-json.sh @@ -70,4 +70,24 @@ test_expect_success 'ls-files --json, fsmonitor extension ' ' ) ' +test_expect_success 'ls-files --json, rerere extension' ' + git init rerere && + ( + cd rerere && + mkdir fi && + test_commit initial fi/le first && + git branch side && + test_commit second fi/le second && + git checkout side && + test_commit third fi/le third && + git checkout master && + git config rerere.enabled true && + test_must_fail git merge side && + echo resolved >fi/le && + git add fi/le && + cp ../filter.sed . && + compare_json rerere + ) +' + test_done diff --git a/t/t3011/rerere b/t/t3011/rerere new file mode 100644 index 0000000000..a8ec4b16ee --- /dev/null +++ b/t/t3011/rerere @@ -0,0 +1,66 @@ +{ + "version": 2, + "oid": , + "mtime_sec": , + "mtime_nsec": , + "entries": [ + { + "id": 0, + "name": "fi/le", + "mode": "100644", + "flags": 0, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 9 + }, + "file_offset": + } + ], + "extensions": { + "TREE": { + "file_offset": , + "ext_size": , + "root": { + "oid": null, + "subdirs": [ + { + "name": "fi", + "oid": null, + "subdirs": [ + ] + } + ] + } + }, + "REUC": { + "file_offset": , + "ext_size": , + "entries": [ + { + "path": "fi/le", + "stages": [ + { + "mode": "100644", + "oid": + }, + { + "mode": "100644", + "oid": + }, + { + "mode": "100644", + "oid": + } + ] + } + ] + } + } From patchwork Mon Jun 24 13:02:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013055 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 231AA1580 for ; Mon, 24 Jun 2019 13:03:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1082C2842E for ; Mon, 24 Jun 2019 13:03:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0308128BB3; Mon, 24 Jun 2019 13:03:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 84E93285E1 for ; Mon, 24 Jun 2019 13:03:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730583AbfFXNDa (ORCPT ); Mon, 24 Jun 2019 09:03:30 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:35154 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730575AbfFXNDa (ORCPT ); Mon, 24 Jun 2019 09:03:30 -0400 Received: by mail-pl1-f196.google.com with SMTP id p1so6877338plo.2 for ; Mon, 24 Jun 2019 06:03:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/2S5IpLFhEb9dDLUGXjl2/30oMiqG8v3lcU0Yp/u58U=; b=iEvjogis4OKDD8VipTR72e24JlmDpGZnduzslo5UAuShg6Mke/yDwGSHaVII3FViQ0 Xmq4SY4S91cPeH0UkNG744MqvbT5Avzhp/bu/M3lQ6X969nGO/QN30DRfxEu1d1RSRdm aro5d3nGfqxwE+TJ6ennrzr5eQnQPrd9hTi9Z3BJQCKh7tKF+c7D5VEAJy61KTMseVeS eZbMks0p8Ip6HwGe6LDJhYCAp1zubFvjkoHhIAoP4lqAliBH/GFyubY9diz6+pBbclMl LRsk9mpsja6NwK7HN6wxypSnIkJOGSb2le68RuRNF6yt7RtUBntAU52ACXVNWG/e+GoJ 97yQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/2S5IpLFhEb9dDLUGXjl2/30oMiqG8v3lcU0Yp/u58U=; b=WIH6ERQ9L0WdUqrNHNmVI/b0ELSCOEWZOHGmlhLi+vzD0J2YsCeF7K58beNHh7acQp SG6wchOK9ghdO54IofsK0BPGvo3bHwq2xCoIpPHoAEKUBLcDaJm1p4m2KUUE0u+2iUpz CSgPceCrk9W1NE55iYKSfu5ZFGlVIC8eutOQx0/0GDIVMDqxuO1IufmZS2+LphbPAJ8E S6i6+x1EWJsnEdj1NL8XJo/ZRN0Ca5i81KrpD010td8QNfy13XhC3vDWRzYc0ZtyFB9e AwHC3f/XokfqjaTRFwdaoJlbU4SylIP0N0ucwPBYIoXroT9lB6FKEg+E+rDi7EWNZjXE c2lQ== X-Gm-Message-State: APjAAAW2XSwEKrLPr8rcv39VQPerLRHcJwslMOLrDnpVgMFJvxblwmzu QunI4C3kl6tURXfxH1edvf9Po/kQ X-Google-Smtp-Source: APXvYqzOWNuNye5lgJgHX8iN9MlqqTsX4FGmh2OHpk8lnKxtaQxE31OuCnvOKHwP1z1NriJi55Ne3Q== X-Received: by 2002:a17:902:110b:: with SMTP id d11mr78458885pla.213.1561381409415; Mon, 24 Jun 2019 06:03:29 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id g2sm10682433pgi.92.2019.06.24.06.03.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:03:28 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:03:24 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 08/10] read-cache.c: dump "EOIE" extension as json Date: Mon, 24 Jun 2019 20:02:24 +0700 Message-Id: <20190624130226.17293-9-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 8 ++++ t/t3011-ls-files-json.sh | 13 ++++++ t/t3011/eoie (new) | 96 ++++++++++++++++++++++++++++++++++++++++ t/test-lib.sh | 4 ++ 4 files changed, 121 insertions(+) diff --git a/read-cache.c b/read-cache.c index e5183636fc..37491dd03d 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1731,6 +1731,14 @@ static int read_index_extension(struct index_state *istate, read_fsmonitor_extension(istate, data, sz); break; case CACHE_EXT_ENDOFINDEXENTRIES: + if (istate->jw) { + /* must be synchronized with read_eoie_extension() */ + jw_object_intmax(istate->jw, "offset", get_be32(data)); + jw_object_string(istate->jw, "oid", + hash_to_hex((const unsigned char*)data + sizeof(uint32_t))); + } + /* already handled in do_read_index() */ + break; case CACHE_EXT_INDEXENTRYOFFSETTABLE: /* already handled in do_read_index() */ break; diff --git a/t/t3011-ls-files-json.sh b/t/t3011-ls-files-json.sh index dc57138f5b..9f4ad4c9cf 100755 --- a/t/t3011-ls-files-json.sh +++ b/t/t3011-ls-files-json.sh @@ -90,4 +90,17 @@ test_expect_success 'ls-files --json, rerere extension' ' ) ' +test_expect_success !SINGLE_CPU 'ls-files --json and multicore extensions' ' + git init eoie && + ( + cd eoie && + git config index.threads 2 && + touch one two three four && + git add . && + cp ../filter.sed . && + strip_number offset && + compare_json eoie + ) +' + test_done diff --git a/t/t3011/eoie b/t/t3011/eoie new file mode 100644 index 0000000000..85ec61517b --- /dev/null +++ b/t/t3011/eoie @@ -0,0 +1,96 @@ +{ + "version": 2, + "oid": , + "mtime_sec": , + "mtime_nsec": , + "entries": [ + { + "id": 0, + "name": "four", + "mode": "100644", + "flags": 0, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 0 + }, + "file_offset": + }, + { + "id": 1, + "name": "one", + "mode": "100644", + "flags": 0, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 0 + }, + "file_offset": + }, + { + "id": 2, + "name": "three", + "mode": "100644", + "flags": 0, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 0 + }, + "file_offset": + }, + { + "id": 3, + "name": "two", + "mode": "100644", + "flags": 0, + "oid": , + "stat": { + "ctime_sec": , + "ctime_nsec": , + "mtime_sec": , + "mtime_nsec": , + "device": , + "inode": , + "uid": , + "gid": , + "size": 0 + }, + "file_offset": + } + ], + "extensions": { + "IEOT": { + "file_offset": , + "ext_size": + }, + "EOIE": { + "file_offset": , + "ext_size": , + "offset": , + "oid": + } + } +} diff --git a/t/test-lib.sh b/t/test-lib.sh index 4b346467df..9d5b273b40 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1611,3 +1611,7 @@ test_lazy_prereq REBASE_P ' test_lazy_prereq FAIL_PREREQS ' test -n "$GIT_TEST_FAIL_PREREQS" ' + +test_lazy_prereq SINGLE_CPU ' + test "$(test-tool online-cpus)" -eq 1 +' From patchwork Mon Jun 24 13:02:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013053 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 41398186E for ; Mon, 24 Jun 2019 13:03:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3114228852 for ; Mon, 24 Jun 2019 13:03:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 25A8F288BD; Mon, 24 Jun 2019 13:03:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9DC9F27FB0 for ; Mon, 24 Jun 2019 13:03:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730629AbfFXNDg (ORCPT ); Mon, 24 Jun 2019 09:03:36 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:33088 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730576AbfFXNDg (ORCPT ); Mon, 24 Jun 2019 09:03:36 -0400 Received: by mail-pf1-f193.google.com with SMTP id x15so7506533pfq.0 for ; Mon, 24 Jun 2019 06:03:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6xp1cwjRAIuDvxzU93ORO/vW6D2vOMdcIrfHaIlX//A=; b=bLY32SnrHssAsIQ3jnRcIcMEhuG0Tk8hUGcWqFAchxcUlE+W2RfOEuuxZPz8p+ntwx mA6h4I65robZG2l5v+fj0+mQLHzOpRLwmZoE8rMi9pHSXkOLlg3uyNfpbWtxAsUg/gdf mE9ooulJ0EC1QKGVOrsGXZC4sT3uPpaD7GoVr9VVDrjfzyNX3XCQKttMJ/+JJdGYW78L S2nH/TBl29QeplVzvQeNPemn2dUFxVKSLd61lTAnb7WvwBAsaWCJ2XxgwWQ3I86liMay gHJCLckKFUPuulkcH55NgIuBVqLdXYs3tnoKGZPJFc0L0cjHQ12fRBUpFI21LLIM3lrH o1gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6xp1cwjRAIuDvxzU93ORO/vW6D2vOMdcIrfHaIlX//A=; b=YG0R4UH67TIPGKjvaJLW+EUwpRDQo+EAwXEMrU4vYjr1LGK6ZtamG2SrL4dbHdqjYR fN9Sao5zGrapNX2pBbsj2RioyyNWcluHOKmyJEhbqSWlUu/yh3iA5rqU70+XKRPHvGFU fISU5ApbRDh+uX8nWiumlDZ0wW+4l34DFUDo5ER+PZ80MD/+xdQMPxINW0soEDnpUgbY IKBtu/MvEhGUh6H9tPBaBrucVBFI74GMzwPi9aPcKezGpqWJIEkyZoFahARhlKY9SHpO pN92+FVAB3/wA3UngNG2mFsmSdxto7Un8fGpNZgYx+wp2QAqSY59FBW5vx3+hy8huNs4 baHg== X-Gm-Message-State: APjAAAWznS2JjiR7+ysasqjEYtN1OUpPfcEjUuP8zVa8TKe6LzeWKT5b 2u3kGEpnbIDiZJeQmSWGQnrY2Qen X-Google-Smtp-Source: APXvYqybMjxG4nObQ76mh+1X7/HfCYi57jMhz+NGVLw3c5OT5PrLK4k48tkKA8jeFX/hctdrvZ0QGA== X-Received: by 2002:a65:55ca:: with SMTP id k10mr71325pgs.14.1561381414683; Mon, 24 Jun 2019 06:03:34 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id n98sm14885550pjc.26.2019.06.24.06.03.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:03:34 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:03:29 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 09/10] read-cache.c: dump "IEOT" extension as json Date: Mon, 24 Jun 2019 20:02:25 +0700 Message-Id: <20190624130226.17293-10-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 43 ++++++++++++++++++++++++++++++++++++------- t/t3011/eoie | 13 ++++++++++++- 2 files changed, 48 insertions(+), 8 deletions(-) diff --git a/read-cache.c b/read-cache.c index 37491dd03d..c26edcc9d9 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1693,6 +1693,7 @@ static int verify_hdr(const struct cache_header *hdr, unsigned long size) return 0; } +static struct index_entry_offset_table *do_read_ieot_extension(struct index_state *, const char *, uint32_t); static int read_index_extension(struct index_state *istate, const char *map, unsigned long *offset) @@ -1740,7 +1741,11 @@ static int read_index_extension(struct index_state *istate, /* already handled in do_read_index() */ break; case CACHE_EXT_INDEXENTRYOFFSETTABLE: - /* already handled in do_read_index() */ + if (istate->jw) { + free(do_read_ieot_extension(istate, data, sz)); + } else { + /* already handled in do_read_index() */ + } break; default: if (*ext < 'A' || 'Z' < *ext) @@ -1938,7 +1943,7 @@ struct index_entry_offset_table struct index_entry_offset entries[FLEX_ARRAY]; }; -static struct index_entry_offset_table *read_ieot_extension(const char *mmap, size_t mmap_size, size_t offset); +static struct index_entry_offset_table *read_ieot_extension(struct index_state *istate, const char *mmap, size_t mmap_size, size_t offset); static void write_ieot_extension(struct strbuf *sb, struct index_entry_offset_table *ieot); static size_t read_eoie_extension(const char *mmap, size_t mmap_size); @@ -2292,7 +2297,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) * to multi-thread the reading of the cache entries. */ if (extension_offset && nr_threads > 1) - ieot = read_ieot_extension(mmap, mmap_size, extension_offset); + ieot = read_ieot_extension(istate, mmap, mmap_size, extension_offset); if (ieot) { src_offset += load_cache_entries_threaded(istate, mmap, mmap_size, nr_threads, ieot); @@ -3634,12 +3639,13 @@ static void write_eoie_extension(struct strbuf *sb, git_hash_ctx *eoie_context, #define IEOT_VERSION (1) -static struct index_entry_offset_table *read_ieot_extension(const char *mmap, size_t mmap_size, size_t offset) +static struct index_entry_offset_table *read_ieot_extension( + struct index_state *istate, + const char *mmap, size_t mmap_size, + size_t offset) { const char *index = NULL; - uint32_t extsize, ext_version; - struct index_entry_offset_table *ieot; - int i, nr; + uint32_t extsize; /* find the IEOT extension */ if (!offset) @@ -3655,6 +3661,17 @@ static struct index_entry_offset_table *read_ieot_extension(const char *mmap, si } if (!index) return NULL; + return do_read_ieot_extension(istate, index, extsize); +} + +static struct index_entry_offset_table *do_read_ieot_extension( + struct index_state *istate, + const char *index, + uint32_t extsize) +{ + uint32_t ext_version; + struct index_entry_offset_table *ieot; + int i, nr; /* validate the version is IEOT_VERSION */ ext_version = get_be32(index); @@ -3670,6 +3687,10 @@ static struct index_entry_offset_table *read_ieot_extension(const char *mmap, si error("invalid number of IEOT entries %d", nr); return NULL; } + if (istate->jw) { + jw_object_intmax(istate->jw, "version", ext_version); + jw_object_inline_begin_array(istate->jw, "entries"); + } ieot = xmalloc(sizeof(struct index_entry_offset_table) + (nr * sizeof(struct index_entry_offset))); ieot->nr = nr; @@ -3678,7 +3699,15 @@ static struct index_entry_offset_table *read_ieot_extension(const char *mmap, si index += sizeof(uint32_t); ieot->entries[i].nr = get_be32(index); index += sizeof(uint32_t); + + if (istate->jw) { + jw_array_inline_begin_object(istate->jw); + jw_object_intmax(istate->jw, "offset", ieot->entries[i].offset); + jw_object_intmax(istate->jw, "count", ieot->entries[i].nr); + jw_end(istate->jw); + } } + jw_end_gently(istate->jw); return ieot; } diff --git a/t/t3011/eoie b/t/t3011/eoie index 85ec61517b..66a0feb3b6 100644 --- a/t/t3011/eoie +++ b/t/t3011/eoie @@ -84,7 +84,18 @@ "extensions": { "IEOT": { "file_offset": , - "ext_size": + "ext_size": , + "version": 1, + "entries": [ + { + "offset": , + "count": 2 + }, + { + "offset": , + "count": 2 + } + ] }, "EOIE": { "file_offset": , From patchwork Mon Jun 24 13:02:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11013057 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4B301580 for ; Mon, 24 Jun 2019 13:03:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A6F2B28761 for ; Mon, 24 Jun 2019 13:03:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9B7BE287BE; Mon, 24 Jun 2019 13:03:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E10B287A1 for ; Mon, 24 Jun 2019 13:03:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730633AbfFXNDm (ORCPT ); Mon, 24 Jun 2019 09:03:42 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:33465 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728919AbfFXNDk (ORCPT ); Mon, 24 Jun 2019 09:03:40 -0400 Received: by mail-pl1-f194.google.com with SMTP id c14so6868518plo.0 for ; Mon, 24 Jun 2019 06:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/kpsTrKeejQKjX//MvqRRoECShM4Yo8nP4Pv1CD9O90=; b=R5bxCJgzR5Uzkw5ZD2T2v6nnQ6gooNWdQZcfjlLX88QLsU3sOYi/rtUW0r3kxrEdzM 7PSvwnVLCBnMHZe51DEgIfhUv269ynBzWHxfHub9EL+5pnBELhVpuSIl/o3w8ABWLaHy 6is/NIvKvaPH5+rbwmO5sIX19fJY6658GHka2WXH6QjUe2NNz4AHVnfzxu103npjJ2yh 3JfSTsbEO1qYsFd9qF30CTm8fbj9h5ySwsEN3mSPZJYKV2jh+PX/x65ofPY0yyzdwTa5 4jlJ4jLD48EuvhBJZ7w2ixZUMUkqfZUrBKt8soqAWuNGFTveIAo+LqdjKM2sfNlz+6l3 Ji5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/kpsTrKeejQKjX//MvqRRoECShM4Yo8nP4Pv1CD9O90=; b=c6Ah6qFl1jC1dgkWqVVGIbRGbMrS9ZWFYZp9fDQ6m8esiObwTSE/oUcPR2MTsgWhYo L2h01gRd8xjt3NgNdHFJL03mYWmaeF+S/eGMAW9GFyLiI/6Ggb/S2SFdV9z+FFy4A8Uc D3QC+EPLSJIF0bzpgEwbNEGzaboxMhs5Qbrv1bxoplXS+N7IEgmw98mRMu8cfnElx4pB GMRpT1q72QMv7DKCTERFlD/i1PjUFc+NctBtBh7Pm2ILmAw15GB1UFKSgqnul4G9tMyA hVi7VTT9HKta6elwl7VI3m1Z5oGlrfcMNbsC2Ia/aCpRYtufx6+0qBedBnwxa2wlbyxy pFjA== X-Gm-Message-State: APjAAAU2FnEYsX0jGZ7+NTfZuPvRk6+l35vLWC7+BQiSl9vsw3pdYUpa loBF3AwHPArEz1V+6yJrzoYLYVrQ X-Google-Smtp-Source: APXvYqy1aah0SnFDQCmX9WmQum+V/QpYm5sZY5/TxyQ6NuIV0mPLZuldjWcKn1HD/BYErREdZqBpPg== X-Received: by 2002:a17:902:2ae7:: with SMTP id j94mr63663961plb.270.1561381419864; Mon, 24 Jun 2019 06:03:39 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id q144sm21115359pfc.103.2019.06.24.06.03.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jun 2019 06:03:39 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Mon, 24 Jun 2019 20:03:35 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFy?= =?utf-8?b?bWFzb24=?= , Johannes Schindelin , =?utf-8?b?Tmd1eeG7hW4g?= =?utf-8?b?VGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH v2 10/10] t3008: use the new SINGLE_CPU prereq Date: Mon, 24 Jun 2019 20:02:26 +0700 Message-Id: <20190624130226.17293-11-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190624130226.17293-1-pclouds@gmail.com> References: <20190624130226.17293-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- t/t3008-ls-files-lazy-init-name-hash.sh | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/t/t3008-ls-files-lazy-init-name-hash.sh b/t/t3008-ls-files-lazy-init-name-hash.sh index 64f047332b..7f918c05f6 100755 --- a/t/t3008-ls-files-lazy-init-name-hash.sh +++ b/t/t3008-ls-files-lazy-init-name-hash.sh @@ -4,15 +4,9 @@ test_description='Test the lazy init name hash with various folder structures' . ./test-lib.sh -if test 1 -eq $($GIT_BUILD_DIR/t/helper/test-tool online-cpus) -then - skip_all='skipping lazy-init tests, single cpu' - test_done -fi - LAZY_THREAD_COST=2000 -test_expect_success 'no buffer overflow in lazy_init_name_hash' ' +test_expect_success !SINGLE_CPU 'no buffer overflow in lazy_init_name_hash' ' ( test_seq $LAZY_THREAD_COST | sed "s/^/a_/" && echo b/b/b &&