diff mbox series

[v4,01/15] scalar: create a rudimentary executable

Message ID 852ec003109b8244e2f9360ec64749779989c4a2.1631630356.git.gitgitgadget@gmail.com (mailing list archive)
State New, archived
Headers show
Series Upstreaming the Scalar command | expand

Commit Message

Johannes Schindelin Sept. 14, 2021, 2:39 p.m. UTC
From: Johannes Schindelin <johannes.schindelin@gmx.de>

The idea of Scalar (https://github.com/microsoft/scalar), and before
that, of VFS for Git, has always been to prove that Git _can_ scale, and
to upstream whatever strategies have been demonstrated to help.

With this patch, we start the journey from that C# project to move what
is left to Git's own `contrib/` directory, reimplementing it in pure C,
with the intention to facilitate integrating the functionality into core
Git all while maintaining backwards-compatibility for existing Scalar
users (which will be much easier when both live in the same worktree).
It has always been the plan to contribute all of the proven strategies
back to core Git.

For example, while the virtual filesystem provided by VFS for Git helped
the team developing the Windows operating system to move onto Git, while
trying to upstream it we realized that it cannot be done: getting the
virtual filesystem to work (which we only managed to implement fully on
Windows, but not on, say, macOS or Linux), and the required server-side
support for the GVFS protocol, made this not quite feasible.

The Scalar project learned from that and tackled the problem with
different tactics: instead of pretending to Git that the working
directory is fully populated, it _specifically_ teaches Git about
partial clone (which is based on VFS for Git's cache server), about
sparse checkout (which VFS for Git tried to do transparently, in the
file system layer), and regularly runs maintenance tasks to keep the
repository in a healthy state.

With partial clone, sparse checkout and `git maintenance` having been
upstreamed, there is little left that `scalar.exe` does which `git.exe`
cannot do. One such thing is that `scalar clone <url>` will
automatically set up a partial, sparse clone, and configure
known-helpful settings from the start.

So let's bring this convenience into Git's tree.

The idea here is that you can (optionally) build Scalar via

	make -C contrib/scalar/Makefile

This will build the `scalar` executable and put it into the
contrib/scalar/ subdirectory.

The slightly awkward addition of the `contrib/scalar/*` bits to the
top-level `Makefile` are actually really required: we want to link to
`libgit.a`, which means that we will need to use the very same `CFLAGS`
and `LDFLAGS` as the rest of Git.

An early development version of this patch tried to replicate all the
conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like
`contrib/svn-fe/Makefile` used to do before it was retired. It turned
out to be quite the whack-a-mole game: the SHA-1-related flags, the
flags enabling/disabling `compat/poll/`, `compat/regex/`,
`compat/win32mmap.c` & friends depending on the current platform... To
put it mildly: it was a major mess.

Instead, this patch makes minimal changes to the top-level `Makefile` so
that the bits in `contrib/scalar/` can be compiled and linked, and
adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a
most minimal way to do the actual compiling.

Note: With this commit, we only establish the infrastructure, no
Scalar functionality is implemented yet; We will do that incrementally
over the next few commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Makefile                  |  8 ++++++++
 contrib/scalar/.gitignore |  2 ++
 contrib/scalar/Makefile   | 34 ++++++++++++++++++++++++++++++++++
 contrib/scalar/scalar.c   | 36 ++++++++++++++++++++++++++++++++++++
 4 files changed, 80 insertions(+)
 create mode 100644 contrib/scalar/.gitignore
 create mode 100644 contrib/scalar/Makefile
 create mode 100644 contrib/scalar/scalar.c

Comments

Ævar Arnfjörð Bjarmason Sept. 24, 2021, 12:52 p.m. UTC | #1
On Tue, Sep 14 2021, Johannes Schindelin via GitGitGadget wrote:


> +int cmd_main(int argc, const char **argv)
> +{
> +	struct strbuf scalar_usage = STRBUF_INIT;
> +	int i;
> +
> +	if (argc > 1) {
> +		argv++;
> +		argc--;
> +
> +		for (i = 0; builtins[i].name; i++)
> +			if (!strcmp(builtins[i].name, argv[0]))
> +				return !!builtins[i].fn(argc, argv);
> +	}
> +
> +	strbuf_addstr(&scalar_usage,
> +		      N_("scalar <command> [<options>]\n\nCommands:\n"));
> +	for (i = 0; builtins[i].name; i++)
> +		strbuf_addf(&scalar_usage, "\t%s\n", builtins[i].name);
> +
> +	usage(scalar_usage.buf);
> +}

In 04/15 you continue and use the parse-options.c API, but here it's the
usage.c API, which is generally being phased out. Any reason for the
difference? It's preferrable not to add new usage() users if we can help
it, I think we'll eventually want to remove it.
Junio C Hamano Sept. 24, 2021, 5:54 p.m. UTC | #2
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> ... the
> usage.c API, which is generally being phased out.

That is news to me.  Any reason why you think so?
Ævar Arnfjörð Bjarmason Sept. 26, 2021, 7:15 p.m. UTC | #3
On Fri, Sep 24 2021, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> ... the
>> usage.c API, which is generally being phased out.
>
> That is news to me.  Any reason why you think so?

Perhaps better phrased as "generally going unused where we're using
parse-options.c", although some quick historical trends I ran as an
ad-hoc show usage() standing still since v1.6.0 in number of builtin*.c
files[1], v.s. a 3x growth in usage_with_options() since then[2].

But in this case we're using an ad-hoc parser in cmd_main(), seemingly
because it ends up copy/pasting a very small part of git.ci
functionality over there, which is something that's been discussed as
something we should move over to parse_options() sooner than later.

It looks like a better approach to just use parse_options() consistently
in scalar.c, as e.g. commit-graph.c, stash.c, multi-pack-index.c
etc. that all implement a similar cmd/subcommand pattern do.

The end-state of duplicating the "-C" and "-c" options from git.c can
then easily be handled by parse-options.c, IIRC the stumbling point in
migrating over git.c was some of the statefulness of other parts
potentially needing incremenatl parsing (i.e. via parse_options_step()
and friends).

1. $ parallel "printf "%s: " {} && git grep -l '\busage\(' {} -- 'builtin*.c' | wc -l" ::: v1.{1..9}.0 v2.{0..32}.0
v1.1.0:0
v1.2.0:0
v1.3.0:0
v1.4.0:20
v1.5.0:51
v1.7.0:35
v1.6.0:40
v1.8.0:34
v2.1.0:32
v1.9.0:33
v2.0.0:33
v2.2.0:32
v2.3.0:32
v2.4.0:32
v2.5.0:32
v2.6.0:31
v2.7.0:31
v2.9.0:29
v2.8.0:30
v2.11.0:29
v2.10.0:29
v2.12.0:29
v2.13.0:29
v2.14.0:31
v2.15.0:31
v2.16.0:31
v2.17.0:31
v2.19.0:32
v2.18.0:31
v2.22.0:31
v2.21.0:32
v2.24.0:31
v2.23.0:31
v2.20.0:32
v2.25.0:30
v2.26.0:30
v2.27.0:30
v2.32.0:29
v2.31.0:30
v2.29.0:31
v2.28.0:29
v2.30.0:31

2. $ parallel "printf "%s: " {} && git grep -l '\busage_with_options\(' {} -- 'builtin*.c' | wc -l" ::: v1.{1..9}.0 v2.{0..32}.0
v1.1.0:0
v1.2.0:0
v1.3.0:0
v1.4.0:0
v1.5.0:0
v1.6.0:19
v1.7.0:33
v1.8.0:42
v2.0.0:42
v1.9.0:42
v2.1.0:43
v2.2.0:43
v2.3.0:43
v2.4.0:43
v2.5.0:44
v2.6.0:45
v2.8.0:44
v2.10.0:45
v2.7.0:44
v2.9.0:45
v2.11.0:45
v2.12.0:45
v2.13.0:46
v2.14.0:47
v2.15.0:47
v2.16.0:47
v2.17.0:47
v2.19.0:50
v2.18.0:49
v2.20.0:52
v2.21.0:52
v2.22.0:53
v2.24.0:54
v2.23.0:54
v2.25.0:56
v2.26.0:55
v2.27.0:55
v2.28.0:55
v2.30.0:58
v2.32.0:60
v2.29.0:58
v2.31.0:58
Junio C Hamano Sept. 27, 2021, 8:32 p.m. UTC | #4
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Fri, Sep 24 2021, Junio C Hamano wrote:
>
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>>
>>> ... the
>>> usage.c API, which is generally being phased out.
>>
>> That is news to me.  Any reason why you think so?
>
> Perhaps better phrased as "generally going unused where we're using
> parse-options.c", ...

Ah, if it was merely an observation of the general trend, then I
agree.  My reaction was primarily "Oh, why is somebody all of a
sudden setting a project decision unilaterally, especially when even
I do not do so very often myself?  Did I miss recent discussion that
resulted in an update to Documentation/ meant for developers?"

Thanks.
diff mbox series

Patch

diff --git a/Makefile b/Makefile
index c3565fc0f8f..2d5c822f7a8 100644
--- a/Makefile
+++ b/Makefile
@@ -2447,6 +2447,10 @@  endif
 .PHONY: objects
 objects: $(OBJECTS)
 
+SCALAR_SOURCES := contrib/scalar/scalar.c
+SCALAR_OBJECTS := $(SCALAR_SOURCES:c=o)
+OBJECTS += $(SCALAR_OBJECTS)
+
 dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)
 dep_dirs := $(addsuffix .depend,$(sort $(dir $(OBJECTS))))
 
@@ -2586,6 +2590,10 @@  $(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)
 
+contrib/scalar/scalar$X: $(SCALAR_OBJECTS) GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
+		$(filter %.o,$^) $(LIBS)
+
 $(LIB_FILE): $(LIB_OBJS)
 	$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^
 
diff --git a/contrib/scalar/.gitignore b/contrib/scalar/.gitignore
new file mode 100644
index 00000000000..ff3d47e84d0
--- /dev/null
+++ b/contrib/scalar/.gitignore
@@ -0,0 +1,2 @@ 
+/*.exe
+/scalar
diff --git a/contrib/scalar/Makefile b/contrib/scalar/Makefile
new file mode 100644
index 00000000000..40c03ad10e1
--- /dev/null
+++ b/contrib/scalar/Makefile
@@ -0,0 +1,34 @@ 
+QUIET_SUBDIR0  = +$(MAKE) -C # space to separate -C and subdir
+QUIET_SUBDIR1  =
+
+ifneq ($(findstring s,$(MAKEFLAGS)),s)
+ifndef V
+	QUIET_SUBDIR0  = +@subdir=
+	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
+			 $(MAKE) $(PRINT_DIR) -C $$subdir
+else
+	export V
+endif
+endif
+
+all:
+
+include ../../config.mak.uname
+-include ../../config.mak.autogen
+-include ../../config.mak
+
+TARGETS = scalar$(X) scalar.o
+GITLIBS = ../../common-main.o ../../libgit.a ../../xdiff/lib.a
+
+all: scalar$X
+
+$(GITLIBS):
+	$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(subst ../../,,$@)
+
+$(TARGETS): $(GITLIBS) scalar.c
+	$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(patsubst %,contrib/scalar/%,$@)
+
+clean:
+	$(RM) $(TARGETS)
+
+.PHONY: all clean FORCE
diff --git a/contrib/scalar/scalar.c b/contrib/scalar/scalar.c
new file mode 100644
index 00000000000..7cff29e0fcd
--- /dev/null
+++ b/contrib/scalar/scalar.c
@@ -0,0 +1,36 @@ 
+/*
+ * The Scalar command-line interface.
+ */
+
+#include "cache.h"
+#include "gettext.h"
+#include "parse-options.h"
+
+static struct {
+	const char *name;
+	int (*fn)(int, const char **);
+} builtins[] = {
+	{ NULL, NULL},
+};
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf scalar_usage = STRBUF_INIT;
+	int i;
+
+	if (argc > 1) {
+		argv++;
+		argc--;
+
+		for (i = 0; builtins[i].name; i++)
+			if (!strcmp(builtins[i].name, argv[0]))
+				return !!builtins[i].fn(argc, argv);
+	}
+
+	strbuf_addstr(&scalar_usage,
+		      N_("scalar <command> [<options>]\n\nCommands:\n"));
+	for (i = 0; builtins[i].name; i++)
+		strbuf_addf(&scalar_usage, "\t%s\n", builtins[i].name);
+
+	usage(scalar_usage.buf);
+}