[v6,00/18] reftable: stop using "git-compat-util.h"

Message

Patrick Steinhardt Feb. 18, 2025, 9:20 a.m. UTC

Hi,

this patch series is the final step to fully decouple the reftable
library from the rest of the Git codebase. The goal of this is to make
the library reusable by other projects like libgit2 by simply copying
over the source files, making Git the canonical upstream for reftable
functionality.

This patch series stops using all kinds of helpers exposed by our
"git-compat-util.h" header and open-codes them instead. In order to keep
us from using these helpers by accident the final step is to pull out
POSIX-related bits and pieces into a new "compat/posix.h" header, which
the reftable library then uses instead of "git-compat-util.h".

Changes in v3:
  - Fix type of `total_read` variable used to track how many bytes we
    have read in `fd_read_lines()`.
  - Drop the patch use root-relative includes again. Let's rather
    discuss this outside of the scope of this series.
  - Link to v2: https://lore.kernel.org/r/[e-mail]

Changes in v4:
  - Fix a couple of now-invalid relative includes that I missed. This
    fixes the build issue in "seen" with Meson.
  - Link to v3: https://lore.kernel.org/r/[e-mail]

Changes in v5:
  - Rework the implementation of `REFTABLE_UNUSED` to use
    `__attribute__((__unused__))`.
  - Link to v4: https://lore.kernel.org/r/[e-mail]

Changes in v6:
  - I have rebased the patch series on top of "master" at 03944513488
    (The eleventh batch, 2025-02-14). On the one hand this gets rid of
    the conflict with the zlib-ng changes, which have meanwhile been
    merged to that branch. And on the other hand it pulls in the new CI
    build for MSVC+Meson, which allows me to have better test coverage
    on Windows to better detect breakage there.
  - Stop moving around Win32 files so that we do the minimum viable
    change in this context. Hopefully, this makes it easier for Dscho to
    integrate into Git for Windows. If it doesn't I don't really have
    any other good ideas for how to do it, so the only alternative would
    be to just ignore Windows altogether.
  - Link to v5: https://lore.kernel.org/r/[e-mail]

Thanks!

Patrick

---
Patrick Steinhardt (18):
      reftable/stack: stop using `read_in_full()`
      reftable/stack: stop using `write_in_full()`
      reftable/blocksource: stop using `xmmap()`
      reftable/record: stop using `COPY_ARRAY()`
      reftable/record: stop using `BUG()` in `reftable_record_init()`
      reftable/record: don't `BUG()` in `reftable_record_cmp()`
      reftable: stop using `BUG()` in trivial cases
      reftable/basics: stop using `st_mult()` in array allocators
      reftable/basics: provide wrappers for big endian conversion
      reftable/reader: stop using `ARRAY_SIZE()` macro
      reftable/system: introduce `reftable_rand()`
      reftable/stack: stop using `sleep_millisec()`
      reftable/basics: stop using `SWAP()` macro
      reftable/basics: introduce `REFTABLE_UNUSED` annotation
      compat/mingw: split out POSIX-related bits
      git-compat-util.h: split out POSIX-emulating bits
      reftable: decouple from Git codebase by pulling in "compat/posix.h"
      Makefile: skip reftable library for Coccinelle

 Makefile                         |   2 +-
 compat/mingw-posix.h             | 431 +++++++++++++++++++++++++++++++
 compat/mingw.h                   | 426 +-----------------------------
 compat/msvc-posix.h              |  33 +++
 compat/msvc.h                    |  30 +--
 compat/posix.h                   | 541 +++++++++++++++++++++++++++++++++++++++
 git-compat-util.h                | 531 +-------------------------------------
 reftable/basics.c                |  19 --
 reftable/basics.h                | 123 ++++++++-
 reftable/block.c                 |  16 +-
 reftable/blocksource.c           |  17 +-
 reftable/iter.c                  |  13 +-
 reftable/merged.c                |  27 +-
 reftable/pq.c                    |  40 ++-
 reftable/pq.h                    |   2 +-
 reftable/reader.c                |  33 ++-
 reftable/record.c                |  84 +++---
 reftable/record.h                |   6 +-
 reftable/stack.c                 |  52 +++-
 reftable/system.c                |   7 +
 reftable/system.h                |   8 +-
 reftable/writer.c                |  27 +-
 t/unit-tests/t-reftable-basics.c |  28 +-
 t/unit-tests/t-reftable-pq.c     |  22 +-
 t/unit-tests/t-reftable-record.c |  42 ++-
 25 files changed, 1405 insertions(+), 1155 deletions(-)

Range-diff versus v5:

 1:  18b6360460f =  1:  3589d759e69 reftable/stack: stop using `read_in_full()`
 2:  4426ec59a5e =  2:  e9fb7b0821f reftable/stack: stop using `write_in_full()`
 3:  4db9255d16d =  3:  f509993ee13 reftable/blocksource: stop using `xmmap()`
 4:  2752fb006c5 =  4:  48247ebf145 reftable/record: stop using `COPY_ARRAY()`
 5:  3bdd13f015e =  5:  795e9650146 reftable/record: stop using `BUG()` in `reftable_record_init()`
 6:  21529172e83 =  6:  8221aec91a9 reftable/record: don't `BUG()` in `reftable_record_cmp()`
 7:  e62c056f8e8 =  7:  8563af898e5 reftable: stop using `BUG()` in trivial cases
 8:  b506a862abc =  8:  447c7166c97 reftable/basics: stop using `st_mult()` in array allocators
 9:  b1749f88610 =  9:  108d40a63f5 reftable/basics: provide wrappers for big endian conversion
10:  f583f0cfb03 = 10:  19068751b6c reftable/reader: stop using `ARRAY_SIZE()` macro
11:  c59328f9299 ! 11:  316ca16a18c reftable/system: introduce `reftable_rand()`
    @@ reftable/system.c
     
      ## reftable/system.h ##
     @@ reftable/system.h: license that can be found in the LICENSE file or at
    - 
      #include "git-compat-util.h"
    + #include "compat/zlib-compat.h"
      
     +/*
     + * Return a random 32 bit integer. This function is expected to return
12:  0657d144327 = 12:  3cf2df99203 reftable/stack: stop using `sleep_millisec()`
13:  504ae579c30 = 13:  5184d4185a6 reftable/basics: stop using `SWAP()` macro
14:  34b0ae5f697 = 14:  a108a876c9a reftable/basics: introduce `REFTABLE_UNUSED` annotation
15:  1552c1f45bf <  -:  ----------- compat/mingw: split out POSIX-related bits
 -:  ----------- > 15:  48186eb7aeb compat/mingw: split out POSIX-related bits
16:  7cebd3c3381 ! 16:  16f2371c868 git-compat-util.h: split out POSIX-emulating bits
    @@ compat/posix.h (new)
     +#endif
     +
     +#if defined(__MINGW32__)
    -+#include "mingw/posix.h"
    ++#include "mingw-posix.h"
     +#elif defined(_MSC_VER)
    -+#include "msvc/posix.h"
    ++#include "msvc-posix.h"
     +#else
     +#include <sys/utsname.h>
     +#include <sys/wait.h>
    @@ git-compat-util.h: static inline int _have_unix_sockets(void)
     @@ git-compat-util.h: static inline int is_xplatform_dir_sep(int c)
      #elif defined(_MSC_VER)
      #include "compat/win32/path-utils.h"
    - #include "compat/msvc/compat-util.h"
    + #include "compat/msvc.h"
     -#else
     -#include <sys/utsname.h>
     -#include <sys/wait.h>
17:  dbb9b816d2e ! 17:  4461c8cf4a3 reftable: decouple from Git codebase by pulling in "compat/posix.h"
    @@ reftable/system.h: license that can be found in the LICENSE file or at
      /* This header glues the reftable library to the rest of Git */
      
     -#include "git-compat-util.h"
    -+#include "../compat/posix.h"
    -+#include <zlib.h>
    ++#include "compat/posix.h"
    + #include "compat/zlib-compat.h"
      
      /*
    -  * Return a random 32 bit integer. This function is expected to return
18:  f7b470f3387 = 18:  3e70da1abc8 Makefile: skip reftable library for Coccinelle

---
base-commit: 03944513488db4a81fdb4c21c3b515e4cb260b05
change-id: 20241119-pks-reftable-drop-git-compat-util-470f2bfde562

Comments

Junio C Hamano Feb. 18, 2025, 6:55 p.m. UTC | #1

Patrick Steinhardt <[e-mail]> writes:

> Changes in v6:
>   - I have rebased the patch series on top of "master" at 03944513488
>     (The eleventh batch, 2025-02-14). On the one hand this gets rid of
>     the conflict with the zlib-ng changes, which have meanwhile been
>     merged to that branch. And on the other hand it pulls in the new CI
>     build for MSVC+Meson, which allows me to have better test coverage
>     on Windows to better detect breakage there.
>   - Stop moving around Win32 files so that we do the minimum viable
>     change in this context. Hopefully, this makes it easier for Dscho to
>     integrate into Git for Windows. If it doesn't I don't really have
>     any other good ideas for how to do it, so the only alternative would
>     be to just ignore Windows altogether.
>   - Link to v5: https://lore.kernel.org/r/[e-mail]

Thanks.  I think the "unlink" thing on Windows I have was queued
depended on the previous iteration of this series so let me make
sure that I'd unentangle it while replacing the topic with this
round.

Will queue.  Thanks.