From patchwork Thu Jul 1 14:47:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12354325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75B9AC11F69 for ; Thu, 1 Jul 2021 14:48:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F1E761414 for ; Thu, 1 Jul 2021 14:48:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233122AbhGAOuj (ORCPT ); Thu, 1 Jul 2021 10:50:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232625AbhGAOu2 (ORCPT ); Thu, 1 Jul 2021 10:50:28 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22FC6C061765 for ; Thu, 1 Jul 2021 07:47:57 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id i94so8602764wri.4 for ; Thu, 01 Jul 2021 07:47:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=wR7kYAWIpdTP/+5YQ3Emp8FTipGBv/xXPqjGmoTjFMw=; b=mJJ7ashoKQVjtWiim0AQV8cbigR0mD0vE0zD+ljBBMJWFDRCCv+LC412YZ46zLpkAS df451Z+rLbrhS59WKI7Hw+CZvkpJmRR2VTzsxzp2DlMPfQsxqJ5bI635YL3oXdlASWoP lL+Fppq+Gw9UGygKyw1JncIapEoZBWgDO+4mz+QwUBm7jE3M2GFvxiyZNahN7HIvr2dF U8GQmFo8TakGmaGmPBAZc7nO7PtoH7XLIU3qxAnlAh3nu45L32OLbE0MsuJGl59BAKy5 QevIWFf+6B68nbwb9fzfjqdJCIxdcoWfoIonVDMK/Z6OquePZeJTovJbOQZfSDL7QCqG 2EoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=wR7kYAWIpdTP/+5YQ3Emp8FTipGBv/xXPqjGmoTjFMw=; b=ApsPouxhFIlOxMizGuWi7gCYcX5T+Eut9enunUImHVsReyJQJoazA0M1sdZURAQVpJ bXTD6I3M2sM26jcwQ48oEfFoiaG2EtC9ruu5HBzXLlQe2KChRtZO74tU+LrL4PohNT3S I3Ar/acd/HhKo9xK6oyConaZC3gIAoHSyDJ404RKSYr7lSKwMYQmW2ZSlFjiKTN+HwfK rYcOzgFgG9+DyKQSR5Wvg/EzlQBwb+jicjqYAu9wXQubCYC8/vQ4KqSGkckIz5e/2sKk JPqk+88QZKQJ7uktB2316hA5CP+M0AA4pwMTQMzIeGhdLEGNc23YDZU123FsQ2jjolof wAtQ== X-Gm-Message-State: AOAM532tkFrcMxENU6Lf7Lgmf0Io0jFzT3l0xY49gJxLUVR+C9uFh7D/ XUttjS+KXCb81PmHHonsbR7pGIVf+oE= X-Google-Smtp-Source: ABdhPJyy9pr+XTYg1uC2mERpEBTbJ4tTrKuCee7espsGVfUANblMV9LiZfSOt1OXfeFiml7gg0kiBg== X-Received: by 2002:a05:6000:42:: with SMTP id k2mr21370wrx.294.1625150875720; Thu, 01 Jul 2021 07:47:55 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u15sm9680812wmq.48.2021.07.01.07.47.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Jul 2021 07:47:55 -0700 (PDT) Message-Id: <37fdce5ec3afaa9aae5001c648fced0675dae0c4.1625150864.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Jul 2021 14:47:27 +0000 Subject: [PATCH v3 17/34] fsmonitor--daemon: define token-ids Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Jeff Hostetler , Derrick Stolee , Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Teach fsmonitor--daemon to create token-ids and define the overall token naming scheme. Signed-off-by: Jeff Hostetler --- builtin/fsmonitor--daemon.c | 116 +++++++++++++++++++++++++++++++++++- 1 file changed, 115 insertions(+), 1 deletion(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index e942f7c5840..e991925fafc 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -93,6 +93,120 @@ static int do_as_client__status(void) } } +/* + * Requests to and from a FSMonitor Protocol V2 provider use an opaque + * "token" as a virtual timestamp. Clients can request a summary of all + * created/deleted/modified files relative to a token. In the response, + * clients receive a new token for the next (relative) request. + * + * + * Token Format + * ============ + * + * The contents of the token are private and provider-specific. + * + * For the built-in fsmonitor--daemon, we define a token as follows: + * + * "builtin" ":" ":" + * + * The "builtin" prefix is used as a namespace to avoid conflicts + * with other providers (such as Watchman). + * + * The is an arbitrary OPAQUE string, such as a GUID, + * UUID, or {timestamp,pid}. It is used to group all filesystem + * events that happened while the daemon was monitoring (and in-sync + * with the filesystem). + * + * Unlike FSMonitor Protocol V1, it is not defined as a timestamp + * and does not define less-than/greater-than relationships. + * (There are too many race conditions to rely on file system + * event timestamps.) + * + * The is a simple integer incremented whenever the + * daemon needs to make its state public. For example, if 1000 file + * system events come in, but no clients have requested the data, + * the daemon can continue to accumulate file changes in the same + * bin and does not need to advance the sequence number. However, + * as soon as a client does arrive, the daemon needs to start a new + * bin and increment the sequence number. + * + * The sequence number serves as the boundary between 2 sets + * of bins -- the older ones that the client has already seen + * and the newer ones that it hasn't. + * + * When a new is created, the is reset to + * zero. + * + * + * About Token Ids + * =============== + * + * A new token_id is created: + * + * [1] each time the daemon is started. + * + * [2] any time that the daemon must re-sync with the filesystem + * (such as when the kernel drops or we miss events on a very + * active volume). + * + * [3] in response to a client "flush" command (for dropped event + * testing). + * + * When a new token_id is created, the daemon is free to discard all + * cached filesystem events associated with any previous token_ids. + * Events associated with a non-current token_id will never be sent + * to a client. A token_id change implicitly means that the daemon + * has gap in its event history. + * + * Therefore, clients that present a token with a stale (non-current) + * token_id will always be given a trivial response. + */ +struct fsmonitor_token_data { + struct strbuf token_id; + struct fsmonitor_batch *batch_head; + struct fsmonitor_batch *batch_tail; + uint64_t client_ref_count; +}; + +static struct fsmonitor_token_data *fsmonitor_new_token_data(void) +{ + static int test_env_value = -1; + static uint64_t flush_count = 0; + struct fsmonitor_token_data *token; + + CALLOC_ARRAY(token, 1); + + strbuf_init(&token->token_id, 0); + token->batch_head = NULL; + token->batch_tail = NULL; + token->client_ref_count = 0; + + if (test_env_value < 0) + test_env_value = git_env_bool("GIT_TEST_FSMONITOR_TOKEN", 0); + + if (!test_env_value) { + struct timeval tv; + struct tm tm; + time_t secs; + + gettimeofday(&tv, NULL); + secs = tv.tv_sec; + gmtime_r(&secs, &tm); + + strbuf_addf(&token->token_id, + "%"PRIu64".%d.%4d%02d%02dT%02d%02d%02d.%06ldZ", + flush_count++, + getpid(), + tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, + tm.tm_hour, tm.tm_min, tm.tm_sec, + (long)tv.tv_usec); + } else { + strbuf_addf(&token->token_id, "test_%08x", test_env_value++); + } + + return token; +} + static ipc_server_application_cb handle_client; static int handle_client(void *data, @@ -283,7 +397,7 @@ static int fsmonitor_run_daemon(void) pthread_mutex_init(&state.main_lock, NULL); state.error_code = 0; - state.current_token_data = NULL; + state.current_token_data = fsmonitor_new_token_data(); /* Prepare to (recursively) watch the directory. */ strbuf_init(&state.path_worktree_watch, 0);