From patchwork Wed Dec 1 00:11:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13010664 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF9E32CA3 for ; Wed, 1 Dec 2021 00:11:29 +0000 (UTC) Received: by mail-pf1-f172.google.com with SMTP id i12so22351408pfd.6 for ; Tue, 30 Nov 2021 16:11:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bceWDZcaOkyzrGLd/xGMAmBBE8fXa94zLV20HlyIv2o=; b=DX9f0RLSwnVavTPKwfaC7eWZL0kUJezFMxTYlG58xbho9/IbYnH7fBdj2hw3ukwiPS HpZmH+Qz3UlvKrkmoyH33X4rGUsHlW0+KEsEN7DSMMjjMLBBhU/NYnTQirEvqJq/rj61 3zUVSSee3ARFbSp0qF1ms4BQU2v2pPK3hlpWg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bceWDZcaOkyzrGLd/xGMAmBBE8fXa94zLV20HlyIv2o=; b=bIPpQGOG7NJVatTvIcvVcwN0/B+oNf8iwjr+5YpRFGS3ZADT/olknh8xyAz7h7sxJ/ oOKcDEcD8b7HjQuDMcQsb7K7rfKv+s+2lu9bbY454+9kEMLbJAoc51Ko1LUJslKdZ7M4 vmITzPaehFzW6b163m6Fo5PYDcTSzCcyQ7NseBVBkQSESYzclSawdWeJ0Q+UMTyLF1Yd JW371i9GCe8ql1be2zm/eTPm2q2fmTz/WYD2kkTbOaNv+ytJCsak8uleNL93t3ws18uo NZEYrCmnbYPdjR38X4r7BMYTXUTqSxdHRMt9ACdkHUqw2ORn3CQSRjFeKOHSZeUOuL7O KCKA== X-Gm-Message-State: AOAM530Q9GVRAVK+p86L0c7l7NtLcgCMERtKHOOnPdLQKieNaGXNjn0k 1Y2ldy1mL9OcE3T4TCJEPXFIXw== X-Google-Smtp-Source: ABdhPJwTd3esRg+rXNxjeYOx3zt/eB7f8lnYoBu5TL4YXZsXFkgaj7M8awLQv8VGa8xw5ufwW/pRGw== X-Received: by 2002:a63:f91f:: with SMTP id h31mr2030519pgi.538.1638317489240; Tue, 30 Nov 2021 16:11:29 -0800 (PST) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id k129sm16702922pgk.72.2021.11.30.16.11.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Nov 2021 16:11:28 -0800 (PST) From: Kees Cook To: Konstantin Ryabitsev Cc: Kees Cook , tools@linux.kernel.org Subject: [PATCH 1/5] patchwork-bot: Note which actions are part of "housekeeping" Date: Tue, 30 Nov 2021 16:11:22 -0800 Message-Id: <20211201001126.4106635-2-keescook@chromium.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201001126.4106635-1-keescook@chromium.org> References: <20211201001126.4106635-1-keescook@chromium.org> Precedence: bulk X-Mailing-List: tools@linux.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1028; h=from:subject; bh=p0nNJyqUltsslCjTRN5hzN2nMrpwxjIPM92Rf6CW+SA=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBhpr2sdE7idsEDPg/Giv5PSItmNdRq/qdDQinvZSJT 7uht+1aJAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCYaa9rAAKCRCJcvTf3G3AJgHkEA Cwb7Ky8w036xFQWmGtnm0rHHcvnm04kBUJaCe4mMw7Xk+1ND/36sNHQzS9Frj9N30JMVK+6V27F/rH AAfmXaoGWYJ3awEn7LrGvjaKHffw2dDRL/wmkkf23oaCs4mwZzWC7oOmaWz/8Hr+Fru1zKs+OCgkiz KGoUWXDpZqvlPwKCErTEvEzMEZWXHnAEEj9Xg/pNg8WBsdgGMARIsKdzdL7IqdCP7bDYFNn4iXB9MO Qi8LPtgnb+/2h1QglJsrxhW2lgs7gqJ+lC6nbkjIOeB/K2w7+A3pX4ynyvdVYcYyjw3dCUf/1FDzS9 HmdVKNwv+/iJuVHGWIQAWtMGJjty0GAK7+JB6Xcc/AKEtLOZL2r6NStbkM4sdDOysayDN3h292ehq9 hMtcVDUKnaxaFTbTCWfAnz2vSEE9Zg4R6OXAh8r1JqvF9KtH40BK2Q5YvbhoGcKZVQF74twccxODQo ld8R8SMbe7Hpyn2NcIXwyUSVplqvK8EXZKuQBfm2DOwgqHe8iRX1vW5IPJLpH3AphHb4KiszvZjOId /ZT2KsHSlylLc3+iuIek2e6Fe3qVpfaizaoc1UxtXhHzacuSzmCyDtgEJVGsGvzfDmO1oRJJG+V7Zg Ni/Lr0G4Hh2Fs7g4kdjtqZfFKPj94vwK1O3KT+9imaNSHokVLIDhL9nTcIiw== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 Expand the --help text slightly to remind users what things the "housekeeping" option does. Signed-off-by: Kees Cook --- git-patchwork-bot.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py index f87af6ef41d6..d1eac6c0c0df 100755 --- a/git-patchwork-bot.py +++ b/git-patchwork-bot.py @@ -1325,7 +1325,7 @@ if __name__ == '__main__': parser.add_argument('-v', '--verbose', action='store_true', default=False, help='Be more verbose in logging output') parser.add_argument('-k', '--housekeeping', action='store_true', default=False, - help='Perform a housekeeping run') + help='Perform a housekeeping run (supersede, archive)') parser.add_argument('--cachedir', default=None, help='Cache directory to use instead of ~/.cache/git-patchwork-bot') parser.add_argument('--domain', default=None, From patchwork Wed Dec 1 00:11:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13010663 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80CD42CA1 for ; Wed, 1 Dec 2021 00:11:29 +0000 (UTC) Received: by mail-pl1-f173.google.com with SMTP id b13so16279163plg.2 for ; Tue, 30 Nov 2021 16:11:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NePa81kB163QyHEWq9gcWkRRPDFvgTP+CZOUP3wpTAs=; b=NfQm3HenJosIN6/6SeUZHKvKD84i6sy1WZ6VlrD3WOM2n9qVcv3KfsUTSt52pf0UHS D4JDjhC5679AgyCnmtHAkJyvDBfMnqU/2mYo3ukI7jcEF+TMAtHN8aS4pzhGq02mokrF 1soLY0Hy6TZQKn9Xh/T8Ip21iQp9R6X6YmHCg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NePa81kB163QyHEWq9gcWkRRPDFvgTP+CZOUP3wpTAs=; b=iZjN823YoxYv5cUpPFNRVeIpbEt7aZ+IRw1SOBeAu1AzkU6dLXK8x88xnFS+RvhUGN ozpE/iNTuaJmnn5J5aLAJpbo0EtRDw4gWBiZlo2yULkXpLGPA0VN/fFRXuZWs068YYeh HDbRGw6Faa3dbyh/fZuq1pOUZ3q61BrQ8JxxAe7KuXt5zATk3tWM4eaw58jFzHzgqMkc ck21ZIxRrpbca94qkABAPIno7y5YgcPiFcyUnYYY2ps5s3dQh9x3jSG6BHoHUKaKdCBx yf5IoYzOhxHyafHyOeVU810THdYWPlI8K6LIN+5VEDVol/eZrUucH7nNbx85vxkUn83T 3n6w== X-Gm-Message-State: AOAM531Qp6DIP4XrI24iVwljkRZ3yS8acauEj90KDegjpv86f25Kn6gk j+NEOSCBznmQyA95K/vgO33Ojg== X-Google-Smtp-Source: ABdhPJxcboJQz4zj/jrn/I6MMfY+ERW4BktFNHCx8rDURPWQQYaj4BwDV5HuRNIjdrhcwMxsVi+AVg== X-Received: by 2002:a17:90a:e7d1:: with SMTP id kb17mr2969544pjb.124.1638317489056; Tue, 30 Nov 2021 16:11:29 -0800 (PST) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id pc10sm4176314pjb.9.2021.11.30.16.11.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Nov 2021 16:11:27 -0800 (PST) From: Kees Cook To: Konstantin Ryabitsev Cc: Kees Cook , tools@linux.kernel.org Subject: [PATCH 2/5] patchwork-bot: Create CACHEDIR if it is missing Date: Tue, 30 Nov 2021 16:11:23 -0800 Message-Id: <20211201001126.4106635-3-keescook@chromium.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201001126.4106635-1-keescook@chromium.org> References: <20211201001126.4106635-1-keescook@chromium.org> Precedence: bulk X-Mailing-List: tools@linux.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=748; h=from:subject; bh=gYNuSa7CqDWoxRa1ApTz8/BbWHcKZPnU0x5DRNkf8Ro=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBhpr2tPWQk95pGxYLnMC+fxBPRQPuNOBO57f6kdCaa GY/O3jGJAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCYaa9rQAKCRCJcvTf3G3AJs7uD/ 9x/C3ODaGesG6NUEKyRRiro3rlv1S1crKGm65Pyu8WbAIg3CVIjcdIIrqEw1j8sLN9z+M2a9wUq3Zl drm1jKGwJVeBq4ad4R9hasTUXOQe1zOUHayu2mW/5kFCfoQ485Z98cCHvrusyT4mdc61QD++6USX7B tZgoC1ffEfxgJIZruOXEytQuR5YqhjphMtdZ8Nd3d0VfcuhfMcTSebl1XcAkTEQT8pvB0vmfwKLg4N hSUWrMhcPddVXK06shTeGUaU8y0mbO3cOJ2TA2yINULFFO1MgnQMmMAh6o8JhwHlpHfKE7sfgB9d6u N2mWcB6BhBm+Nd55cH3NaM82JHcsMuL4HqMANwa8/hUxofOkiyoLhIxeW6rBQwkMPM0ZQuWJSYoIwr Z9eXFFCP7P2Z7KXK7rmntwpvinGIiTq39fz0OLW+xsNP+fpU8heBYcymMYLenddapqxDldkIZegVgi wcv6UIzIFi5pWWMfx71Tyf+540e+/gG3tCguEPndU+M/7uFpohjIUBfxodX4fNo0YHTSM4rIlBzGB3 G8ceIjEc/paWGDfz9zAtHMridAartuS8t4XxiVWhAMZgiEgZHsU3NUlhFefUMCvvc6Sobe+KV8Pgl7 qoaxcbJHIOFgnF43Q7NjLrkObMJlKCbzRQ6bc+DPTVIJfZnF0pYdg8Z/P0cg== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 Create the CACHEDIR if it is missing (e.g. the first time git-patchwork-bot is executed). Signed-off-by: Kees Cook --- git-patchwork-bot.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py index d1eac6c0c0df..dc10fd0effa9 100755 --- a/git-patchwork-bot.py +++ b/git-patchwork-bot.py @@ -1375,6 +1375,9 @@ if __name__ == '__main__': cfgyaml = fh.read() CONFIG = ruamel.yaml.safe_load(cfgyaml) + if not os.path.isdir(CACHEDIR): + os.makedirs(CACHEDIR, exist_ok=True) + if cmdargs.housekeeping: for _pserver, _sconfig in CONFIG['patchworks'].items(): for _pname in _sconfig['projects']: From patchwork Wed Dec 1 00:11:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13010661 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA4C42C99 for ; Wed, 1 Dec 2021 00:11:28 +0000 (UTC) Received: by mail-pj1-f44.google.com with SMTP id x7so16619340pjn.0 for ; Tue, 30 Nov 2021 16:11:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NJj2Xrxr7kYghEILolKhl/zUkFd5bz28t2goqiNtZMU=; b=ATThWKApE/n0kpq0EApJTRmzA6PIfaD4h1H7qT6pnq7HBAPdvRR/4+P1LP6VEkzEol 0PbZyJb0dH8Fob7+fMVSJEH5mG9U5L0GBgvC/YzQRDHHZXFaM14U9gwKBPFGquF6i2tn yZOhJVPQrc8tf6kHhV4/qa+WruFGd21yYZmcE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NJj2Xrxr7kYghEILolKhl/zUkFd5bz28t2goqiNtZMU=; b=vvd6b4tkrBi8ty0IKyjVjwYNLnSVZ8VkZy3/nQML/PB/s2kyQoO+lNtgQSnmIpaRcJ 8LTmUYe1ioQQwkQbDK23lzJdHzuPlUxHdZvG0wctJvyQprh3AYXsYWa/v8z95R9VW/tO oAlzYlbug9OCnI2un90jb/cI5oh0vg8SckUrLYMUkSLzvhr53+kVHgJ3c41DWwaJvlR9 bIfWFCbmxlDyvVKpNvUwH100gItqT2u506dWMSwou3Ym2lSVsK3pasG6F86sr7/fcS+b OBmYZNIp4FA+O8DBPbknQAf7JcTjrZmNj97ZejKwziVqWAl9tYhBtm99LcLiku7/Q8g4 1ZYQ== X-Gm-Message-State: AOAM530jQL8f2T3brthC0CY46aYEXr+8//Y/+PTz33Fhh/2d70/PtMaT iLZ/8GWLkMD33phK9CirSXSfOzOYeKuRaA== X-Google-Smtp-Source: ABdhPJyOi+zFsETxLSlhm7Q5aLuESnvbqyBW2krz60sx2U9AAT0pjX+EPkhTp8kEUBty+FWUnYsTCA== X-Received: by 2002:a17:90a:e00f:: with SMTP id u15mr2889394pjy.123.1638317488236; Tue, 30 Nov 2021 16:11:28 -0800 (PST) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id d1sm23536645pfv.194.2021.11.30.16.11.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Nov 2021 16:11:27 -0800 (PST) From: Kees Cook To: Konstantin Ryabitsev Cc: Kees Cook , tools@linux.kernel.org Subject: [PATCH 3/5] patchwork-bot: Allow limiting heads to single branch Date: Tue, 30 Nov 2021 16:11:24 -0800 Message-Id: <20211201001126.4106635-4-keescook@chromium.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201001126.4106635-1-keescook@chromium.org> References: <20211201001126.4106635-1-keescook@chromium.org> Precedence: bulk X-Mailing-List: tools@linux.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2005; h=from:subject; bh=jFjmKeND7X144NlkWr3Bqrg1VW92lD8E7ZSLWq7llBY=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBhpr2tGpwl4++YrccsAlupkC5RXVBTirabUIaIIQAJ XdU+N8qJAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCYaa9rQAKCRCJcvTf3G3AJhC4EA CA8MXUjqJ73U41d565OM8AHLhGkIdIG+sfEby8sviF6alV9FyZMxHwm25ZT8jpswXhXi9gjqNYzhtk eU6PrORQWvE5dH3PJFJpmiJvN7UdYbHYJo3a8fSeo35WiJOR168GyQgOnhIyRf/sj1vesERtGFuzoO I5wiz8DHeIscIAZZiZXS4w8t02Ro1S0ATPQJSdESi+xbWizzLJfMATb62PypDK9gcgUt9hTtWgCKFV aTIl5I6c0DOgNne0Rer+GwFvy3b7PccXi/uIFXkhMYOUA9BCTLUzb+YQZanRcZzm2nDo+x3NxtnmH/ zklzmuHZCXvuYhfYPKxsDBuEPp2JBJnrw9B1MRwFLqXwb/VxHGYuavL+AwgR37HM50DTgTem7VpQFl xGeSLERfE2L79P7oWpkx4f0XQ0ylBFopMh/iJTCFTnwFWRD43AoHNcjggQfvxf00vXANBAbAxruBtF 6Q41R8wTa3V+LivlDPNKBeuQauI3S5/eb2NY5K0T/HiGWd1II1VhYBiqftMvdg/xBoEVafWqWwwdaX so+ruzmywRUyVGmRW6KRmQAIvzy7I7DH2e9dJKzyv0AgZrAH+DICRcI9c9uEGesdwbRskz3fSfVtSa AuyBFS0d+Ux66NcEGiTr6HHVHzz6xBmgpPFbHkFlO1tg57/nmWtY/TkjxI0w== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 To sanely use a single git repository with multiple worktrees, it must be possible to limit the commits by branch (otherwise all commits are visible for all worktrees). Add an optional "branch" argument for this limit. Signed-off-by: Kees Cook --- git-patchwork-bot.example.yaml | 2 ++ git-patchwork-bot.py | 6 +++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/git-patchwork-bot.example.yaml b/git-patchwork-bot.example.yaml index d6140ed68e53..c3f6c1ea3133 100644 --- a/git-patchwork-bot.example.yaml +++ b/git-patchwork-bot.example.yaml @@ -52,6 +52,8 @@ repos: 'pub/scm/linux/kernel/git/khilman/linux-amlogic.git': # For the notification emails treename: 'khilman/linux-amlogic.git' + # Optional: limit checking to a single branch (instead of --heads) + #branch: 'origin/master' # For the notification emails, must include %s commitlink: 'https://git.kernel.org/khilman/linux-amlogic/c/%.12s' # Only notify for commits where committer email exactly matches one diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py index dc10fd0effa9..fc4d9e384122 100755 --- a/git-patchwork-bot.py +++ b/git-patchwork-bot.py @@ -409,9 +409,9 @@ def git_run_command(gitdir, args, stdin=None): return output -def git_get_repo_heads(gitdir): +def git_get_repo_heads(gitdir, branch): refs = list() - lines = git_get_command_lines(gitdir, ['show-ref', '--heads']) + lines = git_get_command_lines(gitdir, ['show-ref', branch]) if lines is not None: for line in lines: (commit_id, refname) = line.split() @@ -1049,7 +1049,7 @@ def housekeeping(pname): def pwrun(repo, rsettings): - git_heads = git_get_repo_heads(repo) + git_heads = git_get_repo_heads(repo, branch=rsettings.get('branch', '--heads')) if not git_heads: logger.info('Could not get the latest ref in %s', repo) sys.exit(1) From patchwork Wed Dec 1 00:11:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13010662 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBA8429CA for ; Wed, 1 Dec 2021 00:11:28 +0000 (UTC) Received: by mail-pf1-f171.google.com with SMTP id n85so22323564pfd.10 for ; Tue, 30 Nov 2021 16:11:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5HNAI8vlYuB449JtweLE9tRLESJSwBrAfAkGgbwr53c=; b=Gquo8R/QbZj5Mg8ttXV6dbbwBwrjCGThZIJpXKcgq7WTBNVHf6qiLC1J7JNugeLFW7 UYpLrBF2phpBX5I5UbI/F+5npN0tXa+zB8NXWIRKY5aGcaVUl5fpDYlVcQhdmv+yw9Nu SFIazn+bBzReg86PYEisKSUUBL284yCdAJ0Nk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5HNAI8vlYuB449JtweLE9tRLESJSwBrAfAkGgbwr53c=; b=fBe2HR0Nx6LkEoBTEniXCiUq/3kIpFqXjXudh9hQap2pxFQTtayB4sHkTgcJtHOI4K YfrrAZsjBSmviDTFGSybVk/Hb37U7v2BhuJR+LxIjrMkvdiKtdxHD7fEDvR1qu/4HUeN lsWf5l70eYbauzvSEq9otAyzkQeC7c6v7MpkzNwo2qFqi1WLsspBi103ujdP8mEwZdE0 HvcGRuf3712h5hnIIDQ5nr4rqxE1W3BXHvieQ8LQdTZZ2HAPhfY3yRJ5hxkovzut3K1b WG9fZ8ciEuVV2Mws0jnQDqu/QyRkS7n0LVybE++cQCdjIBkTnMrBsTW1Lv+BzA+Yj7uU xONQ== X-Gm-Message-State: AOAM5312yium6RupRaU2AFidgJPsDeoqN09811ZcyHGwIgFS1GEqlmd0 tvoGeF5EQSgcpH8snynPm3W2xUriBkKrIg== X-Google-Smtp-Source: ABdhPJzM7FJKDuACvAfcxLVLYMKiS3vr0CNgir1lHGY8+MY4c6WvCi0npfLr3layR/fkCPPU9ReD8A== X-Received: by 2002:a05:6a00:8cf:b0:4a8:3ae:1a78 with SMTP id s15-20020a056a0008cf00b004a803ae1a78mr2341376pfu.7.1638317488386; Tue, 30 Nov 2021 16:11:28 -0800 (PST) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id 38sm943737pgl.73.2021.11.30.16.11.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Nov 2021 16:11:27 -0800 (PST) From: Kees Cook To: Konstantin Ryabitsev Cc: Kees Cook , tools@linux.kernel.org Subject: [PATCH 4/5] patchwork-bot: Handle repo being a worktree Date: Tue, 30 Nov 2021 16:11:25 -0800 Message-Id: <20211201001126.4106635-5-keescook@chromium.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201001126.4106635-1-keescook@chromium.org> References: <20211201001126.4106635-1-keescook@chromium.org> Precedence: bulk X-Mailing-List: tools@linux.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3091; h=from:subject; bh=ZI+bdv5C4l5ULeF2KK5rA9tnNoMcnBAHJMRmVHeCsiE=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBhpr2t40p33ROj5seEvkZzEOAPwy7Aw4aucrOXriG4 p7o9LqOJAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCYaa9rQAKCRCJcvTf3G3AJkiQD/ 4zbnc48tsUcY5b6VlGf93NMhNLwgN5w3SNE20aGdMb0/tUTB6OSX2cDjPr74BBp5jAgTDnWsh88Br/ q48XWx83CNvUKN7D9SCmmFHIoacFoW8E1Z9p/Ha8c7gC6A8tQxSo5Umc/NhNByZ6lnwzFsY3tW3l02 2ry9DNQ7VwmBEkeQtFBZMPPYVsOm0S4JFZtP2UTAPStoMvueS33CHPxVwlcZ8OHy7X9+XB2tXHymBC uo+L2rc0Ogb0fdMht1C9Qnzrl2XjBWntuMhnx+XdgFPEqvjqnGDsPBfgdO0eqw9tCPOJOFZPNYGSZf HLgl+4aKw310+HydyShJxh1tme2Q6wSBwoSgLNFM18QdH1N+gikYkvI0j2Hyj+SC2ib4eBsFTwTA9f uPvDMcvbRoNnYVplqLDTvLUcvtnKJVHYO3V63ADjLTDMkYF5h6mah3mcEtUGsTL0KC52uxv3qwHh8c Shpu1ppHgAD2okOLqoHUEbtr/DpQv7Macu23NugGZqRotD87WD2uF8l15vgTKr541oWpLNwGEB73s2 9O5xFhQCPYN56GVYjziW2F/VffyrMe/vvTgWaW7iPvS8+U1TLaIrO0hp/YixO2dlLwxy5I/nFIJhGx 8vzlh24/T1mw8s2vhD6BbuqjfnIr1iaYR6OQFieCHwZSgJmpzE6GKb2yVViA== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 Locate the real git location when operating on a worktree, allowing multiple pw.db files to be stored in a single top-level git repo. Signed-off-by: Kees Cook --- git-patchwork-bot.example.yaml | 1 + git-patchwork-bot.py | 27 +++++++++++++++++++++++++-- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/git-patchwork-bot.example.yaml b/git-patchwork-bot.example.yaml index c3f6c1ea3133..94ab334f5043 100644 --- a/git-patchwork-bot.example.yaml +++ b/git-patchwork-bot.example.yaml @@ -53,6 +53,7 @@ repos: # For the notification emails treename: 'khilman/linux-amlogic.git' # Optional: limit checking to a single branch (instead of --heads) + # This becomes a required setting if the repo path is a worktree. #branch: 'origin/master' # For the notification emails, must include %s commitlink: 'https://git.kernel.org/khilman/linux-amlogic/c/%.12s' diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py index fc4d9e384122..9be945d92a8a 100755 --- a/git-patchwork-bot.py +++ b/git-patchwork-bot.py @@ -1054,8 +1054,28 @@ def pwrun(repo, rsettings): logger.info('Could not get the latest ref in %s', repo) sys.exit(1) + dbpath = repo + # If we're aimed at a worktree, move up from the ".git" file to + # the worktree directory. + if not os.path.isdir(dbpath): + gitdir = open(dbpath).readline().strip() + if not gitdir.startswith('gitdir: '): + logger.info('Could not find git tree in %s', dbpath) + sys.exit(1) + gitdir = gitdir.split(' ', 1)[1] + gitdir, worktree = os.path.split(gitdir) + gitdir, category = os.path.split(gitdir) + if category != "worktrees": + logger.info('Could not find git worktree in %s', dbpath) + sys.exit(1) + # To store multiple pw.db files in a single .git directory, + # add a suffix based on the repo treename. + treename = rsettings.get('treename').replace('/', '_') + dbpath = os.path.join(gitdir, f'pw-{treename}.db') + else: + dbpath = os.path.join(dbpath, 'pw.db') + # Do we have a pw.db there yet? - dbpath = os.path.join(repo, 'pw.db') db_exists = os.path.isfile(dbpath) dbconn = sqlite3.connect(dbpath, sqlite3.PARSE_DECLTYPES | sqlite3.PARSE_COLNAMES) c = dbconn.cursor() @@ -1295,10 +1315,13 @@ def check_repos(): for repo in CONFIG['repos']: fullpath = os.path.join(cmdargs.reposdir.rstrip('/'), repo.lstrip('/')) - if not os.path.isdir(fullpath): + if not os.path.exists(fullpath): logger.info('Repository not found: %s', repo) continue settings = CONFIG['repos'][repo] + if not os.path.isdir(fullpath) and settings.get('branch', None) == None: + logger.info('Worktree must specify "branch" setting: %s', repo) + continue logger.info('Processing: %s', repo) pwrun(fullpath, settings) From patchwork Wed Dec 1 00:11:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13010665 Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F2A92C99 for ; Wed, 1 Dec 2021 00:11:30 +0000 (UTC) Received: by mail-pg1-f179.google.com with SMTP id k4so11480749pgb.8 for ; Tue, 30 Nov 2021 16:11:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TQB4EdQwLW0dEuvfGy3zcM1aNhpWIPrjQOaB8Ydmn1w=; b=YwEWOS5bFYL8XlF2paLSl5Rh/Xe9YoRNltcuujAu/CyXfPSSV+S4OHxRdTb8JTZnF2 0hpWBMkm9VcqEoaLOj10TaUFGsXt2+i8BMhNBBaySEc19Ws9ozWAsCVGnNYJGm7VbZlS RrfAKHrtZDwaVF3Co1aWKHE6yodxJcQnpz1nw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TQB4EdQwLW0dEuvfGy3zcM1aNhpWIPrjQOaB8Ydmn1w=; b=UeJ4NWXr3UkKccvE0igTPtX3onB7HsAzga9fHIgfbE7hG5D+1J24tR06eLLLcQ8G5M u8GTyriAvUA/F2jVwUoTSlpax+W6qJ9i6GCM4WeC3YpxzXttmS4eQcxsoMXJ9iN4tSFV l6fkx1P8hf8vCCqm60ePxT/ioAp0vPL2YKIy2WOSEDbt4CxAIcIig8Xw3QUR7Myy98OA QG+9ZbHSxdDk4JUGf6wYs8edhHPQYfFlK80mjS5yaTTRl81Ry+TGSoVToi1zg3RJWwf/ hGaG1Yv2nkRwOFDfqO2Kxh3aqnwbWSWIIjbgVxV6+5/4sKtl592nk1lJY87fFQ0d7QTt yNrw== X-Gm-Message-State: AOAM532X9LkNfYF6iW/w1KX//e8d5lguISRnSK01p7x3LHt5pG6Jn+YL wTJ+237Cyyy7G8QcAU1cHqSvdg== X-Google-Smtp-Source: ABdhPJzAxitikgZ/FDyQOweW9wIu+KM5pmkxMRA5Qreihp6qD6eCpppJdco284CPyDEpnZGnhZJskA== X-Received: by 2002:a05:6a00:1349:b0:4a0:2f9:e3ab with SMTP id k9-20020a056a00134900b004a002f9e3abmr2361753pfu.80.1638317490081; Tue, 30 Nov 2021 16:11:30 -0800 (PST) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id d3sm21976994pfv.57.2021.11.30.16.11.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Nov 2021 16:11:29 -0800 (PST) From: Kees Cook To: Konstantin Ryabitsev Cc: Kees Cook , tools@linux.kernel.org Subject: [PATCH 5/5] patchwork-bot: Allow initial database to skip recent commits Date: Tue, 30 Nov 2021 16:11:26 -0800 Message-Id: <20211201001126.4106635-6-keescook@chromium.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201001126.4106635-1-keescook@chromium.org> References: <20211201001126.4106635-1-keescook@chromium.org> Precedence: bulk X-Mailing-List: tools@linux.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2201; h=from:subject; bh=1RAuWS5nDZVb4yoZ+KhnC3fNNEabyAthcdHbBgRHj5o=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBhpr2tE/IOrk+JZjIGbkZ+HdS/3PzVKoVHksKK4HxZ rCNq0TaJAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCYaa9rQAKCRCJcvTf3G3AJmN1D/ 9tVeBeG29wJP47uA/Hj0pHzdehqSrewlwr82F1kUe+pRfnXULHepnxxppBeMGmKxvDtN1Kj//hCOb/ TSi9qulQ+f6KRKc3wj1GJWltfOvHFNWMNkXv4MwHsplc1XyTzXVm0B7YhlxKonXT5qy23bG35Jwrig 9WwzRM1AlduTwxfmCfGu2dcTCCLkp10of05vZ3nJ5sZ8R1jbNG8bhuYwUnK/al32OAhyH4VErR4nAN u5aX8epSl4FDFMIHsnSTVXmTR2njVU9mCRamdfGzdYLS61zE0pdZYxgifQa6PYWE+Ms296wCQYUw31 xZnkW8aCCnTITTrvCsvJOxvRq12+qUBQhjFO9PttC4z1rkjn7xI46YfLYBnYWYn3eQPJAoRI1sjpVO Ve5AXDjK2+HTERbd5XTpeXey3SSC86jiC3ms398reFSrTD30VlZTE1I6CgpDYBvRL6QQ0y9Ymn9sjW EFbS6oi+f4kw+JQIKvplHnurkqmk+hI4houb+0e6bTYJf7gLbSwOCjLUpaddbfG11KhK1lvFNK7fBg IqnGKr25JMZ3skutrLJp+52lnf4OoPoforwN20HMl21RagUkaHdDun7qYZDUQ6vbpoJVs/QnR4S6uA TULvJnTHGCWN9Mi6KH7SdIEcE7DcEaYcv/o5NLnpx4evHygCJx9/Xqp/Y5nw== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 To test patchwork-bot, it is helpful to create the initial database without a certain number of the latest commits, so that on the next execution, the "missing" commits will appear new, and will be used for processing state changes. Add the --ancestors argument (defaulting to 0, the existing behavior), to provide this ability. Signed-off-by: Kees Cook --- git-patchwork-bot.py | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py index 9be945d92a8a..2ebe5b4e88e4 100755 --- a/git-patchwork-bot.py +++ b/git-patchwork-bot.py @@ -409,13 +409,17 @@ def git_run_command(gitdir, args, stdin=None): return output -def git_get_repo_heads(gitdir, branch): +def git_get_repo_heads(gitdir, branch, ancestry=None): refs = list() lines = git_get_command_lines(gitdir, ['show-ref', branch]) + if ancestry == None: + ancestry = '' + else: + ancestry = f'~{ancestry}' if lines is not None: for line in lines: (commit_id, refname) = line.split() - refs.append((refname, commit_id)) + refs.append((refname, commit_id + ancestry)) return refs @@ -1082,7 +1086,9 @@ def pwrun(repo, rsettings): if not db_exists: db_init_pw_sqlite_db(c) - db_save_repo_heads(c, git_heads) + initial_git_heads = git_get_repo_heads(repo, branch=rsettings.get('branch', '--heads'), + ancestry=cmdargs.ancestors) + db_save_repo_heads(c, initial_git_heads) # Exit early dbconn.commit() return @@ -1353,6 +1359,8 @@ if __name__ == '__main__': help='Cache directory to use instead of ~/.cache/git-patchwork-bot') parser.add_argument('--domain', default=None, help='Domain to use when creating message-ids') + parser.add_argument('--ancestors', default=None, + help='During initial database creation, consider this many ancestor commits as fresh') cmdargs = parser.parse_args()