From patchwork Mon Jun 17 05:21:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Sangorrin X-Patchwork-Id: 10997977 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A327419A2 for ; Mon, 17 Jun 2019 05:21:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8676E284E8 for ; Mon, 17 Jun 2019 05:21:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7A08528814; Mon, 17 Jun 2019 05:21:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CFE84284E8 for ; Mon, 17 Jun 2019 05:21:45 +0000 (UTC) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id A976EA5E; Mon, 17 Jun 2019 05:21:45 +0000 (UTC) X-Original-To: cip-dev@lists.cip-project.org Delivered-To: cip-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 7AD31A5E for ; Mon, 17 Jun 2019 05:21:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mo-csw.securemx.jp (mo-csw1115.securemx.jp [210.130.202.157]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 8CEB72C3 for ; Mon, 17 Jun 2019 05:21:43 +0000 (UTC) Received: by mo-csw.securemx.jp (mx-mo-csw1115) id x5H5LUAF028803; Mon, 17 Jun 2019 14:21:30 +0900 X-Iguazu-Qid: 2wHHcpwUutsh0vKO80 X-Iguazu-QSIG: v=2; s=0; t=1560748890; q=2wHHcpwUutsh0vKO80; m=SOgl3W9ihUBpxsfvfvIHB8++lyBXvQ79ieekKL7G2UU= Received: from imx2.toshiba.co.jp (imx2.toshiba.co.jp [106.186.93.51]) by relay.securemx.jp (mx-mr1110) id x5H5LTV5004676; Mon, 17 Jun 2019 14:21:29 +0900 Received: from enc01.localdomain ([106.186.93.100]) by imx2.toshiba.co.jp with ESMTP id x5H5LTvn013808; Mon, 17 Jun 2019 14:21:29 +0900 (JST) Received: from hop001.toshiba.co.jp ([133.199.164.63]) by enc01.localdomain with ESMTP id x5H5LSej008325; Mon, 17 Jun 2019 14:21:28 +0900 From: Daniel Sangorrin To: ben.hutchings@codethink.co.uk Date: Mon, 17 Jun 2019 14:21:26 +0900 X-TSB-HOP: ON Message-Id: <20190617052127.9571-1-daniel.sangorrin@toshiba.co.jp> X-Mailer: git-send-email 2.17.1 Cc: cip-dev@lists.cip-project.org Subject: [cip-dev] [cip-kernel-sec 1/2] remotes: automatically add remotes from configuration file X-BeenThere: cip-dev@lists.cip-project.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: cip-dev-bounces@lists.cip-project.org Errors-To: cip-dev-bounces@lists.cip-project.org X-Virus-Scanned: ClamAV using ClamSMTP Currently the user is required to create its own remotes by hand. This should not be necessary, because the information is already collected in conf/remotes.yml. For that reason, if we detect that any remote has not been added to the local repo then we will add those. Signed-off-by: Daniel Sangorrin --- README.md | 5 +++-- conf/remotes.yml | 9 ++++++--- scripts/import_stable.py | 9 +++++++++ scripts/templates/issue.html | 7 ++++--- 4 files changed, 22 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 4c5808f..dda94a8 100644 --- a/README.md +++ b/README.md @@ -77,8 +77,9 @@ These files, if they exist, contain a mapping where the keys are default git remote names. The values are also mappings, with the keys: -* `commit_url_prefix`: URL prefix for browsing a commit on a - branch from this remote. +* `git_repo_url`: URL of the remote repository. +* `commit_url_suffix`: URL suffix that gets appended to `git_repo_url` + for browsing a commit on a branch from this remote. * `git_name`: (optional) The name actually used for this git remote, if it's different from the default. diff --git a/conf/remotes.yml b/conf/remotes.yml index 51c523d..446bb15 100644 --- a/conf/remotes.yml +++ b/conf/remotes.yml @@ -1,6 +1,9 @@ torvalds: - commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id= + git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + commit_url_suffix: /commit/?id= stable: - commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit?id= + git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git + commit_url_suffix: /commit?id= cip: - commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git/commit?id= + git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git + commit_url_suffix: /commit?id= diff --git a/scripts/import_stable.py b/scripts/import_stable.py index b3261cf..26e45a9 100755 --- a/scripts/import_stable.py +++ b/scripts/import_stable.py @@ -35,6 +35,9 @@ def update(git_repo, remote_name): subprocess.check_call(['git', 'remote', 'update', remote_name], cwd=git_repo) +def add(git_repo, remote_name, remote_url): + subprocess.check_call(['git', 'remote', 'add', remote_name, remote_url], + cwd=git_repo) def get_backports(git_repo, remotes, branches, debug=False): backports = {} @@ -140,6 +143,12 @@ def main(git_repo, remotes, debug=False): remote_names = set(branch['git_remote'] for branch in branches) for remote_name in remote_names: + import sys + current_remotes = subprocess.check_output(['git', 'remote', 'show'], + cwd=git_repo).decode(sys.stdout.encoding).strip().split('\n') + if remote_name not in current_remotes: + add(git_repo, remotes[remote_name]['git_name'], + remotes[remote_name]['git_repo_url']) update(git_repo, remotes[remote_name]['git_name']) backports = get_backports(git_repo, remotes, branches, debug) c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches) diff --git a/scripts/templates/issue.html b/scripts/templates/issue.html index a3286e1..de42753 100644 --- a/scripts/templates/issue.html +++ b/scripts/templates/issue.html @@ -75,7 +75,8 @@ Status {% for branch, affected in branches %} {% set name = branch.short_name %} - {% set url_prefix = remotes[branch.git_remote].commit_url_prefix %} + {% set git_repo_url = remotes[branch.git_remote].git_repo_url %} + {% set url_suffix = remotes[branch.git_remote].commit_url_suffix %} {{ name }} @@ -84,7 +85,7 @@ {% if issue['fixed-by'] and issue['fixed-by'][name] and issue['fixed-by'][name] != 'never' %} fixed by {% for commit in issue['fixed-by'][name] %} - {{ commit[:12] }}{% if not loop.last %},{% endif %} + {{ commit[:12] }}{% if not loop.last %},{% endif %} {% endfor %} {% else %} never affected @@ -98,7 +99,7 @@ {% if issue['introduced-by'] and issue['introduced-by'][name] and issue['introduced-by'][name] != 'never' %} - introduced by {% for commit in issue['introduced-by'][name] %} - {{ commit[:12] }}{% if not loop.last %},{% endif %} + {{ commit[:12] }}{% if not loop.last %},{% endif %} {% endfor %} {% endif %} {% endif %} From patchwork Mon Jun 17 05:21:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Sangorrin X-Patchwork-Id: 10997979 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 685C119A2 for ; Mon, 17 Jun 2019 05:21:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 57F88284E8 for ; Mon, 17 Jun 2019 05:21:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4923128814; Mon, 17 Jun 2019 05:21:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E5CC7284E8 for ; Mon, 17 Jun 2019 05:21:51 +0000 (UTC) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id DA596A64; Mon, 17 Jun 2019 05:21:51 +0000 (UTC) X-Original-To: cip-dev@lists.cip-project.org Delivered-To: cip-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D2355A5E for ; Mon, 17 Jun 2019 05:21:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mo-csw.securemx.jp (mo-csw1116.securemx.jp [210.130.202.158]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 275992C3 for ; Mon, 17 Jun 2019 05:21:46 +0000 (UTC) Received: by mo-csw.securemx.jp (mx-mo-csw1116) id x5H5LcZp023224; Mon, 17 Jun 2019 14:21:38 +0900 X-Iguazu-Qid: 2wHHQt19eqJjgI86ZE X-Iguazu-QSIG: v=2; s=0; t=1560748898; q=2wHHQt19eqJjgI86ZE; m=HMXvE8jAisdwkEyQGFgPl8U2WiP5eKGM6oVrlIUwAU0= Received: from imx12.toshiba.co.jp (imx12.toshiba.co.jp [61.202.160.132]) by relay.securemx.jp (mx-mr1112) id x5H5LbEV024891; Mon, 17 Jun 2019 14:21:37 +0900 Received: from enc02.toshiba.co.jp ([61.202.160.51]) by imx12.toshiba.co.jp with ESMTP id x5H5LbbR028006; Mon, 17 Jun 2019 14:21:37 +0900 (JST) Received: from hop101.toshiba.co.jp ([133.199.85.107]) by enc02.toshiba.co.jp with ESMTP id x5H5LYvn002706; Mon, 17 Jun 2019 14:21:34 +0900 From: Daniel Sangorrin To: ben.hutchings@codethink.co.uk Date: Mon, 17 Jun 2019 14:21:27 +0900 X-TSB-HOP: ON Message-Id: <20190617052127.9571-2-daniel.sangorrin@toshiba.co.jp> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190617052127.9571-1-daniel.sangorrin@toshiba.co.jp> References: <20190617052127.9571-1-daniel.sangorrin@toshiba.co.jp> Cc: cip-dev@lists.cip-project.org Subject: [cip-dev] [cip-kernel-sec 2/2] git_repo: add a check for the git_repo directory X-BeenThere: cip-dev@lists.cip-project.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: cip-dev-bounces@lists.cip-project.org Errors-To: cip-dev-bounces@lists.cip-project.org X-Virus-Scanned: ClamAV using ClamSMTP Users are required to prepare a local git repository, by default "../kernel", however there is no check at the moment. Alternatively, we could also create a repository for them (mkdir ../kernel; git init .) but that for now let's just spit an error. Signed-off-by: Daniel Sangorrin --- scripts/import_stable.py | 1 + scripts/kernel_sec/branch.py | 11 +++++++++++ scripts/report_affected.py | 1 + scripts/webview.py | 1 + 4 files changed, 14 insertions(+) diff --git a/scripts/import_stable.py b/scripts/import_stable.py index 26e45a9..2d9b66b 100755 --- a/scripts/import_stable.py +++ b/scripts/import_stable.py @@ -179,6 +179,7 @@ if __name__ == '__main__': dest='git_repo', default='../kernel', help=('git repository from which to read commit logs ' '(default: ../kernel)'), + type=kernel_sec.branch.check_git_repo, metavar='DIRECTORY') parser.add_argument('--remote-name', dest='remote_name', action='append', default=[], diff --git a/scripts/kernel_sec/branch.py b/scripts/kernel_sec/branch.py index 3ede64a..0ea5086 100644 --- a/scripts/kernel_sec/branch.py +++ b/scripts/kernel_sec/branch.py @@ -219,3 +219,14 @@ def get_remotes(mappings, mainline=None, stable=None): if stable: remotes['stable']['git_name'] = stable return remotes + + +def check_git_repo(git_repo): + import argparse + if not os.path.isdir(git_repo): + msg = "directory %r not present" % git_repo + raise argparse.ArgumentTypeError(msg) + if not os.path.isdir(os.path.join(git_repo, '.git')): + msg = "directory %r is not a git repository" % git_repo + raise argparse.ArgumentTypeError(msg) + return git_repo diff --git a/scripts/report_affected.py b/scripts/report_affected.py index fac0885..bcb8168 100755 --- a/scripts/report_affected.py +++ b/scripts/report_affected.py @@ -72,6 +72,7 @@ if __name__ == '__main__': dest='git_repo', default='../kernel', help=('git repository from which to read commit logs ' '(default: ../kernel)'), + type=kernel_sec.branch.check_git_repo, metavar='DIRECTORY') parser.add_argument('--remote-name', dest='remote_name', action='append', default=[], diff --git a/scripts/webview.py b/scripts/webview.py index 9e16a0c..e334371 100755 --- a/scripts/webview.py +++ b/scripts/webview.py @@ -202,6 +202,7 @@ if __name__ == '__main__': dest='git_repo', default='../kernel', help=('git repository from which to read commit logs ' '(default: ../kernel)'), + type=kernel_sec.branch.check_git_repo, metavar='DIRECTORY') parser.add_argument('--remote-name', dest='remote_name', action='append', default=[],