From patchwork Fri May 19 09:46:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Fancellu X-Patchwork-Id: 13248129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 90DA9C77B75 for ; Fri, 19 May 2023 09:46:44 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.536874.835600 (Exim 4.92) (envelope-from ) id 1pzwgu-0006k7-13; Fri, 19 May 2023 09:46:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 536874.835600; Fri, 19 May 2023 09:46:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pzwgt-0006k0-Uj; Fri, 19 May 2023 09:46:27 +0000 Received: by outflank-mailman (input) for mailman id 536874; Fri, 19 May 2023 09:46:27 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pzwgt-0006V3-84 for xen-devel@lists.xenproject.org; Fri, 19 May 2023 09:46:27 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 057abb1a-f62a-11ed-b22d-6b7b168915f2; Fri, 19 May 2023 11:46:26 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4D94B2F4; Fri, 19 May 2023 02:47:10 -0700 (PDT) Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1E6843F73F; Fri, 19 May 2023 02:46:24 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 057abb1a-f62a-11ed-b22d-6b7b168915f2 From: Luca Fancellu To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, wei.chen@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH 1/2] xen/misra: add diff-report.py tool Date: Fri, 19 May 2023 10:46:12 +0100 Message-Id: <20230519094613.2134153-2-luca.fancellu@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230519094613.2134153-1-luca.fancellu@arm.com> References: <20230519094613.2134153-1-luca.fancellu@arm.com> MIME-Version: 1.0 Add a new tool, diff-report.py that can be used to make diff between reports generated by xen-analysis.py tool. Currently this tool supports the Xen cppcheck text report format in its operations. The tool prints every finding that is in the report passed with -r (check report) which is not in the report passed with -b (baseline). Signed-off-by: Luca Fancellu Acked-by: Stefano Stabellini Tested-by: Stefano Stabellini --- Changes from v1: - removed 2 method from class ReportEntry that landed there by a mistake on rebase. - Made the script compatible also with python2 (Stefano) --- xen/scripts/diff-report.py | 80 ++++++++++++++ .../xen_analysis/diff_tool/__init__.py | 0 .../xen_analysis/diff_tool/cppcheck_report.py | 44 ++++++++ xen/scripts/xen_analysis/diff_tool/debug.py | 40 +++++++ xen/scripts/xen_analysis/diff_tool/report.py | 100 ++++++++++++++++++ 5 files changed, 264 insertions(+) create mode 100755 xen/scripts/diff-report.py create mode 100644 xen/scripts/xen_analysis/diff_tool/__init__.py create mode 100644 xen/scripts/xen_analysis/diff_tool/cppcheck_report.py create mode 100644 xen/scripts/xen_analysis/diff_tool/debug.py create mode 100644 xen/scripts/xen_analysis/diff_tool/report.py diff --git a/xen/scripts/diff-report.py b/xen/scripts/diff-report.py new file mode 100755 index 000000000000..f97cb2355cc3 --- /dev/null +++ b/xen/scripts/diff-report.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python3 + +from __future__ import print_function +import os +import sys +from argparse import ArgumentParser +from xen_analysis.diff_tool.cppcheck_report import CppcheckReport +from xen_analysis.diff_tool.debug import Debug +from xen_analysis.diff_tool.report import ReportError + + +def log_info(text, end='\n'): + # type: (str, str) -> None + global args + global file_out + + if (args.verbose): + print(text, end=end, file=file_out) + + +def main(argv): + # type: (list) -> None + global args + global file_out + + parser = ArgumentParser(prog="diff-report.py") + parser.add_argument("-b", "--baseline", required=True, type=str, + help="Path to the baseline report.") + parser.add_argument("--debug", action='store_true', + help="Produce intermediate reports during operations.") + parser.add_argument("-o", "--out", default="stdout", type=str, + help="Where to print the tool output. Default is " + "stdout") + parser.add_argument("-r", "--report", required=True, type=str, + help="Path to the 'check report', the one checked " + "against the baseline.") + parser.add_argument("-v", "--verbose", action='store_true', + help="Print more informations during the run.") + + args = parser.parse_args() + + if args.out == "stdout": + file_out = sys.stdout + else: + try: + file_out = open(args.out, "wt") + except OSError as e: + print("ERROR: Issue opening file {}: {}".format(args.out, e)) + sys.exit(1) + + debug = Debug(args) + + try: + baseline_path = os.path.realpath(args.baseline) + log_info("Loading baseline report {}".format(baseline_path), "") + baseline = CppcheckReport(baseline_path) + baseline.parse() + debug.debug_print_parsed_report(baseline) + log_info(" [OK]") + new_rep_path = os.path.realpath(args.report) + log_info("Loading check report {}".format(new_rep_path), "") + new_rep = CppcheckReport(new_rep_path) + new_rep.parse() + debug.debug_print_parsed_report(new_rep) + log_info(" [OK]") + except ReportError as e: + print("ERROR: {}".format(e)) + sys.exit(1) + + output = new_rep - baseline + print(output, end="", file=file_out) + + if len(output) > 0: + sys.exit(1) + + sys.exit(0) + + +if __name__ == "__main__": + main(sys.argv[1:]) diff --git a/xen/scripts/xen_analysis/diff_tool/__init__.py b/xen/scripts/xen_analysis/diff_tool/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/xen/scripts/xen_analysis/diff_tool/cppcheck_report.py b/xen/scripts/xen_analysis/diff_tool/cppcheck_report.py new file mode 100644 index 000000000000..e7e80a9dde84 --- /dev/null +++ b/xen/scripts/xen_analysis/diff_tool/cppcheck_report.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python3 + +import re +from .report import Report, ReportError + + +class CppcheckReport(Report): + def __init__(self, report_path): + # type: (str) -> None + super(CppcheckReport, self).__init__(report_path) + # This matches a string like: + # path/to/file.c(,): + # and captures file name path and line number + # the last capture group is used for text substitution in __str__ + self.__report_entry_regex = re.compile(r'^(.*)\((\d+)(,\d+\):.*)$') + + def parse(self): + # type: () -> None + report_path = self.get_report_path() + try: + with open(report_path, "rt") as infile: + report_lines = infile.readlines() + except OSError as e: + raise ReportError("Issue with reading file {}: {}" + .format(report_path, e)) + for line in report_lines: + entry = self.__report_entry_regex.match(line) + if entry and entry.group(1) and entry.group(2): + file_path = entry.group(1) + line_number = int(entry.group(2)) + self.add_entry(file_path, line_number, line) + else: + raise ReportError("Malformed report entry in file {}:\n{}" + .format(report_path, line)) + + def __str__(self): + # type: () -> str + ret = "" + for entry in self.to_list(): + ret += re.sub(self.__report_entry_regex, + r'{}({}\3'.format(entry.file_path, + entry.line_number), + entry.text) + return ret diff --git a/xen/scripts/xen_analysis/diff_tool/debug.py b/xen/scripts/xen_analysis/diff_tool/debug.py new file mode 100644 index 000000000000..65cca2464110 --- /dev/null +++ b/xen/scripts/xen_analysis/diff_tool/debug.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python3 + +from __future__ import print_function +import os +from .report import Report + + +class Debug: + def __init__(self, args): + self.args = args + + def __get_debug_out_filename(self, path, type): + # type: (str, str) -> str + # Take basename + file_name = os.path.basename(path) + # Split in name and extension + file_name = os.path.splitext(file_name) + if self.args.out != "stdout": + out_folder = os.path.dirname(self.args.out) + else: + out_folder = "./" + dbg_report_path = out_folder + file_name[0] + type + file_name[1] + + return dbg_report_path + + def __debug_print_report(self, report, type): + # type: (Report, str) -> None + report_name = self.__get_debug_out_filename(report.get_report_path(), + type) + try: + with open(report_name, "wt") as outfile: + print(report, end="", file=outfile) + except OSError as e: + print("ERROR: Issue opening file {}: {}".format(report_name, e)) + + def debug_print_parsed_report(self, report): + # type: (Report) -> None + if not self.args.debug: + return + self.__debug_print_report(report, ".parsed") diff --git a/xen/scripts/xen_analysis/diff_tool/report.py b/xen/scripts/xen_analysis/diff_tool/report.py new file mode 100644 index 000000000000..4a303d61b3ea --- /dev/null +++ b/xen/scripts/xen_analysis/diff_tool/report.py @@ -0,0 +1,100 @@ +#!/usr/bin/env python3 + +import os + + +class ReportError(Exception): + pass + + +class Report(object): + class ReportEntry: + def __init__(self, file_path, line_number, entry_text, line_id): + # type: (str, int, list, int) -> None + if not isinstance(line_number, int) or \ + not isinstance(line_id, int): + raise ReportError("ReportEntry constructor wrong type args") + self.file_path = file_path + self.line_number = line_number + self.text = entry_text + self.line_id = line_id + + def __init__(self, report_path): + # type: (str) -> None + self.__entries = {} + self.__path = report_path + self.__last_line_order = 0 + + def parse(self): + # type: () -> None + raise ReportError("Please create a specialised class from 'Report'.") + + def get_report_path(self): + # type: () -> str + return self.__path + + def get_report_entries(self): + # type: () -> dict + return self.__entries + + def add_entry(self, entry_path, entry_line_number, entry_text): + # type: (str, int, str) -> None + entry = Report.ReportEntry(entry_path, entry_line_number, entry_text, + self.__last_line_order) + if entry_path in self.__entries.keys(): + self.__entries[entry_path].append(entry) + else: + self.__entries[entry_path] = [entry] + self.__last_line_order += 1 + + def to_list(self): + # type: () -> list + report_list = [] + for _, entries in self.__entries.items(): + for entry in entries: + report_list.append(entry) + + report_list.sort(key=lambda x: x.line_id) + return report_list + + def __str__(self): + # type: () -> str + ret = "" + for entry in self.to_list(): + ret += entry.file_path + ":" + entry.line_number + ":" + entry.text + + return ret + + def __len__(self): + # type: () -> int + return len(self.to_list()) + + def __sub__(self, report_b): + # type: (Report) -> Report + if self.__class__ != report_b.__class__: + raise ReportError("Diff of different type of report!") + + filename, file_extension = os.path.splitext(self.__path) + diff_report = self.__class__(filename + ".diff" + file_extension) + # Put in the diff report only records of this report that are not + # present in the report_b. + for file_path, entries in self.__entries.items(): + rep_b_entries = report_b.get_report_entries() + if file_path in rep_b_entries.keys(): + # File path exists in report_b, so check what entries of that + # file path doesn't exist in report_b and add them to the diff + rep_b_entries_num = [ + x.line_number for x in rep_b_entries[file_path] + ] + for entry in entries: + if entry.line_number not in rep_b_entries_num: + diff_report.add_entry(file_path, entry.line_number, + entry.text) + else: + # File path doesn't exist in report_b, so add every entry + # of that file path to the diff + for entry in entries: + diff_report.add_entry(file_path, entry.line_number, + entry.text) + + return diff_report