From patchwork Wed Oct 5 14:04:43 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jani Nikula X-Patchwork-Id: 9362971 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 595F2607D6 for ; Wed, 5 Oct 2016 14:05:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 48EA328974 for ; Wed, 5 Oct 2016 14:05:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3C59A2865B; Wed, 5 Oct 2016 14:05:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D42A286BB for ; Wed, 5 Oct 2016 14:05:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754570AbcJEOEv convert rfc822-to-8bit (ORCPT ); Wed, 5 Oct 2016 10:04:51 -0400 Received: from mga06.intel.com ([134.134.136.31]:53259 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752516AbcJEOEt (ORCPT ); Wed, 5 Oct 2016 10:04:49 -0400 Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga104.jf.intel.com with ESMTP; 05 Oct 2016 07:04:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,449,1473145200"; d="scan'208";a="769139782" Received: from jnikula-mobl.fi.intel.com (HELO localhost) ([10.237.72.162]) by FMSMGA003.fm.intel.com with ESMTP; 05 Oct 2016 07:04:43 -0700 From: Jani Nikula To: Daniel Vetter , Julia Lawall Cc: Linux PM list , kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-clk@vger.kernel.org, "linaro-mm-sig\@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra\@vger.kernel.org" , drbd-dev@lists.linbit.com, linux-metag@vger.kernel.org, "linux-arm-kernel\@lists.infradead.org" , "linux-media\@vger.kernel.org" , linux-doc@vger.kernel.org Subject: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) In-Reply-To: Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> Date: Wed, 05 Oct 2016 17:04:43 +0300 Message-ID: <87h98quc1w.fsf@intel.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Wed, 05 Oct 2016, Daniel Vetter wrote: > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > into a lint/checker pass over the entire kernel. I think that'd would > be more robust instead of trying to approximate the real kerneldoc > parser. Otoh that parser is a horror show of a perl/regex driven state > machine ;-) > > Jani, can you pls digg out these patches? Can't find them right now ... Expanding the massive Cc: with linux-doc list... Here goes. It's a quick hack from months ago, but still seems to somewhat work. At least for the kernel-doc parts. The reStructuredText lint part isn't all that great, and doesn't have mapping to line numbers like the Sphinx kernel-doc extension does. Anyway I'm happy how this integrates with kernel build CHECK and C=1/C=2. I guess Julia's goal is to automate the *fixing* of some of the error classes from kernel-doc. Not sure how well this could be made to integrate with any of that. BR, Jani. From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 31 May 2016 18:11:33 +0300 Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst correctness Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo Cc: Jani Nikula Simple kernel-doc and reStructuredText lint tool that can be used independently and as a kernel build CHECK tool to validate kernel-doc comments. Independent usage: $ kernel-doc-rst-lint FILE Kernel CHECK usage: $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) Depends on docutils and the rst-lint package https://pypi.python.org/pypi/restructuredtext_lint Signed-off-by: Jani Nikula --- scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100755 scripts/kernel-doc-rst-lint diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint new file mode 100755 index 000000000000..7e0157679f83 --- /dev/null +++ b/scripts/kernel-doc-rst-lint @@ -0,0 +1,106 @@ +#!/usr/bin/env python +# coding=utf-8 +# +# Copyright © 2016 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +# IN THE SOFTWARE. +# +# Authors: +# Jani Nikula +# +# Simple kernel-doc and reStructuredText lint tool that can be used +# independently and as a kernel build CHECK tool to validate kernel-doc +# comments. +# +# Independent usage: +# $ kernel-doc-rst-lint FILE +# +# Kernel CHECK usage: +# $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) +# +# Depends on docutils and the rst-lint package +# https://pypi.python.org/pypi/restructuredtext_lint +# + +import os +import subprocess +import sys + +from docutils.parsers.rst import directives +from docutils.parsers.rst import Directive +from docutils.parsers.rst import roles +from docutils import nodes, statemachine +import restructuredtext_lint + +class DummyDirective(Directive): + required_argument = 1 + optional_arguments = 0 + option_spec = { } + has_content = True + + def run(self): + return [] + +# Fake the Sphinx C Domain directives and roles +directives.register_directive('c:function', DummyDirective) +directives.register_directive('c:type', DummyDirective) +roles.register_generic_role('c:func', nodes.emphasis) +roles.register_generic_role('c:type', nodes.emphasis) + +# We accept but ignore parameters to be compatible with how the kernel build +# invokes CHECK. +if len(sys.argv) < 2: + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'); + sys.exit(1) + +infile = sys.argv[len(sys.argv) - 1] +cmd = ['scripts/kernel-doc', '-rst', infile] + +try: + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) + out, err = p.communicate() + + # python2 needs conversion to unicode. + # python3 with universal_newlines=True returns strings. + if sys.version_info.major < 3: + out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') + + # kernel-doc errors + sys.stderr.write(err) + if p.returncode != 0: + sys.exit(p.returncode) + + # restructured text errors + lines = statemachine.string2lines(out, 8, convert_whitespace=True) + lint_errors = restructuredtext_lint.lint(out, infile) + for error in lint_errors: + # Ignore INFO + if error.level <= 1: + continue + + print(error.source + ': ' + error.type + ': ' + error.full_message) + if error.line is not None: + print('Context:') + print('\t' + lines[error.line - 1]) + print('\t' + lines[error.line]) + +except Exception as e: + sys.stderr.write(str(e) + '\n') + sys.exit(1)