[v3,1/2] Documentation: dev-tools: Add a section for static analysis tools

Commit Message

Marcelo Schmitt March 30, 2022, 9:49 p.m. UTC

Complement the Kernel Testing Guide documentation page by adding a
section about static analysis tools.

Signed-off-by: Marcelo Schmitt <[e-mail]>
Acked-by: Daniel Latypov <[e-mail]>
Acked-by: Dan Carpenter <[e-mail]>
Acked-by: Julia Lawall <[e-mail]>
Reviewed-by: David Gow <[e-mail]>
Reviewed-by: Shuah Khan <[e-mail]>
---
Change log v2 -> v3:
- Added Julia's acknowledgment tag

Change log v1 -> v2:
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmable

 Documentation/dev-tools/testing-overview.rst | 31 ++++++++++++++++++++
 1 file changed, 31 insertions(+)

Comments

Dongliang Mu March 31, 2022, 4:14 a.m. UTC | #1

On Thu, Mar 31, 2022 at 12:07 PM Marcelo Schmitt
<[e-mail]> wrote:
>
> Complement the Kernel Testing Guide documentation page by adding a
> section about static analysis tools.
>
> Signed-off-by: Marcelo Schmitt <[e-mail]>
> Acked-by: Daniel Latypov <[e-mail]>
> Acked-by: Dan Carpenter <[e-mail]>
> Acked-by: Julia Lawall <[e-mail]>
> Reviewed-by: David Gow <[e-mail]>
> Reviewed-by: Shuah Khan <[e-mail]>
> ---
> Change log v2 -> v3:
> - Added Julia's acknowledgment tag
>
> Change log v1 -> v2:
> - Brought generic tool characteristics to the intro paragraph
> - Made explicit that these tools run at compile time
> - Added a note of caution about false positives
> - Updated Coccinelle info to make it sound better and be more skimmable
>
>  Documentation/dev-tools/testing-overview.rst | 31 ++++++++++++++++++++
>  1 file changed, 31 insertions(+)
>
> diff --git a/Documentation/dev-tools/testing-overview.rst b/Documentation/dev-tools/testing-overview.rst
> index 65feb81edb14..b5e02dd3fd94 100644
> --- a/Documentation/dev-tools/testing-overview.rst
> +++ b/Documentation/dev-tools/testing-overview.rst
> @@ -115,3 +115,34 @@ that none of these errors are occurring during the test.
>  Some of these tools integrate with KUnit or kselftest and will
>  automatically fail tests if an issue is detected.
>
> +Static Analysis Tools
> +=====================
> +
> +In addition to testing a running kernel, one can also analyze kernel source code
> +directly (**at compile time**) using **static analysis** tools. The tools
> +commonly used in the kernel allow one to inspect the whole source tree or just
> +specific files within it. They make it easier to detect and fix problems during
> +the development process.
> +
> +Sparse can help test the kernel by performing type-checking, lock checking,
> +value range checking, in addition to reporting various errors and warnings while
> +examining the code. See the Documentation/dev-tools/sparse.rst documentation
> +page for details on how to use it.
> +
> +Smatch extends Sparse and provides additional checks for programming logic
> +mistakes such as missing breaks in switch statements, unused return values on
> +error checking, forgetting to set an error code in the return of an error path,
> +etc. Smatch also has tests against more serious issues such as integer
> +overflows, null pointer dereferences, and memory leaks. See the project page at
> +http://smatch.sourceforge.net/.
> +
> +Coccinelle is another static analyzer at our disposal. Coccinelle is often used
> +to aid refactoring and collateral evolution of source code, but it can also help
> +to avoid certain bugs that occur in common code patterns. The types of tests
> +available include API tests, tests for correct usage of kernel iterators, checks
> +for the soundness of free operations, analysis of locking behavior, and further
> +tests known to help keep consistent kernel usage. See the
> +Documentation/dev-tools/coccinelle.rst documentation page for details.
> +
> +Beware, though, that static analysis tools suffer from **false positives**.
> +Errors and warns need to be evaluated carefully before attempting to fix them.

Hi Marcelo,

Should we include static analysis tools based on LLVM? For example,
Clang static analysis.

> --
> 2.35.1
>

Marcelo Schmitt March 31, 2022, 12:10 p.m. UTC | #2

Hi Dongliang,

On 03/31, Dongliang Mu wrote:
> On Thu, Mar 31, 2022 at 12:07 PM Marcelo Schmitt
> <[e-mail]> wrote:
> >
> > Complement the Kernel Testing Guide documentation page by adding a
> > section about static analysis tools.
> >
> > Signed-off-by: Marcelo Schmitt <[e-mail]>
> > Acked-by: Daniel Latypov <[e-mail]>
> > Acked-by: Dan Carpenter <[e-mail]>
> > Acked-by: Julia Lawall <[e-mail]>
> > Reviewed-by: David Gow <[e-mail]>
> > Reviewed-by: Shuah Khan <[e-mail]>
> > ---
> > Change log v2 -> v3:
> > - Added Julia's acknowledgment tag
> >
> > Change log v1 -> v2:
> > - Brought generic tool characteristics to the intro paragraph
> > - Made explicit that these tools run at compile time
> > - Added a note of caution about false positives
> > - Updated Coccinelle info to make it sound better and be more skimmable
> >
> >  Documentation/dev-tools/testing-overview.rst | 31 ++++++++++++++++++++
> >  1 file changed, 31 insertions(+)
> >
> > diff --git a/Documentation/dev-tools/testing-overview.rst b/Documentation/dev-tools/testing-overview.rst
> > index 65feb81edb14..b5e02dd3fd94 100644
> > --- a/Documentation/dev-tools/testing-overview.rst
> > +++ b/Documentation/dev-tools/testing-overview.rst
> > @@ -115,3 +115,34 @@ that none of these errors are occurring during the test.
> >  Some of these tools integrate with KUnit or kselftest and will
> >  automatically fail tests if an issue is detected.
> >
> > +Static Analysis Tools
> > +=====================
> > +
> > +In addition to testing a running kernel, one can also analyze kernel source code
> > +directly (**at compile time**) using **static analysis** tools. The tools
> > +commonly used in the kernel allow one to inspect the whole source tree or just
> > +specific files within it. They make it easier to detect and fix problems during
> > +the development process.
> > +
> > +Sparse can help test the kernel by performing type-checking, lock checking,
> > +value range checking, in addition to reporting various errors and warnings while
> > +examining the code. See the Documentation/dev-tools/sparse.rst documentation
> > +page for details on how to use it.
> > +
> > +Smatch extends Sparse and provides additional checks for programming logic
> > +mistakes such as missing breaks in switch statements, unused return values on
> > +error checking, forgetting to set an error code in the return of an error path,
> > +etc. Smatch also has tests against more serious issues such as integer
> > +overflows, null pointer dereferences, and memory leaks. See the project page at
> > +http://smatch.sourceforge.net/.
> > +
> > +Coccinelle is another static analyzer at our disposal. Coccinelle is often used
> > +to aid refactoring and collateral evolution of source code, but it can also help
> > +to avoid certain bugs that occur in common code patterns. The types of tests
> > +available include API tests, tests for correct usage of kernel iterators, checks
> > +for the soundness of free operations, analysis of locking behavior, and further
> > +tests known to help keep consistent kernel usage. See the
> > +Documentation/dev-tools/coccinelle.rst documentation page for details.
> > +
> > +Beware, though, that static analysis tools suffer from **false positives**.
> > +Errors and warns need to be evaluated carefully before attempting to fix them.
> 
> Hi Marcelo,
> 
> Should we include static analysis tools based on LLVM? For example,
> Clang static analysis.

I think that would be a good addition. I haven't checked out Clang tools
though, so it would take me a bit more time to write something about that.

> 
> > --
> > 2.35.1
> >

Jonathan Corbet April 5, 2022, 3:50 p.m. UTC | #3

Marcelo Schmitt <[e-mail]> writes:

> On 03/31, Dongliang Mu wrote:

>> Should we include static analysis tools based on LLVM? For example,
>> Clang static analysis.
>
> I think that would be a good addition. I haven't checked out Clang tools
> though, so it would take me a bit more time to write something about that.

That seems like a good topic for a future patch.  Meanwhile I've applied
this series, thanks.

jon

Patch

diff --git a/Documentation/dev-tools/testing-overview.rst b/Documentation/dev-tools/testing-overview.rst
index 65feb81edb14..b5e02dd3fd94 100644
--- a/Documentation/dev-tools/testing-overview.rst
+++ b/Documentation/dev-tools/testing-overview.rst
@@ -115,3 +115,34 @@  that none of these errors are occurring during the test.
 Some of these tools integrate with KUnit or kselftest and will
 automatically fail tests if an issue is detected.
 
+Static Analysis Tools
+=====================
+
+In addition to testing a running kernel, one can also analyze kernel source code
+directly (**at compile time**) using **static analysis** tools. The tools
+commonly used in the kernel allow one to inspect the whole source tree or just
+specific files within it. They make it easier to detect and fix problems during
+the development process.
+
+Sparse can help test the kernel by performing type-checking, lock checking,
+value range checking, in addition to reporting various errors and warnings while
+examining the code. See the Documentation/dev-tools/sparse.rst documentation
+page for details on how to use it.
+
+Smatch extends Sparse and provides additional checks for programming logic
+mistakes such as missing breaks in switch statements, unused return values on
+error checking, forgetting to set an error code in the return of an error path,
+etc. Smatch also has tests against more serious issues such as integer
+overflows, null pointer dereferences, and memory leaks. See the project page at
+http://smatch.sourceforge.net/.
+
+Coccinelle is another static analyzer at our disposal. Coccinelle is often used
+to aid refactoring and collateral evolution of source code, but it can also help
+to avoid certain bugs that occur in common code patterns. The types of tests
+available include API tests, tests for correct usage of kernel iterators, checks
+for the soundness of free operations, analysis of locking behavior, and further
+tests known to help keep consistent kernel usage. See the
+Documentation/dev-tools/coccinelle.rst documentation page for details.
+
+Beware, though, that static analysis tools suffer from **false positives**.
+Errors and warns need to be evaluated carefully before attempting to fix them.