[v1,00/12] Add Thread Sanitizer support to QEMU

Message

Robert Foley May 29, 2020, 1:21 p.m. UTC

Changes in v1:
- Refocused the patch more on enabling TSan and less on removing warnings.
- Because of this we removed several patches from the series that
  were fixing and/or suppressing warnings.
- Removed from the series some patches that had overlap with the
  per-cpu-lock patch.  We will let the per-cpu-lock patch fix those warnings.
- Removed built-in support for blacklist.   
  The idea here being that all warnings should eventually be fixed and
  this blacklist is for test/debug.
- Enhanced the documentation on TSan support in testing.rst.
- Improved the comments in tsan.h.
- Changed configure to error out if --enable-tsan provided but
  TSan not available.

This patch series continues the work done by Emilio Cota and others to add
Thread Sanitizer (TSan) support to QEMU.

The starting point for this work was Emilio's branch here:
https://github.com/cota/qemu/commits/tsan
specifically this commit: 0be125fc0afd47218b34d2019abdd19b644f3199

The main purpose of this patch is to enable TSan support so that 
QEMU developers can start using the tool.  
We found this tool useful and even ran it on our recent changes in
the cpu-locks series, which fixes many warnings.
Clearly there is work to do here to clean up all the warnings. :)
We have also made an effort to introduce enough of the TSan suppression
mechanisms, so that others can continue this work.

This series adds support for:
- configure option for --enable-tsan.
- testing.rst has the full details on how to use TSan with or without docker,
  including all the suppression mechanisms.
- We added an Ubuntu 20.04 docker that supports TSan builds.
- We added an example blacklist file for files or functions TSan should ignore 
  at compile time.  This can now be specified manually.
- Added a suppression file for TSan to suppress certain warnings at run time.
- Added tsan.h with annotations which also can be used to suppress warnings.

Issues:
- When running docker-test-quick under TSan there are several tests which hang
  - The unit tests which seem to hang under TSan:
    test-char, test-qdev-global-props, and test-qga.  
  - If we comment out those tests, check-unit finishes, albeit with 
    a couple of warnings. :)

Emilio G. Cota (7):
  cpu: convert queued work to a QSIMPLEQ
  thread: add qemu_spin_destroy
  cputlb: destroy CPUTLB with tlb_destroy
  qht: call qemu_spin_destroy for head buckets
  tcg: call qemu_spin_destroy for tb->jmp_lock
  translate-all: call qemu_spin_destroy for PageDesc
  thread: add tsan annotations to QemuSpin

Lingfeng Yang (1):
  configure: add --enable-tsan flag + fiber annotations for
    coroutine-ucontext

Robert Foley (4):
  tests/docker: Added docker build support for TSan.
  include/qemu: Added tsan.h for annotations.
  util: Added tsan annotate for thread name.
  docs: Added details on TSan to testing.rst

 accel/tcg/cputlb.c                         |  15 +++
 accel/tcg/translate-all.c                  |  19 +++-
 configure                                  |  41 ++++++++
 cpus-common.c                              |  25 ++---
 cpus.c                                     |  14 ++-
 docs/devel/testing.rst                     | 113 +++++++++++++++++++++
 exec.c                                     |   1 +
 hw/core/cpu.c                              |   1 +
 include/exec/exec-all.h                    |   8 ++
 include/hw/core/cpu.h                      |   6 +-
 include/qemu/thread.h                      |  38 ++++++-
 include/qemu/tsan.h                        |  71 +++++++++++++
 include/tcg/tcg.h                          |   3 +-
 tcg/tcg.c                                  |  19 +++-
 tests/docker/Makefile.include              |   2 +
 tests/docker/common.rc                     |  19 ++++
 tests/docker/dockerfiles/ubuntu2004.docker |  65 ++++++++++++
 tests/tsan/blacklist.tsan                  |  10 ++
 tests/tsan/suppressions.tsan               |  14 +++
 util/coroutine-ucontext.c                  |  97 ++++++++++++++++--
 util/qemu-thread-posix.c                   |   2 +
 util/qht.c                                 |   1 +
 22 files changed, 547 insertions(+), 37 deletions(-)
 create mode 100644 include/qemu/tsan.h
 create mode 100644 tests/docker/dockerfiles/ubuntu2004.docker
 create mode 100644 tests/tsan/blacklist.tsan
 create mode 100644 tests/tsan/suppressions.tsan