[v3,0/6] kernel: introduce uaccess logging

Message

Peter Collingbourne Dec. 8, 2021, 4:48 a.m. UTC

This patch series introduces a kernel feature known as uaccess
logging, which allows userspace programs to be made aware of the
address and size of uaccesses performed by the kernel during
the servicing of a syscall. More details on the motivation
for and interface to this feature are available in the file
Documentation/admin-guide/uaccess-logging.rst added by the final
patch in the series.

Because we don't have a common kernel entry/exit code path that is used
on all architectures, uaccess logging is only implemented for arm64
and architectures that use CONFIG_GENERIC_ENTRY, i.e. x86 and s390.

The proposed interface is the result of numerous iterations and
prototyping and is based on a proposal by Dmitry Vyukov. The interface
preserves the correspondence between uaccess log identity and syscall
identity while tolerating incoming asynchronous signals in the interval
between setting up the logging and the actual syscall. We considered
a number of alternative designs but rejected them for various reasons:

- The design from v1 of this patch [1] proposed notifying the kernel
  of the address and size of the uaccess buffer via a prctl that
  would also automatically mask and unmask asynchronous signals as
  needed, but this would require multiple syscalls per "real" syscall,
  harming performance.

- We considered extending the syscall calling convention to
  designate currently-unused registers to be used to pass the
  location of the uaccess buffer, but this was rejected for being
  architecture-specific.

- One idea that we considered involved using the stack pointer address
  as a unique identifier for the syscall, but this currently would
  need to be arch-specific as we currently do not appear to have an
  arch-generic way of retrieving the stack pointer; the userspace
  side would also need some arch-specific code for this to work. It's
  also possible that a longjmp() past the signal handler would make
  the stack pointer address not unique enough for this purpose.

We also evaluated implementing this on top of the existing tracepoint
facility, but concluded that it is not suitable for this purpose:

- Tracepoints have a per-task granularity at best, whereas we really want
  to trace per-syscall. This is so that we can exclude syscalls that
  should not be traced, such as syscalls that make up part of the
  sanitizer implementation (to avoid infinite recursion when e.g. printing
  an error report).

- Tracing would need to be synchronous in order to produce useful
  stack traces. For example this could be achieved using the new SIGTRAP
  on perf events mechanism. However, this would require logging each
  access to the stack (in the form of a sigcontext) and this is more
  likely to overflow the stack due to being much larger than a uaccess
  buffer entry as well as being unbounded, in contrast to the bounded
  buffer size passed to prctl(). An approach based on signal handlers is
  also likely to fall foul of the asynchronous signal issues mentioned
  previously, together with needing sigreturn to be handled specially
  (because it copies a sigcontext from userspace) otherwise we could
  never return from the signal handler. Furthermore, arguments to the
  trace events are not available to SIGTRAP. (This on its own wouldn't
  be insurmountable though -- we could add the arguments as fields
  to siginfo.)

- The API in https://www.kernel.org/doc/Documentation/trace/ftrace.txt
  -- e.g. trace_pipe_raw gives access to the internal ring buffer, but
  I don't think it's usable because it's per-CPU and not per-task.

- Tracepoints can be used by eBPF programs, but eBPF programs may
  only be loaded as root, among other potential headaches.

[1] https://lore.kernel.org/all/[e-mail]/

Peter Collingbourne (6):
  include: split out uaccess instrumentation into a separate header
  uaccess-buffer: add core code
  fs: use copy_from_user_nolog() to copy mount() data
  uaccess-buffer: add CONFIG_GENERIC_ENTRY support
  arm64: add support for uaccess logging
  Documentation: document uaccess logging

 Documentation/admin-guide/index.rst           |   1 +
 Documentation/admin-guide/uaccess-logging.rst | 151 ++++++++++++++++++
 arch/Kconfig                                  |   6 +
 arch/arm64/Kconfig                            |   1 +
 arch/arm64/include/asm/thread_info.h          |   7 +-
 arch/arm64/kernel/ptrace.c                    |   7 +
 arch/arm64/kernel/signal.c                    |   5 +
 arch/arm64/kernel/syscall.c                   |   1 +
 fs/exec.c                                     |   3 +
 fs/namespace.c                                |   8 +-
 include/linux/entry-common.h                  |   2 +
 include/linux/instrumented-uaccess.h          |  53 ++++++
 include/linux/instrumented.h                  |  34 ----
 include/linux/sched.h                         |   5 +
 include/linux/thread_info.h                   |   4 +
 include/linux/uaccess-buffer-info.h           |  46 ++++++
 include/linux/uaccess-buffer.h                | 112 +++++++++++++
 include/linux/uaccess.h                       |   2 +-
 include/uapi/linux/prctl.h                    |   3 +
 include/uapi/linux/uaccess-buffer.h           |  27 ++++
 kernel/Makefile                               |   1 +
 kernel/bpf/helpers.c                          |   7 +-
 kernel/entry/common.c                         |  10 ++
 kernel/fork.c                                 |   4 +
 kernel/signal.c                               |   4 +-
 kernel/sys.c                                  |   6 +
 kernel/uaccess-buffer.c                       | 129 +++++++++++++++
 lib/iov_iter.c                                |   2 +-
 lib/usercopy.c                                |   2 +-
 29 files changed, 602 insertions(+), 41 deletions(-)
 create mode 100644 Documentation/admin-guide/uaccess-logging.rst
 create mode 100644 include/linux/instrumented-uaccess.h
 create mode 100644 include/linux/uaccess-buffer-info.h
 create mode 100644 include/linux/uaccess-buffer.h
 create mode 100644 include/uapi/linux/uaccess-buffer.h
 create mode 100644 kernel/uaccess-buffer.c

Comments

Marco Elver Dec. 8, 2021, 3:33 p.m. UTC | #1

On Wed, 8 Dec 2021 at 05:48, Peter Collingbourne <[e-mail]> wrote:
[...]
> Peter Collingbourne (6):
>   include: split out uaccess instrumentation into a separate header
>   uaccess-buffer: add core code
>   fs: use copy_from_user_nolog() to copy mount() data
>   uaccess-buffer: add CONFIG_GENERIC_ENTRY support
>   arm64: add support for uaccess logging
>   Documentation: document uaccess logging

I think it needs to be possible to disable the feature via a Kconfig
option. Not all systems want or could even tolerate the additional
overheads -- even though you say they are minimal elsewhere. For
example, some embedded systems most likely have absolutely no use for
this feature, and the increase in .text might be unacceptable. Certain
features that we usually take for granted are no different (see
init/Kconfig: FUTEX, EPOLL, .. etc). If you'd like it enabled by
default, given the overheads are small enough, it can do "default y"
and be configurable only "if EXPERT".

Is it possible to add a kselftest-style test to
tools/testing/selftests? In addition to the basic tests, can certain
non-trivial properties, like masking of signals, also be tested? I
think that'd be extremely valuable, because I'm sure we'd have to
backport this to several older kernels.

Thanks,
-- Marco

Peter Collingbourne Dec. 9, 2021, 10:12 p.m. UTC | #2

On Wed, Dec 8, 2021 at 7:33 AM Marco Elver <[e-mail]> wrote:
>
> On Wed, 8 Dec 2021 at 05:48, Peter Collingbourne <[e-mail]> wrote:
> [...]
> > Peter Collingbourne (6):
> >   include: split out uaccess instrumentation into a separate header
> >   uaccess-buffer: add core code
> >   fs: use copy_from_user_nolog() to copy mount() data
> >   uaccess-buffer: add CONFIG_GENERIC_ENTRY support
> >   arm64: add support for uaccess logging
> >   Documentation: document uaccess logging
>
> I think it needs to be possible to disable the feature via a Kconfig
> option. Not all systems want or could even tolerate the additional
> overheads -- even though you say they are minimal elsewhere. For
> example, some embedded systems most likely have absolutely no use for
> this feature, and the increase in .text might be unacceptable. Certain
> features that we usually take for granted are no different (see
> init/Kconfig: FUTEX, EPOLL, .. etc). If you'd like it enabled by
> default, given the overheads are small enough, it can do "default y"
> and be configurable only "if EXPERT".

Okay, done.

> Is it possible to add a kselftest-style test to
> tools/testing/selftests? In addition to the basic tests, can certain
> non-trivial properties, like masking of signals, also be tested? I
> think that'd be extremely valuable, because I'm sure we'd have to
> backport this to several older kernels.

Yes, I've added a new patch with a kselftest. (Good thing I did,
because it (together with DEBUG_PREEMPT) uncovered a bug in the
pre/post-exit-loop code. Fixed in v4.)

Peter