| Message ID | 20200115171333.28811-1-kpsingh@chromium.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | MAC and Audit policy using eBPF (KRSI) | expand |
On Wed, Jan 15, 2020 at 9:15 AM KP Singh <kpsingh@chromium.org> wrote: > > From: KP Singh <kpsingh@google.com> > > # Changes since v1 (https://lore.kernel.org/bpf/20191220154208.15895-1-kpsingh@chromium.org/): > > * Eliminate the requirement to maintain LSM hooks separately in > security/bpf/hooks.h Use BPF trampolines to dynamically allocate > security hooks > * Drop the use of securityfs as bpftool provides the required > introspection capabilities. Update the tests to use the bpf_skeleton > and global variables > * Use O_CLOEXEC anonymous fds to represent BPF attachment in line with > the other BPF programs with the possibility to use bpf program pinning > in the future to provide "permanent attachment". > * Drop the logic based on prog names for handling re-attachment. > * Drop bpf_lsm_event_output from this series and send it as a separate > patch. > > # Motivation > > Google does analysis of rich runtime security data to detect and thwart > threats in real-time. Currently, this is done in custom kernel modules > but we would like to replace this with something that's upstream and > useful to others. > > The current kernel infrastructure for providing telemetry (Audit, Perf > etc.) is disjoint from access enforcement (i.e. LSMs). Augmenting the > information provided by audit requires kernel changes to audit, its > policy language and user-space components. Furthermore, building a MAC > policy based on the newly added telemetry data requires changes to > various LSMs and their respective policy languages. > > This patchset proposes a new stackable and privileged LSM which allows > the LSM hooks to be implemented using eBPF. This facilitates a unified > and dynamic (not requiring re-compilation of the kernel) audit and MAC > policy. > > # Why an LSM? > > Linux Security Modules target security behaviours rather than the > kernel's API. For example, it's easy to miss out a newly added system > call for executing processes (eg. execve, execveat etc.) but the LSM > framework ensures that all process executions trigger the relevant hooks > irrespective of how the process was executed. > > Allowing users to implement LSM hooks at runtime also benefits the LSM > eco-system by enabling a quick feedback loop from the security community > about the kind of behaviours that the LSM Framework should be targeting. > > # How does it work? > > The LSM introduces a new eBPF (https://docs.cilium.io/en/v1.6/bpf/) > program type BPF_PROG_TYPE_LSM which can only be attached to LSM hooks. > Attachment requires CAP_SYS_ADMIN for loading eBPF programs and > CAP_MAC_ADMIN for modifying MAC policies. > > The eBPF programs are attached to a separate security_hook_heads > maintained by the BPF LSM for mutable hooks and executed after all the > statically defined hooks (i.e. the ones declared by SELinux, AppArmor, > Smack etc). This also ensures that statically defined LSM hooks retain > the behaviour of "being read-only after init", i.e. __lsm_ro_after_init. > > Upon attachment, a security hook is dynamically allocated with > arch_bpf_prepare_trampoline which generates code to handle the > conversion from the signature of the hook to the BPF context and allows > the JIT'ed BPF program to be called as a C function with the same > arguments as the LSM hooks. If any of the attached eBPF programs returns > an error (like ENOPERM), the behaviour represented by the hook is > denied. > > Audit logs can be written using a format chosen by the eBPF program to > the perf events buffer or to global eBPF variables or maps and can be > further processed in user-space. > > # BTF Based Design > > The current design uses BTF > (https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html, > https://lwn.net/Articles/803258/) which allows verifiable read-only > structure accesses by field names rather than fixed offsets. This allows > accessing the hook parameters using a dynamically created context which > provides a certain degree of ABI stability: > > > // Only declare the structure and fields intended to be used > // in the program > struct vm_area_struct { > unsigned long vm_start; > } __attribute__((preserve_access_index)); > It would be nice to also mention that you don't even have to "re-define" these structs if you use vmlinux.h generated with `bpftool btf dump file <path-to-vm-linux-or-/sys/kernel/btf/vmlinux> format c`. Its output will contain all types of the kernel, including internal ones not exposed through any public headers. And it will also automatically have __attribute__((preserve_access_index)) applied to all structs/unions. It can be pre-generated and checked in somewhere along the application or generated on the fly, if environment and use case allows. > // Declare the eBPF program mprotect_audit which attaches to > // to the file_mprotect LSM hook and accepts three arguments. > SEC("lsm/file_mprotect") > int BPF_PROG(mprotect_audit, struct vm_area_struct *vma, > unsigned long reqprot, unsigned long prot) > { > unsigned long vm_start = vma->vm_start; > > return 0; > } > > By relocating field offsets, BTF makes a large portion of kernel data > structures readily accessible across kernel versions without requiring a > large corpus of BPF helper functions and requiring recompilation with > every kernel version. The BTF type information is also used by the BPF > verifier to validate memory accesses within the BPF program and also > prevents arbitrary writes to the kernel memory. > > The limitations of BTF compatibility are described in BPF Co-Re > (http://vger.kernel.org/bpfconf2019_talks/bpf-core.pdf, i.e. field > renames, #defines and changes to the signature of LSM hooks). > > This design imposes that the MAC policy (eBPF programs) be updated when > the inspected kernel structures change outside of BTF compatibility > guarantees. In practice, this is only required when a structure field > used by a current policy is removed (or renamed) or when the used LSM > hooks change. We expect the maintenance cost of these changes to be > acceptable as compared to the previous design > (https://lore.kernel.org/bpf/20190910115527.5235-1-kpsingh@chromium.org/). > [...]
On Wed, Jan 15 2020 at 18:13, KP Singh wrote: > From: KP Singh <kpsingh@google.com> [...] > KP Singh (10): > bpf: btf: Make some of the API visible outside BTF > bpf: lsm: Add a skeleton and config options > bpf: lsm: Introduce types for eBPF based LSM > bpf: lsm: Add mutable hooks list for the BPF LSM > bpf: lsm: BTF API for LSM hooks > bpf: lsm: Implement attach, detach and execution > bpf: lsm: Make the allocated callback RO+X > tools/libbpf: Add support for BPF_PROG_TYPE_LSM > bpf: lsm: Add selftests for BPF_PROG_TYPE_LSM > bpf: lsm: Add Documentation Just to note internal review cycles at Google; For the whole set: Reviewed-by: Brendan Jackman <jackmanb@chromium.org>
On 15-Jan 14:12, Andrii Nakryiko wrote: > On Wed, Jan 15, 2020 at 9:15 AM KP Singh <kpsingh@chromium.org> wrote: > > > > From: KP Singh <kpsingh@google.com> > > > > # Changes since v1 (https://lore.kernel.org/bpf/20191220154208.15895-1-kpsingh@chromium.org/): > > > > * Eliminate the requirement to maintain LSM hooks separately in > > security/bpf/hooks.h Use BPF trampolines to dynamically allocate > > security hooks > > * Drop the use of securityfs as bpftool provides the required > > introspection capabilities. Update the tests to use the bpf_skeleton > > and global variables > > * Use O_CLOEXEC anonymous fds to represent BPF attachment in line with > > the other BPF programs with the possibility to use bpf program pinning > > in the future to provide "permanent attachment". > > * Drop the logic based on prog names for handling re-attachment. > > * Drop bpf_lsm_event_output from this series and send it as a separate > > patch. > > > > # Motivation > > > > Google does analysis of rich runtime security data to detect and thwart > > threats in real-time. Currently, this is done in custom kernel modules > > but we would like to replace this with something that's upstream and > > useful to others. > > > > The current kernel infrastructure for providing telemetry (Audit, Perf > > etc.) is disjoint from access enforcement (i.e. LSMs). Augmenting the > > information provided by audit requires kernel changes to audit, its > > policy language and user-space components. Furthermore, building a MAC > > policy based on the newly added telemetry data requires changes to > > various LSMs and their respective policy languages. > > > > This patchset proposes a new stackable and privileged LSM which allows > > the LSM hooks to be implemented using eBPF. This facilitates a unified > > and dynamic (not requiring re-compilation of the kernel) audit and MAC > > policy. > > > > # Why an LSM? > > > > Linux Security Modules target security behaviours rather than the > > kernel's API. For example, it's easy to miss out a newly added system > > call for executing processes (eg. execve, execveat etc.) but the LSM > > framework ensures that all process executions trigger the relevant hooks > > irrespective of how the process was executed. > > > > Allowing users to implement LSM hooks at runtime also benefits the LSM > > eco-system by enabling a quick feedback loop from the security community > > about the kind of behaviours that the LSM Framework should be targeting. > > > > # How does it work? > > > > The LSM introduces a new eBPF (https://docs.cilium.io/en/v1.6/bpf/) > > program type BPF_PROG_TYPE_LSM which can only be attached to LSM hooks. > > Attachment requires CAP_SYS_ADMIN for loading eBPF programs and > > CAP_MAC_ADMIN for modifying MAC policies. > > > > The eBPF programs are attached to a separate security_hook_heads > > maintained by the BPF LSM for mutable hooks and executed after all the > > statically defined hooks (i.e. the ones declared by SELinux, AppArmor, > > Smack etc). This also ensures that statically defined LSM hooks retain > > the behaviour of "being read-only after init", i.e. __lsm_ro_after_init. > > > > Upon attachment, a security hook is dynamically allocated with > > arch_bpf_prepare_trampoline which generates code to handle the > > conversion from the signature of the hook to the BPF context and allows > > the JIT'ed BPF program to be called as a C function with the same > > arguments as the LSM hooks. If any of the attached eBPF programs returns > > an error (like ENOPERM), the behaviour represented by the hook is > > denied. > > > > Audit logs can be written using a format chosen by the eBPF program to > > the perf events buffer or to global eBPF variables or maps and can be > > further processed in user-space. > > > > # BTF Based Design > > > > The current design uses BTF > > (https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html, > > https://lwn.net/Articles/803258/) which allows verifiable read-only > > structure accesses by field names rather than fixed offsets. This allows > > accessing the hook parameters using a dynamically created context which > > provides a certain degree of ABI stability: > > > > > > // Only declare the structure and fields intended to be used > > // in the program > > struct vm_area_struct { > > unsigned long vm_start; > > } __attribute__((preserve_access_index)); > > > > It would be nice to also mention that you don't even have to > "re-define" these structs if you use vmlinux.h generated with `bpftool > btf dump file <path-to-vm-linux-or-/sys/kernel/btf/vmlinux> format c`. > Its output will contain all types of the kernel, including internal > ones not exposed through any public headers. And it will also > automatically have __attribute__((preserve_access_index)) applied to > all structs/unions. It can be pre-generated and checked in somewhere > along the application or generated on the fly, if environment and use > case allows. Cool, I will update the documentation to mention this. Thanks! - KP > > > // Declare the eBPF program mprotect_audit which attaches to > > // to the file_mprotect LSM hook and accepts three arguments. > > SEC("lsm/file_mprotect") > > int BPF_PROG(mprotect_audit, struct vm_area_struct *vma, > > unsigned long reqprot, unsigned long prot) > > { > > unsigned long vm_start = vma->vm_start; > > > > return 0; > > } > > > > By relocating field offsets, BTF makes a large portion of kernel data > > structures readily accessible across kernel versions without requiring a > > large corpus of BPF helper functions and requiring recompilation with > > every kernel version. The BTF type information is also used by the BPF > > verifier to validate memory accesses within the BPF program and also > > prevents arbitrary writes to the kernel memory. > > > > The limitations of BTF compatibility are described in BPF Co-Re > > (http://vger.kernel.org/bpfconf2019_talks/bpf-core.pdf, i.e. field > > renames, #defines and changes to the signature of LSM hooks). > > > > This design imposes that the MAC policy (eBPF programs) be updated when > > the inspected kernel structures change outside of BTF compatibility > > guarantees. In practice, this is only required when a structure field > > used by a current policy is removed (or renamed) or when the used LSM > > hooks change. We expect the maintenance cost of these changes to be > > acceptable as compared to the previous design > > (https://lore.kernel.org/bpf/20190910115527.5235-1-kpsingh@chromium.org/). > > > > [...]
From: KP Singh <kpsingh@google.com> # Changes since v1 (https://lore.kernel.org/bpf/20191220154208.15895-1-kpsingh@chromium.org/): * Eliminate the requirement to maintain LSM hooks separately in security/bpf/hooks.h Use BPF trampolines to dynamically allocate security hooks * Drop the use of securityfs as bpftool provides the required introspection capabilities. Update the tests to use the bpf_skeleton and global variables * Use O_CLOEXEC anonymous fds to represent BPF attachment in line with the other BPF programs with the possibility to use bpf program pinning in the future to provide "permanent attachment". * Drop the logic based on prog names for handling re-attachment. * Drop bpf_lsm_event_output from this series and send it as a separate patch. # Motivation Google does analysis of rich runtime security data to detect and thwart threats in real-time. Currently, this is done in custom kernel modules but we would like to replace this with something that's upstream and useful to others. The current kernel infrastructure for providing telemetry (Audit, Perf etc.) is disjoint from access enforcement (i.e. LSMs). Augmenting the information provided by audit requires kernel changes to audit, its policy language and user-space components. Furthermore, building a MAC policy based on the newly added telemetry data requires changes to various LSMs and their respective policy languages. This patchset proposes a new stackable and privileged LSM which allows the LSM hooks to be implemented using eBPF. This facilitates a unified and dynamic (not requiring re-compilation of the kernel) audit and MAC policy. # Why an LSM? Linux Security Modules target security behaviours rather than the kernel's API. For example, it's easy to miss out a newly added system call for executing processes (eg. execve, execveat etc.) but the LSM framework ensures that all process executions trigger the relevant hooks irrespective of how the process was executed. Allowing users to implement LSM hooks at runtime also benefits the LSM eco-system by enabling a quick feedback loop from the security community about the kind of behaviours that the LSM Framework should be targeting. # How does it work? The LSM introduces a new eBPF (https://docs.cilium.io/en/v1.6/bpf/) program type BPF_PROG_TYPE_LSM which can only be attached to LSM hooks. Attachment requires CAP_SYS_ADMIN for loading eBPF programs and CAP_MAC_ADMIN for modifying MAC policies. The eBPF programs are attached to a separate security_hook_heads maintained by the BPF LSM for mutable hooks and executed after all the statically defined hooks (i.e. the ones declared by SELinux, AppArmor, Smack etc). This also ensures that statically defined LSM hooks retain the behaviour of "being read-only after init", i.e. __lsm_ro_after_init. Upon attachment, a security hook is dynamically allocated with arch_bpf_prepare_trampoline which generates code to handle the conversion from the signature of the hook to the BPF context and allows the JIT'ed BPF program to be called as a C function with the same arguments as the LSM hooks. If any of the attached eBPF programs returns an error (like ENOPERM), the behaviour represented by the hook is denied. Audit logs can be written using a format chosen by the eBPF program to the perf events buffer or to global eBPF variables or maps and can be further processed in user-space. # BTF Based Design The current design uses BTF (https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html, https://lwn.net/Articles/803258/) which allows verifiable read-only structure accesses by field names rather than fixed offsets. This allows accessing the hook parameters using a dynamically created context which provides a certain degree of ABI stability: // Only declare the structure and fields intended to be used // in the program struct vm_area_struct { unsigned long vm_start; } __attribute__((preserve_access_index)); // Declare the eBPF program mprotect_audit which attaches to // to the file_mprotect LSM hook and accepts three arguments. SEC("lsm/file_mprotect") int BPF_PROG(mprotect_audit, struct vm_area_struct *vma, unsigned long reqprot, unsigned long prot) { unsigned long vm_start = vma->vm_start; return 0; } By relocating field offsets, BTF makes a large portion of kernel data structures readily accessible across kernel versions without requiring a large corpus of BPF helper functions and requiring recompilation with every kernel version. The BTF type information is also used by the BPF verifier to validate memory accesses within the BPF program and also prevents arbitrary writes to the kernel memory. The limitations of BTF compatibility are described in BPF Co-Re (http://vger.kernel.org/bpfconf2019_talks/bpf-core.pdf, i.e. field renames, #defines and changes to the signature of LSM hooks). This design imposes that the MAC policy (eBPF programs) be updated when the inspected kernel structures change outside of BTF compatibility guarantees. In practice, this is only required when a structure field used by a current policy is removed (or renamed) or when the used LSM hooks change. We expect the maintenance cost of these changes to be acceptable as compared to the previous design (https://lore.kernel.org/bpf/20190910115527.5235-1-kpsingh@chromium.org/). # Why not tracepoints or kprobes? In order to do MAC with tracepoints or kprobes, we would need to override the return value of the security hook. This is not possible with tracepoints or call-site kprobes. Attaching to the return boundary (kretprobe) implies that BPF programs would always get called after all the other LSM hooks are called and clobber the pre-existing LSM semantics. Enforcing MAC policy with an actual LSM helps leverage the verified semantics of the framework. # Usage Examples A simple example and some documentation is included in the patchset. In order to better illustrate the capabilities of the framework some more advanced prototype (not-ready for review) code has also been published separately: * Logging execution events (including environment variables and arguments) https://github.com/sinkap/linux-krsi/blob/patch/v1/examples/samples/bpf/lsm_audit_env.c * Detecting deletion of running executables: https://github.com/sinkap/linux-krsi/blob/patch/v1/examples/samples/bpf/lsm_detect_exec_unlink.c * Detection of writes to /proc/<pid>/mem: https://github.com/sinkap/linux-krsi/blob/patch/v1/examples/samples/bpf/lsm_audit_env.c We have updated Google's internal telemetry infrastructure and have started deploying this LSM on our Linux Workstations. This gives us more confidence in the real-world applications of such a system. KP Singh (10): bpf: btf: Make some of the API visible outside BTF bpf: lsm: Add a skeleton and config options bpf: lsm: Introduce types for eBPF based LSM bpf: lsm: Add mutable hooks list for the BPF LSM bpf: lsm: BTF API for LSM hooks bpf: lsm: Implement attach, detach and execution bpf: lsm: Make the allocated callback RO+X tools/libbpf: Add support for BPF_PROG_TYPE_LSM bpf: lsm: Add selftests for BPF_PROG_TYPE_LSM bpf: lsm: Add Documentation Documentation/security/bpf.rst | 150 ++++++++ Documentation/security/index.rst | 1 + MAINTAINERS | 11 + include/linux/bpf.h | 4 + include/linux/bpf_lsm.h | 98 +++++ include/linux/bpf_types.h | 4 + include/linux/btf.h | 8 + include/uapi/linux/bpf.h | 6 + kernel/bpf/btf.c | 17 + kernel/bpf/syscall.c | 51 ++- kernel/bpf/verifier.c | 74 +++- security/Kconfig | 11 +- security/Makefile | 2 + security/bpf/Kconfig | 25 ++ security/bpf/Makefile | 7 + security/bpf/hooks.c | 337 ++++++++++++++++++ security/bpf/include/bpf_lsm.h | 75 ++++ security/bpf/lsm.c | 92 +++++ security/bpf/ops.c | 30 ++ security/security.c | 24 +- tools/include/uapi/linux/bpf.h | 6 + tools/lib/bpf/bpf.c | 6 +- tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/libbpf.c | 143 +++++++- tools/lib/bpf/libbpf.h | 4 + tools/lib/bpf/libbpf.map | 3 + tools/lib/bpf/libbpf_probes.c | 1 + tools/testing/selftests/bpf/lsm_helpers.h | 19 + .../bpf/prog_tests/lsm_mprotect_audit.c | 58 +++ .../selftests/bpf/progs/lsm_mprotect_audit.c | 48 +++ 30 files changed, 1271 insertions(+), 45 deletions(-) create mode 100644 Documentation/security/bpf.rst create mode 100644 include/linux/bpf_lsm.h create mode 100644 security/bpf/Kconfig create mode 100644 security/bpf/Makefile create mode 100644 security/bpf/hooks.c create mode 100644 security/bpf/include/bpf_lsm.h create mode 100644 security/bpf/lsm.c create mode 100644 security/bpf/ops.c create mode 100644 tools/testing/selftests/bpf/lsm_helpers.h create mode 100644 tools/testing/selftests/bpf/prog_tests/lsm_mprotect_audit.c create mode 100644 tools/testing/selftests/bpf/progs/lsm_mprotect_audit.c