| Message ID | 20231102005521.346983-1-kpsingh@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | Reduce overhead of LSMs with static calls | expand |
On Wed, Nov 1, 2023 at 5:55 PM KP Singh <kpsingh@kernel.org> wrote: > > # Background > > LSM hooks (callbacks) are currently invoked as indirect function calls. These > callbacks are registered into a linked list at boot time as the order of the > LSMs can be configured on the kernel command line with the "lsm=" command line > parameter. > > Indirect function calls have a high overhead due to retpoline mitigation for > various speculative execution attacks. > > Retpolines remain relevant even with newer generation CPUs as recently > discovered speculative attacks, like Spectre BHB need Retpolines to mitigate > against branch history injection and still need to be used in combination with > newer mitigation features like eIBRS. > > This overhead is especially significant for the "bpf" LSM which allows the user > to implement LSM functionality with eBPF program. In order to facilitate this > the "bpf" LSM provides a default callback for all LSM hooks. When enabled, > the "bpf" LSM incurs an unnecessary / avoidable indirect call. This is > especially bad in OS hot paths (e.g. in the networking stack). > This overhead prevents the adoption of bpf LSM on performance critical > systems, and also, in general, slows down all LSMs. > > Since we know the address of the enabled LSM callbacks at compile time and only > the order is determined at boot time, the LSM framework can allocate static > calls for each of the possible LSM callbacks and these calls can be updated once > the order is determined at boot. > > This series is a respin of the RFC proposed by Paul Renauld (renauld@google.com) > and Brendan Jackman (jackmanb@google.com) [1] > > # Performance improvement > > With this patch-set some syscalls with lots of LSM hooks in their path > benefitted at an average of ~3% and I/O and Pipe based system calls benefitting > the most. > > Here are the results of the relevant Unixbench system benchmarks with BPF LSM > and SELinux enabled with default policies enabled with and without these > patches. > > Benchmark Delta(%): (+ is better) > =============================================================================== > Execl Throughput +1.9356 > File Write 1024 bufsize 2000 maxblocks +6.5953 > Pipe Throughput +9.5499 > Pipe-based Context Switching +3.0209 > Process Creation +2.3246 > Shell Scripts (1 concurrent) +1.4975 > System Call Overhead +2.7815 > System Benchmarks Index Score (Partial Only): +3.4859 > > In the best case, some syscalls like eventfd_create benefitted to about ~10%. > The full analysis can be viewed at https://kpsingh.ch/lsm-perf > > [1] https://lore.kernel.org/linux-security-module/20200820164753.3256899-1-jackmanb@chromium.org/ > > > # BPF LSM Side effects > > Patch 4 of the series also addresses the issues with the side effects of the > default value return values of the BPF LSM callbacks and also removes the > overheads associated with them making it deployable at hyperscale. > > # v6 -> v7 > > * Rebased with latest LSM id changes merged > > NOTE: The warning shown by the kernel test bot is spurious, there is no flex array > and it seems to come from an older tool chain. > > https://lore.kernel.org/bpf/202310111711.wLbijitj-lkp@intel.com/ > > # v5 -> v6 > > * Fix a bug in BPF LSM hook toggle logic. > > # v4 -> v5 > > * Rebase to linux-next/master > * Fixed the case where MAX_LSM_COUNT comes to zero when just CONFIG_SECURITY > is compiled in without any other LSM enabled as reported here: > > https://lore.kernel.org/bpf/202309271206.d7fb60f9-oliver.sang@intel.com > > # v3 -> v4 > > * Refactor LSM count macros to use COUNT_ARGS > * Change CONFIG_SECURITY_HOOK_LIKELY likely's default value to be based on > the LSM enabled and have it depend on CONFIG_EXPERT. There are a lot of subtle > options behind CONFIG_EXPERT and this should, hopefully alleviate concerns > about yet another knob. > * __randomize_layout for struct lsm_static_call and, in addition to the cover > letter add performance numbers to 3rd patch and some minor commit message > updates. > * Rebase to linux-next. > > # v2 -> v3 > > * Fixed a build issue on archs which don't have static calls and enable > CONFIG_SECURITY. > * Updated the LSM_COUNT macros based on Andrii's suggestions. > * Changed the security_ prefix to lsm_prefix based on Casey's suggestion. > * Inlined static_branch_maybe into lsm_for_each_hook on Kees' feedback. > > # v1 -> v2 (based on linux-next, next-20230614) > > * Incorporated suggestions from Kees > * Changed the way MAX_LSMs are counted from a binary based generator to a clever header. > * Add CONFIG_SECURITY_HOOK_LIKELY to configure the likelihood of LSM hooks. > > > KP Singh (5): > kernel: Add helper macros for loop unrolling > security: Count the LSMs enabled at compile time > security: Replace indirect LSM hook calls with static calls > bpf: Only enable BPF LSM hooks when an LSM program is attached > security: Add CONFIG_SECURITY_HOOK_LIKELY > > include/linux/bpf_lsm.h | 5 + > include/linux/lsm_count.h | 114 +++++++++++++++++++ > include/linux/lsm_hooks.h | 81 ++++++++++++-- > include/linux/unroll.h | 36 ++++++ > kernel/bpf/trampoline.c | 24 ++++ > security/Kconfig | 11 ++ > security/bpf/hooks.c | 25 ++++- > security/security.c | 226 +++++++++++++++++++++++++------------- > 8 files changed, 434 insertions(+), 88 deletions(-) > create mode 100644 include/linux/lsm_count.h > create mode 100644 include/linux/unroll.h > > -- > 2.42.0.820.g83a721a137-goog > > For the series: Acked-by: Andrii Nakryiko <andrii@kernel.org>
I didn't get your response on https://lkml.kernel.org/r/c588ca5d-c343-4ea2-a1f1-4efe67ebb8e3@I-love.SAKURA.ne.jp . Do you agree that we cannot replace LKM-based LSMs with eBPF-based access control mechanisms, and do you admit that this series makes LKM-based LSMs more difficult to use?
On Thu, Nov 2, 2023 at 10:42 AM Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> wrote: > > I didn't get your response on https://lkml.kernel.org/r/c588ca5d-c343-4ea2-a1f1-4efe67ebb8e3@I-love.SAKURA.ne.jp . > > Do you agree that we cannot replace LKM-based LSMs with eBPF-based access control mechanisms, > and do you admit that this series makes LKM-based LSMs more difficult to use? If you want to do a proper in-tree version of dynamic LSMs. There can be an exported symbol that allocates a dynamic slot and registers LSM hooks to it. This is very doable, but it's not my use case so, I am not going to do it. No it does not make LKM based LSMs difficult to use. I am not ready to have that debate again. I suggested multiple extensions in my replies which you chose to ignore. Regarding BPF it's very much possible, as I suggested many times, you need to rethink about it in terms of implementing policy and not try to dump the whole code and interface into BPF and expect it to work. It will need some design work and that's on you. We can help you, we can also take patches for anything BPF would need to make stuff work (I don't see anything obvious needed yet). But we surely won't write the code for you. >
On 2023/11/02 19:01, KP Singh wrote: > On Thu, Nov 2, 2023 at 10:42 AM Tetsuo Handa > <penguin-kernel@i-love.sakura.ne.jp> wrote: >> >> I didn't get your response on https://lkml.kernel.org/r/c588ca5d-c343-4ea2-a1f1-4efe67ebb8e3@I-love.SAKURA.ne.jp . >> >> Do you agree that we cannot replace LKM-based LSMs with eBPF-based access control mechanisms, >> and do you admit that this series makes LKM-based LSMs more difficult to use? > > If you want to do a proper in-tree version of dynamic LSMs. There can > be an exported symbol that allocates a dynamic slot and registers LSM > hooks to it. This is very doable, but it's not my use case so, I am > not going to do it. > > No it does not make LKM based LSMs difficult to use. I am not ready to > have that debate again. I suggested multiple extensions in my replies > which you chose to ignore. You said I think this needs to be discussed if and when we allow LKM based LSMs." as a response to It is Casey's commitment that the LSM infrastructure will not forbid LKM-based LSMs. We will start allowing LKM-based LSMs. But it is not clear how we can make it possible to allow LKM-based LSMs. , and you suggested combination of static calls (for built-in LSMs) and linked list (for LKM-based LSMs). So, what is your answer to Until I hear the real limitations of using BPF, it's a NAK from me. ? Do you agree to allow dynamically appendable LSM modules?
On Thu, Nov 2, 2023 at 11:30 AM Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> wrote: > > On 2023/11/02 19:01, KP Singh wrote: > > On Thu, Nov 2, 2023 at 10:42 AM Tetsuo Handa > > <penguin-kernel@i-love.sakura.ne.jp> wrote: > >> > >> I didn't get your response on https://lkml.kernel.org/r/c588ca5d-c343-4ea2-a1f1-4efe67ebb8e3@I-love.SAKURA.ne.jp . > >> > >> Do you agree that we cannot replace LKM-based LSMs with eBPF-based access control mechanisms, > >> and do you admit that this series makes LKM-based LSMs more difficult to use? > > > > If you want to do a proper in-tree version of dynamic LSMs. There can > > be an exported symbol that allocates a dynamic slot and registers LSM > > hooks to it. This is very doable, but it's not my use case so, I am > > not going to do it. > > > > No it does not make LKM based LSMs difficult to use. I am not ready to > > have that debate again. I suggested multiple extensions in my replies > > which you chose to ignore. > > You said > > I think this needs to be discussed if and when we allow LKM based LSMs." > > as a response to > > It is Casey's commitment that the LSM infrastructure will not forbid LKM-based LSMs. > We will start allowing LKM-based LSMs. But it is not clear how we can make it possible to > allow LKM-based LSMs. > > , and you suggested combination of static calls (for built-in LSMs) and > linked list (for LKM-based LSMs). Tetsuo, this is exactly a technical solution and it works, a very easy API can be made to enable the LKM based use-case (if the maintainers agree that they want to enable LKM based LSMs) I think what you can do is send patches for an API for LKM based LSMs and have it merged before my series, I will work with the code I have and make LKM based LSMs work. If this work gets merged, and your use-case is accepted (I think I can speak for at least Kees [if not others] too here) we will help you if you get stuck with MAX_LSM_COUNT or a dual static call and linked list based approach. > > So, what is your answer to > > Until I hear the real limitations of using BPF, it's a NAK from me. The burden of proof is on you. All I can say is that if you wanted to implement the policy logic you want, you could. If you cannot, please share specifics and the BPF / LSM community will help. > > ? Do you agree to allow dynamically appendable LSM modules? Nice try, I am for dynamically loadable hook logic, that's why I implemented the BPF LSM. What you want to ask me is am I for LKM LSMs. Regarding the LKM based LSMs, it's my opinion that it should be done with BPF LSM because it's actually safer and does not have the maintenance overhead that a kernel module has. My 0.02c, please share code, specifics and if you like your use case, get it merged with a patch series. From here onwards, I won't be responding to hypotheticals. >
On Thu, Nov 02, 2023 at 01:55:16AM +0100, KP Singh wrote: > NOTE: The warning shown by the kernel test bot is spurious, there is no flex array > and it seems to come from an older tool chain. > > https://lore.kernel.org/bpf/202310111711.wLbijitj-lkp@intel.com/ I was finally able to reproduce this and tracked it down. Fix is here: https://lore.kernel.org/lkml/20231104204334.work.160-kees@kernel.org/ -Kees