From patchwork Sat Mar 12 20:14:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12778927 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BE1DC433EF for ; Sat, 12 Mar 2022 20:14:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E1358D0002; Sat, 12 Mar 2022 15:14:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 38FD08D0001; Sat, 12 Mar 2022 15:14:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 258E38D0002; Sat, 12 Mar 2022 15:14:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 162468D0001 for ; Sat, 12 Mar 2022 15:14:13 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D59B92381B for ; Sat, 12 Mar 2022 20:14:12 +0000 (UTC) X-FDA: 79236835944.11.96B0E26 Received: from out1.migadu.com (out1.migadu.com [91.121.223.63]) by imf25.hostedemail.com (Postfix) with ESMTP id 366B8A001E for ; Sat, 12 Mar 2022 20:14:12 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1647116050; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=LmoxS08scyGyVZqa/3VyRO6NmjBbbNh8W/Hs8Ybg/AU=; b=KWCmw9X7waDLg1FZltp14KfIGyto7rGibooprLEDSWhgrgIdyyrss/jrmCWPML7qrv4h3m s0gyjJwZ6jZkTYilfgDgMqU7xbGhTb9UT4MrHm5Fnh6URXKOyMWzhaExVa9NgXSSccY2c9 CGZrl8lNiUsRHWHw2rTcoSdmyThnBrI= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Andrew Morton Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Vincenzo Frascino , Catalin Marinas , Will Deacon , Mark Rutland , Sami Tolvanen , Peter Collingbourne , Evgenii Stepanov , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH] kasan, scs: collect stack traces from shadow stack Date: Sat, 12 Mar 2022 21:14:06 +0100 Message-Id: <57133fafc4d74377a4a08d98e276d58fe4a127dc.1647115974.git.andreyknvl@google.com> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 366B8A001E X-Stat-Signature: 1abiou1kkpxz7xg1mg3akkbjfufsbr8t Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=KWCmw9X7; spf=pass (imf25.hostedemail.com: domain of andrey.konovalov@linux.dev designates 91.121.223.63 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-HE-Tag: 1647116052-664150 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Currently, KASAN always uses the normal stack trace collection routines, which rely on the unwinder, when saving alloc and free stack traces. Instead of invoking the unwinder, collect the stack trace by copying frames from the Shadow Call Stack whenever it is enabled. This reduces boot time by 30% for all KASAN modes when Shadow Call Stack is enabled. To avoid potentially leaking PAC pointer tags, strip them when saving the stack trace. Signed-off-by: Andrey Konovalov --- Things to consider: We could integrate shadow stack trace collection into kernel/stacktrace.c as e.g. stack_trace_save_shadow(). However, using stack_trace_consume_fn leads to invoking a callback on each saved from, which is undesirable. The plain copy loop is faster. We could add a command line flag to switch between stack trace collection modes. I noticed that Shadow Call Stack might be missing certain frames in stacks originating from a fault that happens in the middle of a function. I am not sure if this case is important to handle though. Looking forward to thoughts and comments. Thanks! --- mm/kasan/common.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index d9079ec11f31..65a0723370c7 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -9,6 +9,7 @@ * Andrey Konovalov */ +#include #include #include #include @@ -21,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -30,12 +32,44 @@ #include "kasan.h" #include "../slab.h" +#ifdef CONFIG_SHADOW_CALL_STACK + +#ifdef CONFIG_ARM64_PTR_AUTH +#define PAC_TAG_RESET(x) (x | GENMASK(63, CONFIG_ARM64_VA_BITS)) +#else +#define PAC_TAG_RESET(x) (x) +#endif + +static unsigned int save_shadow_stack(unsigned long *entries, + unsigned int nr_entries) +{ + unsigned long *scs_sp = task_scs_sp(current); + unsigned long *scs_base = task_scs(current); + unsigned long *frame; + unsigned int i = 0; + + for (frame = scs_sp - 1; frame >= scs_base; frame--) { + entries[i++] = PAC_TAG_RESET(*frame); + if (i >= nr_entries) + break; + } + + return i; +} +#else /* CONFIG_SHADOW_CALL_STACK */ +static inline unsigned int save_shadow_stack(unsigned long *entries, + unsigned int nr_entries) { return 0; } +#endif /* CONFIG_SHADOW_CALL_STACK */ + depot_stack_handle_t kasan_save_stack(gfp_t flags, bool can_alloc) { unsigned long entries[KASAN_STACK_DEPTH]; unsigned int nr_entries; - nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 0); + if (IS_ENABLED(CONFIG_SHADOW_CALL_STACK)) + nr_entries = save_shadow_stack(entries, ARRAY_SIZE(entries)); + else + nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 0); return __stack_depot_save(entries, nr_entries, flags, can_alloc); }