From patchwork Wed Sep 13 17:14:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 13383613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C367EE01F0 for ; Wed, 13 Sep 2023 17:14:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF25C6B0251; Wed, 13 Sep 2023 13:14:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B54176B0252; Wed, 13 Sep 2023 13:14:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CDD56B0254; Wed, 13 Sep 2023 13:14:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8C9466B0251 for ; Wed, 13 Sep 2023 13:14:52 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5CFB51C9FEB for ; Wed, 13 Sep 2023 17:14:52 +0000 (UTC) X-FDA: 81232224024.15.6C218E6 Received: from out-211.mta0.migadu.com (out-211.mta0.migadu.com [91.218.175.211]) by imf23.hostedemail.com (Postfix) with ESMTP id 8F14814001C for ; Wed, 13 Sep 2023 17:14:50 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ODwSiHci; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf23.hostedemail.com: domain of andrey.konovalov@linux.dev designates 91.218.175.211 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694625290; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=Gc6HzykR+A0L83xKMiWxb5E3eMlheJobgexoepcnfG0=; b=4S4ZkArrf2dbnaFePUjWUu5hRo75g0lMDgb9BIlLWT9mk/FhKNZlnt0Avd5AYtLR8xFsaP dwKLFMIwu6yl0v6PPBo0Jaogf0kSa1XB+9XfkE3S4lI3QOkHWEti31eU18LR8jqCm6ECZU PpOe1FYG9GXg/9r2c6VU0Nd5FMNxyjA= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ODwSiHci; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf23.hostedemail.com: domain of andrey.konovalov@linux.dev designates 91.218.175.211 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694625290; a=rsa-sha256; cv=none; b=T1K40V3Va/YrkawRgje0FsgEErLJHfR4pJr+1lDjTCkajU80MUFcZr8K0QFd2hRWU8hzSv nQ7kb6L+Bkfhjc4luWuDsalgAG+ZbZDYYbrBEpJphRNhLWC7JoO2ELVPB66vYW8mTrcF5g Lny71GjDZwvUcm9VfeTthqx+zutIYks= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1694625288; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=Gc6HzykR+A0L83xKMiWxb5E3eMlheJobgexoepcnfG0=; b=ODwSiHcis3mdddO8WgNkr0OX4CFlXGCtxXFw1qh2OkYCRDvsJp2kKgsnNbJ8Ge1HWYw6Gg qFyB7nW3syZF3lt4fEZE89uDLMmQd0mlYzmeI+kPXZAt1II6Wy6iHHDK6XeYOtxVHOkOq+ EPv9hZtm5/AmnbNkbRVPkdVPGebLbzw= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 00/19] stackdepot: allow evicting stack traces Date: Wed, 13 Sep 2023 19:14:25 +0200 Message-Id: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 8F14814001C X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: aoox9ptmny5rjc6usp8uarj3yyymgnkh X-HE-Tag: 1694625290-386683 X-HE-Meta: U2FsdGVkX1+3/cIJoaXVcdCy+py3dWgYmsPq/L0JEw7LeQLAImePRaVn1VOE81vrsO5ZVlc61R3H04NVgyGbeEHoTG/BbAz56q+HXfFI3m2PYSliDnYkh743L3qqwn9mjZD4Jl2TpgS1yypFP2kxx5e0a5+o3o80M8fe2kcmL5e3SB/SQZ7c77OINvN8DzONQESo+HDTd03hXpuSGm4DcGqY/jnMytvMhLTZnXcQ5x4+nbeLvUh9vkFHcsOqj3vSArtKFjYvELXq07CPfqqG1v49CuC27odFZvUlTBXywzGBcvyxDNYiwFo1idPJ5gjGa0o842qXscRX+TuzXkvPofVjZKopseK2vAzR/JKEV5dQdTM2r2dsuCILa+cmSX4fNXFxmORod4W+s8xK1i5YPVCfZRMI5qT1y53gHHrslwNIh9pXn/okILUi00eZ/oB5Gk9Etv6Tq1zh4EJoZGLhNS8psBpMNesZHg8CEjQmb0c/2YgvYbqEZQD7/0gm4AznuCUYXeCdjYI/dj+Yoa9y2Lj0RzEP9HCG97MpCnD5wCTCip3qOnCXjk7hDBRAJhfbCHtHNNUEdVvf14fqerkl0a+B6aLub86Gu5AC7AtsVY2MlhVI1Yqhh/yQeEglQFuUX+Kfzd6ILJGrOThSH2dukHx6K9We+BxnDs5FksUK97N0bWJo6nZdEIT63vbOnOl808ohFUMjpaqN0L3RmzUxXPnFGaudN5GhF57gwI6iaOEdvaADYXTcFfctY6dEsvLH1B82qzX2S+9grHo3Pc2i3JtGaPXI1YOVUuLiIcs7CgP0Rku0KtQr8hAV2GMSNh3E8EH7v5sHkYs4QM92u4cggef9utsT+lpkZtHxLFKPWJvBYjX4SrK8fBVU587/a28EiQlFdwYS2ar9eue3eV9tXKCr4PsIYJSW9pW+UREvDR9HPaWfilbfYPyWJ4e79lw+60AUaJq3oMkKF3Dwd2s zWPi/XHT gXW5DcO7pFdwcpib00aOkgr56zcYeTp1FalbL4K0f012FRdQ8F8j6npIrIJBn4J/rzhXiinVgoS/Pv6A9lmE7Rhl7Ox3RqCiYBRu5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Currently, the stack depot grows indefinitely until it reaches its capacity. Once that happens, the stack depot stops saving new stack traces. This creates a problem for using the stack depot for in-field testing and in production. For such uses, an ideal stack trace storage should: 1. Allow saving fresh stack traces on systems with a large uptime while limiting the amount of memory used to store the traces; 2. Have a low performance impact. Implementing #1 in the stack depot is impossible with the current keep-forever approach. This series targets to address that. Issue #2 is left to be addressed in a future series. This series changes the stack depot implementation to allow evicting unneeded stack traces from the stack depot. The users of the stack depot can do that via new stack_depot_save_flags(STACK_DEPOT_FLAG_GET) and stack_depot_put APIs. Internal changes to the stack depot code include: 1. Storing stack traces in fixed-frame-sized slots; the slot size is controlled via CONFIG_STACKDEPOT_MAX_FRAMES (vs precisely-sized slots in the current implementation); 2. Keeping available slots in a freelist (vs keeping an offset to the next free slot); 3. Using a read/write lock for synchronization (vs a lock-free approach combined with a spinlock). This series also integrates the eviction functionality in the tag-based KASAN modes. Despite wasting some space on rounding up the size of each stack record, with CONFIG_STACKDEPOT_MAX_FRAMES=32, the tag-based KASAN modes end up consuming ~5% less memory in stack depot during boot (with the default stack ring size of 32k entries). The reason for this is the eviction of irrelevant stack traces from the stack depot, which frees up space for other stack traces. For other tools that heavily rely on the stack depot, like Generic KASAN and KMSAN, this change leads to the stack depot capacity being reached sooner than before. However, as these tools are mainly used in fuzzing scenarios where the kernel is frequently rebooted, this outcome should be acceptable. There is no measurable boot time performance impact of these changes for KASAN on x86-64. I haven't done any tests for arm64 modes (the stack depot without performance optimizations is not suitable for intended use of those anyway), but I expect a similar result. Obtaining and copying stack trace frames when saving them into stack depot is what takes the most time. This series does not yet provide a way to configure the maximum size of the stack depot externally (e.g. via a command-line parameter). This will be added in a separate series, possibly together with the performance improvement changes. --- Changes v1->v2: - Rework API to stack_depot_save_flags(STACK_DEPOT_FLAG_GET) + stack_depot_put. - Add CONFIG_STACKDEPOT_MAX_FRAMES Kconfig option. - Switch stack depot to using list_head's. - Assorted minor changes, see the commit message for each path. Andrey Konovalov (19): lib/stackdepot: check disabled flag when fetching lib/stackdepot: simplify __stack_depot_save lib/stackdepot: drop valid bit from handles lib/stackdepot: add depot_fetch_stack helper lib/stackdepot: use fixed-sized slots for stack records lib/stackdepot: fix and clean-up atomic annotations lib/stackdepot: rework helpers for depot_alloc_stack lib/stackdepot: rename next_pool_required to new_pool_required lib/stackdepot: store next pool pointer in new_pool lib/stackdepot: store free stack records in a freelist lib/stackdepot: use read/write lock lib/stackdepot: use list_head for stack record links kmsan: use stack_depot_save instead of __stack_depot_save lib/stackdepot, kasan: add flags to __stack_depot_save and rename lib/stackdepot: add refcount for records lib/stackdepot: allow users to evict stack traces kasan: remove atomic accesses to stack ring entries kasan: check object_size in kasan_complete_mode_report_info kasan: use stack_depot_put for tag-based modes include/linux/stackdepot.h | 59 ++++-- lib/Kconfig | 10 +- lib/stackdepot.c | 410 ++++++++++++++++++++++++------------- mm/kasan/common.c | 7 +- mm/kasan/generic.c | 9 +- mm/kasan/kasan.h | 2 +- mm/kasan/report_tags.c | 27 +-- mm/kasan/tags.c | 24 ++- mm/kmsan/core.c | 7 +- 9 files changed, 356 insertions(+), 199 deletions(-)