From patchwork Fri Nov 13 22:20:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11905129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8052C55ABD for ; Fri, 13 Nov 2020 23:52:12 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6C0F220791 for ; Fri, 13 Nov 2020 23:52:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="uYZTJ43b"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="gd/K1uCb"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="O0Kwm8Q+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C0F220791 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=RmbTxUdAvMPMfX8qhUyYydlIN0qRzQiTs3LzR444kkM=; b=uYZTJ43bdYgoQxvtMBGpvo3Ng oZkf2Ba8QVuP/z5B3l6G0rsuQYqeyLLJ461yDrGxncKTywp5BrKiOVHa2OwnJ4751vGaVn9KZBFgb kb2OImO7sQ2czDcn0Fb2LyCwx2SgpleaygkeoCtZADYJ7BUt8QKqGi+JQp/T/36TTdioaHi03IGdz 9mkl7HuyaKKzgizLVL/b3OhicfZu1jzOrrFsE2z4zfwGuEhx6cGuKJ1CA3Sa83dP74Y2w1zFbPlUd R04uLYojgmpaGWHbVLu7Mztlt3nTh0L8lMCTfFVSdI9gp/UCgiP/LPm9a1dc5dS6eXgDaBa4EvtyK tmilDJANw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdiqh-00030i-RL; Fri, 13 Nov 2020 23:51:23 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdhdi-0007eH-CY for linux-arm-kernel@merlin.infradead.org; Fri, 13 Nov 2020 22:33:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=eyOrtcKen7fgrfwARq8tGqpWknv3CPxMtmzrBg32L54=; b=gd/K1uCb1Vbc0mu8bWqX/QWjAw FCP+K0FI22qNwevUyek12HJIZ17wkpMglWOXuiAKLlf/7UuEuogsdj+t2IfW5U/M+cnmV5ilcqTJd AzTDgTgSZXNOiDJZ/x7L69Da2nCh+GkCaa16FUSakC+e9GbM0Fx1jLTKG6gb/dMeHgdR7vvKl1lr7 lGcqXJrtgMp2ctvc1tp+KiBF50DVVCDCBtTv7T5JocZyVU2hT2b0bvCwpMKXmcGzaXFyTBCdJGuuD +4WMWEqvQK+k43lrEiIX8IbjlUBoNPAkC5OVbTD2ghr/z6vroZuN4aW6JCR8NUDDdb16aOSQTcsmL TDz5KXmA==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdhRH-0000CU-6X for linux-arm-kernel@lists.infradead.org; Fri, 13 Nov 2020 22:21:06 +0000 Received: by mail-wr1-x44a.google.com with SMTP id h11so4651025wrq.20 for ; Fri, 13 Nov 2020 14:21:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=eyOrtcKen7fgrfwARq8tGqpWknv3CPxMtmzrBg32L54=; b=O0Kwm8Q+s44JcGB/AtbKCNpR/VG8oK5sOrln2Ln/mpjlt/J5f19GFlvre+tKbwhytY eJk8paW3Ggm34w8sSwTinp4wMEdtG9WBBLm+el5gAyOVTq5rXVOu9Wmldbgi+6doOooA /YBSmmqDzJadJlZb12kBHqkNWf1ae+E+4T9HYhVUiPxlUvU2bc4UFXMEeQYl4Fv6Z5sI j82jVra71cMq42coFmwruTBcl2+KpUDo32GQcg/wDr5xZ6wE1W/uieh0/qiMZRZRvQLV sPYpMtxafq9Re7uK5aG85xPt8RNC3crN9GedGBUhAdVy7yA/X9WL82N7mqqEh1kv+YZ4 ABvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=eyOrtcKen7fgrfwARq8tGqpWknv3CPxMtmzrBg32L54=; b=Ee7HKWdKwmHquJdXj8P0iF25gOiXn5FAaT4PlzbKetjQ0iYR6SHkZPzh/+i/Auw01U rdmpyPdNwltu12i9JtXoxNovmtdmNc+StqoltZw9v2S5FXyZyB0pmIjYGUzqGsnXq6Gh OgvXDvI/qTvlOboba4u+QD+5nbr8/cLTVcafGwaWH4Zu9ZbZIqlS9sLT/ydH+fDYVKwb 1sD03dsBIDOcE8oms22ZGC1GPqtmaoU8MqbS3Dksqo2FFTPUGR9mNcXw3O2VhNkMgTgj oBAcY2Sz85oplWlsj1/YzfvFrc1XIS1fgzLlCnG0Zqr7t/Dz1L/2jo2xtlN2NVWa7hub XFrw== X-Gm-Message-State: AOAM531VSCVJ1m9pZEXKvOH4CXJbkWVcpI8GM6Wb6i9JmEJj4EydZ76T 4lJU59ak0WCkuzT0JhwhCwAHydBBvA1HVB4i X-Google-Smtp-Source: ABdhPJzcSLgrfEHEsnxNSacjzJpW5iTQm/d7cUrxxvowHRiIiVk33HEA3Z6rKplR2mcWiPF/pgEiyAiPhNzDdAei X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:dc0a:: with SMTP id t10mr4712946wmg.5.1605306060823; Fri, 13 Nov 2020 14:21:00 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:09 +0100 In-Reply-To: Message-Id: <1668fc1d020f9c1f2b8bf57467f0433bfd74d0a3.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 19/19] kasan: update documentation From: Andrey Konovalov To: Andrew Morton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201113_222103_500051_5CBC396C X-CRM114-Status: GOOD ( 34.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arm-kernel@lists.infradead.org, Marco Elver , Catalin Marinas , Kevin Brodsky , Will Deacon , Branislav Rankov , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexander Potapenko , Evgenii Stepanov , Andrey Konovalov , Andrey Ryabinin , Vincenzo Frascino , Dmitry Vyukov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This change updates KASAN documentation to reflect the addition of boot parameters and also reworks and clarifies some of the existing sections, in particular: defines what a memory granule is, mentions quarantine, makes Kunit section more readable. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ib1f83e91be273264b25f42b04448ac96b858849f Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov --- Documentation/dev-tools/kasan.rst | 186 +++++++++++++++++++----------- 1 file changed, 116 insertions(+), 70 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index ffbae8ce5748..0d5d77919b1a 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -4,8 +4,9 @@ The Kernel Address Sanitizer (KASAN) Overview -------- -KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to -find out-of-bound and use-after-free bugs. KASAN has three modes: +KernelAddressSANitizer (KASAN) is a dynamic memory safety error detector +designed to find out-of-bound and use-after-free bugs. KASAN has three modes: + 1. generic KASAN (similar to userspace ASan), 2. software tag-based KASAN (similar to userspace HWASan), 3. hardware tag-based KASAN (based on hardware memory tagging). @@ -39,23 +40,13 @@ CONFIG_KASAN_INLINE. Outline and inline are compiler instrumentation types. The former produces smaller binary while the latter is 1.1 - 2 times faster. Both software KASAN modes work with both SLUB and SLAB memory allocators, -hardware tag-based KASAN currently only support SLUB. -For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. +while the hardware tag-based KASAN currently only support SLUB. + +For better error reports that include stack traces, enable CONFIG_STACKTRACE. To augment reports with last allocation and freeing stack of the physical page, it is recommended to enable also CONFIG_PAGE_OWNER and boot with page_owner=on. -To disable instrumentation for specific files or directories, add a line -similar to the following to the respective kernel Makefile: - -- For a single file (e.g. main.o):: - - KASAN_SANITIZE_main.o := n - -- For all files in one directory:: - - KASAN_SANITIZE := n - Error reports ~~~~~~~~~~~~~ @@ -140,22 +131,75 @@ freed (in case of a use-after-free bug report). Next comes a description of the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. -Reading this part requires some understanding of how KASAN works. - -The state of each 8 aligned bytes of memory is encoded in one shadow byte. -Those 8 bytes can be accessible, partially accessible, freed or be a redzone. -We use the following encoding for each shadow byte: 0 means that all 8 bytes -of the corresponding memory region are accessible; number N (1 <= N <= 7) means -that the first N bytes are accessible, and other (8 - N) bytes are not; -any negative value indicates that the entire 8-byte word is inaccessible. -We use different negative values to distinguish between different kinds of -inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). +Internally KASAN tracks memory state separately for each memory granule, which +is either 8 or 16 aligned bytes depending on KASAN mode. Each number in the +memory state section of the report shows the state of one of the memory +granules that surround the accessed address. + +For generic KASAN the size of each memory granule is 8. The state of each +granule is encoded in one shadow byte. Those 8 bytes can be accessible, +partially accessible, freed or be a part of a redzone. KASAN uses the following +encoding for each shadow byte: 0 means that all 8 bytes of the corresponding +memory region are accessible; number N (1 <= N <= 7) means that the first N +bytes are accessible, and other (8 - N) bytes are not; any negative value +indicates that the entire 8-byte word is inaccessible. KASAN uses different +negative values to distinguish between different kinds of inaccessible memory +like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. For tag-based KASAN this last report section shows the memory tags around the -accessed address (see Implementation details section). +accessed address (see `Implementation details`_ section). + +Boot parameters +~~~~~~~~~~~~~~~ + +Hardware tag-based KASAN mode (see the section about different mode below) is +intended for use in production as a security mitigation. Therefore it supports +boot parameters that allow to disable KASAN competely or otherwise control +particular KASAN features. + +The things that can be controlled are: + +1. Whether KASAN is enabled at all. +2. Whether KASAN collects and saves alloc/free stacks. +3. Whether KASAN panics on a detected bug or not. + +The ``kasan.mode`` boot parameter allows to choose one of three main modes: + +- ``kasan.mode=off`` - KASAN is disabled, no tag checks are performed +- ``kasan.mode=prod`` - only essential production features are enabled +- ``kasan.mode=full`` - all KASAN features are enabled + +The chosen mode provides default control values for the features mentioned +above. However it's also possible to override the default values by providing: + +- ``kasan.stacktrace=off`` or ``=on`` - enable alloc/free stack collection + (default: ``on`` for ``mode=full``, + otherwise ``off``) +- ``kasan.fault=report`` or ``=panic`` - only print KASAN report or also panic + (default: ``report``) + +If ``kasan.mode`` parameter is not provided, it defaults to ``full`` when +``CONFIG_DEBUG_KERNEL`` is enabled, and to ``prod`` otherwise. + +For developers +~~~~~~~~~~~~~~ + +Software KASAN modes use compiler instrumentation to insert validity checks. +Such instrumentation might be incompatible with some part of the kernel, and +therefore needs to be disabled. To disable instrumentation for specific files +or directories, add a line similar to the following to the respective kernel +Makefile: + +- For a single file (e.g. main.o):: + + KASAN_SANITIZE_main.o := n + +- For all files in one directory:: + + KASAN_SANITIZE := n Implementation details @@ -164,10 +208,10 @@ Implementation details Generic KASAN ~~~~~~~~~~~~~ -From a high level, our approach to memory error detection is similar to that -of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to insert checks of shadow -memory on each memory access. +From a high level perspective, KASAN's approach to memory error detection is +similar to that of kmemcheck: use shadow memory to record whether each byte of +memory is safe to access, and use compile-time instrumentation to insert checks +of shadow memory on each memory access. Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and offset to @@ -194,7 +238,10 @@ function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. -Generic KASAN prints up to 2 call_rcu() call stacks in reports, the last one +Generic KASAN is the only mode that delays the reuse of freed object via +quarantine (see mm/kasan/quarantine.c for implementation). + +Generic KASAN prints up to two call_rcu() call stacks in reports, the last one and the second to last. Software tag-based KASAN @@ -304,15 +351,15 @@ therefore be wasteful. Furthermore, to ensure that different mappings use different shadow pages, mappings would have to be aligned to ``KASAN_GRANULE_SIZE * PAGE_SIZE``. -Instead, we share backing space across multiple mappings. We allocate +Instead, KASAN shares backing space across multiple mappings. It allocates a backing page when a mapping in vmalloc space uses a particular page of the shadow region. This page can be shared by other vmalloc mappings later on. -We hook in to the vmap infrastructure to lazily clean up unused shadow +KASAN hooks into the vmap infrastructure to lazily clean up unused shadow memory. -To avoid the difficulties around swapping mappings around, we expect +To avoid the difficulties around swapping mappings around, KASAN expects that the part of the shadow region that covers the vmalloc space will not be covered by the early shadow page, but will be left unmapped. This will require changes in arch-specific code. @@ -323,24 +370,31 @@ architectures that do not have a fixed module region. CONFIG_KASAN_KUNIT_TEST & CONFIG_TEST_KASAN_MODULE -------------------------------------------------- -``CONFIG_KASAN_KUNIT_TEST`` utilizes the KUnit Test Framework for testing. -This means each test focuses on a small unit of functionality and -there are a few ways these tests can be run. +KASAN tests consist on two parts: + +1. Tests that are integrated with the KUnit Test Framework. Enabled with +``CONFIG_KASAN_KUNIT_TEST``. These tests can be run and partially verified +automatically in a few different ways, see the instructions below. -Each test will print the KASAN report if an error is detected and then -print the number of the test and the status of the test: +2. Tests that are currently incompatible with KUnit. Enabled with +``CONFIG_TEST_KASAN_MODULE`` and can only be run as a module. These tests can +only be verified manually, by loading the kernel module and inspecting the +kernel log for KASAN reports. -pass:: +Each KUnit-compatible KASAN test prints a KASAN report if an error is detected. +Then the test prints its number and status. + +When a test passes:: ok 28 - kmalloc_double_kzfree -or, if kmalloc failed:: +When a test fails due to a failed ``kmalloc``:: # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163 Expected ptr is not null, but is not ok 4 - kmalloc_large_oob_right -or, if a KASAN report was expected, but not found:: +When a test fails due to a missing KASAN report:: # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 Expected kasan_data->report_expected == kasan_data->report_found, but @@ -348,46 +402,38 @@ or, if a KASAN report was expected, but not found:: kasan_data->report_found == 0 not ok 28 - kmalloc_double_kzfree -All test statuses are tracked as they run and an overall status will -be printed at the end:: +At the end the cumulative status of all KASAN tests is printed. On success:: ok 1 - kasan -or:: +Or, if one of the tests failed:: not ok 1 - kasan -(1) Loadable Module -~~~~~~~~~~~~~~~~~~~~ + +There are a few ways to run KUnit-compatible KASAN tests. + +1. Loadable module +~~~~~~~~~~~~~~~~~~ With ``CONFIG_KUNIT`` enabled, ``CONFIG_KASAN_KUNIT_TEST`` can be built as -a loadable module and run on any architecture that supports KASAN -using something like insmod or modprobe. The module is called ``test_kasan``. +a loadable module and run on any architecture that supports KASAN by loading +the module with insmod or modprobe. The module is called ``test_kasan``. -(2) Built-In -~~~~~~~~~~~~~ +2. Built-In +~~~~~~~~~~~ With ``CONFIG_KUNIT`` built-in, ``CONFIG_KASAN_KUNIT_TEST`` can be built-in -on any architecure that supports KASAN. These and any other KUnit -tests enabled will run and print the results at boot as a late-init -call. +on any architecure that supports KASAN. These and any other KUnit tests enabled +will run and print the results at boot as a late-init call. -(3) Using kunit_tool -~~~~~~~~~~~~~~~~~~~~~ +3. Using kunit_tool +~~~~~~~~~~~~~~~~~~~ -With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, we can also -use kunit_tool to see the results of these along with other KUnit -tests in a more readable way. This will not print the KASAN reports -of tests that passed. Use `KUnit documentation `_ for more up-to-date -information on kunit_tool. +With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, it's also +possible use ``kunit_tool`` to see the results of these and other KUnit tests +in a more readable way. This will not print the KASAN reports of the tests that +passed. Use `KUnit documentation `_ +for more up-to-date information on ``kunit_tool``. .. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html - -``CONFIG_TEST_KASAN_MODULE`` is a set of KASAN tests that could not be -converted to KUnit. These tests can be run only as a module with -``CONFIG_TEST_KASAN_MODULE`` built as a loadable module and -``CONFIG_KASAN`` built-in. The type of error expected and the -function being run is printed before the expression expected to give -an error. Then the error is printed, if found, and that test -should be interpretted to pass only if the error was the one expected -by the test.