From patchwork Mon Sep 21 13:26:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 11789863 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7EC90139A for ; Mon, 21 Sep 2020 13:27:02 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2C43322574 for ; Mon, 21 Sep 2020 13:27:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="aLLJXYyG"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="lnXG6wiq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2C43322574 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:Mime-Version:Message-Id:Date: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=XmXOE7XYhU0Ae3l8BL7337o3OTbU3qbxjirqBQ1MrvQ=; b=aLLJXYyGgbxQqHtT1WzLAgrOr0 dukf2po+BNWuc9e8whJFxxHl3Hg3NNrst2tBX7niy/SQjTtVcOlQ0CDi7L1Ca3vCmlpjx9Md8AtI9 M9ggldgy1GuuGIOdkwxuyebB1tjuw+QcTv4mSHtv3pYibjyebZY1kKyRQsWQCfjMvYtMSAM8F68tE 1fBKk6h4Qj5QIiGylBlYzVtqBWXRdYTwAEU+Px6bSoUxfuhb0zOy3YlxFie7jcNrXV+Kb7YJZMHZv VJY/2pY+u3XA2Q+rSsWZZ4yDzXRqjcE5YsJRYMjGaeNUX/D1yup59tXj/pVm745Gib3FP5HZP3j3f ogtFoJgw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kKLq7-0007c7-Gv; Mon, 21 Sep 2020 13:26:43 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kKLq3-0007aK-M5 for linux-arm-kernel@lists.infradead.org; Mon, 21 Sep 2020 13:26:40 +0000 Received: by mail-wr1-x44a.google.com with SMTP id r16so5847472wrm.18 for ; Mon, 21 Sep 2020 06:26:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=EdTmVWGaBxdasegvEqn7J8/FntcyBLgQW6ssuU6G+uY=; b=lnXG6wiqGOGBXPQPYdbxIqzXj/EsfLNKDABAmaM/yeVcSkvQbro9/uLwR1z3cLAyKW IvGmn8fhlBoU3TDbuy+xcDsxKmHpaAhmAoFDVGDtIigifLUTi4vKm1mAStZ+9UgM0eat ENjfkuVk4rUGKQJZzDVyoO79zd/5kuthc4MLoJTmXmnr7d7JOFrknBYXnaaRZBpN0RRQ 37S/mOAH++NDMhn4g16G6i+1S+yugEjMY2uQMfaCMAr9K18pHgUavbr+8SDxQVEmKBiM 58aAoxXtjFwsDSR/u6xi3WGKt51fN0LrMAIH6Cxj3HUandudidZ7z0TNVIOW7vo7U6gC QJMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=EdTmVWGaBxdasegvEqn7J8/FntcyBLgQW6ssuU6G+uY=; b=G7wuUNYs+bCnU+8JcL33PVQHmYoYh99urWXhoSGCrZBAcCaQYp8Dokz3PvlCNHZ0gc 7QfdW7r2ztOD7eG5DhTCLBoLrbo8TAGqiqNYxpZ2V+EvRgJgKKdybPF4QQRSXJfGykgE A7ew7B7bDnsOIHRlpy2gb7nVgYEihxegMc7N9VcgL1OK/E0+YEmGsD1OMZ5BPjkBEWL7 8+WmV3voyCUj2tc7Gc78OPO8L+5NBWgbjGXmbMSTPRrGh5cOFab5rw3CvIG2WIaG6HwY unHQNpThhKD23wDy0NKHRrfhSHaKWHN4nTnd+1r5tHnHwjyqt4CvTxjiLBy6byFyb3PI BxiQ== X-Gm-Message-State: AOAM5308dHrKjgiV8ZMDAJN8wFsAoo5zvW6ZHAc5nS2EZixzqKtKRz4R 9XYHSepOUjcgV0GNFQD/GSJdyOXFPQ== X-Google-Smtp-Source: ABdhPJwX1rHHuzQvgCheXOVUsUfAWEjkRMw91N899aD3K9MCxwDTcLNXL2ZcV8kJQBav6XPJiTknJFPm9A== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:f693:9fff:fef4:2449]) (user=elver job=sendgmr) by 2002:a7b:c1d3:: with SMTP id a19mr29455454wmj.19.1600694796072; Mon, 21 Sep 2020 06:26:36 -0700 (PDT) Date: Mon, 21 Sep 2020 15:26:01 +0200 Message-Id: <20200921132611.1700350-1-elver@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.28.0.681.g6f77f65b4e-goog Subject: [PATCH v3 00/10] KFENCE: A low-overhead sampling-based memory safety error detector From: Marco Elver To: elver@google.com, akpm@linux-foundation.org, glider@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200921_092639_742517_318C1F30 X-CRM114-Status: GOOD ( 17.36 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:44a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.0 DKIMWL_WL_MED DKIMwl.org - Medium trust sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, hdanton@sina.com, linux-doc@vger.kernel.org, peterz@infradead.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, edumazet@google.com, hpa@zytor.com, cl@linux.com, will@kernel.org, sjpark@amazon.com, corbet@lwn.net, x86@kernel.org, kasan-dev@googlegroups.com, mingo@redhat.com, vbabka@suse.cz, rientjes@google.com, aryabinin@virtuozzo.com, keescook@chromium.org, paulmck@kernel.org, jannh@google.com, andreyknvl@google.com, bp@alien8.de, luto@kernel.org, Jonathan.Cameron@huawei.com, tglx@linutronix.de, dvyukov@google.com, linux-arm-kernel@lists.infradead.org, gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, penberg@kernel.org, iamjoonsoo.kim@lge.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org This adds the Kernel Electric-Fence (KFENCE) infrastructure. KFENCE is a low-overhead sampling-based memory safety error detector of heap use-after-free, invalid-free, and out-of-bounds access errors. This series enables KFENCE for the x86 and arm64 architectures, and adds KFENCE hooks to the SLAB and SLUB allocators. KFENCE is designed to be enabled in production kernels, and has near zero performance overhead. Compared to KASAN, KFENCE trades performance for precision. The main motivation behind KFENCE's design, is that with enough total uptime KFENCE will detect bugs in code paths not typically exercised by non-production test workloads. One way to quickly achieve a large enough total uptime is when the tool is deployed across a large fleet of machines. KFENCE objects each reside on a dedicated page, at either the left or right page boundaries. The pages to the left and right of the object page are "guard pages", whose attributes are changed to a protected state, and cause page faults on any attempted access to them. Such page faults are then intercepted by KFENCE, which handles the fault gracefully by reporting a memory access error. Guarded allocations are set up based on a sample interval (can be set via kfence.sample_interval). After expiration of the sample interval, the next allocation through the main allocator (SLAB or SLUB) returns a guarded allocation from the KFENCE object pool. At this point, the timer is reset, and the next allocation is set up after the expiration of the interval. To enable/disable a KFENCE allocation through the main allocator's fast-path without overhead, KFENCE relies on static branches via the static keys infrastructure. The static branch is toggled to redirect the allocation to KFENCE. The KFENCE memory pool is of fixed size, and if the pool is exhausted no further KFENCE allocations occur. The default config is conservative with only 255 objects, resulting in a pool size of 2 MiB (with 4 KiB pages). We have verified by running synthetic benchmarks (sysbench I/O, hackbench) that a kernel with KFENCE is performance-neutral compared to a non-KFENCE baseline kernel. KFENCE is inspired by GWP-ASan [1], a userspace tool with similar properties. The name "KFENCE" is a homage to the Electric Fence Malloc Debugger [2]. For more details, see Documentation/dev-tools/kfence.rst added in the series -- also viewable here: https://raw.githubusercontent.com/google/kasan/kfence/Documentation/dev-tools/kfence.rst [1] http://llvm.org/docs/GwpAsan.html [2] https://linux.die.net/man/3/efence v3: * Rewrite SLAB/SLUB patch descriptions to clarify need for 'orig_size'. * Various smaller fixes (see details in patches). v2: https://lkml.kernel.org/r/20200915132046.3332537-1-elver@google.com * Various comment/documentation changes (see details in patches). * Various smaller fixes (see details in patches). * Change all reports to reference the kfence object, "kfence-#nn". * Skip allocation/free internals stack trace. * Rework KMEMLEAK compatibility patch. RFC/v1: https://lkml.kernel.org/r/20200907134055.2878499-1-elver@google.com Alexander Potapenko (6): mm: add Kernel Electric-Fence infrastructure x86, kfence: enable KFENCE for x86 mm, kfence: insert KFENCE hooks for SLAB mm, kfence: insert KFENCE hooks for SLUB kfence, kasan: make KFENCE compatible with KASAN kfence, kmemleak: make KFENCE compatible with KMEMLEAK Marco Elver (4): arm64, kfence: enable KFENCE for ARM64 kfence, lockdep: make KFENCE compatible with lockdep kfence, Documentation: add KFENCE documentation kfence: add test suite Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kfence.rst | 291 +++++++++++ MAINTAINERS | 11 + arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kfence.h | 39 ++ arch/arm64/mm/fault.c | 4 + arch/x86/Kconfig | 2 + arch/x86/include/asm/kfence.h | 60 +++ arch/x86/mm/fault.c | 4 + include/linux/kfence.h | 174 +++++++ init/main.c | 2 + kernel/locking/lockdep.c | 8 + lib/Kconfig.debug | 1 + lib/Kconfig.kfence | 78 +++ mm/Makefile | 1 + mm/kasan/common.c | 7 + mm/kfence/Makefile | 6 + mm/kfence/core.c | 733 +++++++++++++++++++++++++++ mm/kfence/kfence.h | 102 ++++ mm/kfence/kfence_test.c | 777 +++++++++++++++++++++++++++++ mm/kfence/report.c | 219 ++++++++ mm/kmemleak.c | 6 + mm/slab.c | 46 +- mm/slab_common.c | 6 +- mm/slub.c | 72 ++- 25 files changed, 2619 insertions(+), 32 deletions(-) create mode 100644 Documentation/dev-tools/kfence.rst create mode 100644 arch/arm64/include/asm/kfence.h create mode 100644 arch/x86/include/asm/kfence.h create mode 100644 include/linux/kfence.h create mode 100644 lib/Kconfig.kfence create mode 100644 mm/kfence/Makefile create mode 100644 mm/kfence/core.c create mode 100644 mm/kfence/kfence.h create mode 100644 mm/kfence/kfence_test.c create mode 100644 mm/kfence/report.c