From patchwork Tue Sep 15 13:20:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 11776587 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 74239746 for ; Tue, 15 Sep 2020 13:21:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1F21F20829 for ; Tue, 15 Sep 2020 13:21:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="XpeZsX3Z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F21F20829 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DF419900054; Tue, 15 Sep 2020 09:21:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DA4DC90004C; Tue, 15 Sep 2020 09:21:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C930B900054; Tue, 15 Sep 2020 09:21:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0198.hostedemail.com [216.40.44.198]) by kanga.kvack.org (Postfix) with ESMTP id AF4DA90004C for ; Tue, 15 Sep 2020 09:21:00 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 664DB3A82 for ; Tue, 15 Sep 2020 13:21:00 +0000 (UTC) X-FDA: 77265356280.09.slave53_470002a27111 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 3CC42180AD815 for ; Tue, 15 Sep 2020 13:21:00 +0000 (UTC) X-Spam-Summary: 20,1.5,0,5a5ae05f4490d100,d41d8cd98f00b204,3ur9gxwukcl0hoyhujrrjoh.frpolqx0-ppnydfn.ruj@flex--elver.bounces.google.com,,RULES_HIT:2:41:152:355:379:541:800:966:967:973:988:989:1260:1263:1277:1313:1314:1345:1437:1516:1518:1535:1593:1594:1605:1606:1730:1747:1777:1792:1801:2194:2196:2199:2200:2393:2525:2566:2682:2685:2693:2859:2901:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4034:4119:4250:4321:4385:4559:4605:5007:6117:6119:6261:6299:6653:6690:6742:6743:6755:7875:7903:7974:8599:8603:8784:8957:9025:9388:9969:10009:11257:11658:11914:12043:12219:12291:12297:12555:12679:12776:12895:12986:13161:13229:13255:13846:14096:14097:14394:14651:14659:21080:21094:21220:21323:21444:21451:21611:21627:21740:21990:30012:30029:30045:30054:30070:30074,0,RBL:209.85.221.74:@flex--elver.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04y851rotr4j5f665874dz snxqb3dy X-HE-Tag: slave53_470002a27111 X-Filterd-Recvd-Size: 8517 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Tue, 15 Sep 2020 13:20:59 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id r16so1206286wrm.18 for ; Tue, 15 Sep 2020 06:20:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=3v1WW0rh2eoeOYf+0D4JaYfcibqs7KNHQLEBt1uRbAw=; b=XpeZsX3ZL3cx8gGVk2Tbt+U9NXYj/ItbUog5GEEcb1+No+ZzD/4KVzi5GoiJJCB51V bJRNLWSxvPah1fy4Dc/n8t83LiNQ1KIx7CMeQeGEPUKyq+yFPc7/M4/b8aXVpv4SVrUt qrAwousVIXxiFN8JRYIqMaUdJn4bHhIy/i2fPuTRwGqeyG/NBQXymWXoE2IzCFDraXw+ gwh57Oemz7lbxOD9hrLxrDjE+m1Mi/sYgwIJ45oIjXKGYWAIP0HbF/MmFuOCfuU21E8u IvN3YXhohU4OVeavLy81H0DLdYwDou/jdQav1l1PT+jy4PUJ+4a2njVXU7mQiBNkrdOz oRzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=3v1WW0rh2eoeOYf+0D4JaYfcibqs7KNHQLEBt1uRbAw=; b=anpyKinUljIAyRJs819iKCZM5/2LuPB7zb3z0nbAlX9B76kyfx+NLrn2yFQ0W5VjQ2 NYBN7enet+pTQjPXTtFQ5WUbuf0eGRO9EMfVpH/Q7BuQIktTQBMqEjpKWmLaOCOQt0jK jFd13WvRBowad2oWnSM90iM36xN47xGhzN4sVhJKZa/2Q19geAzQrpjGol4kRcVWzM4B xsdaooz2gj/HmRTCxDy/329m2nFchcV77fcwM3lbZZtOKk2RW3BgoZspFW25Zlvix7wA 2p2q1Ga/btZQswZZhS0jY6g7O5mk+8+S1+9/VQb7WenbYofkGMCApPM8+HUFYvKuV8K0 KY3Q== X-Gm-Message-State: AOAM531exOlyTZEtaCVI5ctPpa2wuuvKxMJTlRYvEnMjAYryAfLMz/np nro73kIuUFpJ2OhDBhM9WuH/VtijKA== X-Google-Smtp-Source: ABdhPJz876+N2zJf4oc82/5JSXY1PLMnhaBDskG/2zUt6ueeQCkv0pCYT8As7W2kPVM/TWnB16xSldwfZg== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:f693:9fff:fef4:2449]) (user=elver job=sendgmr) by 2002:a05:600c:2109:: with SMTP id u9mr4457620wml.147.1600176058180; Tue, 15 Sep 2020 06:20:58 -0700 (PDT) Date: Tue, 15 Sep 2020 15:20:36 +0200 Message-Id: <20200915132046.3332537-1-elver@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.28.0.618.gf4bc123cb7-goog Subject: [PATCH v2 00/10] KFENCE: A low-overhead sampling-based memory safety error detector From: Marco Elver To: elver@google.com, akpm@linux-foundation.org, glider@google.com Cc: hpa@zytor.com, paulmck@kernel.org, andreyknvl@google.com, aryabinin@virtuozzo.com, luto@kernel.org, bp@alien8.de, catalin.marinas@arm.com, cl@linux.com, dave.hansen@linux.intel.com, rientjes@google.com, dvyukov@google.com, edumazet@google.com, gregkh@linuxfoundation.org, mingo@redhat.com, jannh@google.com, Jonathan.Cameron@huawei.com, corbet@lwn.net, iamjoonsoo.kim@lge.com, keescook@chromium.org, mark.rutland@arm.com, penberg@kernel.org, peterz@infradead.org, cai@lca.pw, tglx@linutronix.de, vbabka@suse.cz, will@kernel.org, x86@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org X-Rspamd-Queue-Id: 3CC42180AD815 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds the Kernel Electric-Fence (KFENCE) infrastructure. KFENCE is a low-overhead sampling-based memory safety error detector of heap use-after-free, invalid-free, and out-of-bounds access errors. This series enables KFENCE for the x86 and arm64 architectures, and adds KFENCE hooks to the SLAB and SLUB allocators. KFENCE is designed to be enabled in production kernels, and has near zero performance overhead. Compared to KASAN, KFENCE trades performance for precision. The main motivation behind KFENCE's design, is that with enough total uptime KFENCE will detect bugs in code paths not typically exercised by non-production test workloads. One way to quickly achieve a large enough total uptime is when the tool is deployed across a large fleet of machines. KFENCE objects each reside on a dedicated page, at either the left or right page boundaries. The pages to the left and right of the object page are "guard pages", whose attributes are changed to a protected state, and cause page faults on any attempted access to them. Such page faults are then intercepted by KFENCE, which handles the fault gracefully by reporting a memory access error. Guarded allocations are set up based on a sample interval (can be set via kfence.sample_interval). After expiration of the sample interval, the next allocation through the main allocator (SLAB or SLUB) returns a guarded allocation from the KFENCE object pool. At this point, the timer is reset, and the next allocation is set up after the expiration of the interval. To enable/disable a KFENCE allocation through the main allocator's fast-path without overhead, KFENCE relies on static branches via the static keys infrastructure. The static branch is toggled to redirect the allocation to KFENCE. The KFENCE memory pool is of fixed size, and if the pool is exhausted no further KFENCE allocations occur. The default config is conservative with only 255 objects, resulting in a pool size of 2 MiB (with 4 KiB pages). We have verified by running synthetic benchmarks (sysbench I/O, hackbench) that a kernel with KFENCE is performance-neutral compared to a non-KFENCE baseline kernel. KFENCE is inspired by GWP-ASan [1], a userspace tool with similar properties. The name "KFENCE" is a homage to the Electric Fence Malloc Debugger [2]. For more details, see Documentation/dev-tools/kfence.rst added in the series -- also viewable here: https://raw.githubusercontent.com/google/kasan/kfence/Documentation/dev-tools/kfence.rst [1] http://llvm.org/docs/GwpAsan.html [2] https://linux.die.net/man/3/efence v2: * Various comment/documentation changes (see details in patches). * Various smaller fixes (see details in patches). * Change all reports to reference the kfence object, "kfence-#nn". * Skip allocation/free internals stack trace. * Rework KMEMLEAK compatibility patch. RFC/v1: https://lkml.kernel.org/r/20200907134055.2878499-1-elver@google.com Alexander Potapenko (6): mm: add Kernel Electric-Fence infrastructure x86, kfence: enable KFENCE for x86 mm, kfence: insert KFENCE hooks for SLAB mm, kfence: insert KFENCE hooks for SLUB kfence, kasan: make KFENCE compatible with KASAN kfence, kmemleak: make KFENCE compatible with KMEMLEAK Marco Elver (4): arm64, kfence: enable KFENCE for ARM64 kfence, lockdep: make KFENCE compatible with lockdep kfence, Documentation: add KFENCE documentation kfence: add test suite Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kfence.rst | 291 +++++++++++ MAINTAINERS | 11 + arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kfence.h | 39 ++ arch/arm64/mm/fault.c | 4 + arch/x86/Kconfig | 2 + arch/x86/include/asm/kfence.h | 60 +++ arch/x86/mm/fault.c | 4 + include/linux/kfence.h | 174 +++++++ init/main.c | 2 + kernel/locking/lockdep.c | 8 + lib/Kconfig.debug | 1 + lib/Kconfig.kfence | 78 +++ mm/Makefile | 1 + mm/kasan/common.c | 7 + mm/kfence/Makefile | 6 + mm/kfence/core.c | 733 +++++++++++++++++++++++++++ mm/kfence/kfence.h | 102 ++++ mm/kfence/kfence_test.c | 777 +++++++++++++++++++++++++++++ mm/kfence/report.c | 219 ++++++++ mm/kmemleak.c | 6 + mm/slab.c | 46 +- mm/slab_common.c | 6 +- mm/slub.c | 72 ++- 25 files changed, 2619 insertions(+), 32 deletions(-) create mode 100644 Documentation/dev-tools/kfence.rst create mode 100644 arch/arm64/include/asm/kfence.h create mode 100644 arch/x86/include/asm/kfence.h create mode 100644 include/linux/kfence.h create mode 100644 lib/Kconfig.kfence create mode 100644 mm/kfence/Makefile create mode 100644 mm/kfence/core.c create mode 100644 mm/kfence/kfence.h create mode 100644 mm/kfence/kfence_test.c create mode 100644 mm/kfence/report.c Reviewed-by: Dmitry Vyukov