From patchwork Wed Mar 25 16:12:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11458241 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10000913 for ; Wed, 25 Mar 2020 16:13:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B5D4120777 for ; Wed, 25 Mar 2020 16:13:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Wrd4m9KK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5D4120777 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E1246B0037; Wed, 25 Mar 2020 12:13:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6BA7F6B006C; Wed, 25 Mar 2020 12:13:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CE7C6B006E; Wed, 25 Mar 2020 12:13:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0078.hostedemail.com [216.40.44.78]) by kanga.kvack.org (Postfix) with ESMTP id 462DA6B0037 for ; Wed, 25 Mar 2020 12:13:20 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 24B68180CADCD for ; Wed, 25 Mar 2020 16:13:20 +0000 (UTC) X-FDA: 76634379360.25.plate70_74603851c442d X-Spam-Summary: 2,0,0,41ce79441db70497,d41d8cd98f00b204,3hon7xgykca0tyvqr4t11tyr.p1zyv07a-zzx8npx.14t@flex--glider.bounces.google.com,,RULES_HIT:1:2:41:152:355:379:541:800:960:966:968:973:988:989:1042:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2538:2553:2559:2562:2693:2911:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3871:3872:3873:3874:4050:4250:4321:4385:4425:5007:6119:6261:6653:6742:6743:7875:7903:8603:8660:9108:9969:10004:11026:11473:11657:11658:11914:12043:12048:12291:12295:12297:12438:12555:12683:12895:12986:13148:13161:13229:13230:13846:14096:14097:14394:14659:21080:21365:21433:21444:21451:21627:21772:21796:21990:30003:30036:30045:30054:30064:30067:30090,0,RBL:209.85.221.73:@flex--glider.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMA RY:none X-HE-Tag: plate70_74603851c442d X-Filterd-Recvd-Size: 11417 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Wed, 25 Mar 2020 16:13:19 +0000 (UTC) Received: by mail-wr1-f73.google.com with SMTP id f8so1378051wrp.1 for ; Wed, 25 Mar 2020 09:13:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=7Dtu31tIDHcw+zCR3bJfo8J+sswtZptCukakiPro7Cg=; b=Wrd4m9KKfuatTTvlkbvSy8EmiDj7JgzKy9nGVH2jt9xkaIu9JJezGDffEzzh0AO0C+ FIjbWWLNZycvjeNlY4YEwzW07ihOOzTKAlSszTwmbSFJ3gdhjwk5rl+rr29IJGJlNmjZ RuecbYKO49zlRBKIxM/OnA1UzixRo1CLO9bFqYxl/FPqYDycCXC7R94zgEpYT+7PJkj9 XSF17SjSq20JeZx+MbIX2vEhplkE0Lf7aKSd9tLibZcnhndRnX3IW8y8YiGZ+KgEorvX hZ11yrCdU2qU8WGG92ED4Yisb5xHRBCwMVKwHL5u6EfuL+1GLViQxousXg3jLbssdjp9 dYJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=7Dtu31tIDHcw+zCR3bJfo8J+sswtZptCukakiPro7Cg=; b=pBvKWRGu2T561+/PgpiL2sC0M2ccJBlip4NPvmzVch4oM5t8NemeIVifeZxEFVtpOd EBHnvS7YqtEMl4fye0QNx1IYv6h1t1lwTFNpGDGgeeaeCTZ+bFsNLU/QMorAxcr+RqO9 QWd8pQ3WHpJh9NrL3WkO0bMPXcxN7qTeDXpsIu3Npu9MO+pWS92cQL9ZJERy7LidjrjC lTtnNIdlrZ1eehHmFg/+f/ekq0e5S73Nl+GyWD5nW573/tt8XfcCJkpBJpe33o8FY5G7 bK0HxjVX0dgaCDfEac853H2QYuGiKd1VCWxxTGWh79t3Lad99FGFHBbd+7L+Fu+GzboY oDiQ== X-Gm-Message-State: ANhLgQ01ufcU+JutKY2t3jpsjuEoX/nV2pE5gWl/fde5bbxwEhIfDRaR GfCssWOgBE3gRVho5/+0tFUjlINma5s= X-Google-Smtp-Source: ADFU+vtoM3+tpdJx12Tp4R20F1l9cPNGCR9KbiLcDae2PgxIQoHw5qOSO4Ho/3ZF7jgk/PXmoV/H0JKlRsk= X-Received: by 2002:adf:bc04:: with SMTP id s4mr4190860wrg.244.1585152798148; Wed, 25 Mar 2020 09:13:18 -0700 (PDT) Date: Wed, 25 Mar 2020 17:12:18 +0100 In-Reply-To: <20200325161249.55095-1-glider@google.com> Message-Id: <20200325161249.55095-8-glider@google.com> Mime-Version: 1.0 References: <20200325161249.55095-1-glider@google.com> X-Mailer: git-send-email 2.25.1.696.g5e7596f4ac-goog Subject: [PATCH v5 07/38] kmsan: KMSAN compiler API implementation From: glider@google.com To: Jens Axboe , Andy Lutomirski , Wolfram Sang , Christoph Hellwig , Vegard Nossum , Dmitry Vyukov , Marco Elver , Andrey Konovalov , linux-mm@kvack.org Cc: glider@google.com, viro@zeniv.linux.org.uk, adilger.kernel@dilger.ca, akpm@linux-foundation.org, aryabinin@virtuozzo.com, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@infradead.org, darrick.wong@oracle.com, davem@davemloft.net, dmitry.torokhov@gmail.com, ebiggers@google.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, iii@linux.ibm.com, mingo@elte.hu, jasowang@redhat.com, m.szyprowski@samsung.com, mark.rutland@arm.com, martin.petersen@oracle.com, schwidefsky@de.ibm.com, willy@infradead.org, mst@redhat.com, mhocko@suse.com, monstr@monstr.eu, pmladek@suse.com, cai@lca.pw, rdunlap@infradead.org, robin.murphy@arm.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, gor@linux.ibm.com X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: kmsan_instr.c contains the functions called by KMSAN instrumentation. These include functions that: - return shadow/origin pointers for memory accesses; - poison and unpoison local variables; - provide KMSAN context state to pass metadata for function arguments; - perform string operations (mem*) on metadata; - tell KMSAN to report an error. This patch has been split away from the rest of KMSAN runtime to simplify the review process. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Jens Axboe Cc: Andy Lutomirski Cc: Wolfram Sang Cc: Christoph Hellwig Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: Marco Elver Cc: Andrey Konovalov Cc: linux-mm@kvack.org --- v4: - split this patch away as requested by Andrey Konovalov - removed redundant address checks when copying shadow - fix __msan_memmove prototype Change-Id: I826272ed2ebe8ab8ef61a9d4cccdcf07c7b6b499 --- mm/kmsan/kmsan_instr.c | 229 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 229 insertions(+) create mode 100644 mm/kmsan/kmsan_instr.c diff --git a/mm/kmsan/kmsan_instr.c b/mm/kmsan/kmsan_instr.c new file mode 100644 index 0000000000000..0de8aafac5101 --- /dev/null +++ b/mm/kmsan/kmsan_instr.c @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN compiler API. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include "kmsan.h" +#include +#include + +static bool is_bad_asm_addr(void *addr, u64 size, bool is_store) +{ + if ((u64)addr < TASK_SIZE) + return true; + if (!kmsan_get_metadata(addr, size, META_SHADOW)) + return true; + return false; +} + +struct shadow_origin_ptr __msan_metadata_ptr_for_load_n(void *addr, u64 size) +{ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false); +} +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_n); + +struct shadow_origin_ptr __msan_metadata_ptr_for_store_n(void *addr, u64 size) +{ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true); +} +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_n); + +#define DECLARE_METADATA_PTR_GETTER(size) \ +struct shadow_origin_ptr __msan_metadata_ptr_for_load_##size(void *addr) \ +{ \ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false); \ +} \ +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_##size); \ + \ +struct shadow_origin_ptr __msan_metadata_ptr_for_store_##size(void *addr) \ +{ \ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true); \ +} \ +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_##size) + +DECLARE_METADATA_PTR_GETTER(1); +DECLARE_METADATA_PTR_GETTER(2); +DECLARE_METADATA_PTR_GETTER(4); +DECLARE_METADATA_PTR_GETTER(8); + +void __msan_instrument_asm_store(void *addr, u64 size) +{ + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return; + /* + * Most of the accesses are below 32 bytes. The two exceptions so far + * are clwb() (64 bytes) and FPU state (512 bytes). + * It's unlikely that the assembly will touch more than 512 bytes. + */ + if (size > 512) { + WARN_ONCE(1, "assembly store size too big: %d\n", size); + size = 8; + } + if (is_bad_asm_addr(addr, size, /*is_store*/true)) + return; + irq_flags = kmsan_enter_runtime(); + /* Unpoisoning the memory on best effort. */ + kmsan_internal_unpoison_shadow(addr, size, /*checked*/false); + kmsan_leave_runtime(irq_flags); +} +EXPORT_SYMBOL(__msan_instrument_asm_store); + +void *__msan_memmove(void *dst, const void *src, size_t n) +{ + void *result; + + result = __memmove(dst, src, n); + if (!n) + /* Some people call memmove() with zero length. */ + return result; + if (!kmsan_ready || kmsan_in_runtime()) + return result; + + kmsan_memmove_metadata(dst, (void *)src, n); + + return result; +} +EXPORT_SYMBOL(__msan_memmove); + +void *__msan_memmove_nosanitize(void *dst, void *src, u64 n) +{ + return __memmove(dst, src, n); +} +EXPORT_SYMBOL(__msan_memmove_nosanitize); + +void *__msan_memcpy(void *dst, const void *src, u64 n) +{ + void *result; + + result = __memcpy(dst, src, n); + if (!n) + /* Some people call memcpy() with zero length. */ + return result; + + if (!kmsan_ready || kmsan_in_runtime()) + return result; + + kmsan_memcpy_metadata(dst, (void *)src, n); + + return result; +} +EXPORT_SYMBOL(__msan_memcpy); + +void *__msan_memcpy_nosanitize(void *dst, void *src, u64 n) +{ + return __memcpy(dst, src, n); +} +EXPORT_SYMBOL(__msan_memcpy_nosanitize); + +void *__msan_memset(void *dst, int c, size_t n) +{ + void *result; + unsigned long irq_flags; + + result = __memset(dst, c, n); + if (!kmsan_ready || kmsan_in_runtime()) + return result; + + irq_flags = kmsan_enter_runtime(); + /* + * Clang doesn't pass parameter metadata here, so it is impossible to + * use shadow of @c to set up the shadow for @dst. + */ + kmsan_internal_unpoison_shadow(dst, n, /*checked*/false); + kmsan_leave_runtime(irq_flags); + + return result; +} +EXPORT_SYMBOL(__msan_memset); + +void *__msan_memset_nosanitize(void *dst, int c, size_t n) +{ + return __memset(dst, c, n); +} +EXPORT_SYMBOL(__msan_memset_nosanitize); + +depot_stack_handle_t __msan_chain_origin(depot_stack_handle_t origin) +{ + depot_stack_handle_t ret = 0; + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return ret; + + /* Creating new origins may allocate memory. */ + irq_flags = kmsan_enter_runtime(); + ret = kmsan_internal_chain_origin(origin); + kmsan_leave_runtime(irq_flags); + return ret; +} +EXPORT_SYMBOL(__msan_chain_origin); + +void __msan_poison_alloca(void *address, u64 size, char *descr) +{ + depot_stack_handle_t handle; + unsigned long entries[4]; + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return; + + kmsan_internal_memset_shadow(address, -1, size, /*checked*/true); + + entries[0] = KMSAN_ALLOCA_MAGIC_ORIGIN; + entries[1] = (u64)descr; + entries[2] = (u64)__builtin_return_address(0); + entries[3] = (u64)kmsan_internal_return_address(1); + + /* stack_depot_save() may allocate memory. */ + irq_flags = kmsan_enter_runtime(); + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC); + kmsan_leave_runtime(irq_flags); + kmsan_internal_set_origin(address, size, handle); +} +EXPORT_SYMBOL(__msan_poison_alloca); + +void __msan_unpoison_alloca(void *address, u64 size) +{ + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return; + + irq_flags = kmsan_enter_runtime(); + kmsan_internal_unpoison_shadow(address, size, /*checked*/true); + kmsan_leave_runtime(irq_flags); +} +EXPORT_SYMBOL(__msan_unpoison_alloca); + +void __msan_warning(u32 origin) +{ + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return; + irq_flags = kmsan_enter_runtime(); + kmsan_report(origin, /*address*/0, /*size*/0, + /*off_first*/0, /*off_last*/0, /*user_addr*/0, REASON_ANY); + kmsan_leave_runtime(irq_flags); +} +EXPORT_SYMBOL(__msan_warning); + +struct kmsan_context_state *__msan_get_context_state(void) +{ + struct kmsan_context_state *ret; + + ret = kmsan_task_context_state(); + BUG_ON(!ret); + return ret; +} +EXPORT_SYMBOL(__msan_get_context_state);