From patchwork Wed Mar 25 16:12:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11458285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 501EA6CA for ; Wed, 25 Mar 2020 16:14:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 020E920409 for ; Wed, 25 Mar 2020 16:14:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="prjBqCQU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 020E920409 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CFC596B00A0; Wed, 25 Mar 2020 12:14:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C5F886B00A2; Wed, 25 Mar 2020 12:14:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB4066B00A3; Wed, 25 Mar 2020 12:14:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0033.hostedemail.com [216.40.44.33]) by kanga.kvack.org (Postfix) with ESMTP id 878C66B00A0 for ; Wed, 25 Mar 2020 12:14:23 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 6BF4784781 for ; Wed, 25 Mar 2020 16:14:23 +0000 (UTC) X-FDA: 76634382006.07.pull39_7d97cae83b414 X-Spam-Summary: 2,0,0,5ed3f24c9ff08ea3,d41d8cd98f00b204,3xyn7xgykcewuzwrs5u22uzs.q20zw18b-00y9oqy.25u@flex--glider.bounces.google.com,,RULES_HIT:1:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:1981:2194:2199:2393:2559:2562:2636:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3871:3872:3874:4321:4605:5007:6261:6653:6742:6743:7875:7903:7904:8603:9969:10004:11026:11473:11657:11658:11914:12043:12048:12291:12297:12438:12555:12683:12895:12986:13846:14096:14097:14394:14659:21080:21365:21444:21451:21627:21990:30012:30051:30054:30064:30070,0,RBL:209.85.221.74:@flex--glider.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:62,LUA_SUMMARY:none X-HE-Tag: pull39_7d97cae83b414 X-Filterd-Recvd-Size: 14488 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Wed, 25 Mar 2020 16:14:22 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id o18so1372076wrx.9 for ; Wed, 25 Mar 2020 09:14:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=xmEL4PoSL4dRoy+EDuixfj+9Bi6u9R1TX0Usnq36bgE=; b=prjBqCQUE169ghwz94hYnk4baHKSrXUFg7F0ufECHrJWj+Zv6bmmfW+kaM6OypKc48 mXAcJuzRL0uCKPO5Bv1AZTdNqIsZa6B6kn8cniXRBF/nANIlRhS9LjbODWn0TNV0GJwy v8sZumgKsUGTWymfd1wVBnyuiF232VPa6aI3VpAz80GHdYEjDHB4HnJ9xMp6G9yZqR8n 9MjnCMJoKSpK0f4LtHURqqo25gxqAH2xwRY/ZmiJGSS5hEtdVZ1uZ9epXBhgNL85u2p/ tQzX64lrRCoffZiptIwMfeSXQ9o7QXmIBEDDZ5dTr1bBucLbnNgoAxvv+ieFY2i8eKq4 BrSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=xmEL4PoSL4dRoy+EDuixfj+9Bi6u9R1TX0Usnq36bgE=; b=SqEYkRQ+9u0P9QZtgDv+P0rnl67Ye3fMRD5wJfmGHStES/io/F4Ud+zrQC/I3ICNJn OxNoW/fJoQVEOM/MCZk+sRamZXX/7qyV9zpaZqkBFiiiqsW9pyezYsj8M3Kjd7vNZiJq qWfvnJcRp8Vxg65oPbIfAkm+dgPzoT7snaTgMnidEA2tixkX5mEHgLjwYygZwsdylWuK cbEHsbVbpMVq+KTpbkyz3HBzmQNysSjM3mAnTe6gE9n2IMobr3YfPpfTNeis4iqHNLPR UirMKj5Va74VeMmA9mJ4S+TEGfQfQ1yqlEA36DI2hExuR02kohbcNXFPr38lmmXJv5Sm 31qA== X-Gm-Message-State: ANhLgQ2lLWMZlGBJvhfJs09BQE9mloSlOW7mXl8VidIqXEgu4U4KdKVE xx79OPk3jWmAHZRK54/yU+jLv5SEeBU= X-Google-Smtp-Source: ADFU+vt4lZrAygC4LS8Fq0zS/l/3Scsc+fN7a/p63q/Ptkr3O+i7vKKSjeQa4961xbhRBOMvIACKzquD7Z4= X-Received: by 2002:a5d:4683:: with SMTP id u3mr4193014wrq.248.1585152861512; Wed, 25 Mar 2020 09:14:21 -0700 (PDT) Date: Wed, 25 Mar 2020 17:12:38 +0100 In-Reply-To: <20200325161249.55095-1-glider@google.com> Message-Id: <20200325161249.55095-28-glider@google.com> Mime-Version: 1.0 References: <20200325161249.55095-1-glider@google.com> X-Mailer: git-send-email 2.25.1.696.g5e7596f4ac-goog Subject: [PATCH v5 27/38] kmsan: hooks for copy_to_user() and friends From: glider@google.com To: Alexander Viro , Vegard Nossum , Dmitry Vyukov , Marco Elver , Andrey Konovalov , linux-mm@kvack.org Cc: glider@google.com, adilger.kernel@dilger.ca, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@infradead.org, hch@lst.de, darrick.wong@oracle.com, davem@davemloft.net, dmitry.torokhov@gmail.com, ebiggers@google.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, iii@linux.ibm.com, mingo@elte.hu, jasowang@redhat.com, axboe@kernel.dk, m.szyprowski@samsung.com, mark.rutland@arm.com, martin.petersen@oracle.com, schwidefsky@de.ibm.com, willy@infradead.org, mst@redhat.com, mhocko@suse.com, monstr@monstr.eu, pmladek@suse.com, cai@lca.pw, rdunlap@infradead.org, robin.murphy@arm.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, gor@linux.ibm.com, wsa@the-dreams.de X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Memory that is copied from userspace must be unpoisoned. Before copying memory to userspace, check it and report an error if it contains uninitialized bits. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Alexander Viro Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: Marco Elver Cc: Andrey Konovalov Cc: linux-mm@kvack.org --- v3: - fixed compilation errors reported by kbuild test bot v4: - minor variable fixes as requested by Andrey Konovalov - simplified code around copy_to_user() hooks v5: - fixed use of uninitialized value spotted by kbuild test robot Change-Id: I38428b9c7d1909b8441dcec1749b080494a7af99 --- arch/x86/include/asm/uaccess.h | 10 ++++++++++ include/asm-generic/cacheflush.h | 7 ++++++- include/asm-generic/uaccess.h | 12 +++++++++-- include/linux/uaccess.h | 34 ++++++++++++++++++++++++++------ lib/iov_iter.c | 14 +++++++++---- lib/usercopy.c | 8 ++++++-- 6 files changed, 70 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h index 61d93f062a36e..bfb55fdba5df4 100644 --- a/arch/x86/include/asm/uaccess.h +++ b/arch/x86/include/asm/uaccess.h @@ -6,6 +6,7 @@ */ #include #include +#include #include #include #include @@ -174,6 +175,7 @@ __typeof__(__builtin_choose_expr(sizeof(x) > sizeof(0UL), 0ULL, 0UL)) ASM_CALL_CONSTRAINT \ : "0" (ptr), "i" (sizeof(*(ptr)))); \ (x) = (__force __typeof__(*(ptr))) __val_gu; \ + kmsan_unpoison_shadow(&(x), sizeof(*(ptr))); \ __builtin_expect(__ret_gu, 0); \ }) @@ -248,6 +250,7 @@ extern void __put_user_8(void); __chk_user_ptr(ptr); \ might_fault(); \ __pu_val = x; \ + kmsan_check_memory(&(__pu_val), sizeof(*(ptr))); \ switch (sizeof(*(ptr))) { \ case 1: \ __put_user_x(1, __pu_val, ptr, __ret_pu); \ @@ -270,7 +273,9 @@ extern void __put_user_8(void); #define __put_user_size(x, ptr, size, label) \ do { \ + __typeof__(*(ptr)) __pus_val = x; \ __chk_user_ptr(ptr); \ + kmsan_check_memory(&(__pus_val), size); \ switch (size) { \ case 1: \ __put_user_goto(x, ptr, "b", "b", "iq", label); \ @@ -295,7 +300,9 @@ do { \ */ #define __put_user_size_ex(x, ptr, size) \ do { \ + __typeof__(*(ptr)) __puse_val = x; \ __chk_user_ptr(ptr); \ + kmsan_check_memory(&(__puse_val), size); \ switch (size) { \ case 1: \ __put_user_asm_ex(x, ptr, "b", "b", "iq"); \ @@ -363,6 +370,7 @@ do { \ default: \ (x) = __get_user_bad(); \ } \ + kmsan_unpoison_shadow(&(x), size); \ } while (0) #define __get_user_asm(x, addr, err, itype, rtype, ltype, errret) \ @@ -413,6 +421,7 @@ do { \ default: \ (x) = __get_user_bad(); \ } \ + kmsan_unpoison_shadow(&(x), size); \ } while (0) #define __get_user_asm_ex(x, addr, itype, rtype, ltype) \ @@ -433,6 +442,7 @@ do { \ __typeof__(ptr) __pu_ptr = (ptr); \ __typeof__(size) __pu_size = (size); \ __uaccess_begin(); \ + kmsan_check_memory(&(__pu_val), size); \ __put_user_size(__pu_val, __pu_ptr, __pu_size, __pu_label); \ __pu_err = 0; \ __pu_label: \ diff --git a/include/asm-generic/cacheflush.h b/include/asm-generic/cacheflush.h index cac7404b2bdd2..7023c44457ef9 100644 --- a/include/asm-generic/cacheflush.h +++ b/include/asm-generic/cacheflush.h @@ -4,6 +4,7 @@ /* Keep includes the same across arches. */ #include +#include #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0 @@ -99,6 +100,7 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end) #ifndef copy_to_user_page #define copy_to_user_page(vma, page, vaddr, dst, src, len) \ do { \ + kmsan_check_memory(src, len); \ memcpy(dst, src, len); \ flush_icache_user_range(vma, page, vaddr, len); \ } while (0) @@ -106,7 +108,10 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end) #ifndef copy_from_user_page #define copy_from_user_page(vma, page, vaddr, dst, src, len) \ - memcpy(dst, src, len) + do { \ + memcpy(dst, src, len); \ + kmsan_unpoison_shadow(dst, len); \ + } while (0) #endif #endif /* __ASM_CACHEFLUSH_H */ diff --git a/include/asm-generic/uaccess.h b/include/asm-generic/uaccess.h index e935318804f8a..88b626c3ef2de 100644 --- a/include/asm-generic/uaccess.h +++ b/include/asm-generic/uaccess.h @@ -142,7 +142,11 @@ static inline int __access_ok(unsigned long addr, unsigned long size) static inline int __put_user_fn(size_t size, void __user *ptr, void *x) { - return unlikely(raw_copy_to_user(ptr, x, size)) ? -EFAULT : 0; + int n; + + n = raw_copy_to_user(ptr, x, size); + kmsan_copy_to_user(ptr, x, size, n); + return unlikely(n) ? -EFAULT : 0; } #define __put_user_fn(sz, u, k) __put_user_fn(sz, u, k) @@ -203,7 +207,11 @@ extern int __put_user_bad(void) __attribute__((noreturn)); #ifndef __get_user_fn static inline int __get_user_fn(size_t size, const void __user *ptr, void *x) { - return unlikely(raw_copy_from_user(x, ptr, size)) ? -EFAULT : 0; + int res; + + res = raw_copy_from_user(x, ptr, size); + kmsan_unpoison_shadow(x, size - res); + return unlikely(res) ? -EFAULT : 0; } #define __get_user_fn(sz, u, k) __get_user_fn(sz, u, k) diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index 8a215c5c1aed8..b38bdeb135dde 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -5,6 +5,7 @@ #include #include #include +#include #define uaccess_kernel() segment_eq(get_fs(), KERNEL_DS) @@ -58,18 +59,26 @@ static __always_inline __must_check unsigned long __copy_from_user_inatomic(void *to, const void __user *from, unsigned long n) { + unsigned long res; + instrument_copy_from_user(to, from, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + res = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, n - res); + return res; } static __always_inline __must_check unsigned long __copy_from_user(void *to, const void __user *from, unsigned long n) { + unsigned long res; + might_fault(); instrument_copy_from_user(to, from, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + res = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, n - res); + return res; } /** @@ -88,18 +97,26 @@ __copy_from_user(void *to, const void __user *from, unsigned long n) static __always_inline __must_check unsigned long __copy_to_user_inatomic(void __user *to, const void *from, unsigned long n) { + unsigned long res; + instrument_copy_to_user(to, from, n); check_object_size(from, n, true); - return raw_copy_to_user(to, from, n); + res = raw_copy_to_user(to, from, n); + kmsan_copy_to_user((const void *)to, from, n, res); + return res; } static __always_inline __must_check unsigned long __copy_to_user(void __user *to, const void *from, unsigned long n) { + unsigned long res; + might_fault(); instrument_copy_to_user(to, from, n); check_object_size(from, n, true); - return raw_copy_to_user(to, from, n); + res = raw_copy_to_user(to, from, n); + kmsan_copy_to_user((const void *)to, from, n, res); + return res; } #ifdef INLINE_COPY_FROM_USER @@ -107,10 +124,12 @@ static inline __must_check unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n) { unsigned long res = n; + might_fault(); if (likely(access_ok(from, n))) { instrument_copy_from_user(to, from, n); res = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, n - res); } if (unlikely(res)) memset(to + (n - res), 0, res); @@ -125,12 +144,15 @@ _copy_from_user(void *, const void __user *, unsigned long); static inline __must_check unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n) { + unsigned long res = n; + might_fault(); if (access_ok(to, n)) { instrument_copy_to_user(to, from, n); - n = raw_copy_to_user(to, from, n); + res = raw_copy_to_user(to, from, n); + kmsan_copy_to_user(to, from, n, res); } - return n; + return res; } #else extern __must_check unsigned long diff --git a/lib/iov_iter.c b/lib/iov_iter.c index bf538c2bec777..179c28455693d 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -138,20 +138,26 @@ static int copyout(void __user *to, const void *from, size_t n) { + int res; + if (access_ok(to, n)) { instrument_copy_to_user(to, from, n); - n = raw_copy_to_user(to, from, n); + res = raw_copy_to_user(to, from, n); + kmsan_copy_to_user(to, from, n, res); } - return n; + return res; } static int copyin(void *to, const void __user *from, size_t n) { + size_t res; + if (access_ok(from, n)) { instrument_copy_from_user(to, from, n); - n = raw_copy_from_user(to, from, n); + res = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, n - res); } - return n; + return res; } static size_t copy_page_to_iter_iovec(struct page *page, size_t offset, size_t bytes, diff --git a/lib/usercopy.c b/lib/usercopy.c index 4bb1c5e7a3eb0..bf313548c4039 100644 --- a/lib/usercopy.c +++ b/lib/usercopy.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #include /* out-of-line parts */ @@ -13,6 +14,7 @@ unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n if (likely(access_ok(from, n))) { instrument_copy_from_user(to, from, n); res = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, n - res); } if (unlikely(res)) memset(to + (n - res), 0, res); @@ -24,12 +26,14 @@ EXPORT_SYMBOL(_copy_from_user); #ifndef INLINE_COPY_TO_USER unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n) { + unsigned long res; might_fault(); if (likely(access_ok(to, n))) { instrument_copy_to_user(to, from, n); - n = raw_copy_to_user(to, from, n); + res = raw_copy_to_user(to, from, n); + kmsan_copy_to_user(to, from, n, res); } - return n; + return res; } EXPORT_SYMBOL(_copy_to_user); #endif