From patchwork Thu Sep 15 15:03:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12977521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 001A9C6FA8D for ; Thu, 15 Sep 2022 15:04:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8244480008; Thu, 15 Sep 2022 11:04:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7ACA96B0075; Thu, 15 Sep 2022 11:04:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 672CD80008; Thu, 15 Sep 2022 11:04:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 59C8B6B0074 for ; Thu, 15 Sep 2022 11:04:48 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3DAFF808CD for ; Thu, 15 Sep 2022 15:04:48 +0000 (UTC) X-FDA: 79914641856.22.045B4E7 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) by imf14.hostedemail.com (Postfix) with ESMTP id B9D421000B6 for ; Thu, 15 Sep 2022 15:04:47 +0000 (UTC) Received: by mail-ej1-f74.google.com with SMTP id du20-20020a17090772d400b0077df99c5ab3so5451084ejc.3 for ; Thu, 15 Sep 2022 08:04:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date; bh=msgzHzsxvMZcLqG5jRq1WtDuSKgOL7FjSU3mDqU1uyE=; b=OJ1zZnEDisYf20/w2ufGsvf/4BmFJL6F+ZhkDMO9RSLgOWaUeOqgIZIEGBcs1xvjyE Vhi8oq1M5fuOo1BqRow0T2kkjWn7MFIiDl4Vxh0JvAvzMUcoBFNGV5OjD/1YNXh8Cb3e yBOZYjRifZkg/DWICVkMAwQ0rJNw34aIbPzK0IaH5U++++uKm/Nb/YWU7DG9jJPbW4je c+0D9D8GEACiFbk4kbys72G+E4/BBC6t7mtRJi/G2Ikv3nyuJnt9d8HjxthckZDQoevO /FUhBUjORIPB01uQTITnhxROg/kQNGDd3OFt2E1ajNcLp0VYOYpj5PP8IggJI7JbIss8 fOrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date; bh=msgzHzsxvMZcLqG5jRq1WtDuSKgOL7FjSU3mDqU1uyE=; b=6oGUZ8iQPN/Cb4l4YkgUnFmdgDMnkZ6MMXvZ92Pl8CqZeuF7WNygGMRm269QfTSkZU IKxA5vvY3pUnEeK0lAJ1TKHSNztysTn5I7PahlJtrxOP93dAn+MzvDRnZPQxkUaj70Iy x9wiF0yMAfywEcu2z0t/rqb0LwW3mT5zz9RzGg5Rfa+AxIxFsAyJEQ+7ubIDcTWLOozu bNpPwRWuwTWAU38ceC4/tl+SwGuFdz9fBDHYgZEM3FpcpUUgrxnhdZb2Nppf3rxa3gzj NX/rCCXiNgnRNOtBaPOhOljXprc2uF8XAnvJSJD7zxHQUNoaPwcIMdeuicvx7PVeD4Sn oAWA== X-Gm-Message-State: ACrzQf1/k83Huyo6NbMm1UEfZJNk1iQI7+/DL8+CPpt0azWku779SMME 40eUp63aBxMh9iyn2cI2yESCjZgvRhU= X-Google-Smtp-Source: AMsMyM5UV2lppQQoT85C2x0ZnCpLHOXrW/G23K2oT1zkSuuxsEEabT6dnZVkKB+qN2mRnFLrLNuGLySnQgE= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:686d:27b5:495:85b7]) (user=glider job=sendgmr) by 2002:a17:907:7612:b0:771:db66:7b77 with SMTP id jx18-20020a170907761200b00771db667b77mr309998ejc.228.1663254286414; Thu, 15 Sep 2022 08:04:46 -0700 (PDT) Date: Thu, 15 Sep 2022 17:03:37 +0200 In-Reply-To: <20220915150417.722975-1-glider@google.com> Mime-Version: 1.0 References: <20220915150417.722975-1-glider@google.com> X-Mailer: git-send-email 2.37.2.789.g6183377224-goog Message-ID: <20220915150417.722975-4-glider@google.com> Subject: [PATCH v7 03/43] instrumented.h: allow instrumenting both sides of copy_from_user() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Biggers , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Stephen Rothwell , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=OJ1zZnED; spf=pass (imf14.hostedemail.com: domain of 3Dj8jYwYKCDkbgdYZmbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3Dj8jYwYKCDkbgdYZmbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663254287; a=rsa-sha256; cv=none; b=iItUdq/f9jJJTh8GLtacwD91YKfvcvOeLivZcO79NTA2S9ZcQdCYv+rxWLA94rI6EkhHFk nsZ6LuELLefmgdXb+GEGUFHY2Jjstf8E6GtiiWrHueb5zLrAN/704Ym2nG3b8UmSqe5hpz zIyWJOyTHnHPz27orbt0mhWQABCbzHI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663254287; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=msgzHzsxvMZcLqG5jRq1WtDuSKgOL7FjSU3mDqU1uyE=; b=y8rJKI7iosuCkWoYM6N9DbtA9IuuFKJgc6Q205HdstHvrsR+DqMjAzD17ssa2vgo7z5kTX 2bQ9UQ/SexXpAnEgDObIcJkc9m9ITqLROkHp7COkPgATQU3Y9SzOMdRCBxO28xCvMmTT54 VAqavhwMKzPr9u5ZwhxpThdrZ+Mn6FM= X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B9D421000B6 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=OJ1zZnED; spf=pass (imf14.hostedemail.com: domain of 3Dj8jYwYKCDkbgdYZmbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3Dj8jYwYKCDkbgdYZmbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: pituixbaiiwtzu4ptdb7frhk4s7bdjuk X-HE-Tag: 1663254287-204521 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce instrument_copy_from_user_before() and instrument_copy_from_user_after() hooks to be invoked before and after the call to copy_from_user(). KASAN and KCSAN will be only using instrument_copy_from_user_before(), but for KMSAN we'll need to insert code after copy_from_user(). Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- v4: -- fix _copy_from_user_key() in arch/s390/lib/uaccess.c (Reported-by: kernel test robot ) Link: https://linux-review.googlesource.com/id/I855034578f0b0f126734cbd734fb4ae1d3a6af99 --- arch/s390/lib/uaccess.c | 3 ++- include/linux/instrumented.h | 21 +++++++++++++++++++-- include/linux/uaccess.h | 19 ++++++++++++++----- lib/iov_iter.c | 9 ++++++--- lib/usercopy.c | 3 ++- 5 files changed, 43 insertions(+), 12 deletions(-) diff --git a/arch/s390/lib/uaccess.c b/arch/s390/lib/uaccess.c index d7b3b193d1088..58033dfcb6d45 100644 --- a/arch/s390/lib/uaccess.c +++ b/arch/s390/lib/uaccess.c @@ -81,8 +81,9 @@ unsigned long _copy_from_user_key(void *to, const void __user *from, might_fault(); if (!should_fail_usercopy()) { - instrument_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); res = raw_copy_from_user_key(to, from, n, key); + instrument_copy_from_user_after(to, from, n, res); } if (unlikely(res)) memset(to + (n - res), 0, res); diff --git a/include/linux/instrumented.h b/include/linux/instrumented.h index 42faebbaa202a..ee8f7d17d34f5 100644 --- a/include/linux/instrumented.h +++ b/include/linux/instrumented.h @@ -120,7 +120,7 @@ instrument_copy_to_user(void __user *to, const void *from, unsigned long n) } /** - * instrument_copy_from_user - instrument writes of copy_from_user + * instrument_copy_from_user_before - add instrumentation before copy_from_user * * Instrument writes to kernel memory, that are due to copy_from_user (and * variants). The instrumentation should be inserted before the accesses. @@ -130,10 +130,27 @@ instrument_copy_to_user(void __user *to, const void *from, unsigned long n) * @n number of bytes to copy */ static __always_inline void -instrument_copy_from_user(const void *to, const void __user *from, unsigned long n) +instrument_copy_from_user_before(const void *to, const void __user *from, unsigned long n) { kasan_check_write(to, n); kcsan_check_write(to, n); } +/** + * instrument_copy_from_user_after - add instrumentation after copy_from_user + * + * Instrument writes to kernel memory, that are due to copy_from_user (and + * variants). The instrumentation should be inserted after the accesses. + * + * @to destination address + * @from source address + * @n number of bytes to copy + * @left number of bytes not copied (as returned by copy_from_user) + */ +static __always_inline void +instrument_copy_from_user_after(const void *to, const void __user *from, + unsigned long n, unsigned long left) +{ +} + #endif /* _LINUX_INSTRUMENTED_H */ diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index 47e5d374c7ebe..afb18f198843b 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -58,20 +58,28 @@ static __always_inline __must_check unsigned long __copy_from_user_inatomic(void *to, const void __user *from, unsigned long n) { - instrument_copy_from_user(to, from, n); + unsigned long res; + + instrument_copy_from_user_before(to, from, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); + return res; } static __always_inline __must_check unsigned long __copy_from_user(void *to, const void __user *from, unsigned long n) { + unsigned long res; + might_fault(); + instrument_copy_from_user_before(to, from, n); if (should_fail_usercopy()) return n; - instrument_copy_from_user(to, from, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); + return res; } /** @@ -115,8 +123,9 @@ _copy_from_user(void *to, const void __user *from, unsigned long n) unsigned long res = n; might_fault(); if (!should_fail_usercopy() && likely(access_ok(from, n))) { - instrument_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); } if (unlikely(res)) memset(to + (n - res), 0, res); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 4b7fce72e3e52..c3ca28ca68a65 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -174,13 +174,16 @@ static int copyout(void __user *to, const void *from, size_t n) static int copyin(void *to, const void __user *from, size_t n) { + size_t res = n; + if (should_fail_usercopy()) return n; if (access_ok(from, n)) { - instrument_copy_from_user(to, from, n); - n = raw_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); + res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); } - return n; + return res; } static inline struct pipe_buffer *pipe_buf(const struct pipe_inode_info *pipe, diff --git a/lib/usercopy.c b/lib/usercopy.c index 7413dd300516e..1505a52f23a01 100644 --- a/lib/usercopy.c +++ b/lib/usercopy.c @@ -12,8 +12,9 @@ unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n unsigned long res = n; might_fault(); if (!should_fail_usercopy() && likely(access_ok(from, n))) { - instrument_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); } if (unlikely(res)) memset(to + (n - res), 0, res);