From patchwork Wed Mar 20 10:18:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13597689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E245C54E58 for ; Wed, 20 Mar 2024 10:19:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9159C6B0096; Wed, 20 Mar 2024 06:18:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C5EB6B0099; Wed, 20 Mar 2024 06:18:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B5C76B009A; Wed, 20 Mar 2024 06:18:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6976A6B0096 for ; Wed, 20 Mar 2024 06:18:59 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3B07AA1237 for ; Wed, 20 Mar 2024 10:18:59 +0000 (UTC) X-FDA: 81917019198.05.4687B32 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf17.hostedemail.com (Postfix) with ESMTP id 6856E40002 for ; Wed, 20 Mar 2024 10:18:57 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wg7YKLea; spf=pass (imf17.hostedemail.com: domain of 3D7j6ZQYKCKAGLIDERGOOGLE.COMLINUX-MMKVACK.ORG@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3D7j6ZQYKCKAGLIDERGOOGLE.COMLINUX-MMKVACK.ORG@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710929937; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Pe97FWHITdYkiCtSSFDy2YJv9pSAbWKeZZSBy5Rk1y0=; b=WKhUNJEKx79drdmIdZ1NigzoKJPgQ7SGe1rN9IuDmHQ05AMUqtpO3ajlJElM+DVeBzo7nV CivjfAN6bhbdfskl0i48fWXrPtTielqt1SQeV6dTRha+GAM13wErRklyMFpphDxe41LfOH 6gZ9G2kd4Qyp/5FbAPkmr6KaHr3oWY4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wg7YKLea; spf=pass (imf17.hostedemail.com: domain of 3D7j6ZQYKCKAGLIDERGOOGLE.COMLINUX-MMKVACK.ORG@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3D7j6ZQYKCKAGLIDERGOOGLE.COMLINUX-MMKVACK.ORG@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710929937; a=rsa-sha256; cv=none; b=DMsC5sl7DAKRBf4CC3TZh0aAQmjiw5XJp+V6E8ynraFOWCM3zccyn6pt+A0IR9iRPJ1wNR nWxTAeGOG3OGcOZ3kBrwiGxDC88EU07uyyRtjXLEiNfAPVj3tJnK9P2cGFkvM1iZqZvtPD 3+KigwD5VJaOFCMLI+y75+ldysYaWBY= Received: by mail-ej1-f73.google.com with SMTP id a640c23a62f3a-a46fd54fbc1so5139166b.3 for ; Wed, 20 Mar 2024 03:18:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710929936; x=1711534736; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=Pe97FWHITdYkiCtSSFDy2YJv9pSAbWKeZZSBy5Rk1y0=; b=wg7YKLeaqetyGcGIbwtpq3NLbdXrgwTTl6LR/eLhMkElOIdVF+08jxlAjX/3LhTeV2 +2qDYhdCPAc4PUt7NQ2Qb5pCdz9ZcRT1q4vsWN89LRSOFLGDkC/cy5NSrxWdCpnLra62 ovrfS5IkEj9GtJWDkboaaaGslN5idugk0p32SxDrT2idrJuXsdVLbQwSl2IePDjRRnzi KmsQDS//4MzbYFukScFclAY1IrshWdaHD+5DVnLoovBodqhJwLFMfZJnknwkmfXVtfHn cMYrcQIlv8V+/DVSaLFogok5DYJMIVXZljXKFp3OlBz2owEOvvZMKwWjMioYUK4dkyg2 oCTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710929936; x=1711534736; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Pe97FWHITdYkiCtSSFDy2YJv9pSAbWKeZZSBy5Rk1y0=; b=QjPFqtQnMds1BApQ6nSogVoOZ28O1yEbtI/lVbTED+U+JPHUgSs/Xv+hxE4dDAv3gB sPlX77jxYxBMjXK2VlVbxrmO0wpIjptEUELB+TOD4XXvrqG6q8z0z4zjbr1mOEjeKt2Y rFtovi/gmSizh1iZEAAUBaVotthyEbDEw9M6GuNkBm0tU0FfQG0O/GbzaDW2Mi+nmm3c 6jlFNXWWGyigaVVKB26GOpuPyA2Wk/1nheVRw+HRYeOQ8r1CxwGpDbQ5ewl+Bs+KicrQ viYGtvk6+mYvRlmlQcTtYIHdFmRTzZV33tDstssl0HAsuoql7DWBPVJ8tFMZQ8SgoNaZ 55oA== X-Forwarded-Encrypted: i=1; AJvYcCUga9XjAQnHmevGdvJybB/xz0vbj4YGaSHWNCRag9OlFVEPG/sp/Jctkz3W3AHDpaEdm/dvFzdnYZVn7yMirMBg3Xk= X-Gm-Message-State: AOJu0Yx9cWSY6CMfQXcNc4RFKhTYkrCRAQyliysofke+tfbzrzsRcs/7 qxKu8W9p+sMa8cmvaT6ybOuYl/IgLq4c0pKswJWzYOqzqjAfc0g0mcHbXQbItPOX0NyTittTMug qlA== X-Google-Smtp-Source: AGHT+IFgq13bXMs5IFwZBCUn9d8o9CaVR4UgE59VD0DZm2aj6+589s0p0zY44eyaCf7x/WiMresCTSv8JYs= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:2234:4e4b:bcf0:406e]) (user=glider job=sendgmr) by 2002:a05:6402:3893:b0:568:a515:30e8 with SMTP id fd19-20020a056402389300b00568a51530e8mr77979edb.0.1710929935435; Wed, 20 Mar 2024 03:18:55 -0700 (PDT) Date: Wed, 20 Mar 2024 11:18:49 +0100 Mime-Version: 1.0 X-Mailer: git-send-email 2.44.0.291.gc1ea87d7ee-goog Message-ID: <20240320101851.2589698-1-glider@google.com> Subject: [PATCH v2 1/3] mm: kmsan: implement kmsan_memmove() From: Alexander Potapenko To: glider@google.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, tglx@linutronix.de, x86@kernel.org, Tetsuo Handa , Dmitry Vyukov , Marco Elver , Linus Torvalds X-Rspamd-Queue-Id: 6856E40002 X-Rspam-User: X-Stat-Signature: 7oy19o53971z9cyt75pg8p4o1dbnj3m9 X-Rspamd-Server: rspam01 X-HE-Tag: 1710929937-493581 X-HE-Meta: U2FsdGVkX19woer35RTw/Hp1F0iKtlIUhcPTaYYcIa3X4Wm/zi2rhNywPz79CCqLnCyxSCFTvpn8u0rL11dXyrs8zqPsSmEZ6LaT4427mXLAWvVfwY5wrfeFXpuPVvEOPyBafXGpm7Fc0TZFl2bFQ9PXSPX1KrxTGo8bzhd4ZaijghCnPweMbjO1ViipdHf0Qf12uI9kU0saogZFLQ0Fza+wYaBs2vi+UdoQSJn8oYZ49XGyjSst3yleJPDscY4RlX9axQRQA1LlbVQ+o9QvL20cFH/NDGGBYOsIn/AGwL76GDLa96qeLNgToMGdOewbDXJc5trCb1eQmFYiSd+DiYihIFeCTUGFQZwr3AhEHu/JY3flF5dawJqss5ihTHpp3yODUAH8UXuUKt5OMYLPb/zLjBYvaxVN2r3aRl15KlQdDSYrORUjPuGiGU3Fsi2renFKztfcet+G0Jez1qAWE5KlBoCH7N+TmH06OT/zAPexkzXEPf/4OfLHb+TSNSDDxKpbkFVPC3UJkjqekzbSeL5Zx50LDiBwLaKa5cfIrL9eSsql8vOCT+w34x7oCpSPTC/0fjrJHMkyFhTpcOQf/log1Iiq28DbV/oUQKpa6LetWHLt1HKYRzGq1+CreR5pNgdqBH8WQH4v1yF+M1XpHrDAfl+m49e8xaNnBvxoXwgtG9n/cbYBBZs0+s+SlHCDebGGZsmMB7QqvXZ31WiO3hTIo/Ovw+gEXn1+z3OMVOQWYgeNOGNQUj2ld4HyaLiYfwqSUsQT/oHeTNlLSKGpdjeVeMnr3I9tSKiXwDnr/YvOf36bEORGX9n+OCnzGLPFTUc9V0kS0HW7TodYJ/ctVLKM19N9DisvoGbEyn3JEnbGj2NCF59kwtU8gKxRKsZsfNZVxXjMlELFwQkvfITQcGMqvchQVe7NTIKEIsVtJdOjJnL4pnDKbjhjZpsq9vPr8Uoja1aG7uj3g9Ib+cR YsC+O1s+ 5KPglkUIc/HqQJa9hzL7WBcjE4oqSlEfoJ6MNPXLF7PWS6x3V9pnonz6rGoERMQxgsd3A52tn/9BN/8BMTKvRWsvAwmJK0RhDce3Q2AbRSXnnTGnLGxHLPsVaP9jI+3hyPt57MxW2fldqBttnS98m69txtxLZDpfwLSxCio8biI0wvPiV62xEbtAlo0fcrh1VwXS12UzV/YwnkuLDTGKAGWQikeX22i1wAmkWMsR7EWjfT62pmJb4dD/9d1F9y2QiLV/MqNyGcpKKJDR1VMPZtpf3ByKtFa6XnKsDA3XBgV5ygpOYDkojDJVak4cEodaua083Tgv6aCO56JgXCI580gjX7V7dZ0eB9t8BPt91uhGjGF0vH4DYlj8oe+BQUs8beM6B/b5VeD3BB4S9x5x8O9pVEwynDRDjTmLX+hQU1OXR5w7ULoAViz6fBOcGtuW5ldTSUOzojp+/ur3ZUy4FcDSDkv0oaH+bbWOro3wmEeFI8eJnLAUC51qroRFQ9jVWIRAL0B+HpzeMXhNEQZzG1Q7cdcy7PFVV9ssUs8fUuHIuULQW/KvDlSFe4fOLjSzRrvrkPAVnYXH44B4qZYdDTgvXn4OLd39Bxui+wzJCPoYCA3TbIkImWM40oN6SPk1gN5OClGI1+ZqHm+lO0tjRyX85zkwLLBJizbEnjOYGzdDXyNYHBtBra23RLw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Provide a hook that can be used by custom memcpy implementations to tell KMSAN that the metadata needs to be copied. Without that, false positive reports are possible in the cases where KMSAN fails to intercept memory initialization. Link: https://lore.kernel.org/all/3b7dbd88-0861-4638-b2d2-911c97a4cadf@I-love.SAKURA.ne.jp/ Suggested-by: Tetsuo Handa Signed-off-by: Alexander Potapenko Cc: Dmitry Vyukov Cc: Marco Elver Cc: Linus Torvalds Reviewed-by: Marco Elver --- include/linux/kmsan-checks.h | 15 +++++++++++++++ mm/kmsan/hooks.c | 11 +++++++++++ 2 files changed, 26 insertions(+) diff --git a/include/linux/kmsan-checks.h b/include/linux/kmsan-checks.h index c4cae333deec5..e1082dc40abc2 100644 --- a/include/linux/kmsan-checks.h +++ b/include/linux/kmsan-checks.h @@ -61,6 +61,17 @@ void kmsan_check_memory(const void *address, size_t size); void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy, size_t left); +/** + * kmsan_memmove() - Notify KMSAN about a data copy within kernel. + * @to: destination address in the kernel. + * @from: source address in the kernel. + * @size: number of bytes to copy. + * + * Invoked after non-instrumented version (e.g. implemented using assembly + * code) of memmove()/memcpy() is called, in order to copy KMSAN's metadata. + */ +void kmsan_memmove(void *to, const void *from, size_t to_copy); + #else static inline void kmsan_poison_memory(const void *address, size_t size, @@ -78,6 +89,10 @@ static inline void kmsan_copy_to_user(void __user *to, const void *from, { } +static inline void kmsan_memmove(void *to, const void *from, size_t to_copy) +{ +} + #endif #endif /* _LINUX_KMSAN_CHECKS_H */ diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c index 5d6e2dee5692a..364f778ee226d 100644 --- a/mm/kmsan/hooks.c +++ b/mm/kmsan/hooks.c @@ -285,6 +285,17 @@ void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy, } EXPORT_SYMBOL(kmsan_copy_to_user); +void kmsan_memmove(void *to, const void *from, size_t size) +{ + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + kmsan_enter_runtime(); + kmsan_internal_memmove_metadata(to, (void *)from, size); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_memmove); + /* Helper function to check an URB. */ void kmsan_handle_urb(const struct urb *urb, bool is_out) { From patchwork Wed Mar 20 10:18:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13597690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 387E4C54E67 for ; Wed, 20 Mar 2024 10:19:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C72566B009A; Wed, 20 Mar 2024 06:19:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C21916B009B; Wed, 20 Mar 2024 06:19:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE9ED6B009C; Wed, 20 Mar 2024 06:19:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9EC1A6B009A for ; Wed, 20 Mar 2024 06:19:15 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 3D35D1A0BE8 for ; Wed, 20 Mar 2024 10:19:15 +0000 (UTC) X-FDA: 81917019870.21.4A0E19F Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf02.hostedemail.com (Postfix) with ESMTP id 534578001D for ; Wed, 20 Mar 2024 10:19:13 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=oawsCkoM; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3ILj6ZQYKCLEXcZUViXffXcV.TfdcZelo-ddbmRTb.fiX@flex--glider.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3ILj6ZQYKCLEXcZUViXffXcV.TfdcZelo-ddbmRTb.fiX@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710929953; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=m0vBJz04bRZVs+p5tf5O60PjHz2OHj3PY0354SOIqxk=; b=cdErzzZXA+W6DrjIemTylmOJ0jU0YsQ1sfWZxivsVD+nxrvzvteriXYrFDV1yMlCKFUsiY tNf0kLbe+WjUKZB//NB74fxldJBJPnGivAtqTom6y6+JpJffKjlNBwy1t15h+Kj7tu69EF NLsLAeiA2b3NuG2mcn7jVYOh3ZIRBuc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=oawsCkoM; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3ILj6ZQYKCLEXcZUViXffXcV.TfdcZelo-ddbmRTb.fiX@flex--glider.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3ILj6ZQYKCLEXcZUViXffXcV.TfdcZelo-ddbmRTb.fiX@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710929953; a=rsa-sha256; cv=none; b=XR/tKOU1vm6fPB8PvNdLRF7odwot7EWta2LEpR4Gf/b5A9c7c7SrSyN1MUQhlZn7pmN5qv SWuzqVWuMjPO0e1NEyEyG/qt6Gw+RxAsvt/IEzWkrfXHfb10lcF4EIbwJQdjnb7JeEBYf5 QATlLH0/kov6/zqJudpLNUpYEX2c9zQ= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dd169dd4183so7971202276.3 for ; Wed, 20 Mar 2024 03:19:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710929952; x=1711534752; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=m0vBJz04bRZVs+p5tf5O60PjHz2OHj3PY0354SOIqxk=; b=oawsCkoMD4Ysz+cMsrdKSSo+lic307ohSWrbHwSe67/t3MzCYQULIfBmBPs8/ZBQaV Om3l7MW+jDBy+ZGSA9Zt2Fqgf98sMlXsxEHYSO/Iyc+WbTcmcWLsio+G8ygIllB+3JlS 8A0k1+BfPlACZNwxKDHUzizZ1pHZjCDilPGi+6SKPV9uVYtsx9GR/h5eWFeyGr4K23vo QiZEdqINh0FQu09PAqVO9z75ijwNzu5snR3i1E4R9OdIVMA6HvxKZUTIyJZpTjUFuMd1 LZw79DL7ZqXFih72e2XsddYccHUxJV3MRoJl1L5fKSHuhSUrvALlWeTHx/8Gj4W1JEHI 0n1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710929952; x=1711534752; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=m0vBJz04bRZVs+p5tf5O60PjHz2OHj3PY0354SOIqxk=; b=PlVKysbi4grXIvcZfVoBACDXnMIq+WX3JhagZmc9yhF0ekfji8izDHb2ncSkGzSUrb EFVTUSTKC5h0tYOPLvcRkGYfYj/i5xSVXGnIqPz9+4bltstvRiltjRFnFknLTgYchjUl eAKInHCshvbuOMAOvT5tSf9dq2qm284ITE/Xf5iaMq0r5dgPOHoN7sSGbPzqjWJ+IQmC fIWfxoiMQZnx/0qsXHgJmaRCIwUZinwr3EUDoIJThHJt910xL/WR4XNPPh50G4TR9Hrl biNC3IRAvzI7t0O5Ksm4uGi0B21zSGTgGxtTv9v9PIThBLgufWq4Aau7P36Fhdgb0zgd x4sw== X-Forwarded-Encrypted: i=1; AJvYcCWhWppuI+Aw0rfRvzuxb0G9Vwivj3xI2qg9c20ll8NUCEME8DslmBMHgq3r0/B5p2DnCX+r3OlmnriaXvRkSSYNLJA= X-Gm-Message-State: AOJu0YyXjupZjPOqWy99T2yoXpAYgzANtXuK4N7irXGLjImhEDhr9og+ f4vtgkYuZ+V6BpiLkj7wG2HJ8N0pkEnTNDHNje333r8PwS9Ub1nVdseLzJSqB/uJCt1nRbwkYFJ Sgg== X-Google-Smtp-Source: AGHT+IFG8zJJFsZTjI1YIaNEwrks7akz88Yk5exPnSeBisyx550nfOfPmsTtY3RfUy1vhUp7VkINpS74JxE= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:2234:4e4b:bcf0:406e]) (user=glider job=sendgmr) by 2002:a05:6902:2484:b0:dd9:1b94:edb5 with SMTP id ds4-20020a056902248400b00dd91b94edb5mr625397ybb.10.1710929952333; Wed, 20 Mar 2024 03:19:12 -0700 (PDT) Date: Wed, 20 Mar 2024 11:18:50 +0100 In-Reply-To: <20240320101851.2589698-1-glider@google.com> Mime-Version: 1.0 References: <20240320101851.2589698-1-glider@google.com> X-Mailer: git-send-email 2.44.0.291.gc1ea87d7ee-goog Message-ID: <20240320101851.2589698-2-glider@google.com> Subject: [PATCH v2 2/3] instrumented.h: add instrument_memcpy_before, instrument_memcpy_after From: Alexander Potapenko To: glider@google.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, tglx@linutronix.de, x86@kernel.org, Dmitry Vyukov , Marco Elver , Tetsuo Handa , Linus Torvalds X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 534578001D X-Stat-Signature: x4i5gwpnsk7dre3eg59pjy536pgcpkk3 X-Rspam-User: X-HE-Tag: 1710929953-995289 X-HE-Meta: U2FsdGVkX1/K9aI2XhX340Mib/tFUZ3jj9ffSZmYHRYZJEVwvYb2bxlN6BkLy9lbuALGPjSgG6tR+qlBdu08OZtulry+ZMKmT4KeofsW2tY2BpRFksQaIMLy4DpE9yWstUvqZHsJul/GsawxYK9c/iHFH6ENaaxgMUiqoAzU7NK5z+5arY58nVAo2tJj8BAOeILV9Y+J2IL8p2wbHZvfXxK0HxX33Zy4BByq/4bPrjgexYvXaiEOEWRif3F5OMlrhghSD7sa4ADk8w+S/AzCh3DKISU3mKkCtAX0YGZpROLri9TgzDGMX/djA/98UbfuhXwv5OGfA/O7kClR9LXbE7j8gzeGKRXN2eBMcITLIAKGZqHhyZOmExcPi/g4wB/4mFOmy37oW7PUyxE/QwluVNcuVvyEx+f+sbFrKQlzeH5uRCar/H6cW4IsrlWT46C20waUwGgQDHmk8m7YWm4LjfYZz8PF++5gK/hOB6DIBBtgrZpifJcvo5VY+kYBYZhDpdJleScgJNVkgH2TkfNj7FtzW1BwXTGHA8mJL81WTKr+6TU8lHnqPR9oa//F7AIeY2p1VXU1X914oxUlyeow362+OdOcIzgqqtqlAtuSTlMsosNoM9m7aVO6SNp1Pkp0m/UvEHASEwZe2rfRzuJHm4S3r0AWR3ErPa3qW4wgSXy/PaHAmpSfCQp77Yvp0On4QjbCwNYZFBY2HLdz9/irZbq+gnbtyMH60kBL9C46qn24CDaIGPT4ZuXqy5NL7wOsBEWO4KvkyiOPpCAGcVFstTIq8NxJPA2Q1n6I5tOlwY7m5OB3J9MBGKjg0IYmBJHbZRCwu1rDit3MD7i+upMUjG8qBUMoWxQ0o2umwFeyEYqNldc34rXgaMhiYcy57vDalpHTtw+tc9D5I9uGChSXv5JUuxPc9GsOp4cfyHzIgeMyGmKCpP2+MALxNe73SM7X6GqCgLyoRjAgVCCrnF4 tIE7IpAE AAwQqe4w7/AbDEpl89pvtuDFwZ7dSEWfeF7X0FcReR1xNU6Y8+N1Fdeb00GiywP9FCu7a5EUzgiILR1Yrqx1renKwFTJBRdOZ2x9ktMYvW1Hjj96T05zc+3kPfyllfJ1IkSExytDSj68V7V3WGnsjuDcqFCKjcAZo4qZSVSqrRFlmG42hY8cWMALm5YfUlUVflN/kL8ushqS/f5STPyPuGzDkU7oBssWcHT7qQ/Od39m7zHjv/xN9ZLA4xmQ4QFY8YAheD6g0oqExny469soPolMw9Nn+UUhUdCHWYGDtVV2NF3QDgXruM87dTgb5MtP0Va6kOOmKxcqAQeU+0zATJBkUwl5fR71gO0ACy6ghM5aoHNwDUd6O5VwsTKtWcAhlgkt3gTzIfqCV8UITuYLHkJ1nfz8gKUKKL/7G88GWCOjTWfP6yZbkDlvXhRWQ7J5hjBjcN7jX8jYZg5hR+oNl5IwBQuVg4+vfn3p0qNJ1695oHiZoN8dc/5Z5gYcRVptImFaGKU48t2vxsp/RSIfguN/ZwWXISw2W8bpb1+jeyzGWy47ffYdbApQXIYAy8TCJ2iN/dNx57vrlzQmBvvbw7kVYVqrT5K1K93Co3uZ0FnJtfvmhfGVFOUz+MmYxGiNKV7KkPHCsZTZFHFH8Kq+DRvKTlP9tl2KPJBh8PCc1/dM+mL5jW1va7BYevEQEWKFyIQtsuui8MBEwk6f+yq0y37SKpIrO4xl630ItQJthO6e4q2qEs7WJF1aGRNB4Hqrrdv/eEYvblytwqRlnHxtvs386KA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Bug detection tools based on compiler instrumentation may miss memory accesses in custom memcpy implementations (such as copy_mc_to_kernel). Provide instrumentation hooks that tell KASAN, KCSAN, and KMSAN about such accesses. Link: https://lore.kernel.org/all/3b7dbd88-0861-4638-b2d2-911c97a4cadf@I-love.SAKURA.ne.jp/ Signed-off-by: Alexander Potapenko Cc: Dmitry Vyukov Cc: Marco Elver Cc: Tetsuo Handa Cc: Linus Torvalds Reviewed-by: Marco Elver --- v2: fix a copypasto in a comment spotted by Linus --- include/linux/instrumented.h | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/include/linux/instrumented.h b/include/linux/instrumented.h index 1b608e00290aa..711a1f0d1a735 100644 --- a/include/linux/instrumented.h +++ b/include/linux/instrumented.h @@ -147,6 +147,41 @@ instrument_copy_from_user_after(const void *to, const void __user *from, kmsan_unpoison_memory(to, n - left); } +/** + * instrument_memcpy_before - add instrumentation before non-instrumented memcpy + * @to: destination address + * @from: source address + * @n: number of bytes to copy + * + * Instrument memory accesses that happen in custom memcpy implementations. The + * instrumentation should be inserted before the memcpy call. + */ +static __always_inline void instrument_memcpy_before(void *to, const void *from, + unsigned long n) +{ + kasan_check_write(to, n); + kasan_check_read(from, n); + kcsan_check_write(to, n); + kcsan_check_read(from, n); +} + +/** + * instrument_memcpy_after - add instrumentation after non-instrumented memcpy + * @to: destination address + * @from: source address + * @n: number of bytes to copy + * @left: number of bytes not copied (if known) + * + * Instrument memory accesses that happen in custom memcpy implementations. The + * instrumentation should be inserted after the memcpy call. + */ +static __always_inline void instrument_memcpy_after(void *to, const void *from, + unsigned long n, + unsigned long left) +{ + kmsan_memmove(to, from, n - left); +} + /** * instrument_get_user() - add instrumentation to get_user()-like macros * @to: destination variable, may not be address-taken From patchwork Wed Mar 20 10:18:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13597691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80364C54E67 for ; Wed, 20 Mar 2024 10:19:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 174706B0089; Wed, 20 Mar 2024 06:19:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 127856B009C; Wed, 20 Mar 2024 06:19:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F2FEF6B009E; Wed, 20 Mar 2024 06:19:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DE2EC6B009C for ; Wed, 20 Mar 2024 06:19:17 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B93F3A10B9 for ; Wed, 20 Mar 2024 10:19:17 +0000 (UTC) X-FDA: 81917019954.05.1AD8561 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf27.hostedemail.com (Postfix) with ESMTP id 03CED40009 for ; Wed, 20 Mar 2024 10:19:15 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=EppSC5Qi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 3I7j6ZQYKCLQafcXYlaiiafY.Wigfchor-ggepUWe.ila@flex--glider.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3I7j6ZQYKCLQafcXYlaiiafY.Wigfchor-ggepUWe.ila@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710929956; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r71MCuDGNYKDimxiIJKs3mOhEvECUDV+QG0xkV6mxh0=; b=sDydRsByx2rodspwuD7y2ZRKyaMahug0bAan5jIDegtKzS5Uq1OvIBzG2tDI8Hqt9B5ZKr knTonX69NtpIJ42pa19GBdZw1TcPIHVcfbQjFJDwNNaCZ5PNHroVIopZdyaaCjdC2ltzKA JZjerN8K7pE/nVarJbHk9Y0cdH3c35I= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=EppSC5Qi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 3I7j6ZQYKCLQafcXYlaiiafY.Wigfchor-ggepUWe.ila@flex--glider.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3I7j6ZQYKCLQafcXYlaiiafY.Wigfchor-ggepUWe.ila@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710929956; a=rsa-sha256; cv=none; b=Vr2RT4EaTon3fIZsbroGSbCsCuuiqB7LJVqazNt3LSYjjd2XXMQebll6Y5OlSZCXT0SdSi a1Rg7yxlWBmqEtsrOzvE7otWHds9t8P60AI0xI6eaKr6xoB/cHeTlB/XGwuvhRdC4vbJup A0l5ENJ/dHhyuefYcKWJR4kGryDevSQ= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dcbee93a3e1so10176655276.3 for ; Wed, 20 Mar 2024 03:19:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710929955; x=1711534755; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=r71MCuDGNYKDimxiIJKs3mOhEvECUDV+QG0xkV6mxh0=; b=EppSC5QiwL2zANAfq0L3jiGx1/50BgLEZAiwDPMYPORUMhmGDIVrGWqZDQRbiLodmG UIBFjph4OFXG0PF+Pso+YzuZC4bHkdJ9IVSSmuLEmAjVs7UcDiZtVFPs69A8lM/u+NXy TV/fJKRpQjsvDd2PFJqkATvKuc3g1URJct8y7oVYjIv4FB86U2TuCx4UxVNTiT6q640P l36TEVPMIkGtSDc8uRrehgm8EIXDyfoL5YOgtv7leURL/3urwRKt8F6G9hqybSVnRmO9 9Hb85+6DxuC5vMeFwg7vzpo2aO+kt6BwLzVtubcO1UIg5yWTYTFd/bc03MjdeC1dbw4v dZMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710929955; x=1711534755; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=r71MCuDGNYKDimxiIJKs3mOhEvECUDV+QG0xkV6mxh0=; b=GE8Vx0eCLZRCmmeAt2U4FFhglGwszBhPkMTFhVKR0SIQO7RXX0DjKLUXeYE1bIM2NP 5MY5yePt1UfluT66J5Xu+K9kHFUKz7hdPXnUtCae+F0CaRt8IBvjZhf6ZzJifI2nactH pXKEcRX61q/5UpQwxyBnGL/63ir7WLU7vrKd9SLjA2m0I6uchK1eRtE3W6wgcA6e7ybc +CP6SjYIStnwvVSJ0uKS+yEYtHICNt22wXci+QyGUWXb+y5xPQUCxdTqcetIiRHWm19F D7WpE3kyimZ3LX0DxSkV9G6x+2sgiwDSPBf9Vxur3zZ9phtHrQdJ8cyntu7og7pt2+Fq 7Blw== X-Forwarded-Encrypted: i=1; AJvYcCU9OWnZrdD5vcjvTAoVkYr3YmBABZP4delj1e8mfP4O5fd9v2/7Jnv6UUtDjuAK3yUadT3nOLLdrt5jz417wyB/JBM= X-Gm-Message-State: AOJu0Yx+STkBT093Ch6jMjdiG4lq9gEqw0gLyg4RLELIS4NR0gHVWMAd 5LfrN/9R0SCZw1CLI35gXY5teO9vfsfrsWwLxCQz0Vn6qC1+acyaWutAuCgaNku5W7G0of05vWN E0Q== X-Google-Smtp-Source: AGHT+IGiFjO7C+gTehD6m6WxUP0v54ngFYmvEURPJrqbwSTJrYhYcYDkQLRaGgYDZMxUmBfW4tcHUqyzYLs= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:2234:4e4b:bcf0:406e]) (user=glider job=sendgmr) by 2002:a05:6902:1b85:b0:dc2:2ace:860 with SMTP id ei5-20020a0569021b8500b00dc22ace0860mr915079ybb.2.1710929955165; Wed, 20 Mar 2024 03:19:15 -0700 (PDT) Date: Wed, 20 Mar 2024 11:18:51 +0100 In-Reply-To: <20240320101851.2589698-1-glider@google.com> Mime-Version: 1.0 References: <20240320101851.2589698-1-glider@google.com> X-Mailer: git-send-email 2.44.0.291.gc1ea87d7ee-goog Message-ID: <20240320101851.2589698-3-glider@google.com> Subject: [PATCH v2 3/3] x86: call instrumentation hooks from copy_mc.c From: Alexander Potapenko To: glider@google.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, tglx@linutronix.de, x86@kernel.org, Linus Torvalds , Dmitry Vyukov , Marco Elver , Tetsuo Handa X-Rspamd-Queue-Id: 03CED40009 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: m5z9uxjrwegbeqx9o31xe3mdrjm1ec6b X-HE-Tag: 1710929955-188595 X-HE-Meta: U2FsdGVkX1/BjZyX3wCJgD6d86EwZOtin4kBZF/9ZcZAzOGPkceZCipAtllyFBwArAMR5OVaS4OznoyKq6eJ8DfQUa+LKornmE8yxCcGgUMn8xbRbe7EyxX2y2NFsj776o+CBYedV7qoqZX29UWbN7hnBJi1ktI/o2rV+qr8VTPotGFzpdVYwJqf2pOyS3ehc3tx4hgy8CLeZzFEXIZIEt59DIZE4Cv1puu0w4Yq5M9xeAICBvmZSc+kqU342MrsZYVkSilkamhhNl8FWZE5eokYRxVElR1x8uB8K2CDJktG/Sf4zCqSGO3iehCBvLi8QFvOdVMeV4wxLzCjorLTCbGHLghSQZeXMd4rB3BsBe0sKXRTiWjhY8m3WRP8tdDpziulwROZ1T4laFwbMyWHKLUtWLKvY+drAYzEEOvseJYSL79PEE6C1eeGjP/lALc8P/KzS9Cw7CZx7DvQdB28ZbS28e4kgVtMy/nM1eZ3GuM1bDxd9pMpd5vmht+/Ny4h4mAK/tHYj+0UdbS0W060cGfKA0QTCNlFbrxRy+dn/rc/iD29i/vqdOVJ8AK8W5QuiGNtIGPXTRRqE49Q4StFW2d4qpcuVMpQrVQxxxjaXKXdPfcNPUT0j7g7zMgyByDLsd6L6OJ3M3rQMMhns9pTBt1U05o1ToO/j2HBNVxFwE4urJaYgo4FvpAQJlA3GXiR7i/B+iG6MKisvmiIjW7ZUrJMC0swei47/GRTWjXSSvx0HOzMiSj4qZ2r8R6LBlDnvuyRjf010/qzjATHx3idblrq7iNP5BwCe8pjjr86ONn1edfDegzQpE+C618ww/k0PZ0NHGQCkU6+/2TSC8pte9M241dg/N+qsETnPqWNEW2kR6rOmjH+GDMEagBPkB+/aqd3BCzit4ZFTywLJxqTbfzhtW7wIjiOfk6kCGg88aUHtqaaZpZ0MeleRJd85OG7jkqmXF8KdgRksTfGy9a E5MN1LCi 36rcMZyeoWBdwedPszByQOcyDTiYkKntrQRZX7KmCchFLxRC/cfimS4aodlYSCieImDEtn+A4uKfu/rGVHL/VJtpm8VYL40N13rAa45ugDzR1RPvx+Z7x0iwxLU4KbzKg2VUfbxGDQRlkl4TBtxxV/U4YyXwdq/qrEBrnolkA8iJsOigINf93mFubKlGlWmb3ky7rNlAgEA6kIMKVZhMZ+e5VtElKooHLsEhtdHWIcXSm00SfYw/42GKbmr/zQXjKCBeVOkPDXHV9spyQoJt1RvtcNiOrAD5GnIRr41N/qZGqVuxTf+eifrOStwF52c5hMnnEkI+78iuufBkGcpbqqiSrGzBHBNkwN9MyyI+rvYW5W7fYEdbVccsbv6yGd5rwMgoFUhjOxxzDTXm0pk02T46MhnQhm24HjDVmjUboyEXBoV2FqPNHY+QmOvWA+0XFOeh6Q06+hDgtGZadAnvZ/4xKnTU31H/4YZsoj7IgWXaCHTdV4AkRN2KRrnsuip3cDtxvAvdStBeM3YzgKhEeOLnsVZAFU9NEM/Y/NSN+ZgmWJigKNl1c26tNrOtobsVxWgCtZd+svmgRZVwV4bcEFLpFfEKOqoBYx8HbWyUqFTZe9pgmVyQMkI3RKTayBz7WeUXbiZIjHLLF12KUQkqE9LTbCPcDE0/Att8v0FVVTUlPyqQg+Q+mMrbh5NgdtN7SNPLrPf3zMMMc4n2hMkwjZnz6ao6isgcFVtjVcVud7v0D1DYRfff+JgoP5pql8pPgFYOwv4MHPyK6m2RQ8vc2vYqAMA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Memory accesses in copy_mc_to_kernel() and copy_mc_to_user() are performed by assembly routines and are invisible to KASAN, KCSAN, and KMSAN. Add hooks from instrumentation.h to tell the tools these functions have memcpy/copy_from_user semantics. The call to copy_mc_fragile() in copy_mc_fragile_handle_tail() is left intact, because the latter is only called from the assembly implementation of copy_mc_fragile(), so the memory accesses in it are covered by the instrumentation in copy_mc_to_kernel() and copy_mc_to_user(). Link: https://lore.kernel.org/all/3b7dbd88-0861-4638-b2d2-911c97a4cadf@I-love.SAKURA.ne.jp/ Suggested-by: Linus Torvalds Signed-off-by: Alexander Potapenko Cc: Dmitry Vyukov Cc: Marco Elver Cc: Tetsuo Handa Reviewed-by: Marco Elver --- v2: - as requested by Linus Torvalds, move the instrumentation outside the uaccess section --- arch/x86/lib/copy_mc.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/arch/x86/lib/copy_mc.c b/arch/x86/lib/copy_mc.c index 6e8b7e600def5..97e88e58567bf 100644 --- a/arch/x86/lib/copy_mc.c +++ b/arch/x86/lib/copy_mc.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include @@ -61,10 +62,20 @@ unsigned long copy_mc_enhanced_fast_string(void *dst, const void *src, unsigned */ unsigned long __must_check copy_mc_to_kernel(void *dst, const void *src, unsigned len) { - if (copy_mc_fragile_enabled) - return copy_mc_fragile(dst, src, len); - if (static_cpu_has(X86_FEATURE_ERMS)) - return copy_mc_enhanced_fast_string(dst, src, len); + unsigned long ret; + + if (copy_mc_fragile_enabled) { + instrument_memcpy_before(dst, src, len); + ret = copy_mc_fragile(dst, src, len); + instrument_memcpy_after(dst, src, len, ret); + return ret; + } + if (static_cpu_has(X86_FEATURE_ERMS)) { + instrument_memcpy_before(dst, src, len); + ret = copy_mc_enhanced_fast_string(dst, src, len); + instrument_memcpy_after(dst, src, len, ret); + return ret; + } memcpy(dst, src, len); return 0; } @@ -75,6 +86,7 @@ unsigned long __must_check copy_mc_to_user(void __user *dst, const void *src, un unsigned long ret; if (copy_mc_fragile_enabled) { + instrument_copy_to_user(dst, src, len); __uaccess_begin(); ret = copy_mc_fragile((__force void *)dst, src, len); __uaccess_end(); @@ -82,6 +94,7 @@ unsigned long __must_check copy_mc_to_user(void __user *dst, const void *src, un } if (static_cpu_has(X86_FEATURE_ERMS)) { + instrument_copy_to_user(dst, src, len); __uaccess_begin(); ret = copy_mc_enhanced_fast_string((__force void *)dst, src, len); __uaccess_end();