From patchwork Wed Mar 20 10:18:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13597691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80364C54E67 for ; Wed, 20 Mar 2024 10:19:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 174706B0089; Wed, 20 Mar 2024 06:19:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 127856B009C; Wed, 20 Mar 2024 06:19:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F2FEF6B009E; Wed, 20 Mar 2024 06:19:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DE2EC6B009C for ; Wed, 20 Mar 2024 06:19:17 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B93F3A10B9 for ; Wed, 20 Mar 2024 10:19:17 +0000 (UTC) X-FDA: 81917019954.05.1AD8561 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf27.hostedemail.com (Postfix) with ESMTP id 03CED40009 for ; Wed, 20 Mar 2024 10:19:15 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=EppSC5Qi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 3I7j6ZQYKCLQafcXYlaiiafY.Wigfchor-ggepUWe.ila@flex--glider.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3I7j6ZQYKCLQafcXYlaiiafY.Wigfchor-ggepUWe.ila@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710929956; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r71MCuDGNYKDimxiIJKs3mOhEvECUDV+QG0xkV6mxh0=; b=sDydRsByx2rodspwuD7y2ZRKyaMahug0bAan5jIDegtKzS5Uq1OvIBzG2tDI8Hqt9B5ZKr knTonX69NtpIJ42pa19GBdZw1TcPIHVcfbQjFJDwNNaCZ5PNHroVIopZdyaaCjdC2ltzKA JZjerN8K7pE/nVarJbHk9Y0cdH3c35I= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=EppSC5Qi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 3I7j6ZQYKCLQafcXYlaiiafY.Wigfchor-ggepUWe.ila@flex--glider.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3I7j6ZQYKCLQafcXYlaiiafY.Wigfchor-ggepUWe.ila@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710929956; a=rsa-sha256; cv=none; b=Vr2RT4EaTon3fIZsbroGSbCsCuuiqB7LJVqazNt3LSYjjd2XXMQebll6Y5OlSZCXT0SdSi a1Rg7yxlWBmqEtsrOzvE7otWHds9t8P60AI0xI6eaKr6xoB/cHeTlB/XGwuvhRdC4vbJup A0l5ENJ/dHhyuefYcKWJR4kGryDevSQ= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dcbee93a3e1so10176655276.3 for ; Wed, 20 Mar 2024 03:19:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710929955; x=1711534755; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=r71MCuDGNYKDimxiIJKs3mOhEvECUDV+QG0xkV6mxh0=; b=EppSC5QiwL2zANAfq0L3jiGx1/50BgLEZAiwDPMYPORUMhmGDIVrGWqZDQRbiLodmG UIBFjph4OFXG0PF+Pso+YzuZC4bHkdJ9IVSSmuLEmAjVs7UcDiZtVFPs69A8lM/u+NXy TV/fJKRpQjsvDd2PFJqkATvKuc3g1URJct8y7oVYjIv4FB86U2TuCx4UxVNTiT6q640P l36TEVPMIkGtSDc8uRrehgm8EIXDyfoL5YOgtv7leURL/3urwRKt8F6G9hqybSVnRmO9 9Hb85+6DxuC5vMeFwg7vzpo2aO+kt6BwLzVtubcO1UIg5yWTYTFd/bc03MjdeC1dbw4v dZMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710929955; x=1711534755; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=r71MCuDGNYKDimxiIJKs3mOhEvECUDV+QG0xkV6mxh0=; b=GE8Vx0eCLZRCmmeAt2U4FFhglGwszBhPkMTFhVKR0SIQO7RXX0DjKLUXeYE1bIM2NP 5MY5yePt1UfluT66J5Xu+K9kHFUKz7hdPXnUtCae+F0CaRt8IBvjZhf6ZzJifI2nactH pXKEcRX61q/5UpQwxyBnGL/63ir7WLU7vrKd9SLjA2m0I6uchK1eRtE3W6wgcA6e7ybc +CP6SjYIStnwvVSJ0uKS+yEYtHICNt22wXci+QyGUWXb+y5xPQUCxdTqcetIiRHWm19F D7WpE3kyimZ3LX0DxSkV9G6x+2sgiwDSPBf9Vxur3zZ9phtHrQdJ8cyntu7og7pt2+Fq 7Blw== X-Forwarded-Encrypted: i=1; AJvYcCU9OWnZrdD5vcjvTAoVkYr3YmBABZP4delj1e8mfP4O5fd9v2/7Jnv6UUtDjuAK3yUadT3nOLLdrt5jz417wyB/JBM= X-Gm-Message-State: AOJu0Yx+STkBT093Ch6jMjdiG4lq9gEqw0gLyg4RLELIS4NR0gHVWMAd 5LfrN/9R0SCZw1CLI35gXY5teO9vfsfrsWwLxCQz0Vn6qC1+acyaWutAuCgaNku5W7G0of05vWN E0Q== X-Google-Smtp-Source: AGHT+IGiFjO7C+gTehD6m6WxUP0v54ngFYmvEURPJrqbwSTJrYhYcYDkQLRaGgYDZMxUmBfW4tcHUqyzYLs= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:2234:4e4b:bcf0:406e]) (user=glider job=sendgmr) by 2002:a05:6902:1b85:b0:dc2:2ace:860 with SMTP id ei5-20020a0569021b8500b00dc22ace0860mr915079ybb.2.1710929955165; Wed, 20 Mar 2024 03:19:15 -0700 (PDT) Date: Wed, 20 Mar 2024 11:18:51 +0100 In-Reply-To: <20240320101851.2589698-1-glider@google.com> Mime-Version: 1.0 References: <20240320101851.2589698-1-glider@google.com> X-Mailer: git-send-email 2.44.0.291.gc1ea87d7ee-goog Message-ID: <20240320101851.2589698-3-glider@google.com> Subject: [PATCH v2 3/3] x86: call instrumentation hooks from copy_mc.c From: Alexander Potapenko To: glider@google.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, tglx@linutronix.de, x86@kernel.org, Linus Torvalds , Dmitry Vyukov , Marco Elver , Tetsuo Handa X-Rspamd-Queue-Id: 03CED40009 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: m5z9uxjrwegbeqx9o31xe3mdrjm1ec6b X-HE-Tag: 1710929955-188595 X-HE-Meta: U2FsdGVkX1/BjZyX3wCJgD6d86EwZOtin4kBZF/9ZcZAzOGPkceZCipAtllyFBwArAMR5OVaS4OznoyKq6eJ8DfQUa+LKornmE8yxCcGgUMn8xbRbe7EyxX2y2NFsj776o+CBYedV7qoqZX29UWbN7hnBJi1ktI/o2rV+qr8VTPotGFzpdVYwJqf2pOyS3ehc3tx4hgy8CLeZzFEXIZIEt59DIZE4Cv1puu0w4Yq5M9xeAICBvmZSc+kqU342MrsZYVkSilkamhhNl8FWZE5eokYRxVElR1x8uB8K2CDJktG/Sf4zCqSGO3iehCBvLi8QFvOdVMeV4wxLzCjorLTCbGHLghSQZeXMd4rB3BsBe0sKXRTiWjhY8m3WRP8tdDpziulwROZ1T4laFwbMyWHKLUtWLKvY+drAYzEEOvseJYSL79PEE6C1eeGjP/lALc8P/KzS9Cw7CZx7DvQdB28ZbS28e4kgVtMy/nM1eZ3GuM1bDxd9pMpd5vmht+/Ny4h4mAK/tHYj+0UdbS0W060cGfKA0QTCNlFbrxRy+dn/rc/iD29i/vqdOVJ8AK8W5QuiGNtIGPXTRRqE49Q4StFW2d4qpcuVMpQrVQxxxjaXKXdPfcNPUT0j7g7zMgyByDLsd6L6OJ3M3rQMMhns9pTBt1U05o1ToO/j2HBNVxFwE4urJaYgo4FvpAQJlA3GXiR7i/B+iG6MKisvmiIjW7ZUrJMC0swei47/GRTWjXSSvx0HOzMiSj4qZ2r8R6LBlDnvuyRjf010/qzjATHx3idblrq7iNP5BwCe8pjjr86ONn1edfDegzQpE+C618ww/k0PZ0NHGQCkU6+/2TSC8pte9M241dg/N+qsETnPqWNEW2kR6rOmjH+GDMEagBPkB+/aqd3BCzit4ZFTywLJxqTbfzhtW7wIjiOfk6kCGg88aUHtqaaZpZ0MeleRJd85OG7jkqmXF8KdgRksTfGy9a E5MN1LCi 36rcMZyeoWBdwedPszByQOcyDTiYkKntrQRZX7KmCchFLxRC/cfimS4aodlYSCieImDEtn+A4uKfu/rGVHL/VJtpm8VYL40N13rAa45ugDzR1RPvx+Z7x0iwxLU4KbzKg2VUfbxGDQRlkl4TBtxxV/U4YyXwdq/qrEBrnolkA8iJsOigINf93mFubKlGlWmb3ky7rNlAgEA6kIMKVZhMZ+e5VtElKooHLsEhtdHWIcXSm00SfYw/42GKbmr/zQXjKCBeVOkPDXHV9spyQoJt1RvtcNiOrAD5GnIRr41N/qZGqVuxTf+eifrOStwF52c5hMnnEkI+78iuufBkGcpbqqiSrGzBHBNkwN9MyyI+rvYW5W7fYEdbVccsbv6yGd5rwMgoFUhjOxxzDTXm0pk02T46MhnQhm24HjDVmjUboyEXBoV2FqPNHY+QmOvWA+0XFOeh6Q06+hDgtGZadAnvZ/4xKnTU31H/4YZsoj7IgWXaCHTdV4AkRN2KRrnsuip3cDtxvAvdStBeM3YzgKhEeOLnsVZAFU9NEM/Y/NSN+ZgmWJigKNl1c26tNrOtobsVxWgCtZd+svmgRZVwV4bcEFLpFfEKOqoBYx8HbWyUqFTZe9pgmVyQMkI3RKTayBz7WeUXbiZIjHLLF12KUQkqE9LTbCPcDE0/Att8v0FVVTUlPyqQg+Q+mMrbh5NgdtN7SNPLrPf3zMMMc4n2hMkwjZnz6ao6isgcFVtjVcVud7v0D1DYRfff+JgoP5pql8pPgFYOwv4MHPyK6m2RQ8vc2vYqAMA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Memory accesses in copy_mc_to_kernel() and copy_mc_to_user() are performed by assembly routines and are invisible to KASAN, KCSAN, and KMSAN. Add hooks from instrumentation.h to tell the tools these functions have memcpy/copy_from_user semantics. The call to copy_mc_fragile() in copy_mc_fragile_handle_tail() is left intact, because the latter is only called from the assembly implementation of copy_mc_fragile(), so the memory accesses in it are covered by the instrumentation in copy_mc_to_kernel() and copy_mc_to_user(). Link: https://lore.kernel.org/all/3b7dbd88-0861-4638-b2d2-911c97a4cadf@I-love.SAKURA.ne.jp/ Suggested-by: Linus Torvalds Signed-off-by: Alexander Potapenko Cc: Dmitry Vyukov Cc: Marco Elver Cc: Tetsuo Handa Reviewed-by: Marco Elver --- v2: - as requested by Linus Torvalds, move the instrumentation outside the uaccess section --- arch/x86/lib/copy_mc.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/arch/x86/lib/copy_mc.c b/arch/x86/lib/copy_mc.c index 6e8b7e600def5..97e88e58567bf 100644 --- a/arch/x86/lib/copy_mc.c +++ b/arch/x86/lib/copy_mc.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include @@ -61,10 +62,20 @@ unsigned long copy_mc_enhanced_fast_string(void *dst, const void *src, unsigned */ unsigned long __must_check copy_mc_to_kernel(void *dst, const void *src, unsigned len) { - if (copy_mc_fragile_enabled) - return copy_mc_fragile(dst, src, len); - if (static_cpu_has(X86_FEATURE_ERMS)) - return copy_mc_enhanced_fast_string(dst, src, len); + unsigned long ret; + + if (copy_mc_fragile_enabled) { + instrument_memcpy_before(dst, src, len); + ret = copy_mc_fragile(dst, src, len); + instrument_memcpy_after(dst, src, len, ret); + return ret; + } + if (static_cpu_has(X86_FEATURE_ERMS)) { + instrument_memcpy_before(dst, src, len); + ret = copy_mc_enhanced_fast_string(dst, src, len); + instrument_memcpy_after(dst, src, len, ret); + return ret; + } memcpy(dst, src, len); return 0; } @@ -75,6 +86,7 @@ unsigned long __must_check copy_mc_to_user(void __user *dst, const void *src, un unsigned long ret; if (copy_mc_fragile_enabled) { + instrument_copy_to_user(dst, src, len); __uaccess_begin(); ret = copy_mc_fragile((__force void *)dst, src, len); __uaccess_end(); @@ -82,6 +94,7 @@ unsigned long __must_check copy_mc_to_user(void __user *dst, const void *src, un } if (static_cpu_has(X86_FEATURE_ERMS)) { + instrument_copy_to_user(dst, src, len); __uaccess_begin(); ret = copy_mc_enhanced_fast_string((__force void *)dst, src, len); __uaccess_end();