From patchwork Tue Mar 19 16:36:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13596907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 982B6C6FD1F for ; Tue, 19 Mar 2024 16:37:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0ABD36B0089; Tue, 19 Mar 2024 12:37:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 05D6B6B008A; Tue, 19 Mar 2024 12:37:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3CC36B008C; Tue, 19 Mar 2024 12:37:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C81EE6B0089 for ; Tue, 19 Mar 2024 12:37:09 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7F1201C133D for ; Tue, 19 Mar 2024 16:37:09 +0000 (UTC) X-FDA: 81914343378.28.7A980A9 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf17.hostedemail.com (Postfix) with ESMTP id DC92F40015 for ; Tue, 19 Mar 2024 16:37:07 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=sRDt9zQT; spf=pass (imf17.hostedemail.com: domain of 3M7_5ZQYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3M7_5ZQYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710866227; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DNX90sT1lUbl5lD9B0qqgx+siiIHXGud0HrlrLB6kTU=; b=IlLFdBjj+N+FJhGx/g9fPKSjgy7yWJWo5MSP7/BXVqaitLdeBwWw4tvmO9fxnIFr1/56ti /SeTnu26JMGGO44GFvgliVJxH28MhTOHJvzGJi+eeR5U+Uu8v0KJMvdnwo/F7R+hyAvtG5 950QSYwpuXJlBT6osgFKyM+KZKedk8E= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710866227; a=rsa-sha256; cv=none; b=BrYZQmM3zpjdCzYQHVVOrGFU4IvO8m9m1kPaROew6ET5SzKcrIX5KFNgjUkZKoqK87Fe5V LPw3VJR+/vtE/mqzdMEr6PLdlbIrqvOg72+T3cqU+0859lesr4dLET2oF3FJohTEUmHTNi 7xCcqdVGKnQ833EyR9KlDUf0HZnMMo4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=sRDt9zQT; spf=pass (imf17.hostedemail.com: domain of 3M7_5ZQYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3M7_5ZQYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-60a605154d0so500037b3.0 for ; Tue, 19 Mar 2024 09:37:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710866227; x=1711471027; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DNX90sT1lUbl5lD9B0qqgx+siiIHXGud0HrlrLB6kTU=; b=sRDt9zQTs/sJXSMqb2SWs0nlXoQu8+VfLdmdqsaFRCJxCW2uZfCUYiXON41/mC2nUu kx/k/IaONU8qZ0c9CaLQnwa87gZEqXu2SvvyiLXttOSMsEBgGo9xAdyfxsMw0Wchxcpb 9cLRVyI/chwTxGcAOYc70UYQvur+jGDIYR4Q7pw/kr6b3cGBUN7D64RoYiJF6TCeqcR9 mjilWQRQMK2c6gOaZMM0qCK/xVp0ArrdmJV3j1jC/scGE+9B9lixFOywRQwiYI3haBg+ 3ID0fU670l47jn9iiwvbX59RQqzs6l8NyJiYrozZ9B6nDtGF4Qf3KkOhhqF3BLay4OvH wdlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710866227; x=1711471027; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DNX90sT1lUbl5lD9B0qqgx+siiIHXGud0HrlrLB6kTU=; b=gD25OgYJuQeJNtjy/vP7jSomL2GNlkU0WvQif6wmsvGY43oPXxfU0DoBiffwxedahW Jtsa3o23F2w+H232kdimvmjQHRrfCiJCoOUIdibXjzNNZoKx2H57XsVSV7vnKGAKz2uf xl+TNi144aMFcZv8ghBUUjiIO18cOft/oU2pYu1GmJdNzjwmjxblx7obnfCLBWYuAw/Q 1V1OMmqQm910ycP6h59bDis0ZTAt50/T2GA0aMw9S3fNmiWk9n2OXGceR2NNJlz95t7m zBXdCQGCb5yFBfKzZ8kp8S1t3mOvyqHN2+2BhDxOAqsLBax1Xg+kTGsJYglFtOcRoKq9 EStg== X-Forwarded-Encrypted: i=1; AJvYcCXZYkW4Hl/vJE3Xq6MMc0qTdZ72YBHeZnCtUxRyW2DPOJzfm9+4ZWtlOgHCr/5LLwDGI95NkB6zAXGLec5NRTjz3RQ= X-Gm-Message-State: AOJu0YzCAkd+y57lzQAlZKOgzF5g8eHrd391zPpKUjzuGf2R9ZZmMIMc 1JzeUdFEJLCuYuA0ElI6Ai+v8ob3PmHJZViVsujMi6qGPlMi8cR2BH7zhLon/ej0e4fkj6qcjvq EJA== X-Google-Smtp-Source: AGHT+IEuFUPrKGZNxAUQCkAZHdAFv9OY/U9m9EfSTuCmaxi85ZlQiw/125AE2F106cBa+RSZ0S9g4AMYkZ4= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:2234:4e4b:bcf0:406e]) (user=glider job=sendgmr) by 2002:a0d:d950:0:b0:60a:1844:74ef with SMTP id b77-20020a0dd950000000b0060a184474efmr742423ywe.1.1710866227040; Tue, 19 Mar 2024 09:37:07 -0700 (PDT) Date: Tue, 19 Mar 2024 17:36:56 +0100 In-Reply-To: <20240319163656.2100766-1-glider@google.com> Mime-Version: 1.0 References: <20240319163656.2100766-1-glider@google.com> X-Mailer: git-send-email 2.44.0.291.gc1ea87d7ee-goog Message-ID: <20240319163656.2100766-3-glider@google.com> Subject: [PATCH v1 3/3] x86: call instrumentation hooks from copy_mc.c From: Alexander Potapenko To: glider@google.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, tglx@linutronix.de, x86@kernel.org, Linus Torvalds , Dmitry Vyukov , Marco Elver , Tetsuo Handa X-Rspamd-Queue-Id: DC92F40015 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: affitres173ux8y5w7kpsuab1cpyektn X-HE-Tag: 1710866227-886166 X-HE-Meta: U2FsdGVkX199WdG8b/u7cKQeIcS0VbzbOB3cPFSsweGt0iyXX9PwWeIVXfKpcZfh3VcpalSt/Kf5Z6j7+9zmi1L63fpXVvyz0vny9PsojJz115TUoaltWO8OuaGDzdcl+OH53erK+1nTRtXP61TTIVWIn4j9vT0ysk06TKlYjKkBMokOOP2JU284K8aTMWQZssvjzmHaeCBB/uQ4S8K9761uCtF+ZybR76CpOl8hLxlqxJAAMI7bncUzTC0ca6m+jfQgSadUFDrIm/nVRlXm3sRWi6D9cMYLQkyjlAm3Aspm8sRIbtZ+A1chfnX666zzsqtNWD/honO4Lq1j81yaO9q2yibhLlVG/8XcNpkBT+o9/VsnH7ZlVRhzADd4xQvB+DvvF7laXpxTM0aE0AVZcSrUckqRxNN5oMsphuKlcoWzq5gezxnzP3AA1KWw53mwPI+qmNebphEwRBN/Rk3ov/7Jyc/jOzL7ysDmZY7QjJgSDtY0FJx65ZSl9gjxD/IHgKbgNvZUzb9m2n5kwQMzYGbU7AGI64Bpzl0z5KxtubZjINLsZqUwGNZFKfD+JUI+aL5U+F0BHkkWrbZVplH3EIIm0veR/Hp/NK9C8pEx90FX6RVg119BWDRju8AdGZ1YddgfPzWz6CkS+64G4s1CiuX/xQRDP6mp3CltAYzAkUavr+mOUjRDqOmri9635dUTiFrcNIlS1eZQyePYY8uqEkOgtrIlaKm5FZA/tBvwoaZFkNNxWMklcspELfg6RZjx3RIJivr1NvzkjlhwXY2HrRVLLg2LMFi4ECUAg+B7VrhagM80W8bWhw2hClgpsZ+ZcsUaVFekO52KsO7Pb0Tin0IDxG7O10mlOi4OIZBFMbQ+hVPO4Bygg+VIRrOg3Ftkm9MoY0dWfwH0x1UOqI3r086U0gyLlq11R8uq6I/3HGesdhPJMSIq8qzH1aRyzqFxMcJZspX77CFMFHEehu8 gFJ26wDV kh/SqqH0mc1g4mSRwa0s1Oz3NxMgv8gD+poWkjzwqT01ehX2FhVrfiPqjr76dvm68kt/xtJ7NZ2Eax8RLwziidTu+4qdTZUS3HyfOZd9w8DO7waZms9IrUnrJ1lPHYZXRD40K/NnQ4pOGhwPtvYOKdbyynjkw4o+m+yFIK1ivPB1eD3/gmKjA/adSu0o5UHRoDJ/2Jmp+6sprSpOWd5gXGuo2B5Cn97yKdNr39Nv6qdWuK4wDwAtxO9rtbdFCqvIeUcj9N1ht13xF0lYzIS8zv5q3K/qTHfMU7TcO+6o6RWMgNUjlgkSTfeT+1XIksVms7McX/K+pttUzPdWms7Q6DWlZMjTmXfDzJoIE0mDiQpOc8S/MwQdyu+cZKqK0uGWKoDYl1nGxExbJ+G4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Memory accesses in copy_mc_to_kernel() and copy_mc_to_user() are performed by assembly routines and are invisible to KASAN, KCSAN, and KMSAN. Add hooks from instrumentation.h to tell the tools these functions have memcpy/copy_from_user semantics. The call to copy_mc_fragile() in copy_mc_fragile_handle_tail() is left intact, because the latter is only called from the assembly implementation of copy_mc_fragile(), so the memory accesses in it are covered by the instrumentation in copy_mc_to_kernel() and copy_mc_to_user(). Link: https://lore.kernel.org/all/3b7dbd88-0861-4638-b2d2-911c97a4cadf@I-love.SAKURA.ne.jp/ Suggested-by: Linus Torvalds Signed-off-by: Alexander Potapenko Cc: Dmitry Vyukov Cc: Marco Elver Cc: Tetsuo Handa --- arch/x86/lib/copy_mc.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/arch/x86/lib/copy_mc.c b/arch/x86/lib/copy_mc.c index 6e8b7e600def5..e8aec0dbe6bcf 100644 --- a/arch/x86/lib/copy_mc.c +++ b/arch/x86/lib/copy_mc.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include @@ -61,10 +62,20 @@ unsigned long copy_mc_enhanced_fast_string(void *dst, const void *src, unsigned */ unsigned long __must_check copy_mc_to_kernel(void *dst, const void *src, unsigned len) { - if (copy_mc_fragile_enabled) - return copy_mc_fragile(dst, src, len); - if (static_cpu_has(X86_FEATURE_ERMS)) - return copy_mc_enhanced_fast_string(dst, src, len); + unsigned long ret; + + if (copy_mc_fragile_enabled) { + instrument_memcpy_before(dst, src, len); + ret = copy_mc_fragile(dst, src, len); + instrument_memcpy_after(dst, src, len, ret); + return ret; + } + if (static_cpu_has(X86_FEATURE_ERMS)) { + instrument_memcpy_before(dst, src, len); + ret = copy_mc_enhanced_fast_string(dst, src, len); + instrument_memcpy_after(dst, src, len, ret); + return ret; + } memcpy(dst, src, len); return 0; } @@ -76,6 +87,7 @@ unsigned long __must_check copy_mc_to_user(void __user *dst, const void *src, un if (copy_mc_fragile_enabled) { __uaccess_begin(); + instrument_copy_to_user(dst, src, len); ret = copy_mc_fragile((__force void *)dst, src, len); __uaccess_end(); return ret; @@ -83,6 +95,7 @@ unsigned long __must_check copy_mc_to_user(void __user *dst, const void *src, un if (static_cpu_has(X86_FEATURE_ERMS)) { __uaccess_begin(); + instrument_copy_to_user(dst, src, len); ret = copy_mc_enhanced_fast_string((__force void *)dst, src, len); __uaccess_end(); return ret;