From patchwork Tue Mar 19 16:36:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13596905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E4A9C54E71 for ; Tue, 19 Mar 2024 16:37:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B31F06B0083; Tue, 19 Mar 2024 12:37:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AE1CD6B0085; Tue, 19 Mar 2024 12:37:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A9BE6B0088; Tue, 19 Mar 2024 12:37:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8BD9B6B0083 for ; Tue, 19 Mar 2024 12:37:04 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 08A70A11B9 for ; Tue, 19 Mar 2024 16:37:04 +0000 (UTC) X-FDA: 81914343168.20.356F16A Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf08.hostedemail.com (Postfix) with ESMTP id 74A20160016 for ; Tue, 19 Mar 2024 16:37:02 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=zzhQoLcz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of 3Lb_5ZQYKCMguzwrs5u22uzs.q20zw18B-00y9oqy.25u@flex--glider.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3Lb_5ZQYKCMguzwrs5u22uzs.q20zw18B-00y9oqy.25u@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710866222; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Pe97FWHITdYkiCtSSFDy2YJv9pSAbWKeZZSBy5Rk1y0=; b=zVqN9mYZwN5wxfUlz0WMsX6hk64jqB9nYDJTt3W2Q5L/y0x0A+Nh9PUBVIjmOkZW+svQxS 2uonO4xaqNkxgqqHAtxYuXkyXWqA1OwNTYDXy0WGa1w2Zv8gZnABrnPR6R9UsaGdYXUnKX MyJGdMbD61HIk+gDdXAC78NPLrl4yE4= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=zzhQoLcz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of 3Lb_5ZQYKCMguzwrs5u22uzs.q20zw18B-00y9oqy.25u@flex--glider.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3Lb_5ZQYKCMguzwrs5u22uzs.q20zw18B-00y9oqy.25u@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710866222; a=rsa-sha256; cv=none; b=4254Mo2bMk30jY3U6nWdT/wqQypsr8OWuZj9IVIMgr+nBz/sHYuHSEl6SNAdyngT9igYsJ 1RNb8/ZFRXFnCPOiUN3XgngM4QUwcHIohCZfWpQGYaOZL5oTLornhGYwlmrlnDDf/7SMfx Ww5uPckEVJJpEN12nvASbOUcNQ2YiDA= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-60a2b53b99eso124173087b3.3 for ; Tue, 19 Mar 2024 09:37:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710866221; x=1711471021; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=Pe97FWHITdYkiCtSSFDy2YJv9pSAbWKeZZSBy5Rk1y0=; b=zzhQoLczN9J20T4Nmd91lgUKW44PbEXVpkqw+8Gczpc3klJh5Hvv5/L0Bv43+P3SBs zb7p2ZfP1tuQwUqU49qdfcoAW0un71XZaOVW35xwpMTA66P+ygCH3WT8oOP3B06W8D8s umNBFQYk+MVNpoJyA5/DiTkFxu8nl1Rw2uyyi+bNH7wwvmN3sldVuY144fsWsaSheePI 9oLO460D4eIZdYPDbpz5lgf8/kf+I/jQzqZbu8eVTnycwumx+wlsGyJ+mPN+UXLlQeyr HiTHrGuww40h9SMABVCH7g1hfk1eflUkrhT466yP0LFq5V94ry8ELIFEmpv8HpabhHvC ffgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710866221; x=1711471021; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Pe97FWHITdYkiCtSSFDy2YJv9pSAbWKeZZSBy5Rk1y0=; b=v8UrwdwcsqH6ugFav93mXvvQSKDunDPTvMXYpvg/DVZLyqENcSRuJqUb5IF6u/QoTV XBdmI8msu7KQamJAis+nFVuhj11weJLyICCUNbu0xnOovOTs59Ee+dWwfwR4pUQ7YTYQ Mnz9CWJE4W6Nf6R9KY6JRARFVKUVtuPcPBJ0zJwU7ucVEIvQnyx6ugBs/f1nY4Te7i1f V36DRmSTWAuuR02vHhc1p8mjZvGzeJaXBMJZq1lfpEaBM3jrSKzgmh+ZJ7ELfKvqgO/7 NclFeaDi2Fts8jzNkBW7C86CUILQFcXLsxIvYBGLiWFJb/mHXeVuJuojJJX3HvGSv6gO U3Gg== X-Forwarded-Encrypted: i=1; AJvYcCV2Nt/MB8nyN6kuN4q5W2ygJXX0AB0Q3xCQIFFpTDLGuMiC/+QkmBDN8oH6a90XGeA19GWCYsQiWSm54J2SWlaafoY= X-Gm-Message-State: AOJu0YwO1VRyQGMfCeKYpdpxVOMQE4vNVzfe97VbuIfGNyqZE0UWH4qt O2Zta1lMc9DWxq2Pbun5iHFWSTBD399kuuw6O5QTEtS95Mss9mJ6q+Ea06igr3aDIvl68UANkQc Aeg== X-Google-Smtp-Source: AGHT+IEz30VBiIgqKB2uLCoDK46fqUug5yjDscbPTqKHvlX6fQbTCw7l6zHUNahzxmdhiA2a5cr8piuX0n4= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:2234:4e4b:bcf0:406e]) (user=glider job=sendgmr) by 2002:a81:4941:0:b0:610:c60a:bd27 with SMTP id w62-20020a814941000000b00610c60abd27mr1741328ywa.0.1710866221483; Tue, 19 Mar 2024 09:37:01 -0700 (PDT) Date: Tue, 19 Mar 2024 17:36:54 +0100 Mime-Version: 1.0 X-Mailer: git-send-email 2.44.0.291.gc1ea87d7ee-goog Message-ID: <20240319163656.2100766-1-glider@google.com> Subject: [PATCH v1 1/3] mm: kmsan: implement kmsan_memmove() From: Alexander Potapenko To: glider@google.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, tglx@linutronix.de, x86@kernel.org, Tetsuo Handa , Dmitry Vyukov , Marco Elver , Linus Torvalds X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 74A20160016 X-Stat-Signature: xguy73fx743586sh1r883dtqooqtx936 X-HE-Tag: 1710866222-399471 X-HE-Meta: U2FsdGVkX1/hED9HNg67olAyDBol88AkCjB4NGqAIuWI2cEO5uujK8N0EAhyd0iAaAz3X7DaBtNyJTN6SGvoXumc2ITRvmE8m0MBDDiBGBWfBy5t6OSlqQUPpbyB4BK8K0b/MIdwCtCA3kB1T5mXCDxPlyoHMS/zmsJkAfWyvcdoL/pKJ0Qnw70jnKRoIAvtfq4lvRTfhTddCvBBUO3GItYvVPXhj+7h7kZcohQVNnu5bVrPh1/pZ7REoKoWSaUgxrOqcM60P3PqrAznEmtyEDO7zZH+DWWaLQW+g/Y2yUh96dNAJsZ1pKcpFvIlewuHfU9Jzl0VYhyDlFIlRF6MNu1GcdDi5Zn0IxXKbyL8/6irrxNMFitIlm5YF4Owe8MuSxspfzYQKfwJoaCiV2l3IcUk8yI5n+h+x9wSC3JzSm2vCitAzJyRV2SG+RDLRGLrzJxnqJR1EB5he4aMLYKrumC6nOIbjT0LWbvOt8qZTRlMCnqMfABXPcRwU9l2dh4/HzdDm//AA9Y8obdMo94dOdGl8IV7kCq2NZMTUHe9EKrV8iaWKc1/U1zXF+2QkPXFj52C8uvRXFDSw7J7E17Wbgwk/A2ElkkI01sAoIXQ5yOPnt6A5a2ZmnPgPkgfto4i/gmk6KT8pqKPVhqxSQeLKBg+qMjVLYWtIpJU8CORizMhbuN1BOKQCE4N0EkcHK9N8RBlwIHkzRGt0fax5wsV2VBUO1Q/W1UdCie2XwV0hOmH9wpUGwd2joof5zWuNKBLlcUu6TiwBOg3N/XcEulurANq1kptRXc920KkiEh8OXLaZgPpDbTrVzr5hbe6Ajt1/cSUD6ywo9n9f8I/zeryORcWAAvBIMbpro6LLNQmM66LErniTBDuqFTm2dB5j1zmdfMroOh9z/UvB9hDwOVvjNHN75/vPt6Sk5W0MP+pcu2W+4cfDFZX1N7K7A0BZm+aMHHF34F71L4eRCuY4d+ TXE9msXi pyYf0aAJCEVxFC4h6PKUTHGBxF6jinTUjHXZBY7SPzqrlNenoQTBwEUgQtFwVJdRJuprOMPQiQZ1mVyQAmz/8UoJaAz3djWOyE6PBpZGLp730lU7JiFH0KVVQE4UrMzhALEDET+mTzt9yzI02lcr7T75S1fRbH1KHq4y2URY8An3e7dMxrWahOQRNLx6qTXjqND8E1MR6CtfuBancLy6Q0H4PO3EDQvOJ7YOh9Q+il0uqUYNoUSgYfrb8FWtNiW3kTv1Ei0O8rw5FLVbhAJFLjJupT4F6/BjvVca/mWwYoW1O0nkMbLu7BKIu8vkokuygm7P/MRfaOl+MsWBoC7tKQ/SQfgYKwOXMezMv6dQVdc9f+OB7WgJ0ytylk4UUBIi3AMEXW1upVsxT0I4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Provide a hook that can be used by custom memcpy implementations to tell KMSAN that the metadata needs to be copied. Without that, false positive reports are possible in the cases where KMSAN fails to intercept memory initialization. Link: https://lore.kernel.org/all/3b7dbd88-0861-4638-b2d2-911c97a4cadf@I-love.SAKURA.ne.jp/ Suggested-by: Tetsuo Handa Signed-off-by: Alexander Potapenko Cc: Dmitry Vyukov Cc: Marco Elver Cc: Linus Torvalds --- include/linux/kmsan-checks.h | 15 +++++++++++++++ mm/kmsan/hooks.c | 11 +++++++++++ 2 files changed, 26 insertions(+) diff --git a/include/linux/kmsan-checks.h b/include/linux/kmsan-checks.h index c4cae333deec5..e1082dc40abc2 100644 --- a/include/linux/kmsan-checks.h +++ b/include/linux/kmsan-checks.h @@ -61,6 +61,17 @@ void kmsan_check_memory(const void *address, size_t size); void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy, size_t left); +/** + * kmsan_memmove() - Notify KMSAN about a data copy within kernel. + * @to: destination address in the kernel. + * @from: source address in the kernel. + * @size: number of bytes to copy. + * + * Invoked after non-instrumented version (e.g. implemented using assembly + * code) of memmove()/memcpy() is called, in order to copy KMSAN's metadata. + */ +void kmsan_memmove(void *to, const void *from, size_t to_copy); + #else static inline void kmsan_poison_memory(const void *address, size_t size, @@ -78,6 +89,10 @@ static inline void kmsan_copy_to_user(void __user *to, const void *from, { } +static inline void kmsan_memmove(void *to, const void *from, size_t to_copy) +{ +} + #endif #endif /* _LINUX_KMSAN_CHECKS_H */ diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c index 5d6e2dee5692a..364f778ee226d 100644 --- a/mm/kmsan/hooks.c +++ b/mm/kmsan/hooks.c @@ -285,6 +285,17 @@ void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy, } EXPORT_SYMBOL(kmsan_copy_to_user); +void kmsan_memmove(void *to, const void *from, size_t size) +{ + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + kmsan_enter_runtime(); + kmsan_internal_memmove_metadata(to, (void *)from, size); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_memmove); + /* Helper function to check an URB. */ void kmsan_handle_urb(const struct urb *urb, bool is_out) { From patchwork Tue Mar 19 16:36:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13596906 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5C03CD11BF for ; Tue, 19 Mar 2024 16:37:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 390616B0088; Tue, 19 Mar 2024 12:37:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 31A2A6B0089; Tue, 19 Mar 2024 12:37:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1BA006B008A; Tue, 19 Mar 2024 12:37:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0C5386B0088 for ; Tue, 19 Mar 2024 12:37:07 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AC28DA1908 for ; Tue, 19 Mar 2024 16:37:06 +0000 (UTC) X-FDA: 81914343252.24.D36352A Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf27.hostedemail.com (Postfix) with ESMTP id F004C40016 for ; Tue, 19 Mar 2024 16:37:04 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="zB10/20+"; spf=pass (imf27.hostedemail.com: domain of 3ML_5ZQYKCMsx2zuv8x55x2v.t532z4BE-331Crt1.58x@flex--glider.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3ML_5ZQYKCMsx2zuv8x55x2v.t532z4BE-331Crt1.58x@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710866225; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k4w49nfOoRyLntHEj6xF46Ct9MhFoMry9itEqGksxZs=; b=pKTVZb3Tq/ApKkb07Z5/HCt1Apgpfh5wFI6Sd3GBUaHV7hJ6V/gASRxjcMz/0CAl/QuyXd n8NhYv+PWK4bIS7jrHoPHHecohVtsj9+fI+RiKl/yQ6Au602PyagTPKrl3LbCrgFKAIwvZ blNuzb+nAdYC+IxD0uxLRU83h1BfDpk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710866225; a=rsa-sha256; cv=none; b=zZZDnKAp2vhIaclw2fnro/LC3CiSDP5oSP4AjS0GgNxlcH8LrOedXVGnhDyLCSlpFM5uAh pO3B3hNpLZ1J5dPYCzj9lR8hGtX0WiScTHaih/8ggQ4+G+NDXcVpkGu9Sgz0CInRAUo0Ua tS7zD+Ox9gyZs7/mmuPXb6jS4GPr9Ls= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="zB10/20+"; spf=pass (imf27.hostedemail.com: domain of 3ML_5ZQYKCMsx2zuv8x55x2v.t532z4BE-331Crt1.58x@flex--glider.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3ML_5ZQYKCMsx2zuv8x55x2v.t532z4BE-331Crt1.58x@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-60cc8d4e1a4so109045827b3.3 for ; Tue, 19 Mar 2024 09:37:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710866224; x=1711471024; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=k4w49nfOoRyLntHEj6xF46Ct9MhFoMry9itEqGksxZs=; b=zB10/20+CwRw8Q4t17bNntR88uwevQ4m88Qp4Q/qt79XIKcLLVf+etV+7UcLKZabPl LokU2o+yNejED+DQgKLRWoYF8GDjSjoFFK7wT27CxeqjbccaJvk0FgF+/36vTBagMOBS hN9lgzyA8DF+ihCOI+lLIirNE3QMoCbXq//LKeZ9kc+c3ZmWBjQvWDTVc39tMO7wvNg5 lMS9kN7a9wp9GHBJTY5o5QR55Cj5TGvzUkoBTEAZGOHJ1DF6IuRmq1Wwz1x3H+lxajnN tycBLrUT7BqIQAqmeJhs30W8zKdzAqf9KixDIzMzHa8V1KUDpPDEigweGT29GHhOxSSH IiQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710866224; x=1711471024; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=k4w49nfOoRyLntHEj6xF46Ct9MhFoMry9itEqGksxZs=; b=mVNinoVskILlMoCKPBc1Wj3yCgaAlJe/X8wjJ70BHsndcCN7AYtqqI6v3QiI1wAVlj IQBVoQLD7qxuQE4xpXfWevhlUsH0SO+E+Cckxj0i9eGmJo7PJmndGCR8DR+ocBONEP8p yv80l3kE5GrkGg5srn/f1NpWx0vg4GsPNkt0lat37uTR0oeUtGRkza+/UQ29EwWR+Q8X FCX8/hfM4dFWPM78eSI4FY4P1RuxLad+pAAW6LtgXZdsIKbZVnwXf15cnPu0eJSvlCL8 Jcny6QLQXKC2fGRypUhwJ+Mvmuk6DNKdZwD5b1+geTJovuZPMt1TFcaQ2kemeUiEWu2a /BTQ== X-Forwarded-Encrypted: i=1; AJvYcCWq4NKMEqjOQTi09SicTuTBK3cKhL5GSHlA+EHFWCzEp7qt8zYiEJ+erc+tQHtSLim1VbJ5LV5hztOJigpmz6dLZSQ= X-Gm-Message-State: AOJu0Yz4HSyk7nyTf5LxiZMeb6mS9yWwakV304kZk0JZ1v7NVXl6HAkQ vHstIyVdcD+bm3FDv+nbNkHLeWO/v55gnfhPi8tE0dqt0e2bycSTP66X0SbHtpGhdmt5uOuCmJ0 6+w== X-Google-Smtp-Source: AGHT+IE2b5132XOPzd/fmc5XbsJpCg/epKdPr4l7eMqrWTqkXllllbtWoCJvakx0igbVQQYRl57Kyez91XQ= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:2234:4e4b:bcf0:406e]) (user=glider job=sendgmr) by 2002:a0d:d413:0:b0:60a:3d7e:2b98 with SMTP id w19-20020a0dd413000000b0060a3d7e2b98mr825284ywd.4.1710866224161; Tue, 19 Mar 2024 09:37:04 -0700 (PDT) Date: Tue, 19 Mar 2024 17:36:55 +0100 In-Reply-To: <20240319163656.2100766-1-glider@google.com> Mime-Version: 1.0 References: <20240319163656.2100766-1-glider@google.com> X-Mailer: git-send-email 2.44.0.291.gc1ea87d7ee-goog Message-ID: <20240319163656.2100766-2-glider@google.com> Subject: [PATCH v1 2/3] instrumented.h: add instrument_memcpy_before, instrument_memcpy_after From: Alexander Potapenko To: glider@google.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, tglx@linutronix.de, x86@kernel.org, Dmitry Vyukov , Marco Elver , Tetsuo Handa , Linus Torvalds X-Stat-Signature: q1qrfisjy9fko5pcfwtjiwuczh4fhfks X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: F004C40016 X-Rspam-User: X-HE-Tag: 1710866224-915877 X-HE-Meta: U2FsdGVkX19WhwDxRUwqj7vB16YvhVSpysEq5MszP8XPtppLGnTAV35FLRAuYi4dm4z08A/wkMIZ0EASItfXZQT2k8FFW1tIKI3rFDComazcCXLgp1/xHL7NrOpVoi+bafeKN2vPK2BqosO69NuBaAZrYHNVauI9rKNV8ZoagO5fnD1VAgh7nxfQ3Ow/PzE2dNgszFre5HSh5KC3QHLqDnGh5+1pm/48qwuj++soNnPZTJ9JW9ig/sfa3dg/+wNoy51nflXOW2IWsQEdXPaHMGEN/wyydOBm4Fed76mXrzeKvqnEAuN7hr6n+QbzZ6cffFvN7nj0Z6yZDx3SOy4o1nNcPMrG/gbRrRwqQD5FNJSkuPtqm3G9FoIEP49n2BEsrhd3NFCAT+NiLV9Mb5NVI61K84X/fm34++vy2XwkS8UAGyj19O40OxIBWO96g1lIoTYCJExJnTZIfcRkh6PM69r7iP7OYVDjpSgGZa394hIwplFPlmxW1hQcQGG79B70HyiaTJHbIje2861CuzBNQPWviLHojTLa+c+f/Rqg/XTggXDPGX0K4ZyHdcYHqgi6A/MJQ6TMJt7C0XHnk9szC02PS1swXJ6qKnkvpZ7Tw7chFfY+r4+ZBsAgWR0KgJlx/aWL4DwGpdirxQX1I02I9KoPCqu/fzCQ7QT3j7+LRgMkW6Tp7WzgRbt+Zxe0MRX2nMZ3How345ewMvfZQ6S/biyD56x4wxz7Cer9X2F1TXO+Seyxc2Kg/3O9Z6VBl9pZG6rCVwnmvIFLaJfadQZVtEeKw7SfoI6J+gP2z06+OGDX49R7yRDVxDGkv03h6VpUq+HjrS+y/b7iykg7MXibc0+K5xT0wL3oIGDI3k9/Cb9fllFRh6ZK4pccHQlQFZiGxrsIc7PZa5ZvwpReGlLIY5loBOTGxY93kzuEEnVNCSb+EuFriTNSEs9RB/Ap4v3xAQdMC7IwfEODtoMqYhk g4yOPipp 083CRLwxbZrUuH8yIphZDnqKIOumMqj5h5BjomiAaY3XOxiDpBMYOdDGeFM5XeRpbsg+lnQ308qoLMNWYG7gFWPZs0XLRJ0iFC53gUli7ijB9kvWclzVsP8F/xpQ4olFWRVc07eq+uZRO8Rf8fQVS0fRoY5uIywrYz7OrS/Px8QfyqCA59hUHMagBR9P6yFgEApHx/fA3blujY40xBcw+bw3HWfqE9BQsu1wKL7xRxNm3DijPVdNeIF0NTJ2cFjxbKO1cNHX7V14GcY5/Q8rmmsINdUjF1fBQ8p7yV+5DXddGmJ7HgvkEc6ly6+4+zMM3D5CPZG4EczeKeMF+GGiovgtJnC8By+U/lWcwTd9dpmNffxls69wKIPd1Q2gAtR+IqEFHs7XSGkNUQWA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Bug detection tools based on compiler instrumentation may miss memory accesses in custom memcpy implementations (such as copy_mc_to_kernel). Provide instrumentation hooks that tell KASAN, KCSAN, and KMSAN about such accesses. Link: https://lore.kernel.org/all/3b7dbd88-0861-4638-b2d2-911c97a4cadf@I-love.SAKURA.ne.jp/ Signed-off-by: Alexander Potapenko Cc: Dmitry Vyukov Cc: Marco Elver Cc: Tetsuo Handa Cc: Linus Torvalds --- include/linux/instrumented.h | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/include/linux/instrumented.h b/include/linux/instrumented.h index 1b608e00290aa..f5f81f02506eb 100644 --- a/include/linux/instrumented.h +++ b/include/linux/instrumented.h @@ -147,6 +147,41 @@ instrument_copy_from_user_after(const void *to, const void __user *from, kmsan_unpoison_memory(to, n - left); } +/** + * instrument_memcpy_before - add instrumentation before non-instrumented memcpy + * @to: destination address + * @from: source address + * @n: number of bytes to copy + * + * Instrument memory accesses that happen in custom memcpy implementations. The + * instrumentation should be inserted before the memcpy call. + */ +static __always_inline void instrument_memcpy_before(void *to, const void *from, + unsigned long n) +{ + kasan_check_write(to, n); + kasan_check_read(from, n); + kcsan_check_write(to, n); + kcsan_check_read(from, n); +} + +/** + * instrument_memcpy_after - add instrumentation before non-instrumented memcpy + * @to: destination address + * @from: source address + * @n: number of bytes to copy + * @left: number of bytes not copied (if known) + * + * Instrument memory accesses that happen in custom memcpy implementations. The + * instrumentation should be inserted after the memcpy call. + */ +static __always_inline void instrument_memcpy_after(void *to, const void *from, + unsigned long n, + unsigned long left) +{ + kmsan_memmove(to, from, n - left); +} + /** * instrument_get_user() - add instrumentation to get_user()-like macros * @to: destination variable, may not be address-taken From patchwork Tue Mar 19 16:36:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13596907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 982B6C6FD1F for ; Tue, 19 Mar 2024 16:37:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0ABD36B0089; Tue, 19 Mar 2024 12:37:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 05D6B6B008A; Tue, 19 Mar 2024 12:37:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3CC36B008C; Tue, 19 Mar 2024 12:37:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C81EE6B0089 for ; Tue, 19 Mar 2024 12:37:09 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7F1201C133D for ; Tue, 19 Mar 2024 16:37:09 +0000 (UTC) X-FDA: 81914343378.28.7A980A9 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf17.hostedemail.com (Postfix) with ESMTP id DC92F40015 for ; Tue, 19 Mar 2024 16:37:07 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=sRDt9zQT; spf=pass (imf17.hostedemail.com: domain of 3M7_5ZQYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3M7_5ZQYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710866227; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DNX90sT1lUbl5lD9B0qqgx+siiIHXGud0HrlrLB6kTU=; b=IlLFdBjj+N+FJhGx/g9fPKSjgy7yWJWo5MSP7/BXVqaitLdeBwWw4tvmO9fxnIFr1/56ti /SeTnu26JMGGO44GFvgliVJxH28MhTOHJvzGJi+eeR5U+Uu8v0KJMvdnwo/F7R+hyAvtG5 950QSYwpuXJlBT6osgFKyM+KZKedk8E= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710866227; a=rsa-sha256; cv=none; b=BrYZQmM3zpjdCzYQHVVOrGFU4IvO8m9m1kPaROew6ET5SzKcrIX5KFNgjUkZKoqK87Fe5V LPw3VJR+/vtE/mqzdMEr6PLdlbIrqvOg72+T3cqU+0859lesr4dLET2oF3FJohTEUmHTNi 7xCcqdVGKnQ833EyR9KlDUf0HZnMMo4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=sRDt9zQT; spf=pass (imf17.hostedemail.com: domain of 3M7_5ZQYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3M7_5ZQYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-60a605154d0so500037b3.0 for ; Tue, 19 Mar 2024 09:37:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710866227; x=1711471027; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DNX90sT1lUbl5lD9B0qqgx+siiIHXGud0HrlrLB6kTU=; b=sRDt9zQTs/sJXSMqb2SWs0nlXoQu8+VfLdmdqsaFRCJxCW2uZfCUYiXON41/mC2nUu kx/k/IaONU8qZ0c9CaLQnwa87gZEqXu2SvvyiLXttOSMsEBgGo9xAdyfxsMw0Wchxcpb 9cLRVyI/chwTxGcAOYc70UYQvur+jGDIYR4Q7pw/kr6b3cGBUN7D64RoYiJF6TCeqcR9 mjilWQRQMK2c6gOaZMM0qCK/xVp0ArrdmJV3j1jC/scGE+9B9lixFOywRQwiYI3haBg+ 3ID0fU670l47jn9iiwvbX59RQqzs6l8NyJiYrozZ9B6nDtGF4Qf3KkOhhqF3BLay4OvH wdlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710866227; x=1711471027; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DNX90sT1lUbl5lD9B0qqgx+siiIHXGud0HrlrLB6kTU=; b=gD25OgYJuQeJNtjy/vP7jSomL2GNlkU0WvQif6wmsvGY43oPXxfU0DoBiffwxedahW Jtsa3o23F2w+H232kdimvmjQHRrfCiJCoOUIdibXjzNNZoKx2H57XsVSV7vnKGAKz2uf xl+TNi144aMFcZv8ghBUUjiIO18cOft/oU2pYu1GmJdNzjwmjxblx7obnfCLBWYuAw/Q 1V1OMmqQm910ycP6h59bDis0ZTAt50/T2GA0aMw9S3fNmiWk9n2OXGceR2NNJlz95t7m zBXdCQGCb5yFBfKzZ8kp8S1t3mOvyqHN2+2BhDxOAqsLBax1Xg+kTGsJYglFtOcRoKq9 EStg== X-Forwarded-Encrypted: i=1; AJvYcCXZYkW4Hl/vJE3Xq6MMc0qTdZ72YBHeZnCtUxRyW2DPOJzfm9+4ZWtlOgHCr/5LLwDGI95NkB6zAXGLec5NRTjz3RQ= X-Gm-Message-State: AOJu0YzCAkd+y57lzQAlZKOgzF5g8eHrd391zPpKUjzuGf2R9ZZmMIMc 1JzeUdFEJLCuYuA0ElI6Ai+v8ob3PmHJZViVsujMi6qGPlMi8cR2BH7zhLon/ej0e4fkj6qcjvq EJA== X-Google-Smtp-Source: AGHT+IEuFUPrKGZNxAUQCkAZHdAFv9OY/U9m9EfSTuCmaxi85ZlQiw/125AE2F106cBa+RSZ0S9g4AMYkZ4= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:2234:4e4b:bcf0:406e]) (user=glider job=sendgmr) by 2002:a0d:d950:0:b0:60a:1844:74ef with SMTP id b77-20020a0dd950000000b0060a184474efmr742423ywe.1.1710866227040; Tue, 19 Mar 2024 09:37:07 -0700 (PDT) Date: Tue, 19 Mar 2024 17:36:56 +0100 In-Reply-To: <20240319163656.2100766-1-glider@google.com> Mime-Version: 1.0 References: <20240319163656.2100766-1-glider@google.com> X-Mailer: git-send-email 2.44.0.291.gc1ea87d7ee-goog Message-ID: <20240319163656.2100766-3-glider@google.com> Subject: [PATCH v1 3/3] x86: call instrumentation hooks from copy_mc.c From: Alexander Potapenko To: glider@google.com, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, tglx@linutronix.de, x86@kernel.org, Linus Torvalds , Dmitry Vyukov , Marco Elver , Tetsuo Handa X-Rspamd-Queue-Id: DC92F40015 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: affitres173ux8y5w7kpsuab1cpyektn X-HE-Tag: 1710866227-886166 X-HE-Meta: U2FsdGVkX199WdG8b/u7cKQeIcS0VbzbOB3cPFSsweGt0iyXX9PwWeIVXfKpcZfh3VcpalSt/Kf5Z6j7+9zmi1L63fpXVvyz0vny9PsojJz115TUoaltWO8OuaGDzdcl+OH53erK+1nTRtXP61TTIVWIn4j9vT0ysk06TKlYjKkBMokOOP2JU284K8aTMWQZssvjzmHaeCBB/uQ4S8K9761uCtF+ZybR76CpOl8hLxlqxJAAMI7bncUzTC0ca6m+jfQgSadUFDrIm/nVRlXm3sRWi6D9cMYLQkyjlAm3Aspm8sRIbtZ+A1chfnX666zzsqtNWD/honO4Lq1j81yaO9q2yibhLlVG/8XcNpkBT+o9/VsnH7ZlVRhzADd4xQvB+DvvF7laXpxTM0aE0AVZcSrUckqRxNN5oMsphuKlcoWzq5gezxnzP3AA1KWw53mwPI+qmNebphEwRBN/Rk3ov/7Jyc/jOzL7ysDmZY7QjJgSDtY0FJx65ZSl9gjxD/IHgKbgNvZUzb9m2n5kwQMzYGbU7AGI64Bpzl0z5KxtubZjINLsZqUwGNZFKfD+JUI+aL5U+F0BHkkWrbZVplH3EIIm0veR/Hp/NK9C8pEx90FX6RVg119BWDRju8AdGZ1YddgfPzWz6CkS+64G4s1CiuX/xQRDP6mp3CltAYzAkUavr+mOUjRDqOmri9635dUTiFrcNIlS1eZQyePYY8uqEkOgtrIlaKm5FZA/tBvwoaZFkNNxWMklcspELfg6RZjx3RIJivr1NvzkjlhwXY2HrRVLLg2LMFi4ECUAg+B7VrhagM80W8bWhw2hClgpsZ+ZcsUaVFekO52KsO7Pb0Tin0IDxG7O10mlOi4OIZBFMbQ+hVPO4Bygg+VIRrOg3Ftkm9MoY0dWfwH0x1UOqI3r086U0gyLlq11R8uq6I/3HGesdhPJMSIq8qzH1aRyzqFxMcJZspX77CFMFHEehu8 gFJ26wDV kh/SqqH0mc1g4mSRwa0s1Oz3NxMgv8gD+poWkjzwqT01ehX2FhVrfiPqjr76dvm68kt/xtJ7NZ2Eax8RLwziidTu+4qdTZUS3HyfOZd9w8DO7waZms9IrUnrJ1lPHYZXRD40K/NnQ4pOGhwPtvYOKdbyynjkw4o+m+yFIK1ivPB1eD3/gmKjA/adSu0o5UHRoDJ/2Jmp+6sprSpOWd5gXGuo2B5Cn97yKdNr39Nv6qdWuK4wDwAtxO9rtbdFCqvIeUcj9N1ht13xF0lYzIS8zv5q3K/qTHfMU7TcO+6o6RWMgNUjlgkSTfeT+1XIksVms7McX/K+pttUzPdWms7Q6DWlZMjTmXfDzJoIE0mDiQpOc8S/MwQdyu+cZKqK0uGWKoDYl1nGxExbJ+G4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Memory accesses in copy_mc_to_kernel() and copy_mc_to_user() are performed by assembly routines and are invisible to KASAN, KCSAN, and KMSAN. Add hooks from instrumentation.h to tell the tools these functions have memcpy/copy_from_user semantics. The call to copy_mc_fragile() in copy_mc_fragile_handle_tail() is left intact, because the latter is only called from the assembly implementation of copy_mc_fragile(), so the memory accesses in it are covered by the instrumentation in copy_mc_to_kernel() and copy_mc_to_user(). Link: https://lore.kernel.org/all/3b7dbd88-0861-4638-b2d2-911c97a4cadf@I-love.SAKURA.ne.jp/ Suggested-by: Linus Torvalds Signed-off-by: Alexander Potapenko Cc: Dmitry Vyukov Cc: Marco Elver Cc: Tetsuo Handa --- arch/x86/lib/copy_mc.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/arch/x86/lib/copy_mc.c b/arch/x86/lib/copy_mc.c index 6e8b7e600def5..e8aec0dbe6bcf 100644 --- a/arch/x86/lib/copy_mc.c +++ b/arch/x86/lib/copy_mc.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include @@ -61,10 +62,20 @@ unsigned long copy_mc_enhanced_fast_string(void *dst, const void *src, unsigned */ unsigned long __must_check copy_mc_to_kernel(void *dst, const void *src, unsigned len) { - if (copy_mc_fragile_enabled) - return copy_mc_fragile(dst, src, len); - if (static_cpu_has(X86_FEATURE_ERMS)) - return copy_mc_enhanced_fast_string(dst, src, len); + unsigned long ret; + + if (copy_mc_fragile_enabled) { + instrument_memcpy_before(dst, src, len); + ret = copy_mc_fragile(dst, src, len); + instrument_memcpy_after(dst, src, len, ret); + return ret; + } + if (static_cpu_has(X86_FEATURE_ERMS)) { + instrument_memcpy_before(dst, src, len); + ret = copy_mc_enhanced_fast_string(dst, src, len); + instrument_memcpy_after(dst, src, len, ret); + return ret; + } memcpy(dst, src, len); return 0; } @@ -76,6 +87,7 @@ unsigned long __must_check copy_mc_to_user(void __user *dst, const void *src, un if (copy_mc_fragile_enabled) { __uaccess_begin(); + instrument_copy_to_user(dst, src, len); ret = copy_mc_fragile((__force void *)dst, src, len); __uaccess_end(); return ret; @@ -83,6 +95,7 @@ unsigned long __must_check copy_mc_to_user(void __user *dst, const void *src, un if (static_cpu_has(X86_FEATURE_ERMS)) { __uaccess_begin(); + instrument_copy_to_user(dst, src, len); ret = copy_mc_enhanced_fast_string((__force void *)dst, src, len); __uaccess_end(); return ret;