From patchwork Thu Sep 7 13:06:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13376428 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57217EC8724 for ; Thu, 7 Sep 2023 13:06:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B887B8E000F; Thu, 7 Sep 2023 09:06:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B38978D0005; Thu, 7 Sep 2023 09:06:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A001E8E000F; Thu, 7 Sep 2023 09:06:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9222C8D0005 for ; Thu, 7 Sep 2023 09:06:50 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5426C1402A9 for ; Thu, 7 Sep 2023 13:06:50 +0000 (UTC) X-FDA: 81209826180.08.1339BFB Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf06.hostedemail.com (Postfix) with ESMTP id 5D43C180003 for ; Thu, 7 Sep 2023 13:06:48 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=LTbd9FN+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of 35sr5ZAYKCJE163yzC19916z.x97638FI-775Gvx5.9C1@flex--glider.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=35sr5ZAYKCJE163yzC19916z.x97638FI-775Gvx5.9C1@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694092008; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=pguS5nzVV0mhBjB/CrXM2tdNaYwlS7Rq+h24iaGnEBI=; b=wrQLf3833H2M1tC4Nj0xhh+vtl3KgVuL61iVKUM39B/iu/bpBeo5scZACfo42/eQph1qpD vW46IrPynhLbIAGFzoD9qsbpHGXEGgYG3sOuOWY7JUZL4AkFmf4qOy/rvY3u6RmBz6d/yC dHRK2bVEvDsMVA8Bq3KUrisT3oiJgKk= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=LTbd9FN+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of 35sr5ZAYKCJE163yzC19916z.x97638FI-775Gvx5.9C1@flex--glider.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=35sr5ZAYKCJE163yzC19916z.x97638FI-775Gvx5.9C1@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694092008; a=rsa-sha256; cv=none; b=f2EIy6IJr9/9wP0Ng8TA9+tavLrRaDVR9fN77cQFzFQ1WJiTPp5qIZo7/2KM8hZOVBKEri drUjwEiTURooIFMQuYg1qAg5c4Ti5TJ25kLo6LSoiIEf/7aKyjd+BzdQ0XN7kc/SMkggi9 34EYMz8V4FOkb4ynaJaLNK7i86fZWNs= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-3fe1521678fso6514945e9.1 for ; Thu, 07 Sep 2023 06:06:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1694092007; x=1694696807; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=pguS5nzVV0mhBjB/CrXM2tdNaYwlS7Rq+h24iaGnEBI=; b=LTbd9FN+8uq0lOwXX5w2XCHVw9l9A84j3TfPg546N/jDDwnVGLh5oFUoLx0uwkSg1q mXHYC9UI0Nb3UKzdezmzDkjNGfBtcpyTl/cuZU1IAs5Kgkmi4LICCLgj9ILPeK7r0aWH YrYLcTyWQ1q1AoTxVyc3GjVL1rw7/xTt/ckCtS4BIF7Zp88A1QXIbLROYS5pCcAoMlVy qUpgqv2cS1uOXwJJv3Lu4ZGKpTc8+cKuc+qN5/5E0F4DACCP+qpXtWKtyNiN+VvU+5d+ QQ9P5vtL1A72RVBAIYRfqUci0MnF+7hUZzTcXz7cr5M2vVlIDsecXKUry10nkEuXkclK uP2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694092007; x=1694696807; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=pguS5nzVV0mhBjB/CrXM2tdNaYwlS7Rq+h24iaGnEBI=; b=Sn+jjnCCBc8Q22oJILJx1E5Baw+RfOsdGY32s7Sr8Y84TzB8JkGaydPebd0prrV3pL vy5M8dkLSsSJoHz7v/9gVoSYoklXpcl2XkvxPKfDt3UVjQ7f4teo2IZtZqpIQcEi2nGb p2iJEHertirRs9U3uzkj4YRpflhFs4LZQ/c/adNohuAHGHLczR3U9Q8ZR1PsGRFLKXll lOz6ckpEkR9UJmIaU9bQ6y5W2UGQLd/aEzZsovRezowcwhp2Y3/NQlR79ZSo1d2qslq0 pfOYQmnj183AzHHYTXd+Y+9SsWw4m3XEWbjJOqkREo74cF1kvVex5e1wqdvFTcFve4ok 6sEg== X-Gm-Message-State: AOJu0YyaxKmZOtRv/J4w/YOxx5pRID8h/JFlAifN2UDu+bUq6UtZh6Py ReZv3xO5xrhlKSP33/nO64G4p2+liko= X-Google-Smtp-Source: AGHT+IGBqXTREGTdC9uanYMcN82i8XIFX52t7L2XmvkIcAbdZnLizPEjhlk0WM1DzQMASN5p1z0+EuxsWCw= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:33dd:e36e:b6dc:1a69]) (user=glider job=sendgmr) by 2002:a05:600c:2d84:b0:402:eacb:a797 with SMTP id i4-20020a05600c2d8400b00402eacba797mr47988wmg.4.1694092006816; Thu, 07 Sep 2023 06:06:46 -0700 (PDT) Date: Thu, 7 Sep 2023 15:06:41 +0200 Mime-Version: 1.0 X-Mailer: git-send-email 2.42.0.283.g2d96d420d3-goog Message-ID: <20230907130642.245222-1-glider@google.com> Subject: [PATCH 1/2] kmsan: simplify kmsan_internal_memmove_metadata() From: Alexander Potapenko To: glider@google.com, dvyukov@google.com, elver@google.com, akpm@linux-foundation.org, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 5D43C180003 X-Stat-Signature: oc5nbc9fwsxe6f6n8ukof8dnuma5iske X-Rspam-User: X-HE-Tag: 1694092008-639562 X-HE-Meta: U2FsdGVkX18ZnUccjaKHMCdHle+9I954GatnzYbQdeev4pI9eMX4YPZ8t1quX15Y6gYaBS8WfyoyOCfeFz4TpYhwmjZZq0DZJAd1NASfYL91v/VS8ziugJ9WKhDgDdPy9fMfUvD9Tlcw+GeCad4PzBkZ+Yt2sk4+7X2lucRS6QM3RACrLhd9a3Q7c0TxIAn17cP1x13q4vPURODfBPPgGimmcijH2bsYruicENLcoCEnL25NRl/DbGtjIWUvDBwkpWnwj+8FUX7AHv3zZL9EbZDmfQHNdGh2zDp+ziXlexdxwKbhEwFlGty6AIA2QXnwm7VaP9GmB0laFYHP3aCCoxbGvyayHRKcIBWk9Vft4fHAy6pVdICC63BmZ2Bj5xORx2JH98rePitGPXA+u5LRXOSiVeuog/rwISGcHWicyzJh1XUMHGElyN90ZUqi79ExYwxydpFfOgEZLhwrWoCMOO2Ta4VYfaLqdQ+HzuKjKEYBJ4gP/8VR2zLN011o9HAGtlEuu7nGP+vUB+NOtFuF51pOvOTEgIyz/IoocClzVwokSPOeq2o8y6smkNgL3vrOPtEqjItmWr0xqGsmarcihpG3Vco/FbfO8MxlutTKdsASLSfBhu6nBdaDb1LsUYXH8RQGwlSMUNRJXjhXhYvZErtWLJGq1goMrN0Zc16qVk3NHB9S4oHg2LgDnO+Bk6f29l7jU8w/LVYgNfdc35tSOvdle/8XBzuuawctzEtLdjb/oMSCKz159cG6U2mvGkAvFBoHlTNYRSWywxxJEMY1m/KL1CgbBiXS5jf2oeI5ivwzS+SGvratgrw8P6+pK0s99eWQAJrBYpg/ZdvQB+7KHQlTbVBf3Vpa6UH2ZHJ+pM4YZmddJ0YToo3Y5noR/BAcNlq0y71tzdRIVa9yABc+bOW8gW1GrnPfpM+b/af+HzTXqQfZwrW5UWMYSrr0jAh0JaV+pCEpMFGoeyXMIuv nVJQuMEf clcoPKsWHMgElYV0tpJuO/7RGNrI4EaIWABbDq9zO3Wv06NyPebIkkQPfnW4LGxUtY4dv2Viod7K1B/ikk3/gheU58kVuicxB/iBmNxeHSfaD3gNYwFZKlfgzMEOdX/EXneYaLz62DFMuMyW49q0+MOc2giOIcIkEbTpIUKkTG0PmG9elTcyBaKv2Xfb9aZ6xkoK72JsUzUWMrdOrdMu2pPtV+f9gaOuxUAtlcICA3X5vLFAaRYhbBrX0Nh8RhnxtToRUIBONRM1cNaTeXwqiIdO7Z4c0J3icTIVxdUNIQwWwUajen2AeOJTuXMzLnfxBoQdZoJlCo3dQCAfky4Eg6oS5kOgHy2xcmBsAzGil9o4HNWQCsX+nVihPkrjhmRw7od3OEVk/ZiGOQmu3ZuGrV8wigFX9xEXezugFwkmBjHEhvj4QQanKzufCcubG2x6Mrc9o9TmsSet4hr0wX7E3aB2ZgfWSqkoXyLQn8aQ0yeS1AKB4NgI/9b4qDLzGsVK/h9pIYSnGTdWVtbKhRfuIxrx3MNe0sYcryfJ52pe1JxqsFf7g0rcIb4gqEQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: kmsan_internal_memmove_metadata() is the function that implements copying metadata every time memcpy()/memmove() is called. Because shadow memory stores 1 byte per each byte of kernel memory, copying the shadow is trivial and can be done by a single memmove() call. Origins, on the other hand, are stored as 4-byte values corresponding to every aligned 4 bytes of kernel memory. Therefore, if either the source or the destination of kmsan_internal_memmove_metadata() is unaligned, the number of origin slots corresponding to the source or destination may differ: 1) memcpy(0xffff888080a00000, 0xffff888080900000, 4) copies 1 origin slot into 1 origin slot: src (0xffff888080900000): xxxx src origins: o111 dst (0xffff888080a00000): xxxx dst origins: o111 2) memcpy(0xffff888080a00001, 0xffff888080900000, 4) copies 1 origin slot into 2 origin slots: src (0xffff888080900000): xxxx src origins: o111 dst (0xffff888080a00000): .xxx x... dst origins: o111 o111 3) memcpy(0xffff888080a00000, 0xffff888080900001, 4) copies 2 origin slots into 1 origin slot: src (0xffff888080900000): .xxx x... src origins: o111 o222 dst (0xffff888080a00000): xxxx dst origins: o111 (or o222) Previously, kmsan_internal_memmove_metadata() tried to solve this problem by copying min(src_slots, dst_slots) as is and cloning the missing slot on one of the ends, if needed. This was error-prone even in the simple cases where 4 bytes were copied, and did not account for situations where the total number of nonzero origin slots could have increased by more than one after copying: memcpy(0xffff888080a00000, 0xffff888080900002, 8) src (0xffff888080900002): ..xx .... xx.. src origins: o111 0000 o222 dst (0xffff888080a00000): xx.. ..xx o111 0000 (or 0000 o222) The new implementation simply copies the shadow byte by byte, and updates the corresponding origin slot, if the shadow byte is nonzero. This approach can handle complex cases with mixed initialized and uninitialized bytes. Similarly to KMSAN inline instrumentation, latter writes to bytes sharing the same origin slots take precedence. Signed-off-by: Alexander Potapenko Acked-by: Marco Elver --- mm/kmsan/core.c | 127 ++++++++++++------------------------------------ 1 file changed, 31 insertions(+), 96 deletions(-) diff --git a/mm/kmsan/core.c b/mm/kmsan/core.c index 3adb4c1d3b193..c19f47af04241 100644 --- a/mm/kmsan/core.c +++ b/mm/kmsan/core.c @@ -83,131 +83,66 @@ depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags, /* Copy the metadata following the memmove() behavior. */ void kmsan_internal_memmove_metadata(void *dst, void *src, size_t n) { + depot_stack_handle_t prev_old_origin = 0, prev_new_origin = 0; + int i, iter, step, src_off, dst_off, oiter_src, oiter_dst; depot_stack_handle_t old_origin = 0, new_origin = 0; - int src_slots, dst_slots, i, iter, step, skip_bits; depot_stack_handle_t *origin_src, *origin_dst; - void *shadow_src, *shadow_dst; - u32 *align_shadow_src, shadow; + u8 *shadow_src, *shadow_dst; + u32 *align_shadow_dst; bool backwards; shadow_dst = kmsan_get_metadata(dst, KMSAN_META_SHADOW); if (!shadow_dst) return; KMSAN_WARN_ON(!kmsan_metadata_is_contiguous(dst, n)); + align_shadow_dst = + (u32 *)ALIGN_DOWN((u64)shadow_dst, KMSAN_ORIGIN_SIZE); shadow_src = kmsan_get_metadata(src, KMSAN_META_SHADOW); if (!shadow_src) { - /* - * @src is untracked: zero out destination shadow, ignore the - * origins, we're done. - */ - __memset(shadow_dst, 0, n); + /* @src is untracked: mark @dst as initialized. */ + kmsan_internal_unpoison_memory(dst, n, /*checked*/ false); return; } KMSAN_WARN_ON(!kmsan_metadata_is_contiguous(src, n)); - __memmove(shadow_dst, shadow_src, n); - origin_dst = kmsan_get_metadata(dst, KMSAN_META_ORIGIN); origin_src = kmsan_get_metadata(src, KMSAN_META_ORIGIN); KMSAN_WARN_ON(!origin_dst || !origin_src); - src_slots = (ALIGN((u64)src + n, KMSAN_ORIGIN_SIZE) - - ALIGN_DOWN((u64)src, KMSAN_ORIGIN_SIZE)) / - KMSAN_ORIGIN_SIZE; - dst_slots = (ALIGN((u64)dst + n, KMSAN_ORIGIN_SIZE) - - ALIGN_DOWN((u64)dst, KMSAN_ORIGIN_SIZE)) / - KMSAN_ORIGIN_SIZE; - KMSAN_WARN_ON((src_slots < 1) || (dst_slots < 1)); - KMSAN_WARN_ON((src_slots - dst_slots > 1) || - (dst_slots - src_slots < -1)); backwards = dst > src; - i = backwards ? min(src_slots, dst_slots) - 1 : 0; - iter = backwards ? -1 : 1; - - align_shadow_src = - (u32 *)ALIGN_DOWN((u64)shadow_src, KMSAN_ORIGIN_SIZE); - for (step = 0; step < min(src_slots, dst_slots); step++, i += iter) { - KMSAN_WARN_ON(i < 0); - shadow = align_shadow_src[i]; - if (i == 0) { - /* - * If @src isn't aligned on KMSAN_ORIGIN_SIZE, don't - * look at the first @src % KMSAN_ORIGIN_SIZE bytes - * of the first shadow slot. - */ - skip_bits = ((u64)src % KMSAN_ORIGIN_SIZE) * 8; - shadow = (shadow >> skip_bits) << skip_bits; + step = backwards ? -1 : 1; + iter = backwards ? n - 1 : 0; + src_off = (u64)src % KMSAN_ORIGIN_SIZE; + dst_off = (u64)dst % KMSAN_ORIGIN_SIZE; + + /* Copy shadow bytes one by one, updating the origins if necessary. */ + for (i = 0; i < n; i++, iter += step) { + oiter_src = (iter + src_off) / KMSAN_ORIGIN_SIZE; + oiter_dst = (iter + dst_off) / KMSAN_ORIGIN_SIZE; + if (!shadow_src[iter]) { + shadow_dst[iter] = 0; + if (!align_shadow_dst[oiter_dst]) + origin_dst[oiter_dst] = 0; + continue; } - if (i == src_slots - 1) { - /* - * If @src + n isn't aligned on - * KMSAN_ORIGIN_SIZE, don't look at the last - * (@src + n) % KMSAN_ORIGIN_SIZE bytes of the - * last shadow slot. - */ - skip_bits = (((u64)src + n) % KMSAN_ORIGIN_SIZE) * 8; - shadow = (shadow << skip_bits) >> skip_bits; - } - /* - * Overwrite the origin only if the corresponding - * shadow is nonempty. - */ - if (origin_src[i] && (origin_src[i] != old_origin) && shadow) { - old_origin = origin_src[i]; - new_origin = kmsan_internal_chain_origin(old_origin); + shadow_dst[iter] = shadow_src[iter]; + old_origin = origin_src[oiter_src]; + if (old_origin == prev_old_origin) + new_origin = prev_new_origin; + else { /* * kmsan_internal_chain_origin() may return * NULL, but we don't want to lose the previous * origin value. */ + new_origin = kmsan_internal_chain_origin(old_origin); if (!new_origin) new_origin = old_origin; } - if (shadow) - origin_dst[i] = new_origin; - else - origin_dst[i] = 0; - } - /* - * If dst_slots is greater than src_slots (i.e. - * dst_slots == src_slots + 1), there is an extra origin slot at the - * beginning or end of the destination buffer, for which we take the - * origin from the previous slot. - * This is only done if the part of the source shadow corresponding to - * slot is non-zero. - * - * E.g. if we copy 8 aligned bytes that are marked as uninitialized - * and have origins o111 and o222, to an unaligned buffer with offset 1, - * these two origins are copied to three origin slots, so one of then - * needs to be duplicated, depending on the copy direction (@backwards) - * - * src shadow: |uuuu|uuuu|....| - * src origin: |o111|o222|....| - * - * backwards = 0: - * dst shadow: |.uuu|uuuu|u...| - * dst origin: |....|o111|o222| - fill the empty slot with o111 - * backwards = 1: - * dst shadow: |.uuu|uuuu|u...| - * dst origin: |o111|o222|....| - fill the empty slot with o222 - */ - if (src_slots < dst_slots) { - if (backwards) { - shadow = align_shadow_src[src_slots - 1]; - skip_bits = (((u64)dst + n) % KMSAN_ORIGIN_SIZE) * 8; - shadow = (shadow << skip_bits) >> skip_bits; - if (shadow) - /* src_slots > 0, therefore dst_slots is at least 2 */ - origin_dst[dst_slots - 1] = - origin_dst[dst_slots - 2]; - } else { - shadow = align_shadow_src[0]; - skip_bits = ((u64)dst % KMSAN_ORIGIN_SIZE) * 8; - shadow = (shadow >> skip_bits) << skip_bits; - if (shadow) - origin_dst[0] = origin_dst[1]; - } + origin_dst[oiter_dst] = new_origin; + prev_new_origin = new_origin; + prev_old_origin = old_origin; } } From patchwork Thu Sep 7 13:06:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13376429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91316EC8727 for ; Thu, 7 Sep 2023 13:06:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EBD0A440185; Thu, 7 Sep 2023 09:06:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E6D3B8D0005; Thu, 7 Sep 2023 09:06:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D34A7440185; Thu, 7 Sep 2023 09:06:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C5CAE8D0005 for ; Thu, 7 Sep 2023 09:06:53 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A18E3160167 for ; Thu, 7 Sep 2023 13:06:53 +0000 (UTC) X-FDA: 81209826306.20.EB4F14F Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf15.hostedemail.com (Postfix) with ESMTP id D4867A003F for ; Thu, 7 Sep 2023 13:06:51 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=KviyaW2Y; spf=pass (imf15.hostedemail.com: domain of 36sr5ZAYKCJU5A723G5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--glider.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=36sr5ZAYKCJU5A723G5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694092011; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=s8i7Jhe9BViu4aMBPbLDU6gPTC4qxWjzqtUEiNX0CLE=; b=1GzOpLmcO5+UlfLZOBNLRttl/GvjrDSX3rFY5siidY4+kUnzMeVR9BDqv/NoLmi6f2OPW4 42qbXoDY1DnNV2beElIbkf+sbeHAye2e2fPyK5LYblmexK194wR/slo8Q4W/of/xIaBfyC C10DF67xEXGBoh07xyFUUZDki171oRg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694092011; a=rsa-sha256; cv=none; b=HCxQTGY/uM68KB9Bzwfwbcyc8ubZIOGNviHVwhU0QPnL0Qs4E2aaPBa3CDAI6LqrctWIYP KDc8XW310kOd/6XTeiZmvbiUNVICb2Yh9Qgaqi51v1tTj/z41g7d0qtQsAtOK7LK0c3IdR ziopo8+d2OuYODja4BDBBlLPpvpW4cQ= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=KviyaW2Y; spf=pass (imf15.hostedemail.com: domain of 36sr5ZAYKCJU5A723G5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--glider.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=36sr5ZAYKCJU5A723G5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5958487ca15so10379967b3.1 for ; Thu, 07 Sep 2023 06:06:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1694092011; x=1694696811; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=s8i7Jhe9BViu4aMBPbLDU6gPTC4qxWjzqtUEiNX0CLE=; b=KviyaW2YfhnGn9sh37dnFiDNOqZyteohmNEiE0UxzGj/Fosk5wLMbrO1X97uxZlcI3 Se8iiNaBSIHYmPMhds8NaRaLZ9kxIdFJr21mjE+ZtAREBTPdGQzrHDupVFaq/Zit0p7w 5LPleExc4hgFpArDDF9FAMv54DaRXkU11ahn3Ip95GX88HdrzP9Pktqch2kR/k4yoLFG LxcXg+Y+tjLSJN7ogGSmP7+qOK6IOCwF83Stcjs+rJ4IaCqK24GVwci5A923nabucw6g H6gEpWprTRXJwpi1G4OTrJgeVZ2yi9ofquGBcL5ywrSAzuY2tyLQHR1UV2Qzo8JT7g9q GQzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694092011; x=1694696811; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=s8i7Jhe9BViu4aMBPbLDU6gPTC4qxWjzqtUEiNX0CLE=; b=fBkSCfRrVH27uXhbRgnXnKHFT21D/kypuD7gYrYsU5v/Wd8KKtgw3Sq7I5YLC9EIX7 TcmcZ2g6fBnJ3sZj9CbMTE7L/Et4VqnrLKCahHteahs2i7U19kz17YOx7MtCDWKW90UL Clk5Be1N/OHzABU+QWoyqwrjzzCAI1Ylg0NOi7G5XbgjjQQWJKcIqdviFGlIi3TSsr30 rxCtuydPzp+TNDYZV8ArtSgCEAlo2qnkSek736aHFisX3O52OZRSjJR5q8G6yIKQ2GlO NWE93DjKoUFKeGceSihAzYqVwzRz0a3ehAO1fOgG4DP5RY8xr93NhJ6mTzy/jB21+rjk 9fCQ== X-Gm-Message-State: AOJu0YznCYZ6jO4BAwx+41GiMjI2bCsepYHlF6P2dzLutSp5j5ohZ5rW SgQkt5r9rYjs30B0XZfEWMZ+eagNqEc= X-Google-Smtp-Source: AGHT+IEevIijC6sdQUSkGV/Z3VUPtyWQWTY5lHP0KbgSWdAqKt9EUNHzvlQ9r/pckae1MLXf3sJctovYv6g= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:33dd:e36e:b6dc:1a69]) (user=glider job=sendgmr) by 2002:a25:d20f:0:b0:d78:215f:ba5f with SMTP id j15-20020a25d20f000000b00d78215fba5fmr466389ybg.9.1694092010883; Thu, 07 Sep 2023 06:06:50 -0700 (PDT) Date: Thu, 7 Sep 2023 15:06:42 +0200 In-Reply-To: <20230907130642.245222-1-glider@google.com> Mime-Version: 1.0 References: <20230907130642.245222-1-glider@google.com> X-Mailer: git-send-email 2.42.0.283.g2d96d420d3-goog Message-ID: <20230907130642.245222-2-glider@google.com> Subject: [PATCH 2/2] kmsan: prevent optimizations in memcpy tests From: Alexander Potapenko To: glider@google.com, dvyukov@google.com, elver@google.com, akpm@linux-foundation.org, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com X-Rspamd-Queue-Id: D4867A003F X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 9ge3hq1df4rrrucgdzmtxaqnuww4imfj X-HE-Tag: 1694092011-318038 X-HE-Meta: U2FsdGVkX18F1Y2xEWKJvdt57fBCVP7mYULl3IzdOqWulBhHLyOpg0ed15QeAYD7WMjij3JBpwF2we7TQJ/NgCZHXICp/WMmAGVny9sMp4qamzqvFrlC/ahO9C2j3BPjjSivud3b51+n7z1JMYPJOzogUUZiyddlrruur5eYthYxfvMZ8+aUNCHXCactLGKqmFKYNzqdyOE3MKUYW5dYsAI4c1r7htWLIuHIhIKjzD6OenmNAJ7jM+xoSOS+/cTGrCKkqFU+02XGkutFI1yImo67jb0sUtg0SXf2r6xMLyHLzsCnC1KLqeXrUzxr/DyuQViA5x7VEMXMeqxgX1UH8h0wLS6dM4CrnT/2jb3cw736ytKL1Pbb6DKdZzR3+d2bLYATgwJOWQw1gzRt+d48YUgBX5JCgzlDPaKbxIPtDYtedjYLBlKWKJLSTb+SvhGrm9PBd0OsilrObMjmZ5SuILrmwdGlrDw9LzAotgYcEdwNvRI6Y98Qt1wJSyOieB8b953dbbqNRQB39IhUIzoHxp3QgrzDOE4w2Ci4rCg1RERa8y4TkZ/Kmcpwv9cC3wJ67in2U5KknjqTPetTQeS+IQk55n+lBiNd8lWXWfTfXqXIvtDKk7uzFmc1DD+hQtHqU1zkmNrsWdcb6FaVDFXR4DinySwV+CVl5PU9+O91wTZbF6NOiHVVhX4IPnchgDwcbMSVDfC1vl3WMhjiiB5yEvi5TyPUuI6yN+aid3gsXqarJR2WLVyXcn1R2dILOfmZgn52hmjYi2QcEXVsAgb0PJ4YDlglcUx+vzjzDDwB8+avyTd4vC/xfNY1I0Knd5qP7aHnsgt1QCHp15ftUsBV1JwtvbuFN4yPWLtmABXUOKNCqn91iN2FbJvnbQqXHESvgag/NA5C2LgEzTM/nXtOJU4TQzeAEpwl3VdysB3JMFuRf46IDpPQtRw2Dhn1TrANOUMv4mtJQfNctlK81Oq ZWqOumk7 9TYBo1Qqefaw3QFJCjERoXGZFQ0goAWpqIh7mqWZeQHRGpkVSWR38wWCQyAH0G7CPhpIxtxaup2msXWASjYUwCPaEUmccCzwe0e+q91n1OTBSORcqOyBuThoxzwlMInm7zFyhz8u43TPr9yyzlhmQQKXMElkQoQMMvVnYrhFVDsGIGwy322YZbQVQ8qirqvdNRzcPP5eLPyVdRDJRC5bJrgiK6g+ionfvBe9VlDm/YuJnXU47ayFBOMjBKAb8vsq4F4Or/gyqgWpy1Bl5B9X/cNXg75pTZ89zb14ywRl9bAK3zxIL8zvUkb7opAfn/XWfn/tURgNXqNYiKsX7q1lxnuVoVu0u/POxZMLBL62UJ0T6CpZxdGAJnAbmgwbSMPi8qd/2orx/lEzqPiojji7uBtG6cnpNOD7Ndba8Yh1tKVvRdc6g9S/EZ7w4lTyVFxpfDWn1gSES/kXYrSHua34ChsgcQPiekc//DXJd3iZ7+/Yu6DszwCqx/SRHeALCWuNNxytX6p1LvoAzp7M1USNRmMXNvYkUHp8KizuWoIaxRCS3b3OV26/qQfZcmZDB80iyuzdsVPra+Gd/k6/YAO/KeKZs71okt7GeWUGUP8UXV5YGcKg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Clang 18 learned to optimize away memcpy() calls of small uninitialized scalar values. To ensure that memcpy tests in kmsan_test.c still perform calls to memcpy() (which KMSAN replaces with __msan_memcpy()), declare a separate memcpy_noinline() function with volatile parameters, which won't be optimized. Also retire DO_NOT_OPTIMIZE(), as memcpy_noinline() is apparently enough. Signed-off-by: Alexander Potapenko --- mm/kmsan/kmsan_test.c | 37 ++++++++++++++----------------------- 1 file changed, 14 insertions(+), 23 deletions(-) diff --git a/mm/kmsan/kmsan_test.c b/mm/kmsan/kmsan_test.c index 312989aa2865c..0c32c917b489a 100644 --- a/mm/kmsan/kmsan_test.c +++ b/mm/kmsan/kmsan_test.c @@ -407,33 +407,25 @@ static void test_printk(struct kunit *test) KUNIT_EXPECT_TRUE(test, report_matches(&expect)); } -/* - * Prevent the compiler from optimizing @var away. Without this, Clang may - * notice that @var is uninitialized and drop memcpy() calls that use it. - * - * There is OPTIMIZER_HIDE_VAR() in linux/compier.h that we cannot use here, - * because it is implemented as inline assembly receiving @var as a parameter - * and will enforce a KMSAN check. Same is true for e.g. barrier_data(var). - */ -#define DO_NOT_OPTIMIZE(var) barrier() +/* Prevent the compiler from inlining a memcpy() call. */ +static noinline void *memcpy_noinline(volatile void *dst, + const volatile void *src, size_t size) +{ + return memcpy(dst, src, size); +} -/* - * Test case: ensure that memcpy() correctly copies initialized values. - * Also serves as a regression test to ensure DO_NOT_OPTIMIZE() does not cause - * extra checks. - */ +/* Test case: ensure that memcpy() correctly copies initialized values. */ static void test_init_memcpy(struct kunit *test) { EXPECTATION_NO_REPORT(expect); volatile int src; volatile int dst = 0; - DO_NOT_OPTIMIZE(src); src = 1; kunit_info( test, "memcpy()ing aligned initialized src to aligned dst (no reports)\n"); - memcpy((void *)&dst, (void *)&src, sizeof(src)); + memcpy_noinline((void *)&dst, (void *)&src, sizeof(src)); kmsan_check_memory((void *)&dst, sizeof(dst)); KUNIT_EXPECT_TRUE(test, report_matches(&expect)); } @@ -451,8 +443,7 @@ static void test_memcpy_aligned_to_aligned(struct kunit *test) kunit_info( test, "memcpy()ing aligned uninit src to aligned dst (UMR report)\n"); - DO_NOT_OPTIMIZE(uninit_src); - memcpy((void *)&dst, (void *)&uninit_src, sizeof(uninit_src)); + memcpy_noinline((void *)&dst, (void *)&uninit_src, sizeof(uninit_src)); kmsan_check_memory((void *)&dst, sizeof(dst)); KUNIT_EXPECT_TRUE(test, report_matches(&expect)); } @@ -474,8 +465,9 @@ static void test_memcpy_aligned_to_unaligned(struct kunit *test) kunit_info( test, "memcpy()ing aligned uninit src to unaligned dst (UMR report)\n"); - DO_NOT_OPTIMIZE(uninit_src); - memcpy((void *)&dst[1], (void *)&uninit_src, sizeof(uninit_src)); + kmsan_check_memory(&uninit_src, sizeof(uninit_src)); + memcpy_noinline((void *)&dst[1], (void *)&uninit_src, + sizeof(uninit_src)); kmsan_check_memory((void *)dst, 4); KUNIT_EXPECT_TRUE(test, report_matches(&expect)); } @@ -498,8 +490,8 @@ static void test_memcpy_aligned_to_unaligned2(struct kunit *test) kunit_info( test, "memcpy()ing aligned uninit src to unaligned dst - part 2 (UMR report)\n"); - DO_NOT_OPTIMIZE(uninit_src); - memcpy((void *)&dst[1], (void *)&uninit_src, sizeof(uninit_src)); + memcpy_noinline((void *)&dst[1], (void *)&uninit_src, + sizeof(uninit_src)); kmsan_check_memory((void *)&dst[4], sizeof(uninit_src)); KUNIT_EXPECT_TRUE(test, report_matches(&expect)); } @@ -513,7 +505,6 @@ static void test_memcpy_aligned_to_unaligned2(struct kunit *test) \ kunit_info(test, \ "memset" #size "() should initialize memory\n"); \ - DO_NOT_OPTIMIZE(uninit); \ memset##size((uint##size##_t *)&uninit, 0, 1); \ kmsan_check_memory((void *)&uninit, sizeof(uninit)); \ KUNIT_EXPECT_TRUE(test, report_matches(&expect)); \