From patchwork Fri Feb 19 00:22:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 12094573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37EF2C433DB for ; Fri, 19 Feb 2021 00:22:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 94B3164EC0 for ; Fri, 19 Feb 2021 00:22:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 94B3164EC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E3C8F6B0005; Thu, 18 Feb 2021 19:22:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DEB3C6B006C; Thu, 18 Feb 2021 19:22:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB2D16B006E; Thu, 18 Feb 2021 19:22:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0233.hostedemail.com [216.40.44.233]) by kanga.kvack.org (Postfix) with ESMTP id B6EAD6B0005 for ; Thu, 18 Feb 2021 19:22:32 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 7ABE475B0 for ; Fri, 19 Feb 2021 00:22:32 +0000 (UTC) X-FDA: 77833116144.19.149D152 Received: from mail-qt1-f201.google.com (mail-qt1-f201.google.com [209.85.160.201]) by imf12.hostedemail.com (Postfix) with ESMTP id 3AA2F130 for ; Fri, 19 Feb 2021 00:22:28 +0000 (UTC) Received: by mail-qt1-f201.google.com with SMTP id z3so2246880qtv.20 for ; Thu, 18 Feb 2021 16:22:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=U/iWzYWPjCN+XW1A2BqmtUjlIIZumhGvP/yo+EmOxYg=; b=G7bAXsSfm+ZNftqz1PsApsrrwJT8rMmTOEArec4kk5JfXkHSn5yAxj9DM4umoFPnWp RGmtL7d5uL9H7zRtAjVAbKIkaAF8ufdT94u9akurg8SFStJwtepBldAe7CzJ3uhcaMzp 9WEo3HH6yOSmwI+Lcgdd4NIQ6NS6Js00aG5ddncE4w0J4+Ks2433wNoGqaOZDjsXit0l tdPTVlwbwaNJ+GN5c/McwoIy7K8gu4vVTR2V/16/tycbQ9v2qp4R93lfHwogeNIl0Kjn NzIwqCWwxxy9qLwZ9Mf402aMCg10rkgFlP6McJvh2yH2Fz6dXez8C9TT4sRw8BpZBGe+ MWNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=U/iWzYWPjCN+XW1A2BqmtUjlIIZumhGvP/yo+EmOxYg=; b=EbdLQHgGU3KbKaMDuoIUVLLl7iaWbS3DZuZJzARpSU4JyOBeC4+n/zDDwieRwqmYG3 56MLXb8CthYQSLBexdqAA29dl+dGBwt6J/dvzfnJcF0zfhm34p/Tzsdw9akU4SfWMofD BnDnga/MJqdTlq/P8Bjw83QLQ4qxeNVAzT6+1g/UCcKg+Qc5epZ+jTf8ad7OQWHeNyRC X4jg1qYYvMqieYCMSeYx0vcsPwnZ6D7BJWd6W9IK5Lp/VhC70mrCW+ZRPWuqppk4iqJl H6Av6SWWNUgv3W/K6UB7MfHhRiZO3+oOIejUEMkuGae/vZBFC6BdX4/F2M5xaPyJRD5Q qDIA== X-Gm-Message-State: AOAM531fyI13SoVLPtWa6i1atlrPcngjIGkmuJ18WIdvRx82zo+/9LCZ eCqbSPR5EphAjaaazDRRtVIvRu8U5BgPwNFQ X-Google-Smtp-Source: ABdhPJyFsVJtAau0Mc2BHt611wI2ENN7q1M05EoS8ziF6kD66XPpoztxHVTEQqqAY5LyC/zaszm81323HxFZMODP X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:2d89:512e:587f:6e72]) (user=andreyknvl job=sendgmr) by 2002:a0c:8b8a:: with SMTP id r10mr6826124qva.52.1613694151028; Thu, 18 Feb 2021 16:22:31 -0800 (PST) Date: Fri, 19 Feb 2021 01:22:23 +0100 Message-Id: Mime-Version: 1.0 X-Mailer: git-send-email 2.30.0.617.g56c4b15f3c-goog Subject: [PATCH v2 1/2] kasan: initialize shadow to TAG_INVALID for SW_TAGS From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Peter Collingbourne , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , Christoph Hellwig , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3AA2F130 X-Stat-Signature: 7f316e1cgxmkiqqt5uaqj97jnkjo8jz4 Received-SPF: none (flex--andreyknvl.bounces.google.com>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from="<3xwQvYAoKCJQyB1F2M8BJ94CC492.0CA96BIL-AA8Jy08.CF4@flex--andreyknvl.bounces.google.com>"; helo=mail-qt1-f201.google.com; client-ip=209.85.160.201 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1613694148-71445 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, KASAN_SW_TAGS uses 0xFF as the default tag value for unallocated memory. The underlying idea is that since that memory hasn't been allocated yet, it's only supposed to be dereferenced through a pointer with the native 0xFF tag. While this is a good idea in terms on consistency, practically it doesn't bring any benefit. Since the 0xFF pointer tag is a match-all tag, it doesn't matter what tag the accessed memory has. No accesses through 0xFF-tagged pointers are considered buggy by KASAN. This patch changes the default tag value for unallocated memory to 0xFE, which is the tag KASAN uses for inaccessible memory. This doesn't affect accesses through 0xFF-tagged pointer to this memory, but this allows KASAN to detect wild and large out-of-bounds invalid memory accesses through otherwise-tagged pointers. This is a prepatory patch for the next one, which changes the tag-based KASAN modes to not poison the boot memory. Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 14f72ec96492..44c147dae7e3 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -30,7 +30,8 @@ struct kunit_kasan_expectation { /* Software KASAN implementations use shadow memory. */ #ifdef CONFIG_KASAN_SW_TAGS -#define KASAN_SHADOW_INIT 0xFF +/* This matches KASAN_TAG_INVALID. */ +#define KASAN_SHADOW_INIT 0xFE #else #define KASAN_SHADOW_INIT 0 #endif From patchwork Fri Feb 19 00:22:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 12094575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E310C433E6 for ; Fri, 19 Feb 2021 00:22:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7814F64EC0 for ; Fri, 19 Feb 2021 00:22:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7814F64EC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 00A5B6B006C; Thu, 18 Feb 2021 19:22:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EFCF96B006E; Thu, 18 Feb 2021 19:22:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEE1E6B0070; Thu, 18 Feb 2021 19:22:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0055.hostedemail.com [216.40.44.55]) by kanga.kvack.org (Postfix) with ESMTP id C9F036B006C for ; Thu, 18 Feb 2021 19:22:34 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9BADF6C3E for ; Fri, 19 Feb 2021 00:22:34 +0000 (UTC) X-FDA: 77833116228.05.3510B15 Received: from mail-qk1-f201.google.com (mail-qk1-f201.google.com [209.85.222.201]) by imf09.hostedemail.com (Postfix) with ESMTP id 3A0F4600249E for ; Fri, 19 Feb 2021 00:22:31 +0000 (UTC) Received: by mail-qk1-f201.google.com with SMTP id d66so2450399qke.9 for ; Thu, 18 Feb 2021 16:22:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=hrKEbBMgsE8JTGg8XVXnIk1GaeWuGUQgi5pPA+IiEQ4=; b=ak7Lim4r6UGdoX3Z1PzU0EoMlkFZ/AwbBJ5Xeb7WEfkr3zrhXbt30Zi0mxiTLr1gM3 oOJsBHdsz3YfynCuuPUwXxPVzGpQsbNeL10ozod7WcGa112DcPa9aYLmwJqLAYf+23aN 1xOTtE1TMka7NPRtgbayLjP6VxKPa+JTnee8iqw0eH4kyTV1qOEYf/ZF/gICBeh4RiFv 8aCTOJMmkkZSZ2DOYbIGhlXMUqZZ2kCZi+CutvvE8liFmyynXXxxhPhSdP9MmkV+xM2v 2VssB8mFg+9TgHPgUmX4078E4QPksWkhXQJnJfj+fr2HMge/AdaAn31PTd+odLG3IxQP qNxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=hrKEbBMgsE8JTGg8XVXnIk1GaeWuGUQgi5pPA+IiEQ4=; b=S49tdy8p6K/2ovOV1VAsZ/miGt4fBgty1+VsJvJkCk5/HTm5rU7Yky7Q60G+OhpCqM xy0sGw4Jj7FaTdx9P0gFZKR0iKApxQ43943RTQGbNWXhS3S5cuCMcIFA1oNKLRGQVdOR +a4GrMM1iGShvLQOo2ygeY2S5QyLfoVSsUeDwyx+wNWzG5z90ddT7tEMLjDk/+ojOPVj BQXkU6QrNqrzPOJuV564ecwUFGAQiyNLLQ6zHFE9MK7hFsGmjj/wM0ECbvpcJA4MXh3W uaIR4pqt8rAoZod+L9OwyEXnx6fUnysdALP2T2flszki5QyllEH52bOZ67yJd94cYtyi 3isQ== X-Gm-Message-State: AOAM5306AqoanB6Pg22QPf9qv4+K/bSGHu1jdZ6aXzzQV8egGlo2Vlf5 c9dPVJSHl/0nEQbTpAM9yabdWJpbXx3biIfA X-Google-Smtp-Source: ABdhPJw4Yi61ckpHWJpC6IgjpY8vd4yFN412cUOB1rVjce5SvKJbPNZGR7ts+bVgdWqFpnTQ6UbzOjNQeAja1IV2 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:2d89:512e:587f:6e72]) (user=andreyknvl job=sendgmr) by 2002:a0c:9e13:: with SMTP id p19mr7008653qve.12.1613694153272; Thu, 18 Feb 2021 16:22:33 -0800 (PST) Date: Fri, 19 Feb 2021 01:22:24 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.30.0.617.g56c4b15f3c-goog Subject: [PATCH v2 2/2] mm, kasan: don't poison boot memory with tag-based modes From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Peter Collingbourne , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , Christoph Hellwig , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Stat-Signature: opk8x3y7arbecczs9ebnzozmsxxcc63i X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 3A0F4600249E Received-SPF: none (flex--andreyknvl.bounces.google.com>: No applicable sender policy available) receiver=imf09; identity=mailfrom; envelope-from="<3yQQvYAoKCJY0D3H4OADLB6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--andreyknvl.bounces.google.com>"; helo=mail-qk1-f201.google.com; client-ip=209.85.222.201 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1613694151-929541 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: During boot, all non-reserved memblock memory is exposed to page_alloc via memblock_free_pages->__free_pages_core(). This results in kasan_free_pages() being called, which poisons that memory. Poisoning all that memory lengthens boot time. The most noticeable effect is observed with the HW_TAGS mode. A boot-time impact may potentially also affect systems with large amount of RAM. This patch changes the tag-based modes to not poison the memory during the memblock->page_alloc transition. An exception is made for KASAN_GENERIC. Since it marks all new memory as accessible, not poisoning the memory released from memblock will lead to KASAN missing invalid boot-time accesses to that memory. With KASAN_SW_TAGS, as it uses the invalid 0xFE tag as the default tag for all memory, it won't miss bad boot-time accesses even if the poisoning of memblock memory is removed. With KASAN_HW_TAGS, the default memory tags values are unspecified. Therefore, if memblock poisoning is removed, this KASAN mode will miss the mentioned type of boot-time bugs with a 1/16 probability. This is taken as an acceptable trafe-off. Internally, the poisoning is removed as follows. __free_pages_core() is used when exposing fresh memory during system boot and when onlining memory during hotplug. This patch adds a new FPI_SKIP_KASAN_POISON flag and passes it to __free_pages_ok() through free_pages_prepare() from __free_pages_core(). If FPI_SKIP_KASAN_POISON is set, kasan_free_pages() is not called. All memory allocated normally when the boot is over keeps getting poisoned as usual. Reviewed-by: Catalin Marinas Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Only drop memblock poisoning for tag-based KASAN modes. --- mm/page_alloc.c | 45 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 34 insertions(+), 11 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0b55c9c95364..c89e7b107514 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -108,6 +108,17 @@ typedef int __bitwise fpi_t; */ #define FPI_TO_TAIL ((__force fpi_t)BIT(1)) +/* + * Don't poison memory with KASAN (only for the tag-based modes). + * During boot, all non-reserved memblock memory is exposed to page_alloc. + * Poisoning all that memory lengthens boot time, especially on systems with + * large amount of RAM. This flag is used to skip that poisoning. + * This is only done for the tag-based KASAN modes, as those are able to + * detect memory corruptions with the memory tags assigned by default. + * All memory allocated normally after boot gets poisoned as usual. + */ +#define FPI_SKIP_KASAN_POISON ((__force fpi_t)BIT(2)) + /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */ static DEFINE_MUTEX(pcp_batch_high_lock); #define MIN_PERCPU_PAGELIST_FRACTION (8) @@ -384,10 +395,15 @@ static DEFINE_STATIC_KEY_TRUE(deferred_pages); * on-demand allocation and then freed again before the deferred pages * initialization is done, but this is not likely to happen. */ -static inline void kasan_free_nondeferred_pages(struct page *page, int order) +static inline void kasan_free_nondeferred_pages(struct page *page, int order, + fpi_t fpi_flags) { - if (!static_branch_unlikely(&deferred_pages)) - kasan_free_pages(page, order); + if (static_branch_unlikely(&deferred_pages)) + return; + if (!IS_ENABLED(CONFIG_KASAN_GENERIC) && + (fpi_flags & FPI_SKIP_KASAN_POISON)) + return; + kasan_free_pages(page, order); } /* Returns true if the struct page for the pfn is uninitialised */ @@ -438,7 +454,14 @@ defer_init(int nid, unsigned long pfn, unsigned long end_pfn) return false; } #else -#define kasan_free_nondeferred_pages(p, o) kasan_free_pages(p, o) +static inline void kasan_free_nondeferred_pages(struct page *page, int order, + fpi_t fpi_flags) +{ + if (!IS_ENABLED(CONFIG_KASAN_GENERIC) && + (fpi_flags & FPI_SKIP_KASAN_POISON)) + return; + kasan_free_pages(page, order); +} static inline bool early_page_uninitialised(unsigned long pfn) { @@ -1216,7 +1239,7 @@ static void kernel_init_free_pages(struct page *page, int numpages) } static __always_inline bool free_pages_prepare(struct page *page, - unsigned int order, bool check_free) + unsigned int order, bool check_free, fpi_t fpi_flags) { int bad = 0; @@ -1290,7 +1313,7 @@ static __always_inline bool free_pages_prepare(struct page *page, debug_pagealloc_unmap_pages(page, 1 << order); - kasan_free_nondeferred_pages(page, order); + kasan_free_nondeferred_pages(page, order, fpi_flags); return true; } @@ -1303,7 +1326,7 @@ static __always_inline bool free_pages_prepare(struct page *page, */ static bool free_pcp_prepare(struct page *page) { - return free_pages_prepare(page, 0, true); + return free_pages_prepare(page, 0, true, FPI_NONE); } static bool bulkfree_pcp_prepare(struct page *page) @@ -1323,9 +1346,9 @@ static bool bulkfree_pcp_prepare(struct page *page) static bool free_pcp_prepare(struct page *page) { if (debug_pagealloc_enabled_static()) - return free_pages_prepare(page, 0, true); + return free_pages_prepare(page, 0, true, FPI_NONE); else - return free_pages_prepare(page, 0, false); + return free_pages_prepare(page, 0, false, FPI_NONE); } static bool bulkfree_pcp_prepare(struct page *page) @@ -1533,7 +1556,7 @@ static void __free_pages_ok(struct page *page, unsigned int order, int migratetype; unsigned long pfn = page_to_pfn(page); - if (!free_pages_prepare(page, order, true)) + if (!free_pages_prepare(page, order, true, fpi_flags)) return; migratetype = get_pfnblock_migratetype(page, pfn); @@ -1570,7 +1593,7 @@ void __free_pages_core(struct page *page, unsigned int order) * Bypass PCP and place fresh pages right to the tail, primarily * relevant for memory onlining. */ - __free_pages_ok(page, order, FPI_TO_TAIL); + __free_pages_ok(page, order, FPI_TO_TAIL | FPI_SKIP_KASAN_POISON); } #ifdef CONFIG_NEED_MULTIPLE_NODES