From patchwork Fri Nov 13 22:20:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11905093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F339BC55ABD for ; Fri, 13 Nov 2020 22:43:32 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7393722252 for ; Fri, 13 Nov 2020 22:43:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="JXYuiSdX"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Wogiwfd7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7393722252 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=84K3XwNfIlb54hGqNTxIS8j99Gr2rSCf8AqQr3E+SJQ=; b=JXYuiSdXIxiYHtck0KT5NEh5c FLZ+Ei8PHvgpu8YxCLUGV+1WgdOTwxN/5PoaTb4fRWv2IbY92KG+MjN5qAxbVYKOqmvnX/+Do7CZY 3ZKWfUl6uquCdnotX4WWfxz4UQzuZkolZRhGGVzefJ59HnskD9DLxLuxxpnIUB8RHg30v8YSgtwo2 lghcKrTWBwXrNz2coNnN+ZglzUvOnlA4m9PiRY0a+zz2BAU9tiiVm/KWJtDjrMTHH/SWltLvuYJmi gFYmg2O1TTvQeixTKM9KyMR8+gQcquY3jhQ1pLvimnfj6DYTo2osQ+asA7dkdZ+b3DghY5kUYyGZs tnPlVY31g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdhmJ-0003Mn-0j; Fri, 13 Nov 2020 22:42:47 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdhRK-0001le-UR for linux-arm-kernel@lists.infradead.org; Fri, 13 Nov 2020 22:21:12 +0000 Received: by mail-qk1-x749.google.com with SMTP id q25so7579081qkm.17 for ; Fri, 13 Nov 2020 14:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=VblPcwxF/Hme6lihpSqzEG9568rY0eTc9RDR6WgTVWw=; b=Wogiwfd7B8TITgy/dJX2SuNVBCGj1sMmA/wWfqUVUW625r77eKdpvQaDaSu3aBij18 zEhjdHmrCW/gZ7A+s95KqIfE8DNCf76WA+Nww0tJUkPRyTX4kRL6suTfbHi2qlyVafam rV+hV43GvnYUfMMgdDZbuxRdthx5R+R5/Y2EHztggMs7SRvCIai8cg3H8bvmqhAi17Nt 36le6MqSx5RpWtq05dKx2kbxkwESQg8uMTwlplCYYGRgGySvTwMQEijRKAHcWAwock2d 3rNzJuPXD5wPMUIWUOa1SBHN9tz+5KvOK3ae7A6N6hVkTe9bRJuMUVjKXK9evatpgUtX DvbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=VblPcwxF/Hme6lihpSqzEG9568rY0eTc9RDR6WgTVWw=; b=W3EiQkCGHEMg+3Zm1GT0tJKV0qHOZGo4P/i0HSR0JKv4r1i3nw4+51Ikr0nl5yYbSx gV2tLKoNfIc8dSBkcfPkKE4dG/S3RpAOxkX3f83++4iV/iGhU1+TL66jK9ri8rpY/AjF u0kpvv9Zm2fP9o3QMma4eV2czgzHOOWS7BTVfoGYdom9drXBvnsDLOzHKQ7Xcev1AC4Z sE05UmgwSLh6fsKYGZjdEq2IucNw5wDXdhTDw6EQ88TSlr7HCYMQLe1msaEg7XQSL0w1 Lipkzfp2tdEYP4KwM1QWPUyzEPrU4/sNA1FlohxU9C+0m0teuRV97euZstJli27zO4Cu jkhQ== X-Gm-Message-State: AOAM530py9HXjbAyweejSS4jgrBFWMU5SE+IS5alZGGBcto6FDtM6Lgb T1WsVFWuhdNUl/t6yAaelvBLX+UAm8tNWyog X-Google-Smtp-Source: ABdhPJxPL1Ew5OnNzRmKgE563d6wfxOAtulSj3z/jp9Yh8hehdEomONqkappFCtKfgmpNirQUmmu+Nt5HUJLowo+ X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:9e53:: with SMTP id z19mr4596536qve.23.1605306058182; Fri, 13 Nov 2020 14:20:58 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:08 +0100 In-Reply-To: Message-Id: <6f0a1e72783ddac000ac08e7315b1d7c0ca4ec51.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 18/19] kasan, mm: allow cache merging with no metadata From: Andrey Konovalov To: Andrew Morton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201113_172107_556530_46F19D89 X-CRM114-Status: GOOD ( 17.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arm-kernel@lists.infradead.org, Marco Elver , Catalin Marinas , Kevin Brodsky , Will Deacon , Branislav Rankov , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexander Potapenko , Evgenii Stepanov , Andrey Konovalov , Andrey Ryabinin , Vincenzo Frascino , Dmitry Vyukov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov --- include/linux/kasan.h | 21 +++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 3 ++- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 16cf53eac29b..173a8e81d001 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -81,17 +81,30 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static __always_inline bool kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline bool kasan_enabled(void) { return true; } -#endif + +#endif /* CONFIG_KASAN_HW_TAGS */ + +slab_flags_t __kasan_never_merge(void); +static __always_inline slab_flags_t kasan_never_merge(void) +{ + if (kasan_enabled()) + return __kasan_never_merge(); + return 0; +} void __kasan_unpoison_range(const void *addr, size_t size); static __always_inline void kasan_unpoison_range(const void *addr, size_t size) @@ -238,6 +251,10 @@ static inline bool kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(void) +{ + return 0; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} diff --git a/mm/kasan/common.c b/mm/kasan/common.c index cf874243efab..a5a4dcb1254d 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -87,6 +87,17 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(void) +{ + if (kasan_stack_collection_enabled()) + return SLAB_KASAN; + return 0; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; diff --git a/mm/slab_common.c b/mm/slab_common.c index 0b5ae1819a8b..075b23ce94ec 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -54,7 +55,7 @@ static DECLARE_WORK(slab_caches_to_rcu_destroy_work, */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_KASAN) + SLAB_FAILSLAB | kasan_never_merge()) #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT)