From patchwork Mon Nov 23 20:14:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11926705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BA84C388F9 for ; Mon, 23 Nov 2020 20:40:05 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0A86220721 for ; Mon, 23 Nov 2020 20:40:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="RySbyPV6"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Q7pV78bn"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Va6STH1a" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A86220721 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NJ/MSjv1P9dQvVDbCBuOym1GjqOiZPU9ABIgJ62h6EE=; b=RySbyPV6rK85YZ+tx0gqeiI/i fSjy+Z7qqAT4qQvnQ9V/GAYIVvTkl1kwRvlJby/VicffiMV56ConPGIzWh5KxzffbarV6TuDepVOz Ibzgcf4/qHuWZOJgyCGz6SQ43Me78fgXyJPn6FdAHgg7VtZyLIWCe4ndc50iU/WId1d8gNWSV/zTz bKsOmjYCt8UEf+xrXf1oLBJuurzxAMYoRyZ1QOpiSAlsiy8LFpR0HvgMyVzaQmPr1S4mDmj6ZfF1W XM5lvhWlyZ4NnUPQkCKlCCEj2Aw4Qih/qx0iEOwWallFgRjJqfdm5RWQXGOMGTTDCD0vCkmWS7Sl4 8EmgYz8kQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1khIcH-0004Ze-Ek; Mon, 23 Nov 2020 20:39:17 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1khIO2-0006KM-P1 for linux-arm-kernel@merlin.infradead.org; Mon, 23 Nov 2020 20:24:34 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=jlSh18JA+tyc0YTMYNGOzncWOn17Ui2VWegewVhQQAI=; b=Q7pV78bncCTa3GroED4vJlpL75 7FoS4oPDHxQmtMK4zqiP982xFXinoZsbAAIIpVMWZoddDKua6bDI1cxkCRvwIj7xlfD0oIoT+yoCr SZsxxjcnFPUusZWDr8rjuA3zitocffIWFCHNqnGaoXoXgtBCuY2qZQRoPxZp5sBrbpgExYxSo/PkT Za3cLFB2zrZv/2j4G5rflLVR0nfc7mDUVISKxmk6Ha2KQmtLNH27MZs2fxkZKP4eJsycR/VcNqOQc vAtk/OkMWfvhqhX+kannezBz6lDOUT0DXnW74KxQDxzYdrqAIcjxto/JkcWt9rDbu077zMTHq5hi/ 9Nu9RLNQ==; Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1khIFU-0006PT-JS for linux-arm-kernel@lists.infradead.org; Mon, 23 Nov 2020 20:15:51 +0000 Received: by mail-qt1-x84a.google.com with SMTP id y5so14465900qtb.13 for ; Mon, 23 Nov 2020 12:15:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=jlSh18JA+tyc0YTMYNGOzncWOn17Ui2VWegewVhQQAI=; b=Va6STH1aX4wYunU0iMuz4BvFHnwWptWCMWXmt/xofeqd3owQuGpwtvjYit4rl6mlW2 hhm5iDeTzV3r8r1qtqi5fdwTQs/TSfD/bCoDouUVq/mqil3HrggfywpVxpheL6ZEWpIl zDXMe40hvq88yS3PuIjxsWaxgjZyKl/B05KVNcChbJkk49tAW2TDEREdWyEig5SkLtvs 5K5C3H1Yizp2NapbQ8UOeKQZdiMwc5N0OfbB8Dc3yzVaEvt/fAhl005CJycepaLAgkY1 zyVIgqpv8eZepV66RBshT0P3UBs6jkxe5tAKTYnNSfeugX8uOp+AF7kVSkrFVfKZ4WBw kmbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=jlSh18JA+tyc0YTMYNGOzncWOn17Ui2VWegewVhQQAI=; b=pqu6hRp0ka7PDlq0j4EUb/7MT6hZuuD08KHE5P+7Qt9p/A/n+kdDASlxBidDsBPrUs PrA9/7B1dgZGghoIApEjaGSdYVGXz0WVQL2vqDk2impiljYkzb9ftFSKqnVp1kR+Xx2j wwvmzc+425yMqecHnMo6jZojrjd/O1JWbhpzzd85wZqgMvXCi6QGHew/dWUw47NjiyW4 p/rAKDcryG+C2ggzpoMclaC0lUi+ud7j382gGFwaIQGghnIQnl0N5mVFKbhd3EH0EFi9 me33KDxtj2Q3w7pqOwialEUNS73lHyVHEIyqBc2C9kCXkg/UT/sOFnsUWksg7wFO7Px0 neag== X-Gm-Message-State: AOAM531J8hlIfLq9JuIv38ct0jHJQM9Jm24AnPtec3ea2IoZZGv15EGQ sg6ThX3jSJxlyV4DhJ+KUZJBheSTILoxB7Qu X-Google-Smtp-Source: ABdhPJxiz/yK/mWtOWOXNmT0lUD67I1dkQtTBgb4Q9Y4cmrtA6Kl6akYtyMC6LEbXAx67mmzam5gwyx0dWQN53nE X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:804a:: with SMTP id 68mr1172222qva.1.1606162536399; Mon, 23 Nov 2020 12:15:36 -0800 (PST) Date: Mon, 23 Nov 2020 21:14:48 +0100 In-Reply-To: Message-Id: <37497e940bfd4b32c0a93a702a9ae4cf061d5392.1606162397.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.454.gaff20da3a2-goog Subject: [PATCH mm v4 18/19] kasan, mm: allow cache merging with no metadata From: Andrey Konovalov To: Andrew Morton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201123_201545_123470_2F0DE1B7 X-CRM114-Status: GOOD ( 16.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arm-kernel@lists.infradead.org, Marco Elver , Catalin Marinas , Kevin Brodsky , Will Deacon , Branislav Rankov , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexander Potapenko , Evgenii Stepanov , Andrey Konovalov , Andrey Ryabinin , Vincenzo Frascino , Dmitry Vyukov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba --- include/linux/kasan.h | 21 +++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 3 ++- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 2610438120ce..83860aa4e89c 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -81,17 +81,30 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static __always_inline bool kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline bool kasan_enabled(void) { return true; } -#endif + +#endif /* CONFIG_KASAN_HW_TAGS */ + +slab_flags_t __kasan_never_merge(void); +static __always_inline slab_flags_t kasan_never_merge(void) +{ + if (kasan_enabled()) + return __kasan_never_merge(); + return 0; +} void __kasan_unpoison_range(const void *addr, size_t size); static __always_inline void kasan_unpoison_range(const void *addr, size_t size) @@ -238,6 +251,10 @@ static inline bool kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(void) +{ + return 0; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 249ccba1ecf5..da79f340f3a6 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -87,6 +87,17 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(void) +{ + if (kasan_stack_collection_enabled()) + return SLAB_KASAN; + return 0; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; diff --git a/mm/slab_common.c b/mm/slab_common.c index 0b5ae1819a8b..075b23ce94ec 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -54,7 +55,7 @@ static DECLARE_WORK(slab_caches_to_rcu_destroy_work, */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_KASAN) + SLAB_FAILSLAB | kasan_never_merge()) #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT)