From patchwork Fri Nov 13 22:20:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904991 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B80AD697 for ; Fri, 13 Nov 2020 22:21:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 80E34206F9 for ; Fri, 13 Nov 2020 22:21:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Wogiwfd7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 80E34206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 639826B00A4; Fri, 13 Nov 2020 17:21:00 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5E6C26B00A6; Fri, 13 Nov 2020 17:21:00 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 487716B00A7; Fri, 13 Nov 2020 17:21:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id 083F86B00A4 for ; Fri, 13 Nov 2020 17:20:59 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A34172497 for ; Fri, 13 Nov 2020 22:20:59 +0000 (UTC) X-FDA: 77480816238.10.food92_4b16ab127312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 7E85B16A069 for ; Fri, 13 Nov 2020 22:20:59 +0000 (UTC) X-Spam-Summary: 1,0,0,2d3e53b98550f31f,d41d8cd98f00b204,3ygavxwokcjc1e4i5pbemc7ff7c5.3fdc9elo-ddbm13b.fi7@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2731:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3354:3865:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4118:4250:4321:4385:4425:4605:5007:6261:6653:6742:7901:8603:9025:9969:10004:10400:10450:10455:11026:11232:11473:11658:11914:12043:12295:12296:12297:12438:12555:12679:12701:12737:12895:12986:13161:13229:14181:14394:14659:14721:19904:19999:21063:21080:21365:21444:21450:21451:21627:21740:21939:21966:21990:30003:30012:30054:30070,0,RBL:209.85.219.73:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04ygg4c6qxkjmxj94ufhty7 93ajzdyp X-HE-Tag: food92_4b16ab127312 X-Filterd-Recvd-Size: 7067 Received: from mail-qv1-f73.google.com (mail-qv1-f73.google.com [209.85.219.73]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:58 +0000 (UTC) Received: by mail-qv1-f73.google.com with SMTP id 12so7003482qvr.8 for ; Fri, 13 Nov 2020 14:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=VblPcwxF/Hme6lihpSqzEG9568rY0eTc9RDR6WgTVWw=; b=Wogiwfd7B8TITgy/dJX2SuNVBCGj1sMmA/wWfqUVUW625r77eKdpvQaDaSu3aBij18 zEhjdHmrCW/gZ7A+s95KqIfE8DNCf76WA+Nww0tJUkPRyTX4kRL6suTfbHi2qlyVafam rV+hV43GvnYUfMMgdDZbuxRdthx5R+R5/Y2EHztggMs7SRvCIai8cg3H8bvmqhAi17Nt 36le6MqSx5RpWtq05dKx2kbxkwESQg8uMTwlplCYYGRgGySvTwMQEijRKAHcWAwock2d 3rNzJuPXD5wPMUIWUOa1SBHN9tz+5KvOK3ae7A6N6hVkTe9bRJuMUVjKXK9evatpgUtX DvbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=VblPcwxF/Hme6lihpSqzEG9568rY0eTc9RDR6WgTVWw=; b=BDogbmgo+fAx6Y35NMOY3sCQd0sVJBTISIs+ekDuoF46msT6woagn8K9W/E/qhGFJk MAHSJIbrrmX+7MRGEFI71exTpDpTJx2dyL74eFDk/GJpVldEjPN+WWfgim6Mj8FUXJOo bQS+WUt7MKH7GE7giE5LB04FHS4gqYqZUzqGwgFquyL6i0UF8SVBQkilRZ1txXPAUhko 5TGjOftEZ8BazVDTcDc7bwjwxkxLUBHe9VcAOG13Rr3BtQY6CncW3zEUKQNXBa8aZ+m/ Rf2aVgDCwoNV+Q33lM42696cx+1Ugjhl9of4saNu4hBKFsHH5SDyRAHhxLzA1JI4PwCD 79Gg== X-Gm-Message-State: AOAM532PBMLSqyyITLhP57TeMcaFRf7hCcPpwkcl064QhYI/27BEPY4v ojtHRfgcwdezGxJy3ikIf7JQhNgpUrOCzYYs X-Google-Smtp-Source: ABdhPJxPL1Ew5OnNzRmKgE563d6wfxOAtulSj3z/jp9Yh8hehdEomONqkappFCtKfgmpNirQUmmu+Nt5HUJLowo+ X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:9e53:: with SMTP id z19mr4596536qve.23.1605306058182; Fri, 13 Nov 2020 14:20:58 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:08 +0100 In-Reply-To: Message-Id: <6f0a1e72783ddac000ac08e7315b1d7c0ca4ec51.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 18/19] kasan, mm: allow cache merging with no metadata From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov , Vincenzo Frascino X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov --- include/linux/kasan.h | 21 +++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 3 ++- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 16cf53eac29b..173a8e81d001 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -81,17 +81,30 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static __always_inline bool kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline bool kasan_enabled(void) { return true; } -#endif + +#endif /* CONFIG_KASAN_HW_TAGS */ + +slab_flags_t __kasan_never_merge(void); +static __always_inline slab_flags_t kasan_never_merge(void) +{ + if (kasan_enabled()) + return __kasan_never_merge(); + return 0; +} void __kasan_unpoison_range(const void *addr, size_t size); static __always_inline void kasan_unpoison_range(const void *addr, size_t size) @@ -238,6 +251,10 @@ static inline bool kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(void) +{ + return 0; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} diff --git a/mm/kasan/common.c b/mm/kasan/common.c index cf874243efab..a5a4dcb1254d 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -87,6 +87,17 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(void) +{ + if (kasan_stack_collection_enabled()) + return SLAB_KASAN; + return 0; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; diff --git a/mm/slab_common.c b/mm/slab_common.c index 0b5ae1819a8b..075b23ce94ec 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -54,7 +55,7 @@ static DECLARE_WORK(slab_caches_to_rcu_destroy_work, */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_KASAN) + SLAB_FAILSLAB | kasan_never_merge()) #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT)