From patchwork Tue Nov 10 22:20:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87D9DC55ABD for ; Tue, 10 Nov 2020 22:33:01 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 095162068D for ; Tue, 10 Nov 2020 22:33:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="j3EIvcMR"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="B6QwfVBA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 095162068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GH96qVxwimkPf5RWVMX2M6oRGjq2ciq4zVxN8TLjPZY=; b=j3EIvcMRSwQcAKnQmRRQ26Kda 9/z8qOCmb0UJDgTAX2KLLutWltWlrbwB4FW4jnNwVKq0xPFcY4WYNN5LdmWe/4itKqZQZrRLWLhnd 9d4pyjrgUyp+UOs/oLkAiMPR0uY1D/squOmMox168ITP83h/rjfRLRIqMAhDuEqsSh24zmhfR75AK XTYuFOurRGz+oQsqLWpprzwUvpOJCLBMV/dLLufZWR7GgHHXZrIyRLTeQWh5UsBV8oxd560tgY64A iKYE26QqH5lxicAi+dCqXa8f7HdjpqYGHy7fMWg94Q7sDqLilJEGgUjExoNi7ggZvEDUnyV7Lztnx LNqUpIKWg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccAD-0004s2-S3; Tue, 10 Nov 2020 22:30:58 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0C-0000BZ-3r for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:20:46 +0000 Received: by mail-wm1-x349.google.com with SMTP id u207so1866177wmu.4 for ; Tue, 10 Nov 2020 14:20:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=SMV12gvgXWyti1wWozncOBF2sC/NzCwG5tM1W9vYuH4=; b=B6QwfVBAz7pkQ6yN3GQqARn2ii4rZdYmih8xg6h0+kbT6exEYUg4pd4Iohk94eyM1c mfyXHR7Pxl38SAXC+UEdfwWOGQNZDQyHqw0WxKX+1aldC5Nh33y9y1wUbwjCW1D0yajz 82APgf+BpR2WC3RpKvYiDkV8kFzJxpoXE7CFk42ZB+MpjTlBxJm0XvKG7IxF4CW0/YAU G5dkj7NUrzW4l90Z8ZrgBn308BRtQCP8KJTgZyqtc9ySrVSwXbVeMNlFKR3/b04gGUiT bClkE+hnLWgDfQGY229UlKDA4DC0pu/Iq0yDc0SmjQZcRbEmjzC5MnBcn/JR/TO1dWNI M5UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SMV12gvgXWyti1wWozncOBF2sC/NzCwG5tM1W9vYuH4=; b=cYOkY1jbgdp90RuxX1fZMMp4T1nHFiF65oLGLyaUGen9xJiSqzTEOGJSk53BvsToRx 8phrYmHrIkroP1N6r2h0bwCqhSchHfYzxPCO5mE4KoVQU0w5r7d1UdWJgS+v03S+hSxU gw0bpLAo7H+Phm7ljjHnSfDKYoGHR7tHiyAttOGe7qMMg9Tmv3G0UJilTPZnZ8M1A5sd 36LiC8r2xmyEEPKYKFJ2dKwuXL3tqUvGUrFjAG8RELOr5kawctBAfSNZa3vKId6oHtwp gUNd/MovbPgnh8vIbBGlDZpDQfF41QUXyrgPULSVqCc9qxVeN/JtGSvGhlst+obqCYwf 2m5g== X-Gm-Message-State: AOAM531a9SuENeNc91iRz9avOq+QEFPjHEiLm0vjvKYDRXoa4GmsCGyq abs2sJTyGkRwn30MoT+xZ73gHiA50NJMcCIf X-Google-Smtp-Source: ABdhPJxOgxlfhZQUpHjQ0Geh6GQq8dMdva8blwAYkX9VCtBB6RV5i3/j12u3n9DxhI2tM3WRQFA2yXCA2IHyyaf/ X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:5f45:: with SMTP id t66mr315013wmb.20.1605046831528; Tue, 10 Nov 2020 14:20:31 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:05 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 01/20] kasan: simplify quarantine_put call site From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172036_230522_451D2C1A X-CRM114-Status: GOOD ( 14.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move get_free_info() call into quarantine_put() to simplify the call site. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Iab0f04e7ebf8d83247024b7190c67c3c34c7940f Reviewed-by: Marco Elver --- mm/kasan/common.c | 2 +- mm/kasan/kasan.h | 5 ++--- mm/kasan/quarantine.c | 3 ++- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 2bb0ef6da6bd..5712c66c11c1 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -308,7 +308,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, kasan_set_free_info(cache, object, tag); - quarantine_put(get_free_info(cache, object), cache); + quarantine_put(cache, object); return IS_ENABLED(CONFIG_KASAN_GENERIC); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 21fe75c66f26..c2c40ec1544d 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -214,12 +214,11 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, #if defined(CONFIG_KASAN_GENERIC) && \ (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) -void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); +void quarantine_put(struct kmem_cache *cache, void *object); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); #else -static inline void quarantine_put(struct kasan_free_meta *info, - struct kmem_cache *cache) { } +static inline void quarantine_put(struct kmem_cache *cache, void *object) { } static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 580ff5610fc1..a0792f0d6d0f 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -161,11 +161,12 @@ static void qlist_free_all(struct qlist_head *q, struct kmem_cache *cache) qlist_init(q); } -void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache) +void quarantine_put(struct kmem_cache *cache, void *object) { unsigned long flags; struct qlist_head *q; struct qlist_head temp = QLIST_INIT; + struct kasan_free_meta *info = get_free_info(cache, object); /* * Note: irq must be disabled until after we move the batch to the From patchwork Tue Nov 10 22:20:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B552C388F7 for ; Tue, 10 Nov 2020 22:32:59 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16CFB2068D for ; Tue, 10 Nov 2020 22:32:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Te/1kfMU"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="CJ8BfCyP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16CFB2068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=AUcIfj0UN4t8X8f+YxJvipoXXhOg61fa9e8Q+JCF93A=; b=Te/1kfMUYFhJQVy9DksEaLxkM Sf8jUxfEWlj2EZAM4u5jPdQjuqaV1C3MYTeOzNosXO+WJFGkDPyKsZAftA+pkPtGNN4mSutsW+atM Uf2iodtxY552VNihdISFhN5EyNTgfHFsVZK2yzpSp6k7aNs6ElHtdfx+rsRgNJiMvNoTklkD7lgz1 /vvx9kav5pEEvo9cpou3EnUiDLMVrxzmdVd+rMHCl72vYcSyIxBF4FlJybZDpbfN8qZy2KLbSyD3i 0GMPLD6fx9l5SBpaHJSogNvW5Fp0rNjsDf2EdBN4VijqUnm/RL74UhWNGGPFOy6p5pFPUYqQSbkSu fFwgueEiQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccBX-0005Qu-TF; Tue, 10 Nov 2020 22:32:20 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0F-0000CI-3T for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:20:52 +0000 Received: by mail-wm1-x349.google.com with SMTP id 14so1386164wmg.1 for ; Tue, 10 Nov 2020 14:20:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=JDIZOBUdSYTr2qi1CrbHHoKD73faXUnEima7jMLorCA=; b=CJ8BfCyPbncwSwtW5Nci8JD1n3yUcmS6Qi/Zxu+AO5koJd203DZOBn0oO6Cv8JYj7C sLWi5+XypMyXhXDToFYMh5hN5cdjZulh23MD2DytKQfTKC5mwvnUM8j+mNToUnyFM1mN +nMOjhzrbVe7lVJ+TpZaeqRxbhBhrHdFviv4Q69TIi+QYWDXVgUHRliNYRiUIQIk2jlI lQa6dollahx6B5ha3GuwMdCmN8IKGeABTfSNFBIqeOHHs7zsOo+qiUTOofI3E43ndV36 ioZ/4b5sm7nyrNtTpoRwb7Vt3vS1rm8q8u6CnUtCuTFxjDgLDbDk8DH3AVh+f5nfv4/d lo2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=JDIZOBUdSYTr2qi1CrbHHoKD73faXUnEima7jMLorCA=; b=I5AlbSVMujq6Hj89/TIsfmfOTOcY3DnwZ55xAIeYK3CNBcthU3Iyqs8HzLST/NlRKv dHHKbuua95omsQbEgDL5W9D35CdMtsnDpHwLrpxuGjG4sdymTMayFTjsmx5q+qw2gOEx g38CAjpRqZsmAQdYiutbldkI1OZbSyp2oDBLrJQ1QckqstIsQ5JKWljEOa88IwucL3aM dwwYAFBcvaz4IFcrX10QcMNYkbZPwlY49KzjN6SrR8zMt+Zskzbn1BT5TBpys8Kqr5eZ 4IlsmB85ghldxwjdPuEAhQQwTsaXn4v6uwIYdIIcY12s7Y7TZyZt04G3N0Lb7i+mgjNn CnHg== X-Gm-Message-State: AOAM530Q01UURGvbuzaxPpqmS9uVVgKF4qXX55HqwrXJ957IaMKDFbK8 nk5m0iW5maHGkvMU/PjDm6ran8XCov9w2/1z X-Google-Smtp-Source: ABdhPJySAq2llW78O6AAjPm3restP8s7ySQayVVxoAa3/zeNctA1gJD24wEL3ercpCiB7rt/FGljH7Ku8auUbxvP X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:bac1:: with SMTP id k184mr290033wmf.76.1605046833741; Tue, 10 Nov 2020 14:20:33 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:06 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 02/20] kasan: rename get_alloc/free_info From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172039_844041_C1DCD19A X-CRM114-Status: GOOD ( 19.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Rename get_alloc_info() and get_free_info() to kasan_get_alloc_meta() and kasan_get_free_meta() to better reflect what those do and avoid confusion with kasan_set_free_info(). No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Ib6e4ba61c8b12112b403d3479a9799ac8fff8de1 Reviewed-by: Marco Elver --- mm/kasan/common.c | 16 ++++++++-------- mm/kasan/generic.c | 12 ++++++------ mm/kasan/hw_tags.c | 4 ++-- mm/kasan/kasan.h | 8 ++++---- mm/kasan/quarantine.c | 4 ++-- mm/kasan/report.c | 12 ++++++------ mm/kasan/report_sw_tags.c | 2 +- mm/kasan/sw_tags.c | 4 ++-- 8 files changed, 31 insertions(+), 31 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 5712c66c11c1..8fd04415d8f4 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -175,14 +175,14 @@ size_t kasan_metadata_size(struct kmem_cache *cache) sizeof(struct kasan_free_meta) : 0); } -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) +struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, + const void *object) { return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; } -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) +struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, + const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; @@ -259,13 +259,13 @@ static u8 assign_tag(struct kmem_cache *cache, const void *object, void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { - struct kasan_alloc_meta *alloc_info; + struct kasan_alloc_meta *alloc_meta; if (!(cache->flags & SLAB_KASAN)) return (void *)object; - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -345,7 +345,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) - kasan_set_track(&get_alloc_info(cache, object)->alloc_track, flags); + kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); return set_tag(object, tag); } diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index adb254df1b1d..d259e4c3aefd 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -329,7 +329,7 @@ void kasan_record_aux_stack(void *addr) { struct page *page = kasan_addr_to_page(addr); struct kmem_cache *cache; - struct kasan_alloc_meta *alloc_info; + struct kasan_alloc_meta *alloc_meta; void *object; if (!(page && PageSlab(page))) @@ -337,13 +337,13 @@ void kasan_record_aux_stack(void *addr) cache = page->slab_cache; object = nearest_obj(cache, page, addr); - alloc_info = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); /* * record the last two call_rcu() call stacks. */ - alloc_info->aux_stack[1] = alloc_info->aux_stack[0]; - alloc_info->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); + alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; + alloc_meta->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); } void kasan_set_free_info(struct kmem_cache *cache, @@ -351,7 +351,7 @@ void kasan_set_free_info(struct kmem_cache *cache, { struct kasan_free_meta *free_meta; - free_meta = get_free_info(cache, object); + free_meta = kasan_get_free_meta(cache, object); kasan_set_track(&free_meta->free_track, GFP_NOWAIT); /* @@ -365,5 +365,5 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK) return NULL; - return &get_free_info(cache, object)->free_track; + return &kasan_get_free_meta(cache, object)->free_track; } diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 0080b78ec843..70b88dd40cd8 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -66,7 +66,7 @@ void kasan_set_free_info(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); } @@ -75,6 +75,6 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); return &alloc_meta->free_track[0]; } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index c2c40ec1544d..db8a7a508121 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -148,10 +148,10 @@ struct kasan_free_meta { #endif }; -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object); -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object); +struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, + const void *object); +struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, + const void *object); void kasan_poison_memory(const void *address, size_t size, u8 value); diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index a0792f0d6d0f..0da3d37e1589 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -166,7 +166,7 @@ void quarantine_put(struct kmem_cache *cache, void *object) unsigned long flags; struct qlist_head *q; struct qlist_head temp = QLIST_INIT; - struct kasan_free_meta *info = get_free_info(cache, object); + struct kasan_free_meta *meta = kasan_get_free_meta(cache, object); /* * Note: irq must be disabled until after we move the batch to the @@ -179,7 +179,7 @@ void quarantine_put(struct kmem_cache *cache, void *object) local_irq_save(flags); q = this_cpu_ptr(&cpu_quarantine); - qlist_put(q, &info->quarantine_link, cache->size); + qlist_put(q, &meta->quarantine_link, cache->size); if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE)) { qlist_move_all(q, &temp); diff --git a/mm/kasan/report.c b/mm/kasan/report.c index ce06005d4052..0cac53a57c14 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -164,12 +164,12 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, static void describe_object(struct kmem_cache *cache, void *object, const void *addr, u8 tag) { - struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object); + struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); if (cache->flags & SLAB_KASAN) { struct kasan_track *free_track; - print_track(&alloc_info->alloc_track, "Allocated"); + print_track(&alloc_meta->alloc_track, "Allocated"); pr_err("\n"); free_track = kasan_get_free_track(cache, object, tag); if (free_track) { @@ -178,14 +178,14 @@ static void describe_object(struct kmem_cache *cache, void *object, } #ifdef CONFIG_KASAN_GENERIC - if (alloc_info->aux_stack[0]) { + if (alloc_meta->aux_stack[0]) { pr_err("Last call_rcu():\n"); - print_stack(alloc_info->aux_stack[0]); + print_stack(alloc_meta->aux_stack[0]); pr_err("\n"); } - if (alloc_info->aux_stack[1]) { + if (alloc_meta->aux_stack[1]) { pr_err("Second to last call_rcu():\n"); - print_stack(alloc_info->aux_stack[1]); + print_stack(alloc_meta->aux_stack[1]); pr_err("\n"); } #endif diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index aebc44a29e83..317100fd95b9 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -46,7 +46,7 @@ const char *get_bug_type(struct kasan_access_info *info) if (page && PageSlab(page)) { cache = page->slab_cache; object = nearest_obj(cache, page, (void *)addr); - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); for (i = 0; i < KASAN_NR_FREE_STACKS; i++) if (alloc_meta->free_pointer_tag[i] == tag) diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index dfe707dd8d0d..3bffb489b144 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -174,7 +174,7 @@ void kasan_set_free_info(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; u8 idx = 0; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY idx = alloc_meta->free_track_idx; @@ -191,7 +191,7 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; int i = 0; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { From patchwork Tue Nov 10 22:20:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E2D4C4742C for ; Tue, 10 Nov 2020 22:33:39 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D55852068D for ; Tue, 10 Nov 2020 22:33:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="OiqzgGQW"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="lCftT4m0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D55852068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=uyc5xVNA0s3jISE79iqTTZ7EPh6NsguRoX1c+mBDwAA=; b=OiqzgGQWaZZlDm8WmrLjMGciL aJ+HeLF3eRpyI4ijxQ+ifIQGbkuAPPGf1xQwhlEq01PPVdtVN2+Y6gfW427YsXXnIfdtryKjCTIcL 8lsB8qy0+3jKMcAc0NJ5QNfNQFtyfGZTf9Qmoa6ImmzndXm3jEHzZXUulK54d7sAN1nolcj/l1Z4A fYTda3ueAtoz0G33uKmGESDEWf+YzSEkOFbI+HUt662X7eGOsbt5claNBzf5oK2IbCv0ZmjM2Z0d6 +b/CMjJ2+v0YlFfdxT9RJn90qKzLiL/jdACGhJwu3XhCqmGCcjdyVu1Y/rd8LjnSPz54CCFJ38WGy beSzAzMDA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccCI-0005oL-Sp; Tue, 10 Nov 2020 22:33:07 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0J-0000Ds-QP for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:20:53 +0000 Received: by mail-wr1-x44a.google.com with SMTP id b6so6159678wrn.17 for ; Tue, 10 Nov 2020 14:20:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=gNopVjIT8PLD1roCXuMREiwLLvujReoC9+pcOjj05mE=; b=lCftT4m06a6889qXGQTef3H4ubHV8kKj9dGlz5f67eoPKWdz/PHnNnYEjABMQLk8d9 KObCel4cnvyGAfTLT3glV7CaPrDc0oVmARZammV+1pVoZ/2jSlMDKbk5t8ghqy2BrnAQ 6wiA3BXXhqVlgixBVMEYRnob8fIiyC/M+a6xp4u0aNMUCSSJaunS7d4ecvp+s9fgBTTl ILcUALwdqyXB6pkQZeuRw2BZ9ehON8RQWBZCFPbVG5/Tjjcf9C/MjxhSUXZvdV4Wx2sm tXxLD2ziia9ZukMCBDzC8zBSj5kuUeYjuML+1LS3YfaqORZ/tW8RY/g6NyBPCfrCuo0E RgsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=gNopVjIT8PLD1roCXuMREiwLLvujReoC9+pcOjj05mE=; b=LhbqDZ1U9W4I2dYvSdzmaRIwtLZ0CQ2+qUK4hGjLh9yQqfkZmy8yCE6jYMWlYgz23j y2WSOYDhkaTNGXtBHMVD8TfmB8UB3ovbBDuEYPlqB9efIGmZmiT0/bMPK/L6lmDM1rtP 4jHzKC/Hc3q9JmfSYAFNU4fWgIkBeoJz22cxjWIZAcfTuBW7ff80vrdbzsKhBuDcVjL2 A9T63valIhUD1FuMzPpi+smeBGetlT8CzbhlpW/ynLl+tI695divaXj7TNjfYm5FYirt wd2C1Vytk3U1tB4FzyrKbLvTzPYkw7IuLmmDS/YtbsFp3FovhI56RBzVTsC6RBXSAoge 4kTQ== X-Gm-Message-State: AOAM533JoZs19U/5jnYofcEKNWOA7FJBNJJ4u9nWE5thbyJW2+CI1ZNk gloA4uGaztDEkhdoMewUN+DX95yFsLl/EzsT X-Google-Smtp-Source: ABdhPJywHqOrMb83vobFQz6uQH+1U+IiE7bFzlSvWOmiu58rv8s7DM6ShUb7PXTFUABDWl/FwVrflkCpjtBdifZz X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:3803:: with SMTP id f3mr304885wma.14.1605046836014; Tue, 10 Nov 2020 14:20:36 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:07 +0100 In-Reply-To: Message-Id: <5302e6d48429465259bd0868a7dc357290a2e8a5.1605046662.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 03/20] kasan: introduce set_alloc_info From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172044_187211_6334CA26 X-CRM114-Status: GOOD ( 12.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add set_alloc_info() helper and move kasan_set_track() into it. This will simplify the code for one of the upcoming changes. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/I0316193cbb4ecc9b87b7c2eee0dd79f8ec908c1a Reviewed-by: Marco Elver --- mm/kasan/common.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 8fd04415d8f4..a880e5a547ed 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -318,6 +318,11 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } +static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) +{ + kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); +} + static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags, bool keep_tag) { @@ -345,7 +350,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) - kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); + set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); } From patchwork Tue Nov 10 22:20:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BB69C388F7 for ; Tue, 10 Nov 2020 22:34:25 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CF18E2068D for ; Tue, 10 Nov 2020 22:34:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="UooeAdqQ"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Lw7P323O" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF18E2068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Eiuwe9ZxlgzkK1ApszIEQegHUApEPLB8PwkcTwccVUE=; b=UooeAdqQ2x0JeX/ZPdAw/Kqss rO/U/AGBajgyNzrU5Go3eP5bqWH5jiXwUNt8GOLEqpUuOg+rxO2tUZAzPNtLKVc6TJGMHKNXuxlkJ P/RiE6J2d3I18eVRY+aS3v0sunVk1JwaYA+JCyNbbdwQNlMun0YGTeKGoI90l5cBIZxjs3CFvFMHa I5XzE4q88zUq61zydPoY0BA18wmSNHwg1bVaVwqwwlDlebw5HRlkpAGjzn6aFn/SZCvq58FOuZsna K76USJc8xxAi5DchUNDMrXMMSrUD81oEtYGKXGnuvPliOAOg6JvWItZsQgtM6R9VHJvEszdBHgsPd 5fe95MWwA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccBx-0005dT-Sr; Tue, 10 Nov 2020 22:32:46 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0J-0000EQ-MB for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:20:56 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id ca17so86048qvb.1 for ; Tue, 10 Nov 2020 14:20:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=tEdtRhABI1InYFpD5ltmIm/evviW/mtmaENcJYt1lmc=; b=Lw7P323OF/2i72kWQuJ4l8inThWAnZLsOW9ThJ12RKxjoJxpzU7FXD1KV+6+q4wFpg l/r4+g+lNCXzgzXgK1lygyyUKCtzwpfHl/3OlTnMhhMRfVDtB2PdYon8ei3bzhD2Vqtt Wo3uUi7dX96xLBACxd53V6ueiGeSZymzkpvnmaVcbe+7AHrMitDwaS52oyIZ68DrTouD k4aflKbowlt0EK7lot+JEbexzRr4RzOeeQC1v96jGdm7Fx8vBNCN0QhyaFYecRjubCKf y4BPPQEw0TRxyeg4FETviyB1G271O7T0w4aKkUqb+DvRw5ABtXAGXbngmp+oaS/px1Mw omsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tEdtRhABI1InYFpD5ltmIm/evviW/mtmaENcJYt1lmc=; b=Ijwm4sytMG9+U/0FeK3FtOcPRdk2ZZbi35NKOEMNu523A9Ig+3Tgmmt81PbZtEbUmW FucEKKgF4G1bNOSyrCu3P+m+l4KRn4Y5uCu1elJxO32DK53YntlXknfcpkbmpSqsJkoJ 6GRWU7U2gaSscOAE1sHi3aSGn8DW3Sq2lip3xazkXru8UULKZt+C+YWQgsHf3NT7vpA1 xjPKM0TaQPn2ltmkAPhrv1+RhuUwnQCdVPPUJsz37bH5tuO9xSokR+lzxIUsj7wzg0pa YdJND8sqnVzqljt+7Mf3jjt4PYgWEt2Ugv5Hee0qqF4yxYH/6zOksPAD76+cN48iajq6 78Bg== X-Gm-Message-State: AOAM533DmJe52XcnrG0w6XXCcPwchAYEtmfsRv/T9Y8IwLAVeiMIl5Zn hjpbuIgCGy2sNmV57x9FPBwUEZOeMiK9YIZl X-Google-Smtp-Source: ABdhPJxqgq6UfWHyoMS9rSSpgTD3+0B/UNYGVxM4mfhk/kYijiMY8uMFuj8WQnHSvWCojmtQAUsNukGtycyjiBdC X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:6214:6a7:: with SMTP id s7mr22500256qvz.2.1605046838323; Tue, 10 Nov 2020 14:20:38 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:08 +0100 In-Reply-To: Message-Id: <7e95d4739f5617b2c1acf52f37e01f1ca83750b5.1605046662.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 04/20] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172044_221861_7E8BDCDE X-CRM114-Status: GOOD ( 16.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There's a config option CONFIG_KASAN_STACK that has to be enabled for KASAN to use stack instrumentation and perform validity checks for stack variables. There's no need to unpoison stack when CONFIG_KASAN_STACK is not enabled. Only call kasan_unpoison_task_stack[_below]() when CONFIG_KASAN_STACK is enabled. Note, that CONFIG_KASAN_STACK is an option that is currently always defined when CONFIG_KASAN is enabled, and therefore has to be tested with #if instead of #ifdef. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/If8a891e9fe01ea543e00b576852685afec0887e3 Reviewed-by: Marco Elver Acked-by: Catalin Marinas --- arch/arm64/kernel/sleep.S | 2 +- arch/x86/kernel/acpi/wakeup_64.S | 2 +- include/linux/kasan.h | 10 ++++++---- mm/kasan/common.c | 2 ++ 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S index ba40d57757d6..bdadfa56b40e 100644 --- a/arch/arm64/kernel/sleep.S +++ b/arch/arm64/kernel/sleep.S @@ -133,7 +133,7 @@ SYM_FUNC_START(_cpu_resume) */ bl cpu_do_resume -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK mov x0, sp bl kasan_unpoison_task_stack_below #endif diff --git a/arch/x86/kernel/acpi/wakeup_64.S b/arch/x86/kernel/acpi/wakeup_64.S index c8daa92f38dc..5d3a0b8fd379 100644 --- a/arch/x86/kernel/acpi/wakeup_64.S +++ b/arch/x86/kernel/acpi/wakeup_64.S @@ -112,7 +112,7 @@ SYM_FUNC_START(do_suspend_lowlevel) movq pt_regs_r14(%rax), %r14 movq pt_regs_r15(%rax), %r15 -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK /* * The suspend path may have poisoned some areas deeper in the stack, * which we now need to unpoison. diff --git a/include/linux/kasan.h b/include/linux/kasan.h index f22bdef82111..b9b9db335d87 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -76,8 +76,6 @@ static inline void kasan_disable_current(void) {} void kasan_unpoison_memory(const void *address, size_t size); -void kasan_unpoison_task_stack(struct task_struct *task); - void kasan_alloc_pages(struct page *page, unsigned int order); void kasan_free_pages(struct page *page, unsigned int order); @@ -122,8 +120,6 @@ void kasan_restore_multi_shot(bool enabled); static inline void kasan_unpoison_memory(const void *address, size_t size) {} -static inline void kasan_unpoison_task_stack(struct task_struct *task) {} - static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} @@ -175,6 +171,12 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK +void kasan_unpoison_task_stack(struct task_struct *task); +#else +static inline void kasan_unpoison_task_stack(struct task_struct *task) {} +#endif + #ifdef CONFIG_KASAN_GENERIC void kasan_cache_shrink(struct kmem_cache *cache); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a880e5a547ed..a3e67d49b893 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -58,6 +58,7 @@ void kasan_disable_current(void) } #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +#if CONFIG_KASAN_STACK static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) { void *base = task_stack_page(task); @@ -84,6 +85,7 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) kasan_unpoison_memory(base, watermark - base); } +#endif /* CONFIG_KASAN_STACK */ void kasan_alloc_pages(struct page *page, unsigned int order) { From patchwork Tue Nov 10 22:20:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3BDAC388F7 for ; Tue, 10 Nov 2020 22:35:10 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5CA832068D for ; Tue, 10 Nov 2020 22:35:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Fpada9em"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="anCq4Edn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5CA832068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LTHoC41fL9smTyywo94pGiykyheP2NVHOoC/X1a5jBA=; b=Fpada9emivdorhuftn2sU92Mp X9jebl/gwV3YeqSB2B0PeWWC6IbvWzJMQLTd/gxTi3FeE0DcWBbpvwHBfU+zDTha/bFuwH8AusARi QUtAPsLDAm0KcJyBUdV8bTpn0+Ge1lZzF2sKhCIX4X2vtcST7Sppuwo8+WUIX6/Copk+WZwKnxqIp 4TnZvkfd5oq3B16H6xogYDIBpX+y5orTMt18rb4vLIuBI6WvmLomiyYZA/P/en4LiT+RZeXr0MoFQ cDhPdRpPG1xQPUlqtalm/2V8haNpZBq5j5EGAzrLnqK7wjtqZKgfq5ARHWF3f7IZHV7LKFr9qmwfx r+Ks0WtTg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccCf-0005wa-Mz; Tue, 10 Nov 2020 22:33:29 +0000 Received: from mail-lf1-x14a.google.com ([2a00:1450:4864:20::14a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0M-0000H3-B9 for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:20:56 +0000 Received: by mail-lf1-x14a.google.com with SMTP id c65so80687lfg.13 for ; Tue, 10 Nov 2020 14:20:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=iclGPRxOEu/Y72cezYx1VmP9mDKpdmhcr9SlAN2oUkA=; b=anCq4EdnMSfYibAYpKrbP8hc679KEJvqPcdaN5/ex3VCos1k3cAadzjksNjGffBqyb 9lgrUg03iHa8hf4SaVEeE+mhD1mCHcljJ1Z7Ec/ZghnpiPfwDV4+tZOz03wyyuzCkWN9 T4v3wHx6wv0e4wpOUdjnOJR7SC5EgpvpDGy+L1000n3KvhEK62jVSzkseAUDrGlOW3r0 g8ZvAhT1g1U4eFHzE8Pxdb2eONP3u7u7CYSfsjuF0yjCdSWIeOtzbawwk3RpvEMvyPf3 Z/XsxkAcX/vePoc584Itp3epvSDKWY3tEZhC9BFnXQ7YFDQwHJPCHiGaCKIKNknphdDi D6Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=iclGPRxOEu/Y72cezYx1VmP9mDKpdmhcr9SlAN2oUkA=; b=EVTVuhov16UjPuspHIsJNznjdhO/rRG0YoY93exPVWjcKuD91FhjKMuT286oYQaXfX tyEY730Z8Pl7MJAl40Mc89prjZw/FpR9pr+ZvF0+5UXFNwh5a0nfOv+hN+1mRkPkcvmZ aWmCNiQvbp391A8rcxCls2ZKklRALlfTI6PgXGNPz+CBu6cKjjavqU48MoaP0mpiGkdE DS2jDlir+sOxuvemcSfi80Z2QeMjA9GBb7R4QjViOLVXxS6HHcCvzCHIL61MUusldX6s q6nS9KYc7fODGh2hNAuXfN0xAP63Rw2VVha4PdsbHhgBYU0n+ztFrnbRnU2tl+BmHJE4 TysQ== X-Gm-Message-State: AOAM530alUH+M2tCGMCTBvvOhSTC9wJQSS4n2kgx14BzaL5oIw8f8UHc 036f2sOj65lScCYEsMqBncbK/OToDoeuYJh5 X-Google-Smtp-Source: ABdhPJz/vW2PmB5nvPRSNdQS5v8e1QIaJCbChgmywQIwIw3qCBt2wYADSNx1GYS36rwbNkkvrKv/2+gFZwdZs5i0 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a2e:320c:: with SMTP id y12mr9612197ljy.362.1605046840773; Tue, 10 Nov 2020 14:20:40 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:09 +0100 In-Reply-To: Message-Id: <3443e106c40799e5dc3981dec2011379f3cbbb0c.1605046662.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 05/20] kasan: allow VMAP_STACK for HW_TAGS mode From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172046_622341_DA7F1F04 X-CRM114-Status: GOOD ( 12.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Even though hardware tag-based mode currently doesn't support checking vmalloc allocations, it doesn't use shadow memory and works with VMAP_STACK as is. Change VMAP_STACK definition accordingly. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/I3552cbc12321dec82cd7372676e9372a2eb452ac Reviewed-by: Marco Elver Acked-by: Catalin Marinas --- arch/Kconfig | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 56b6ccc0e32d..7e7d14fae568 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -914,16 +914,16 @@ config VMAP_STACK default y bool "Use a virtually-mapped stack" depends on HAVE_ARCH_VMAP_STACK - depends on !KASAN || KASAN_VMALLOC + depends on !KASAN || KASAN_HW_TAGS || KASAN_VMALLOC help Enable this if you want the use virtually-mapped kernel stacks with guard pages. This causes kernel stack overflows to be caught immediately rather than causing difficult-to-diagnose corruption. - To use this with KASAN, the architecture must support backing - virtual mappings with real shadow memory, and KASAN_VMALLOC must - be enabled. + To use this with software KASAN modes, the architecture must support + backing virtual mappings with real shadow memory, and KASAN_VMALLOC + must be enabled. config ARCH_OPTIONAL_KERNEL_RWX def_bool n From patchwork Tue Nov 10 22:20:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52F87C4742C for ; Tue, 10 Nov 2020 22:34:37 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CB6972068D for ; Tue, 10 Nov 2020 22:34:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="QmE7m7kZ"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="ljjlpu85" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB6972068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=4RZ41Q75Du3qLh4lvMdHmQetXjue8QcrnYR9mb3CmWg=; b=QmE7m7kZtR/yissvKtmDklKFR 9dqq/6qv5X7SORF3dSX1b1MbgDlpAaBXEbVYN2Q9zW0Ja4emfi3otR5MGn152d5/wqB5Cdi3lEpmQ TbhvbslsjkUwjKVaaNtCs+38WNUFZtRetZbMUwJsxfvZGmWmeyUxMMKvILIw/w9l1A9Q2CvuIlTeR ISJ87y17MLm8p7s1Icv2CX7CZ1sfWm74duKUxvt6vaA2xPBpf4pJ5SHyXY7ue3+Ap0xYk9Uc1+87b vJkNc17eTgauu4qFXZqPpbLpuwt9HqYXsqPz+VjFT97sHpA1voHsGKb26+WVEtQdfUr+LibrXV+u+ fGF7hWlXA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccD7-00069l-H3; Tue, 10 Nov 2020 22:33:57 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0O-0000Ho-T3 for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:00 +0000 Received: by mail-wr1-x44a.google.com with SMTP id 67so6236110wra.2 for ; Tue, 10 Nov 2020 14:20:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=gyGUZGx6qFH2jkAU5L1Nk+VZLXl2e4B3h6WrDO5AJuc=; b=ljjlpu8510giM6ompwxwC9fW/XdFC8qbLxu53fZ6U03b6EYiKpJWB0pnDuMqHGR+jN r5TObzaeFfHiWBeyCqwixdqdi5C/59498RU2S9dfH8AcpG/uescXFFAI4vk1MFIFdUxs aKXAW+Igl/dpH9QlRqyELk9F5/tXGDupazh551//E9tyUg+sj67imZtPjKwh90Ayg1OS vnltrTuRHfB3/tkF78B29cR78uIqwsQ9sGY6GIW8WBL94NHckeVhPN4/F3EutOAkS+Ln 5/kQaGmI93yINQy4OgjBA4Oz+orNqVJURp3LVEBZBzBxMrSgh386MQ6eowcq/tXep6fs h8og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=gyGUZGx6qFH2jkAU5L1Nk+VZLXl2e4B3h6WrDO5AJuc=; b=ZYw3pY/N2QvtRyuQ1+0gy50W6zPQyIO/84xHBO4fkmWINzEO8cC3iCFGO+fI4zUyOF 6dasP5b95O14LZIXnYegsY4l9z5Ciz4/tR1XBnwR0mg4++tK3bgGwhp973dd79RMr/+S mcRlJj2rXmSDe1vGkBPlvdRBC1yxwDOo6/9DcqIGW75VW4xFmx/GrP/QWnLYrf0V9bEb 0GNsN197htnlf627ZhCkFTbXPIozInjh89inQ7E+paCSlxRABFO/yZ/uilAIJboiWSbu jeTn5nMrE8XdlP9D8FJ7Kryvji9X/qpkGl+vN35V8rWZTrsjUNz1LprVeQKQLZBrhKYy yp1Q== X-Gm-Message-State: AOAM533VQHId7JYtBRc83mADGV7liyUkyHQCXvXhno+SBfyw6udNtadH FOl7/oYXWGUdVWVtqluT5ea3+WkhGhrPYY4q X-Google-Smtp-Source: ABdhPJwekepriFSyzIoCXRNKCGpvFiq2NnAR4bAyn052jaQHzEPlcFfRQSyz8Y0DdueqyddIpnCAMwTZ+WqCAiQP X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a5d:630b:: with SMTP id i11mr5840517wru.404.1605046843285; Tue, 10 Nov 2020 14:20:43 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:10 +0100 In-Reply-To: Message-Id: <462c375f39ba8c4c105b3a9bf3b5db17f3720159.1605046662.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 06/20] kasan: remove __kasan_unpoison_stack From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172049_948907_AF90542D X-CRM114-Status: GOOD ( 12.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There's no need for __kasan_unpoison_stack() helper, as it's only currently used in a single place. Removing it also removes unneeded arithmetic. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Ie5ba549d445292fe629b4a96735e4034957bcc50 Reviewed-by: Marco Elver --- mm/kasan/common.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a3e67d49b893..9008fc6b0810 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -59,18 +59,12 @@ void kasan_disable_current(void) #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ #if CONFIG_KASAN_STACK -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - kasan_unpoison_memory(base, size); -} - /* Unpoison the entire stack for a task. */ void kasan_unpoison_task_stack(struct task_struct *task) { - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); + void *base = task_stack_page(task); + + kasan_unpoison_memory(base, THREAD_SIZE); } /* Unpoison the stack for the current task beyond a watermark sp value. */ From patchwork Tue Nov 10 22:20:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18F36C388F7 for ; Tue, 10 Nov 2020 23:47:35 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8FF82207E8 for ; Tue, 10 Nov 2020 23:47:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="J9UMh+ob"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="blIC1ANB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8FF82207E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kYvN0+UtP9eaEpHpIqw7n3jilcl0RCSJAn7C1wmePZk=; b=J9UMh+obkTSzgsB0H/nfzMZpI OmjFiBar6z6QwiC/lSVa7irLlzWDs+aty8tLOO1G5izysahU8wt4QbsBJjZHmul/VQ5KmhfyU5BHM GBGJqMZvo0pgAcgwHnxzk6W599mLOAnZe/uKy8CfcwC+mOggdssiCP14TkBWwCytvTtycNMuQI/C8 xZrnP8Gp2YX4zGCYUyl/9/vw4CV4tWOu7JY2k/OU7otHFmS1BIhGTzyux+iOEtcfPp7O7J2+M/OzN BonK9XjMsge0dq87Nk8dv/HvnNkIdlOB0zRBKMyhDv6s9DoZqmwFeibmDIyNb1sUEJpFfnTJNvuQE KJVz/2grg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcdLR-0002c0-Ad; Tue, 10 Nov 2020 23:46:37 +0000 Received: from mail-wr1-f73.google.com ([209.85.221.73]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0Q-0000Jf-NO for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:05 +0000 Received: by mail-wr1-f73.google.com with SMTP id r16so6159486wrw.22 for ; Tue, 10 Nov 2020 14:20:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=syFMy85hNfCVsv9zzw9k8mLn//1pQqq/P081Jgl6REI=; b=blIC1ANBVNd6EZzMzsRlw3d4rt6B2ipZm892/MrZ7S1nhqnNWiOA7JvDTY6AMSh02M BaJg3PuQURSenuj+wfR6ssW9FkwcQYO8Bhz4ora+aWSAu2niRzurqnxkfhFeYyK3J1sx OUU+UGqUoiZj02mRPHxnU87w0tZwOz1ThhGd/L9AhjpdD+Y6HbrDz6zGAGcDgoyAbyqH VkiZY8f5IVx43dKCVdn+K+DOFwhCdfSNxSWATQocZECaN2AE7QOaWvyTt4Tbao9PL1yC Ill13wq+v6ZjomJNuAjfAriY8SteAWp9M5/wp68psNH5GzRU0n8ecJFQr4fffIhLEYGH F6uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=syFMy85hNfCVsv9zzw9k8mLn//1pQqq/P081Jgl6REI=; b=exl1pvIo8NcufEEIteLtoOHgekQP2NFIG2PANa22fH5S2/eslfR+bnevohKC24anUR G8YKoQflR13J+/HU0XTI0SI0AwHaPR4YCJx0smNyB6PZmw3N8lfQQUkan8nFdCtLImxA KWwCwZNDn1kLwTIQWPGmqq0iv0Q0wuN5K691t0UCT76V5ZfI+urf62TtgSyN96wrpu2V c3p0A4QBXMW97GwaQ10kgKv1vMEmjtm3dbPG+wlkCUJZ3CARmyKl0cfpnjYItCfpCa7g WUsKNlvLVovDkbPX4vb09jRQ349PNdcJ0KEzGmf0Vn6ZdLQ61LO+KfDZyv81OOfXYX4Y 3fyQ== X-Gm-Message-State: AOAM532q//xMbQv4XATBsx+e1pz08SSzCdSSpsTKzwJoMOMhiDfqZQEt Y8nTWMSjN+EPQTm2Rm56ob5iAnRQOAbTPfRv X-Google-Smtp-Source: ABdhPJyjyI1u9TfjaCaGRNEf+0oiJ6LkqtM1n6xNaRSxKrLTlg5Z8+icNw9Kwhx+L+60LftELeAmSKSqIcXD2xjN X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a5d:4ac1:: with SMTP id y1mr26982536wrs.27.1605046845650; Tue, 10 Nov 2020 14:20:45 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:11 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 07/20] kasan: inline kasan_reset_tag for tag-based modes From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172051_192231_B955F6B3 X-CRM114-Status: GOOD ( 17.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Using kasan_reset_tag() currently results in a function call. As it's called quite often from the allocator code, this leads to a noticeable slowdown. Move it to include/linux/kasan.h and turn it into a static inline function. Also remove the now unneeded reset_tag() internal KASAN macro and use kasan_reset_tag() instead. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/I4d2061acfe91d480a75df00b07c22d8494ef14b5 Reviewed-by: Marco Elver --- include/linux/kasan.h | 5 ++++- mm/kasan/common.c | 6 +++--- mm/kasan/hw_tags.c | 9 ++------- mm/kasan/kasan.h | 4 ---- mm/kasan/report.c | 4 ++-- mm/kasan/report_hw_tags.c | 2 +- mm/kasan/report_sw_tags.c | 4 ++-- mm/kasan/shadow.c | 4 ++-- mm/kasan/sw_tags.c | 9 ++------- 9 files changed, 18 insertions(+), 29 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index b9b9db335d87..53c8e8b12fbc 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -193,7 +193,10 @@ static inline void kasan_record_aux_stack(void *ptr) {} #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) -void *kasan_reset_tag(const void *addr); +static inline void *kasan_reset_tag(const void *addr) +{ + return (void *)arch_kasan_reset_tag(addr); +} bool kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 9008fc6b0810..a266b90636a1 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -174,14 +174,14 @@ size_t kasan_metadata_size(struct kmem_cache *cache) struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object) { - return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; + return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset; } struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; + return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } void kasan_poison_slab(struct page *page) @@ -278,7 +278,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, tag = get_tag(object); tagged_object = object; - object = reset_tag(object); + object = kasan_reset_tag(object); if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != object)) { diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 70b88dd40cd8..49ea5f5c5643 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -30,20 +30,15 @@ void kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void *kasan_reset_tag(const void *addr) -{ - return reset_tag(addr); -} - void kasan_poison_memory(const void *address, size_t size, u8 value) { - hw_set_mem_tag_range(reset_tag(address), + hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), value); } void kasan_unpoison_memory(const void *address, size_t size) { - hw_set_mem_tag_range(reset_tag(address), + hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index db8a7a508121..8a5501ef2339 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -246,15 +246,11 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) return addr; } #endif -#ifndef arch_kasan_reset_tag -#define arch_kasan_reset_tag(addr) ((void *)(addr)) -#endif #ifndef arch_kasan_get_tag #define arch_kasan_get_tag(addr) 0 #endif #define set_tag(addr, tag) ((void *)arch_kasan_set_tag((addr), (tag))) -#define reset_tag(addr) ((void *)arch_kasan_reset_tag(addr)) #define get_tag(addr) arch_kasan_get_tag(addr) #ifdef CONFIG_KASAN_HW_TAGS diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 0cac53a57c14..25ca66c99e48 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -328,7 +328,7 @@ void kasan_report_invalid_free(void *object, unsigned long ip) unsigned long flags; u8 tag = get_tag(object); - object = reset_tag(object); + object = kasan_reset_tag(object); #if IS_ENABLED(CONFIG_KUNIT) if (current->kunit_test) @@ -361,7 +361,7 @@ static void __kasan_report(unsigned long addr, size_t size, bool is_write, disable_trace_on_warning(); tagged_addr = (void *)addr; - untagged_addr = reset_tag(tagged_addr); + untagged_addr = kasan_reset_tag(tagged_addr); info.access_addr = tagged_addr; if (addr_has_metadata(untagged_addr)) diff --git a/mm/kasan/report_hw_tags.c b/mm/kasan/report_hw_tags.c index da543eb832cd..57114f0e14d1 100644 --- a/mm/kasan/report_hw_tags.c +++ b/mm/kasan/report_hw_tags.c @@ -22,7 +22,7 @@ const char *get_bug_type(struct kasan_access_info *info) void *find_first_bad_addr(void *addr, size_t size) { - return reset_tag(addr); + return kasan_reset_tag(addr); } void metadata_fetch_row(char *buffer, void *row) diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index 317100fd95b9..7604b46239d4 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -41,7 +41,7 @@ const char *get_bug_type(struct kasan_access_info *info) int i; tag = get_tag(info->access_addr); - addr = reset_tag(info->access_addr); + addr = kasan_reset_tag(info->access_addr); page = kasan_addr_to_page(addr); if (page && PageSlab(page)) { cache = page->slab_cache; @@ -72,7 +72,7 @@ const char *get_bug_type(struct kasan_access_info *info) void *find_first_bad_addr(void *addr, size_t size) { u8 tag = get_tag(addr); - void *p = reset_tag(addr); + void *p = kasan_reset_tag(addr); void *end = p + size; while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 616ac64c4a21..8e4fa9157a0b 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -81,7 +81,7 @@ void kasan_poison_memory(const void *address, size_t size, u8 value) * some of the callers (e.g. kasan_poison_object_data) pass tagged * addresses to this function. */ - address = reset_tag(address); + address = kasan_reset_tag(address); shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); @@ -98,7 +98,7 @@ void kasan_unpoison_memory(const void *address, size_t size) * some of the callers (e.g. kasan_unpoison_object_data) pass tagged * addresses to this function. */ - address = reset_tag(address); + address = kasan_reset_tag(address); kasan_poison_memory(address, size, tag); diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index 3bffb489b144..d1af6f6c6d12 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -67,11 +67,6 @@ u8 random_tag(void) return (u8)(state % (KASAN_TAG_MAX + 1)); } -void *kasan_reset_tag(const void *addr) -{ - return reset_tag(addr); -} - bool check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { @@ -107,7 +102,7 @@ bool check_memory_region(unsigned long addr, size_t size, bool write, if (tag == KASAN_TAG_KERNEL) return true; - untagged_addr = reset_tag((const void *)addr); + untagged_addr = kasan_reset_tag((const void *)addr); if (unlikely(untagged_addr < kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) { return !kasan_report(addr, size, write, ret_ip); @@ -126,7 +121,7 @@ bool check_memory_region(unsigned long addr, size_t size, bool write, bool check_invalid_free(void *addr) { u8 tag = get_tag(addr); - u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(reset_tag(addr))); + u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(kasan_reset_tag(addr))); return (shadow_byte == KASAN_TAG_INVALID) || (tag != KASAN_TAG_KERNEL && tag != shadow_byte); From patchwork Tue Nov 10 22:20:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FDE9C388F7 for ; Tue, 10 Nov 2020 22:35:31 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EDDD72068D for ; Tue, 10 Nov 2020 22:35:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="J5dkTuTE"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="X/G1FHBp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDDD72068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kz2XUJBMKDKVxGdPOh43oyUogwTrZAA5nEQMP41Gwa0=; b=J5dkTuTEiS/P1+yMKfu+UjKB0 XQ17zcXfbaUZu33v8OuUlhUd+jCITJPKRmeHCYjQbLNlYcUwSP69IKtCovNOdQS4GSD8/Ggk8y+OF 9sVgrXY2b49PIDBMLRWGA+y0d8lHwuBSWzEP0j00IXWCpH/9rETH0dYcm+KYgBe5MbKRMayCpfeN+ s5rXGOz/qH13imN3St+SqTeKNgXst12jhYGcUrH3rjh9Sl1bMgJLuqW+rI6vkoK1cH8IJSo/UY59L hlmVOenAsHmkaB5gNBbkZIf3snPu8G1JoRBLMbpVu148DULfT9ug9xmNbftX+i6avltg4PAWjAbxs PgR64XK5w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccE2-0006Yb-Dt; Tue, 10 Nov 2020 22:34:54 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0U-0000Kc-Vj for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:04 +0000 Received: by mail-wr1-x449.google.com with SMTP id z13so247636wrm.19 for ; Tue, 10 Nov 2020 14:20:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=/rSFl10DA/GX54GBDw6zGUYBErxFDPI5QaAJ7Zds8ak=; b=X/G1FHBpaTNPbceckVWXnlaQTlozNeiCpeU7tMmREOJimTXkmCTjt6rRjgNPSeQ0pw q73UVtucfzggJP2pLLg0d8XHXT0ngy/FLNTkV+unC3tseUSKbxmMqR1RfIvjsfMbA2Eq fniu1B6bL+1GI15kaPYNzXCk/yeIk98ad1Jm+d8i/5KDIlsK5BgcmP/efX5QPF1s8Si/ 6NXEIxz4MDViC1Q1XgbEBgCSkG1swl6ak5j3v5axkuBmgM9UBfbh1fuNYORk8kWZXP9Q iGwaOfp4qXX5szQspMdAW9gYIF9nebgteHML8xT3tOJCbTT3oOXCOGM54M/KBM4bacSy YBMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=/rSFl10DA/GX54GBDw6zGUYBErxFDPI5QaAJ7Zds8ak=; b=hdgohbTHIUcN1DK7LOVk9X/FnW9gFSsIcZFsrM6yEy1+1sEJjtKsyUh/bqaNFbcLL0 IN1GX0PJcmN/HZvYzouO/eIDsqiy7f6yTqlaBYZbbY6Zdl0d1153cfeEXoBqiypwGxMV O4SaLkU88nA0N0smtk62w5dKTLXig1VeBszAR1DuVWKpDTtzQftGtx/7AF+kYXg3JVxO RaDT1sx/bHvmJYSNCCLlr/rwB5YdYWmZXMXGC8gimgJRrrDKOD5liklC6BHVn6rM6vaL ezM8N3DnX4LisXcUEzqGj8utr+rBfjg15r4bRHh/pUHmKYKkM/9skF4w4ccSsaJFcqZm 0HLA== X-Gm-Message-State: AOAM533BmW6AHDwMWL/REDgbgLzBgDPUsAVSKFx3ZKMQ4TRl/G64yFYQ zs+L9XG8okf4iidab1g0HqVjMy7aFevApaql X-Google-Smtp-Source: ABdhPJyY/bBCadcyM/PRgjtzpu0TT6SQ1RzDsL0gjh3vrXFFiPqpWwP79w3xbEXBJdKIjt2rfzmw9gOcpXSqt2FT X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a5d:6692:: with SMTP id l18mr14959782wru.44.1605046848070; Tue, 10 Nov 2020 14:20:48 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:12 +0100 In-Reply-To: Message-Id: <73399d4c0644266d61ad81eb391f5ee10c09e098.1605046662.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 08/20] kasan: inline random_tag for HW_TAGS From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172055_173175_AD4DB1B4 X-CRM114-Status: GOOD ( 14.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Using random_tag() currently results in a function call. Move its definition to mm/kasan/kasan.h and turn it into a static inline function for hardware tag-based mode to avoid uneeded function calls. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Iac5b2faf9a912900e16cca6834d621f5d4abf427 Reviewed-by: Marco Elver --- mm/kasan/hw_tags.c | 5 ----- mm/kasan/kasan.h | 34 +++++++++++++++++----------------- 2 files changed, 17 insertions(+), 22 deletions(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 49ea5f5c5643..1476ac07666e 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -42,11 +42,6 @@ void kasan_unpoison_memory(const void *address, size_t size) round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } -u8 random_tag(void) -{ - return hw_get_random_tag(); -} - bool check_invalid_free(void *addr) { u8 ptr_tag = get_tag(addr); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 8a5501ef2339..7498839a15d3 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -188,6 +188,12 @@ static inline bool addr_has_metadata(const void *addr) #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) +void print_tags(u8 addr_tag, const void *addr); +#else +static inline void print_tags(u8 addr_tag, const void *addr) { } +#endif + bool check_invalid_free(void *addr); void *find_first_bad_addr(void *addr, size_t size); @@ -223,23 +229,6 @@ static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif -#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) - -void print_tags(u8 addr_tag, const void *addr); - -u8 random_tag(void); - -#else - -static inline void print_tags(u8 addr_tag, const void *addr) { } - -static inline u8 random_tag(void) -{ - return 0; -} - -#endif - #ifndef arch_kasan_set_tag static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) { @@ -279,6 +268,17 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #endif /* CONFIG_KASAN_HW_TAGS */ +#ifdef CONFIG_KASAN_SW_TAGS +u8 random_tag(void); +#elif defined(CONFIG_KASAN_HW_TAGS) +#define random_tag() hw_get_random_tag() +#else +static inline u8 random_tag(void) +{ + return 0; +} +#endif + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. From patchwork Tue Nov 10 22:20:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7408C388F7 for ; Tue, 10 Nov 2020 22:36:01 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 199112068D for ; Tue, 10 Nov 2020 22:36:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="IXdME7KX"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="HbmemqL7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 199112068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=+NyZTyBRT9Qq6CZZADsJeB5ylIWczWwf5S1mcaN4qIU=; b=IXdME7KXgUdN2JWeb0drJxP8A apywOZxcg6rPRQvKykvhnj6rurBYWYESh0md9SYYvmO8s+msvLynTh/h59xRfMLmvHkAj5zgPkvBK FtOSB+ytz0g9yDDWDDnOewMy2IoEfNsAUrlnGMQVZc1kbA6Ry5dI3I50ENR215sNczWqcIC63GRK9 VrsgS+KsmL9VaZ3mVY5jLtJyiw2gD/IUlUZ9NsbGL9CG/z7p6V8yWsuyJ/8YMvINX/6s17lTeB1od CQjPyjpEEmCOhn6JJFhlnNDWXlaB2ZTDLGvrFPPW7sRwUZ91yGWq2G8gctwb2bbNpleQ2YXMFnpEy AztMwPWAQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccEV-0006o0-WD; Tue, 10 Nov 2020 22:35:24 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0W-0000Mj-F9 for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:05 +0000 Received: by mail-qk1-x74a.google.com with SMTP id x2so181240qkd.23 for ; Tue, 10 Nov 2020 14:20:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=WOMasW4WeK4i4aCbvBDhnou95P0GnHEsdPLivTBemP4=; b=HbmemqL7hGON1dZUtHAIO/Ny5Z5yXZ4JDxyVyJy817JkkYWiD/QtbND8oC8ab7jCnI 5Wbp10QDuzYLJu6lNMQ/cilMKfhcOQKdxLo2b2RlUEr9KAST0GqOKpk56Qy0CPfK5YuM QKo9+lLsoO9x4ILXIt3sKjhcJ8pdIqjOatB0zmydqRDWzWGHfAAw6YSENtb7o5FoF2si lv2RAbcm26eD/cbihEmRP5cwTzv9aHIH9H1rKOeKMG6R8p0oNMZ/gBRAvVBPI42BEMDc psc8VI/Zf64SE7/LFloA+ZdjypcKgOsrKstJM34+FhTA1A+KoSlI7qAqvqmoxKnZxc2h Df1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=WOMasW4WeK4i4aCbvBDhnou95P0GnHEsdPLivTBemP4=; b=D4Is1Za0xt9SBv4D2EJIHgEt3HUAqsXDu3Vf8IXsxRK5Mcle1IJYjxHhy515F5QgTo yQK4wybiSvT7tmgQ84JyRm+MVPhGBGUQzJpUO1nEIaiMMfl92tdBuIe4b6p2nHyb930M 8B86Qtydfip6DZbhq+MLrq6opQ9MMKMwc69p6YhVD7+Q13utOplffwFrxkzOD54vbAjS JLLtUWt84GuKfBME15+iHhbQHSaSPd6kzyIM6LSR8YHxxqXw+Eps4gxIMUhWvleQCfEq PG/uCx0jJsblxi5Ugt1J8WddsePSLI/1RcnGL4Jaa6K4YcqQGjwbMl3cDcUwrDKc4bn6 F6Xw== X-Gm-Message-State: AOAM533rZ9eGIYGO0/MWpTsX5cbmorbqZgYZ3bBmKYSn8aQCP2GelDuU sig5lpO9Xx0e79pMYMIgV8/vwMPhRO0/KMDp X-Google-Smtp-Source: ABdhPJxV3RzH8+DI2bYMl115wXV/xV/YUfiel/zd2PCNwgEqHTxxwM0dmmBMZ8TRZ+cgt1xGGP23OM9y5Oi9AE76 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:9e2f:: with SMTP id p47mr14534879qve.11.1605046850374; Tue, 10 Nov 2020 14:20:50 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:13 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 09/20] kasan: inline kasan_poison_memory and check_invalid_free From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172056_831137_BCCB31A3 X-CRM114-Status: GOOD ( 14.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Using kasan_poison_memory() or check_invalid_free() currently results in function calls. Move their definitions to mm/kasan/kasan.h and turn them into static inline functions for hardware tag-based mode to avoid unneeded function calls. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Ia9d8191024a12d1374675b3d27197f10193f50bb Reviewed-by: Marco Elver --- mm/kasan/hw_tags.c | 15 --------------- mm/kasan/kasan.h | 28 ++++++++++++++++++++++++---- 2 files changed, 24 insertions(+), 19 deletions(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 1476ac07666e..0303e49904b4 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -30,27 +30,12 @@ void kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void kasan_poison_memory(const void *address, size_t size, u8 value) -{ - hw_set_mem_tag_range(kasan_reset_tag(address), - round_up(size, KASAN_GRANULE_SIZE), value); -} - void kasan_unpoison_memory(const void *address, size_t size) { hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } -bool check_invalid_free(void *addr) -{ - u8 ptr_tag = get_tag(addr); - u8 mem_tag = hw_get_mem_tag(addr); - - return (mem_tag == KASAN_TAG_INVALID) || - (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); -} - void kasan_set_free_info(struct kmem_cache *cache, void *object, u8 tag) { diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 7498839a15d3..ab7314418604 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -153,8 +153,6 @@ struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object); -void kasan_poison_memory(const void *address, size_t size, u8 value); - #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) static inline const void *kasan_shadow_to_mem(const void *shadow_addr) @@ -194,8 +192,6 @@ void print_tags(u8 addr_tag, const void *addr); static inline void print_tags(u8 addr_tag, const void *addr) { } #endif -bool check_invalid_free(void *addr); - void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); void metadata_fetch_row(char *buffer, void *row); @@ -279,6 +275,30 @@ static inline u8 random_tag(void) } #endif +#ifdef CONFIG_KASAN_HW_TAGS + +static inline void kasan_poison_memory(const void *address, size_t size, u8 value) +{ + hw_set_mem_tag_range(kasan_reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), value); +} + +static inline bool check_invalid_free(void *addr) +{ + u8 ptr_tag = get_tag(addr); + u8 mem_tag = hw_get_mem_tag(addr); + + return (mem_tag == KASAN_TAG_INVALID) || + (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); +} + +#else /* CONFIG_KASAN_HW_TAGS */ + +void kasan_poison_memory(const void *address, size_t size, u8 value); +bool check_invalid_free(void *addr); + +#endif /* CONFIG_KASAN_HW_TAGS */ + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. From patchwork Tue Nov 10 22:20:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D86FC55ABD for ; Tue, 10 Nov 2020 23:47:33 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 33B2A207E8 for ; Tue, 10 Nov 2020 23:47:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="v9eingry"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="tjOoGMIf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 33B2A207E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ksC0nMwXmxHD/lBXhIWtrPCSiho6pTTvfrGB0c80ClI=; b=v9eingryIEHvlkFSIgKLjETI7 xnb6VOm1D7W6k73JX+a2EN1FH7nWLXZS0TfCYyejSkuTR3TNvuvMoIjTodvJB45nopuwJ//45OFjL 6Sx5nPun9lJVR51M3clm0yvWWRY2ADOkZ4xT1ueBK48iWjPicr6lUyOgvycm0ZRFiRZgFIS+Pgni6 WcdQaIb2ztg8YDF2RDys04fhjVqwvrA1nmcR//DgMJvB7vSXrLdN8Qo8xxexSFtxJPlcgDQ8G/gas YX3Vw+IebrgJ1UM3XMj4eaZmusUBikvJ4qizU22IcPGVicIrGczfsK/ya/rpqVS6fKPFAwAbrHcur olrJQVjnw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcdLS-0002c9-Fy; Tue, 10 Nov 2020 23:46:38 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0a-0000N7-17 for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:12 +0000 Received: by mail-wr1-x44a.google.com with SMTP id q15so6192689wrw.8 for ; Tue, 10 Nov 2020 14:20:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=tuOEp1iVth583MBjSMMuaXP+cup9Jpeadd/jLYbzv5o=; b=tjOoGMIfi1Gy2x6aR7moxABHWyMdUQ9vFeQAZjlOfMm28OF0UgUGZZCPOYiyrWRuhp Zvfv6lWIXOW8Qhgx/K5jP91DGmLRB1FDX+2dINo9CsLqWdwLa/qTm5toRnoVWB6TgXCe 9vdlGqXBFqgVzxUOwRyn4y9EVlt5WJDMAId3+JtmEXPcHm65v4Q3UGdFLTKZyscgUFiH 5/tbS8FhH3SEYuHH4FYln33WWUmUTTwywFL50omBB/idcdGGItMwcrqRfMcvcWPfvIU1 Z5WBIbk0dyF4xuunIyE2hOWre7Qwjxc0UCO5kNa7aXBPIwh7vv3mMCzJhyvyTDoNSjXL y0SA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tuOEp1iVth583MBjSMMuaXP+cup9Jpeadd/jLYbzv5o=; b=M1kVhWwmfdiRGS486ltOakK8LElMjLN3UMXPdbX0HWAtBpJcgnIpcoTf18X7asx9R1 b0XfCaJrMEcWmJ1UL8P5uSolQTg8CRzKAZr/cPVLMcPzUQdMMz65H1Z1e4SaxvrHMb4h 7ch0phG/vhvdBkXMQmPdh8iyUJkLkZFYNPwHK7G5GcHdXeK1bzBjX8Tb+82V4U7UriAg qeGNp/Z331b2v3anlkyLwE6v0ov/Xc0t49N1UYNf36rXOpgkxRVCj1rdsEkhyUIsECpc Ho0qRCQ3hVICfG1B40APNP7q5TDihKVa5W87CMw9IG8gQxLjViAvT4O9gzdW0lrwmgKs tLCg== X-Gm-Message-State: AOAM533cA6bAEu1Wc8bqw9sqgpWF8Z1fUhpvvnscdZnXYOe7P2wB1iIG QIxci8owfjTWvWsIVDQ+XO9YTwbnueTwp1JQ X-Google-Smtp-Source: ABdhPJxtiBqyG4xwe2EmyTCmp+GdZ0f2gBwn+LXwUoSWPlDe2zEXDIKB2AYn330Kqc9Cuu2p95cOSOPMIUl2u6W+ X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a7b:c458:: with SMTP id l24mr281240wmi.136.1605046852883; Tue, 10 Nov 2020 14:20:52 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:14 +0100 In-Reply-To: Message-Id: <0a9b63bff116734ab63d99ebd09c244332d71958.1605046662.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 10/20] kasan: inline and rename kasan_unpoison_memory From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172100_600239_1ED966F4 X-CRM114-Status: GOOD ( 17.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently kasan_unpoison_memory() is used as both an external annotation and as an internal memory poisoning helper. Rename external annotation to kasan_unpoison_data() and inline the internal helper for hardware tag-based mode to avoid undeeded function calls. There's the external annotation kasan_unpoison_slab() that is currently defined as static inline and uses kasan_unpoison_memory(). With this change it's turned into a function call. Overall, this results in the same number of calls for hardware tag-based mode as kasan_unpoison_memory() is now inlined. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ia7c8b659f79209935cbaab3913bf7f082cc43a0e --- include/linux/kasan.h | 16 ++++++---------- kernel/fork.c | 2 +- mm/kasan/common.c | 10 ++++++++++ mm/kasan/hw_tags.c | 6 ------ mm/kasan/kasan.h | 7 +++++++ mm/slab_common.c | 2 +- 6 files changed, 25 insertions(+), 18 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 53c8e8b12fbc..f1a5042ae4fc 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -74,14 +74,15 @@ static inline void kasan_disable_current(void) {} #ifdef CONFIG_KASAN -void kasan_unpoison_memory(const void *address, size_t size); - void kasan_alloc_pages(struct page *page, unsigned int order); void kasan_free_pages(struct page *page, unsigned int order); void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags); +void kasan_unpoison_data(const void *address, size_t size); +void kasan_unpoison_slab(const void *ptr); + void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); @@ -106,11 +107,6 @@ struct kasan_cache { int free_meta_offset; }; -size_t __ksize(const void *); -static inline void kasan_unpoison_slab(const void *ptr) -{ - kasan_unpoison_memory(ptr, __ksize(ptr)); -} size_t kasan_metadata_size(struct kmem_cache *cache); bool kasan_save_enable_multi_shot(void); @@ -118,8 +114,6 @@ void kasan_restore_multi_shot(bool enabled); #else /* CONFIG_KASAN */ -static inline void kasan_unpoison_memory(const void *address, size_t size) {} - static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} @@ -127,6 +121,9 @@ static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} +static inline void kasan_unpoison_data(const void *address, size_t size) { } +static inline void kasan_unpoison_slab(const void *ptr) { } + static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} @@ -166,7 +163,6 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object, return false; } -static inline void kasan_unpoison_slab(const void *ptr) { } static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ diff --git a/kernel/fork.c b/kernel/fork.c index 1c905e4290ab..883898487b3f 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -226,7 +226,7 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node) continue; /* Mark stack accessible for KASAN. */ - kasan_unpoison_memory(s->addr, THREAD_SIZE); + kasan_unpoison_data(s->addr, THREAD_SIZE); /* Clear stale pointers from reused stack. */ memset(s->addr, 0, THREAD_SIZE); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a266b90636a1..4598c1364f19 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -184,6 +184,16 @@ struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } +void kasan_unpoison_data(const void *address, size_t size) +{ + kasan_unpoison_memory(address, size); +} + +void kasan_unpoison_slab(const void *ptr) +{ + kasan_unpoison_memory(ptr, __ksize(ptr)); +} + void kasan_poison_slab(struct page *page) { unsigned long i; diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 0303e49904b4..838b29e44e32 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -30,12 +30,6 @@ void kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void kasan_unpoison_memory(const void *address, size_t size) -{ - hw_set_mem_tag_range(kasan_reset_tag(address), - round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); -} - void kasan_set_free_info(struct kmem_cache *cache, void *object, u8 tag) { diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index ab7314418604..2d3c99125996 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -283,6 +283,12 @@ static inline void kasan_poison_memory(const void *address, size_t size, u8 valu round_up(size, KASAN_GRANULE_SIZE), value); } +static inline void kasan_unpoison_memory(const void *address, size_t size) +{ + hw_set_mem_tag_range(kasan_reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); +} + static inline bool check_invalid_free(void *addr) { u8 ptr_tag = get_tag(addr); @@ -295,6 +301,7 @@ static inline bool check_invalid_free(void *addr) #else /* CONFIG_KASAN_HW_TAGS */ void kasan_poison_memory(const void *address, size_t size, u8 value); +void kasan_unpoison_memory(const void *address, size_t size); bool check_invalid_free(void *addr); #endif /* CONFIG_KASAN_HW_TAGS */ diff --git a/mm/slab_common.c b/mm/slab_common.c index 53d0f8bb57ea..f1b0c4a22f08 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1176,7 +1176,7 @@ size_t ksize(const void *objp) * We assume that ksize callers could use whole allocated area, * so we need to unpoison this area. */ - kasan_unpoison_memory(objp, size); + kasan_unpoison_data(objp, size); return size; } EXPORT_SYMBOL(ksize); From patchwork Tue Nov 10 22:20:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F65AC4742C for ; Tue, 10 Nov 2020 23:47:35 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AA44F20809 for ; Tue, 10 Nov 2020 23:47:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="mAqoj7+c"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="gvbp41Wo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA44F20809 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=oxuwsUIpCZ7/hU2M+Ws7J40xAX4C/QhCGluiLlPpQVY=; b=mAqoj7+c76D1tJcpSM5FRlRcD yL6RCeK3l6w6M6ORQLVa/Hq3w4+8X5GBTmVyOGHIixo+mKlWE96G73CJNmk06/KmMLhLgZU96UVh9 LJGXahh6uBp8jTOW8U0DjOKSAAIj+pCFoMXucrQfTy7Su+31QOVVWCSnzmUGyeP4IAdNdmDpA+54z 1AXi6dJnWXKkiUVuCo/8uNVUxxnnypmlNnPO4u6rB1Tivty/ySJErfbDI/W5ZXVGLGvX197eLQf+O BK37Gm3EsEFmD9iLaFOo5VbHuHt20NAYND0Ovj08keN2OBQBQaEJ07KBvxzON9MH1EgQiJ8fxcztM tXn8ZTgIg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcdLT-0002cN-IL; Tue, 10 Nov 2020 23:46:39 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0a-0000Ox-ED for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:16 +0000 Received: by mail-qk1-x749.google.com with SMTP id t64so218459qkd.5 for ; Tue, 10 Nov 2020 14:20:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=C1AHhA2JiIvkEyQnY3sopRuy3W9NzHQWsdv6OutYhlM=; b=gvbp41WoAzww8TYRMkKnNegELr+4/Y7ni0aHb9h9e2Ek6BRCEc8wNmGcSuLXFrDdju 8HFuejYaUD5zzx3ycw8T9elfW5ANAXirMImZOqD/oLXj2HVCz1N+dvi6nzg7Myeom8qF oOZF84RbQOqOFCChHPxL2C9Kv2NiHuBbkcZR0NRfazYmTWAYEXJi2p8kpQi/cHe8EVed DCIt1qRmzd0sV/2tF7FAS3pmWCL+sTMRTblqS0nTQeWXc5hqnMTNDeFFFsurS6+KxYhI 7v2Sg3o9W2+dQYcGdjjASONR6vn+tDZdwD1OYziEKxKg/WI6riY39Z6vo/ae7LBj+tGb 5yoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=C1AHhA2JiIvkEyQnY3sopRuy3W9NzHQWsdv6OutYhlM=; b=RUm1ipR3koVQBZCq4m+OTHmhmYrIigIIjBlZ/CpoIf+YrP7fVcSaALn1vh7G8kS3QD ciStbMqeTF/+KtWp05YR3B0gWavZw4t9R0fZj0N3JR3c9lI66ToqND1FI75qnZofd4/z cLYQbIhB8gLiTBjCTPUt9RF4O4NLs5PkmGIdbYQnZSjW3x6KXzkgGcisvZmveBHRQ6BK T4uPaiH1PfF+2kb/TjD2JA9DAncMEKk/qMKIUI9v+4muxMP4igDWWMVHfvpeS/qkPKEs m29j4I+pZBFxoEZA9uWXMNOXPJkwnyW3+gJbCmlexe7OJhA8W5a+KxF/xW3KijGlTqEj frbA== X-Gm-Message-State: AOAM531WK1W4teY221jopQNARtU5uIeCf2hUknPNo+dd1TNOZbPTQ1uu DM/NNDc4SG7LYVZcm8kd0HeCFxCQUcbWUyhD X-Google-Smtp-Source: ABdhPJzULwIjhucxHMEL1ve5N+U+spPW+S0asJ3yxR73Odh5s887/rzw9Q2Ud8xCU6UwkQF11IYxomADZ/OybFc8 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:6214:1541:: with SMTP id t1mr21263869qvw.61.1605046855110; Tue, 10 Nov 2020 14:20:55 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:15 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 11/20] kasan: add and integrate kasan boot parameters From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172101_383717_75A44761 X-CRM114-Status: GOOD ( 24.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hardware tag-based KASAN mode is intended to eventually be used in production as a security mitigation. Therefore there's a need for finer control over KASAN features and for an existence of a kill switch. This change adds a few boot parameters for hardware tag-based KASAN that allow to disable or otherwise control particular KASAN features. The features that can be controlled are: 1. Whether KASAN is enabled at all. 2. Whether KASAN collects and saves alloc/free stacks. 3. Whether KASAN panics on a detected bug or not. With this change a new boot parameter kasan.mode allows to choose one of three main modes: - kasan.mode=off - KASAN is disabled, no tag checks are performed - kasan.mode=prod - only essential production features are enabled - kasan.mode=full - all KASAN features are enabled The chosen mode provides default control values for the features mentioned above. However it's also possible to override the default values by providing: - kasan.stacktrace=off/on - enable alloc/free stack collection (default: on for mode=full, otherwise off) - kasan.fault=report/panic - only report tag fault or also panic (default: report) If kasan.mode parameter is not provided, it defaults to full when CONFIG_DEBUG_KERNEL is enabled, and to prod otherwise. It is essential that switching between these modes doesn't require rebuilding the kernel with different configs, as this is required by the Android GKI (Generic Kernel Image) initiative [1]. [1] https://source.android.com/devices/architecture/kernel/generic-kernel-image Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/If7d37003875b2ed3e0935702c8015c223d6416a4 Acked-by: Catalin Marinas --- mm/kasan/common.c | 22 +++++-- mm/kasan/hw_tags.c | 152 +++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.h | 16 +++++ mm/kasan/report.c | 14 ++++- 4 files changed, 197 insertions(+), 7 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 4598c1364f19..efad5ed6a3bd 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -129,6 +129,11 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, unsigned int redzone_size; int redzone_adjust; + if (!kasan_stack_collection_enabled()) { + *flags |= SLAB_KASAN; + return; + } + /* Add alloc meta. */ cache->kasan_info.alloc_meta_offset = *size; *size += sizeof(struct kasan_alloc_meta); @@ -165,6 +170,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, size_t kasan_metadata_size(struct kmem_cache *cache) { + if (!kasan_stack_collection_enabled()) + return 0; return (cache->kasan_info.alloc_meta_offset ? sizeof(struct kasan_alloc_meta) : 0) + (cache->kasan_info.free_meta_offset ? @@ -267,11 +274,13 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; + if (kasan_stack_collection_enabled()) { + if (!(cache->flags & SLAB_KASAN)) + return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); + } if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -308,6 +317,9 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE); + if (!kasan_stack_collection_enabled()) + return false; + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || unlikely(!(cache->flags & SLAB_KASAN))) return false; @@ -355,7 +367,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (cache->flags & SLAB_KASAN) + if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN)) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 838b29e44e32..2f6f0261af8c 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -8,6 +8,8 @@ #define pr_fmt(fmt) "kasan: " fmt +#include +#include #include #include #include @@ -17,9 +19,104 @@ #include "kasan.h" +enum kasan_arg_mode { + KASAN_ARG_MODE_DEFAULT, + KASAN_ARG_MODE_OFF, + KASAN_ARG_MODE_PROD, + KASAN_ARG_MODE_FULL, +}; + +enum kasan_arg_stacktrace { + KASAN_ARG_STACKTRACE_DEFAULT, + KASAN_ARG_STACKTRACE_OFF, + KASAN_ARG_STACKTRACE_ON, +}; + +enum kasan_arg_fault { + KASAN_ARG_FAULT_DEFAULT, + KASAN_ARG_FAULT_REPORT, + KASAN_ARG_FAULT_PANIC, +}; + +static enum kasan_arg_mode kasan_arg_mode __ro_after_init; +static enum kasan_arg_stacktrace kasan_arg_stacktrace __ro_after_init; +static enum kasan_arg_fault kasan_arg_fault __ro_after_init; + +/* Whether KASAN is enabled at all. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_flag_enabled); +EXPORT_SYMBOL(kasan_flag_enabled); + +/* Whether to collect alloc/free stack traces. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_flag_stacktrace); + +/* Whether panic or disable tag checking on fault. */ +bool kasan_flag_panic __ro_after_init; + +/* kasan.mode=off/prod/full */ +static int __init early_kasan_mode(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_mode = KASAN_ARG_MODE_OFF; + else if (!strcmp(arg, "prod")) + kasan_arg_mode = KASAN_ARG_MODE_PROD; + else if (!strcmp(arg, "full")) + kasan_arg_mode = KASAN_ARG_MODE_FULL; + else + return -EINVAL; + + return 0; +} +early_param("kasan.mode", early_kasan_mode); + +/* kasan.stack=off/on */ +static int __init early_kasan_flag_stacktrace(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_stacktrace = KASAN_ARG_STACKTRACE_OFF; + else if (!strcmp(arg, "on")) + kasan_arg_stacktrace = KASAN_ARG_STACKTRACE_ON; + else + return -EINVAL; + + return 0; +} +early_param("kasan.stacktrace", early_kasan_flag_stacktrace); + +/* kasan.fault=report/panic */ +static int __init early_kasan_fault(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "report")) + kasan_arg_fault = KASAN_ARG_FAULT_REPORT; + else if (!strcmp(arg, "panic")) + kasan_arg_fault = KASAN_ARG_FAULT_PANIC; + else + return -EINVAL; + + return 0; +} +early_param("kasan.fault", early_kasan_fault); + /* kasan_init_hw_tags_cpu() is called for each CPU. */ void kasan_init_hw_tags_cpu(void) { + /* + * There's no need to check that the hardware is MTE-capable here, + * as this function is only called for MTE-capable hardware. + */ + + /* If KASAN is disabled, do nothing. */ + if (kasan_arg_mode == KASAN_ARG_MODE_OFF) + return; + hw_init_tags(KASAN_TAG_MAX); hw_enable_tagging(); } @@ -27,6 +124,61 @@ void kasan_init_hw_tags_cpu(void) /* kasan_init_hw_tags() is called once on boot CPU. */ void kasan_init_hw_tags(void) { + /* If hardware doesn't support MTE, do nothing. */ + if (!system_supports_mte()) + return; + + /* If KASAN is disabled, do nothing. */ + if (kasan_arg_mode == KASAN_ARG_MODE_OFF) + return; + + /* Choose KASAN mode if kasan boot parameter is not provided. */ + if (kasan_arg_mode == KASAN_ARG_MODE_DEFAULT) { + if (IS_ENABLED(CONFIG_DEBUG_KERNEL)) + kasan_arg_mode = KASAN_ARG_MODE_FULL; + else + kasan_arg_mode = KASAN_ARG_MODE_PROD; + } + + /* Preset parameter values based on the mode. */ + switch (kasan_arg_mode) { + case KASAN_ARG_MODE_OFF: + return; + case KASAN_ARG_MODE_PROD: + static_branch_enable(&kasan_flag_enabled); + break; + case KASAN_ARG_MODE_FULL: + static_branch_enable(&kasan_flag_enabled); + static_branch_enable(&kasan_flag_stacktrace); + break; + default: + break; + } + + /* Now, optionally override the presets. */ + + switch (kasan_arg_stacktrace) { + case KASAN_ARG_STACKTRACE_OFF: + static_branch_disable(&kasan_flag_stacktrace); + break; + case KASAN_ARG_STACKTRACE_ON: + static_branch_enable(&kasan_flag_stacktrace); + break; + default: + break; + } + + switch (kasan_arg_fault) { + case KASAN_ARG_FAULT_REPORT: + kasan_flag_panic = false; + break; + case KASAN_ARG_FAULT_PANIC: + kasan_flag_panic = true; + break; + default: + break; + } + pr_info("KernelAddressSanitizer initialized\n"); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 2d3c99125996..5eff3d9f624e 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -5,6 +5,22 @@ #include #include +#ifdef CONFIG_KASAN_HW_TAGS +#include +DECLARE_STATIC_KEY_FALSE(kasan_flag_stacktrace); +static inline bool kasan_stack_collection_enabled(void) +{ + return static_branch_unlikely(&kasan_flag_stacktrace); +} +#else +static inline bool kasan_stack_collection_enabled(void) +{ + return true; +} +#endif + +extern bool kasan_flag_panic __ro_after_init; + #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #else diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 25ca66c99e48..7d86af340148 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -99,6 +99,10 @@ static void end_report(unsigned long *flags) panic_on_warn = 0; panic("panic_on_warn set ...\n"); } +#ifdef CONFIG_KASAN_HW_TAGS + if (kasan_flag_panic) + panic("kasan.fault=panic set ...\n"); +#endif kasan_enable_current(); } @@ -161,8 +165,8 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, (void *)(object_addr + cache->object_size)); } -static void describe_object(struct kmem_cache *cache, void *object, - const void *addr, u8 tag) +static void describe_object_stacks(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) { struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); @@ -190,7 +194,13 @@ static void describe_object(struct kmem_cache *cache, void *object, } #endif } +} +static void describe_object(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) +{ + if (kasan_stack_collection_enabled()) + describe_object_stacks(cache, object, addr, tag); describe_object_addr(cache, object, addr); } From patchwork Tue Nov 10 22:20:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B16FEC55ABD for ; Tue, 10 Nov 2020 23:47:36 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3E365207E8 for ; Tue, 10 Nov 2020 23:47:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="qxovB6Bp"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="pkAAotds" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E365207E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fv4kCwzQQIv9FHYW57UI2PmI7AnW2x9cehJX0koHasA=; b=qxovB6BpSH5eXhU95QePcWde7 IGgis24QIw7XSTq9eQdChrBnvuiQF+B8Sr3rqmvqJb8iI76QvrQX94MlRf40ORqkbwH2bDYBcBy5D 4voE6vJvP9nOAFTjHe2XBLvsxBd0dTXxq2KrOpEwcKYIrQUsxDeDj7BpFQJCSEk/Z1Xan6hY6SRdu JSGeDhnSDzR85X5lMr5LEc8CNATloeamUhid4DHFlJB+iHzxWR19/We4cafujXfGpXWyiklwocQkv WKkhMTvpYS05ODQjLEWJpRVVK4LmKeOl7ehVcYw/a90zIl96++KIy9L6cf3RdqB30OFTTxrXZ/GfG rnbJZuGNg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcdLU-0002cd-NP; Tue, 10 Nov 2020 23:46:40 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0d-0000Pf-F7 for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:27 +0000 Received: by mail-wm1-x349.google.com with SMTP id k128so1861828wme.7 for ; Tue, 10 Nov 2020 14:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=o1/1nos0zFZPL9mIFzV6FCy+ZXGKjAVVVyDw35QavK4=; b=pkAAotdsfAMsRZfaOpyxx7Rz2i5IkNySqQ7JPwm6WoXvf8tTRT3Q89WWzscOmM4lDh 1KGi706m82ncxB6mVp4FBO1UcMmWP8Gb0P8957QptILHOywu1qNMwfuKYw3HLwJ9BIMN WNTnXdEzAFWj/LpZ/WJjlmW5C8Hyetp98G3fl0GF/uO7ZxWVbMSnZNzJngYMewened/U 8ELme7jBq1mJS2gzSdgGD8AqztiMTRgJDxQbGuOpP/ukvKwuWvZDTgQrTDnnTg0IZYvm 0uC0bb03Tj4o97c1kQQIztRVIJcUsn4FyB+PpcD9AtSK1NOnzl4c5/8ycTVhjFz1ZQAz fHkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=o1/1nos0zFZPL9mIFzV6FCy+ZXGKjAVVVyDw35QavK4=; b=iWFKEw2NSGsMe8AGVOvJGxkuAYthHba7BJHcLaKPJj6IzzMW0s5T9zHRIJsrWKhh2T F6w4y8mMmxRzjQ+HFLrbpfjSy6F5MLMkgW2ZtnSA0oifOHTDokBw4K4+kHPog7Zz9BbY zPyiPqrCwc47y72MfE/aDF1PaYgAME7saHkQjYmsbJ5C+hYYH1PcrzFJUbiurVKSGLk2 MQjv2af/Ka90GJ4tANlm/fWTAZUGhUEweJhTOiBKFudnpn6KA0rQWF+C2UHMAHYjwbGX gljvm8MYQEN0rd40o96ZgkDmd+l49DKrH2pRcl4UM63UA7W/d30Rm61Wa1QszcQze0LE 6y/Q== X-Gm-Message-State: AOAM530WV4j0VYVD3ENP1OfjpP3U37w1GYHGS4Vezq4j2awfKqO1yJ1l d/QJ4rsQmBugJgWZWpJsT8ujWMxIiHSboRTV X-Google-Smtp-Source: ABdhPJzmqSrplyUCQ4ds7REpp9Bb8+zoLNTCA47/0SwzZE9WT287XXOluXwd62ndMzbeEtx/MmT/5J+W8mDSzLx6 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:6000:10e:: with SMTP id o14mr26646190wrx.225.1605046857657; Tue, 10 Nov 2020 14:20:57 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:16 +0100 In-Reply-To: Message-Id: <22c1a837d4d0c0b241a700c88f180f5e831a1953.1605046662.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 12/20] kasan, mm: check kasan_enabled in annotations From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172103_999080_CC0E452D X-CRM114-Status: GOOD ( 18.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Declare the kasan_enabled static key in include/linux/kasan.h and in include/linux/mm.h and check it in all kasan annotations. This allows to avoid any slowdown caused by function calls when kasan_enabled is disabled. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/I2589451d3c96c97abbcbf714baabe6161c6f153e --- include/linux/kasan.h | 220 ++++++++++++++++++++++++++++++++---------- include/linux/mm.h | 22 +++-- mm/kasan/common.c | 60 ++++++------ 3 files changed, 216 insertions(+), 86 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index f1a5042ae4fc..779f8e703982 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -2,6 +2,7 @@ #ifndef _LINUX_KASAN_H #define _LINUX_KASAN_H +#include #include struct kmem_cache; @@ -74,56 +75,179 @@ static inline void kasan_disable_current(void) {} #ifdef CONFIG_KASAN -void kasan_alloc_pages(struct page *page, unsigned int order); -void kasan_free_pages(struct page *page, unsigned int order); +struct kasan_cache { + int alloc_meta_offset; + int free_meta_offset; +}; -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags); +#ifdef CONFIG_KASAN_HW_TAGS +DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); +static inline kasan_enabled(void) +{ + return static_branch_likely(&kasan_flag_enabled); +} +#else +static inline kasan_enabled(void) +{ + return true; +} +#endif -void kasan_unpoison_data(const void *address, size_t size); -void kasan_unpoison_slab(const void *ptr); +void __kasan_alloc_pages(struct page *page, unsigned int order); +static inline void kasan_alloc_pages(struct page *page, unsigned int order) +{ + if (kasan_enabled()) + __kasan_alloc_pages(page, order); +} -void kasan_poison_slab(struct page *page); -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); -void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, - const void *object); +void __kasan_free_pages(struct page *page, unsigned int order); +static inline void kasan_free_pages(struct page *page, unsigned int order) +{ + if (kasan_enabled()) + __kasan_free_pages(page, order); +} -void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, - gfp_t flags); -void kasan_kfree_large(void *ptr, unsigned long ip); -void kasan_poison_kfree(void *ptr, unsigned long ip); -void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags); -void * __must_check kasan_krealloc(const void *object, size_t new_size, - gfp_t flags); +void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags); +static inline void kasan_cache_create(struct kmem_cache *cache, + unsigned int *size, slab_flags_t *flags) +{ + if (kasan_enabled()) + __kasan_cache_create(cache, size, flags); +} -void * __must_check kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags); -bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); +size_t __kasan_metadata_size(struct kmem_cache *cache); +static inline size_t kasan_metadata_size(struct kmem_cache *cache) +{ + if (kasan_enabled()) + return __kasan_metadata_size(cache); + return 0; +} -struct kasan_cache { - int alloc_meta_offset; - int free_meta_offset; -}; +void __kasan_unpoison_data(const void *addr, size_t size); +static inline void kasan_unpoison_data(const void *addr, size_t size) +{ + if (kasan_enabled()) + __kasan_unpoison_data(addr, size); +} + +void __kasan_unpoison_slab(const void *ptr); +static inline void kasan_unpoison_slab(const void *ptr) +{ + if (kasan_enabled()) + __kasan_unpoison_slab(ptr); +} + +void __kasan_poison_slab(struct page *page); +static inline void kasan_poison_slab(struct page *page) +{ + if (kasan_enabled()) + return __kasan_poison_slab(page); +} + +void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object); +static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +{ + if (kasan_enabled()) + return __kasan_unpoison_object_data(cache, object); +} + +void __kasan_poison_object_data(struct kmem_cache *cache, void *object); +static inline void kasan_poison_object_data(struct kmem_cache *cache, void *object) +{ + if (kasan_enabled()) + __kasan_poison_object_data(cache, object); +} + +void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, + const void *object); +static inline void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) +{ + if (kasan_enabled()) + return __kasan_init_slab_obj(cache, object); + return (void *)object; +} + +bool __kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); +static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip) +{ + if (kasan_enabled()) + return __kasan_slab_free(s, object, ip); + return false; +} + +void * __must_check __kasan_slab_alloc(struct kmem_cache *s, + void *object, gfp_t flags); +static inline void * __must_check kasan_slab_alloc(struct kmem_cache *s, + void *object, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_slab_alloc(s, object, flags); + return object; +} -size_t kasan_metadata_size(struct kmem_cache *cache); +void * __must_check __kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags); +static inline void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_kmalloc(s, object, size, flags); + return (void *)object; +} + +void * __must_check __kasan_kmalloc_large(const void *ptr, + size_t size, gfp_t flags); +static inline void * __must_check kasan_kmalloc_large(const void *ptr, + size_t size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_kmalloc_large(ptr, size, flags); + return (void *)ptr; +} + +void * __must_check __kasan_krealloc(const void *object, + size_t new_size, gfp_t flags); +static inline void * __must_check kasan_krealloc(const void *object, + size_t new_size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_krealloc(object, new_size, flags); + return (void *)object; +} + +void __kasan_poison_kfree(void *ptr, unsigned long ip); +static inline void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_poison_kfree(ptr, ip); +} + +void __kasan_kfree_large(void *ptr, unsigned long ip); +static inline void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_kfree_large(ptr, ip); +} bool kasan_save_enable_multi_shot(void); void kasan_restore_multi_shot(bool enabled); #else /* CONFIG_KASAN */ +static inline kasan_enabled(void) +{ + return false; +} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} - static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} - -static inline void kasan_unpoison_data(const void *address, size_t size) { } -static inline void kasan_unpoison_slab(const void *ptr) { } - +static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } +static inline void kasan_unpoison_data(const void *address, size_t size) {} +static inline void kasan_unpoison_slab(const void *ptr) {} static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} @@ -134,36 +258,32 @@ static inline void *kasan_init_slab_obj(struct kmem_cache *cache, { return (void *)object; } - -static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +static inline bool kasan_slab_free(struct kmem_cache *s, void *object, + unsigned long ip) { - return ptr; + return false; +} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; } -static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} -static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags) { return (void *)object; } +static inline void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + return (void *)ptr; +} static inline void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags) { return (void *)object; } - -static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) -{ - return object; -} -static inline bool kasan_slab_free(struct kmem_cache *s, void *object, - unsigned long ip) -{ - return false; -} - -static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } +static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} +static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} #endif /* CONFIG_KASAN */ diff --git a/include/linux/mm.h b/include/linux/mm.h index 0793d03a4183..8d84a6b2fa3c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -31,6 +31,7 @@ #include #include #include +#include struct mempolicy; struct anon_vma; @@ -1414,22 +1415,30 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) #endif /* CONFIG_NUMA_BALANCING */ #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) + static inline u8 page_kasan_tag(const struct page *page) { - return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; + if (kasan_enabled()) + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; + return 0xff; } static inline void page_kasan_tag_set(struct page *page, u8 tag) { - page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); - page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; + if (kasan_enabled()) { + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; + } } static inline void page_kasan_tag_reset(struct page *page) { - page_kasan_tag_set(page, 0xff); + if (kasan_enabled()) + page_kasan_tag_set(page, 0xff); } -#else + +#else /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ + static inline u8 page_kasan_tag(const struct page *page) { return 0xff; @@ -1437,7 +1446,8 @@ static inline u8 page_kasan_tag(const struct page *page) static inline void page_kasan_tag_set(struct page *page, u8 tag) { } static inline void page_kasan_tag_reset(struct page *page) { } -#endif + +#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ static inline struct zone *page_zone(const struct page *page) { diff --git a/mm/kasan/common.c b/mm/kasan/common.c index efad5ed6a3bd..385863eaec2c 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -81,7 +81,7 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ -void kasan_alloc_pages(struct page *page, unsigned int order) +void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; unsigned long i; @@ -95,7 +95,7 @@ void kasan_alloc_pages(struct page *page, unsigned int order) kasan_unpoison_memory(page_address(page), PAGE_SIZE << order); } -void kasan_free_pages(struct page *page, unsigned int order) +void __kasan_free_pages(struct page *page, unsigned int order) { if (likely(!PageHighMem(page))) kasan_poison_memory(page_address(page), @@ -122,8 +122,8 @@ static inline unsigned int optimal_redzone(unsigned int object_size) object_size <= (1 << 16) - 1024 ? 1024 : 2048; } -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) +void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) { unsigned int orig_size = *size; unsigned int redzone_size; @@ -168,7 +168,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *flags |= SLAB_KASAN; } -size_t kasan_metadata_size(struct kmem_cache *cache) +size_t __kasan_metadata_size(struct kmem_cache *cache) { if (!kasan_stack_collection_enabled()) return 0; @@ -191,17 +191,17 @@ struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } -void kasan_unpoison_data(const void *address, size_t size) +void __kasan_unpoison_data(const void *addr, size_t size) { - kasan_unpoison_memory(address, size); + kasan_unpoison_memory(addr, size); } -void kasan_unpoison_slab(const void *ptr) +void __kasan_unpoison_slab(const void *ptr) { kasan_unpoison_memory(ptr, __ksize(ptr)); } -void kasan_poison_slab(struct page *page) +void __kasan_poison_slab(struct page *page) { unsigned long i; @@ -211,12 +211,12 @@ void kasan_poison_slab(struct page *page) KASAN_KMALLOC_REDZONE); } -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object) { kasan_unpoison_memory(object, cache->object_size); } -void kasan_poison_object_data(struct kmem_cache *cache, void *object) +void __kasan_poison_object_data(struct kmem_cache *cache, void *object) { kasan_poison_memory(object, round_up(cache->object_size, KASAN_GRANULE_SIZE), @@ -269,7 +269,7 @@ static u8 assign_tag(struct kmem_cache *cache, const void *object, #endif } -void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, +void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_meta; @@ -288,7 +288,7 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, return (void *)object; } -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, +static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { u8 tag; @@ -331,9 +331,9 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, return IS_ENABLED(CONFIG_KASAN_GENERIC); } -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) { - return __kasan_slab_free(cache, object, ip, true); + return ____kasan_slab_free(cache, object, ip, true); } static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) @@ -341,7 +341,7 @@ static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); } -static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, +static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags, bool keep_tag) { unsigned long redzone_start; @@ -373,20 +373,20 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, return set_tag(object, tag); } -void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, - gfp_t flags) +void * __must_check __kasan_slab_alloc(struct kmem_cache *cache, + void *object, gfp_t flags) { - return __kasan_kmalloc(cache, object, cache->object_size, flags, false); + return ____kasan_kmalloc(cache, object, cache->object_size, flags, false); } -void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, - size_t size, gfp_t flags) +void * __must_check __kasan_kmalloc(struct kmem_cache *cache, const void *object, + size_t size, gfp_t flags) { - return __kasan_kmalloc(cache, object, size, flags, true); + return ____kasan_kmalloc(cache, object, size, flags, true); } -EXPORT_SYMBOL(kasan_kmalloc); +EXPORT_SYMBOL(__kasan_kmalloc); -void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, +void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; @@ -411,7 +411,7 @@ void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, return (void *)ptr; } -void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) +void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; @@ -421,13 +421,13 @@ void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); + return __kasan_kmalloc_large(object, size, flags); else - return __kasan_kmalloc(page->slab_cache, object, size, + return ____kasan_kmalloc(page->slab_cache, object, size, flags, true); } -void kasan_poison_kfree(void *ptr, unsigned long ip) +void __kasan_poison_kfree(void *ptr, unsigned long ip) { struct page *page; @@ -440,11 +440,11 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) } kasan_poison_memory(ptr, page_size(page), KASAN_FREE_PAGE); } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); + ____kasan_slab_free(page->slab_cache, ptr, ip, false); } } -void kasan_kfree_large(void *ptr, unsigned long ip) +void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); From patchwork Tue Nov 10 22:20:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9329CC388F7 for ; Tue, 10 Nov 2020 23:47:32 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3CFB1207E8 for ; Tue, 10 Nov 2020 23:47:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="EftlDDcA"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Sorzbxjl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3CFB1207E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=uiJ6yo17oEWh2D6WQGw+UTz4Bza9/Qxdp5ZGLuzPtYA=; b=EftlDDcAJ7SCOw2MtdI1jAs1X fbwrv2nM6ljZRMfXLhIKNX69uwTqP2gvSadF7wz31b8CNoRIPEmr4rbzRGKFKEKoHVRFiyvXUPZJ+ 15rSvoFwujL/dVWXipu0yDwa4+g53PijoEwjUehPqAud8kzy5l9gqMRZzE+ASR7ZTrY3QA2EQYXzy 1p14INZ1yjEEt9dEw33zfdaGgxCHn9Ihb9whNWsu0USDeqxHZ//0GC8SW0ZmgS5NrWOki6N1ykvxx FsrUuCfeGnxBwM9/fbOkl8E2Hh3C+tesl6oplHC6F3rxvaw8xuOI/ICR7jctdTZBCW6B7eQXvVPPu StmrQboUA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcdLV-0002cp-VN; Tue, 10 Nov 2020 23:46:41 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0h-0000Qj-2x for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:29 +0000 Received: by mail-qt1-x84a.google.com with SMTP id o1so2160623qtp.7 for ; Tue, 10 Nov 2020 14:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=JVfCx5YoVAhsMAMzq5Ib4xNDNXTPugCAXEdF/9ZfK1c=; b=Sorzbxjl434teafjW3faZG7zFZzfm93UE5ni1XpfIncUp10ibp8jpLYD70flrD0+2/ /ZaY5ksYh6XtmAQzE15CeAKLXWStbYLxHdCNnUebaCqUpyj9z4JlZq4J3u8IfE10A5Lq Zr3qjbG+frRqzlU2wYjRNGXaopuLOQnurA51QAgFYIIrikpen/tETHXDcodSPHKR/9By HM+aXQXai3muVVFES7+dvfTZwk3Q1hnpbedVcj8nRXP8nq6OW22Mr8dJd3a/ILVJSdD3 So1MJMa8xAODLMxUsWoHgul0P/PCAhB0ZKwHZbNh8Nu2VglU+Jp9Lss9YGFtCDcHEgIE Xc4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=JVfCx5YoVAhsMAMzq5Ib4xNDNXTPugCAXEdF/9ZfK1c=; b=meT2OUChPWwAVOvnXWP2fiYwDd6ugVlSm0d1Rv3uzueV0gIEjxfNG/o9So67cwFYhM b1aa7eS4AwR5tgTYJmoyFzl1/LejCb5uuCoP3rnRkTk4hfUZRsJF7PUrQBDgJF0+PRsf DPwpM0nV5wNB3lzv+4BouKx8LlWeEXLYMboRsXbZjg92q1zxusJS4PytcAXck7JFlBpu 6vyiHN+/XdMNYlxP/Z/mnsOD7tAhot2p80MIM7mkK4FR+/ND+WuVE9tfdyh67B+HnzkI b3o/rjs9JWUtZr1qX5SxrJWOBtHMvuDoFw6GZ3WxLY4FCJ48piPzVnXcT+dbqaKUyimH r3Dg== X-Gm-Message-State: AOAM532LRGmDI53+kahYHR2udA+boowGwt7bWHrH+WEEiM/mzWRwzsEU qotqNvtbFy308QjU9FryQcjqfC8LP939WO1J X-Google-Smtp-Source: ABdhPJz9X4I0fQKQ6rAPw02HsIflV/nkdacrsdfDSd0mERCmB5vYjtdDNjyd6StrpyBIrGYzVfJD2gDIvmTN5dGg X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:e790:: with SMTP id x16mr12244792qvn.21.1605046859712; Tue, 10 Nov 2020 14:20:59 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:17 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 13/20] kasan: simplify kasan_poison_kfree From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172108_335668_080DCA86 X-CRM114-Status: GOOD ( 13.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kasan_poison_kfree() is currently only called for mempool allocations that are backed by either kmem_cache_alloc() or kmalloc(). Therefore, the page passed to kasan_poison_kfree() is always PageSlab() and there's no need to do the check. Remove it. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/If31f88726745da8744c6bea96fb32584e6c2778c Reviewed-by: Marco Elver --- mm/kasan/common.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 385863eaec2c..819403548f2e 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -432,16 +432,7 @@ void __kasan_poison_kfree(void *ptr, unsigned long ip) struct page *page; page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - kasan_poison_memory(ptr, page_size(page), KASAN_FREE_PAGE); - } else { - ____kasan_slab_free(page->slab_cache, ptr, ip, false); - } + ____kasan_slab_free(page->slab_cache, ptr, ip, false); } void __kasan_kfree_large(void *ptr, unsigned long ip) From patchwork Tue Nov 10 22:20:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8351AC388F7 for ; Tue, 10 Nov 2020 23:47:45 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 03FCE207E8 for ; Tue, 10 Nov 2020 23:47:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="s2EPVCXp"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="pbUFHUwD"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="nWOy4NUD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 03FCE207E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2ZMwZ4w3FAr2k9MF8BbIfwk8ud+xFcojRs6AvC/l8ho=; b=s2EPVCXpMw6rU5uDWJ1a1SUuG hTnvR6cEs/psx2nfdOfgYmvGIubsjBYeSmwIMnLbi1rAqAwGO46HBN2qMJbO5Yan5nVqHhs5vtiMq L4lozRnOnUDa7EGJc7nMhyb0Jf0OvoviM8xz0+jRtHH1Ij+INZOcf7V7VdgVpil/w2o/UgYcbN5ko VWh9Vc2FNVKjb4qhCBZ/R+LoN5yqaQwI4KXbzdVKaTCDPSxPKEch/g6nUh5w4nKa48iI4XCIY4Y8u cjL8dLI0YegAw4iVIVk497DgHmuKKeDbQKV9Yp99UScWPEbH2s54mC3URudt7gjyqA1K7DZKp0MLd e1pg03xBQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcdLa-0002f7-FU; Tue, 10 Nov 2020 23:46:46 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccB2-0005D9-Su for linux-arm-kernel@merlin.infradead.org; Tue, 10 Nov 2020 22:31:49 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=GOGJv8k49mZ7izGKRKf2lz4nE5+QAATrVFhHt9hU4vk=; b=pbUFHUwDEe+qZjrZZA1LN1AKfP SA9DdHCV+Hs3TFZZAt2xFOjlZ/4VT4ZvMUkA1UYK/0tKSpLl+yoZjoO3l6dU2teQv4hR5OduBrQKr 9SGk2PLf80ONQpSnyycG3Udip6Vn070z5XHuybEDoksXEQBHOUy8QIg9Fyema6VeexaPYtAaDTU3D czpEOr29Zp1R0+sH5USGkVGaf5Tcm1HA1QkoqQokFXmNZnf7M2JCY9Ofn/u0BZLKwT1y6xr9SqWiy MEn9UF3uvMbcMvCN+SBRZieV0kzG3GP6H5TnWte4LuSlq8tHWX8XOIRWcvCkGwKvnTeTLL15UGYTT 5wSS5kFg==; Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0f-0008SV-4d for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:11 +0000 Received: by mail-wm1-x349.google.com with SMTP id 8so1231189wmg.6 for ; Tue, 10 Nov 2020 14:21:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=GOGJv8k49mZ7izGKRKf2lz4nE5+QAATrVFhHt9hU4vk=; b=nWOy4NUDHFOJEkxt4HeQuuF7ItikIUR8FEDsoSuy97fbvULA7lagn7eN3aC6vpstwF xnpFycGGrQwC8h/x/TXHWw0CW+G4Fjd4Slr9kUN2ThO1uqV1a17ThYYrVviz2QzvLoEO oWgn9eARxtVrIcMhHtbgK5vf157PytkMdr0TozM3dqyYiXD2fg1TZOmiSH2tfoQNyg/p JnBT4n1XAgbGk9xq5rgIyLRbUehOrg15Syn4bQ069qKNf+WaK4exp2ytMFY2Ps2HZxgG ZfB1I1UrMfNw1Gz86u3BfC24ZC1yd4ha4aaLaa9otfPc1GMwphpeAJEn2q3FmZCaeW05 uz7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=GOGJv8k49mZ7izGKRKf2lz4nE5+QAATrVFhHt9hU4vk=; b=jSwHYc/WdbrGwMrh/QZQGPBXWo48w3wxB81wZ4/SmajWOt0FYfFsyMfhuBe+6Ay7WR Vnyzm2k4sOqEH1KFh/tdPKGfgMdiwK5+BZeVOVlsn3dpr0nhp0R48imhtVVbRHT1c10Q LjAWdSGjBNlyBGHq34EnMUqUbnq/kZtxmH/9QNFv5BMlqtTvrntwQ5s1jhDUPfzoHZVb TfTGM1t/5fLHwxHiscWttZ+a5UU71P/WWcvT8Xg7cy+eCoT2Rn2Bxy81Fedc4Nhn8gbk 3Ei2tJebNG+2dWyWV36fqGUZnexgheHclHk/ChTYRc/hGbFiUqxc4TO7zElRBDaAUZkw eiwg== X-Gm-Message-State: AOAM533XSyM6uay8+pKqek8s6P71XBLO2RDjvdo6uTOp1M45H51NAlk1 YrlcmtMD1MJHSeh5IJcGKp9rKybxho7kMqM1 X-Google-Smtp-Source: ABdhPJye7812Tl8RDG0JMOImadD+I+Ef01WcUzHZ3QitQU3NwVahUAiU8MWUn0N3DMR/yxe7K7D4GjlglR/6liPr X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a7b:cbd7:: with SMTP id n23mr302280wmi.142.1605046862286; Tue, 10 Nov 2020 14:21:02 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:18 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 14/20] kasan, mm: rename kasan_poison_kfree From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_222106_643203_6B496B6A X-CRM114-Status: GOOD ( 13.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Rename kasan_poison_kfree() to kasan_slab_free_mempool() as it better reflects what this annotation does. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/I5026f87364e556b506ef1baee725144bb04b8810 --- include/linux/kasan.h | 16 ++++++++-------- mm/kasan/common.c | 16 ++++++++-------- mm/mempool.c | 2 +- 3 files changed, 17 insertions(+), 17 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 779f8e703982..534ab3e2935a 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -177,6 +177,13 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned return false; } +void __kasan_slab_free_mempool(void *ptr, unsigned long ip); +static inline void kasan_slab_free_mempool(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_slab_free_mempool(ptr, ip); +} + void * __must_check __kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); static inline void * __must_check kasan_slab_alloc(struct kmem_cache *s, @@ -217,13 +224,6 @@ static inline void * __must_check kasan_krealloc(const void *object, return (void *)object; } -void __kasan_poison_kfree(void *ptr, unsigned long ip); -static inline void kasan_poison_kfree(void *ptr, unsigned long ip) -{ - if (kasan_enabled()) - __kasan_poison_kfree(ptr, ip); -} - void __kasan_kfree_large(void *ptr, unsigned long ip); static inline void kasan_kfree_large(void *ptr, unsigned long ip) { @@ -263,6 +263,7 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object, { return false; } +static inline void kasan_slab_free_mempool(void *ptr, unsigned long ip) {} static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags) { @@ -282,7 +283,6 @@ static inline void *kasan_krealloc(const void *object, size_t new_size, { return (void *)object; } -static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} #endif /* CONFIG_KASAN */ diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 819403548f2e..60793f8695a8 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -336,6 +336,14 @@ bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return ____kasan_slab_free(cache, object, ip, true); } +void __kasan_slab_free_mempool(void *ptr, unsigned long ip) +{ + struct page *page; + + page = virt_to_head_page(ptr); + ____kasan_slab_free(page->slab_cache, ptr, ip, false); +} + static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) { kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); @@ -427,14 +435,6 @@ void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flag flags, true); } -void __kasan_poison_kfree(void *ptr, unsigned long ip) -{ - struct page *page; - - page = virt_to_head_page(ptr); - ____kasan_slab_free(page->slab_cache, ptr, ip, false); -} - void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) diff --git a/mm/mempool.c b/mm/mempool.c index f473cdddaff0..b1f39fa75ade 100644 --- a/mm/mempool.c +++ b/mm/mempool.c @@ -104,7 +104,7 @@ static inline void poison_element(mempool_t *pool, void *element) static __always_inline void kasan_poison_element(mempool_t *pool, void *element) { if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc) - kasan_poison_kfree(element, _RET_IP_); + kasan_slab_free_mempool(element, _RET_IP_); else if (pool->alloc == mempool_alloc_pages) kasan_free_pages(element, (unsigned long)pool->pool_data); } From patchwork Tue Nov 10 22:20:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 736BBC388F7 for ; Tue, 10 Nov 2020 22:36:38 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DDAFA205ED for ; Tue, 10 Nov 2020 22:36:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="VpBEqbud"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="uANlXR3C" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DDAFA205ED Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=+abstXjLoCrKOsI1EAxAxVhw/DyenPY41YxF+0B0mwc=; b=VpBEqbud6wzPq/jVuJedLmH7l FUM5JvTEjgRebtX0V6eYRUzvKmeFBU24/x/6znPaJa/ERWH3CsdxdNKmJTDIt0gKrDzjT0ZI2TD7O bQfbsMB7iO+fUMDsMtvVTYR9clgj11HDKt50gGKmNiaWOKWBfZ8xdRCTcFHz9WsEo0frVu4tcbrrm SsE2IyUDYp/xMUm4LxdjHkb7R1ShX2BodaPfXsQCXUowmdpFJyosTNi8T52LPmbVZxoQOejdsXhM8 Z/a3ikhiKuHSjyai6kTy+4h1jPOPHRJLW/4yWzBlT/1aDa7V6Yq7qk3ekhWeBeMZpTgqaWGmP0OaB gMBThGxAQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccFC-00076F-HQ; Tue, 10 Nov 2020 22:36:06 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0k-0000TJ-Uj for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:32 +0000 Received: by mail-wr1-x449.google.com with SMTP id w17so5440680wrp.11 for ; Tue, 10 Nov 2020 14:21:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=pq+zpW/Hw4XDGIR7N3eqf6sjflzq5qp4Op5FmJmOOa8=; b=uANlXR3CSpy0OVPmNQ73PxY92qov20UVaMIwXUyPX+fAIprSkZe9uSPL4bQVMEC/m/ t6wBvgClC8i+jwTY287EzT22FDgFPZZShu17Fwwgr+vWpgadpDkWYTs9vdSp0HV+9VZq Kx+M2yf4bagD7qpj2zf7ge0r5Zz+1FN0ycJy5Vu0t8QfpOdv9cijYOOLD8Vq/+Peju7k u3843665kwiM52RAxyenrbPXVhQuJ48TUwhSG4+Wo05tfFV9dEDRxFmFk0olCq4CS7+H zj5ZcfrDTG76qZOUCtww5kQJxsH6bWa0hEpqDIpxqTvvTx3BnIhmBpX9ELGPzultfHVR wsDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=pq+zpW/Hw4XDGIR7N3eqf6sjflzq5qp4Op5FmJmOOa8=; b=NwxCHlbEVjS98hDCUH7Jw9OzTTSoumDWctUezBJVBfiFQeh3F8tSx5pqZ1Iq76FpT0 AUel2/7nRLEmiAN72ATpBd4t4dDMRqCQ2cS8Pih/P/6xveH1tKbrJ360rEplmpsoWylr a8mlIUbpcKodGsDO2Boy+y1G9DTcUAhqlejEH/U1yfSsQYEn2RHiCgPS1TDgfpDsI0jY yJpuOdICL4kvOKvDAHpPjvskZfLoron5xTagskArZwLlTVVEY1isOO30e1pYI0+0DWMb aotBMkeq/5SCeiW6TL5pv9YzlpFf1dhBESZkT1czAkZhFKAJBE+yNxDUIkHT60bFBKgT T7IQ== X-Gm-Message-State: AOAM532D9s7YWdkn13u5GHQFS43iy9gCsptNuOM1uKSa3/sAb1k5tjVe 7Sv8DdEbw2v1ECEuZJBn5coXoQSOK1Me8aij X-Google-Smtp-Source: ABdhPJww9dm2MDRGqKJLg0tAFFPdHdF+3n5YlkqvRFYDl3KYCnu7Zb3EA2VEUW1wWZgN/7f9Jq+e6O9RKx82b94E X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:b387:: with SMTP id c129mr303876wmf.58.1605046864904; Tue, 10 Nov 2020 14:21:04 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:19 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 15/20] kasan: don't round_up too much From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172111_704178_2F0773A6 X-CRM114-Status: GOOD ( 13.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For hardware tag-based mode kasan_poison_memory() already rounds up the size. Do the same for software modes and remove round_up() from the common code. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Ib397128fac6eba874008662b4964d65352db4aa4 Reviewed-by: Marco Elver --- mm/kasan/common.c | 8 ++------ mm/kasan/shadow.c | 1 + 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 60793f8695a8..69ab880abacc 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -218,9 +218,7 @@ void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object) void __kasan_poison_object_data(struct kmem_cache *cache, void *object) { - kasan_poison_memory(object, - round_up(cache->object_size, KASAN_GRANULE_SIZE), - KASAN_KMALLOC_REDZONE); + kasan_poison_memory(object, cache->object_size, KASAN_KMALLOC_REDZONE); } /* @@ -293,7 +291,6 @@ static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, { u8 tag; void *tagged_object; - unsigned long rounded_up_size; tag = get_tag(object); tagged_object = object; @@ -314,8 +311,7 @@ static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, return true; } - rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); - kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE); + kasan_poison_memory(object, cache->object_size, KASAN_KMALLOC_FREE); if (!kasan_stack_collection_enabled()) return false; diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 8e4fa9157a0b..3f64c9ecbcc0 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -82,6 +82,7 @@ void kasan_poison_memory(const void *address, size_t size, u8 value) * addresses to this function. */ address = kasan_reset_tag(address); + size = round_up(size, KASAN_GRANULE_SIZE); shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); From patchwork Tue Nov 10 22:20:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77341C4742C for ; Tue, 10 Nov 2020 22:38:05 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F36A8206B6 for ; Tue, 10 Nov 2020 22:38:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="WOk4/Atm"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="QP7TuVPx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F36A8206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=PYHVGfIfYqkNCAtkYRqxu4IJAWna9zNqio1VT5mAWaA=; b=WOk4/Atme9ejDL/0sPP/av9fT O2crtdNl7stao/ffy5hoUA3WrY2hGjVpBi0/kU3gmaae3o0i4CWGzTCV+Tzpbm3/jKB6ro/oTE3Ks Y7qU0v3ZcWUwrl4euOyedQZhARaA0uBPlCp7jYx8MyH8tLIP/+B790k2EPkUWdwYC/D6NoZ3555Fq Vw8YoSdC+iQE3uNpWqjCqjBcTXmlr6g+GI2REU8hvph1cKevwqQhpX7WfqSOyoFo57Cyfkzur/FT0 +fj6glfspn5pnNGKInw9mezi1zjDBqXQoNot4EgEyiNm+9iH6ySQmYHYShDRxu5uNHYsOvZLU/Miv FGDSh79aQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccFT-0007Ev-Kd; Tue, 10 Nov 2020 22:36:23 +0000 Received: from mail-ed1-x54a.google.com ([2a00:1450:4864:20::54a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0q-0000Us-Tx for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:37 +0000 Received: by mail-ed1-x54a.google.com with SMTP id l24so1540409edt.16 for ; Tue, 10 Nov 2020 14:21:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=UUb16eez1SBPDEo4INNWK7vyjFOxlmk1eUi1izfNnGk=; b=QP7TuVPxCEiWmmHgnEKJnNoGay3AQqR5W+nqoq/mV5uvCvY6CbJgGKtXz82KEG4kId KNMjnlvqyunp1Kqhwt3peDe0Vg4O+DQxvkw+MnRVzCq5Nvf0Ou/xBeqbAPV2XSIrqswu 9aKboBYNZGPP5klIdn8CMN07pjj/wj34ufercro9wLmyR/w6wCqGr66F7ztDXO4P9Y5P QoahP92EwMw+TLRC0EtcxvwFqReTRl6Xn1duRONdFajcKPyltvk3QlSDJFIhoKFiJo9K an+ewDBQH6xWC/JrFoJ/Z1WbXQtbEDByyh/6FVx/JJ1FMmIHAZ+hzFOiqtvuORTqVwpA AXpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=UUb16eez1SBPDEo4INNWK7vyjFOxlmk1eUi1izfNnGk=; b=Vb18fZM35PjuGuhe02FueWUUdkPfukGP2jyu7sstlIz4DHJW+6Fya0BGLoltGiqE0l +BeWP7zpp/keumTHH2o8r5oZXK9Cta+x/nIT/uvLG/wAV+rV5oa5VCj3P3RrRWOhc18V KBSxjauSYErIlyCQkwVqy8P6rTqK7vm6Oo7fWcQfiuYyjWjZ0OgjtLRQc1OeiCjAAVtr ssRKomDHdfVcbjDj8v/7oZ1ymdHqP5SKq61MBQdx9zwWUC3QZnhh8AfruiGVCW3SMf9p s/R4nMlZoOd26IoVhK5Yfl0kJQD/Oe/FkOQLEqFhcUrwu7YKInKq8Bx5TuL/ZDVlGbsI iG0Q== X-Gm-Message-State: AOAM531VRKPrcz5U7WNxrH1SD76FAWIOG3wMzY4SCVPnOAri0iXBnve9 KY7F4uDOPx4Do+LjF4wZNIjZ7qv/sy2GZUVP X-Google-Smtp-Source: ABdhPJy1APWb8IPIno5fyCxmFvAiY4V9ib3nnVEVsGk5gNukRVXnCQVuL8cF06lZdJfE+QXxRpZDFZp+aOWcujy4 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a17:906:b043:: with SMTP id bj3mr21951070ejb.543.1605046867367; Tue, 10 Nov 2020 14:21:07 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:20 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 16/20] kasan: simplify assign_tag and set_tag calls From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172117_468255_83983129 X-CRM114-Status: GOOD ( 15.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org set_tag() already ignores the tag for the generic mode, so just call it as is. Add a check for the generic mode to assign_tag(), and simplify its call in ____kasan_kmalloc(). Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/I18905ca78fb4a3d60e1a34a4ca00247272480438 Reviewed-by: Marco Elver --- mm/kasan/common.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 69ab880abacc..40ff3ce07a76 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -238,6 +238,9 @@ void __kasan_poison_object_data(struct kmem_cache *cache, void *object) static u8 assign_tag(struct kmem_cache *cache, const void *object, bool init, bool keep_tag) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return 0xff; + /* * 1. When an object is kmalloc()'ed, two hooks are called: * kasan_slab_alloc() and kasan_kmalloc(). We assign the @@ -280,8 +283,8 @@ void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, __memset(alloc_meta, 0, sizeof(*alloc_meta)); } - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) - object = set_tag(object, assign_tag(cache, object, true, false)); + /* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */ + object = set_tag(object, assign_tag(cache, object, true, false)); return (void *)object; } @@ -362,9 +365,7 @@ static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, KASAN_GRANULE_SIZE); redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_GRANULE_SIZE); - - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) - tag = assign_tag(cache, object, false, keep_tag); + tag = assign_tag(cache, object, false, keep_tag); /* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */ kasan_unpoison_memory(set_tag(object, tag), size); From patchwork Tue Nov 10 22:20:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFE5EC4742C for ; Tue, 10 Nov 2020 22:37:38 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6769C2068D for ; Tue, 10 Nov 2020 22:37:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="QgdO4A8N"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="wL8k50M1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6769C2068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=YOerkGaWfQsULtsxXgnEixy89I8IEuAC5cA5Pe0zTeo=; b=QgdO4A8NL93cXz6UE7mYV/2Oz L4j+3RpjFbcA9lq1a3B15bnBUzQ4hYnLXHy40zAKSHgNQgRVk8DnMhJCy2Y5srYU/0TeGQ87mILoA zm5MlHlBZ8oSOmISW2W9dgTLG8PHiFx11YOGOhRiMMf1ep/upzrf+/lmiEnqhNFKWWZOrzxLNU1fK 7MHbZixPXGHitQrFRT8B4kA3XgNfnt0vqk0Jzw0feT5WhFA6+tw+w6GobTvpDMJQ3U50f9A4Ejj+X QTFdQ8DQkUPBTx1xzCmYmhmGNrHwbyu8ZCHSFhLCG/2HfO+/S4jTkBGfswX2nK45eRhOyCQF2AerO jhCExwdig==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccG3-0007To-EA; Tue, 10 Nov 2020 22:36:59 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0s-0000Va-Rz for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:41 +0000 Received: by mail-wr1-x449.google.com with SMTP id y2so3571898wrl.3 for ; Tue, 10 Nov 2020 14:21:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=MgiGs2FK0RQ4JNcJFnsxjsC4BWa7qeyNid6Uf5IWwBE=; b=wL8k50M1AQHx0VbQL3xi2XuIKzEGXDqvuUVrYzcIQ94olxXzgz0KX0q6k1f4qHRqNr llatx03+5CGH/foSZCzS8+mbo9AsyH1v/WHAAFQvSGtKVFIsW7M2JVSL/RY77Ni7+UKv FEqOLEs61nCxCIKEpESSzhVsL4LPWnOcrnfbp/V6bK7D8N8vSYSVH1Vn1WBuBPKz1Xto AJ3cdtZ2Uum98aAUASzeWGCdAOuYczsaP5Ke24OgbosVmgCMZ6eufONNfo/5pI4knmaW 5ez6cUJOUeYVfi5WRIAL3JvIxyUF+UP4ZCh+C3+BgUUsOc/4Liwv6fER/rLna14Yiuni 6Jfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MgiGs2FK0RQ4JNcJFnsxjsC4BWa7qeyNid6Uf5IWwBE=; b=AnMqy3beGdOISJhX0xjPgcd2V7bBrowDwySu4k00KeK5Rjy52FRVkdSdCGzfaqH0gE 2dDYqGaSLRyaUalvVA7CefQNxAvi829z2XuX9ZkpMHZVl2W8iCnFkzGa3jzEuq8CNp7d g7p4xwCuJL5c62lL8S9ndGH0+JiHwHnLVHCSB5sO8uThDgUmwtuutR0+5xrmeERBm1kf ekb9Hgh5WvBr7YsXo6wmdVPzgD64Wa+E6eeAwcYOS1YRRAV/R0jgMiQroyCoistwF5zq avc8t3MifXGos5acehfDSvgBNJEVE96+xTm5DwKiGswtzjyI2/oBmvwO9mbmuPymXR+q UQiw== X-Gm-Message-State: AOAM5300Xu0Ml+7aIqtdxIsOJK51a/kau7ZJppDHXkvi+x9L1L6MBwZ3 orA11Npopo9Kk7yhQJzh0rkOIkF1pcvAk94j X-Google-Smtp-Source: ABdhPJz8cb5hUXqbVxJ0PZ94Oo4nqDbSIaKQRp5eSLyEzi5Tgzn73vxMVKemqTEYC3dSSCt7/YBWoWzIAj4/5hyP X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:600c:210a:: with SMTP id u10mr304573wml.98.1605046870141; Tue, 10 Nov 2020 14:21:10 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:21 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 17/20] kasan: clarify comment in __kasan_kfree_large From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172119_074982_7E96CDB5 X-CRM114-Status: GOOD ( 13.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently it says that the memory gets poisoned by page_alloc code. Clarify this by mentioning the specific callback that poisons the memory. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/I1334dffb69b87d7986fab88a1a039cc3ea764725 Reviewed-by: Marco Elver --- mm/kasan/common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 40ff3ce07a76..4360292ad7f3 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -436,5 +436,5 @@ void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ + /* The object will be poisoned by kasan_free_pages(). */ } From patchwork Tue Nov 10 22:20:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 307A2C388F7 for ; Tue, 10 Nov 2020 22:40:47 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 27A7320781 for ; Tue, 10 Nov 2020 22:40:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="LclLGZ6U"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="HiQa0Kqy"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="FOzDKlnS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 27A7320781 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ZwLUfAGN8m3HJIkfgipJUOMfxQXO2Sjleb1TGeE3uUs=; b=LclLGZ6UV2L/XI0uQYN2kCCTr AqO0zXwBsvSJo4shXsdwT4Ug8DAHUKIKtv7TSaAmo+J/0HA8CqUFrtv4+0gt7ONM58Uih8JZVo8rV fBW3TH+B5zUk4ZS6TU31Mu8f2LdKsPADen7b9z3h4WhN8M4N/yfMgdKVKdY5voqdwL881VAtL+3Wk cw8EfwzkXwzcegJRZxkcBKivDDUZRgEa13dvWdLZ+JFXrWdTab2RxCGFVjtNx+D8m22wcP/O4AR2j FtFfMtgirrBuHI8Qq+R0MEG9We0Lut4hCt0wqrTh8z21ufUnFmDXn1sEBhguWhb8xvfwowUq0eMSy c6cVPn7kw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccJ6-0000TK-0v; Tue, 10 Nov 2020 22:40:08 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccAt-0005D9-Lf for linux-arm-kernel@merlin.infradead.org; Tue, 10 Nov 2020 22:31:40 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=2zUWPQplFymp9Hmc6hlFXFyiGUCdCRWGOE8USzzNHUI=; b=HiQa0KqyjvJG2qAZ+hyu1v4HEN kpTSevUaJyhBkd5HctjVKd4l6hhBrMDT2VTIMUijUIidS19B9RpZ/ktDJ+VIqzYgEZMPTgzHmb0eC SuVHBaaswM3I0MDQqbITFWXYGVcWGmLZFmVisLLvF+CSEbqhT4TNNbWCBuF2YEgAAmzkSlKCMx0z9 dSgQa7IDEjiryvDNHSpCNq0GrfaaQ5Lh/h5aj0nyKuIVi6iC9oYJAO9ZA8+rL80Z9ZHPuiDu58JqC dSClKO9iUQ/95wlow25VbwHaINOKXSWPQ+3SQt9xxi0QWb8Q01XdLppg+r8HgJJBmVqeIZ21Dq3Dr Io1dzu+g==; Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0s-0008Vh-7U for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:27 +0000 Received: by mail-wm1-x34a.google.com with SMTP id 3so1859351wms.9 for ; Tue, 10 Nov 2020 14:21:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=2zUWPQplFymp9Hmc6hlFXFyiGUCdCRWGOE8USzzNHUI=; b=FOzDKlnSb1ljG+EB9YZK8MuDZK2QVgd+vEBRnNGIOsmByT0hZp/QxZV6N4gx4LwXJK 5bqmlJT0Ia2Z5GfrC2o5VNeOqhwMdDfBEK5I1e30ArlD3PqzxgilBWDRDZNSW3wVe13g hH4eXpXNXFuyyG+Dyes1ZPj7n+X9GJ8Ont3iQGNAfFku5e3CHFS8NgDUJgsKA44VgJUN NJX0T32+oNuqqt2pHbILme31X+iXy2f9LTYP5wT+2D0j3UXcH4AK0aTE9GAd9k/FiXti 25LGAFGL6fGj/J/KTejxXNuuZ9DEncV/6PiM0x3n570H6xpmiKr5+umAvXLXQ++Ok6LY Y5QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=2zUWPQplFymp9Hmc6hlFXFyiGUCdCRWGOE8USzzNHUI=; b=QuqcWVxbcYdo+opAaE3yPJp0emOdbsguEFtUg3upsiHz5W8ksXBM4fFU+QE8JWg2NZ uuOJcaYThUvqVDgMMxmiG1euYdq7Rr0q0Dcfxczi8HU1SFkH5SySHoGoeCi5PScMHttg H1LGf6tP2jH6pc26FP+mskTuYg5bEVl/sMO0vgG3anvpimYVlJOtLx0ZhbXoX4ccs86B LFG3CE7gzn2/ZLyiSqzoJY2mSWJvUmXUyjSe/5lgipnBtPXZUYc7jg3qwO9TZZNF+/nv V6n5mgeKJvluupMWUy56XXBAdRRqvfMgZuGSS63oK+0USeKzxXoXfzncLyrBIsxi7sgr ckFA== X-Gm-Message-State: AOAM531MJToH1ATQeb7caAjdra+SnMOCbrUhovlLBIB/OvQyrSYY2BT0 u2kt8/iA/+mor5VH/UmJvSJizlkrZrYArcjq X-Google-Smtp-Source: ABdhPJx6NSCyprfzCnoG3Bu9L/e53v6n3157momnnnlvzOzEvt/D47UjDjWZMq8vbeLYznH/vm0QqoXPT8slgNqO X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a5d:6310:: with SMTP id i16mr26060595wru.284.1605046872700; Tue, 10 Nov 2020 14:21:12 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:22 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 18/20] kasan: clean up metadata allocation and usage From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_222118_549069_55BFC0CD X-CRM114-Status: GOOD ( 33.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KASAN marks caches that are sanitized with the SLAB_KASAN cache flag. Currently if the metadata that is appended after the object (stores e.g. stack trace ids) doesn't fit into KMALLOC_MAX_SIZE (can only happen with SLAB, see the comment in the patch), KASAN turns off sanitization completely. With this change sanitization of the object data is always enabled. However the metadata is only stored when it fits. Instead of checking for SLAB_KASAN flag accross the code to find out whether the metadata is there, use cache->kasan_info.alloc/free_meta_offset. As 0 can be a valid value for free_meta_offset, introduce KASAN_NO_FREE_META as an indicator that the free metadata is missing. Along the way rework __kasan_cache_create() and add claryfying comments. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Icd947e2bea054cb5cfbdc6cf6652227d97032dcb --- mm/kasan/common.c | 112 +++++++++++++++++++++++++------------- mm/kasan/generic.c | 15 ++--- mm/kasan/hw_tags.c | 6 +- mm/kasan/kasan.h | 13 ++++- mm/kasan/quarantine.c | 8 +++ mm/kasan/report.c | 43 ++++++++------- mm/kasan/report_sw_tags.c | 7 ++- mm/kasan/sw_tags.c | 4 ++ 8 files changed, 138 insertions(+), 70 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 4360292ad7f3..940b42231069 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -109,9 +109,6 @@ void __kasan_free_pages(struct page *page, unsigned int order) */ static inline unsigned int optimal_redzone(unsigned int object_size) { - if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) - return 0; - return object_size <= 64 - 16 ? 16 : object_size <= 128 - 32 ? 32 : @@ -125,47 +122,79 @@ static inline unsigned int optimal_redzone(unsigned int object_size) void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { - unsigned int orig_size = *size; + unsigned int ok_size; unsigned int redzone_size; - int redzone_adjust; + unsigned int optimal_size; + + /* + * SLAB_KASAN is used to mark caches as ones that are sanitized by + * KASAN. Currently this is used in two places: + * 1. In slab_ksize() when calculating the size of the accessible + * memory within the object. + * 2. In slab_common.c to prevent merging of sanitized caches. + */ + *flags |= SLAB_KASAN; - if (!kasan_stack_collection_enabled()) { - *flags |= SLAB_KASAN; + if (!kasan_stack_collection_enabled()) return; - } - /* Add alloc meta. */ + ok_size = *size; + + /* Add alloc meta into redzone. */ cache->kasan_info.alloc_meta_offset = *size; *size += sizeof(struct kasan_alloc_meta); - /* Add free meta. */ - if (IS_ENABLED(CONFIG_KASAN_GENERIC) && - (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta))) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); + /* + * If alloc meta doesn't fit, don't add it. + * This can only happen with SLAB, as it has KMALLOC_MAX_SIZE equal + * to KMALLOC_MAX_CACHE_SIZE and doesn't fall back to page_alloc for + * larger sizes. + */ + if (*size > KMALLOC_MAX_SIZE) { + cache->kasan_info.alloc_meta_offset = 0; + *size = ok_size; + /* Continue, since free meta might still fit. */ } - redzone_size = optimal_redzone(cache->object_size); - redzone_adjust = redzone_size - (*size - cache->object_size); - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + redzone_size)); + /* Only the generic mode uses free meta or flexible redzones. */ + if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) { + cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META; + return; + } /* - * If the metadata doesn't fit, don't enable KASAN at all. + * Add free meta into redzone when it's not possible to store + * it in the object. This is the case when: + * 1. Object is SLAB_TYPESAFE_BY_RCU, which means that is can + * be touched after it was freed, or + * 2. Object has a constructor, which means it's expected to + * retain its content until the next allocation, or + * 3. Object is too small. + * Otherwise cache->kasan_info.free_meta_offset = 0 is implied. */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; + if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + ok_size = *size; + + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + + /* If free meta doesn't fit, don't add it. */ + if (*size > KMALLOC_MAX_SIZE) { + cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META; + *size = ok_size; + } } - *flags |= SLAB_KASAN; + redzone_size = optimal_redzone(cache->object_size); + /* Calculate size with optimal redzone. */ + optimal_size = cache->object_size + redzone_size; + /* Limit it with KMALLOC_MAX_SIZE (relevant for SLAB only). */ + if (optimal_size > KMALLOC_MAX_SIZE) + optimal_size = KMALLOC_MAX_SIZE; + /* Use optimal size if the size with added metas is not large enough. */ + if (*size < optimal_size) + *size = optimal_size; } size_t __kasan_metadata_size(struct kmem_cache *cache) @@ -181,15 +210,21 @@ size_t __kasan_metadata_size(struct kmem_cache *cache) struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object) { + if (!cache->kasan_info.alloc_meta_offset) + return NULL; return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset; } +#ifdef CONFIG_KASAN_GENERIC struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + if (cache->kasan_info.free_meta_offset == KASAN_NO_FREE_META) + return NULL; return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } +#endif void __kasan_unpoison_data(const void *addr, size_t size) { @@ -276,11 +311,9 @@ void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; if (kasan_stack_collection_enabled()) { - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + if (alloc_meta) + __memset(alloc_meta, 0, sizeof(*alloc_meta)); } /* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */ @@ -319,8 +352,7 @@ static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, if (!kasan_stack_collection_enabled()) return false; - if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || - unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine)) return false; kasan_set_free_info(cache, object, tag); @@ -345,7 +377,11 @@ void __kasan_slab_free_mempool(void *ptr, unsigned long ip) static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); + struct kasan_alloc_meta *alloc_meta; + + alloc_meta = kasan_get_alloc_meta(cache, object); + if (alloc_meta) + kasan_set_track(&alloc_meta->alloc_track, flags); } static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, @@ -372,7 +408,7 @@ static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN)) + if (kasan_stack_collection_enabled()) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index d259e4c3aefd..97e39516f8fe 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -338,10 +338,10 @@ void kasan_record_aux_stack(void *addr) cache = page->slab_cache; object = nearest_obj(cache, page, addr); alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return; - /* - * record the last two call_rcu() call stacks. - */ + /* Record the last two call_rcu() call stacks. */ alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; alloc_meta->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); } @@ -352,11 +352,11 @@ void kasan_set_free_info(struct kmem_cache *cache, struct kasan_free_meta *free_meta; free_meta = kasan_get_free_meta(cache, object); - kasan_set_track(&free_meta->free_track, GFP_NOWAIT); + if (!free_meta) + return; - /* - * the object was freed and has free track set - */ + kasan_set_track(&free_meta->free_track, GFP_NOWAIT); + /* The object was freed and has free track set. */ *(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREETRACK; } @@ -365,5 +365,6 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK) return NULL; + /* Free meta must be present with KASAN_KMALLOC_FREETRACK. */ return &kasan_get_free_meta(cache, object)->free_track; } diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 2f6f0261af8c..c3d2a21d925d 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -188,7 +188,8 @@ void kasan_set_free_info(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; alloc_meta = kasan_get_alloc_meta(cache, object); - kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); + if (alloc_meta) + kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); } struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, @@ -197,5 +198,8 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return NULL; + return &alloc_meta->free_track[0]; } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 5eff3d9f624e..88892c05eb7d 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -154,20 +154,31 @@ struct kasan_alloc_meta { struct qlist_node { struct qlist_node *next; }; + +/* + * Generic mode either stores free meta in the object itself or in the redzone + * after the object. In the former case free meta offset is 0, in the latter + * case it has some sane value smaller than INT_MAX. Use INT_MAX as free meta + * offset when free meta isn't present. + */ +#define KASAN_NO_FREE_META (INT_MAX) + struct kasan_free_meta { +#ifdef CONFIG_KASAN_GENERIC /* This field is used while the object is in the quarantine. * Otherwise it might be used for the allocator freelist. */ struct qlist_node quarantine_link; -#ifdef CONFIG_KASAN_GENERIC struct kasan_track free_track; #endif }; struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object); +#ifdef CONFIG_KASAN_GENERIC struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object); +#endif #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 0da3d37e1589..23f6bfb1e73f 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -135,7 +135,12 @@ static void qlink_free(struct qlist_node *qlink, struct kmem_cache *cache) if (IS_ENABLED(CONFIG_SLAB)) local_irq_save(flags); + /* + * As the object now gets freed from the quaratine, assume that its + * free track is now longer valid. + */ *(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREE; + ___cache_free(cache, object, _THIS_IP_); if (IS_ENABLED(CONFIG_SLAB)) @@ -168,6 +173,9 @@ void quarantine_put(struct kmem_cache *cache, void *object) struct qlist_head temp = QLIST_INIT; struct kasan_free_meta *meta = kasan_get_free_meta(cache, object); + if (!meta) + return; + /* * Note: irq must be disabled until after we move the batch to the * global quarantine. Otherwise quarantine_remove_cache() can miss diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 7d86af340148..6a95ad2dee91 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -168,32 +168,35 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, static void describe_object_stacks(struct kmem_cache *cache, void *object, const void *addr, u8 tag) { - struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); - - if (cache->flags & SLAB_KASAN) { - struct kasan_track *free_track; + struct kasan_alloc_meta *alloc_meta; + struct kasan_track *free_track; + alloc_meta = kasan_get_alloc_meta(cache, object); + if (alloc_meta) { print_track(&alloc_meta->alloc_track, "Allocated"); pr_err("\n"); - free_track = kasan_get_free_track(cache, object, tag); - if (free_track) { - print_track(free_track, "Freed"); - pr_err("\n"); - } + } + + free_track = kasan_get_free_track(cache, object, tag); + if (free_track) { + print_track(free_track, "Freed"); + pr_err("\n"); + } #ifdef CONFIG_KASAN_GENERIC - if (alloc_meta->aux_stack[0]) { - pr_err("Last call_rcu():\n"); - print_stack(alloc_meta->aux_stack[0]); - pr_err("\n"); - } - if (alloc_meta->aux_stack[1]) { - pr_err("Second to last call_rcu():\n"); - print_stack(alloc_meta->aux_stack[1]); - pr_err("\n"); - } -#endif + if (!alloc_meta) + return; + if (alloc_meta->aux_stack[0]) { + pr_err("Last call_rcu():\n"); + print_stack(alloc_meta->aux_stack[0]); + pr_err("\n"); } + if (alloc_meta->aux_stack[1]) { + pr_err("Second to last call_rcu():\n"); + print_stack(alloc_meta->aux_stack[1]); + pr_err("\n"); + } +#endif } static void describe_object(struct kmem_cache *cache, void *object, diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index 7604b46239d4..11dc8739e500 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -48,9 +48,10 @@ const char *get_bug_type(struct kasan_access_info *info) object = nearest_obj(cache, page, (void *)addr); alloc_meta = kasan_get_alloc_meta(cache, object); - for (i = 0; i < KASAN_NR_FREE_STACKS; i++) - if (alloc_meta->free_pointer_tag[i] == tag) - return "use-after-free"; + if (alloc_meta) + for (i = 0; i < KASAN_NR_FREE_STACKS; i++) + if (alloc_meta->free_pointer_tag[i] == tag) + return "use-after-free"; return "out-of-bounds"; } diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index d1af6f6c6d12..be10d16bd129 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -170,6 +170,8 @@ void kasan_set_free_info(struct kmem_cache *cache, u8 idx = 0; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return; #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY idx = alloc_meta->free_track_idx; @@ -187,6 +189,8 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, int i = 0; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return NULL; #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { From patchwork Tue Nov 10 22:20:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 727C3C388F7 for ; Tue, 10 Nov 2020 22:41:25 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F3B232068D for ; Tue, 10 Nov 2020 22:41:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Yg7QfOJl"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="LpMJwR/j"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="MvPjJraT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F3B232068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5JB3NiBo+ZrP5Nh3l4xB+o5oyr35w4L2w2x57uFIHCI=; b=Yg7QfOJlqpEqeBgnRmXqzPb1g 3SRn5NXDur8YvyKbOQdbFFDJ9dVcQi9IO07/o9mjJN5+yXRpzXFkCT7D3w/xum9a0QY89o98fQQrY EXQAKdFnaktebKp9bMbr74rW1B9exvAu52eRcxwubSdKxvN4e6WSsN1AE1/3aPcIZFnwq1YeCCQ7K ceFc61cL/iPD/Hwcrex8HLpjeuSSMhmaYe1T4Mkm8wRjLSIOcar1d/HkAYx4fdEdKlGu3U10pK9kw FOri/F+Lxw/bmbUDeMTWH3gm/AdSkkmlmHtuLX/LWy3c2xNA/XvoJ6iN4/InXIi+oYvDIV8NmXD2x i2Kg1ywIg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccJj-0000gL-9b; Tue, 10 Nov 2020 22:40:47 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccAy-0005D9-0C for linux-arm-kernel@merlin.infradead.org; Tue, 10 Nov 2020 22:31:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=NSEnP5/Ymw2+CzfuUjUL3yzFjWGlVw3jfopzq6nBhYw=; b=LpMJwR/jy6xxNHE/J1MlNldMT9 u1ipI2dKIFy5x573Rt0yH75ITl9Sp/Am4zubcEKfbQzfOs+1hmFQFNTKoT9wH8W3Y6Hr09FDQvC/R gg9Of0U3WUawpbHV4R8Uo8x3f/phRAoDUnzvJAomeRYrskq70KHyMaKUt/kY+yzhVRuxvrK8lkmJf 4evwazBXdb3vjTV1S/oM71+HT+XkKNbbNVuIYnzwfeBKZp8+MfEnWROHd3yzFl3ciCBlZ4rbgBZUS UE8ekuuESsG6ky3k6yLDrm2sQjgYUdhEe5bkneXKw7qBC4ceIyeBxO2SFnwx8ACeyv0c104bZLPg+ oL167ljQ==; Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc0r-00005D-IQ for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:23 +0000 Received: by mail-wm1-x34a.google.com with SMTP id a130so1677797wmf.0 for ; Tue, 10 Nov 2020 14:21:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=NSEnP5/Ymw2+CzfuUjUL3yzFjWGlVw3jfopzq6nBhYw=; b=MvPjJraTvvciMCrmEVBHJf7JZffHC8y5WsKXsz7dKaBKOd+ahVD8jotAMioOOY+Zy8 +oIjs5zBxjMN13SJYSVufJUR8ZSJRp8wcFRg6ysEbVOpnOQ5vU6igGKyRCAtUMHR205f 7QyvhDr6wwOcsOkI4zxzz6jcwQtJl0AqqctzgvATGiNPeP/sVqQ+6ox1fbYOYABAYPvk 7DaxX13UIIfsr9P6vPscgt+nvmJbOlQRSXO/sZhZ6Ht3GxlE3kgYYtamuzZF9LJ/Pvpz BQVuFuMfR8gf1A4I0Rij0OHDKjK8tIBD4Mhw3LvAXhWqwcwJRbF96SPZigccwisHP0Br nfyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=NSEnP5/Ymw2+CzfuUjUL3yzFjWGlVw3jfopzq6nBhYw=; b=Yr5HlI9sTPPLAVDK7SV6sGweZmiI5BxrNsWQRgstyMrZZztJNOYNiw/yQzZB3raXv5 S6s+dMU9tujXfZJtea7n3UI1+euHwnDmUS96kuficd65u6ryhDxxC0MNe8dnLrzZ2U87 5gw+a/r0brAYKZJi8lTt+S3xY8kZusWgBD+uOIGnYT2aBJv7wXHHK2xH09Veww7tyqS0 W4Mfe0zLfRr03YezSjInLvDe68hFgBU6EnQtAg8RGz2LJaMYhHXH+6ZCNohEOgkOvWYl ZqrxpGdE6vCOYCyyFpu4Ml8zmPvc9sHCdcTaLKEfpuSVR9iAER3cGnurC4lcIYjVQO88 cufg== X-Gm-Message-State: AOAM533+pHdTwMFPbE6qkc9ZIzTWbcbYlixMxls9zJZu5OkYKYLzRDZd 8KBk9QR56IWcCA6DpWW1DClkr11q+xt8a6Jk X-Google-Smtp-Source: ABdhPJx5R9Lwi7vyMEdhZoBWGFA+5/tY8OWnRDBKQTuXeI+HTirxH7sptEemR57Y7sCa/FhiaUQSY6Jezq4w1pp+ X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a7b:c772:: with SMTP id x18mr262720wmk.185.1605046875625; Tue, 10 Nov 2020 14:21:15 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:23 +0100 In-Reply-To: Message-Id: <936c0c198145b663e031527c49a6895bd21ac3a0.1605046662.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 19/20] kasan, mm: allow cache merging with no metadata From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_222117_833342_96CB61A8 X-CRM114-Status: GOOD ( 16.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba --- include/linux/kasan.h | 26 ++++++++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 11 ++++++++--- 3 files changed, 43 insertions(+), 5 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 534ab3e2935a..c754eca356f7 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -81,17 +81,35 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static inline kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +slab_flags_t __kasan_never_merge(slab_flags_t flags); +static inline slab_flags_t kasan_never_merge(slab_flags_t flags) +{ + if (kasan_enabled()) + return __kasan_never_merge(flags); + return flags; +} + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline kasan_enabled(void) { return true; } -#endif + +static inline slab_flags_t kasan_never_merge(slab_flags_t flags) +{ + return flags; +} + +#endif /* CONFIG_KASAN_HW_TAGS */ void __kasan_alloc_pages(struct page *page, unsigned int order); static inline void kasan_alloc_pages(struct page *page, unsigned int order) @@ -240,6 +258,10 @@ static inline kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(slab_flags_t flags) +{ + return flags; +} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} static inline void kasan_cache_create(struct kmem_cache *cache, diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 940b42231069..25b18c145b06 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -81,6 +81,17 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(slab_flags_t flags) +{ + if (kasan_stack_collection_enabled()) + return flags; + return flags & ~SLAB_KASAN; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; diff --git a/mm/slab_common.c b/mm/slab_common.c index f1b0c4a22f08..3042ee8ea9ce 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -49,12 +50,16 @@ static DECLARE_WORK(slab_caches_to_rcu_destroy_work, slab_caches_to_rcu_destroy_workfn); /* - * Set of flags that will prevent slab merging + * Set of flags that will prevent slab merging. + * Use slab_never_merge() instead. */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ SLAB_FAILSLAB | SLAB_KASAN) +/* KASAN allows merging in some configurations and will remove SLAB_KASAN. */ +#define slab_never_merge() (kasan_never_merge(SLAB_NEVER_MERGE)) + #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT) @@ -164,7 +169,7 @@ static unsigned int calculate_alignment(slab_flags_t flags, */ int slab_unmergeable(struct kmem_cache *s) { - if (slab_nomerge || (s->flags & SLAB_NEVER_MERGE)) + if (slab_nomerge || (s->flags & slab_never_merge())) return 1; if (s->ctor) @@ -198,7 +203,7 @@ struct kmem_cache *find_mergeable(unsigned int size, unsigned int align, size = ALIGN(size, align); flags = kmem_cache_flags(size, flags, name, NULL); - if (flags & SLAB_NEVER_MERGE) + if (flags & slab_never_merge()) return NULL; list_for_each_entry_reverse(s, &slab_caches, list) { From patchwork Tue Nov 10 22:20:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11895735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5374C4742C for ; Tue, 10 Nov 2020 22:39:17 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 11BDC2068D for ; Tue, 10 Nov 2020 22:39:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="WOBCfbCm"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="NtqZsgjs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 11BDC2068D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=UNcAkABDVC5qgyNa1dz9exf1ynbH5cNCNotwNsE94kM=; b=WOBCfbCmob+xEC65jF0skL0L6 SqQUX3RtexvxY7QUSy8lCclgY5Ao9gNhIxHfwO6Kh9JExFlojdGJn+w8do+lmDKc2cTo9egFVnDJp Mc3mOs0TsIyGw7Ku5lyPQPzAUHtPZ51aWvryw7P7SemK4vPbyb893h5WPrT3FdVkTYKzSoqMeJqZu JRs52jJmbwLkTTGIKNP1vQRCXW/IZr1eLyFQ/q5/S25N5ZsFzBH3Fy5gLVYFtoEskFcVCzrw1v4M7 +AeUta8mZfiCVtbt2BLW8072hxgecaiFA1rirQxQAq6a4t8UJ+lhHynLzQtVr2LMHLEehJbHAxwfS 1iJYLLR7Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kccGY-0007it-6j; Tue, 10 Nov 2020 22:37:30 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kcc10-0000YO-0F for linux-arm-kernel@lists.infradead.org; Tue, 10 Nov 2020 22:21:50 +0000 Received: by mail-qk1-x74a.google.com with SMTP id c18so199292qkl.15 for ; Tue, 10 Nov 2020 14:21:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=hhAVeCS4JTcfsOdyr55ktLnnnel/q5W+4IYGph2ifVQ=; b=NtqZsgjsLakRDBLENq4QMY0X9HIgXySW03WWmgVqGyEYFqKXkjIr1dEai5KtjCKEW3 7K2btai6v2nhe+5GvDMjBlpzdb+L+gFrW0dsbOE3XONOKvcmOP84oxhz669e/Xv9pema AuFjujh/ijtNIT6DJ/yWdkkj6fZlw+UOPnNyLXJ48oiP2FBrAyyAJepdIaIb7X23kmcQ gGiLq8nwuid0DGkV9GC3ezMJdnVkK/gKzPZYVCyvbc7Uxuj45Uwy46M3IAVTYdpb/GQo RpYViBibwA69YuCVWHp62UdWt3CgA5Eym+yKlD5Ar4VDr7xMgH1UOG4msMzpsOHEPAHV n9nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=hhAVeCS4JTcfsOdyr55ktLnnnel/q5W+4IYGph2ifVQ=; b=YbcQ2dGtn31SgHTu4+kg9Iy/9qpkiv3JXWrHTny8rDZMqcFNNaSAiqukt40Y/+mC3P 4rucWBrDQ+RRfJFGbFUhTl+lXoJUY4d9jDSqerAimmv6nept7pCQp9Mr1qlkYzMzeaip DQKUm9+KCvBrTTT5tLKgTzPAe/n81vqFCK9qD1FKWEAwltKdigJmVHHjFuJnLTtcpTMB +e8mpjiHnbRfB8gomM2WVoOFs6XjZMiROBAFT5Bg4zb0H5yI/JV3hDcEDK0bDyzolC6H 1UsRAifnMit78gHulwoBaCYJHH/uVOBRnRXaBVhTePVCVrW4aA+dzrh/GBrmpnxxI5CA x3rg== X-Gm-Message-State: AOAM530N3pOBZABbRX1bvWGdyODohu0hiqVhndpBMZp1rrFz9ma5Jzqv 1Gh4rmG+V66cDTZiuMrVK8veQQnPLBxt7jKT X-Google-Smtp-Source: ABdhPJxdW7ElRQssXr8ri/yytQPlY79pRGXXiR3orScqYpcHOLCE1EbY/q0D4TfAlEFaD8Q8q9+owBaUWU1bDOJV X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:ad4:4673:: with SMTP id z19mr9694086qvv.60.1605046877787; Tue, 10 Nov 2020 14:21:17 -0800 (PST) Date: Tue, 10 Nov 2020 23:20:24 +0100 In-Reply-To: Message-Id: <6103d1aaacea96a94ca4632f78bcd801e4fbc9c4.1605046662.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.222.g5d2a92d10f8-goog Subject: [PATCH v2 20/20] kasan: update documentation From: Andrey Konovalov To: Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201110_172126_885060_22E59D99 X-CRM114-Status: GOOD ( 32.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Catalin Marinas , Kevin Brodsky , Will Deacon , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Konovalov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This change updates KASAN documentation to reflect the addition of boot parameters and also reworks and clarifies some of the existing sections, in particular: defines what a memory granule is, mentions quarantine, makes Kunit section more readable. Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 180 +++++++++++++++++++----------- 1 file changed, 113 insertions(+), 67 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index 422f8ee1bb17..f2da2b09e5c7 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -6,6 +6,7 @@ Overview KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to find out-of-bound and use-after-free bugs. KASAN has three modes: + 1. generic KASAN (similar to userspace ASan), 2. software tag-based KASAN (similar to userspace HWASan), 3. hardware tag-based KASAN (based on hardware memory tagging). @@ -39,23 +40,13 @@ CONFIG_KASAN_INLINE. Outline and inline are compiler instrumentation types. The former produces smaller binary while the latter is 1.1 - 2 times faster. Both software KASAN modes work with both SLUB and SLAB memory allocators, -hardware tag-based KASAN currently only support SLUB. -For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. +while the hardware tag-based KASAN currently only support SLUB. + +For better error reports that include stack traces, enable CONFIG_STACKTRACE. To augment reports with last allocation and freeing stack of the physical page, it is recommended to enable also CONFIG_PAGE_OWNER and boot with page_owner=on. -To disable instrumentation for specific files or directories, add a line -similar to the following to the respective kernel Makefile: - -- For a single file (e.g. main.o):: - - KASAN_SANITIZE_main.o := n - -- For all files in one directory:: - - KASAN_SANITIZE := n - Error reports ~~~~~~~~~~~~~ @@ -140,16 +131,20 @@ freed (in case of a use-after-free bug report). Next comes a description of the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. -Reading this part requires some understanding of how KASAN works. - -The state of each 8 aligned bytes of memory is encoded in one shadow byte. -Those 8 bytes can be accessible, partially accessible, freed or be a redzone. -We use the following encoding for each shadow byte: 0 means that all 8 bytes -of the corresponding memory region are accessible; number N (1 <= N <= 7) means -that the first N bytes are accessible, and other (8 - N) bytes are not; -any negative value indicates that the entire 8-byte word is inaccessible. -We use different negative values to distinguish between different kinds of -inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). +Internally KASAN tracks memory state separately for each memory granule, which +is either 8 or 16 aligned bytes depending on KASAN mode. Each number in the +memory state section of the report shows the state of one of the memory +granules that surround the accessed address. + +For generic KASAN the size of each memory granule is 8. The state of each +granule is encoded in one shadow byte. Those 8 bytes can be accessible, +partially accessible, freed or be a part of a redzone. KASAN uses the following +encoding for each shadow byte: 0 means that all 8 bytes of the corresponding +memory region are accessible; number N (1 <= N <= 7) means that the first N +bytes are accessible, and other (8 - N) bytes are not; any negative value +indicates that the entire 8-byte word is inaccessible. KASAN uses different +negative values to distinguish between different kinds of inaccessible memory +like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. @@ -157,6 +152,55 @@ the accessed address is partially accessible. For tag-based KASAN this last report section shows the memory tags around the accessed address (see Implementation details section). +Boot parameters +~~~~~~~~~~~~~~~ + +Hardware tag-based KASAN mode (see the section about different mode below) is +intended for use in production as a security mitigation. Therefore it supports +boot parameters that allow to disable KASAN competely or otherwise control +particular KASAN features. + +The things that can be controlled are: + +1. Whether KASAN is enabled at all. +2. Whether KASAN collects and saves alloc/free stacks. +3. Whether KASAN panics on a detected bug or not. + +The ``kasam.mode`` boot parameter allows to choose one of three main modes: + +- ``kasan.mode=off`` - KASAN is disabled, no tag checks are performed +- ``kasan.mode=prod`` - only essential production features are enabled +- ``kasan.mode=full`` - all KASAN features are enabled + +The chosen mode provides default control values for the features mentioned +above. However it's also possible to override the default values by providing: + +- ``kasan.stacktrace=off`` or ``=on`` - enable alloc/free stack collection + (default: ``on`` for ``mode=full``, + otherwise ``off``) +- ``kasan.fault=report`` or ``=panic`` - only print KASAN report or also panic + (default: ``report``) + +If ``kasan.mode parameter`` is not provided, it defaults to ``full`` when +``CONFIG_DEBUG_KERNEL`` is enabled, and to ``prod`` otherwise. + +For developers +~~~~~~~~~~~~~~ + +Software KASAN modes use compiler instrumentation to insert validity checks. +Such instrumentation might be incompatible with some part of the kernel, and +therefore needs to be disabled. To disable instrumentation for specific files +or directories, add a line similar to the following to the respective kernel +Makefile: + +- For a single file (e.g. main.o):: + + KASAN_SANITIZE_main.o := n + +- For all files in one directory:: + + KASAN_SANITIZE := n + Implementation details ---------------------- @@ -164,10 +208,10 @@ Implementation details Generic KASAN ~~~~~~~~~~~~~ -From a high level, our approach to memory error detection is similar to that -of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to insert checks of shadow -memory on each memory access. +From a high level perspective, KASAN's approach to memory error detection is +similar to that of kmemcheck: use shadow memory to record whether each byte of +memory is safe to access, and use compile-time instrumentation to insert checks +of shadow memory on each memory access. Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and offset to @@ -194,7 +238,10 @@ function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. -Generic KASAN prints up to 2 call_rcu() call stacks in reports, the last one +Generic KASAN is the only mode that delays the reuse of freed object via +quarantine (see mm/kasan/quarantine.c for implementation). + +Generic KASAN prints up to two call_rcu() call stacks in reports, the last one and the second to last. Software tag-based KASAN @@ -302,15 +349,15 @@ therefore be wasteful. Furthermore, to ensure that different mappings use different shadow pages, mappings would have to be aligned to ``KASAN_GRANULE_SIZE * PAGE_SIZE``. -Instead, we share backing space across multiple mappings. We allocate +Instead, KASAN shares backing space across multiple mappings. It allocates a backing page when a mapping in vmalloc space uses a particular page of the shadow region. This page can be shared by other vmalloc mappings later on. -We hook in to the vmap infrastructure to lazily clean up unused shadow +KASAN hooks in to the vmap infrastructure to lazily clean up unused shadow memory. -To avoid the difficulties around swapping mappings around, we expect +To avoid the difficulties around swapping mappings around, KASAN expects that the part of the shadow region that covers the vmalloc space will not be covered by the early shadow page, but will be left unmapped. This will require changes in arch-specific code. @@ -321,24 +368,31 @@ architectures that do not have a fixed module region. CONFIG_KASAN_KUNIT_TEST & CONFIG_TEST_KASAN_MODULE -------------------------------------------------- -``CONFIG_KASAN_KUNIT_TEST`` utilizes the KUnit Test Framework for testing. -This means each test focuses on a small unit of functionality and -there are a few ways these tests can be run. +KASAN tests consist on two parts: + +1. Tests that are integrated with the KUnit Test Framework. Enabled with +``CONFIG_KASAN_KUNIT_TEST``. These tests can be run and partially verified +automatically in a few different ways, see the instructions below. -Each test will print the KASAN report if an error is detected and then -print the number of the test and the status of the test: +2. Tests that are currently incompatible with Kunit. Enabled with +``CONFIG_TEST_KASAN_MODULE`` and can only be run as a module. These tests can +only be verified manually, by loading the kernel module and inspecting the +kernel log for KASAN reports. -pass:: +Each KUNIT-compatible KASAN test prints a KASAN report if an error is detected. +Then the test prints its number and status. + +When a test passes:: ok 28 - kmalloc_double_kzfree -or, if kmalloc failed:: +When a test fails due to a failed ``kmalloc``:: # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163 Expected ptr is not null, but is not ok 4 - kmalloc_large_oob_right -or, if a KASAN report was expected, but not found:: +When a test fails due to a missing KASAN report:: # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 Expected kasan_data->report_expected == kasan_data->report_found, but @@ -346,46 +400,38 @@ or, if a KASAN report was expected, but not found:: kasan_data->report_found == 0 not ok 28 - kmalloc_double_kzfree -All test statuses are tracked as they run and an overall status will -be printed at the end:: +At the end the cumulative status of all KASAN tests is printed. On success:: ok 1 - kasan -or:: +Or, if one of the tests failed:: not ok 1 - kasan -(1) Loadable Module -~~~~~~~~~~~~~~~~~~~~ + +There are a few ways to run Kunit-compatible KASAN tests. + +1. Loadable module +~~~~~~~~~~~~~~~~~~ With ``CONFIG_KUNIT`` enabled, ``CONFIG_KASAN_KUNIT_TEST`` can be built as -a loadable module and run on any architecture that supports KASAN -using something like insmod or modprobe. The module is called ``test_kasan``. +a loadable module and run on any architecture that supports KASAN by loading +the module with insmod or modprobe. The module is called ``test_kasan``. -(2) Built-In -~~~~~~~~~~~~~ +2. Built-In +~~~~~~~~~~~ With ``CONFIG_KUNIT`` built-in, ``CONFIG_KASAN_KUNIT_TEST`` can be built-in -on any architecure that supports KASAN. These and any other KUnit -tests enabled will run and print the results at boot as a late-init -call. +on any architecure that supports KASAN. These and any other KUnit tests enabled +will run and print the results at boot as a late-init call. -(3) Using kunit_tool -~~~~~~~~~~~~~~~~~~~~~ +3. Using kunit_tool +~~~~~~~~~~~~~~~~~~~ -With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, we can also -use kunit_tool to see the results of these along with other KUnit -tests in a more readable way. This will not print the KASAN reports -of tests that passed. Use `KUnit documentation `_ for more up-to-date -information on kunit_tool. +With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, it's also +possible use ``kunit_tool`` to see the results of these and other KUnit tests +in a more readable way. This will not print the KASAN reports of the tests that +passed. Use `KUnit documentation `_ +for more up-to-date information on ``kunit_tool``. .. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html - -``CONFIG_TEST_KASAN_MODULE`` is a set of KASAN tests that could not be -converted to KUnit. These tests can be run only as a module with -``CONFIG_TEST_KASAN_MODULE`` built as a loadable module and -``CONFIG_KASAN`` built-in. The type of error expected and the -function being run is printed before the expression expected to give -an error. Then the error is printed, if found, and that test -should be interpretted to pass only if the error was the one expected -by the test.