From patchwork Thu Nov 5 00:02:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9D17C00A89 for ; Thu, 5 Nov 2020 04:24:21 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2FEA420825 for ; Thu, 5 Nov 2020 04:24:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="xGYEzAwI"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="FdrQhzry"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="TtFB7Uef" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2FEA420825 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=e9odu6nAYelfxHH0aMsDQxVWFm0/4FJrUjyjwWos5Oc=; b=xGYEzAwIx4+PdlPjaaUZmSryR /FoqgHDtUY2NIdMQzwmaWxI+QHW7oA2y39FMtmo0ISVFFpxAYSYM/FSF0fNpL5wwvlrxZYUy+iHwJ nf+bfdPanlniL8pLZye698IOKyIYrzqPFqh51upv0pPc7XQeOfIx2ugGQDaEvleajVKFC7edgnq7X 0yzZ1tjQAJj7XIiM8xqKYFckAR1mK7/EhDKLhU2IT/YK9K8NNz9NYjdeuJ69uXSePT90JF6E8Etso qCiQ4QftKwXNDAggzdVop2ZPNGKq+Mb2k5Nz6ykA9zQh8qW11qELqZjot9hYYM0LZl1NSSvUGiVjG q+toeNXXw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaWnQ-0006J5-3h; Thu, 05 Nov 2020 04:22:48 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3v-000145-NR for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Ocomz11lJ9KE0anC5CzM99SKlSLQ0mxGbD3mCLha9hU=; b=FdrQhzryDZdfdg1Xn4HYTqt2GF sIUqgTDEq16cGwGJTFXba5TQb3M87p52K80v3Rn12WcM8wi+dfP8N/XXxc9Gr8/hVZswEZpejAgqc 6pBgWu3OulHa6BEddJTnYq0AFy10OGXff0XRWZIMR0DHTRxeMi56QrewxA03HacQ1gQvN5+Goq6oa MzLam6fZbXYZ78Qe0fskUQv9z6YbMAaMxdt6DVb6STy+Y5aLDbhPSdMHDd9iHX0cr0rToEg+PZ0Ux h7DpBdud/5XCDTrcc+sUVR2By7z7sNxzuXYsF3y283RnH97AFi1PR23T3FpTnDeolBvPCir8LnbSA 2B4pEQ1Q==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSkd-0000mO-N8 for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:03:43 +0000 Received: by mail-wr1-x449.google.com with SMTP id m20so84785wrb.21 for ; Wed, 04 Nov 2020 16:03:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=Ocomz11lJ9KE0anC5CzM99SKlSLQ0mxGbD3mCLha9hU=; b=TtFB7UefTiyPVVP+/kDhkkT/QxrHuTo+WYQ5Nvww663l3VqibyXgl53WW2WtoolN3r wYf3SBvI/y4/ED+RCCuwa5/pUZX+bY9wrT2Bkruuc8LKJG6Bx3xM1sQnPbF8GcOx8QmF mz7OfwCckO0gu8SIJZZwnULGnyIrzmYFoX7VIZiDvB2t8IjiO5TJICScNGjWfBA3MowY ev/8Po/TamKjG9Q11/jRnwJFgLkXoxfftwqGS4W/r0zgAIkuTcAbu3CICmE1xGNfhTHR o/P2LK0i0XtpoXyE+a1Wqq6TJ8dXDS83gGcR1mvWnopVytCSBBuVDZGQMksX5hYTNfHR TmAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Ocomz11lJ9KE0anC5CzM99SKlSLQ0mxGbD3mCLha9hU=; b=XRB7T/i666LNg1PFH7OI71wkGvZQUEOncYK/j7Ommz2yN3g2JnEWl8462gmgwqRV5I +IFK3wiFN/5Co8gZU9vpl1UV9kNl3mQuQVH5Gjk7X3Py5/FMHRrG5tvzimdJicwQGtgw XrQusrYFjrL4N7wykF2qVE2AQnD0gu7p2mIdub8PO7R55SfTYq4Z83DCzLRWmk12g9dE a+hFJfhxffoFkHTWqb8CIIECgyxgtn35OJdsKC4+GaQOAWWvl6UHG73slvMgBi8104kJ 4c4FJp49gCKKVZ3mwsbRgzzWnIpKnOZ1hFXCdufCa8V483kIoPBbzBryYOW9T5DY22Ve kONQ== X-Gm-Message-State: AOAM5327q2ZYOAC+anjEB/h1MsMjtlFCb7JzRYmIwTAQvmc4tVka4ZN+ oYqh0rpxhhlsujawG/d3oJyMcF1yqSJx4i0K X-Google-Smtp-Source: ABdhPJxLptJ2NppRam2U5gruDbgx9bSTsC/cH6+ET8/IVRhCo/jAA8pQOAD70gM5B+ZoIP4EkLlv3XYmcaaXloEn X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:600c:22d3:: with SMTP id 19mr178957wmg.161.1604534557830; Wed, 04 Nov 2020 16:02:37 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:11 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 01/20] kasan: simplify quarantine_put call site From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000339_996444_901E3214 X-CRM114-Status: GOOD ( 12.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move get_free_info() call into quarantine_put() to simplify the call site. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Iab0f04e7ebf8d83247024b7190c67c3c34c7940f --- mm/kasan/common.c | 2 +- mm/kasan/kasan.h | 5 ++--- mm/kasan/quarantine.c | 3 ++- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 2bb0ef6da6bd..5712c66c11c1 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -308,7 +308,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, kasan_set_free_info(cache, object, tag); - quarantine_put(get_free_info(cache, object), cache); + quarantine_put(cache, object); return IS_ENABLED(CONFIG_KASAN_GENERIC); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index b0a57d8f9803..994be9979ffd 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -214,12 +214,11 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, #if defined(CONFIG_KASAN_GENERIC) && \ (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) -void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); +void quarantine_put(struct kmem_cache *cache, void *object); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); #else -static inline void quarantine_put(struct kasan_free_meta *info, - struct kmem_cache *cache) { } +static inline void quarantine_put(struct kmem_cache *cache, void *object) { } static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 580ff5610fc1..a0792f0d6d0f 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -161,11 +161,12 @@ static void qlist_free_all(struct qlist_head *q, struct kmem_cache *cache) qlist_init(q); } -void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache) +void quarantine_put(struct kmem_cache *cache, void *object) { unsigned long flags; struct qlist_head *q; struct qlist_head temp = QLIST_INIT; + struct kasan_free_meta *info = get_free_info(cache, object); /* * Note: irq must be disabled until after we move the batch to the From patchwork Thu Nov 5 00:02:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF171C00A89 for ; Thu, 5 Nov 2020 05:10:40 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4BF482151B for ; Thu, 5 Nov 2020 05:10:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="B/LBtXIG"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="PS9X/wSH"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="QP+KU7EF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BF482151B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ewioDlKkNzSBeQY8sYsyK36E94H7uk8JTgqiqUKbpiI=; b=B/LBtXIGT02hbuiKq9OF5liON upqKnWeM0366xHgE6FuIDF7cpora+0KzpAjFuXBeyZM67DoB0QRYnkNg5o+7s5CeleQWW7hhQnpnN j7AIzoahysYFQboUISlvl6YHmOcQX3smiFtAD3IAg+8HDihtpRefSOea7ats+P20Hub9NJ323uYip NqhZWtkW6y3UUC5ruQK8xfeflhgqkWWRbex75mkQLL3Jo95tmHXPSufla9qWtiQJRlDggCnxiiAzW wFq4FzzALyTbUMuuWA9kEFb+kuAdtPIXf+T7MQeDxIgSke1K6esI5O2eWE/cj1bFP7OTieOHNKZNX Ty4CyilEA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXVp-0006Fb-Ex; Thu, 05 Nov 2020 05:08:41 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3u-000145-Lw for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=XrlC+aEcWbd3aE57QC+9E5FwHOZXpbQu57tR6JwJ5+c=; b=PS9X/wSHbWMxihOnAZv794+w4p ehM2Ts3c8U9GlYHsC+EBbHh1qH5CL5I1kcM/yGCDH0VK3B9eGazMs+8ugtlKnQSx8J2maeT2TTxyx Jt3uMQ8eGai+8gPFOrldbmcjAffxggf7Y3rWnTCJEy/eFSxZkyLusfD3AycatzapeTkhXfIwlf3Kb kaHe7m7kz+yluaVaT2mhV8RoMzzuDTN7PEC7NE9EUZfmebosQXNq0eTRqUl3NJlnYpWntjwB4lCCK 7H8C476YWoVOPgwl0sXDLOYRN7Zu0BsH95OolDMfO+R5MpXsosXYSeKllQ8+k6ByiYf0n0q/bWB86 6TiB82nA==; Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSkk-0000nA-5E for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:03:49 +0000 Received: by mail-qk1-x74a.google.com with SMTP id x5so47729qkn.2 for ; Wed, 04 Nov 2020 16:03:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=XrlC+aEcWbd3aE57QC+9E5FwHOZXpbQu57tR6JwJ5+c=; b=QP+KU7EF4Sooeu0hjMOHvPUzQF5nQTLtWDZRHBDFWo88qNIpvOh07JBtE2fO5pXWaa R2nTiR6er0b98Pvy+W+XlgRLf7zR1sLuN2//KVj6b9PXc+/aOMRBigj8v/7QVPoiL9Ju b6m8aQKraSwfZVHEyK2pDWQmH7p0DsDzXA1VjjlXo1z1H+JkXjMnD36++hDR3wGQj5KR ytFLM4fhOYCgVsdpyorwZk9kuclI6166ICSVqKPWb3/ae2h4dablms2ZyUMkE+6e6LDQ L+5MmUM+ILWRfLpr4UWDqPvyPANmq8PiziVXC4DHIVZ/e2nd1cOvIXLV1P39AQpQ1ZiN dJEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=XrlC+aEcWbd3aE57QC+9E5FwHOZXpbQu57tR6JwJ5+c=; b=Ia0mBIHWL3VBp3tdW70ud/LfLRCwxev+8BAPs8oOcPtcaHZB+4l+6tGlvKWeZ9ehsR X6/2Zq39YK9C7kK/0z0sQ0wh1UxKngh49qHzEwfNmR1VF7M+pH9zHsmrHhwTZ+ffFmex 8dGW7R+BDBQ5n2rm13v8UFteNWd88eEOrXOcG6yoMz3K6E/I5sKkkJcjfIrns+/ksqJX m7bGVj2fd6tjnIc1LaFqmRQd4kLzwR3GPRA/RTYtBQyZqcix098QfwAYOOi0gRPt28Fr sMNBP8rmhfAdkU/+7w0jAaA3mf8VimbbGhW+Bq5VfDT6XSg3h6PEbWT5IJ+qhNeKXjLg 9gUQ== X-Gm-Message-State: AOAM533hH8lWTsUA9cqM8YciN3UpUJE9O/4EHgJFdbwvrduTopMxFUHj WmHqFa0JO6FCDtp1te77lutAPTjuf6CqXQ9R X-Google-Smtp-Source: ABdhPJxFHKaDtHayh34H7x/LU75rTQ4Ma+mGckmHaWDT2iy4/BgEE9wclXaglhHGlAM5BH0Ftdvx8GrFbsQPy5MT X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:ecc8:: with SMTP id o8mr293863qvq.54.1604534560168; Wed, 04 Nov 2020 16:02:40 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:12 +0100 In-Reply-To: Message-Id: <7fb01c82767f6ec2ef804ec4689b7a9620b5bb4d.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 02/20] kasan: rename get_alloc/free_info From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000346_357966_597E2156 X-CRM114-Status: GOOD ( 18.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Rename get_alloc_info() and get_free_info() to kasan_get_alloc_meta() and kasan_get_free_meta() to better reflect what those do and avoid confusion with kasan_set_free_info(). No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Ib6e4ba61c8b12112b403d3479a9799ac8fff8de1 --- mm/kasan/common.c | 16 ++++++++-------- mm/kasan/generic.c | 12 ++++++------ mm/kasan/hw_tags.c | 4 ++-- mm/kasan/kasan.h | 8 ++++---- mm/kasan/quarantine.c | 4 ++-- mm/kasan/report.c | 12 ++++++------ mm/kasan/report_sw_tags.c | 2 +- mm/kasan/sw_tags.c | 4 ++-- 8 files changed, 31 insertions(+), 31 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 5712c66c11c1..8fd04415d8f4 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -175,14 +175,14 @@ size_t kasan_metadata_size(struct kmem_cache *cache) sizeof(struct kasan_free_meta) : 0); } -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) +struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, + const void *object) { return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; } -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) +struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, + const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; @@ -259,13 +259,13 @@ static u8 assign_tag(struct kmem_cache *cache, const void *object, void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { - struct kasan_alloc_meta *alloc_info; + struct kasan_alloc_meta *alloc_meta; if (!(cache->flags & SLAB_KASAN)) return (void *)object; - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -345,7 +345,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) - kasan_set_track(&get_alloc_info(cache, object)->alloc_track, flags); + kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); return set_tag(object, tag); } diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index adb254df1b1d..d259e4c3aefd 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -329,7 +329,7 @@ void kasan_record_aux_stack(void *addr) { struct page *page = kasan_addr_to_page(addr); struct kmem_cache *cache; - struct kasan_alloc_meta *alloc_info; + struct kasan_alloc_meta *alloc_meta; void *object; if (!(page && PageSlab(page))) @@ -337,13 +337,13 @@ void kasan_record_aux_stack(void *addr) cache = page->slab_cache; object = nearest_obj(cache, page, addr); - alloc_info = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); /* * record the last two call_rcu() call stacks. */ - alloc_info->aux_stack[1] = alloc_info->aux_stack[0]; - alloc_info->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); + alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; + alloc_meta->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); } void kasan_set_free_info(struct kmem_cache *cache, @@ -351,7 +351,7 @@ void kasan_set_free_info(struct kmem_cache *cache, { struct kasan_free_meta *free_meta; - free_meta = get_free_info(cache, object); + free_meta = kasan_get_free_meta(cache, object); kasan_set_track(&free_meta->free_track, GFP_NOWAIT); /* @@ -365,5 +365,5 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK) return NULL; - return &get_free_info(cache, object)->free_track; + return &kasan_get_free_meta(cache, object)->free_track; } diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 25ae7b43db87..d858aeb7387f 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -62,7 +62,7 @@ void kasan_set_free_info(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); } @@ -71,6 +71,6 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); return &alloc_meta->free_track[0]; } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 994be9979ffd..5513b4685007 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -148,10 +148,10 @@ struct kasan_free_meta { #endif }; -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object); -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object); +struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, + const void *object); +struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, + const void *object); void kasan_poison_memory(const void *address, size_t size, u8 value); diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index a0792f0d6d0f..0da3d37e1589 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -166,7 +166,7 @@ void quarantine_put(struct kmem_cache *cache, void *object) unsigned long flags; struct qlist_head *q; struct qlist_head temp = QLIST_INIT; - struct kasan_free_meta *info = get_free_info(cache, object); + struct kasan_free_meta *meta = kasan_get_free_meta(cache, object); /* * Note: irq must be disabled until after we move the batch to the @@ -179,7 +179,7 @@ void quarantine_put(struct kmem_cache *cache, void *object) local_irq_save(flags); q = this_cpu_ptr(&cpu_quarantine); - qlist_put(q, &info->quarantine_link, cache->size); + qlist_put(q, &meta->quarantine_link, cache->size); if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE)) { qlist_move_all(q, &temp); diff --git a/mm/kasan/report.c b/mm/kasan/report.c index ce06005d4052..0cac53a57c14 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -164,12 +164,12 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, static void describe_object(struct kmem_cache *cache, void *object, const void *addr, u8 tag) { - struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object); + struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); if (cache->flags & SLAB_KASAN) { struct kasan_track *free_track; - print_track(&alloc_info->alloc_track, "Allocated"); + print_track(&alloc_meta->alloc_track, "Allocated"); pr_err("\n"); free_track = kasan_get_free_track(cache, object, tag); if (free_track) { @@ -178,14 +178,14 @@ static void describe_object(struct kmem_cache *cache, void *object, } #ifdef CONFIG_KASAN_GENERIC - if (alloc_info->aux_stack[0]) { + if (alloc_meta->aux_stack[0]) { pr_err("Last call_rcu():\n"); - print_stack(alloc_info->aux_stack[0]); + print_stack(alloc_meta->aux_stack[0]); pr_err("\n"); } - if (alloc_info->aux_stack[1]) { + if (alloc_meta->aux_stack[1]) { pr_err("Second to last call_rcu():\n"); - print_stack(alloc_info->aux_stack[1]); + print_stack(alloc_meta->aux_stack[1]); pr_err("\n"); } #endif diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index aebc44a29e83..317100fd95b9 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -46,7 +46,7 @@ const char *get_bug_type(struct kasan_access_info *info) if (page && PageSlab(page)) { cache = page->slab_cache; object = nearest_obj(cache, page, (void *)addr); - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); for (i = 0; i < KASAN_NR_FREE_STACKS; i++) if (alloc_meta->free_pointer_tag[i] == tag) diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index dfe707dd8d0d..3bffb489b144 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -174,7 +174,7 @@ void kasan_set_free_info(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; u8 idx = 0; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY idx = alloc_meta->free_track_idx; @@ -191,7 +191,7 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; int i = 0; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { From patchwork Thu Nov 5 00:02:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6421CC388F9 for ; Thu, 5 Nov 2020 05:14:03 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AE2A62083B for ; Thu, 5 Nov 2020 05:14:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="JDYQRqxr"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="UwrCYcnl"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="TKDRruDc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE2A62083B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=8HEHAwtuxp4zao36wKO2ABmbHTlULKy2EEaCOvl5yIE=; b=JDYQRqxrQf3u4Y9oDfPBfdhpG IfxM+uL4wN4TOXK8yDjoYg/CQ3V1AF18f40PLFvhfIFf+8oQPIVk6rzxt2oWUxKB9Dc169ZTRW60c zzAqLBqc0pfmHtC+XOw6cX+aapTqpQpw07k2eb0eRuk2aOSeGcu839IcwbMHmE8K4G50CqS3kM1pf 3uF4puScdV/MML5nKbaOh+R0Ch4ELPHsspVypiXLb+JSC8C4SiMaMfG0RLzaR2xxWzGUq9vaNk5/T u6AphaSq5RoJYrykAsJ2zQ8ockXp71Q4SO2EPyHbIcPG3tc8YoZ0NZ9qjIOXbfbmEvXZOaR+rcxdZ oTK/DgQCg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXYa-0007QY-HB; Thu, 05 Nov 2020 05:11:32 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3u-00016k-Nx for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=vCuap+Z40WCfgwTrVBGRXx2rimwF3+Nybx2YvcmvQUo=; b=UwrCYcnlYctrO9fAt9BtGExukz pe8geK2HnmBpGElsbcviTWBLNx/vwZ39RdBlzKieEFGxn/ItVvNEobmEYAnxVy99s1z/UjZbtBNxx 5BWENF6EOkBhQb7sD8jpDqEGIFbXon2zDw7PTYweVCHk9b+/vf2Y5SabNt0w4wwlMsDmO8iuAJgYx cP6XNjRzc87ZjY1PmWEsHBUU0IZTWU9314u2k7oKSQ/iQkAj7ml4R5FCNclSV2wtzQnW1EHgYBAjH g1g3djrE2vNjbMafj1YNqarBsMXherE30W/Zy3oWkQBw5kMKPIbXquu0GgQTzvBSzj/FTN9OCudpZ mLmt9t+Q==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSkj-0000n9-DY for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:03:47 +0000 Received: by mail-wr1-x44a.google.com with SMTP id w6so103144wrk.1 for ; Wed, 04 Nov 2020 16:03:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=vCuap+Z40WCfgwTrVBGRXx2rimwF3+Nybx2YvcmvQUo=; b=TKDRruDc0EPKdW5qP5kZx93U6ILuVOKa9NRxOsJBSbrsNehB2La/XSXsAe/9hQyaMk yCK6FVozl9zmqc2IUI/9cYSE7LNyU/0mpvnpHD2T8bl7AVgkRNS2UNY1OunTzaHIAXdt FE91Mn1bMT+89GiGiG+pSKXKCCCi7QTzrqooRDTGgQ6Ctjnx1U3rrYtD1VR4mL6FgcFK c/85JUoGFBsSf4jLoyMoeAlfhJucBZFp+w6YrIBmmw+RbFgil2rQH+u6WqYjog3CXg9X fu8SkOWMFfLel2sGmtiWpsbMBhumZ52ebQMgrd624Xe4e6DKIvYVbygT2KF7gS/efkdd Cm0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=vCuap+Z40WCfgwTrVBGRXx2rimwF3+Nybx2YvcmvQUo=; b=hnhxH/hjIs6JLH+wqReol6IuCUvRpItCeIa5cXE3fGx2CwBeGvKPzNLRRO3EANx2Gy OlXR7nQK3JADVvPOFGrhp+ERFh/alMwJAy9BiddHpwsQCbIOLI8xWfGgBwCCMAlvNbJf PvAPpkPxXU3d1sPwpeK1F3uz2gpEaVh22toTm70+qpq9VhcJO5b0KeEH7mgsl2mco+Bt QuPWOSizwfe/Pbc4P9Ju8KzBU0YeFDyF4fnBhg3jNPULwCgzwF8aHINjp1abtEQMwFdU vlWHJwsu8Igo7vuZdfFJ46GIJevYaZ3cGrlBpYup42oEumkEaUsBN0NpW2NmdtskZXk+ DPvA== X-Gm-Message-State: AOAM531+7GLnisIyBWLE3qRsIoLBDDRYc7u+f3soC20WNkAs1RtFvdeo yQqGra1PdexEqZvtHbdVbCXzLB83KPJzbhli X-Google-Smtp-Source: ABdhPJwuuyHLeDdD+KHqiqHb6fDGG/rhTQChDK6P5WB9OmWfgwhMcOyy6D+DkRkEMkd2PMl1yMhO3tDZC6Em3M7m X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a5d:4d0d:: with SMTP id z13mr477608wrt.23.1604534562699; Wed, 04 Nov 2020 16:02:42 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:13 +0100 In-Reply-To: Message-Id: <4a32aecc6761e93d792cb2b78af86689025627bd.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 03/20] kasan: introduce set_alloc_info From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000345_581877_705D118A X-CRM114-Status: GOOD ( 10.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add set_alloc_info() helper and move kasan_set_track() into it. This will simplify the code for one of the upcoming changes. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/I0316193cbb4ecc9b87b7c2eee0dd79f8ec908c1a --- mm/kasan/common.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 8fd04415d8f4..a880e5a547ed 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -318,6 +318,11 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } +static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) +{ + kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); +} + static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags, bool keep_tag) { @@ -345,7 +350,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) - kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); + set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); } From patchwork Thu Nov 5 00:02:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C0E7C00A89 for ; Thu, 5 Nov 2020 04:24:45 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BC6EA20795 for ; Thu, 5 Nov 2020 04:24:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="s7Mk4dy+"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="RHfIBJGI"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="PRQZtB1d" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BC6EA20795 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=M6hnCYCv8zmGWktrGOkW6ubQAFdn3UCjF5fkyWLQDr8=; b=s7Mk4dy+r/O06wUvv+B/S3w38 UGk7Q2rXcSUlIbuM34fBFvRrliaGHC1smQdFTfoMhFNxgVWPl70VA0mUPBHWGeNuSB2fpO32Cv1Pv P94GX835FfR/YwufUBPJ/1UiHbHPH2Yrd2qaliPTACF9Ub54W+nAdQxq3KlJS/R18/aPAXd/ewPh0 LI8ZiU5M3me6C/HoHtYG9fE7zOMzYjIjvGTyHxDokevmzpHE0x6IRLjw4tHvmHA19gBNk97UMGnc6 6rNT/teoTlABkyIVpSlT+39gqALDNobMrp5MGokYJPW3ppZS0+VEOJFi8bNbu8rwzO0LTh8i/M/bR 3suOW8ISg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaWnj-0006Pt-Vd; Thu, 05 Nov 2020 04:23:08 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3s-00016k-UY for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=75cYXImCPyiWQ9SKzSdNGkBAnxoIkaWvKX26xtnhUe0=; b=RHfIBJGIapbZ80NiMIeneE92WT n2Q9Y4cu1Mt7jB5yVNhM1PpaviCb+gAibad76iGB1lZtyOLycLUvzjHFOIYzZpaymNSjsGOtJ1hHQ WGjDt8OdqPCCu/6mEgEBWcvslHkpkxWgQr4CmyGDLfBe6Haf/OF119PXwOyjVEe6GOsmg42WmxvUM BjVibITT5JxHWtKGAPu7x7MOT6H0QZzy9jp+/eJjuH4lXprC4SFTTv9c8NgVKRWoevSw+85YbmTDQ q6e7Co1mGZfhwg/qiCK1gNsirAmyvJVf9dAuciAQX9Ui+cMe9TKcWVyiz4tCp0mZ8mu7TlwaMkLB3 srP7Pa6w==; Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSkm-0000oC-R0 for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:03:53 +0000 Received: by mail-qv1-xf49.google.com with SMTP id dd7so50032qvb.6 for ; Wed, 04 Nov 2020 16:03:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=75cYXImCPyiWQ9SKzSdNGkBAnxoIkaWvKX26xtnhUe0=; b=PRQZtB1dufXLPUzgGaqIbaWhMXmesRjAadeDWIt74dEP8Lfsgm2ryEy6s7q8fzmlIV IWJOdm09vXFCNhnpM8EQA7rqqWqjMR9RQQq+tdXuXK3ckY+h7vf59jeSgqtX019vBSLK j8ygqaBGulNwu0YsR9bBAAvVI10PL05Y3wtvXuXQxic2EsEZyEkhcax1ETvG7h0+ymCX e50LJMX71RYCnHJ+gYJyjhKhP+EGhLi7Hl2CmIlnTEpAV9+5QzbT4cIQ3P6MRc7P6q8P Mm3sjdpxfFKeB/L9pqme/H+/pNL5HjbpvJH5LX3yE30dm0HKjUCMmqsnucEmfIgriAJ6 xrLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=75cYXImCPyiWQ9SKzSdNGkBAnxoIkaWvKX26xtnhUe0=; b=r7+P/284kK3Vjh/Io2w38/Z8Vsfd6h1DU/NRLECWO974El6DX63Ka32ePzzNWkT1Qq Rc6z2sNWVpV87qcH5CO472/Aw/7W0+TItXIM9W3Vj7dnCeIFDOEAMFXsFI4StjU5ZVKM UoeRrvJJ4GFrGMY/h8CvNANSa1roULSdf7+rXfXE1pTKYId1i1hfV0zeNXp/ou/qteuc sJeIe01bS27cgC+lyfaL4zudZHf/VBu11cArY++vk4eqZygO+Cb4A4/9FCPR5EBk36ta CI4RQv6ETXfz/V3OfIfT0OS70sKO1xDkyJFCSmWHYgbq1smf5fv7bi6/66bO5Z+UZ4vf Cm1g== X-Gm-Message-State: AOAM5328vcGDbYcZ5jP1vx1wyKTm28fGZb+MR+1scHa+oCUJ0h/KaWiC c8ZVZc+O5m/eKKoPRLsV4Z2Cg5M2/FCofwBd X-Google-Smtp-Source: ABdhPJw7NtC8X2ma+DihzOyHxYwzEo/bH3H1mERPSZkcC0cm1czCOnJJGd/mJicu4ukRtxMsLfXt8UL78l7IEHSt X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:ad4:4443:: with SMTP id l3mr356156qvt.53.1604534564913; Wed, 04 Nov 2020 16:02:44 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:14 +0100 In-Reply-To: Message-Id: <16e48c6ca3f6ea0cb80d3555723a723bef85270d.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 04/20] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000349_117175_B775CC15 X-CRM114-Status: GOOD ( 14.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There's a config option CONFIG_KASAN_STACK that has to be enabled for KASAN to use stack instrumentation and perform validity checks for stack variables. There's no need to unpoison stack when CONFIG_KASAN_STACK is not enabled. Only call kasan_unpoison_task_stack[_below]() when CONFIG_KASAN_STACK is enabled. Note, that CONFIG_KASAN_STACK is an option that is currently always defined when CONFIG_KASAN is enabled, and therefore has to be tested with #if instead of #ifdef. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/If8a891e9fe01ea543e00b576852685afec0887e3 --- arch/arm64/kernel/sleep.S | 2 +- arch/x86/kernel/acpi/wakeup_64.S | 2 +- include/linux/kasan.h | 10 ++++++---- mm/kasan/common.c | 2 ++ 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S index ba40d57757d6..bdadfa56b40e 100644 --- a/arch/arm64/kernel/sleep.S +++ b/arch/arm64/kernel/sleep.S @@ -133,7 +133,7 @@ SYM_FUNC_START(_cpu_resume) */ bl cpu_do_resume -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK mov x0, sp bl kasan_unpoison_task_stack_below #endif diff --git a/arch/x86/kernel/acpi/wakeup_64.S b/arch/x86/kernel/acpi/wakeup_64.S index c8daa92f38dc..5d3a0b8fd379 100644 --- a/arch/x86/kernel/acpi/wakeup_64.S +++ b/arch/x86/kernel/acpi/wakeup_64.S @@ -112,7 +112,7 @@ SYM_FUNC_START(do_suspend_lowlevel) movq pt_regs_r14(%rax), %r14 movq pt_regs_r15(%rax), %r15 -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK /* * The suspend path may have poisoned some areas deeper in the stack, * which we now need to unpoison. diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 42a556c5d67c..2c37a39b76ed 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -74,8 +74,6 @@ static inline void kasan_disable_current(void) {} void kasan_unpoison_memory(const void *address, size_t size); -void kasan_unpoison_task_stack(struct task_struct *task); - void kasan_alloc_pages(struct page *page, unsigned int order); void kasan_free_pages(struct page *page, unsigned int order); @@ -120,8 +118,6 @@ void kasan_restore_multi_shot(bool enabled); static inline void kasan_unpoison_memory(const void *address, size_t size) {} -static inline void kasan_unpoison_task_stack(struct task_struct *task) {} - static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} @@ -173,6 +169,12 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK +void kasan_unpoison_task_stack(struct task_struct *task); +#else +static inline void kasan_unpoison_task_stack(struct task_struct *task) {} +#endif + #ifdef CONFIG_KASAN_GENERIC void kasan_cache_shrink(struct kmem_cache *cache); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a880e5a547ed..a3e67d49b893 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -58,6 +58,7 @@ void kasan_disable_current(void) } #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +#if CONFIG_KASAN_STACK static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) { void *base = task_stack_page(task); @@ -84,6 +85,7 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) kasan_unpoison_memory(base, watermark - base); } +#endif /* CONFIG_KASAN_STACK */ void kasan_alloc_pages(struct page *page, unsigned int order) { From patchwork Thu Nov 5 00:02:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2E2AC00A89 for ; Thu, 5 Nov 2020 04:33:01 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 537EB20729 for ; Thu, 5 Nov 2020 04:33:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="UoYo0zP/"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="afuiVOrN"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="o1BHF1EV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 537EB20729 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=nFJBxHZ6AZ0inMeGg/lLXNK4GcpIeZzQf69SqIpgfHQ=; b=UoYo0zP/gANBRiAjtY0+jou3z vUfkZf4JLRd31D5JyWOgjtHDyxdFVtfyzJo0GxNjhavrimvaWWoRO8b6+CKvqHVd5oLd7S2PsE4m8 TAPHOzrFA+Ehbie16HB9DkToELc7S56kRHxFBCBDnPzMzxuPJVtVJ6ScG3XJB0nyu7Yp+ROrfBnFh 8kAVAMbXQEv5Sn5qAB41KO1bdz/xeUjJ4uc4JZP9ntMsx6pCkl0RJ8/BgmLVUu8MVA+jxWqZgrcjW uGIiNcVSH6d6f4x0mzi8Qq2UMuPiATf0x4+fx7jpTcpeLDOWkM3a2A5HthFmaSKRukgp+J88O3e9m Xh04v4sQQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaWwk-0002N6-4T; Thu, 05 Nov 2020 04:32:26 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3t-000145-QZ for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=z6j3WUTg8MpmEPLYdtLimxVBLIkxxjwsBk3bJk+FZ/c=; b=afuiVOrNuq65jGpeuaEORjQFo0 1kOk3Un/ePsC2ryGKday8wc1RFCOywDA/a3KJ9X2aaq8oXpI+Jw+S4wP+llFZNkeOK1lk+gjhayAX VpUM6tBKYj++pUL++xPSDAbw/bGDVcJFr1wE8/Y3mGoB8iSJhAjdEhQp2TQ1+EM9rTL1LrVhjizWM LczrbvQ4XqdMpzzfNUlphAhgWS/gZINXjLBVxq/6KCLN/IkymzeUBZeS3o4JOYTtsLix4N5KSJvAO /9wvKHJF/40estYC80+oBViLJAM8P7/mLwfdnhd0RPoz8FVImiiWcCv1ggP8IkzMs+uqXK3s9mxcl JO7mSHyw==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSkm-0000oA-93 for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:03:52 +0000 Received: by mail-wr1-x44a.google.com with SMTP id m4so82487wrq.23 for ; Wed, 04 Nov 2020 16:03:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=z6j3WUTg8MpmEPLYdtLimxVBLIkxxjwsBk3bJk+FZ/c=; b=o1BHF1EVbMTrooct4IuVuzwUadRydSkmLut5zGwLfIW86KeTj+T7+3qQd8RFCYYVk1 ZMQRhpeEUB0u7Y/NHq7fRwnmcx3ssRcGQXKNVGsCHvT9J0ec5Uo0HSkVZwIlNtxDiWBV 1AYf95vBSDu2mEJnYgYDgiUD2HOPIlRa801YsQlyBFlE4oaIsvs/R1fU90M1/78nGH31 rSpbf1mVkKGWUcgjBXgG/dZMJz3TUYg1+KwIJX3sm4kRmuAVkTFAYArvZdBWFDmDFy5c 75+gVsds4JozQ2HcmfCmLiO0RpdOJkKQTUuAkL7OUg7yjQY0LQHEif8Q02Q9qTWV0+Hq AH5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=z6j3WUTg8MpmEPLYdtLimxVBLIkxxjwsBk3bJk+FZ/c=; b=swmnwJKjV9arYd+mEkzZa1asWRvzbkDGEOY+svrNeu2bT/F2DhJQ4qJFcXGscEDm+T Q5f7o/BXlOPq3LUj037Od4bV5KygU66NPS/oUzCwl6O1//skduv3EsEg4hl+rxBeiMS6 2imxqW05brv9gG6HlcqdWLzOQkdEM0BPsVEsUi89tjfBGDtvOm0xCxFuxoPSBfaFksRw Vpl54GEuTPg3Bze6rnsL9HZUbmVEUnZLcVxiZLH9qC2Bn3haFeAdPAkXs6KX7hIfMeB1 JJdpYP12RApVcoxnK8+TayA8ABFnHlh2XpEGHrMGpVtHwDPS/LrunEc+j8v2BhvilSDJ y09w== X-Gm-Message-State: AOAM531sXdBgCIWr2R+Bv3K2tmX8AuRC9o+VEyUNRk5rPifLtmlUhRI2 ibd7JHtBMmpjERuXuYeEaIan5szT6XJ0IBEE X-Google-Smtp-Source: ABdhPJyW9CVHpcZlYLIGEMvyGyz16yUK8Wds51tx5Sii32bcO3BS7futrB9CgRdyZWCAoRPp5flAxMpdXR2/WiKH X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:adf:a315:: with SMTP id c21mr514082wrb.272.1604534567466; Wed, 04 Nov 2020 16:02:47 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:15 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 05/20] kasan: allow VMAP_STACK for HW_TAGS mode From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000348_524058_99A806AD X-CRM114-Status: GOOD ( 11.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Even though hardware tag-based mode currently doesn't support checking vmalloc allocations, it doesn't use shadow memory and works with VMAP_STACK as is. Change VMAP_STACK definition accordingly. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/I3552cbc12321dec82cd7372676e9372a2eb452ac --- arch/Kconfig | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 56b6ccc0e32d..7e7d14fae568 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -914,16 +914,16 @@ config VMAP_STACK default y bool "Use a virtually-mapped stack" depends on HAVE_ARCH_VMAP_STACK - depends on !KASAN || KASAN_VMALLOC + depends on !KASAN || KASAN_HW_TAGS || KASAN_VMALLOC help Enable this if you want the use virtually-mapped kernel stacks with guard pages. This causes kernel stack overflows to be caught immediately rather than causing difficult-to-diagnose corruption. - To use this with KASAN, the architecture must support backing - virtual mappings with real shadow memory, and KASAN_VMALLOC must - be enabled. + To use this with software KASAN modes, the architecture must support + backing virtual mappings with real shadow memory, and KASAN_VMALLOC + must be enabled. config ARCH_OPTIONAL_KERNEL_RWX def_bool n From patchwork Thu Nov 5 00:02:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFB14C00A89 for ; Thu, 5 Nov 2020 04:25:23 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 84AB920795 for ; Thu, 5 Nov 2020 04:25:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="soqVr30i"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="MjHqa6Wv"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="EIGseFkU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84AB920795 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yiHYbBi5ldB5gJydRqD8Xiuffx/oFxU4Gy0nuajN15w=; b=soqVr30iQwK7v0lQh+hqXXdit s/yMa5sswYeShIj0KIUu5a5Goh9tKbIRhmvd5OGwEtpwbKl/tcbPVAeKxZFx+Os1VFe0237SZL6/7 kvwQ5MckRHr9gl+zlizZ63uRClfjjaI/hxoZAaayfvgH5j5vEcUAAWKGZXrYj9TqlxrOY2dZYeYeC wQtSa0EaCQQm0JnZYTqYwe6dzD0L+OMs9WBwpLw9siBr8V+XMg4MCPgey0684o59Wux7Dh+UShuCl wcQTjXKOAWibOuLifhojZp7tse039OZXFyF0E/NxFAIzVwdcIUs+enyV7gs75xYn0WNbGRWKgfMz3 wnaKuK60g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaWoL-0006fs-Go; Thu, 05 Nov 2020 04:23:45 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3s-000145-Ud for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=JW8D6ibToVOb7LpAkCihsputAW6li7gT66iFpWFJlB0=; b=MjHqa6Wv8TOxj9HyXjWTfBVxjV 1ShMDTfMZoxor5EUIXathH0c1PWXlXt/QaJTCRXrzT9uRSWg5LVzM+/p+I1p+QeXhRZZM2GnM5YWP XFWA3cK0s7sn6Nym6Yz560b6EZHeIOmmhlBZHpd4ZG8ewjQrgst13cPHot0OJ/xp59E4hDc9C2bGz hpESCSIYWYwzGuccHm5AxKzqrsoOrm7TLqGVv798kpNHq6zSPlST5zK8B82L5elnnakcjTVzSIc9s e1t+NwowAeotEuiKnFUrC5dLAuNJtpE6LeXzs79hFwqaIpoj3dqloi2fwqPrjlKEvQ50L0ZCt37DB cXdGZ2Eg==; Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSkq-0000ot-5C for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:03:54 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id y11so452975ybm.22 for ; Wed, 04 Nov 2020 16:03:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=JW8D6ibToVOb7LpAkCihsputAW6li7gT66iFpWFJlB0=; b=EIGseFkUbu4j97m+Rhe8ifiJEJ0/vSgBaoaF2oPB6YJuPwOq2yFl8o3MdZTisWpG8v 7KPp5oAaiwsPmjeCUi37l89Z7cZJJj/hhe5YwAUSX9ohPHFQHG9rviik167v9qNGYxg+ P0LNwwi8goSo5pXkQCm2SaSogyReaDgIOTbI5K97sZj9flQ0v+LrD0Pth2BoTDM0Nawq hRM42A6VpyD9FTb4ipeHNtA1FxJV0NecpqgkotIXjwA3kZts685NAweHbMSnkr4kQD/y wrASYwOEExzSGG5j537dCqcaS992EojrvnZ4VEjnIXPSewtClP3KbuoKPCc76tmqA1U1 kE1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=JW8D6ibToVOb7LpAkCihsputAW6li7gT66iFpWFJlB0=; b=AZQvwNdAjdADJGfBUhHHficavg3AuJo8VVqWRgRRVyMumHSXqBp3q8BwmrgXiu+8r6 1ggECxJsuPUoIF1ADgpHpMnK9scFA4IhGBloOQMNMyE+eKEkMfI9c8MaMd16C3NcEnFX +3QWs96KT+ASLGfGMTvBt4uKBst8dx3QQs/MddW+v/l/n1WvHOUuEu6U5blyPRoAOtUb 72GcY12nFI0U8UIP6nSlLt1YICr0BYgP5ewEQoDcK6QMBAwcCCFmndlRMOCWYkzPf041 t+yY28Q3NP1yJpqfynpPvOXoNlX2WN7SLMS2mDlM1W3aEEofIsqbsWsC6EpuGBHvbVIJ wBjA== X-Gm-Message-State: AOAM532wrqOvRGO9YIv8zfMkcMFDbUL2fneuZyV/2Hp87HZ1azhoSlK9 a1ZvDUTmYBGeciHIfSdzwQNeE6Z8VEr4HBig X-Google-Smtp-Source: ABdhPJyMEsriifNDEGfN+BopmjlIR9t9CjJFRB97WqiYs1BD5RFz7wmlp/gc3XkSEP7qscjhGorWWs7vzkyRjDGQ X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a25:48c:: with SMTP id 134mr48783ybe.158.1604534569629; Wed, 04 Nov 2020 16:02:49 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:16 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 06/20] kasan: remove __kasan_unpoison_stack From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000352_468989_9FF396B0 X-CRM114-Status: GOOD ( 11.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There's no need for __kasan_unpoison_stack() helper, as it's only currently used in a single place. Removing it also removes unneeded arithmetic. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Ie5ba549d445292fe629b4a96735e4034957bcc50 --- mm/kasan/common.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a3e67d49b893..9008fc6b0810 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -59,18 +59,12 @@ void kasan_disable_current(void) #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ #if CONFIG_KASAN_STACK -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - kasan_unpoison_memory(base, size); -} - /* Unpoison the entire stack for a task. */ void kasan_unpoison_task_stack(struct task_struct *task) { - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); + void *base = task_stack_page(task); + + kasan_unpoison_memory(base, THREAD_SIZE); } /* Unpoison the stack for the current task beyond a watermark sp value. */ From patchwork Thu Nov 5 00:02:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14536C00A89 for ; Thu, 5 Nov 2020 05:03:04 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7D3A72151B for ; Thu, 5 Nov 2020 05:03:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Aau9DTJG"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Ut7YIsAf"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="DwhOeulf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7D3A72151B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tc3g/72+gnNdu9h8PIDrcA/7Zv02pZgARGIM70HzzPI=; b=Aau9DTJGren83VX8BAqfrPqtn W6icjHd1Od94dVMwK5epgR2KuHNF3E5/OclRJJJ9ryilz+wJr4rP1XTbAu9cfFOsBkrEcBHstilPv NBfDWhG4ShJ8F2+xNozh/apVWUEPcxhoXA8vyFV7LZJeBQdUCVjqpz3ywHTAcVH/x8mjlrygWrbh9 NRpn9gH22DqHILI7Q62AUr2fOw2SHLtTNFwd7lM73LjPEE+/B/tCFGpfxBZPuB+1O+aliLC3sGMYO ldVVTGaKU59aqTeU6Zolh6gsFZYRxJdLJN7zlXR+UYRccO9qEdmoDg3wqXCpu2yeCpkzJ6RSZMu6V Ftus56N3w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXOm-0002yo-7k; Thu, 05 Nov 2020 05:01:24 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3r-00016k-TN for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=/zD7WZPyOqitLWvnpsu4/VKPhFmibJDXB/9ojacn4YU=; b=Ut7YIsAfGVisrVSHxS82EXIB/s SNVNuAR8vFlK36F6CBilSXZsGlRxsAuOTol20pAJsX2n2A+X+QMrExqU44zzve4roGMJbyyA9BlHL zGhThHbZPqtDR5tVXDhmXXNk3WGX3sPo+zC8KSwhSZyK4HQ7j6z2n24qSf1gnIvz/8U3982w7Ipcd GwjCgD92iB4IJrDBQfrNh96NPa0rJrnz3NfNl9/VNfMhKuECMlEcityePN7MImrIHi+9xMg3edz7A NbodHFKrH8aFXjudpILvQ+PjS9qt6jUDLjqogN/c03OKoF7EimF2Ep2DzfDsXLM0PgM/loN7PxaX3 jRMLgRNA==; Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSkt-0000pm-2m for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:03:58 +0000 Received: by mail-qk1-x74a.google.com with SMTP id f126so28167qke.17 for ; Wed, 04 Nov 2020 16:03:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=/zD7WZPyOqitLWvnpsu4/VKPhFmibJDXB/9ojacn4YU=; b=DwhOeulf583kaNb7jVHS5Ju0oa1dB7TrWi/R74xwn9ypCjTjRyYu4S8mpn18QDz9h6 pT0vF/yvBwh0+yLH/hwX3jrJc3VBDxn//Ereo6E9UElkpvh2qx9EdDD8loDgsfhHC8zh /H81ILLovNBrBT7gnCrmEGE3Hhz43hkPqoNJgdV+2pOGCS1BFKJ7i13Zp1VZHe6XFaIp jn4gXLXl+JEcEF3ImyfvsKrHHAE1zL5Sklo2KONJoFsQ6K4WK/fRubaG3V2m+OzSf0fK 4EUGukjbRkVVQM+qlOKiwuaeSn3pwok6hhAoIzm5rWvkIQ7+3EagaNQNqGwDm+x+1tJZ 6coQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=/zD7WZPyOqitLWvnpsu4/VKPhFmibJDXB/9ojacn4YU=; b=iw4V1O7C85gC7SUdYnV1NLTtwqBLGTcGa+lA35LekcjH17WSdmQLxddnBnROH1XAx5 6TaxO/gFNvQieFIf3HYE9tHVDF5a0Ez6z6r7hq0KPXyYVyCrEzufwDq3K5sOo1T83oQi 1fHLCtrr/MkcAHgY4oUwlG0/vvyR+lJ/sZrISlL0DQ1/uLVNiEJ12xMrI3J6ujQH5Zrs GXTK+uaABEhKadss9QPbET2cCgP5nM/Z2ClIr/lXETxDX/x72axp3V5nI9gAJEldVveK IX9hgzfELM4YpLZzvKnhQSAHBaKPBDCkbPmP6J3hEnZk3gY3olUNZFCDRNXwLPUoqUzZ Nx/w== X-Gm-Message-State: AOAM532CUxBStCKdVfLgb6Q69Xm/clxTu8gc/XvidbqFc4MeTiJuoRZX 4hJF4osMrvtSGlqlRBld3ymvrj7H3dWUk0Aj X-Google-Smtp-Source: ABdhPJx8UCj7hDKh8Iv5Gz2Yjdfp1h/FlWdPy/wN7f+g1RtFnkwPWjZFqkJyR10HO5jKvmjyPuN4iXlFK++lmqPj X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:ad4:45ca:: with SMTP id v10mr546432qvt.36.1604534571983; Wed, 04 Nov 2020 16:02:51 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:17 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 07/20] kasan: inline kasan_reset_tag for tag-based modes From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000355_346122_423F959B X-CRM114-Status: GOOD ( 16.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Using kasan_reset_tag() currently results in a function call. As it's called quite often from the allocator code, this leads to a noticeable slowdown. Move it to include/linux/kasan.h and turn it into a static inline function. Also remove the now unneeded reset_tag() internal KASAN macro and use kasan_reset_tag() instead. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/I4d2061acfe91d480a75df00b07c22d8494ef14b5 --- include/linux/kasan.h | 5 ++++- mm/kasan/common.c | 6 +++--- mm/kasan/hw_tags.c | 9 ++------- mm/kasan/kasan.h | 4 ---- mm/kasan/report.c | 4 ++-- mm/kasan/report_hw_tags.c | 2 +- mm/kasan/report_sw_tags.c | 4 ++-- mm/kasan/shadow.c | 4 ++-- mm/kasan/sw_tags.c | 9 ++------- 9 files changed, 18 insertions(+), 29 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 2c37a39b76ed..0211a4ec5d87 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -191,7 +191,10 @@ static inline void kasan_record_aux_stack(void *ptr) {} #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) -void *kasan_reset_tag(const void *addr); +static inline void *kasan_reset_tag(const void *addr) +{ + return (void *)arch_kasan_reset_tag(addr); +} bool kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 9008fc6b0810..a266b90636a1 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -174,14 +174,14 @@ size_t kasan_metadata_size(struct kmem_cache *cache) struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object) { - return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; + return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset; } struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; + return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } void kasan_poison_slab(struct page *page) @@ -278,7 +278,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, tag = get_tag(object); tagged_object = object; - object = reset_tag(object); + object = kasan_reset_tag(object); if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != object)) { diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index d858aeb7387f..fe8e6c8e6319 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -26,20 +26,15 @@ void kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void *kasan_reset_tag(const void *addr) -{ - return reset_tag(addr); -} - void kasan_poison_memory(const void *address, size_t size, u8 value) { - hw_set_mem_tag_range(reset_tag(address), + hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), value); } void kasan_unpoison_memory(const void *address, size_t size) { - hw_set_mem_tag_range(reset_tag(address), + hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 5513b4685007..e9c7d061fbe5 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -246,15 +246,11 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) return addr; } #endif -#ifndef arch_kasan_reset_tag -#define arch_kasan_reset_tag(addr) ((void *)(addr)) -#endif #ifndef arch_kasan_get_tag #define arch_kasan_get_tag(addr) 0 #endif #define set_tag(addr, tag) ((void *)arch_kasan_set_tag((addr), (tag))) -#define reset_tag(addr) ((void *)arch_kasan_reset_tag(addr)) #define get_tag(addr) arch_kasan_get_tag(addr) #ifdef CONFIG_KASAN_HW_TAGS diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 0cac53a57c14..25ca66c99e48 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -328,7 +328,7 @@ void kasan_report_invalid_free(void *object, unsigned long ip) unsigned long flags; u8 tag = get_tag(object); - object = reset_tag(object); + object = kasan_reset_tag(object); #if IS_ENABLED(CONFIG_KUNIT) if (current->kunit_test) @@ -361,7 +361,7 @@ static void __kasan_report(unsigned long addr, size_t size, bool is_write, disable_trace_on_warning(); tagged_addr = (void *)addr; - untagged_addr = reset_tag(tagged_addr); + untagged_addr = kasan_reset_tag(tagged_addr); info.access_addr = tagged_addr; if (addr_has_metadata(untagged_addr)) diff --git a/mm/kasan/report_hw_tags.c b/mm/kasan/report_hw_tags.c index da543eb832cd..57114f0e14d1 100644 --- a/mm/kasan/report_hw_tags.c +++ b/mm/kasan/report_hw_tags.c @@ -22,7 +22,7 @@ const char *get_bug_type(struct kasan_access_info *info) void *find_first_bad_addr(void *addr, size_t size) { - return reset_tag(addr); + return kasan_reset_tag(addr); } void metadata_fetch_row(char *buffer, void *row) diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index 317100fd95b9..7604b46239d4 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -41,7 +41,7 @@ const char *get_bug_type(struct kasan_access_info *info) int i; tag = get_tag(info->access_addr); - addr = reset_tag(info->access_addr); + addr = kasan_reset_tag(info->access_addr); page = kasan_addr_to_page(addr); if (page && PageSlab(page)) { cache = page->slab_cache; @@ -72,7 +72,7 @@ const char *get_bug_type(struct kasan_access_info *info) void *find_first_bad_addr(void *addr, size_t size) { u8 tag = get_tag(addr); - void *p = reset_tag(addr); + void *p = kasan_reset_tag(addr); void *end = p + size; while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 616ac64c4a21..8e4fa9157a0b 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -81,7 +81,7 @@ void kasan_poison_memory(const void *address, size_t size, u8 value) * some of the callers (e.g. kasan_poison_object_data) pass tagged * addresses to this function. */ - address = reset_tag(address); + address = kasan_reset_tag(address); shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); @@ -98,7 +98,7 @@ void kasan_unpoison_memory(const void *address, size_t size) * some of the callers (e.g. kasan_unpoison_object_data) pass tagged * addresses to this function. */ - address = reset_tag(address); + address = kasan_reset_tag(address); kasan_poison_memory(address, size, tag); diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index 3bffb489b144..d1af6f6c6d12 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -67,11 +67,6 @@ u8 random_tag(void) return (u8)(state % (KASAN_TAG_MAX + 1)); } -void *kasan_reset_tag(const void *addr) -{ - return reset_tag(addr); -} - bool check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { @@ -107,7 +102,7 @@ bool check_memory_region(unsigned long addr, size_t size, bool write, if (tag == KASAN_TAG_KERNEL) return true; - untagged_addr = reset_tag((const void *)addr); + untagged_addr = kasan_reset_tag((const void *)addr); if (unlikely(untagged_addr < kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) { return !kasan_report(addr, size, write, ret_ip); @@ -126,7 +121,7 @@ bool check_memory_region(unsigned long addr, size_t size, bool write, bool check_invalid_free(void *addr) { u8 tag = get_tag(addr); - u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(reset_tag(addr))); + u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(kasan_reset_tag(addr))); return (shadow_byte == KASAN_TAG_INVALID) || (tag != KASAN_TAG_KERNEL && tag != shadow_byte); From patchwork Thu Nov 5 00:02:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCC19C00A89 for ; Thu, 5 Nov 2020 05:03:29 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D8212083B for ; Thu, 5 Nov 2020 05:03:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="V9lTLeDj"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="p67rrdlX"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="cMYeDNAI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D8212083B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=KwmBp53CdNzCQIt8XhHMA1N7EuuXLuiEYT7j3E25NEQ=; b=V9lTLeDjdy41KRt8k/iKyibbC MAq/PYcI8ape1W3pxogJYXG1pmP2rRCPMK4umlTFUDnsd8b2/2LtvLf+247EybQuZ6O5UlDF14u0u h5gvCGZVrtuK9cZSAQFHDRTu7lfU+yhox1yUvQEo5APegVWUdjG8q0H33u9TWeEdMdYfY15rBWeHJ IqtmSZ1rKTgLCTegkNiByqGftBdKcSF/gPLlSvb+E9bUlqW4QQOfMll7BXgkvLNYvjzliYx/qmbZA P8KxbEDPrYFttQ+IVisEa0mqKPSuO6B6XRBO96tib2VU3N4/vKlXNqfMPlhxsksbTcA3bbWOstijm 6V8WtQuFA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXOv-00033B-1X; Thu, 05 Nov 2020 05:01:33 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3r-000145-TG for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=mRk4x8ffqejU12g7nFdHunjF+UNrFXF8Jgp9G1eFhMI=; b=p67rrdlXvmTq2/KdqA5fnzXskn 2vZLbV4o8E5vW/wp14TYd7/F7g2PYcaPseDwt4r7I2wXN8if76mym1xtxWvbMTALIAMYzloMu5chK 5dd5dLCS7Z1vQrpOXtLWCLpI30ofvnPABBKMEbaNBH4otd2ph18Esbd5kCbeZiHhzP7H7OzYKWfVC cXsZEDnHrtmSpYGs9DmKxrLFwGSE0WT/8BACSrmJYTSuE6aT5oZLpegmYtjtIdb7TalZDenaVoMi6 MDcJZsqrLozb2nbN8w4ju3HJ9gP1uEF7S4G/50PjP4T0B2tDo7sC+eCXW+0rcChkz4epQBWujbAVq /wv/EE8g==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSkv-0000q3-Lb for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:00 +0000 Received: by mail-wr1-x449.google.com with SMTP id w1so99597wrr.5 for ; Wed, 04 Nov 2020 16:03:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=mRk4x8ffqejU12g7nFdHunjF+UNrFXF8Jgp9G1eFhMI=; b=cMYeDNAIEtLr1J3RVSdvrmgQMONBQrVYYerYkXCetvmGBA2xUeYAgNtkEzxajLT3CI Bh1DopQkey4TaU1/5/i1N7npPVtiA/6+48OP7fv9WSaHFiK0vGvalq1IIyODIovLziQ6 zMS4w9QJ63hGFrVjV3+FF8tTat/0fKVwSKWNijqQLrijIlcvDOkez4BWmkgXGFA9J0aB 6Z7OHg1UfOyKJi6jTLQEvCR+dVhIDQbvKXe/QHUgcqVkfITeCc8cvOBbeOhVboFi7Xqz K+At9OBj/xD9fE95zOkrKIEfvez3xnemsuQTm53Fi0esSdQzKOjhX1YcdHiNvLB6WEkw qHUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=mRk4x8ffqejU12g7nFdHunjF+UNrFXF8Jgp9G1eFhMI=; b=kbmVOAoHVWMZFo5r6bF+DrJ9jjiV4rhR3wz69PPNf9dlEpnuFw0DE9yZ7AK12N8Jut 2QrjiDIuSgamPI6aV3WIvXNuBYJvQIURunYU292U0tjok+SCPG0vGl99YObW6PwAXQLP M2drgM8pZCPSH3Xi/+03nq6/hv9+icLQxcKtMXyXLp7PjragYffZNX9vdmSnuXyu4gwC hpCZUpLRqAyMN5rDvJPCrSpm58FXKaaWeZl1Rm9JOsg9l/G6hFPoSMFj1hUGaNvy4L+m h/QVbTvl1HLPksA/ydJ+8fCuPdxgnNbGjoXiF6AJgMFzfawYYhhPwPvh4Z4CcdqXYTNm 2keg== X-Gm-Message-State: AOAM530arKtcyZzEZZkd3rOcDxrmrk4aSUiYqJPXK5F2xhs57UunrVGQ q1LigMMtGIwbu77cTfByaXe7LEi+pSsOhRU+ X-Google-Smtp-Source: ABdhPJxaB9bS1n4FopLulDs5jJOJUms2oIgnCPMGCHt0rWTOkeMjdf4qr41+n+qwT4dQ3uLhsi4LP6SGG6zkwG0L X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:6000:36f:: with SMTP id f15mr532689wrf.78.1604534574449; Wed, 04 Nov 2020 16:02:54 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:18 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 08/20] kasan: inline random_tag for HW_TAGS From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000357_869192_68611188 X-CRM114-Status: GOOD ( 12.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Using random_tag() currently results in a function call. Move its definition to mm/kasan/kasan.h and turn it into a static inline function for hardware tag-based mode to avoid uneeded function calls. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Iac5b2faf9a912900e16cca6834d621f5d4abf427 --- mm/kasan/hw_tags.c | 5 ----- mm/kasan/kasan.h | 34 +++++++++++++++++----------------- 2 files changed, 17 insertions(+), 22 deletions(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index fe8e6c8e6319..d5824530fd15 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -38,11 +38,6 @@ void kasan_unpoison_memory(const void *address, size_t size) round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } -u8 random_tag(void) -{ - return hw_get_random_tag(); -} - bool check_invalid_free(void *addr) { u8 ptr_tag = get_tag(addr); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index e9c7d061fbe5..d7a03eab5814 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -188,6 +188,12 @@ static inline bool addr_has_metadata(const void *addr) #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) +void print_tags(u8 addr_tag, const void *addr); +#else +static inline void print_tags(u8 addr_tag, const void *addr) { } +#endif + bool check_invalid_free(void *addr); void *find_first_bad_addr(void *addr, size_t size); @@ -223,23 +229,6 @@ static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif -#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) - -void print_tags(u8 addr_tag, const void *addr); - -u8 random_tag(void); - -#else - -static inline void print_tags(u8 addr_tag, const void *addr) { } - -static inline u8 random_tag(void) -{ - return 0; -} - -#endif - #ifndef arch_kasan_set_tag static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) { @@ -275,6 +264,17 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #endif /* CONFIG_KASAN_HW_TAGS */ +#ifdef CONFIG_KASAN_SW_TAGS +u8 random_tag(void); +#elif defined(CONFIG_KASAN_HW_TAGS) +#define random_tag() hw_get_random_tag() +#else +static inline u8 random_tag(void) +{ + return 0; +} +#endif + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. From patchwork Thu Nov 5 00:02:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67229C00A89 for ; Thu, 5 Nov 2020 05:16:07 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E223C2083B for ; Thu, 5 Nov 2020 05:16:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="LxkULcBW"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="T1Qe8AH1"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="DWEvy7mM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E223C2083B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=q6KLPBcKUHPYR8Gx5ufaJ402QCKzjSG6/CaFneiAM20=; b=LxkULcBWvaHsYuZZWW2kwtwhD Oyl7vpoQ+XvZs2k/NHwqNuRMXWz/2lGV3hz1f+/FnRydeymO8dTrerzMbhMSK8Ob5HBEloIGsh084 JKKoO7xufE4BbZPxYT76Hg9uR0Nsa2f6oERuIMjaE4MP5BFTsAEo7LfdgDrvAnxVLnE/fWu/hsALy 0q2oLgwSWCJtyff31AzMxx8R9qCBs5CW5YgMapUdVY8vtYssQMV0j3lGLtmUy8zsh1YNM9NQ1lGzj z9UgybZ+mCWk97Rof5I04YugWbIhWWPcIdzh7bKXgi/ChJJZmnzSJhE0UkmUVzICiNCX5p/BuS9Yo ufIumfv0A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXcC-0000dS-Ee; Thu, 05 Nov 2020 05:15:16 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3q-00016k-Pw for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=TVZ5KfDqV4sczuDDUV9gNsUWnBSctv8RhcIj1oMqFVo=; b=T1Qe8AH1OMF1S6rzPVhKcJK0/J R8PXYGTlf93trtIZ38ZiuM+MNfE8gLn5K74JFE+SRyHpB5Efcf53FZbbuCwgU3YIAnfW4LkfyOfSz Fykj9fL3qQNAufz/4jqRtCGEhlphpns0JTJES/jjFZ4B9qtbiRRrE6FaL0NmE2SyezcipfPXDF8+U kC1RjNnfoXbgKCZoXds+GVETqpD9XRSUzA8xXwybwQuX4pASTiwrXMYlS45pMDBN3bPUKxEyb07Xs OQgo8VY2xqddNXseT3sQXf8HPwKMiIC25/DdIg9RbqNYEiwgA+voST3tJgybetUpf215gL6GaI2Pp t5ZIVZtQ==; Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSky-0000qO-IM for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:03 +0000 Received: by mail-wm1-x34a.google.com with SMTP id c10so469wmh.6 for ; Wed, 04 Nov 2020 16:04:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=TVZ5KfDqV4sczuDDUV9gNsUWnBSctv8RhcIj1oMqFVo=; b=DWEvy7mMEfr+/axjvw6t2VjDZPS+lO0xs+0DiZYZF+nJ/ZsK5Vhqm1kX0FdXvUY7yG 7jCjsxmBJd4av9dmmVxOp0gRjRs2QCY5L3otIaMyH7pQ4nKoFxVscyQ9krNdoXU0alvh DhuBuZL5pei0BWn8T0hoN4WkbQNcs3/Rb0i/xpksyeduhrc5PXjjCgF0P/3A4mmuVE7V Gazmv0f/MfckBnt0ufqq6PUfKCFQTkU73BJsVkhUSWBmR1xBSsc52x/QycRA9GeNSXQZ hS0WV28RuFuQLkl6WpqfqAT0/gIyIKYt0um0SMwnhJGYhi5Qc34vqM0lhPtMDl383/Em hqXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=TVZ5KfDqV4sczuDDUV9gNsUWnBSctv8RhcIj1oMqFVo=; b=Z7E3tXejE8p106Ue+uUissszpndtmTkRpCDqP0UJOapEma5LEPTyBojxdtrqGRITIY am/K5b8iV9A1cX+qdTb59wwgHcXIAnezKTlJw3HUV1KrJcKoh99Y1hCb+jPfGtOmJ8rS u+07aTOX6K3octYmDehT+8QqgPtxv+Nl95Ddo9F8eDU7Yfy+5C/zCU2f/fZrhpewOd6K VE34p9jYaRjFFpn904GLfo62YzidrbG8MFm0/6FRPjiIzgVAGFw3c/Stw2ZfAjM00VZf B56H36/KJWR/5GONi0RlbvWGGWVb0G/1E263L4aQhpLUu3Dyb7COJebJSt5nrbyBa1df xdIw== X-Gm-Message-State: AOAM5309p21+BmCnugYynR4Jg+8qR+vZYg8XIR9jpWJ0yNmZ8SWSAlfZ fYpoZmIJ0LMdnV6fGyc5X5yeEANthWultVAs X-Google-Smtp-Source: ABdhPJzWht9sgT8VGHbs3poE0UdD+kavb+Z/NCfdyaj2htoI0/JLAcuZvn5uAHlVsbj+IJFNKhvHkMoibdhWjIwx X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:4d05:: with SMTP id o5mr23559wmh.94.1604534576841; Wed, 04 Nov 2020 16:02:56 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:19 +0100 In-Reply-To: Message-Id: <7a831f5b5876f468545d637775d5440d49d31400.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 09/20] kasan: inline kasan_poison_memory and check_invalid_free From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000400_754550_9B7E15A5 X-CRM114-Status: GOOD ( 11.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Using kasan_poison_memory() or check_invalid_free() currently results in function calls. Move their definitions to mm/kasan/kasan.h and turn them into static inline functions for hardware tag-based mode to avoid unneeded function calls. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Ia9d8191024a12d1374675b3d27197f10193f50bb --- mm/kasan/hw_tags.c | 15 --------------- mm/kasan/kasan.h | 28 ++++++++++++++++++++++++---- 2 files changed, 24 insertions(+), 19 deletions(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index d5824530fd15..9d7b1f1a2553 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -26,27 +26,12 @@ void kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void kasan_poison_memory(const void *address, size_t size, u8 value) -{ - hw_set_mem_tag_range(kasan_reset_tag(address), - round_up(size, KASAN_GRANULE_SIZE), value); -} - void kasan_unpoison_memory(const void *address, size_t size) { hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } -bool check_invalid_free(void *addr) -{ - u8 ptr_tag = get_tag(addr); - u8 mem_tag = hw_get_mem_tag(addr); - - return (mem_tag == KASAN_TAG_INVALID) || - (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); -} - void kasan_set_free_info(struct kmem_cache *cache, void *object, u8 tag) { diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index d7a03eab5814..73364acf6ec8 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -153,8 +153,6 @@ struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object); -void kasan_poison_memory(const void *address, size_t size, u8 value); - #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) static inline const void *kasan_shadow_to_mem(const void *shadow_addr) @@ -194,8 +192,6 @@ void print_tags(u8 addr_tag, const void *addr); static inline void print_tags(u8 addr_tag, const void *addr) { } #endif -bool check_invalid_free(void *addr); - void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); void metadata_fetch_row(char *buffer, void *row); @@ -275,6 +271,30 @@ static inline u8 random_tag(void) } #endif +#ifdef CONFIG_KASAN_HW_TAGS + +static inline void kasan_poison_memory(const void *address, size_t size, u8 value) +{ + hw_set_mem_tag_range(kasan_reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), value); +} + +static inline bool check_invalid_free(void *addr) +{ + u8 ptr_tag = get_tag(addr); + u8 mem_tag = hw_get_mem_tag(addr); + + return (mem_tag == KASAN_TAG_INVALID) || + (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); +} + +#else /* CONFIG_KASAN_HW_TAGS */ + +void kasan_poison_memory(const void *address, size_t size, u8 value); +bool check_invalid_free(void *addr); + +#endif /* CONFIG_KASAN_HW_TAGS */ + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. From patchwork Thu Nov 5 00:02:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32929C00A89 for ; Thu, 5 Nov 2020 05:15:27 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 713672087D for ; Thu, 5 Nov 2020 05:15:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="i1Pot2wK"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="bqex/nfZ"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="iH4SUzBB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 713672087D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=T2chLS5lPcZZl31ZrByDOB3oZNfiZlcog7aXnS+f6ko=; b=i1Pot2wKjExB1+h2v4kVCe8K+ llonthDdcEgEkMryUBD+/Cne/EPUEKyo55Pex1lsGHnVUckHIa414OnnmVUEPejltedUqJJYWUHUH N6jHMeiGAVjBSP90XDd4KnqZdYWcQK3k/vsztfWmaPWzCxxe1IG7HxsgkVz0avQuUTV49QKRp0ZvR QYv+ncdehlZfKZR7IxgB6gsC1gm53EgLOMrstAwOne63YEeQYGU7AFIfmxUNA2q+Hj4ZdMsxIOVtV hk9oeKEWZyIf9yDG28rNuHqReQGjit/9yW4V0rOFpulPwkEerND0p07C9cnKXWN3eXlEfYAP8hBUm lkdiwv4OQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXbF-0000CG-LV; Thu, 05 Nov 2020 05:14:17 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3q-000145-Pq for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=IR+L7pzOJ3S8sSm3PMiXuYbX9hdb+iZBoVgJRWY7cXM=; b=bqex/nfZ6YOWUESMRC6yZUeuwD W8xo1Ki4WVloh4q5ELwl+fvZnEFinmfsJrbjBWGfgKWY/zuy3dKAuCJopZ22kQIVrPpNjH0/y+pAV gwlg1e8q/+gf5Lnv6azo0QQvXlN7KywhffN5XIN/KGqrPqwDXWItDeC1fzNi4Y3jab9EY5wa9mt7t yyzuLbOoZoGrETLVj6bZ5wtbQsE/ulyaue1Jir2lhr0iauRgHoofAG90Wfkoj5EHA97i+bEFDjK4/ t8oRY/IZUUsCTJCqjpz1mWe0xVizgEYosEwwXhdFzlM7mz52/kWS6HEta/8I6MQymTDSZqsD2pOTg PDCu+kOQ==; Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSky-0000qN-LE for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:03 +0000 Received: by mail-wm1-x349.google.com with SMTP id y1so2259wma.5 for ; Wed, 04 Nov 2020 16:04:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=IR+L7pzOJ3S8sSm3PMiXuYbX9hdb+iZBoVgJRWY7cXM=; b=iH4SUzBBRR5CmezCpCssl96xuxWU5rYOC5p49knORlt/DlKkf4bUYRgUYkQfHbyqzI Wjp3S6zP0hsWuTRlT3kZrfLV35zK4lCv78SBodcqjwFi0y5k1iYJU4rPz7GCGMl8l2Yf Xgpd8Ec4O9XN/osspKXmsXrdmeI+tsrmxx2qHfYA1mRMz7vR/7y6s+EJU7woeySmHJ+x R75NynG563yeZ+l2+qkvA4ZWEKnIBqPguaJjoKaxkseImgY7sUuOO0WaupodY4v8b7l4 L0A5JKrS9qa5u95L3dpVvEIUM5i4CDf4fEr8Jba3GgECYdL8KdeUeNrUvFq6f/0Bkw5D aJQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=IR+L7pzOJ3S8sSm3PMiXuYbX9hdb+iZBoVgJRWY7cXM=; b=Sas9m5F2UeoTLfAe25FDH5piPLAMYLBAK8lZMnhb1ix5zEOCDdHhctLU6hWZuTdkAr XhH7jwFjsvvbq0bhY4oQtn9dwSa2BTyCWBQOaYBRa4wXyPfrPNFzIawD6pXYEfL+fCZq 0U9q6NiyFD7IR/BnCl2OYIBYZPBbT/GReTX+dzd6a5sdbZGOahYo4Jt9RaiIhWKB1YKc 9en1MuNO8YNURI1oRl85s/o23OTp+34U826q5ZvwkZKkWZ3D1tj6HLd/WPpjSL3zzqGW SN5OIMefFheUQpt4CnhpKcRy9c2/lLyuhVbY0N2T7BkorQVuzkzH2pyrHZ3mmShNYxj/ gbcw== X-Gm-Message-State: AOAM530Z1YeQcZ+c8Swgmsw/WJtAgoUZG3N0bhdMQ4H3r7oUETtiV57l c2YkneAz2jarBTh0FodLlaY4w9jUvPc7yfyR X-Google-Smtp-Source: ABdhPJziGq/j8udcryfhR03OaOpym1lVl1to/3Mi30OCnUP7ip/1hRB5UF2s45H8QUldbWJDkwFxs5TLqGB7SkVw X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:bc08:: with SMTP id m8mr185656wmf.137.1604534579112; Wed, 04 Nov 2020 16:02:59 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:20 +0100 In-Reply-To: Message-Id: <82219b5988592173ba4fbf07abcb7009e3d7265a.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 10/20] kasan: inline and rename kasan_unpoison_memory From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000400_855167_3540EF6E X-CRM114-Status: GOOD ( 16.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently kasan_unpoison_memory() is used as both an external annotation and as an internal memory poisoning helper. Rename external annotation to kasan_unpoison_data() and inline the internal helper for hardware tag-based mode to avoid undeeded function calls. There's the external annotation kasan_unpoison_slab() that is currently defined as static inline and uses kasan_unpoison_memory(). With this change it's turned into a function call. Overall, this results in the same number of calls for hardware tag-based mode as kasan_unpoison_memory() is now inlined. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ia7c8b659f79209935cbaab3913bf7f082cc43a0e --- include/linux/kasan.h | 16 ++++++---------- kernel/fork.c | 2 +- mm/kasan/common.c | 10 ++++++++++ mm/kasan/hw_tags.c | 6 ------ mm/kasan/kasan.h | 7 +++++++ mm/slab_common.c | 2 +- 6 files changed, 25 insertions(+), 18 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 0211a4ec5d87..34236f134472 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -72,14 +72,15 @@ static inline void kasan_disable_current(void) {} #ifdef CONFIG_KASAN -void kasan_unpoison_memory(const void *address, size_t size); - void kasan_alloc_pages(struct page *page, unsigned int order); void kasan_free_pages(struct page *page, unsigned int order); void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags); +void kasan_unpoison_data(const void *address, size_t size); +void kasan_unpoison_slab(const void *ptr); + void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); @@ -104,11 +105,6 @@ struct kasan_cache { int free_meta_offset; }; -size_t __ksize(const void *); -static inline void kasan_unpoison_slab(const void *ptr) -{ - kasan_unpoison_memory(ptr, __ksize(ptr)); -} size_t kasan_metadata_size(struct kmem_cache *cache); bool kasan_save_enable_multi_shot(void); @@ -116,8 +112,6 @@ void kasan_restore_multi_shot(bool enabled); #else /* CONFIG_KASAN */ -static inline void kasan_unpoison_memory(const void *address, size_t size) {} - static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} @@ -125,6 +119,9 @@ static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} +static inline void kasan_unpoison_data(const void *address, size_t size) { } +static inline void kasan_unpoison_slab(const void *ptr) { } + static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} @@ -164,7 +161,6 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object, return false; } -static inline void kasan_unpoison_slab(const void *ptr) { } static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ diff --git a/kernel/fork.c b/kernel/fork.c index 463ef51f2b05..d6ff6b5650aa 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -226,7 +226,7 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node) continue; /* Mark stack accessible for KASAN. */ - kasan_unpoison_memory(s->addr, THREAD_SIZE); + kasan_unpoison_data(s->addr, THREAD_SIZE); /* Clear stale pointers from reused stack. */ memset(s->addr, 0, THREAD_SIZE); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a266b90636a1..4598c1364f19 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -184,6 +184,16 @@ struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } +void kasan_unpoison_data(const void *address, size_t size) +{ + kasan_unpoison_memory(address, size); +} + +void kasan_unpoison_slab(const void *ptr) +{ + kasan_unpoison_memory(ptr, __ksize(ptr)); +} + void kasan_poison_slab(struct page *page) { unsigned long i; diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 9d7b1f1a2553..bd8bf05c8034 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -26,12 +26,6 @@ void kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void kasan_unpoison_memory(const void *address, size_t size) -{ - hw_set_mem_tag_range(kasan_reset_tag(address), - round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); -} - void kasan_set_free_info(struct kmem_cache *cache, void *object, u8 tag) { diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 73364acf6ec8..ba850285a360 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -279,6 +279,12 @@ static inline void kasan_poison_memory(const void *address, size_t size, u8 valu round_up(size, KASAN_GRANULE_SIZE), value); } +static inline void kasan_unpoison_memory(const void *address, size_t size) +{ + hw_set_mem_tag_range(kasan_reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); +} + static inline bool check_invalid_free(void *addr) { u8 ptr_tag = get_tag(addr); @@ -291,6 +297,7 @@ static inline bool check_invalid_free(void *addr) #else /* CONFIG_KASAN_HW_TAGS */ void kasan_poison_memory(const void *address, size_t size, u8 value); +void kasan_unpoison_memory(const void *address, size_t size); bool check_invalid_free(void *addr); #endif /* CONFIG_KASAN_HW_TAGS */ diff --git a/mm/slab_common.c b/mm/slab_common.c index 53d0f8bb57ea..f1b0c4a22f08 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1176,7 +1176,7 @@ size_t ksize(const void *objp) * We assume that ksize callers could use whole allocated area, * so we need to unpoison this area. */ - kasan_unpoison_memory(objp, size); + kasan_unpoison_data(objp, size); return size; } EXPORT_SYMBOL(ksize); From patchwork Thu Nov 5 00:02:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FE42C00A89 for ; Thu, 5 Nov 2020 05:29:14 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E77952083B for ; Thu, 5 Nov 2020 05:29:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="sYezTeYK"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="jJiJE+d8"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="MX4puu5v" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E77952083B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=6LfuQuW4erGsuTde37GIgbTfP1pPcjagK4kUvqRsrPw=; b=sYezTeYK5VSAz4fp7si4IlXlW u8B6HIVOez8aSwUfNozHAdBAjASJh+m8j2eoTDWgNeFFfJ2Jy+GhvKmP5dr93Ph/lvGC6JznAX8Kr MdpKs/ZDJvm1nLCKEk+M6EmjszFTuhCYaggpA75qOTLLjkpa59vUVtFbVwy+T4L80xltAfVNzj4Cy AFB8PVo/nNct1aJ7zrcvXqaHyWcKTemkO2KKnSlXC2DtzaTRtObdSCRp/kgyH2cwwSIp8nTo9pC+3 XOhqKjpoSofLCYsW4P5WrdplbfJD0IX+PhAzkz/PL9eR1E5bysJwAKV8+Dk9Pt48crEJtwQNOK+il UIfHCVP7A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXng-0006ED-O3; Thu, 05 Nov 2020 05:27:08 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3p-00016k-Jh for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=YHPz8JrRkO/XRi71L68uJnMTHNWIIJmflnYXoYMB1HU=; b=jJiJE+d8+1gU0/MiKh0ta8/07c ncH2jIvAHQ4MxCpC7PwNkIjTegEDIQWHwrxejbbhDvbvZYDdWi5h3NFDLbSAkXX1Y7szsKLHooMm9 JAFJ0aIeCBY0qrVtPIx5o0XcQZp4qk1IPKCt/Mc8ma9e2QxG+yTAXkO5i3STpw+x19oGwZ5/frAlS Ah7gEwiJLHaKloCDYOSkTPUGm/9yrpE0HS2qWc5FmfY3YIDuJalffu/R/aSzdc6zwM3uVhExxHRO4 37Z6X0Q2ncWNz8+6zxUkgw0Y6+RNZRRhnBUV2QFxAOD9l5YRULdeZtHB89S8MfMc+xXP11yNuqdwy 032Hb1WA==; Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSl2-0000qu-1h for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:07 +0000 Received: by mail-qk1-x74a.google.com with SMTP id z28so45614qkj.4 for ; Wed, 04 Nov 2020 16:04:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=YHPz8JrRkO/XRi71L68uJnMTHNWIIJmflnYXoYMB1HU=; b=MX4puu5vY5ArnM06rFh+Cesd3ekV0Pt8TXclcxkRsmn9ZJcjqSxu5S4dY6abyKf7/d fPBWZ1NcRQyBMo8+o0SGloqmEoVjT24NRRNqMHykXQ2LwZSyF77DyOl1BHVE8V5D64QO pa51EuoTzpt+vlTQKxImAS83FZCb82pvJk7wtJnQvTmg5D6jKcX7oJTm6QURngCsnwQM vaqaEgbJlp+et0wpzHH0o5KK4L3N3fk2igBMo8GHqoxXk2KQ6GrEJZ8msKygxFZIo7/0 56U1QvIKMOKf6pjQoDsq7IKEEtfN2EXrVv1VoEl6xXsp8JVNVJ8WpnPI98b6peFxEftd md1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=YHPz8JrRkO/XRi71L68uJnMTHNWIIJmflnYXoYMB1HU=; b=PoHca7Qr5C7JRjMJnvxTxCBVH0WjX5flknvljCo2ufiJRaH8tehkgdpYfQq2dpUz0i nWAjX4voN6DmnJ+yC/1Ou6VHuqsff1Jg9Qizm/hjGOj0p9dnCNO+V1r2h77c7Ol6M214 Vde9fg1Qcv5A+sSg8/T99eqBQwJK36bvOOuUEUKKANGhsSJbG3mPW1EUQbWoZm9nI3ry BABRT520gEvs1wUlSjo1Afz70iU5h+jNLsCpcz+uGmD5ZeuQTj8NJgui/oQbW6R2HBph kHQDoQSURL6OXHO91+/W2IY/zvRPU1vkR+tf4UMnsUWjiYIpID4vMbtxJtG18P031tBR 2gxA== X-Gm-Message-State: AOAM5307PVP33D4ofP4RRGUXsFbErbLc+pnx384YRgSzZHvb6lL9Gr+k TRovhIFWo6Tp/nUlGxBANWHf8Sx/40QzsOIx X-Google-Smtp-Source: ABdhPJyy9dYPH53G7mNH8StH+XA61fxu2VluKrIUzwGUzcdlLu3J13fpNk0YwaAKhzuBscpovrcCh9MFsI99dfia X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:ad4:560d:: with SMTP id ca13mr435510qvb.2.1604534581496; Wed, 04 Nov 2020 16:03:01 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:21 +0100 In-Reply-To: Message-Id: <050977b6a6e0baee4afb4e701b600af32ee85ee6.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 11/20] kasan: add and integrate kasan boot parameters From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000404_254133_7BE7D622 X-CRM114-Status: GOOD ( 25.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hardware tag-based KASAN mode is intended to eventually be used in production as a security mitigation. Therefore there's a need for finer control over KASAN features and for an existence of a kill switch. This change adds a few boot parameters for hardware tag-based KASAN that allow to disable or otherwise control particular KASAN features. The features that can be controlled are: 1. Whether KASAN is enabled at all. 2. Whether KASAN collects and saves alloc/free stacks. 3. Whether KASAN panics on a detected bug or not. With this change a new boot parameter kasan.mode allows to choose one of three main modes: - kasan.mode=off - KASAN is disabled, no tag checks are performed - kasan.mode=prod - only essential production features are enabled - kasan.mode=full - all KASAN features are enabled The chosen mode provides default control values for the features mentioned above. However it's also possible to override the default values by providing: - kasan.stack=off/on - enable stacks collection (default: on for mode=full, otherwise off) - kasan.fault=report/panic - only report tag fault or also panic (default: report) If kasan.mode parameter is not provided, it defaults to full when CONFIG_DEBUG_KERNEL is enabled, and to prod otherwise. It is essential that switching between these modes doesn't require rebuilding the kernel with different configs, as this is required by the Android GKI (Generic Kernel Image) initiative [1]. [1] https://source.android.com/devices/architecture/kernel/generic-kernel-image Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/If7d37003875b2ed3e0935702c8015c223d6416a4 --- mm/kasan/common.c | 22 +++++-- mm/kasan/hw_tags.c | 144 +++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.h | 16 +++++ mm/kasan/report.c | 14 ++++- 4 files changed, 189 insertions(+), 7 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 4598c1364f19..efad5ed6a3bd 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -129,6 +129,11 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, unsigned int redzone_size; int redzone_adjust; + if (!kasan_stack_collection_enabled()) { + *flags |= SLAB_KASAN; + return; + } + /* Add alloc meta. */ cache->kasan_info.alloc_meta_offset = *size; *size += sizeof(struct kasan_alloc_meta); @@ -165,6 +170,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, size_t kasan_metadata_size(struct kmem_cache *cache) { + if (!kasan_stack_collection_enabled()) + return 0; return (cache->kasan_info.alloc_meta_offset ? sizeof(struct kasan_alloc_meta) : 0) + (cache->kasan_info.free_meta_offset ? @@ -267,11 +274,13 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; + if (kasan_stack_collection_enabled()) { + if (!(cache->flags & SLAB_KASAN)) + return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); + } if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -308,6 +317,9 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE); + if (!kasan_stack_collection_enabled()) + return false; + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || unlikely(!(cache->flags & SLAB_KASAN))) return false; @@ -355,7 +367,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (cache->flags & SLAB_KASAN) + if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN)) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index bd8bf05c8034..52984825c75f 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -8,6 +8,8 @@ #define pr_fmt(fmt) "kasan: " fmt +#include +#include #include #include #include @@ -17,11 +19,153 @@ #include "kasan.h" +enum kasan_arg_mode { + KASAN_ARG_MODE_DEFAULT, + KASAN_ARG_MODE_OFF, + KASAN_ARG_MODE_PROD, + KASAN_ARG_MODE_FULL, +}; + +enum kasan_arg_stacks { + KASAN_ARG_STACKS_DEFAULT, + KASAN_ARG_STACKS_OFF, + KASAN_ARG_STACKS_ON, +}; + +enum kasan_arg_fault { + KASAN_ARG_FAULT_DEFAULT, + KASAN_ARG_FAULT_REPORT, + KASAN_ARG_FAULT_PANIC, +}; + +static enum kasan_arg_mode kasan_arg_mode __ro_after_init; +static enum kasan_arg_stacks kasan_arg_stacks __ro_after_init; +static enum kasan_arg_fault kasan_arg_fault __ro_after_init; + +/* Whether KASAN is enabled at all. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_flag_enabled); +EXPORT_SYMBOL(kasan_flag_enabled); + +/* Whether to collect alloc/free stack traces. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_flag_stacks); + +/* Whether panic or disable tag checking on fault. */ +bool kasan_flag_panic __ro_after_init; + +/* kasan.mode=off/prod/full */ +static int __init early_kasan_mode(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_mode = KASAN_ARG_MODE_OFF; + else if (!strcmp(arg, "prod")) + kasan_arg_mode = KASAN_ARG_MODE_PROD; + else if (!strcmp(arg, "full")) + kasan_arg_mode = KASAN_ARG_MODE_FULL; + else + return -EINVAL; + + return 0; +} +early_param("kasan.mode", early_kasan_mode); + +/* kasan.stack=off/on */ +static int __init early_kasan_flag_stacks(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_stacks = KASAN_ARG_STACKS_OFF; + else if (!strcmp(arg, "on")) + kasan_arg_stacks = KASAN_ARG_STACKS_ON; + else + return -EINVAL; + + return 0; +} +early_param("kasan.stacks", early_kasan_flag_stacks); + +/* kasan.fault=report/panic */ +static int __init early_kasan_fault(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "report")) + kasan_arg_fault = KASAN_ARG_FAULT_REPORT; + else if (!strcmp(arg, "panic")) + kasan_arg_fault = KASAN_ARG_FAULT_PANIC; + else + return -EINVAL; + + return 0; +} +early_param("kasan.fault", early_kasan_fault); + /* kasan_init_hw_tags() is called for each CPU. */ void kasan_init_hw_tags(void) { + /* Choose KASAN mode if kasan boot parameter is not provided. */ + if (kasan_arg_mode == KASAN_ARG_MODE_DEFAULT) { + if (IS_ENABLED(CONFIG_DEBUG_KERNEL)) + kasan_arg_mode = KASAN_ARG_MODE_FULL; + else + kasan_arg_mode = KASAN_ARG_MODE_PROD; + } + + /* If KASAN isn't enabled, do nothing. */ + if (kasan_arg_mode == KASAN_ARG_MODE_OFF) + return; + + /* Only process the boot parameters on boot CPU. */ + if (smp_processor_id() == 0) { + /* Preset parameter values based on the mode. */ + switch (kasan_arg_mode) { + case KASAN_ARG_MODE_OFF: + return; + case KASAN_ARG_MODE_PROD: + static_branch_enable(&kasan_flag_enabled); + break; + case KASAN_ARG_MODE_FULL: + static_branch_enable(&kasan_flag_enabled); + static_branch_enable(&kasan_flag_stacks); + break; + default: + break; + } + + /* Now, optionally override the presets. */ + + switch (kasan_arg_stacks) { + case KASAN_ARG_STACKS_OFF: + static_branch_disable(&kasan_flag_stacks); + break; + case KASAN_ARG_STACKS_ON: + static_branch_enable(&kasan_flag_stacks); + break; + default: + break; + } + + switch (kasan_arg_fault) { + case KASAN_ARG_FAULT_REPORT: + kasan_flag_panic = false; + break; + case KASAN_ARG_FAULT_PANIC: + kasan_flag_panic = true; + break; + default: + break; + } + } + + /* Init tags for each CPU. */ hw_init_tags(KASAN_TAG_MAX); + /* Only print the message on boot CPU. */ if (smp_processor_id() == 0) pr_info("KernelAddressSanitizer initialized\n"); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index ba850285a360..8a4cd9618142 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -5,6 +5,22 @@ #include #include +#ifdef CONFIG_KASAN_HW_TAGS +#include +DECLARE_STATIC_KEY_FALSE(kasan_flag_stacks); +static inline bool kasan_stack_collection_enabled(void) +{ + return static_branch_unlikely(&kasan_flag_stacks); +} +#else +static inline bool kasan_stack_collection_enabled(void) +{ + return true; +} +#endif + +extern bool kasan_flag_panic __ro_after_init; + #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #else diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 25ca66c99e48..7d86af340148 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -99,6 +99,10 @@ static void end_report(unsigned long *flags) panic_on_warn = 0; panic("panic_on_warn set ...\n"); } +#ifdef CONFIG_KASAN_HW_TAGS + if (kasan_flag_panic) + panic("kasan.fault=panic set ...\n"); +#endif kasan_enable_current(); } @@ -161,8 +165,8 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, (void *)(object_addr + cache->object_size)); } -static void describe_object(struct kmem_cache *cache, void *object, - const void *addr, u8 tag) +static void describe_object_stacks(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) { struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); @@ -190,7 +194,13 @@ static void describe_object(struct kmem_cache *cache, void *object, } #endif } +} +static void describe_object(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) +{ + if (kasan_stack_collection_enabled()) + describe_object_stacks(cache, object, addr, tag); describe_object_addr(cache, object, addr); } From patchwork Thu Nov 5 00:02:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 325A4C00A89 for ; Thu, 5 Nov 2020 05:28:32 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 824DF2083B for ; Thu, 5 Nov 2020 05:28:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="1Tr2bdJS"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="OBUNL921"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="YYO+EYSi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 824DF2083B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=G/GYw4IATAFPGNxYPb5VjBHkZN26wJQMz2Bir211nAA=; b=1Tr2bdJSIAWVo8WO0IQDBrrRO vqeJsK1pu028YoGNQayd0lQftm35l0WpyjvdPJqax5a39vYHXtglXxFSnfMkoTS66h6wwVbx1TnV7 fDF50goAxa4XXsSKY1PnUEBZEe7K8on59+uRO8BvD3KmzTwyVTZ9C5IGeIoKbstxf6DNjIucDYIrr zmqTp7xLJDHzrszQVktYFDuxoN0tmXgEab0YR9keSpXAaxjo59IfcyWVrBVO+EfEp7NV9XbN59Kbj Qt+ht1J2FwQ2MULcRSbWavbRFbtLMuGesFLLETXd/aOO2iLJJDVIozI5+DhvtRAtEgDe9zkukcZVq FQNUqO4rQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXmr-0005or-PN; Thu, 05 Nov 2020 05:26:18 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3p-000145-GK for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=DKUUdpjKzvhCcD6WgylaP6PBqV8CAT8cUIeT7JoiXzA=; b=OBUNL921CRA+8hQ7xRdFkMRmby lgm2u2EQhbjDnPsS5BeUp39CadKpcKoJtMFWoYHNnm56KiAdHiT//PJWKPbbru+sIGrHsd0EO3r8j 92D2UAZo+soyqvqV7dIrPAYGF8gpIZsDVt71Nv8tlwra6ggOSq/vFD09JyxcTtXaZ8keWdc4Sgb92 tA2wSClSjec7p8H6iI2+zKx3IMyBIp6Nn3JRtJd6gp1ImQw+125MYqUEDmEXW0eIqZwFDtnF/8Ov+ Zd9xxOxGx4/aHTa5KNaWgO5M7RmaqjkTbt3Jsso8+qocVeNzw6EtADg+8VVRekotGm1lic0SV3E5A kE8tNp/w==; Received: from mail-ed1-x549.google.com ([2a00:1450:4864:20::549]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSl4-0000rE-NT for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:10 +0000 Received: by mail-ed1-x549.google.com with SMTP id bs10so34844edb.22 for ; Wed, 04 Nov 2020 16:04:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=DKUUdpjKzvhCcD6WgylaP6PBqV8CAT8cUIeT7JoiXzA=; b=YYO+EYSiaJ4twGy9EpxxBHKXOuR9KlXFe4cIaNXWeybPoBnqdJf0ObepEkZt9/3nqr XorjNTapwBd1Hi0Ay4/6IxlLU/mNzbKNCTwpyPwb69amCwDDOBtsWiGqX1jcIzes780S jzKY/cMEj5TJIU9YcUgElw4N5D5SaC38/eUqKFQOJedrsFRN111a14pDJxVK1dz9LFlO mlrTgLBPjldgQRgCFwRpmCw9EPeM7p80jlopEISYUMtRTzmnW87B3W66/grcDzFuJPrI q2aY8dS+pQqTgUW4B8NSd/i6q8IzDIJh6H1Gs2CqOKok1WSVbLmq/TDrAIfb/tEn/uHq gA5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=DKUUdpjKzvhCcD6WgylaP6PBqV8CAT8cUIeT7JoiXzA=; b=WveiEwX7zyxBynlI8J1p1yIl5HFxnso4YD83zmGUl8gy1p9wcFmT1qMZQXy0/CO/ra mKXHmQt/z1CMoG9LuiNZyD8WCbYnVsLr/F1+V+xvSxVkLyNq+hCjanY6lLKhpK6+4rG9 P1Lgby/oM7QWX3DzuLr8sypcgrEo7gRMU30PfJb2ahHmmZprqvf1sKkX5+dYFHUu7r6b Y5/b3znOClq3ZhjSz2KLuph2UIWxsl5q4h12dodubcrabV4AJuRo0QnIxss7tRm3fJlI YVvNYynStWWHaDFMXReDnp0wv+B5R5NNaopxBWgJP9PsHBNvD3SoiRI5IxiFDpB1QsfJ CtoA== X-Gm-Message-State: AOAM530tHNA2y0PKjROrjEIvsB3s1u4T8cjxZcQbh9l20ETTZCKqFNfA AYVfjA9aKhrP80uL2T4Kindzy6g424rZJnB/ X-Google-Smtp-Source: ABdhPJw9sKQRzZDY8K5Pn8mocA1Ivc0tkhgl0uN9rBcUneHik+B5q2nnAB7+jcQ+79C0iislPbJ5TmUzPcKEhMyu X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:6402:14cf:: with SMTP id f15mr356123edx.18.1604534583946; Wed, 04 Nov 2020 16:03:03 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:22 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 12/20] kasan, mm: check kasan_enabled in annotations From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000406_970414_FCBA37F6 X-CRM114-Status: GOOD ( 15.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Declare the kasan_enabled static key in include/linux/kasan.h and in include/linux/mm.h and check it in all kasan annotations. This allows to avoid any slowdown caused by function calls when kasan_enabled is disabled. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/I2589451d3c96c97abbcbf714baabe6161c6f153e --- include/linux/kasan.h | 220 ++++++++++++++++++++++++++++++++---------- include/linux/mm.h | 22 +++-- mm/kasan/common.c | 60 ++++++------ 3 files changed, 216 insertions(+), 86 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 34236f134472..ae1046fc74e5 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -2,6 +2,7 @@ #ifndef _LINUX_KASAN_H #define _LINUX_KASAN_H +#include #include struct kmem_cache; @@ -72,56 +73,179 @@ static inline void kasan_disable_current(void) {} #ifdef CONFIG_KASAN -void kasan_alloc_pages(struct page *page, unsigned int order); -void kasan_free_pages(struct page *page, unsigned int order); +struct kasan_cache { + int alloc_meta_offset; + int free_meta_offset; +}; -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags); +#ifdef CONFIG_KASAN_HW_TAGS +DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); +static inline kasan_enabled(void) +{ + return static_branch_likely(&kasan_flag_enabled); +} +#else +static inline kasan_enabled(void) +{ + return true; +} +#endif -void kasan_unpoison_data(const void *address, size_t size); -void kasan_unpoison_slab(const void *ptr); +void __kasan_alloc_pages(struct page *page, unsigned int order); +static inline void kasan_alloc_pages(struct page *page, unsigned int order) +{ + if (kasan_enabled()) + __kasan_alloc_pages(page, order); +} -void kasan_poison_slab(struct page *page); -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); -void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, - const void *object); +void __kasan_free_pages(struct page *page, unsigned int order); +static inline void kasan_free_pages(struct page *page, unsigned int order) +{ + if (kasan_enabled()) + __kasan_free_pages(page, order); +} -void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, - gfp_t flags); -void kasan_kfree_large(void *ptr, unsigned long ip); -void kasan_poison_kfree(void *ptr, unsigned long ip); -void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags); -void * __must_check kasan_krealloc(const void *object, size_t new_size, - gfp_t flags); +void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags); +static inline void kasan_cache_create(struct kmem_cache *cache, + unsigned int *size, slab_flags_t *flags) +{ + if (kasan_enabled()) + __kasan_cache_create(cache, size, flags); +} -void * __must_check kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags); -bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); +size_t __kasan_metadata_size(struct kmem_cache *cache); +static inline size_t kasan_metadata_size(struct kmem_cache *cache) +{ + if (kasan_enabled()) + return __kasan_metadata_size(cache); + return 0; +} -struct kasan_cache { - int alloc_meta_offset; - int free_meta_offset; -}; +void __kasan_unpoison_data(const void *addr, size_t size); +static inline void kasan_unpoison_data(const void *addr, size_t size) +{ + if (kasan_enabled()) + __kasan_unpoison_data(addr, size); +} + +void __kasan_unpoison_slab(const void *ptr); +static inline void kasan_unpoison_slab(const void *ptr) +{ + if (kasan_enabled()) + __kasan_unpoison_slab(ptr); +} + +void __kasan_poison_slab(struct page *page); +static inline void kasan_poison_slab(struct page *page) +{ + if (kasan_enabled()) + return __kasan_poison_slab(page); +} + +void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object); +static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +{ + if (kasan_enabled()) + return __kasan_unpoison_object_data(cache, object); +} + +void __kasan_poison_object_data(struct kmem_cache *cache, void *object); +static inline void kasan_poison_object_data(struct kmem_cache *cache, void *object) +{ + if (kasan_enabled()) + __kasan_poison_object_data(cache, object); +} + +void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, + const void *object); +static inline void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) +{ + if (kasan_enabled()) + return __kasan_init_slab_obj(cache, object); + return (void *)object; +} + +bool __kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); +static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip) +{ + if (kasan_enabled()) + return __kasan_slab_free(s, object, ip); + return false; +} + +void * __must_check __kasan_slab_alloc(struct kmem_cache *s, + void *object, gfp_t flags); +static inline void * __must_check kasan_slab_alloc(struct kmem_cache *s, + void *object, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_slab_alloc(s, object, flags); + return object; +} -size_t kasan_metadata_size(struct kmem_cache *cache); +void * __must_check __kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags); +static inline void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_kmalloc(s, object, size, flags); + return (void *)object; +} + +void * __must_check __kasan_kmalloc_large(const void *ptr, + size_t size, gfp_t flags); +static inline void * __must_check kasan_kmalloc_large(const void *ptr, + size_t size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_kmalloc_large(ptr, size, flags); + return (void *)ptr; +} + +void * __must_check __kasan_krealloc(const void *object, + size_t new_size, gfp_t flags); +static inline void * __must_check kasan_krealloc(const void *object, + size_t new_size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_krealloc(object, new_size, flags); + return (void *)object; +} + +void __kasan_poison_kfree(void *ptr, unsigned long ip); +static inline void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_poison_kfree(ptr, ip); +} + +void __kasan_kfree_large(void *ptr, unsigned long ip); +static inline void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_kfree_large(ptr, ip); +} bool kasan_save_enable_multi_shot(void); void kasan_restore_multi_shot(bool enabled); #else /* CONFIG_KASAN */ +static inline kasan_enabled(void) +{ + return false; +} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} - static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} - -static inline void kasan_unpoison_data(const void *address, size_t size) { } -static inline void kasan_unpoison_slab(const void *ptr) { } - +static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } +static inline void kasan_unpoison_data(const void *address, size_t size) {} +static inline void kasan_unpoison_slab(const void *ptr) {} static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} @@ -132,36 +256,32 @@ static inline void *kasan_init_slab_obj(struct kmem_cache *cache, { return (void *)object; } - -static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +static inline bool kasan_slab_free(struct kmem_cache *s, void *object, + unsigned long ip) { - return ptr; + return false; +} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; } -static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} -static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags) { return (void *)object; } +static inline void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + return (void *)ptr; +} static inline void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags) { return (void *)object; } - -static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) -{ - return object; -} -static inline bool kasan_slab_free(struct kmem_cache *s, void *object, - unsigned long ip) -{ - return false; -} - -static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } +static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} +static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} #endif /* CONFIG_KASAN */ diff --git a/include/linux/mm.h b/include/linux/mm.h index 0793d03a4183..8d84a6b2fa3c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -31,6 +31,7 @@ #include #include #include +#include struct mempolicy; struct anon_vma; @@ -1414,22 +1415,30 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) #endif /* CONFIG_NUMA_BALANCING */ #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) + static inline u8 page_kasan_tag(const struct page *page) { - return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; + if (kasan_enabled()) + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; + return 0xff; } static inline void page_kasan_tag_set(struct page *page, u8 tag) { - page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); - page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; + if (kasan_enabled()) { + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; + } } static inline void page_kasan_tag_reset(struct page *page) { - page_kasan_tag_set(page, 0xff); + if (kasan_enabled()) + page_kasan_tag_set(page, 0xff); } -#else + +#else /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ + static inline u8 page_kasan_tag(const struct page *page) { return 0xff; @@ -1437,7 +1446,8 @@ static inline u8 page_kasan_tag(const struct page *page) static inline void page_kasan_tag_set(struct page *page, u8 tag) { } static inline void page_kasan_tag_reset(struct page *page) { } -#endif + +#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ static inline struct zone *page_zone(const struct page *page) { diff --git a/mm/kasan/common.c b/mm/kasan/common.c index efad5ed6a3bd..385863eaec2c 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -81,7 +81,7 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ -void kasan_alloc_pages(struct page *page, unsigned int order) +void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; unsigned long i; @@ -95,7 +95,7 @@ void kasan_alloc_pages(struct page *page, unsigned int order) kasan_unpoison_memory(page_address(page), PAGE_SIZE << order); } -void kasan_free_pages(struct page *page, unsigned int order) +void __kasan_free_pages(struct page *page, unsigned int order) { if (likely(!PageHighMem(page))) kasan_poison_memory(page_address(page), @@ -122,8 +122,8 @@ static inline unsigned int optimal_redzone(unsigned int object_size) object_size <= (1 << 16) - 1024 ? 1024 : 2048; } -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) +void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) { unsigned int orig_size = *size; unsigned int redzone_size; @@ -168,7 +168,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *flags |= SLAB_KASAN; } -size_t kasan_metadata_size(struct kmem_cache *cache) +size_t __kasan_metadata_size(struct kmem_cache *cache) { if (!kasan_stack_collection_enabled()) return 0; @@ -191,17 +191,17 @@ struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } -void kasan_unpoison_data(const void *address, size_t size) +void __kasan_unpoison_data(const void *addr, size_t size) { - kasan_unpoison_memory(address, size); + kasan_unpoison_memory(addr, size); } -void kasan_unpoison_slab(const void *ptr) +void __kasan_unpoison_slab(const void *ptr) { kasan_unpoison_memory(ptr, __ksize(ptr)); } -void kasan_poison_slab(struct page *page) +void __kasan_poison_slab(struct page *page) { unsigned long i; @@ -211,12 +211,12 @@ void kasan_poison_slab(struct page *page) KASAN_KMALLOC_REDZONE); } -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object) { kasan_unpoison_memory(object, cache->object_size); } -void kasan_poison_object_data(struct kmem_cache *cache, void *object) +void __kasan_poison_object_data(struct kmem_cache *cache, void *object) { kasan_poison_memory(object, round_up(cache->object_size, KASAN_GRANULE_SIZE), @@ -269,7 +269,7 @@ static u8 assign_tag(struct kmem_cache *cache, const void *object, #endif } -void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, +void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_meta; @@ -288,7 +288,7 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, return (void *)object; } -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, +static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { u8 tag; @@ -331,9 +331,9 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, return IS_ENABLED(CONFIG_KASAN_GENERIC); } -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) { - return __kasan_slab_free(cache, object, ip, true); + return ____kasan_slab_free(cache, object, ip, true); } static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) @@ -341,7 +341,7 @@ static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); } -static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, +static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags, bool keep_tag) { unsigned long redzone_start; @@ -373,20 +373,20 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, return set_tag(object, tag); } -void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, - gfp_t flags) +void * __must_check __kasan_slab_alloc(struct kmem_cache *cache, + void *object, gfp_t flags) { - return __kasan_kmalloc(cache, object, cache->object_size, flags, false); + return ____kasan_kmalloc(cache, object, cache->object_size, flags, false); } -void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, - size_t size, gfp_t flags) +void * __must_check __kasan_kmalloc(struct kmem_cache *cache, const void *object, + size_t size, gfp_t flags) { - return __kasan_kmalloc(cache, object, size, flags, true); + return ____kasan_kmalloc(cache, object, size, flags, true); } -EXPORT_SYMBOL(kasan_kmalloc); +EXPORT_SYMBOL(__kasan_kmalloc); -void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, +void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; @@ -411,7 +411,7 @@ void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, return (void *)ptr; } -void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) +void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; @@ -421,13 +421,13 @@ void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); + return __kasan_kmalloc_large(object, size, flags); else - return __kasan_kmalloc(page->slab_cache, object, size, + return ____kasan_kmalloc(page->slab_cache, object, size, flags, true); } -void kasan_poison_kfree(void *ptr, unsigned long ip) +void __kasan_poison_kfree(void *ptr, unsigned long ip) { struct page *page; @@ -440,11 +440,11 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) } kasan_poison_memory(ptr, page_size(page), KASAN_FREE_PAGE); } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); + ____kasan_slab_free(page->slab_cache, ptr, ip, false); } } -void kasan_kfree_large(void *ptr, unsigned long ip) +void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); From patchwork Thu Nov 5 00:02:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25E0BC00A89 for ; Thu, 5 Nov 2020 03:38:05 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9DEBD2072E for ; Thu, 5 Nov 2020 03:38:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="EPuXzTTU"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="pdY+e1OM"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="U8jhak8M" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9DEBD2072E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=8guCawMFq4d5/5f4Z58iyQPrhBv2NM5OVWtDIr6F0oQ=; b=EPuXzTTUTxu7RtP/NxkGat7pt lghizcDEFVv1YJNSqCrOZxzr+9mq2RIEFy+un2Fp6FBpbEzVC9TkPV/pUiqCJ33G66PpTWlZSC/Ry 4TaDSUZiG9/qb0693XCyngTFKROdjOyorUiCz/IHcXF7DgXQz06R9i70BWYNbVEqexOqhqb4uy9wm 2jYo59PHAMPtgLGqMiKA25Y7K5zUSsHSmDX2gSgCKe8njPHvaKCZfATRYmyKe9fQxK2bwHVszV/wt L6mz3H9niunnLWYBBrW2SISAFdsJMDBHCxR+FywFbme5IicNfePUuK8o4QK8kqjQd+lMOekIm4a9X osBsd9hLQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW4t-0001zb-CP; Thu, 05 Nov 2020 03:36:47 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3o-000145-LO for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:40 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=BMZ/M+WrZmZXK8ABsLpLLUMO5suYjkxoKy6+yIMHKNI=; b=pdY+e1OMG/PxvUQ/iiFSUY4YTK smkVYR70ioqjneFMPc9A9ay5u0HVbKgc8opSaSzblrZ7Ta0eou8ZuqmNzcamGDHxaur3B2f1sAK3m esQrhL7ymPvqkX1NFdgXENV0EXW3IpOOocWRPtuUT+zO8iT3TA1d4y5CS5vU/sGRL1lQwKHBuHkDG NAIG/j2InPFHWO/Dwst4Hg08RFTc6iAOqKPMrkA6ul7483zEsYubpZcuMu8bZxgvH8IwVofQD5ypF hh4d9T0NSojTP+R/sG/rqJiCA9nucMSjrRJLDwq8SnVA8EfShEwLDfAFsxw7SmVSwMptI45ec51fA WZb9/nSg==; Received: from mail-ej1-x64a.google.com ([2a00:1450:4864:20::64a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSl8-0000rV-4I for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:13 +0000 Received: by mail-ej1-x64a.google.com with SMTP id nt22so92186ejb.17 for ; Wed, 04 Nov 2020 16:04:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=BMZ/M+WrZmZXK8ABsLpLLUMO5suYjkxoKy6+yIMHKNI=; b=U8jhak8MkkwTo4VVA2S4aMMc3rysJoG7VHeplDyb4fXuf1THbqycXNlTuCEhZq1ndK AhCyWSIEEOYjSrqyLHpxcdGv3yzV87WfK1+ghuNzSnULiKfdJVOxe6sUUQd7sxT+81Pm 6XhrLx6kwXsOktWPr/cM0ZinyJvTm5jvezSnFF3iKwjECgARv++K8S8XZ4/9MBv984h3 M3fq09VNCB+Bg+9rcUoAMcvxXA54/qhwAgrNEF7lkW3u5fOaa5RoV0kgdWx3fxww+s6M Ria7FGAAeatBPhxai7ZCgPgRfE1vFFHs7XGihw6Ux0UtxsTqIWhVPGuecdGyuf1tCe/W JiOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=BMZ/M+WrZmZXK8ABsLpLLUMO5suYjkxoKy6+yIMHKNI=; b=s22s2DAIFMSNhVgz++GZw6NX1BI6gBSkDdQGTOWd0nMGZ7u4AA+doyfp88R2PnQ3Vr ztH9ParNFmEJ2pnmsVyUwdpbWoJb//zTFYMqLL+N7GFohzA79AwLrk8Vd8uVv2JhB4To rGwlHyHvZFGW/9Bi2hAp/LzbDa8wMtA+j6zs0tWxbO/fb0eMO95bR+tAxd5uxlH+lPvf 6TD7wAB7N/lJFIGoOsf0fbxM6j1KyMTqSa91JCSj1m8PGedvGrFqbwUgNBhmh/mcXMbj k+e8jFR4sBHiJx0NutAmqKUUCY5lGHvT+6hmIQAqyDV5UyI+syI/yXQHiz2xglnv7Ydu Ekkg== X-Gm-Message-State: AOAM531W3fCh9kYTuNv73oDaRP+WBJpOqfapqKy9jqWEYesrHLdoeJgI Cy7sCIDjta3uH3amNYRYJD128d1AHHwi+nKH X-Google-Smtp-Source: ABdhPJweTaJkd6b3wl4vljsjzSaHEeYVHgfBL2uu8wiqY4d4DlmNQcw7W5IBJIWQpkZx+af/OCfJ+MPWnW+0Gr4h X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a17:906:134e:: with SMTP id x14mr575217ejb.173.1604534586349; Wed, 04 Nov 2020 16:03:06 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:23 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 13/20] kasan: simplify kasan_poison_kfree From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000410_307869_F3542FE2 X-CRM114-Status: GOOD ( 11.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kasan_poison_kfree() is currently only called for mempool allocations that are backed by either kmem_cache_alloc() or kmalloc(). Therefore, the page passed to kasan_poison_kfree() is always PageSlab() and there's no need to do the check. Remove it. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/If31f88726745da8744c6bea96fb32584e6c2778c --- mm/kasan/common.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 385863eaec2c..819403548f2e 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -432,16 +432,7 @@ void __kasan_poison_kfree(void *ptr, unsigned long ip) struct page *page; page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - kasan_poison_memory(ptr, page_size(page), KASAN_FREE_PAGE); - } else { - ____kasan_slab_free(page->slab_cache, ptr, ip, false); - } + ____kasan_slab_free(page->slab_cache, ptr, ip, false); } void __kasan_kfree_large(void *ptr, unsigned long ip) From patchwork Thu Nov 5 00:02:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8860C4741F for ; Thu, 5 Nov 2020 06:36:52 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 35D8E2151B for ; Thu, 5 Nov 2020 06:36:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="hZvwkIKs"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ChWFw5hM"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="sFpQqhhE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 35D8E2151B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SaDDqKyQP3rfYwGhaTS0zu8ZIgUu2RhSoGpmEq6sEkg=; b=hZvwkIKsDuSbcYSeE3p20bp2K 3A32TcEXiBGnMmf1faG2Dm++ZXsSzzInSGLN3HzFK6c5QouPxUvK7pySMuue1CvQf+2kNexVqGPvE Tn3cUj8wnlXmYUDQJfPgtmHJmwJbOWAPpMUx/5k3jUke1PWa/2uGxujy8CBiSHaV9tQhxHUQtglEW H+xaP03oOT21sIQ60foWaZrpaezkihNaAvXBLmCKPNO/2IOnNzMMFPpmllXCcxJ+pLhXlOohsmqCq 4mv93124cNLEcaPYa0t4SxCDONu8vak4xleMmjrzpaTPkjFs/FZNBQFiTmLOOS7S7hfA5UDY1LKPe bs5T1tjeA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaYs6-0007IK-Go; Thu, 05 Nov 2020 06:35:46 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3n-000145-MH for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0USIvWPWD+0X5uGT95blxzhIaksN6b9CBpQaBzMOnSY=; b=ChWFw5hMXgRJPz1Q0KIpMEfaB/ ofNL3WXpcpwXO+AnZfRqGFREioE9faJMt892rDGLzEGHmpeeIdMQxVuLJR0J7vLKEBAFlhQyyzAD0 BQRkqHWC+TnDfb0NDtGXJ9+H1WIkb2KZU0SK9PzWSDWNAnRUwWWRhU47gwMrQAqMJUnwqgayNbKDW sx+JR7ogdOojemI3A8RcAlDYST3gU4Jcvqd+ynPuvR+/erGC7cmW8TEZf6g9Z8PI+es8ZdeJ9766u Vsofm5OcFbnN7WZJ9lsCXLlrAbKynFOr5hLbiy2NTV9oZRp57azx1dd3Uzu+P4vsYQiwCCrAC5z2+ mDBUOOjw==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSlB-0000s3-Hm for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:16 +0000 Received: by mail-wr1-x449.google.com with SMTP id m4so82758wrq.23 for ; Wed, 04 Nov 2020 16:04:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=0USIvWPWD+0X5uGT95blxzhIaksN6b9CBpQaBzMOnSY=; b=sFpQqhhEG5gmFI9GQ8viL3pqjWr2ET4+vulUOd3uW9uXSre4REhA33qyv+t4dVSEU7 w545R8LNs86kAkwY0TXSEvxLnddxzfkbaplqkm43YTr/brE+bJoOY+C2aatYwHPrKd8V lyF0Dk29ZIFpFIJ1Og+2xmXpbHGti9zQJ89WmkxMG1/0OCal1u616x08x92MgB7fVD/D G69WQShQeI4m/Swv42Hyjt3WeTnoKf7SUs6+1sXJtxjv/UHA5ZAhdSY1gHBAtl08c45b vDih+OYOVaIk+p10CXSYtIivWlmHY49fZ1jflJr3KJySxpjBR7RHRbYJf7/j6E5/VucA 0Etg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=0USIvWPWD+0X5uGT95blxzhIaksN6b9CBpQaBzMOnSY=; b=ocAu2YXs/wkW3iHQ0phCv4HVz1X2Z7v3BhZWWL4qNeors5EeynxG1mZEv/u7/anRRN L3SONj6PVpzRw0Xd+1Nxmfum5h1b4vnu4vfIqlU3o9kM4ijyg3gH2Gdi8kYsVEmVUtjo QjoPkpS2YTgVS7PWm7ML8RrhjV9bsiQCbq/Bsb243Lrpm0Gpg53eNEVq5FUTzxf6ulHO ADOuBmPwRNVECGeWUSFKsIF9bN/qxakn2Kjd7fMAgd0YLlj8ogs6AUo3As9BC0tL3sbZ n5yGNC95fs6kpqaLDeJCaMX5GhPATZQ7Bh08baa2/VaDb6YLrasAi84fOgUeMrFtvQPQ ux8w== X-Gm-Message-State: AOAM532noBPqkJ+XdpIR90DmhGAcCcDlY0crUkTnNhZfMYEdjntyBqPU veVU6YgnsU7rV8vQ+ZmJoZq/UMwYbaeQizWs X-Google-Smtp-Source: ABdhPJxw/lGpcSbzl86AC+uSuIK+hwvLKzT3r8ACA6Gr0SlWmsWXhA12+uWjp+hh9NvD3Cd9/idF6XWVIRZbxQIj X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a5d:4c4f:: with SMTP id n15mr475161wrt.137.1604534591024; Wed, 04 Nov 2020 16:03:11 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:25 +0100 In-Reply-To: Message-Id: <3a3e6dfe3ad355bb5ffc3cc34769cb97aec650d2.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 15/20] kasan: don't round_up too much From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000413_741466_99989FE0 X-CRM114-Status: GOOD ( 11.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For hardware tag-based mode kasan_poison_memory() already rounds up the size. Do the same for software modes and remove round_up() from the common code. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/Ib397128fac6eba874008662b4964d65352db4aa4 --- mm/kasan/common.c | 8 ++------ mm/kasan/shadow.c | 1 + 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 60793f8695a8..69ab880abacc 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -218,9 +218,7 @@ void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object) void __kasan_poison_object_data(struct kmem_cache *cache, void *object) { - kasan_poison_memory(object, - round_up(cache->object_size, KASAN_GRANULE_SIZE), - KASAN_KMALLOC_REDZONE); + kasan_poison_memory(object, cache->object_size, KASAN_KMALLOC_REDZONE); } /* @@ -293,7 +291,6 @@ static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, { u8 tag; void *tagged_object; - unsigned long rounded_up_size; tag = get_tag(object); tagged_object = object; @@ -314,8 +311,7 @@ static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, return true; } - rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); - kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE); + kasan_poison_memory(object, cache->object_size, KASAN_KMALLOC_FREE); if (!kasan_stack_collection_enabled()) return false; diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 8e4fa9157a0b..3f64c9ecbcc0 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -82,6 +82,7 @@ void kasan_poison_memory(const void *address, size_t size, u8 value) * addresses to this function. */ address = kasan_reset_tag(address); + size = round_up(size, KASAN_GRANULE_SIZE); shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); From patchwork Thu Nov 5 00:02:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EEB5C00A89 for ; Thu, 5 Nov 2020 05:32:10 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CB1092083B for ; Thu, 5 Nov 2020 05:32:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="cQwmy7Gc"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="cBI1znwq"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="QLuAbvZH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB1092083B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=amqobPqYoUfeCoIDTbX5vy3jU+Ebtwlx48wbupNwRHw=; b=cQwmy7GcvwOQTK4kMMnzrYsc5 DiVxdYc+se/og3ZO3NB/NMLfLUsAZVdHy0QgCq3+11v8QerBXddmzHhX9IujvIyU+dRzJFWviNgDM sD8dRHRQiK++raxBkRebCanr+fOIOJj0gL4D4xCMTKFy/KAkxfSQP9X1PtMQUAxLyatbXJshdtCjT 7O6tWb0vBoXnxqf8KODB8fWXAXKOfo3M21hRJ3RaIqNh5VXO4RDwjYiX+YJOjPJBZ+WBNMU65UgTc +QCngyZsoAefCD/hk3BNDolqW2GwKR5j01Pn9UlTH5lUT3Icvgyuk+AkwlOeaMzLyDrt6hOa+i63K UM9CdPCog==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXqZ-0007dm-Nv; Thu, 05 Nov 2020 05:30:08 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3n-00016k-M6 for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=KFrv2CeYC66xd7QOOp7G/eCqjg0M9rtzwQxn1AQSEo8=; b=cBI1znwqBarY6ZZ2m3dr5g+C/4 2qfRjJ+uuT2wDcwImAAt0jZ9fEltR4ejJvpl/ZNWnaz74BSEfqjljln1I+RyXv5higu55SK1PHGr1 tsULmXOriiJpfJzHgkmxowTD0203A80pA1/ZiOF/0/Qsn6LVSLHE5uUjFg74BQXJuZY2Y0IiYOfM4 oJDQ3T2Tj0AHIh7WZSCmcsFfj314R5bTz2aEg6SW9wkpnCLsUN7WfiibWQOgHBXRcJlN7cvqn7N6q esMRw7hfUZ2Xqoz52EoSVC+MCbkljy7ZRZe5nli03Zt8A/E9cFjejVYrwIoP76/nm8Y0UKG00WNV9 6aGUh0UQ==; Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSlE-0000sO-3y for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:18 +0000 Received: by mail-wm1-x349.google.com with SMTP id y187so5842wmy.3 for ; Wed, 04 Nov 2020 16:04:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=KFrv2CeYC66xd7QOOp7G/eCqjg0M9rtzwQxn1AQSEo8=; b=QLuAbvZHFP2W58yhtR9vhBr55r8Hyf3S4Rs3PezTxrfLFhxyd7JT09PKnYS8oSsPPq gdRKQvVLLsai3OEzz4dyfxqzwtdGBwLP/JIK3nrPx65ohFOxXi8w41i17w7g8HswL0pM QsnWuFYb+DhqJ6xTCmfTf5pevVgWDcXouuFmLtrqO1Pnya8cbLcz2uUW+4+tBIDfCvWR NIBMGmdmtEOwmByJGW5M96WjPsZbCZh7VmiInUy7NkcWeP7l8Bovd9DsLTRHc6Tgd9yI lZqfbycXbtZhTw1t+934rBC373gyowgCXZJ0eV1iEKb+i/tEaSfzGY1H5Xkynl1uQ7FS 4S7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KFrv2CeYC66xd7QOOp7G/eCqjg0M9rtzwQxn1AQSEo8=; b=qA6SzM+9fxp1Iz2WBSfgZnJ4F+bM7eJH6JIURRhhiu+o2imk6N9M4uvPsJl64J6bkq OdH65pxWA3htchfCQ4gGY2yJvBmJZcEqODwEwHEhUL9/iQ6XcN5vl7GX5ToXhv/1fL3S f4VJuFa9g0B9+R18/ejarfjVMLk/f2astE1DYm8aAUyHRa8bXE1QrHd/F2MOhnC9hszj SmhguEr10PMYXoog/SCepoEhHwF7mJFtNOOGLdyTVFq+U5ZGIpjl45B8iRpWo0WWlyIb m0hJRtpIuYq4YtSz5qOYNGBnm5GlW0qHMDJfn6MErB2aQe4xvol5ypuI/ZYA6/40E5/p FV2Q== X-Gm-Message-State: AOAM530W1chffewOpjgmATFPoxnv9pEdNmNIot/iFluu/C0SF8RXZW6w m/TNbDuulQvnwo/vzFSGB2/UyJbxPxVM7uN5 X-Google-Smtp-Source: ABdhPJwQ27fD7Fc3ZNzuubZFZKWZfrYiXQ310vGkklvY3Ifyj2TvUAmaupP6a4qWeV/wlXVXw6iDgvs3qJM0Dpq1 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:9d02:: with SMTP id g2mr24703wme.110.1604534593196; Wed, 04 Nov 2020 16:03:13 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:26 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 16/20] kasan: simplify assign_tag and set_tag calls From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000416_464200_AF9F9749 X-CRM114-Status: GOOD ( 12.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org set_tag() already ignores the tag for the generic mode, so just call it as is. Add a check for the generic mode to assign_tag(), and simplify its call in ____kasan_kmalloc(). Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/I18905ca78fb4a3d60e1a34a4ca00247272480438 --- mm/kasan/common.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 69ab880abacc..40ff3ce07a76 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -238,6 +238,9 @@ void __kasan_poison_object_data(struct kmem_cache *cache, void *object) static u8 assign_tag(struct kmem_cache *cache, const void *object, bool init, bool keep_tag) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return 0xff; + /* * 1. When an object is kmalloc()'ed, two hooks are called: * kasan_slab_alloc() and kasan_kmalloc(). We assign the @@ -280,8 +283,8 @@ void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, __memset(alloc_meta, 0, sizeof(*alloc_meta)); } - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) - object = set_tag(object, assign_tag(cache, object, true, false)); + /* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */ + object = set_tag(object, assign_tag(cache, object, true, false)); return (void *)object; } @@ -362,9 +365,7 @@ static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, KASAN_GRANULE_SIZE); redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_GRANULE_SIZE); - - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) - tag = assign_tag(cache, object, false, keep_tag); + tag = assign_tag(cache, object, false, keep_tag); /* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */ kasan_unpoison_memory(set_tag(object, tag), size); From patchwork Thu Nov 5 00:02:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4A8EC00A89 for ; Thu, 5 Nov 2020 04:13:12 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0AAB420732 for ; Thu, 5 Nov 2020 04:13:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="tDyGRZfh"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="tQ3NnttN"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="iKpb0OkO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0AAB420732 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=30b4FgCn3Cz2rNkm82IkuRhgJ04HfEq0AfPKP3lEsyw=; b=tDyGRZfhU/zO9oMEVwhElSbVK cRLkcpxjwdGDJ7SDK4K7Ohr2nHQsfC07R2B0WNynbrMM0KAZkjGJUv2nkp/1kXtuETGagyXa9E8Rn xwkobIAQvF4DFwbamJSfG08mVwb/VSgq5iTv9aZX5prMJyR1o9U4So0WyHAb8IJB09USAsFfKwCz2 gp+oyprWwdzOe9DF9sOkC8/ELm5oZk8oWYTu2WiXJo66e8mRKl5EFEEb/MtPsG+m3fYtYZuG2kBFk FyYtNs58m0/2njiXRb9q+klc3GjaAyb63Tgyg6KG6YmVOAiyiF0Oh2yKiB59qN4WVbZZjgRYqA7ig QI5lM1egg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaWcg-0004Yk-FA; Thu, 05 Nov 2020 04:11:42 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3m-00016k-OT for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:38 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Z3EYeS+tIqBFVXo6hPJUkFm8u6eCOLhA7qycqAqlcHg=; b=tQ3NnttNJlHacGo0SyWJkU2VfP MuH4JIskhlzevg/x+LyFsaD42Girp/pM3gk3WkkHVe0N6Cb1z2LRc9YRPBW5IpyDrN7uSEMSQ05uR wUwfM+wl/Z7wUfIU7t4mEnUHGM0IFGPiGtlWH6ThoKzAeYicMsjKdS7HIaamsEw2WJOEi3TfIZaKn jj2aSOAJFoIEumiiAcjUQpph2NCEHExBklCdNcPUBSSvlc1EyC1GC4sWdcyKeuB7IFaItqYXRP88y fcWXiZVn/sFexwrDcB7tHqHWMv68VHB1CTY4e2LlNr1ulc9VQivttf5a57CI9HU3uIqW6qPbJi1zH UH/YWhlQ==; Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSlE-0000sQ-Uy for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:19 +0000 Received: by mail-qt1-x849.google.com with SMTP id t19so110282qta.21 for ; Wed, 04 Nov 2020 16:04:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=Z3EYeS+tIqBFVXo6hPJUkFm8u6eCOLhA7qycqAqlcHg=; b=iKpb0OkOxyAj1vbZYd8fEFbo90igQaXc+iJXc+9v0Cnyse6LjyC8e1nDAbQ8npvTyB 6c0yKUJnxJfUXo6LthJnmi4BtVKlRc9st0yW+p5e437FSeJIy1JNB38//QnbDQi8yvJD 8poDc5IXP7uVlrIMc6PADLpsNjP4ntVvu4fD/ArtGI0BcsxYTalDT3LKcCaYLS0/tX40 DF6PcU4VsSj6fVtsORvXz3B/FT4bbLunxCJUqQnKWVUgPSC2J9VcbTnOHmRPGpXy5buS 1wPU09WC6/4kv6Pr/Z2vNysZ9hOswfuOdo+FZprVcJtIH6brNop3j6rOpq169/0ReoIb q8rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Z3EYeS+tIqBFVXo6hPJUkFm8u6eCOLhA7qycqAqlcHg=; b=gj8hj8yO8qptePMF2LL1bE8CBMEdF+8XY/tI4tDQUgMY8+bdOIhW+QC8F/4Myq+6/H h1K5FqC7GGh0PQjH4ubLeNOBFn7htMf2y1jX+ndJIC5DXkZam+wFjAvt3yHsMR8CjYy+ ycg3K9X5Osc1lnqLDqa6vVNBaLsFxd0bEBuVFgEA99sx1aY6tIvIzB+kLj8bH36CBZ8N Tl/xqvWnTNakudpO9Un3TDYqznvPt+7tfVcSQUx2fPkyFaC3XyKBwuAyq7YPmeY6PHg9 tTnZ6mN33G2hpzXU5o4KW0/dx4NyKDXlFzQmix2+HKY8vCfasKqFh9UDCCscd7yGmhca rf5A== X-Gm-Message-State: AOAM532eHFxsYvRbTRvOg8JIOYFWNqYwaMhPafAZQCFlWjYzTi+wJnaF t5j2vpvW0oKuuWO+gyyV1UriZv+fO7NDbKmG X-Google-Smtp-Source: ABdhPJzWMcXzK46NzTRvzzutnp1BqgiDGWeQpHI4uE0DQoI/1yGhlrV8gjgvofQbANK7f7lDyvWyDo2WjxVJIhjI X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:f7cb:: with SMTP id f11mr315849qvo.34.1604534595295; Wed, 04 Nov 2020 16:03:15 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:27 +0100 In-Reply-To: Message-Id: <1f2309b6c4aa9554b298e82bd830aca7dc6877c3.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 17/20] kasan: clarify comment in __kasan_kfree_large From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000417_111052_44168D11 X-CRM114-Status: GOOD ( 12.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently it says that the memory gets poisoned by page_alloc code. Clarify this by mentioning the specific callback that poisons the memory. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Link: https://linux-review.googlesource.com/id/I1334dffb69b87d7986fab88a1a039cc3ea764725 --- mm/kasan/common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 40ff3ce07a76..4360292ad7f3 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -436,5 +436,5 @@ void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ + /* The object will be poisoned by kasan_free_pages(). */ } From patchwork Thu Nov 5 00:02:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9A73C00A89 for ; Thu, 5 Nov 2020 05:17:51 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 020632083B for ; Thu, 5 Nov 2020 05:17:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="wvXz55w1"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="VF31R+CU"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="vy1Rrcsl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 020632083B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ljPgxyY16Cu+zBZmK0o1iXoupAxWZlVNymcW01gG69U=; b=wvXz55w186ImQmcKTkENp66lN zydehdAUxk9OVYyPLJ7iTioGtlKdww1fkZ7saOEY/cFrrrBtXvcLs0JnfzUdus9ioilg0XXaVciGV Xu2JK7y6fyT5ayj2teh0wxBkTgunT7N5M8M3GNzu20JSEugzEeefkER1s3xyTDYT9W3puoH6Qqj4d vglmQ3lNtNpDji+hOyY/ozq8SnP5kiAv/AtvcMZAlZ2ZWg8kvTd8S7H4lwGxj8H5o+sTD7VkOaPHH xcyN/V5IowHgLWQKgCznzuOmrY5R7N0tgf+UO2hYCT/BUFBH8pl1P9gY/ZAf5ix/W61U3N5aMaoaM OUMlGypMg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXdu-0001Uv-Df; Thu, 05 Nov 2020 05:17:02 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3l-00016k-La for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=DV2ANgqfhcrmLoftngxLgTiwbMOTC2d/bt5e9qthpHc=; b=VF31R+CUMUZ/jpHwTrnZKjUyiR qHG9Q5F124iOzYU9iuTxkKwftkwWsxjeYxNh2LcgYbtWzMDbyWxzqaZpoTF7Wi29Eoq/oEfxGDohX jxc6KvX5AFhbyVUL+p4Sl5MqiQuCY3T82srvtztMPS4nKup78LWD+hpAK0HbIEZRjxCi9O1o7xOZU I2GEj4Dequs8Sm+9ihnOn3AWVFyBk7fhjPx1ZiBEVHDXeAxpit9q2uNpVi/k1kG1priv5weHxqWVy XaKW1ZGR+VuaYCHWcKodAp9fdyjZVIF3HU1U+pOtMUUEh10lmTUZcYBlOosPxv1tEHS9WO8h+MuWB Ybg8WGXw==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSlK-0000tb-Lp for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:26 +0000 Received: by mail-wr1-x44a.google.com with SMTP id h8so97215wrt.9 for ; Wed, 04 Nov 2020 16:04:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=DV2ANgqfhcrmLoftngxLgTiwbMOTC2d/bt5e9qthpHc=; b=vy1RrcslAZvwTGzhpUrQomYop6w65FwF/woriGWL9Dpn77y8Zhknz80DtYOCv63Zyw p/LWOXTei2nebiizP1bqEea9glW4tIT6+knngWT0+bcFrBf7WI5Qh2sp4J6Adk0zyDph eHkcEprMLqoAeP0GYVVJGqhXqBlQet6Tvvy/3iyH/A9tBDxnC9C0xXujQi8vz0j5SWDO +Koqc1t5qOSzvWdScaFLe2k8GBxIrqGgbKEVHjKJk+qrLD7FzpfscoBIi5pw0fSyPNL6 ZwRi0io7/7F1fcQ3MEFw5iBS53w7jQmoxltWJEUmIBNQLTb1qHErWmdSyHnA5nY7lhBC sAPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=DV2ANgqfhcrmLoftngxLgTiwbMOTC2d/bt5e9qthpHc=; b=M/RGNaVKUdhAnPhNcv9mXYrn9AcXr+YQ9zJOGz9sKrcMWzlKrIvYVH/SFtn/VD92J5 79J7bdo8EqdmWnJQpWfSJ42zbM5ZNvdrkHTIEFXjF6r+8cnnAAxJFUkUWm3NC3LmIaNb hHA+zzLpeOOE6SAjP33kUTM6ymgLxAmy9mnllYI15/QJLYtiRLPyKey99Kei4yICoq9t 3hwsE2WbNDQmbmvnfxYuY3yX+UEZmf5OfkgdavVBWN7pH0pakVA1eSZ1Hyxx91XJBKaF y1xfmiJ82Z9C9M4Mow2jyFWJxG4d0X4BNlD6iKa1hEfxqbxEAb3dPX2Xst+7JksXupkw 8xHQ== X-Gm-Message-State: AOAM531PU8BwzuKCnU9IFgDdPXxNm14JI12NQ/sfOBP+TwSycuvKOiwU Bf4BsJqUs95r1FNzWDWMtDnH410H2aj6sLEd X-Google-Smtp-Source: ABdhPJwB+j9nNwO12SyduLwM4V0CVjarMVdrOjoUvnK/EOOzJZD9qReGloZ6w37jfrFKiubnnLMxnbunULEgcS6u X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a5d:5612:: with SMTP id l18mr537149wrv.372.1604534599527; Wed, 04 Nov 2020 16:03:19 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:28 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 18/20] kasan: clean up metadata allocation and usage From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000422_916240_2FEC715F X-CRM114-Status: GOOD ( 32.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KASAN marks caches that are sanitized with the SLAB_KASAN cache flag. Currently if the metadata that is appended after the object (stores e.g. stack trace ids) doesn't fit into KMALLOC_MAX_SIZE (can only happen with SLAB, see the comment in the patch), KASAN turns off sanitization completely. With this change sanitization of the object data is always enabled. However the metadata is only stored when it fits. Instead of checking for SLAB_KASAN flag accross the code to find out whether the metadata is there, use cache->kasan_info.alloc/free_meta_offset. As 0 can be a valid value for free_meta_offset, introduce KASAN_NO_FREE_META as an indicator that the free metadata is missing. Along the way rework __kasan_cache_create() and add claryfying comments. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Icd947e2bea054cb5cfbdc6cf6652227d97032dcb --- mm/kasan/common.c | 112 +++++++++++++++++++++++++------------- mm/kasan/generic.c | 15 ++--- mm/kasan/hw_tags.c | 6 +- mm/kasan/kasan.h | 13 ++++- mm/kasan/quarantine.c | 8 +++ mm/kasan/report.c | 43 ++++++++------- mm/kasan/report_sw_tags.c | 7 ++- mm/kasan/sw_tags.c | 4 ++ 8 files changed, 138 insertions(+), 70 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 4360292ad7f3..940b42231069 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -109,9 +109,6 @@ void __kasan_free_pages(struct page *page, unsigned int order) */ static inline unsigned int optimal_redzone(unsigned int object_size) { - if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) - return 0; - return object_size <= 64 - 16 ? 16 : object_size <= 128 - 32 ? 32 : @@ -125,47 +122,79 @@ static inline unsigned int optimal_redzone(unsigned int object_size) void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { - unsigned int orig_size = *size; + unsigned int ok_size; unsigned int redzone_size; - int redzone_adjust; + unsigned int optimal_size; + + /* + * SLAB_KASAN is used to mark caches as ones that are sanitized by + * KASAN. Currently this is used in two places: + * 1. In slab_ksize() when calculating the size of the accessible + * memory within the object. + * 2. In slab_common.c to prevent merging of sanitized caches. + */ + *flags |= SLAB_KASAN; - if (!kasan_stack_collection_enabled()) { - *flags |= SLAB_KASAN; + if (!kasan_stack_collection_enabled()) return; - } - /* Add alloc meta. */ + ok_size = *size; + + /* Add alloc meta into redzone. */ cache->kasan_info.alloc_meta_offset = *size; *size += sizeof(struct kasan_alloc_meta); - /* Add free meta. */ - if (IS_ENABLED(CONFIG_KASAN_GENERIC) && - (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta))) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); + /* + * If alloc meta doesn't fit, don't add it. + * This can only happen with SLAB, as it has KMALLOC_MAX_SIZE equal + * to KMALLOC_MAX_CACHE_SIZE and doesn't fall back to page_alloc for + * larger sizes. + */ + if (*size > KMALLOC_MAX_SIZE) { + cache->kasan_info.alloc_meta_offset = 0; + *size = ok_size; + /* Continue, since free meta might still fit. */ } - redzone_size = optimal_redzone(cache->object_size); - redzone_adjust = redzone_size - (*size - cache->object_size); - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + redzone_size)); + /* Only the generic mode uses free meta or flexible redzones. */ + if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) { + cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META; + return; + } /* - * If the metadata doesn't fit, don't enable KASAN at all. + * Add free meta into redzone when it's not possible to store + * it in the object. This is the case when: + * 1. Object is SLAB_TYPESAFE_BY_RCU, which means that is can + * be touched after it was freed, or + * 2. Object has a constructor, which means it's expected to + * retain its content until the next allocation, or + * 3. Object is too small. + * Otherwise cache->kasan_info.free_meta_offset = 0 is implied. */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; + if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + ok_size = *size; + + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + + /* If free meta doesn't fit, don't add it. */ + if (*size > KMALLOC_MAX_SIZE) { + cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META; + *size = ok_size; + } } - *flags |= SLAB_KASAN; + redzone_size = optimal_redzone(cache->object_size); + /* Calculate size with optimal redzone. */ + optimal_size = cache->object_size + redzone_size; + /* Limit it with KMALLOC_MAX_SIZE (relevant for SLAB only). */ + if (optimal_size > KMALLOC_MAX_SIZE) + optimal_size = KMALLOC_MAX_SIZE; + /* Use optimal size if the size with added metas is not large enough. */ + if (*size < optimal_size) + *size = optimal_size; } size_t __kasan_metadata_size(struct kmem_cache *cache) @@ -181,15 +210,21 @@ size_t __kasan_metadata_size(struct kmem_cache *cache) struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object) { + if (!cache->kasan_info.alloc_meta_offset) + return NULL; return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset; } +#ifdef CONFIG_KASAN_GENERIC struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + if (cache->kasan_info.free_meta_offset == KASAN_NO_FREE_META) + return NULL; return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } +#endif void __kasan_unpoison_data(const void *addr, size_t size) { @@ -276,11 +311,9 @@ void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; if (kasan_stack_collection_enabled()) { - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + if (alloc_meta) + __memset(alloc_meta, 0, sizeof(*alloc_meta)); } /* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */ @@ -319,8 +352,7 @@ static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, if (!kasan_stack_collection_enabled()) return false; - if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || - unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine)) return false; kasan_set_free_info(cache, object, tag); @@ -345,7 +377,11 @@ void __kasan_slab_free_mempool(void *ptr, unsigned long ip) static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); + struct kasan_alloc_meta *alloc_meta; + + alloc_meta = kasan_get_alloc_meta(cache, object); + if (alloc_meta) + kasan_set_track(&alloc_meta->alloc_track, flags); } static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, @@ -372,7 +408,7 @@ static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN)) + if (kasan_stack_collection_enabled()) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index d259e4c3aefd..97e39516f8fe 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -338,10 +338,10 @@ void kasan_record_aux_stack(void *addr) cache = page->slab_cache; object = nearest_obj(cache, page, addr); alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return; - /* - * record the last two call_rcu() call stacks. - */ + /* Record the last two call_rcu() call stacks. */ alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; alloc_meta->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); } @@ -352,11 +352,11 @@ void kasan_set_free_info(struct kmem_cache *cache, struct kasan_free_meta *free_meta; free_meta = kasan_get_free_meta(cache, object); - kasan_set_track(&free_meta->free_track, GFP_NOWAIT); + if (!free_meta) + return; - /* - * the object was freed and has free track set - */ + kasan_set_track(&free_meta->free_track, GFP_NOWAIT); + /* The object was freed and has free track set. */ *(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREETRACK; } @@ -365,5 +365,6 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK) return NULL; + /* Free meta must be present with KASAN_KMALLOC_FREETRACK. */ return &kasan_get_free_meta(cache, object)->free_track; } diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 52984825c75f..a0bc7db4e8ff 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -176,7 +176,8 @@ void kasan_set_free_info(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; alloc_meta = kasan_get_alloc_meta(cache, object); - kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); + if (alloc_meta) + kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); } struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, @@ -185,5 +186,8 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return NULL; + return &alloc_meta->free_track[0]; } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 8a4cd9618142..14ab24931287 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -154,20 +154,31 @@ struct kasan_alloc_meta { struct qlist_node { struct qlist_node *next; }; + +/* + * Generic mode either stores free meta in the object itself or in the redzone + * after the object. In the former case free meta offset is 0, in the latter + * case it has some sane value smaller than INT_MAX. Use INT_MAX as free meta + * offset when free meta isn't present. + */ +#define KASAN_NO_FREE_META (INT_MAX) + struct kasan_free_meta { +#ifdef CONFIG_KASAN_GENERIC /* This field is used while the object is in the quarantine. * Otherwise it might be used for the allocator freelist. */ struct qlist_node quarantine_link; -#ifdef CONFIG_KASAN_GENERIC struct kasan_track free_track; #endif }; struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object); +#ifdef CONFIG_KASAN_GENERIC struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object); +#endif #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 0da3d37e1589..23f6bfb1e73f 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -135,7 +135,12 @@ static void qlink_free(struct qlist_node *qlink, struct kmem_cache *cache) if (IS_ENABLED(CONFIG_SLAB)) local_irq_save(flags); + /* + * As the object now gets freed from the quaratine, assume that its + * free track is now longer valid. + */ *(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREE; + ___cache_free(cache, object, _THIS_IP_); if (IS_ENABLED(CONFIG_SLAB)) @@ -168,6 +173,9 @@ void quarantine_put(struct kmem_cache *cache, void *object) struct qlist_head temp = QLIST_INIT; struct kasan_free_meta *meta = kasan_get_free_meta(cache, object); + if (!meta) + return; + /* * Note: irq must be disabled until after we move the batch to the * global quarantine. Otherwise quarantine_remove_cache() can miss diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 7d86af340148..6a95ad2dee91 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -168,32 +168,35 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, static void describe_object_stacks(struct kmem_cache *cache, void *object, const void *addr, u8 tag) { - struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); - - if (cache->flags & SLAB_KASAN) { - struct kasan_track *free_track; + struct kasan_alloc_meta *alloc_meta; + struct kasan_track *free_track; + alloc_meta = kasan_get_alloc_meta(cache, object); + if (alloc_meta) { print_track(&alloc_meta->alloc_track, "Allocated"); pr_err("\n"); - free_track = kasan_get_free_track(cache, object, tag); - if (free_track) { - print_track(free_track, "Freed"); - pr_err("\n"); - } + } + + free_track = kasan_get_free_track(cache, object, tag); + if (free_track) { + print_track(free_track, "Freed"); + pr_err("\n"); + } #ifdef CONFIG_KASAN_GENERIC - if (alloc_meta->aux_stack[0]) { - pr_err("Last call_rcu():\n"); - print_stack(alloc_meta->aux_stack[0]); - pr_err("\n"); - } - if (alloc_meta->aux_stack[1]) { - pr_err("Second to last call_rcu():\n"); - print_stack(alloc_meta->aux_stack[1]); - pr_err("\n"); - } -#endif + if (!alloc_meta) + return; + if (alloc_meta->aux_stack[0]) { + pr_err("Last call_rcu():\n"); + print_stack(alloc_meta->aux_stack[0]); + pr_err("\n"); } + if (alloc_meta->aux_stack[1]) { + pr_err("Second to last call_rcu():\n"); + print_stack(alloc_meta->aux_stack[1]); + pr_err("\n"); + } +#endif } static void describe_object(struct kmem_cache *cache, void *object, diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index 7604b46239d4..11dc8739e500 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -48,9 +48,10 @@ const char *get_bug_type(struct kasan_access_info *info) object = nearest_obj(cache, page, (void *)addr); alloc_meta = kasan_get_alloc_meta(cache, object); - for (i = 0; i < KASAN_NR_FREE_STACKS; i++) - if (alloc_meta->free_pointer_tag[i] == tag) - return "use-after-free"; + if (alloc_meta) + for (i = 0; i < KASAN_NR_FREE_STACKS; i++) + if (alloc_meta->free_pointer_tag[i] == tag) + return "use-after-free"; return "out-of-bounds"; } diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index d1af6f6c6d12..be10d16bd129 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -170,6 +170,8 @@ void kasan_set_free_info(struct kmem_cache *cache, u8 idx = 0; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return; #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY idx = alloc_meta->free_track_idx; @@ -187,6 +189,8 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, int i = 0; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return NULL; #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { From patchwork Thu Nov 5 00:02:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 244F0C388F9 for ; Thu, 5 Nov 2020 04:13:13 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8F7ED20786 for ; Thu, 5 Nov 2020 04:13:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="wWQGWEhW"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Burz7TvE"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="qJE0zOp+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F7ED20786 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/e4XEUofn30JAYsd+ANNsmMG/ikHh8qbOMVhv24Om/g=; b=wWQGWEhWHPQddFOiRVOauiyly iVkDIgNDExXJKfPwrY0ncSotGyiF4bF0kBhw6sLu+nIgGANKPf9Kl7/gIiVb/QffGx5hFZ/TwZT5e dBQKGJEcEMt3GVTUGOohvvic6Z0hUAHRohzjzrRY5aOj9xNxE1g6+iKRA/bwxp1NS8A10FIZULz0B OSboYnLCxqkqTYveiKgPUgaa12rT0nTh/mmQz5CJ/XrFopU9cI2r8QlQq2AV6I4Zby6c+v5gCgk/M YS6wr8H1JaVMY9Vaqnhi7hAxzYCI6UCZJ82ro3S21jKDSmonMsNLi7xBISOrJCKEbu8ns2wAf44Hc APKODbtCw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaWcl-0004Zr-4t; Thu, 05 Nov 2020 04:11:47 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3m-000145-NE for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:38 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=BKsLmNNgmZaUnh/EpgjKJBz6eZxr51wf9zKFrfwsdMY=; b=Burz7TvE3aKSZBr+nQc6lp3eFr jkzXPH/hx8qJCdW159qp10pb7qlfzKumczZzE2Ywzb6RcQiyXVcJiFPvg6DdyWbL4vRvCTVvvT5IY KZ4ivexfllpcNOQkKfW9cu6tE0lEsMQegTxlgvbHtfa3udVie3Knjqk2ROIUXuySrK2kKpzglF9sw 9Fm/DodwvfmdLJZ8y6CJACdSg0U9RLE212gkCsMS/T3ECPSKkhlHscrdsMN7R9oaE3O+zu7cmNXZC FZx8vNZ/+J+uR89TTSYKv2WWfLJQS4VviX6agafx6lgh1MSlWGPyf//K6438F+1rcRxiMpl2Yxam/ 3FPhtWcw==; Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSlK-0000tc-Nb for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:25 +0000 Received: by mail-wm1-x34a.google.com with SMTP id y1so2489wma.5 for ; Wed, 04 Nov 2020 16:04:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=BKsLmNNgmZaUnh/EpgjKJBz6eZxr51wf9zKFrfwsdMY=; b=qJE0zOp+CKttdyhR8H016jLOb78DZeJauMV8UV6YZ6W6V0nNZuzfcr66fKWF+xyu0j Hu93tQ9+h2euPfBrbAxeuBkQNmB8XDwFPIUVJgoex7AOSAWddfvqlgdW3pAWZhXwKd6X b+4Ap2alZM5FYcGzH2cFewAp6b7/bUbZIK3yBJc2oRUFxZfbXaPfwTPIUBN3UXPtMd8B 0s2vwIlVrgjH1M0Fqhf9QXQfXantgORoY4k0jb4EKhOV+mnANLXMQSac5m11GpbObnx4 yfQ1qeP54OrJQ6bj36qKUqzswK3NjKcjADRI71Uarnr5Yd65cdFHT2Dm5JIJEZXHacUn FVew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=BKsLmNNgmZaUnh/EpgjKJBz6eZxr51wf9zKFrfwsdMY=; b=mx5Au7SimyA939GyyIdYBqbwwKP/sVbEqIqZGqZ2XxJr39i7TzeQeuZyyxgQSzoY1X F7O68z/LlIORh4uarNyOP3A6uEQoN5aBgZnkRNlzjJtWsCWpOdHqiHlSNJTvzA4W4x4K JwYdwQRAr7MaldUSPevuZ8SNKX0win7319+wGqPA9EF/1h5VG54xQuKDRoIIHfO321Gy 7eX0pKDOJSk341FVtXcUkh3VA9wdVgInZeNfXF/YTNv4mLaWS12ewxmIzmYUZz6QJxb+ TgY7Mbl+f1RE8kOz4anYnWNzFkVCx/uxNUGK4E4rXq5nGvr7MHllEc2QGw5le6ayS8U7 WAAw== X-Gm-Message-State: AOAM5337s0gixs0vsEgp1mi287l0am0rkHSDG/XFD509bEt1kWhe14nq RqM/DLGHenEG9qlmfeXeXtcUtXZKdBNFFmxG X-Google-Smtp-Source: ABdhPJwrvNjDqk1pGCj7inLtCzHsZII7Iw+NeQV+f26/W8zfFQBJG75O6f9HjOB9QVSFdbKpg6sk4TSVpyEf1Ad2 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:6302:: with SMTP id x2mr196504wmb.121.1604534601835; Wed, 04 Nov 2020 16:03:21 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:29 +0100 In-Reply-To: Message-Id: <17ecf27ee7b275869047bef91558bd263dd243f1.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 19/20] kasan, mm: allow cache merging with no metadata From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000422_916254_F9C7B1F8 X-CRM114-Status: GOOD ( 16.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba --- include/linux/kasan.h | 26 ++++++++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 11 ++++++++--- 3 files changed, 43 insertions(+), 5 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index d47601517dad..fb8ba4719e3b 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -79,17 +79,35 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static inline kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +slab_flags_t __kasan_never_merge(slab_flags_t flags); +static inline slab_flags_t kasan_never_merge(slab_flags_t flags) +{ + if (kasan_enabled()) + return __kasan_never_merge(flags); + return flags; +} + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline kasan_enabled(void) { return true; } -#endif + +static inline slab_flags_t kasan_never_merge(slab_flags_t flags) +{ + return flags; +} + +#endif /* CONFIG_KASAN_HW_TAGS */ void __kasan_alloc_pages(struct page *page, unsigned int order); static inline void kasan_alloc_pages(struct page *page, unsigned int order) @@ -238,6 +256,10 @@ static inline kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(slab_flags_t flags) +{ + return flags; +} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} static inline void kasan_cache_create(struct kmem_cache *cache, diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 940b42231069..25b18c145b06 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -81,6 +81,17 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(slab_flags_t flags) +{ + if (kasan_stack_collection_enabled()) + return flags; + return flags & ~SLAB_KASAN; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; diff --git a/mm/slab_common.c b/mm/slab_common.c index f1b0c4a22f08..3042ee8ea9ce 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -49,12 +50,16 @@ static DECLARE_WORK(slab_caches_to_rcu_destroy_work, slab_caches_to_rcu_destroy_workfn); /* - * Set of flags that will prevent slab merging + * Set of flags that will prevent slab merging. + * Use slab_never_merge() instead. */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ SLAB_FAILSLAB | SLAB_KASAN) +/* KASAN allows merging in some configurations and will remove SLAB_KASAN. */ +#define slab_never_merge() (kasan_never_merge(SLAB_NEVER_MERGE)) + #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT) @@ -164,7 +169,7 @@ static unsigned int calculate_alignment(slab_flags_t flags, */ int slab_unmergeable(struct kmem_cache *s) { - if (slab_nomerge || (s->flags & SLAB_NEVER_MERGE)) + if (slab_nomerge || (s->flags & slab_never_merge())) return 1; if (s->ctor) @@ -198,7 +203,7 @@ struct kmem_cache *find_mergeable(unsigned int size, unsigned int align, size = ALIGN(size, align); flags = kmem_cache_flags(size, flags, name, NULL); - if (flags & SLAB_NEVER_MERGE) + if (flags & slab_never_merge()) return NULL; list_for_each_entry_reverse(s, &slab_caches, list) { From patchwork Thu Nov 5 00:02:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B225C00A89 for ; Thu, 5 Nov 2020 05:06:48 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 86A3A2065E for ; Thu, 5 Nov 2020 05:06:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="YPQKgJt5"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="dwpJI5rq"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="lkii/bEU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 86A3A2065E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=vIuSBj9XmncyQQ4dGqOGiZeYiSZaP/tStA5XNM8BgAA=; b=YPQKgJt5NSG9zE+Ksd7LC2aKV femezpMterdEzWqXobDiowMDYmVoC+XWzEefBx160tyfRtD3nBWXmqc/PjUGBAxKtssF+jOSnZIn4 4BuUg0Z2xCA3Cgd6Ds/5mDQ7DTypflFK2UNO9H7h8yfXCMJWZ0sP+pcbFe+HLtXF3hBi0TLUMsmoT 4S9/MU6SFpU0gh2OJVCKOHGNH2l9i3DH8rISZJQnwtOOrRvQCaVJRfYG2lfgnZXX0i797FPgTJ3cQ PbqQjOkvCU53+KwIDKyPnQDPjUHDyMYChrdKRFyEyYpk6Xbw624QPY+Q2mbZ2CfW5ACw8Pu6uIeQP cv+/ZWNlQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaXRx-0004ah-Lx; Thu, 05 Nov 2020 05:04:42 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW3k-00016k-Er for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:35:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=acYpqJrzd4w8wpKjZSmquNJqz361BSwiUmG0Zr7FYY8=; b=dwpJI5rqFE9Us7VXaaSk6T4Exf x4TEXT1Z8xMhm2pP6A5Dxef3X0eClMuI5asy1ZHTpFQP1hT3PO5eE/H+xTq9s3bjbZ3ksoQhxxaj2 UuQGMP2B/KKAXVOKB8m+oq4DgeAstXf+Sk7isjecD05x/cb2CeGesh2ngwasLFis91PUefJT0eHTs At1ZQzA8fnawgXAMejJo8jZJ7DniJyWRNgNktnkCx6oPXqZ7ED+snOWmP5YoEn17jhdAAcp3O2HZA mqMfjQelj5Tt/lrTJhAQspC9daWd4rU1tT+V3GtBlO1q/wE3GqWkhrtvm0ZT8xMH7qcF3ly5k7sil fsd3Fk5w==; Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaSlO-0000u9-7q for linux-arm-kernel@lists.infradead.org; Thu, 05 Nov 2020 00:04:29 +0000 Received: by mail-qt1-x849.google.com with SMTP id i15so130481qti.7 for ; Wed, 04 Nov 2020 16:04:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=acYpqJrzd4w8wpKjZSmquNJqz361BSwiUmG0Zr7FYY8=; b=lkii/bEUPCqi4JnZZip2pucMb9g5gTFdgPsLWQRtV/FprAqhxujlo1v8yx8U+bPEfu QHWLb9dY4h6BTR7YBf+SsZkgGUBYGEmUY2kHUFFUnu13xTw2i58bfRPeSkr7Ato/b4gO lh0cvHlvBCXJmqoI/txScPZKTEDqaB96jbixATZbceLsNgq83PSfTbuyBXncl1na6lt+ XbhcNN/tjOAqMlaucYihNhWnAoEshGarqlyQSciCI66CqnvuAOelgYu4hAwqz5rY51WY CNt8ygAN/lK0twQzvFPmF3SGJukgeexL/rz5jOkKDN5ctnW2q8aLthUt7YBJwxO88V4X 3kNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=acYpqJrzd4w8wpKjZSmquNJqz361BSwiUmG0Zr7FYY8=; b=dEvXJSbhMIZZQonYj6F8klPVkgHkE4SBcuZk02BNNxxbVC7aVfDtlUdu6nmnIVBd+W ciikEIqlGwMCl1tGuOZcU4no5b917v/9wgMUJvRgX5CePglYNDqKEeFtw6WMfNCZ4bsm w35oL4TufxofC7MbZo0u4XV2zx1f2ANqHse9KiA0RFSEf+7FLmExyCDC8y2wlfetFIhu IPK2AUVis67uwdBEAf1Leqkbh6EAiuwDnjo28xuAAMu5T3A1oWrAxz/mHYGH1GLPwEqF fjndiN2Vijy6+nUsQFTA06Niir+71kUZf2lKmdXK3W9JR4BfrKOAHO3u/v69OyMRdnIT v8Ow== X-Gm-Message-State: AOAM532ELUgPqZBwnGxv85rY1umZ9yTCKZeTheCnMHo+e11lhkmf4jfU tP/2dHPKVeu++FtcMfZk9ckAA+AWxdjxvln9 X-Google-Smtp-Source: ABdhPJxpWYi8TBwRYN+5nuWxOibbHATeaFRQ+/WQ3g7/C4NxCyrcC1oHgPV5+3pc1pJHPlI39lsx7rrb/k1+eYR1 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:6214:174f:: with SMTP id dc15mr302926qvb.26.1604534604164; Wed, 04 Nov 2020 16:03:24 -0800 (PST) Date: Thu, 5 Nov 2020 01:02:30 +0100 In-Reply-To: Message-Id: <83d168b5877883a194af0eda02d6e523cd17484c.1604534322.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 20/20] kasan: update documentation From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201105_000426_469109_4BEFF91C X-CRM114-Status: GOOD ( 34.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Branislav Rankov , Andrey Konovalov , Kevin Brodsky , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This change updates KASAN documentation to reflect the addition of boot parameters and also reworks and clarifies some of the existing sections, in particular: defines what a memory granule is, mentions quarantine, makes Kunit section more readable. Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 180 +++++++++++++++++++----------- 1 file changed, 113 insertions(+), 67 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index 422f8ee1bb17..ae1e732ebdfa 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -6,6 +6,7 @@ Overview KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to find out-of-bound and use-after-free bugs. KASAN has three modes: + 1. generic KASAN (similar to userspace ASan), 2. software tag-based KASAN (similar to userspace HWASan), 3. hardware tag-based KASAN (based on hardware memory tagging). @@ -39,23 +40,13 @@ CONFIG_KASAN_INLINE. Outline and inline are compiler instrumentation types. The former produces smaller binary while the latter is 1.1 - 2 times faster. Both software KASAN modes work with both SLUB and SLAB memory allocators, -hardware tag-based KASAN currently only support SLUB. -For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. +while the hardware tag-based KASAN currently only support SLUB. + +For better error reports that include stack traces, enable CONFIG_STACKTRACE. To augment reports with last allocation and freeing stack of the physical page, it is recommended to enable also CONFIG_PAGE_OWNER and boot with page_owner=on. -To disable instrumentation for specific files or directories, add a line -similar to the following to the respective kernel Makefile: - -- For a single file (e.g. main.o):: - - KASAN_SANITIZE_main.o := n - -- For all files in one directory:: - - KASAN_SANITIZE := n - Error reports ~~~~~~~~~~~~~ @@ -140,16 +131,20 @@ freed (in case of a use-after-free bug report). Next comes a description of the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. -Reading this part requires some understanding of how KASAN works. - -The state of each 8 aligned bytes of memory is encoded in one shadow byte. -Those 8 bytes can be accessible, partially accessible, freed or be a redzone. -We use the following encoding for each shadow byte: 0 means that all 8 bytes -of the corresponding memory region are accessible; number N (1 <= N <= 7) means -that the first N bytes are accessible, and other (8 - N) bytes are not; -any negative value indicates that the entire 8-byte word is inaccessible. -We use different negative values to distinguish between different kinds of -inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). +Internally KASAN tracks memory state separately for each memory granule, which +is either 8 or 16 aligned bytes depending on KASAN mode. Each number in the +memory state section of the report shows the state of one of the memory +granules that surround the accessed address. + +For generic KASAN the size of each memory granule is 8. The state of each +granule is encoded in one shadow byte. Those 8 bytes can be accessible, +partially accessible, freed or be a part of a redzone. KASAN uses the following +encoding for each shadow byte: 0 means that all 8 bytes of the corresponding +memory region are accessible; number N (1 <= N <= 7) means that the first N +bytes are accessible, and other (8 - N) bytes are not; any negative value +indicates that the entire 8-byte word is inaccessible. KASAN uses different +negative values to distinguish between different kinds of inaccessible memory +like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. @@ -157,6 +152,55 @@ the accessed address is partially accessible. For tag-based KASAN this last report section shows the memory tags around the accessed address (see Implementation details section). +Boot parameters +~~~~~~~~~~~~~~~ + +Hardware tag-based KASAN mode (see the section about different mode below) is +intended for use in production as a security mitigation. Therefore it supports +boot parameters that allow to disable KASAN competely or otherwise control +particular KASAN features. + +The things that can be controlled are: + +1. Whether KASAN is enabled at all. +2. Whether KASAN collects and saves alloc/free stacks. +3. Whether KASAN panics on a detected bug or not. + +The ``kasam.mode`` boot parameter allows to choose one of three main modes: + +- ``kasan.mode=off`` - KASAN is disabled, no tag checks are performed +- ``kasan.mode=prod`` - only essential production features are enabled +- ``kasan.mode=full`` - all KASAN features are enabled + +The chosen mode provides default control values for the features mentioned +above. However it's also possible to override the default values by providing: + +- ``kasan.stack=off`` or ``=on`` - enable alloc/free stacks collection + (default: ``on`` for ``mode=full``, + otherwise ``off``) +- ``kasan.fault=report`` or ``=panic`` - only print KASAN report or also panic + (default: ``report``) + +If ``kasan.mode parameter`` is not provided, it defaults to ``full`` when +``CONFIG_DEBUG_KERNEL`` is enabled, and to ``prod`` otherwise. + +For developers +~~~~~~~~~~~~~~ + +Software KASAN modes use compiler instrumentation to insert validity checks. +Such instrumentation might be incompatible with some part of the kernel, and +therefore needs to be disabled. To disable instrumentation for specific files +or directories, add a line similar to the following to the respective kernel +Makefile: + +- For a single file (e.g. main.o):: + + KASAN_SANITIZE_main.o := n + +- For all files in one directory:: + + KASAN_SANITIZE := n + Implementation details ---------------------- @@ -164,10 +208,10 @@ Implementation details Generic KASAN ~~~~~~~~~~~~~ -From a high level, our approach to memory error detection is similar to that -of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to insert checks of shadow -memory on each memory access. +From a high level perspective, KASAN's approach to memory error detection is +similar to that of kmemcheck: use shadow memory to record whether each byte of +memory is safe to access, and use compile-time instrumentation to insert checks +of shadow memory on each memory access. Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and offset to @@ -194,7 +238,10 @@ function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. -Generic KASAN prints up to 2 call_rcu() call stacks in reports, the last one +Generic KASAN is the only mode that delays the reuse of freed object via +quarantine (see mm/kasan/quarantine.c for implementation). + +Generic KASAN prints up to two call_rcu() call stacks in reports, the last one and the second to last. Software tag-based KASAN @@ -302,15 +349,15 @@ therefore be wasteful. Furthermore, to ensure that different mappings use different shadow pages, mappings would have to be aligned to ``KASAN_GRANULE_SIZE * PAGE_SIZE``. -Instead, we share backing space across multiple mappings. We allocate +Instead, KASAN shares backing space across multiple mappings. It allocates a backing page when a mapping in vmalloc space uses a particular page of the shadow region. This page can be shared by other vmalloc mappings later on. -We hook in to the vmap infrastructure to lazily clean up unused shadow +KASAN hooks in to the vmap infrastructure to lazily clean up unused shadow memory. -To avoid the difficulties around swapping mappings around, we expect +To avoid the difficulties around swapping mappings around, KASAN expects that the part of the shadow region that covers the vmalloc space will not be covered by the early shadow page, but will be left unmapped. This will require changes in arch-specific code. @@ -321,24 +368,31 @@ architectures that do not have a fixed module region. CONFIG_KASAN_KUNIT_TEST & CONFIG_TEST_KASAN_MODULE -------------------------------------------------- -``CONFIG_KASAN_KUNIT_TEST`` utilizes the KUnit Test Framework for testing. -This means each test focuses on a small unit of functionality and -there are a few ways these tests can be run. +KASAN tests consist on two parts: + +1. Tests that are integrated with the KUnit Test Framework. Enabled with +``CONFIG_KASAN_KUNIT_TEST``. These tests can be run and partially verified +automatically in a few different ways, see the instructions below. -Each test will print the KASAN report if an error is detected and then -print the number of the test and the status of the test: +2. Tests that are currently incompatible with Kunit. Enabled with +``CONFIG_TEST_KASAN_MODULE`` and can only be run as a module. These tests can +only be verified manually, by loading the kernel module and inspecting the +kernel log for KASAN reports. -pass:: +Each KUNIT-compatible KASAN test prints a KASAN report if an error is detected. +Then the test prints its number and status. + +When a test passes:: ok 28 - kmalloc_double_kzfree -or, if kmalloc failed:: +When a test fails due to a failed ``kmalloc``:: # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163 Expected ptr is not null, but is not ok 4 - kmalloc_large_oob_right -or, if a KASAN report was expected, but not found:: +When a test fails due to a missing KASAN report:: # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 Expected kasan_data->report_expected == kasan_data->report_found, but @@ -346,46 +400,38 @@ or, if a KASAN report was expected, but not found:: kasan_data->report_found == 0 not ok 28 - kmalloc_double_kzfree -All test statuses are tracked as they run and an overall status will -be printed at the end:: +At the end the cumulative status of all KASAN tests is printed. On success:: ok 1 - kasan -or:: +Or, if one of the tests failed:: not ok 1 - kasan -(1) Loadable Module -~~~~~~~~~~~~~~~~~~~~ + +There are a few ways to run Kunit-compatible KASAN tests. + +1. Loadable module +~~~~~~~~~~~~~~~~~~ With ``CONFIG_KUNIT`` enabled, ``CONFIG_KASAN_KUNIT_TEST`` can be built as -a loadable module and run on any architecture that supports KASAN -using something like insmod or modprobe. The module is called ``test_kasan``. +a loadable module and run on any architecture that supports KASAN by loading +the module with insmod or modprobe. The module is called ``test_kasan``. -(2) Built-In -~~~~~~~~~~~~~ +2. Built-In +~~~~~~~~~~~ With ``CONFIG_KUNIT`` built-in, ``CONFIG_KASAN_KUNIT_TEST`` can be built-in -on any architecure that supports KASAN. These and any other KUnit -tests enabled will run and print the results at boot as a late-init -call. +on any architecure that supports KASAN. These and any other KUnit tests enabled +will run and print the results at boot as a late-init call. -(3) Using kunit_tool -~~~~~~~~~~~~~~~~~~~~~ +3. Using kunit_tool +~~~~~~~~~~~~~~~~~~~ -With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, we can also -use kunit_tool to see the results of these along with other KUnit -tests in a more readable way. This will not print the KASAN reports -of tests that passed. Use `KUnit documentation `_ for more up-to-date -information on kunit_tool. +With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, it's also +possible use ``kunit_tool`` to see the results of these and other KUnit tests +in a more readable way. This will not print the KASAN reports of the tests that +passed. Use `KUnit documentation `_ +for more up-to-date information on ``kunit_tool``. .. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html - -``CONFIG_TEST_KASAN_MODULE`` is a set of KASAN tests that could not be -converted to KUnit. These tests can be run only as a module with -``CONFIG_TEST_KASAN_MODULE`` built as a loadable module and -``CONFIG_KASAN`` built-in. The type of error expected and the -function being run is printed before the expression expected to give -an error. Then the error is printed, if found, and that test -should be interpretted to pass only if the error was the one expected -by the test.