From patchwork Wed Sep 8 13:27:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 12481157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21571C433EF for ; Wed, 8 Sep 2021 13:27:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BDC0061164 for ; Wed, 8 Sep 2021 13:27:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BDC0061164 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 3BCAE6B006C; Wed, 8 Sep 2021 09:27:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 36CE86B0071; Wed, 8 Sep 2021 09:27:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25B486B0072; Wed, 8 Sep 2021 09:27:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0061.hostedemail.com [216.40.44.61]) by kanga.kvack.org (Postfix) with ESMTP id 177796B006C for ; Wed, 8 Sep 2021 09:27:38 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B4C0C1839261F for ; Wed, 8 Sep 2021 13:27:37 +0000 (UTC) X-FDA: 78564483354.38.81258B6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf20.hostedemail.com (Postfix) with ESMTP id 3F0C6D0000AF for ; Wed, 8 Sep 2021 13:27:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631107656; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=4zJCMyAEhGNeqtEq1uHSEAAohctgkdWhGGLuAjpd4pI=; b=WhWCvsq94uaKR5T+/+I5mSL64ZbuhtRciK/1HOyqTFadJZgpUKfRxWpd+6+Kkj3Be0HtFM zjoHnNJmQH1yL4mKZlfBrU8QRF+rbMZHPgeS+q9p3UO5lRrwqJEmVdDSeCqriG2zvfN8i/ 8G9aLvCYKUcTJRoXzoQStewaFJH0j5Y= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-416--9IIZwwuMYWWGNmY0bHy9w-1; Wed, 08 Sep 2021 09:27:35 -0400 X-MC-Unique: -9IIZwwuMYWWGNmY0bHy9w-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 82ED0802929; Wed, 8 Sep 2021 13:27:34 +0000 (UTC) Received: from t480s.redhat.com (unknown [10.39.195.121]) by smtp.corp.redhat.com (Postfix) with ESMTP id 71CE4194B9; Wed, 8 Sep 2021 13:27:28 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: David Hildenbrand , Ping Fang , Andrew Morton , Uladzislau Rezki , Roman Gushchin , Michal Hocko , Oscar Salvador , linux-mm@kvack.org Subject: [PATCH v1] mm/vmalloc: fix exact allocations with an alignment > 1 Date: Wed, 8 Sep 2021 15:27:27 +0200 Message-Id: <20210908132727.16165-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WhWCvsq9; spf=none (imf20.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: 1ixgsszirokf44rfp8mf3au86871cxcx X-Rspamd-Queue-Id: 3F0C6D0000AF X-Rspamd-Server: rspam04 X-HE-Tag: 1631107657-598295 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: find_vmap_lowest_match() is imprecise such that it won't always find "the first free block ... that will accomplish the request" if an alignment > 1 was specified: especially also when the alignment is PAGE_SIZE. Unfortuantely, the way the vmalloc data structures were designed, propagating the max size without alignment information through the tree, it's hard to make it precise again when an alignment > 1 is specified. The issue is that in order to be able to eventually align later, find_vmap_lowest_match() has to search for a slightly bigger area and might skip some applicable subtrees just by lookign at the result of get_subtree_max_size(). While this usually doesn't matter, it matters for exact allocations as performed by KASAN when onlining a memory block, when the free block exactly matches the request. (mm/kasn/shadow.c:kasan_mem_notifier()). In case we online memory blocks out of order (not lowest to highest address), find_vmap_lowest_match() with PAGE_SIZE alignment will reject an exact allocation if it corresponds exactly to a free block. (there are some corner cases where it would still work, if we're lucky and hit the first is_within_this_va() inside the while loop) [root@vm-0 fedora]# echo online > /sys/devices/system/memory/memory82/state [root@vm-0 fedora]# echo online > /sys/devices/system/memory/memory83/state [root@vm-0 fedora]# echo online > /sys/devices/system/memory/memory85/state [root@vm-0 fedora]# echo online > /sys/devices/system/memory/memory84/state [ 223.858115] vmap allocation for size 16777216 failed: use vmalloc= to increase size [ 223.859415] bash: vmalloc: allocation failure: 16777216 bytes, mode:0x6000c0(GFP_KERNEL), nodemask=(null),cpuset=/,mems_allowed=0 [ 223.860992] CPU: 4 PID: 1644 Comm: bash Kdump: loaded Not tainted 4.18.0-339.el8.x86_64+debug #1 [ 223.862149] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 223.863580] Call Trace: [ 223.863946] dump_stack+0x8e/0xd0 [ 223.864420] warn_alloc.cold.90+0x8a/0x1b2 [ 223.864990] ? zone_watermark_ok_safe+0x300/0x300 [ 223.865626] ? slab_free_freelist_hook+0x85/0x1a0 [ 223.866264] ? __get_vm_area_node+0x240/0x2c0 [ 223.866858] ? kfree+0xdd/0x570 [ 223.867309] ? kmem_cache_alloc_node_trace+0x157/0x230 [ 223.868028] ? notifier_call_chain+0x90/0x160 [ 223.868625] __vmalloc_node_range+0x465/0x840 [ 223.869230] ? mark_held_locks+0xb7/0x120 While we could fix this in kasan_mem_notifier() by passing an alignment of "1", this is actually an implementation detail of vmalloc and to be handled internally. So use an alignment of 1 when calling find_vmap_lowest_match() for exact allocations that are expected to succeed -- if the given range can satisfy the alignment requirements. Fixes: 68ad4a330433 ("mm/vmalloc.c: keep track of free blocks for vmap allocation") Reported-by: Ping Fang Cc: Andrew Morton Cc: Uladzislau Rezki (Sony) Cc: Roman Gushchin Cc: Michal Hocko Cc: Oscar Salvador Cc: linux-mm@kvack.org Signed-off-by: David Hildenbrand Tested-by: David Hildenbrand Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) base-commit: 7d2a07b769330c34b4deabeed939325c77a7ec2f diff --git a/mm/vmalloc.c b/mm/vmalloc.c index d5cd52805149..c6071f5f8de3 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -1153,7 +1153,8 @@ is_within_this_va(struct vmap_area *va, unsigned long size, /* * Find the first free block(lowest start address) in the tree, * that will accomplish the request corresponding to passing - * parameters. + * parameters. Note that with an alignment > 1, this function + * can be imprecise and skip applicable free blocks. */ static __always_inline struct vmap_area * find_vmap_lowest_match(unsigned long size, @@ -1396,7 +1397,15 @@ __alloc_vmap_area(unsigned long size, unsigned long align, enum fit_type type; int ret; - va = find_vmap_lowest_match(size, align, vstart); + /* + * For exact allocations, ignore the alignment, such that + * find_vmap_lowest_match() won't search for a bigger free area just + * able to align later and consequently fail the search. + */ + if (vend - vstart == size && IS_ALIGNED(vstart, align)) + va = find_vmap_lowest_match(size, 1, vstart); + else + va = find_vmap_lowest_match(size, align, vstart); if (unlikely(!va)) return vend;