From patchwork Fri Apr 1 18:21:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12798592 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC5F4C433EF for ; Fri, 1 Apr 2022 18:25:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8BD528D000B; Fri, 1 Apr 2022 14:21:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89F8B8D0003; Fri, 1 Apr 2022 14:21:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 75B558D000B; Fri, 1 Apr 2022 14:21:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0244.hostedemail.com [216.40.44.244]) by kanga.kvack.org (Postfix) with ESMTP id 689B38D0003 for ; Fri, 1 Apr 2022 14:21:29 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 2A4F11830451B for ; Fri, 1 Apr 2022 18:21:19 +0000 (UTC) X-FDA: 79309127478.16.26A56B1 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf13.hostedemail.com (Postfix) with ESMTP id A9E0220013 for ; Fri, 1 Apr 2022 18:21:18 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 72FE6B82590; Fri, 1 Apr 2022 18:21:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F190C3410F; Fri, 1 Apr 2022 18:21:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1648837276; bh=j3G5qLSFoKcDWuso+x7UqSY0ZG6NlWyASPqJ1eYQrJE=; h=Date:To:From:In-Reply-To:Subject:From; b=xwVG9jsTYWxFeX0EIHI7Rm+8T6JoFAbTCBUwVjF6YU1EBYuJAq/blaFdWGYHb+mzG FGEDE75OS48xwkw1EUnHQajzjome11aexWWqGjgGMd1NRrAvN8oJ/eZIQMsFfEuCmM DGxIsPc7TwyS6sJ+vnQd+W+U3lGBoBCI7FFx5Kgk= Date: Fri, 01 Apr 2022 11:21:15 -0700 To: roman.gushchin@linux.dev,glider@google.com,elver@google.com,dvyukov@google.com,duanxiongchun@bytedance.com,songmuchun@bytedance.com,akpm@linux-foundation.org,patches@lists.linux.dev,linux-mm@kvack.org,mm-commits@vger.kernel.org,torvalds@linux-foundation.org,akpm@linux-foundation.org From: Andrew Morton In-Reply-To: Subject: [patch 09/16] mm: kfence: fix objcgs vector allocation Message-Id: <20220401182116.0F190C3410F@smtp.kernel.org> Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=xwVG9jsT; spf=pass (imf13.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 6feaexe4t1pqqndh48gbs3rghes57uym X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A9E0220013 X-HE-Tag: 1648837278-877414 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: kfence: fix objcgs vector allocation If the kfence object is allocated to be used for objects vector, then this slot of the pool eventually being occupied permanently since the vector is never freed. The solutions could be 1) freeing vector when the kfence object is freed or 2) allocating all vectors statically. Since the memory consumption of object vectors is low, it is better to chose 2) to fix the issue and it is also can reduce overhead of vectors allocating in the future. Link: https://lkml.kernel.org/r/20220328132843.16624-1-songmuchun@bytedance.com Fixes: d3fb45f370d9 ("mm, kfence: insert KFENCE hooks for SLAB") Signed-off-by: Muchun Song Reviewed-by: Marco Elver Reviewed-by: Roman Gushchin Cc: Alexander Potapenko Cc: Dmitry Vyukov Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- mm/kfence/core.c | 11 ++++++++++- mm/kfence/kfence.h | 3 +++ 2 files changed, 13 insertions(+), 1 deletion(-) --- a/mm/kfence/core.c~mm-kfence-fix-objcgs-vector-allocation +++ a/mm/kfence/core.c @@ -566,6 +566,8 @@ static unsigned long kfence_init_pool(vo * enters __slab_free() slow-path. */ for (i = 0; i < KFENCE_POOL_SIZE / PAGE_SIZE; i++) { + struct slab *slab = page_slab(&pages[i]); + if (!i || (i % 2)) continue; @@ -573,7 +575,11 @@ static unsigned long kfence_init_pool(vo if (WARN_ON(compound_head(&pages[i]) != &pages[i])) return addr; - __SetPageSlab(&pages[i]); + __folio_set_slab(slab_folio(slab)); +#ifdef CONFIG_MEMCG + slab->memcg_data = (unsigned long)&kfence_metadata[i / 2 - 1].objcg | + MEMCG_DATA_OBJCGS; +#endif } /* @@ -1033,6 +1039,9 @@ void __kfence_free(void *addr) { struct kfence_metadata *meta = addr_to_metadata((unsigned long)addr); +#ifdef CONFIG_MEMCG + KFENCE_WARN_ON(meta->objcg); +#endif /* * If the objects of the cache are SLAB_TYPESAFE_BY_RCU, defer freeing * the object, as the object page may be recycled for other-typed --- a/mm/kfence/kfence.h~mm-kfence-fix-objcgs-vector-allocation +++ a/mm/kfence/kfence.h @@ -89,6 +89,9 @@ struct kfence_metadata { struct kfence_track free_track; /* For updating alloc_covered on frees. */ u32 alloc_stack_hash; +#ifdef CONFIG_MEMCG + struct obj_cgroup *objcg; +#endif }; extern struct kfence_metadata kfence_metadata[CONFIG_KFENCE_NUM_OBJECTS];