[11/19] mm, slub: fix mismatch between reconstructed freelist depth and cnt

Commit Message

Andrew Morton Oct. 18, 2021, 10:15 p.m. UTC

From: Miaohe Lin <[e-mail]>
Subject: mm, slub: fix mismatch between reconstructed freelist depth and cnt

If object's reuse is delayed, it will be excluded from the reconstructed
freelist.  But we forgot to adjust the cnt accordingly.  So there will be
a mismatch between reconstructed freelist depth and cnt.  This will lead
to free_debug_processing() complaining about freelist count or a incorrect
slub inuse count.

Link: https://lkml.kernel.org/r/[e-mail]
Fixes: c3895391df38 ("kasan, slub: fix handling of kasan_slab_free hook")
Signed-off-by: Miaohe Lin <[e-mail]>
Reviewed-by: Vlastimil Babka <[e-mail]>
Cc: Andrey Konovalov <[e-mail]>
Cc: Andrey Ryabinin <[e-mail]>
Cc: Bharata B Rao <[e-mail]>
Cc: Christoph Lameter <[e-mail]>
Cc: David Rientjes <[e-mail]>
Cc: Faiyaz Mohammed <[e-mail]>
Cc: Greg Kroah-Hartman <[e-mail]>
Cc: Joonsoo Kim <[e-mail]>
Cc: Kees Cook <[e-mail]>
Cc: Pekka Enberg <[e-mail]>
Cc: Roman Gushchin <[e-mail]>
Cc: <[e-mail]>
Signed-off-by: Andrew Morton <[e-mail]>
---

 mm/slub.c |   11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

Patch

--- a/mm/slub.c~mm-slub-fix-mismatch-between-reconstructed-freelist-depth-and-cnt
+++ a/mm/slub.c
@@ -1701,7 +1701,8 @@  static __always_inline bool slab_free_ho
 }
 
 static inline bool slab_free_freelist_hook(struct kmem_cache *s,
-					   void **head, void **tail)
+					   void **head, void **tail,
+					   int *cnt)
 {
 
 	void *object;
@@ -1728,6 +1729,12 @@  static inline bool slab_free_freelist_ho
 			*head = object;
 			if (!*tail)
 				*tail = object;
+		} else {
+			/*
+			 * Adjust the reconstructed freelist depth
+			 * accordingly if object's reuse is delayed.
+			 */
+			--(*cnt);
 		}
 	} while (object != old_tail);
 
@@ -3480,7 +3487,7 @@  static __always_inline void slab_free(st
 	 * With KASAN enabled slab_free_freelist_hook modifies the freelist
 	 * to remove objects, whose reuse must be delayed.
 	 */
-	if (slab_free_freelist_hook(s, &head, &tail))
+	if (slab_free_freelist_hook(s, &head, &tail, &cnt))
 		do_slab_free(s, page, head, tail, cnt, addr);
 }