From patchwork Wed May 11 08:12:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wonhyuk Yang X-Patchwork-Id: 12845905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CFA5C433EF for ; Wed, 11 May 2022 08:12:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD3736B0073; Wed, 11 May 2022 04:12:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A82B26B0075; Wed, 11 May 2022 04:12:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94BA96B0078; Wed, 11 May 2022 04:12:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 86C3C6B0073 for ; Wed, 11 May 2022 04:12:17 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 5D34281E37 for ; Wed, 11 May 2022 08:12:17 +0000 (UTC) X-FDA: 79452744714.03.AEB1D97 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf15.hostedemail.com (Postfix) with ESMTP id B58F1A009A for ; Wed, 11 May 2022 08:12:04 +0000 (UTC) Received: by mail-pl1-f175.google.com with SMTP id m12so1164157plb.4 for ; Wed, 11 May 2022 01:12:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=cqJR2AA+wNKWbyVIetsUsVHvhXM2cLpfd5b5B8Xn3tg=; b=jFlZHXvADIRkCRlklvNg7gh4pcEQYsdUWPiarBS4j4Xq6JltD0PiSGyqh5Vl+xq54U bFbwlaGBKNGqmGUR9BAiKyjE3GAERMDhpJU2EbAKPidHXgPHmh5+kT28J/ou05ShmjRP eR9l9we6/YnhjzicYiGOYzTfnj1c4Tl6uKDaUf/OprDbU2Aj4vbv6zJwRtJZZ9bs6m+S VCcR4KInrRuphEK/qmSbOZ/OKPPsUis6flBysYoPwZOaZowlGPafw+HjYvnVZcT/AtN7 F0vISpDdaZdrfge/HN0XEaG9bnC89q0ZV11LE8Fi8S3e9TtlizKJbTXWSFnsYllSyYil d7pA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=cqJR2AA+wNKWbyVIetsUsVHvhXM2cLpfd5b5B8Xn3tg=; b=auOFlxMOJy0FeMd7a94vHsuTFee0FrmY8rY0bN+dHUt4Nv6a+t3GD/re+iy71Xqsbs HsoYuEKvDJhsB5KtXPf4LGk9HFF5kzRNGceMOX4lyLsEJjzoiXpRlRAZE50taeYzQsMK treCDV7drfqcAmXlEl7WBG4GHq2DSzyRxcoDl8uPiZsSmzvf65WoJMOmiKcn3wA/mogr MWyg/mYOjM7sWZx/gah5ZqbBFDryDZrNQoS2Ed3x63JiaU0uI+PDf5juWNMLvF5FkGMq lknVMTDAvl85GWLcFRGcOCDOeATV+D3EjmJyRjQD02Kt74bAPh9Z8e4S/0YXiYn7tf8x YHcA== X-Gm-Message-State: AOAM533tcJUZ76XJociGJXe5N/6IjUlVUgiE2YvyYB+MLSp27qRhw8tT /VHCL3riOLGDj5xl5fXs8tE= X-Google-Smtp-Source: ABdhPJzOpnwc+Ic2DdJylptmY/bldO6LmURa5sfn+gm7gn+nyabdPinrJUTlGA11hWuAbdSA4Y4NJQ== X-Received: by 2002:a17:90a:f30d:b0:1dc:b437:4ab with SMTP id ca13-20020a17090af30d00b001dcb43704abmr4067693pjb.23.1652256735731; Wed, 11 May 2022 01:12:15 -0700 (PDT) Received: from localhost.localdomain ([125.131.156.123]) by smtp.gmail.com with ESMTPSA id a4-20020a17090ad80400b001d8ace370cbsm3294259pjv.54.2022.05.11.01.12.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 May 2022 01:12:14 -0700 (PDT) From: Wonhyuk Yang To: Mel Gorman , Steven Rostedt , Ingo Molnar , Andrew Morton Cc: Wonhyuk Yang , Baik Song An , Hong Yeon Kim , Taeung Song , linuxgeek@linuxgeek.io, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH] mm/page_alloc: Fix tracepoint mm_page_alloc_zone_locked() Date: Wed, 11 May 2022 17:12:07 +0900 Message-Id: <20220511081207.132034-1-vvghjk1234@gmail.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: B58F1A009A X-Stat-Signature: omz1pc4pxj79pnam7htko9jjhngtr697 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=jFlZHXvA; spf=pass (imf15.hostedemail.com: domain of vvghjk1234@gmail.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=vvghjk1234@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1652256724-212173 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, trace point mm_page_alloc_zone_locked() doesn't show correct information. First, when alloc_flag has ALLOC_HARDER/ALLOC_CMA, page can be allocated from MIGRATE_HIGHATOMIC/MIGRATE_CMA. Nevertheless, tracepoint use requested migration type not MIGRATE_HIGHATOMIC and MIGRATE_CMA. Second, after Commit 44042b4498728 ("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists") percpu-list can store high order pages. But trace point determine whether it is a refiil of percpu-list by comparing requested order and 0. To handle these problems, use cached migration type by get_pcppage_migratetype() instead of requested migration type. Then, make mm_page_alloc_zone_locked() be called only two contexts (rmqueue_bulk, rmqueue). With a new argument called percpu_refill, it can show whether it is a refill of percpu-list correctly. Cc: Baik Song An Cc: Hong Yeon Kim Cc: Taeung Song Cc: linuxgeek@linuxgeek.io Signed-off-by: Wonhyuk Yang --- include/trace/events/kmem.h | 14 +++++++++----- mm/page_alloc.c | 18 +++++++++--------- 2 files changed, 18 insertions(+), 14 deletions(-) diff --git a/include/trace/events/kmem.h b/include/trace/events/kmem.h index ddc8c944f417..f89fb3afcd46 100644 --- a/include/trace/events/kmem.h +++ b/include/trace/events/kmem.h @@ -229,20 +229,23 @@ TRACE_EVENT(mm_page_alloc, DECLARE_EVENT_CLASS(mm_page, - TP_PROTO(struct page *page, unsigned int order, int migratetype), + TP_PROTO(struct page *page, unsigned int order, int migratetype, + int percpu_refill), - TP_ARGS(page, order, migratetype), + TP_ARGS(page, order, migratetype, percpu_refill), TP_STRUCT__entry( __field( unsigned long, pfn ) __field( unsigned int, order ) __field( int, migratetype ) + __field( int, percpu_refill ) ), TP_fast_assign( __entry->pfn = page ? page_to_pfn(page) : -1UL; __entry->order = order; __entry->migratetype = migratetype; + __entry->percpu_refill = percpu_refill; ), TP_printk("page=%p pfn=0x%lx order=%u migratetype=%d percpu_refill=%d", @@ -250,14 +253,15 @@ DECLARE_EVENT_CLASS(mm_page, __entry->pfn != -1UL ? __entry->pfn : 0, __entry->order, __entry->migratetype, - __entry->order == 0) + __entry->percpu_refill) ); DEFINE_EVENT(mm_page, mm_page_alloc_zone_locked, - TP_PROTO(struct page *page, unsigned int order, int migratetype), + TP_PROTO(struct page *page, unsigned int order, int migratetype, + int percpu_refill), - TP_ARGS(page, order, migratetype) + TP_ARGS(page, order, migratetype, percpu_refill) ); TRACE_EVENT(mm_page_pcpu_drain, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0e42038382c1..0351808322ba 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2999,7 +2999,7 @@ __rmqueue(struct zone *zone, unsigned int order, int migratetype, zone_page_state(zone, NR_FREE_PAGES) / 2) { page = __rmqueue_cma_fallback(zone, order); if (page) - goto out; + return page; } } retry: @@ -3012,9 +3012,6 @@ __rmqueue(struct zone *zone, unsigned int order, int migratetype, alloc_flags)) goto retry; } -out: - if (page) - trace_mm_page_alloc_zone_locked(page, order, migratetype); return page; } @@ -3028,6 +3025,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order, int migratetype, unsigned int alloc_flags) { int i, allocated = 0; + int mt; /* * local_lock_irq held so equivalent to spin_lock_irqsave for @@ -3055,7 +3053,9 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order, */ list_add_tail(&page->lru, list); allocated++; - if (is_migrate_cma(get_pcppage_migratetype(page))) + mt = get_pcppage_migratetype(page); + trace_mm_page_alloc_zone_locked(page, order, mt, true); + if (is_migrate_cma(mt)) __mod_zone_page_state(zone, NR_FREE_CMA_PAGES, -(1 << order)); } @@ -3704,6 +3704,7 @@ struct page *rmqueue(struct zone *preferred_zone, { unsigned long flags; struct page *page; + int mt; if (likely(pcp_allowed_order(order))) { /* @@ -3735,16 +3736,15 @@ struct page *rmqueue(struct zone *preferred_zone, */ if (order > 0 && alloc_flags & ALLOC_HARDER) { page = __rmqueue_smallest(zone, order, MIGRATE_HIGHATOMIC); - if (page) - trace_mm_page_alloc_zone_locked(page, order, migratetype); } if (!page) { page = __rmqueue(zone, order, migratetype, alloc_flags); if (!page) goto failed; } - __mod_zone_freepage_state(zone, -(1 << order), - get_pcppage_migratetype(page)); + mt = get_pcppage_migratetype(page); + trace_mm_page_alloc_zone_locked(page, order, mt, false); + __mod_zone_freepage_state(zone, -(1 << order), mt); spin_unlock_irqrestore(&zone->lock, flags); } while (check_new_pages(page, order));