From patchwork Tue Nov 22 13:12:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 13052336 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 722EFC4332F for ; Tue, 22 Nov 2022 13:12:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D65896B0075; Tue, 22 Nov 2022 08:12:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D15A56B0078; Tue, 22 Nov 2022 08:12:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C04C76B007B; Tue, 22 Nov 2022 08:12:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B27EA6B0075 for ; Tue, 22 Nov 2022 08:12:55 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 710F5C0307 for ; Tue, 22 Nov 2022 13:12:55 +0000 (UTC) X-FDA: 80161118310.20.BCFF6E8 Received: from outbound-smtp58.blacknight.com (outbound-smtp58.blacknight.com [46.22.136.242]) by imf05.hostedemail.com (Postfix) with ESMTP id F1BA310000E for ; Tue, 22 Nov 2022 13:12:53 +0000 (UTC) Received: from mail.blacknight.com (pemlinmail02.blacknight.ie [81.17.254.11]) by outbound-smtp58.blacknight.com (Postfix) with ESMTPS id 9CB50FA8FA for ; Tue, 22 Nov 2022 13:12:52 +0000 (GMT) Received: (qmail 21501 invoked from network); 22 Nov 2022 13:12:52 -0000 Received: from unknown (HELO morpheus.112glenside.lan) (mgorman@techsingularity.net@[84.203.198.246]) by 81.17.254.9 with ESMTPA; 22 Nov 2022 13:12:52 -0000 From: Mel Gorman To: Andrew Morton Cc: Hugh Dickins , Yu Zhao , Vlastimil Babka , Marcelo Tosatti , Michal Hocko , Marek Szyprowski , LKML , Linux-MM , Mel Gorman Subject: [PATCH 1/2] mm/page_alloc: Leave IRQs enabled for per-cpu page allocations -fix Date: Tue, 22 Nov 2022 13:12:28 +0000 Message-Id: <20221122131229.5263-2-mgorman@techsingularity.net> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20221122131229.5263-1-mgorman@techsingularity.net> References: <20221122131229.5263-1-mgorman@techsingularity.net> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.136.242 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669122774; a=rsa-sha256; cv=none; b=VNmpgO0NkOF8ryvbW/d+PkMSQeAYrQK272ZuKlPR6VDXZbNvXp9oQqGyOfkxLVFkd0MqfW N/QQp18f94MnPR3e28crZH9lcWp2SaAv0X7PaDiETCLC5JzHP9i1sXoXiVcA2n6nqDuGmZ OYo0WVJV4pVjI/YMyh6Nynrclm/boHQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669122774; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qBCaONlLg6w/Q2XsXyH9oJ9r8IGo/Zer6iXJkU1HRaE=; b=fiXAw2TMENFfjlRgZwkQx2MtrsDWZWeh1iRMH1UQw1HiVmwBbG3ZKck7JkHX+sSO2v20CY OXQslUM73TDXaa4Z8g0hrVYXVUHLYqF4aaSEAekzafoA2cze75zqa82n6qpNRoCXX7y+X3 2L8RhfzxHdIyO8FErOPFpB+15HJrzlA= X-Rspam-User: Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.136.242 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none X-Stat-Signature: a7bm3nt9898apbo8q5ybda4hhtu8c147 X-Rspamd-Queue-Id: F1BA310000E X-Rspamd-Server: rspam09 X-HE-Tag: 1669122773-63497 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As noted by Vlastimil Babka, the migratetype might be wrong if a PCP was not locked so check the migrate type early. Similarly the !pcp check is generally unlikely so explicitly tagging it makes sense. This is a fix for the mm-unstable patch mm-page_alloc-leave-irqs-enabled-for-per-cpu-page-allocations.patch Reported-by: Vlastimil Babka Signed-off-by: Mel Gorman Acked-by: Vlastimil Babka --- mm/page_alloc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 323fec05c4c6..445066617204 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3516,6 +3516,7 @@ void free_unref_page_list(struct list_head *list) struct zone *zone = page_zone(page); list_del(&page->lru); + migratetype = get_pcppage_migratetype(page); /* Different zone, different pcp lock. */ if (zone != locked_zone) { @@ -3530,7 +3531,7 @@ void free_unref_page_list(struct list_head *list) */ pcp_trylock_prepare(UP_flags); pcp = pcp_spin_trylock(zone->per_cpu_pageset); - if (!pcp) { + if (unlikely(!pcp)) { pcp_trylock_finish(UP_flags); free_one_page(zone, page, page_to_pfn(page), 0, migratetype, FPI_NONE); @@ -3545,7 +3546,6 @@ void free_unref_page_list(struct list_head *list) * Non-isolated types over MIGRATE_PCPTYPES get added * to the MIGRATE_MOVABLE pcp list. */ - migratetype = get_pcppage_migratetype(page); if (unlikely(migratetype >= MIGRATE_PCPTYPES)) migratetype = MIGRATE_MOVABLE; From patchwork Tue Nov 22 13:12:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 13052337 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E06A3C4332F for ; Tue, 22 Nov 2022 13:13:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8356E6B0078; Tue, 22 Nov 2022 08:13:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E6246B007B; Tue, 22 Nov 2022 08:13:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6AE358E0001; Tue, 22 Nov 2022 08:13:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5DF4C6B0078 for ; Tue, 22 Nov 2022 08:13:05 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2508D80520 for ; Tue, 22 Nov 2022 13:13:05 +0000 (UTC) X-FDA: 80161118730.21.F583177 Received: from outbound-smtp15.blacknight.com (outbound-smtp15.blacknight.com [46.22.139.232]) by imf27.hostedemail.com (Postfix) with ESMTP id 8323C40011 for ; Tue, 22 Nov 2022 13:13:04 +0000 (UTC) Received: from mail.blacknight.com (pemlinmail02.blacknight.ie [81.17.254.11]) by outbound-smtp15.blacknight.com (Postfix) with ESMTPS id 41E251C3840 for ; Tue, 22 Nov 2022 13:13:03 +0000 (GMT) Received: (qmail 22136 invoked from network); 22 Nov 2022 13:13:03 -0000 Received: from unknown (HELO morpheus.112glenside.lan) (mgorman@techsingularity.net@[84.203.198.246]) by 81.17.254.9 with ESMTPA; 22 Nov 2022 13:13:02 -0000 From: Mel Gorman To: Andrew Morton Cc: Hugh Dickins , Yu Zhao , Vlastimil Babka , Marcelo Tosatti , Michal Hocko , Marek Szyprowski , LKML , Linux-MM , Mel Gorman Subject: [PATCH 2/2] mm/page_alloc: Simplify locking during free_unref_page_list Date: Tue, 22 Nov 2022 13:12:29 +0000 Message-Id: <20221122131229.5263-3-mgorman@techsingularity.net> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20221122131229.5263-1-mgorman@techsingularity.net> References: <20221122131229.5263-1-mgorman@techsingularity.net> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669122784; a=rsa-sha256; cv=none; b=1KVNQQ78cVtxQOLB3Tg7SPlPTSEdo7LmmC4aCieyrTsI9SHyPjb0EKDpk19OVKZxqh1fz1 2bOJ6WEzkMhuhjlsYtB9S7PZwZt3hZOolr6X756e+FuB/o8yxgfGjw/hY1mMpvZOypglg2 OOEZamIHNY1aK8iZMTCMjOfkN293St0= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; spf=pass (imf27.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.139.232 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669122784; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XK344uuYkxcmc+eZLLnSr5Eg380aQR5V754TdgFCINA=; b=2lbnhudjFVQhOj6TZJsezTCTuojrmc1py3u5XOFlice1w9/AIQJ9Kog7BZ5SjyYHQ0yOMH PX3pMJF0U842WNNPniReKRo6iOwVD1fCn3cWETXXYeZ34iJubhjFCbEMQOXFZNu4TSV+2m YCexPSrr6C9f5+von79FFUnjk34NtTE= X-Stat-Signature: 8tsjhdjgsrzziwu9d9zpdqefifo9mozf X-Rspamd-Queue-Id: 8323C40011 X-Rspam-User: Authentication-Results: imf27.hostedemail.com; dkim=none; spf=pass (imf27.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.139.232 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none X-Rspamd-Server: rspam02 X-HE-Tag: 1669122784-435526 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: While freeing a large list, the zone lock will be released and reacquired to avoid long hold times since commit c24ad77d962c ("mm/page_alloc.c: avoid excessive IRQ disabled times in free_unref_page_list()"). As suggested by Vlastimil Babka, the lockrelease/reacquire logic can be simplified by reusing the logic that acquires a different lock when changing zones. Signed-off-by: Mel Gorman Reviewed-by: Vlastimil Babka --- mm/page_alloc.c | 25 +++++++++---------------- 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 445066617204..08e32daf0918 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3518,13 +3518,19 @@ void free_unref_page_list(struct list_head *list) list_del(&page->lru); migratetype = get_pcppage_migratetype(page); - /* Different zone, different pcp lock. */ - if (zone != locked_zone) { + /* + * Either different zone requiring a different pcp lock or + * excessive lock hold times when freeing a large list of + * pages. + */ + if (zone != locked_zone || batch_count == SWAP_CLUSTER_MAX) { if (pcp) { pcp_spin_unlock(pcp); pcp_trylock_finish(UP_flags); } + batch_count = 0; + /* * trylock is necessary as pages may be getting freed * from IRQ or SoftIRQ context after an IO completion. @@ -3539,7 +3545,6 @@ void free_unref_page_list(struct list_head *list) continue; } locked_zone = zone; - batch_count = 0; } /* @@ -3551,19 +3556,7 @@ void free_unref_page_list(struct list_head *list) trace_mm_page_free_batched(page); free_unref_page_commit(zone, pcp, page, migratetype, 0); - - /* - * Guard against excessive lock hold times when freeing - * a large list of pages. Lock will be reacquired if - * necessary on the next iteration. - */ - if (++batch_count == SWAP_CLUSTER_MAX) { - pcp_spin_unlock(pcp); - pcp_trylock_finish(UP_flags); - batch_count = 0; - pcp = NULL; - locked_zone = NULL; - } + batch_count++; } if (pcp) {