From patchwork Wed Sep 28 22:33:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Berger X-Patchwork-Id: 12993207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9130EC32771 for ; Wed, 28 Sep 2022 22:34:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 27D546B0083; Wed, 28 Sep 2022 18:34:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 22D956B0085; Wed, 28 Sep 2022 18:34:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A6F06B0087; Wed, 28 Sep 2022 18:34:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F141F6B0083 for ; Wed, 28 Sep 2022 18:34:07 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C94D840F54 for ; Wed, 28 Sep 2022 22:34:07 +0000 (UTC) X-FDA: 79962948534.27.D8BCCA9 Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) by imf17.hostedemail.com (Postfix) with ESMTP id 3FADB4000E for ; Wed, 28 Sep 2022 22:34:07 +0000 (UTC) Received: by mail-qv1-f41.google.com with SMTP id ml1so9061757qvb.1 for ; Wed, 28 Sep 2022 15:34:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=hndRuqLYmpsdza/I5YqR10YZRjB6Gw7lqCVZwv7bT84=; b=W5KTlkD6VDvTed2L2KfHwXo9j86RBLv0DNqupCjdd7UJXUFKgQ6ICn1v22QPaFF653 UUH7LRcXhW62qq39IHwi34e5XSq8nSnOM5SAyWJvUpLscbaqAkYsuFZsSYCAiBqDk+WE 12K8uhBSmyEFWms9uxioDGzI3h9oFNasFPBomC7bWYUMDQ+i5OMAZnByjvqWrIn0HhIf KJhuzCmgX/7mBYgrmftfQadDSto2kyNLE2imb9nYnmLFfllPp4mAvrPZGMDuVzxUOjDG 9lGghwppmTYcB6NOl9/Hb1f4XFaxXyms0ePckizh45ccJw6S8t2QhVjg7VgrvpJxSSdo T/Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=hndRuqLYmpsdza/I5YqR10YZRjB6Gw7lqCVZwv7bT84=; b=svnfyc/42SDMXIXCV08L5sNwSqiEl/cPmWnL86La0XOZuP7Pz4WBIodg4U5rxk5GEg fM+D0MK+AINufeAZN9w+UUn2sWSsS7wx6bera2nZ8FRrpFyPQe5+lzvq/vZj09y8dvP9 WBYOQ7SOjqmlKr9AYP2vE2fM4nl86KxMqHfn+XibouGuzvVyOw25nVFtwAT5IkiwSgrc 0S4EfkNeVmoeBTe5pl1Tj6cGuy1GVm7q42kue6mP2Lp95B8xBw50tnWEmocMXyi57a4S 1pufjhaRPe7CGpLufeVhV5MBLQ989YqBy+wufMB8WwH9QcRrVaknnzP4fpb23xpeXkCQ PK6g== X-Gm-Message-State: ACrzQf3qJoRzcxF5DlJFfDdp1XzXfsATC4VPwG3bkkfEDj6vsbl//Hs9 tMIL1eekZWTSfljsNIZ+A6o= X-Google-Smtp-Source: AMsMyM6Ccu/5uERbT7jxjfR+0FE1IFewQaesnxuWr2WVMdlCZf4bfvdaCH8iSFcEiEy/T6yQAHFDhA== X-Received: by 2002:ad4:4ea7:0:b0:4ad:656a:260c with SMTP id ed7-20020ad44ea7000000b004ad656a260cmr268848qvb.51.1664404446501; Wed, 28 Sep 2022 15:34:06 -0700 (PDT) Received: from stbirv-lnx-3.igp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j188-20020a37b9c5000000b006bb83c2be40sm3963481qkf.59.2022.09.28.15.34.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Sep 2022 15:34:05 -0700 (PDT) From: Doug Berger To: Andrew Morton Cc: Jonathan Corbet , Mike Rapoport , Borislav Petkov , "Paul E. McKenney" , Neeraj Upadhyay , Randy Dunlap , Damien Le Moal , Muchun Song , KOSAKI Motohiro , Mel Gorman , Mike Kravetz , Florian Fainelli , David Hildenbrand , Oscar Salvador , Michal Hocko , Joonsoo Kim , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Doug Berger Subject: [PATCH v2 8/9] mm/page_alloc: make alloc_contig_pages DMB aware Date: Wed, 28 Sep 2022 15:33:00 -0700 Message-Id: <20220928223301.375229-9-opendmb@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220928223301.375229-1-opendmb@gmail.com> References: <20220928223301.375229-1-opendmb@gmail.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664404447; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hndRuqLYmpsdza/I5YqR10YZRjB6Gw7lqCVZwv7bT84=; b=t22K2ngKuegFEoitTJItI92k4bwV5I8k5MFFZTfPv845rdE6d2BuisnzQS3CEd5lEqgsl/ lrHBmpaBrouhxwyMmkgNT6bzKFkpZpCl0fD4l48khs5qWcUUMCVkfd9t+RArMxKq14ctnx 3qeWNlf1kWbVfvIHm9L8zg4Nl01KYQE= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=W5KTlkD6; spf=pass (imf17.hostedemail.com: domain of opendmb@gmail.com designates 209.85.219.41 as permitted sender) smtp.mailfrom=opendmb@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664404447; a=rsa-sha256; cv=none; b=nw9iop4sAFciCoocymE6+nhI7GpnsSYWZLkYt3OtRqp4H51PwNhUZTTsYrWbMEDwpmYMlp W4CFejYuHP4VEAXbNBPtY74RssxOdSDEVLtJZ1ijqFlzt+9p0C0DVhe4mM/ALgNLKPb2+6 +gbROY5b6Q9j9+LVDrJ0xBGVPTueFVw= X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3FADB4000E Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=W5KTlkD6; spf=pass (imf17.hostedemail.com: domain of opendmb@gmail.com designates 209.85.219.41 as permitted sender) smtp.mailfrom=opendmb@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: 14r5jgs6jc38i97knur4xbt7h9pxcjuz X-HE-Tag: 1664404447-139921 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Designated Movable Blocks are skipped when attempting to allocate contiguous pages. Doing per page validation across all spanned pages within a zone can be extra inefficient when Designated Movable Blocks create large overlaps between zones. Use dmb_intersects() within pfn_range_valid_contig as an early check to signal the range is not valid. The zone_movable_pfn array which represents the start of non- overlapped ZONE_MOVABLE on the node is now preserved to be used at runtime to skip over any DMB-only portion of the zone. Signed-off-by: Doug Berger --- mm/page_alloc.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cd31f26b0d21..c07111a897c0 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -435,7 +435,7 @@ static unsigned long required_kernelcore_percent __initdata; static unsigned long required_movablecore __initdata; static unsigned long required_movablecore_percent __initdata; static unsigned long min_dmb_pfn[MAX_NUMNODES] __initdata; -static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata; +static unsigned long zone_movable_pfn[MAX_NUMNODES]; bool mirrored_kernelcore __initdata_memblock; /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */ @@ -9369,6 +9369,9 @@ static bool pfn_range_valid_contig(struct zone *z, unsigned long start_pfn, unsigned long i, end_pfn = start_pfn + nr_pages; struct page *page; + if (dmb_intersects(start_pfn, end_pfn)) + return false; + for (i = start_pfn; i < end_pfn; i++) { page = pfn_to_online_page(i); if (!page) @@ -9425,7 +9428,10 @@ struct page *alloc_contig_pages(unsigned long nr_pages, gfp_t gfp_mask, gfp_zone(gfp_mask), nodemask) { spin_lock_irqsave(&zone->lock, flags); - pfn = ALIGN(zone->zone_start_pfn, nr_pages); + if (zone_idx(zone) == ZONE_MOVABLE && zone_movable_pfn[nid]) + pfn = ALIGN(zone_movable_pfn[nid], nr_pages); + else + pfn = ALIGN(zone->zone_start_pfn, nr_pages); while (zone_spans_last_pfn(zone, pfn, nr_pages)) { if (pfn_range_valid_contig(zone, pfn, nr_pages)) { /*