From patchwork Fri Jan 4 12:50:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 10748339 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C55E14E5 for ; Fri, 4 Jan 2019 12:53:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 09C852810E for ; Fri, 4 Jan 2019 12:53:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EE5852836F; Fri, 4 Jan 2019 12:53:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 62AB02810E for ; Fri, 4 Jan 2019 12:53:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 77B828E00E6; Fri, 4 Jan 2019 07:53:48 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 72B258E00AE; Fri, 4 Jan 2019 07:53:48 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 61ADA8E00E6; Fri, 4 Jan 2019 07:53:48 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 057328E00AE for ; Fri, 4 Jan 2019 07:53:48 -0500 (EST) Received: by mail-ed1-f72.google.com with SMTP id b3so35181738edi.0 for ; Fri, 04 Jan 2019 04:53:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=qvPONPnRj0V1IpypwsEizwqGPfeoFo2DW9REbulI7jQ=; b=SHwLHK5MAEmuJEAJytR2uQD5maUZDNWuNsM8VT6wR+r5IPaArxUz24fsGTM+Ec2b4t PZe700cVqi5LYRTT5f8Vk9GdcLpSu9FD8Y43oFelcmtm4x76n1pwqXkyNKgvPu/iMKyh zkkOASDb+8+FfRI/QEW0i6Uk34LlqXL9rz1P+MK0mjcBD4/jKtaXgcXRAMwKk2MJ6pVM hhVjRmT2pDo2+4m6fyCc62XVfalZP6BNZgI+0raN+rCiaBj7mfQrdhNYpLob9AdQGAwd Ta1c3heXAgzlTE4waRLJ0ZlYNsf9swO5WbJUtTa2pxh62W6tJxH7iLMvT7liAYbwgmYW Ntrg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.15 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net X-Gm-Message-State: AA+aEWZYV2yf0hl3pb8v65zH637bpyJkaOb8z+SgcnFboSco9KJ//hqO kVAuEm7klMRVjwWuUcmMKP13qHW6maSeeBpXgr5o6XIzNZmux7DavwMpCsrEO0tK3KHIQo5Sil+ qhz6O42E4EG8XqcI9bTWdX97WIDJ+N4KX0d30Sl9qGKDfs/DvrMm9JURb+3rXD11rFg== X-Received: by 2002:a50:b902:: with SMTP id m2mr44510589ede.108.1546606427503; Fri, 04 Jan 2019 04:53:47 -0800 (PST) X-Google-Smtp-Source: AFSGD/XYnlOnU4oDRSPptjj/+qXI5VacQO9ua6pW1lcXJznrey22hngZiBPcU2WsuMWQDWm6NVeR X-Received: by 2002:a50:b902:: with SMTP id m2mr44510547ede.108.1546606426448; Fri, 04 Jan 2019 04:53:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1546606426; cv=none; d=google.com; s=arc-20160816; b=rKhLkO8DfAr/q3Th70MtfLwYrbZqE4M65jKn//bNo0VwY1V58mjuhOiVomTiZl2Kll XQgx9zh1KVyFeuly8YtuIwGflWzLqSzMWjepD5RmO6NooTBNNbxPE+eo4Q6nnmbHM9jB s6qbAgF2EaEgJD3u3fOfm9hExtrT9QpeEuInEV7bpXSZ6Spgx2mp0lgQOyN1sWNMaexY TFByYi9RXExzIgb7AnyxEVoBE1AXBvqk/uuU83blM7jj+hXMkZN95AbGkSXWmH93NwI/ F1eT/cLnBx7IFYYlXDcrxNxZGrDwlPUIf3oPvFgD/XdYk/gE1/o5Mt3CxOT8tlDliAFY r9rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=qvPONPnRj0V1IpypwsEizwqGPfeoFo2DW9REbulI7jQ=; b=Q/ji2aOdY4XFe3JK/60TLxpD1jKoM+oe9ivQEWg+r3T6KdcB+BFC4xIqIaIdvGp7pc 8zJIhEC+aFB6C//UjwNglqQbyejtngsfpic03hrXGyopVmxPVI2TOwe7ebqmwhi1MH0e ZxtFyrdBxu28rjUKnEkxnF0eVL7HoLOdvBDLpomIg+/wOktH/vfNekRWPwT/P5o+L5KB IpX0KYiDSgaV3AxeNZSvKzolfvdtchOJrcBfg/+4rz+YK2XfpRBrat8x3xNkXAYU8baf jM2c+G7upgTFPEUJA6E1LsTBdnyvQTcU0QjyfCnzccqegY2R/h890P9dKGYpb7YqY7/D 4osA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.15 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net Received: from outbound-smtp10.blacknight.com (outbound-smtp10.blacknight.com. [46.22.139.15]) by mx.google.com with ESMTPS id b56si4122501eda.336.2019.01.04.04.53.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Jan 2019 04:53:46 -0800 (PST) Received-SPF: pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.15 as permitted sender) client-ip=46.22.139.15; Authentication-Results: mx.google.com; spf=pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.15 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net Received: from mail.blacknight.com (pemlinmail03.blacknight.ie [81.17.254.16]) by outbound-smtp10.blacknight.com (Postfix) with ESMTPS id 17B421C18D7 for ; Fri, 4 Jan 2019 12:53:46 +0000 (GMT) Received: (qmail 9216 invoked from network); 4 Jan 2019 12:53:46 -0000 Received: from unknown (HELO stampy.163woodhaven.lan) (mgorman@techsingularity.net@[37.228.229.96]) by 81.17.254.9 with ESMTPA; 4 Jan 2019 12:53:45 -0000 From: Mel Gorman To: Linux-MM Cc: David Rientjes , Andrea Arcangeli , Vlastimil Babka , ying.huang@intel.com, kirill@shutemov.name, Andrew Morton , Linux List Kernel Mailing , Mel Gorman Subject: [PATCH 20/25] mm, compaction: Reduce unnecessary skipping of migration target scanner Date: Fri, 4 Jan 2019 12:50:06 +0000 Message-Id: <20190104125011.16071-21-mgorman@techsingularity.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190104125011.16071-1-mgorman@techsingularity.net> References: <20190104125011.16071-1-mgorman@techsingularity.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The fast isolation of pages can move the scanner faster than is necessary depending on the contents of the free list. This patch will only allow the fast isolation to initialise the scanner and advance it slowly. The primary means of moving the scanner forward is via the linear scanner to reduce the likelihood the migration source/target scanners meet prematurely triggering a rescan. 4.20.0 4.20.0 noresched-v2r15 slowfree-v2r15 Amean fault-both-1 0.00 ( 0.00%) 0.00 * 0.00%* Amean fault-both-3 2736.50 ( 0.00%) 2512.53 ( 8.18%) Amean fault-both-5 4133.70 ( 0.00%) 4159.43 ( -0.62%) Amean fault-both-7 5738.61 ( 0.00%) 5950.15 ( -3.69%) Amean fault-both-12 9392.82 ( 0.00%) 8674.38 ( 7.65%) Amean fault-both-18 13257.15 ( 0.00%) 12850.79 ( 3.07%) Amean fault-both-24 16859.44 ( 0.00%) 17242.86 ( -2.27%) Amean fault-both-30 16249.30 ( 0.00%) 19404.18 * -19.42%* Amean fault-both-32 14904.71 ( 0.00%) 16200.79 ( -8.70%) The impact to latency, success rates and scan rates is marginal but avoiding unnecessary restarts is important. It helps later patches that are more careful about how pageblocks are treated as earlier iterations of those patches hit corner cases where the restarts were punishing and very visible. Signed-off-by: Mel Gorman Acked-by: Vlastimil Babka --- mm/compaction.c | 27 ++++++++++----------------- 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 75eb0d40d4d7..6c5552c6d8f9 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -324,10 +324,9 @@ static void update_cached_migrate(struct compact_control *cc, unsigned long pfn) * future. The information is later cleared by __reset_isolation_suitable(). */ static void update_pageblock_skip(struct compact_control *cc, - struct page *page, unsigned long nr_isolated) + struct page *page, unsigned long pfn) { struct zone *zone = cc->zone; - unsigned long pfn; if (cc->no_set_skip_hint) return; @@ -335,13 +334,8 @@ static void update_pageblock_skip(struct compact_control *cc, if (!page) return; - if (nr_isolated) - return; - set_pageblock_skip(page); - pfn = page_to_pfn(page); - /* Update where async and sync compaction should restart */ if (pfn < zone->compact_cached_free_pfn) zone->compact_cached_free_pfn = pfn; @@ -359,7 +353,7 @@ static inline bool pageblock_skip_persistent(struct page *page) } static inline void update_pageblock_skip(struct compact_control *cc, - struct page *page, unsigned long nr_isolated) + struct page *page, unsigned long pfn) { } @@ -450,7 +444,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc, bool strict) { int nr_scanned = 0, total_isolated = 0; - struct page *cursor, *valid_page = NULL; + struct page *cursor; unsigned long flags = 0; bool locked = false; unsigned long blockpfn = *start_pfn; @@ -477,9 +471,6 @@ static unsigned long isolate_freepages_block(struct compact_control *cc, if (!pfn_valid_within(blockpfn)) goto isolate_fail; - if (!valid_page) - valid_page = page; - /* * For compound pages such as THP and hugetlbfs, we can save * potentially a lot of iterations if we skip them at once. @@ -576,10 +567,6 @@ static unsigned long isolate_freepages_block(struct compact_control *cc, if (strict && blockpfn < end_pfn) total_isolated = 0; - /* Update the pageblock-skip if the whole pageblock was scanned */ - if (blockpfn == end_pfn) - update_pageblock_skip(cc, valid_page, total_isolated); - cc->total_free_scanned += nr_scanned; if (total_isolated) count_compact_events(COMPACTISOLATED, total_isolated); @@ -1295,8 +1282,10 @@ fast_isolate_freepages(struct compact_control *cc) } } - if (highest && highest > cc->zone->compact_cached_free_pfn) + if (highest && highest >= cc->zone->compact_cached_free_pfn) { + highest -= pageblock_nr_pages; cc->zone->compact_cached_free_pfn = highest; + } cc->total_free_scanned += nr_scanned; if (!page) @@ -1376,6 +1365,10 @@ static void isolate_freepages(struct compact_control *cc) isolate_freepages_block(cc, &isolate_start_pfn, block_end_pfn, freelist, false); + /* Update the skip hint if the full pageblock was scanned */ + if (isolate_start_pfn == block_end_pfn) + update_pageblock_skip(cc, page, block_start_pfn); + /* Are enough freepages isolated? */ if (cc->nr_freepages >= cc->nr_migratepages) { if (isolate_start_pfn >= block_end_pfn) {