From patchwork Wed Sep 19 03:17:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 10605205 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 067E5112B for ; Wed, 19 Sep 2018 03:18:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA3A72B8AC for ; Wed, 19 Sep 2018 03:18:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DDD962B8FF; Wed, 19 Sep 2018 03:18:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E6602B8AC for ; Wed, 19 Sep 2018 03:18:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0AAC48E0004; Tue, 18 Sep 2018 23:18:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 00BDE8E0001; Tue, 18 Sep 2018 23:18:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E19468E0004; Tue, 18 Sep 2018 23:18:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 9802B8E0001 for ; Tue, 18 Sep 2018 23:18:32 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id b69-v6so2045507pfc.20 for ; Tue, 18 Sep 2018 20:18:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=tFZLsRjo1xZOn/CddR3VikCmk4h0He503FmrRdcNnmw=; b=HfCLhxBmzfTB4TLvLMvS6DUKlAvCiY5YmLZUMInbVbVowCVfR6LPtg6Dij8vlizk6s H7seCuiXN5ficbRcxzQyCO9Udfu3KXve3rfUAfozrcqDpA3NpGjfkCynsMHEKMz//xUT c1QhRp2kWCLX5OiHumIJaCieEMD2ihjIii0IdQ6EaFeYB/xFP5XWd826Ok9RcWT+hjeZ Mxc8Ya5ITTZqKO2l93Vbz409JjlrWnQffQCRpejIYF5vHIkq2aQHrk2gxSQhuLvd64zZ Rqm2tswTT+Ft1XH4rU7PBc3VG8h9kRaS82dDJWbGPzS/2We4iKIIW7SF9SXZFrwW+lM8 qaVw== X-Gm-Message-State: APzg51ClvVuF/U7a7pLpzhsLhKVLRy4dTlh0gQbC07+oWdsjpIgNdxSa N1vbcn38NDd+SkIqudbp5i8JiMKLwuJwGjSzD3847S8rpXtOOffxTbOvVM/T4YNZiG9yi+h79z6 qFoI4zRYbhL74lC8UzSJlUeEswifIO04oT6qOixZnfyPwhv0s3gmqLcoKexjdKmVNg6mA7gFA8v dTAdtiiFJ+b1+dZzCWQVXteb48lynB2JyVq52GynwxDiBUme6dhc/iALU/KVPS33jz2PaO6h77d yAq/8f/BEEutgM4oyLHKCcvWKJcvLFY/lhF1kXcOam74k/dN2UfaZ4Bpk5pvD4wsWfbGslKVpQk i9u0kQO2UI/jOZu2X+B14eEC5/OpKO1CAhnpNqKi8a9hU+pFQDDd0CarP3NGdFeCJesf+jnOM0S / X-Received: by 2002:a63:6849:: with SMTP id d70-v6mr30107531pgc.7.1537327112287; Tue, 18 Sep 2018 20:18:32 -0700 (PDT) X-Received: by 2002:a63:6849:: with SMTP id d70-v6mr30107484pgc.7.1537327111167; Tue, 18 Sep 2018 20:18:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537327111; cv=none; d=google.com; s=arc-20160816; b=yXYaTGaZNX4Z9qyeeutGQLSGz8kD9Ps4BTncFINh1uVIO6uSy+lyuNgjd8jMftodkH NABPrrmnLD3E8NBB8aAz+IJ88dqs37XlSzmYN0Pca4ECketokvIPicfdyNqqGdEh+Ufa 6VjjsUMP1PpFCLJjK7n6tsAr96QJffroF4oy52Lfqor2g67lJA+4ZhY67tA2yyAcKxuR brEBJhos1EomSmIUfZjwRe5Fj6D5Ob640re2AH1817fjFvs95p63os8s6buMBzWHll+6 EMI+ucczMs/0dGf++aqw3qckuvBu5XWEfw+M92KyvW7eGJzW8OiD2xKaIrZQyqK9y9Qj rpIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=tFZLsRjo1xZOn/CddR3VikCmk4h0He503FmrRdcNnmw=; b=vM90w1Mam5duPRQ7UI+a9OG7jqo+N/heoQkxk6TuI0cp77KHBN7CzGDU7HtYTBmEql fdiRm8De+m4aOheGtL9HiHkRUQii55l4S7KpsBIc8FaqA7lXkzDJMXmvMY/ON94+m0p/ 4umcIuDoGQ/JT4rUk30IjSp++aqxZhd8RdcRKcIIqhbFY209FY9RIQr6sNWo3FZN/7JM /GWtdOxQp+eM5RAQL6aBs5TUHkrLeNwj3R8sM6D7BFH/JmpMiRuVaIHX8B0ygd414Jfx jSGHqMyLRCgYFVkhsaGmGrg9lP6BWFlhGnEeTuSd3bNIftLfThWx1V3tRbe1V9VJWqpm vOJA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=M8yNmtoy; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b7-v6sor3893334plx.0.2018.09.18.20.18.30 for (Google Transport Security); Tue, 18 Sep 2018 20:18:31 -0700 (PDT) Received-SPF: pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=M8yNmtoy; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=tFZLsRjo1xZOn/CddR3VikCmk4h0He503FmrRdcNnmw=; b=M8yNmtoyM3dZYI37J9nYwmfjENyH5RM0ghWBNLqTV0LKGCuSuK8BawFFxpzPE+dTyL AblfR71Ry57bED2sczs+/E59H1wfKPyNY4nxVHqIixXeFczzydTHtQcNYki0lUpwGTpB R0v5YoHPU6MY2h0VSzn5fGgXBdR6zrm1dQ9pImaFee2tRCh4EemRoc3G3RfHLV007UEC f5pSQb3gZBLVSDxVfz/c5YSsFUJNQ+mnStt+eEfMWfD3Id6aYimBZ6InJskf+awaoJzw r23EIPpgthYfASMZ9rMC5QRH0zHBe8jDgByx3CB/1bJKe4X1SU8fvR85VQdIyviMQkW1 P4OQ== X-Google-Smtp-Source: ANB0VdbGsTeEzzjb6bAO7GzZjbJwYp2NopC/5k0XctYv/OvCEjXWq5YYXA48ZTChQfyMQLH9Z2R6DA== X-Received: by 2002:a17:902:a504:: with SMTP id s4-v6mr33051978plq.101.1537327110390; Tue, 18 Sep 2018 20:18:30 -0700 (PDT) Received: from mylaptop.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id o20-v6sm53087673pfj.35.2018.09.18.20.18.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Sep 2018 20:18:29 -0700 (PDT) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Andrew Morton , KAMEZAWA Hiroyuki , Mel Gorman , Greg Kroah-Hartman , Pavel Tatashin , Michal Hocko , Bharata B Rao , Dan Williams , "H. Peter Anvin" , "Kirill A . Shutemov" Subject: [PATCH 1/3] mm/isolation: separate the isolation and migration ops in offline memblock Date: Wed, 19 Sep 2018 11:17:44 +0800 Message-Id: <1537327066-27852-2-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1537327066-27852-1-git-send-email-kernelfans@gmail.com> References: <1537327066-27852-1-git-send-email-kernelfans@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The current design of start_isolate_page_range() relies on MIGRATE_ISOLATE to run against other threads. Hence the callers of start_isolate_page_range() can only do the isolation by themselves. But in this series, a suggested mem offline seq splits the pageblock's isolation and migration on a memblock, i.e. -1. call start_isolate_page_range() on a batch of memblock -2. call __offline_pages() on each memblock. This requires the ability to allow __offline_pages() to reuse the isolation About the mark of isolation, it is not preferable to do it in memblock, because at this level, pageblock is used, and the memblock should be hidden. On the other hand, isolation and compaction can not run in parallel, the PB_migrate_skip bit can be reused to mark the isolation result of previous ops, as used by this patch. Also the prototype of start_isolate_page_range() is changed to tell __offline_pages cases from temporary isolation e.g. alloc_contig_range() Signed-off-by: Pingfan Liu Cc: Andrew Morton Cc: KAMEZAWA Hiroyuki Cc: Mel Gorman Cc: Greg Kroah-Hartman Cc: Pavel Tatashin Cc: Michal Hocko Cc: Bharata B Rao Cc: Dan Williams Cc: "H. Peter Anvin" Cc: Kirill A. Shutemov --- include/linux/page-isolation.h | 4 ++-- include/linux/pageblock-flags.h | 2 ++ mm/memory_hotplug.c | 6 +++--- mm/page_alloc.c | 4 ++-- mm/page_isolation.c | 28 +++++++++++++++++++++++----- 5 files changed, 32 insertions(+), 12 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 4ae347c..dcc2bd1 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -47,7 +47,7 @@ int move_freepages_block(struct zone *zone, struct page *page, */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, - unsigned migratetype, bool skip_hwpoisoned_pages); + unsigned int migratetype, bool skip_hwpoisoned_pages, bool reuse); /* * Changes MIGRATE_ISOLATE to MIGRATE_MOVABLE. @@ -55,7 +55,7 @@ start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, */ int undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, - unsigned migratetype); + unsigned int migratetype, bool reuse); /* * Test all pages in [start_pfn, end_pfn) are isolated or not. diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flags.h index 9132c5c..80c5341 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -31,6 +31,8 @@ enum pageblock_bits { PB_migrate_end = PB_migrate + 3 - 1, /* 3 bits required for migrate types */ PB_migrate_skip,/* If set the block is skipped by compaction */ + PB_isolate_skip = PB_migrate_skip, + /* isolation and compaction do not concur */ /* * Assume the bits will always align on a word. If this assumption diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 9eea6e8..228de4d 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1616,7 +1616,7 @@ static int __ref __offline_pages(unsigned long start_pfn, /* set above range as isolated */ ret = start_isolate_page_range(start_pfn, end_pfn, - MIGRATE_MOVABLE, true); + MIGRATE_MOVABLE, true, true); if (ret) return ret; @@ -1662,7 +1662,7 @@ static int __ref __offline_pages(unsigned long start_pfn, We cannot do rollback at this point. */ offline_isolated_pages(start_pfn, end_pfn); /* reset pagetype flags and makes migrate type to be MOVABLE */ - undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE); + undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE, true); /* removal success */ adjust_managed_page_count(pfn_to_page(start_pfn), -offlined_pages); zone->present_pages -= offlined_pages; @@ -1697,7 +1697,7 @@ static int __ref __offline_pages(unsigned long start_pfn, ((unsigned long long) end_pfn << PAGE_SHIFT) - 1); memory_notify(MEM_CANCEL_OFFLINE, &arg); /* pushback to free area */ - undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE); + undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE, true); return ret; } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 05e983f..a0ae259 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7882,7 +7882,7 @@ int alloc_contig_range(unsigned long start, unsigned long end, ret = start_isolate_page_range(pfn_max_align_down(start), pfn_max_align_up(end), migratetype, - false); + false, false); if (ret) return ret; @@ -7967,7 +7967,7 @@ int alloc_contig_range(unsigned long start, unsigned long end, done: undo_isolate_page_range(pfn_max_align_down(start), - pfn_max_align_up(end), migratetype); + pfn_max_align_up(end), migratetype, false); return ret; } diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 43e0856..36858ab 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -15,8 +15,18 @@ #define CREATE_TRACE_POINTS #include +#define get_pageblock_isolate_skip(page) \ + get_pageblock_flags_group(page, PB_isolate_skip, \ + PB_isolate_skip) +#define clear_pageblock_isolate_skip(page) \ + set_pageblock_flags_group(page, 0, PB_isolate_skip, \ + PB_isolate_skip) +#define set_pageblock_isolate_skip(page) \ + set_pageblock_flags_group(page, 1, PB_isolate_skip, \ + PB_isolate_skip) + static int set_migratetype_isolate(struct page *page, int migratetype, - bool skip_hwpoisoned_pages) + bool skip_hwpoisoned_pages, bool reuse) { struct zone *zone; unsigned long flags, pfn; @@ -33,8 +43,11 @@ static int set_migratetype_isolate(struct page *page, int migratetype, * If it is already set, then someone else must have raced and * set it before us. Return -EBUSY */ - if (is_migrate_isolate_page(page)) + if (is_migrate_isolate_page(page)) { + if (reuse && get_pageblock_isolate_skip(page)) + ret = 0; goto out; + } pfn = page_to_pfn(page); arg.start_pfn = pfn; @@ -75,6 +88,8 @@ static int set_migratetype_isolate(struct page *page, int migratetype, int mt = get_pageblock_migratetype(page); set_pageblock_migratetype(page, MIGRATE_ISOLATE); + if (reuse) + set_pageblock_isolate_skip(page); zone->nr_isolate_pageblock++; nr_pages = move_freepages_block(zone, page, MIGRATE_ISOLATE, NULL); @@ -185,7 +200,7 @@ __first_valid_page(unsigned long pfn, unsigned long nr_pages) * prevents two threads from simultaneously working on overlapping ranges. */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, - unsigned migratetype, bool skip_hwpoisoned_pages) + unsigned int migratetype, bool skip_hwpoisoned_pages, bool reuse) { unsigned long pfn; unsigned long undo_pfn; @@ -199,7 +214,8 @@ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, pfn += pageblock_nr_pages) { page = __first_valid_page(pfn, pageblock_nr_pages); if (page && - set_migratetype_isolate(page, migratetype, skip_hwpoisoned_pages)) { + set_migratetype_isolate(page, migratetype, + skip_hwpoisoned_pages, reuse)) { undo_pfn = pfn; goto undo; } @@ -222,7 +238,7 @@ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, * Make isolated pages available again. */ int undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, - unsigned migratetype) + unsigned int migratetype, bool reuse) { unsigned long pfn; struct page *page; @@ -236,6 +252,8 @@ int undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, page = __first_valid_page(pfn, pageblock_nr_pages); if (!page || !is_migrate_isolate_page(page)) continue; + if (reuse) + clear_pageblock_isolate_skip(page); unset_migratetype_isolate(page, migratetype); } return 0;