From patchwork Fri Jan 4 12:49:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 10748311 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D084D13B5 for ; Fri, 4 Jan 2019 12:51:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C11FB282E8 for ; Fri, 4 Jan 2019 12:51:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B2DB9283C9; Fri, 4 Jan 2019 12:51:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3556F282E8 for ; Fri, 4 Jan 2019 12:51:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6948A8E00DB; Fri, 4 Jan 2019 07:51:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 61B1F8E00AE; Fri, 4 Jan 2019 07:51:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50B398E00DB; Fri, 4 Jan 2019 07:51:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id E6CF88E00AE for ; Fri, 4 Jan 2019 07:51:55 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id l45so35123592edb.1 for ; Fri, 04 Jan 2019 04:51:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=8kl/Dp+/KzdTN0WWTLWu4uZwc5c5hUQZGkU37edu/xc=; b=sEbAUHDGgLn02Et+NQacob0kCVFuUph79Y9DyWo0i1Ku51gNLKEzZvoyx2AVz7M14S NHQZOOxnBTJ9XbUTI7Zu37Dt8ncFmBdOU9EocB506/Jj6OMAxpG+Os4tWl1ed/fvL+9J 8lubSS3f094MkpZgbs6jVy+A4yXLg9LuedAPqxXBO2YY6L05YOPmZtNGYX6L3YTH7sWz J0tcZTsgnLwjXkHfbP3cVDb7uffY/yghiKEFzg2SzD7c5BlehW9YVK7aTD3ug8jIHvbo TM+WX2jWTDyJw+FI4L1BfOHFadMg+Elu/z9CmwVe6hKkT2P25Y0ekQFyQJyCCLK/TEzW IvWQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.17 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net X-Gm-Message-State: AA+aEWZY0SuQf1NAcsWexE3hctEvzCPXJWa98ommCyquXYxHU4a3idPO xQ97ZMZpKV/Zfigy8SXx5MjEh78YBzvSd/bHSeQljGbfL6+R9i+vZ4LADIncEEh6lhRzppvdSWg u48/B5BIVBGCwq8nATZLQQx8a7CEzb3Qu4uk4kXrSZkKsbxGj9hjBxsnXEcJnbDlKsQ== X-Received: by 2002:a17:906:2972:: with SMTP id x18-v6mr39434227ejd.28.1546606315406; Fri, 04 Jan 2019 04:51:55 -0800 (PST) X-Google-Smtp-Source: AFSGD/WBbqcRetPE/kOJezIOCmvk43IZYISfWTT7KbfKJd6POuDv/+TM4bZ84VDXAXsde5wCkBWs X-Received: by 2002:a17:906:2972:: with SMTP id x18-v6mr39434176ejd.28.1546606314356; Fri, 04 Jan 2019 04:51:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1546606314; cv=none; d=google.com; s=arc-20160816; b=ltpS3CJ5589zc3FM6ypyOqEcEnTigpsA/mZsekNShqp04yPaAAtJvVHxjJpqkS8xLf yu/kBNNz7TLjm7Hzrb6YzI/jz0ZY3tp3c7rKw97h8WEQ2YIfYICihjTFdfvysE+Fi8AO 8b1g1/kIi7eYNAhES0q65TlmfxenXg/Gy6SiOjQLfpnJiXnmK5Afzlswpf5brXk5I0xn ZD4+N5hjF43F727wHzbodUPIF5kETS7Io7Icqew5Irmb3Z5g4nylVzjM1Ec4PuRewqxl IhJs3PWxUcX9dEKm+1BrNJCJUJxZ6G+ORN2Hb5P1Tv7RAsjxPYntM+SUU1Nrvjlna3dS jgWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=8kl/Dp+/KzdTN0WWTLWu4uZwc5c5hUQZGkU37edu/xc=; b=eAB3Z60xs2z9DWbsOaBin1JfciUhmisnsSbw7xGVGWzl21/cMFKJnEf+NRInC18+6s oQwQQ+JqiqmVIlSYGQhH0b8ExX6JMz4h5JZ88q8VUGUti3Q3hQsEDGcX6LUpOJ00eCZ1 1YuCIk6HbzPq1lcXx0mIbFClWkUZ/c/MH4VMAT1R9pAE63SE6EnlX0h/PeLs/iijBDxF OOrCwnWr/ijAAsMhWFB4ZlWFK1xxoiDI6RCMnYByO6PrSKjINQqImbx3pYIFrEaNr4Q2 GYCAjHQI1Vgt0v0LfDuLK9XbdcUVq0moE1HTiw+0hA4r1OikMyVtHgHRIpvVWCDve4M8 zvsQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.17 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net Received: from outbound-smtp12.blacknight.com (outbound-smtp12.blacknight.com. [46.22.139.17]) by mx.google.com with ESMTPS id j8si668671edh.289.2019.01.04.04.51.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Jan 2019 04:51:54 -0800 (PST) Received-SPF: pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.17 as permitted sender) client-ip=46.22.139.17; Authentication-Results: mx.google.com; spf=pass (google.com: domain of mgorman@techsingularity.net designates 46.22.139.17 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net Received: from mail.blacknight.com (pemlinmail03.blacknight.ie [81.17.254.16]) by outbound-smtp12.blacknight.com (Postfix) with ESMTPS id 015471C1E3F for ; Fri, 4 Jan 2019 12:51:54 +0000 (GMT) Received: (qmail 1701 invoked from network); 4 Jan 2019 12:51:53 -0000 Received: from unknown (HELO stampy.163woodhaven.lan) (mgorman@techsingularity.net@[37.228.229.96]) by 81.17.254.9 with ESMTPA; 4 Jan 2019 12:51:53 -0000 From: Mel Gorman To: Linux-MM Cc: David Rientjes , Andrea Arcangeli , Vlastimil Babka , ying.huang@intel.com, kirill@shutemov.name, Andrew Morton , Linux List Kernel Mailing , Mel Gorman Subject: [PATCH 09/25] mm, compaction: Use the page allocator bulk-free helper for lists of pages Date: Fri, 4 Jan 2019 12:49:55 +0000 Message-Id: <20190104125011.16071-10-mgorman@techsingularity.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190104125011.16071-1-mgorman@techsingularity.net> References: <20190104125011.16071-1-mgorman@techsingularity.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP release_pages() is a simpler version of free_unref_page_list() but it tracks the highest PFN for caching the restart point of the compaction free scanner. This patch optionally tracks the highest PFN in the core helper and converts compaction to use it. The performance impact is limited but it should reduce lock contention slightly in some cases. The main benefit is removing some partially duplicated code. Signed-off-by: Mel Gorman --- include/linux/gfp.h | 7 ++++++- mm/compaction.c | 12 +++--------- mm/page_alloc.c | 10 +++++++++- 3 files changed, 18 insertions(+), 11 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 5f5e25fd6149..9e58799b730f 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -543,7 +543,12 @@ void * __meminit alloc_pages_exact_nid(int nid, size_t size, gfp_t gfp_mask); extern void __free_pages(struct page *page, unsigned int order); extern void free_pages(unsigned long addr, unsigned int order); extern void free_unref_page(struct page *page); -extern void free_unref_page_list(struct list_head *list); +extern void __free_page_list(struct list_head *list, bool dropref, unsigned long *highest_pfn); + +static inline void free_unref_page_list(struct list_head *list) +{ + return __free_page_list(list, false, NULL); +} struct page_frag_cache; extern void __page_frag_cache_drain(struct page *page, unsigned int count); diff --git a/mm/compaction.c b/mm/compaction.c index 8bf2090231a3..8f0ce44dba41 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -52,16 +52,10 @@ static inline void count_compact_events(enum vm_event_item item, long delta) static unsigned long release_freepages(struct list_head *freelist) { - struct page *page, *next; - unsigned long high_pfn = 0; + unsigned long high_pfn; - list_for_each_entry_safe(page, next, freelist, lru) { - unsigned long pfn = page_to_pfn(page); - list_del(&page->lru); - __free_page(page); - if (pfn > high_pfn) - high_pfn = pfn; - } + __free_page_list(freelist, true, &high_pfn); + INIT_LIST_HEAD(freelist); return high_pfn; } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cde5dac6229a..57ba9d1da519 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2876,18 +2876,26 @@ void free_unref_page(struct page *page) /* * Free a list of 0-order pages */ -void free_unref_page_list(struct list_head *list) +void __free_page_list(struct list_head *list, bool dropref, + unsigned long *highest_pfn) { struct page *page, *next; unsigned long flags, pfn; int batch_count = 0; + if (highest_pfn) + *highest_pfn = 0; + /* Prepare pages for freeing */ list_for_each_entry_safe(page, next, list, lru) { + if (dropref) + WARN_ON_ONCE(!put_page_testzero(page)); pfn = page_to_pfn(page); if (!free_unref_page_prepare(page, pfn)) list_del(&page->lru); set_page_private(page, pfn); + if (highest_pfn && pfn > *highest_pfn) + *highest_pfn = pfn; } local_irq_save(flags);