From patchwork Fri Jun 15 15:57:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Tatashin X-Patchwork-Id: 10466837 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 870F960532 for ; Fri, 15 Jun 2018 15:58:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7603628D98 for ; Fri, 15 Jun 2018 15:58:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A97528DFD; Fri, 15 Jun 2018 15:58:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 049EA28DF8 for ; Fri, 15 Jun 2018 15:58:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0761D6B0003; Fri, 15 Jun 2018 11:58:06 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 025576B0006; Fri, 15 Jun 2018 11:58:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E57D06B0007; Fri, 15 Jun 2018 11:58:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f197.google.com (mail-qk0-f197.google.com [209.85.220.197]) by kanga.kvack.org (Postfix) with ESMTP id BE5056B0003 for ; Fri, 15 Jun 2018 11:58:05 -0400 (EDT) Received: by mail-qk0-f197.google.com with SMTP id i64-v6so8258750qkh.14 for ; Fri, 15 Jun 2018 08:58:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id; bh=yUWqIp9+8RD7OjliaE/vj9Mpm4ihOlswZO8+N4srvus=; b=uWDUzTnPLNAx6K8ycAc/kINAWL+1jpA9Q0ZsGFGtk+u5+fCVT7v27F/GF7n4hElINW M7e9iNEU/jXu/IU6dnPHQPXWnmobhzN1i30NZO5WRqYOdNG2+RbfYVDvE8Ehb6mSiQ6l F6uoQMitXyY8iE61gpc225A6kHo6nF3885xWMmoTqVGtJ/vKihfr+ADfqiXtgtWZKY69 8XsMmBns8ZoGJudceuP6x8JTJ4Y/NmFFr+Ufht/Pbkg3EaONzQmzLJTE3p0dlrZPL/Ce oW4Mw1aExxP6kw0JZFp29GvVJpbyIjoYyeRf2Yg2FY9k6PAfoKzVm3UTniYSNcjaDubM 709Q== X-Gm-Message-State: APt69E1hWvAts5hJwtuREB64PALArIYIXFwosS9M7vfvhIP1GeSv6Scd kmfPuuIrF+Cog7zvvvHBhvXYU/uhs1Zvep5Yw/pPpg7XJJXEZrg5ixmocpB76JDQYZM12+Z0IYS GK3YFLgTrUQLDbsvAU+te8vKEy+mRpSQoXhLpS8idejlEZC0cutWEj4Y3VwePe7pAHw== X-Received: by 2002:ac8:16b3:: with SMTP id r48-v6mr1974368qtj.273.1529078285566; Fri, 15 Jun 2018 08:58:05 -0700 (PDT) X-Google-Smtp-Source: ADUXVKK/zb7cr177JdXyOanj/ZegpZhFKtdd6WHVxDdT41HSDcNeLEuPpU90TYLL8pVvUVkQ1TXE X-Received: by 2002:ac8:16b3:: with SMTP id r48-v6mr1974334qtj.273.1529078285008; Fri, 15 Jun 2018 08:58:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529078284; cv=none; d=google.com; s=arc-20160816; b=sXI4JwEvF4jgmbNXDhRho8Wh5Vcq3D6QKknde0Do2UPbVddio0bHN34PLdm1k/Tske nqOC2e8guigluZGEeXvUoS7gXPUDnCB0NS2WDZl8+vo3hCv+TEvxgGE5aX5w2TBOOMAd sZs5/m7KZhtBW66oJ8sT5tQnH7EI8gPnyj8lU9nfuSqHjyiZZ0uYZ1ddK5w3VbLRdExb 3lpnMrt51JTL+yP8RQ+rwBMEBr42FJj8Z9id7XPr3iQWzlP4szyHtMIRL8KK4YDG+k2b W23HD1Eh7yzCpFhHybMQUGgW1kdjVz++YDa6lb7B9CeCjMhTN6WGJwcc3DF20FHrwDie XHWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:to:from:dkim-signature :arc-authentication-results; bh=yUWqIp9+8RD7OjliaE/vj9Mpm4ihOlswZO8+N4srvus=; b=W0RLSt5OVr9nLBZZXyO8NSuV2g8BOPmCrqo82yLhZcaIpRw79lm2lHPmPTDbjRThd9 Uz9uQGIdwTmO0qV8zD/Rt6WqKfinNNvzgPd3CVsp1paepXIus0HLCyL3x0d30N1E6Put 1uiyt+QYn+cnL+tFZM9wHapiWqjOFL/OP1bHslmD2HdsgBAEEyxLyexxQ3AUByC+8ACc QdZK5SqdShRhDYdUwbEUL/SKy+/RRVI4rH1MMi5t65hCVbYJwWlwk42Lhij3q6c//2fH RVmSYnbwSsvNVp4mAwwbnv05CznHRDhHCdh9/HNOZzlJB1/b9DPvCp2qWRlbUZ/hlbiC aPsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2017-10-26 header.b="E/vIE3kE"; spf=pass (google.com: domain of pasha.tatashin@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=pasha.tatashin@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id t127-v6si5809796qkc.292.2018.06.15.08.58.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Jun 2018 08:58:04 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2017-10-26 header.b="E/vIE3kE"; spf=pass (google.com: domain of pasha.tatashin@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=pasha.tatashin@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w5FFsT8M069912; Fri, 15 Jun 2018 15:57:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id; s=corp-2017-10-26; bh=yUWqIp9+8RD7OjliaE/vj9Mpm4ihOlswZO8+N4srvus=; b=E/vIE3kEeE3sd8WX1d1/I8VDn4TTYq0r0VSY0hQOrv67kBnzOmVEmdaHiHWAqt10T5lo R5oB2i58cOXDEKoLXz9oiBBeFOU12W6EJ3K9SjZ/XNdjoe1+SADRM4k7fwjzUXM94j1E zD9+SKJXcd1eiHZQP1pzuR4IN8wXl2Whdu3A4Er5nXV6zP6aU9bVW6Gv8MwKnXUk3ZFL DyOczKwtEw8zKwAqTLRKRlrmP5CGDTd9vQKSLkb0rRnvuRx2+QkAdK6VECETqt5kTWb/ J2n/L2esp69QW/v3gRR0STl/DhauGuwCz6O/3/8TG+YkCJQymMwfiPFVwzbdyhCfkGyE dg== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2130.oracle.com with ESMTP id 2jk0xr1k28-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Jun 2018 15:57:42 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w5FFvfGm021529 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Jun 2018 15:57:42 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w5FFveue022206; Fri, 15 Jun 2018 15:57:40 GMT Received: from localhost.localdomain (/73.69.118.222) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Jun 2018 08:57:40 -0700 From: Pavel Tatashin To: steven.sistare@oracle.com, daniel.m.jordan@oracle.com, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, mhocko@suse.com, n-horiguchi@ah.jp.nec.com, linux-mm@kvack.org, osalvador@suse.de, willy@infradead.org, mingo@kernel.org, dan.j.williams@intel.com, ying.huang@intel.com Subject: [PATCH] mm: skip invalid pages block at a time in zero_resv_unresv Date: Fri, 15 Jun 2018 11:57:33 -0400 Message-Id: <20180615155733.1175-1-pasha.tatashin@oracle.com> X-Mailer: git-send-email 2.17.1 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8924 signatures=668702 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1805220000 definitions=main-1806150171 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The role of zero_resv_unavail() is to make sure that every struct page that is allocated but is not backed by memory that is accessible by kernel is zeroed and not in some uninitialized state. Since struct pages are allocated in blocks (2M pages in x86 case), we can skip pageblock_nr_pages at a time, when the first one is found to be invalid. This optimization may help since now on x86 every hole in e820 maps is marked as reserved in memblock, and thus will go through this function. This function is called before sched_clock() is initialized, so I used my x86 early boot clock patches to measure the performance improvement. With 1T hole on i7-8700 currently we would take 0.606918s of boot time, but with this optimization 0.001103s. Signed-off-by: Pavel Tatashin Reviewed-by: Oscar Salvador Reviewed-by: Naoya Horiguchi --- mm/page_alloc.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1521100f1e63..94f1b3201735 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6404,8 +6404,11 @@ void __paginginit zero_resv_unavail(void) pgcnt = 0; for_each_resv_unavail_range(i, &start, &end) { for (pfn = PFN_DOWN(start); pfn < PFN_UP(end); pfn++) { - if (!pfn_valid(ALIGN_DOWN(pfn, pageblock_nr_pages))) + if (!pfn_valid(ALIGN_DOWN(pfn, pageblock_nr_pages))) { + pfn = ALIGN_DOWN(pfn, pageblock_nr_pages) + + pageblock_nr_pages - 1; continue; + } mm_zero_struct_page(pfn_to_page(pfn)); pgcnt++; }