From patchwork Wed Jul 24 17:50:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 11057471 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7FA0138D for ; Wed, 24 Jul 2019 17:50:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96DD0208C2 for ; Wed, 24 Jul 2019 17:50:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8A8F02875F; Wed, 24 Jul 2019 17:50:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4DD16208C2 for ; Wed, 24 Jul 2019 17:50:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D93EB8E000B; Wed, 24 Jul 2019 13:50:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D466D8E0005; Wed, 24 Jul 2019 13:50:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C322C8E000B; Wed, 24 Jul 2019 13:50:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ua1-f69.google.com (mail-ua1-f69.google.com [209.85.222.69]) by kanga.kvack.org (Postfix) with ESMTP id 9D3818E0005 for ; Wed, 24 Jul 2019 13:50:34 -0400 (EDT) Received: by mail-ua1-f69.google.com with SMTP id p13so4782888uad.11 for ; Wed, 24 Jul 2019 10:50:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NX/s/A5OLVDNZD4fNK87u3UpZfdfRBOaB3nXPLcsPas=; b=E3/BVO0tchn7ogJimN9dpVMOjdokSv8yq+vkJJLImwL0gEok8UePxZhe4LtFhZHnRb vtLSsVZ+O+P2zGwcJU3Aqqothkt1xgBRfr/slz2EssPghlJkB8xTh9lPGIdsu4cVkESz s5sFShbdJWAtGeW4nouePPIhtEyk1UdRUtpYsSscLfBcgdM/bWXI0qymgeyZdJWoyima T+q5zArT3H4feYVS8OeFNYI6CMaJGYg1BMoh6/RpPX7F7Yfvmr7dDLbc0HizbaXjDDnC HWO3bRrya7y1z0Gm5dcqNmKb9bnYSy/abJSbN/Bx9+XETh2ema/TEoCKqyRS/0qY3Cmv 9nMA== X-Gm-Message-State: APjAAAXrJhL7n6a+LX31PfC/MjVFDirMWa0yKB0iOIwWnShnMq7yi6oh hnuvc3BoAcqchRJQKw0YGgZSX+GpnCDo4tsB/VLWO08KOBZBGil9N90tdU/25+nQoCSAokDdRWE TCG32lLaWYe6ujCVTMHH1Ue06Z4Ry2Eq6clgCiukxEWMm3hU7aaEO0UM+ljOgVMYO/Q== X-Received: by 2002:a67:d58a:: with SMTP id m10mr54727041vsj.15.1563990634285; Wed, 24 Jul 2019 10:50:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqxvRjxzKbb2lZC8ZAmdao3Mbju+kbxkxlk02s32HOzywS37mtRymLdMpj4FPlJYgCXGsuCo X-Received: by 2002:a67:d58a:: with SMTP id m10mr54726999vsj.15.1563990633775; Wed, 24 Jul 2019 10:50:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1563990633; cv=none; d=google.com; s=arc-20160816; b=IvceYcYVl6YYwEp18Z9dbn/ZCgVFoG4SXgPqzJoDLJWBU7pGYjOSb0YPzBU7WiGAqR vJ5Md/gpG3cDtl1ljPK+Nos2Wv7QpgUpwy25anoRgO7wPRVUSQmwciG1aOk38i2C8gAB 4gYg10x+zC29ItXF4ro5jkXj/W+5F05GA9tjD7hpaH0CdW7JtaLCsjzR7b+q4PDs6P5Z glYvknX1hgwRc8v7p3QUYmzXmFcLuGqyeSdGQ+VdFi3P2BQgUtWU7OXkkXdnZiBULmUV ucnL13mSTlYzkyTDMmWI07Lsx4edzYgAIltqPVMSKC5A/qXnG0AaV/pqtzAN9z6O6mpZ 904w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=NX/s/A5OLVDNZD4fNK87u3UpZfdfRBOaB3nXPLcsPas=; b=ii5JnZ9luFJRXXEjjCxQgovFZNrLPuryQQKpbXmhFQRLKEP3xXHDlbl7+MCO4P+RzI T3v9C3g+HPPK9NZ/1o2D7XC5RHAsI3N52zpeMFmkyTYo85T0AmeSG2YY2h0RblojH8ez YELp56rO7NRsekSrh6FT7IGfN+iOJ7qk4rU40JtJU5mOMMU/nDqiIh+5ttsTj6XZi0Md ho7BSCFkbyr2tVe5ydh+9JmYpEGYMDJwM4Y8BWC1i7wLpw0p5JYo3qJl+lmjLor0VCpx Wvk+FrEpdZLqc3otRLgIk7gTvT57IF2gRQ+rkgboeMQ+EltViWeRKnxAYB7S3eIzcmm6 ZBaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=lkP+47eS; spf=pass (google.com: domain of mike.kravetz@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2120.oracle.com (aserp2120.oracle.com. [141.146.126.78]) by mx.google.com with ESMTPS id u62si3167436vsu.198.2019.07.24.10.50.33 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2019 10:50:33 -0700 (PDT) Received-SPF: pass (google.com: domain of mike.kravetz@oracle.com designates 141.146.126.78 as permitted sender) client-ip=141.146.126.78; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=lkP+47eS; spf=pass (google.com: domain of mike.kravetz@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x6OHdfUU134750; Wed, 24 Jul 2019 17:50:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=NX/s/A5OLVDNZD4fNK87u3UpZfdfRBOaB3nXPLcsPas=; b=lkP+47eSi6lNelJK4kH8PvDDkmDI4KwgfHOhbC5Lp/E4hBF1MY44i5mbaW+evRfZPfVk R0E2hYBeVSx9InL7HjrmIAPY9jsoPtf007rUUpbwwU4kMeu5SwrRL/OaqngDPPnDxpRy AejMJlnkqk7kWLvjxvzRIKfTgu0PhLBMNO6svrheEboBLTglFaBcAp8E2cYbg0nGn284 JFJedRP7waYMAXMhlD29ywDn9b0RVde89rNmiQn3yImeq7Z8T8QNjtY/7dNVqr76HLnN PdLeHUc6OyBQYDGzt2p6cj94UzzJYNWaB+pds0fvtZPlZ+ppJhobynJmlWoICT2GKsSF 4w== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 2tx61by15s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Jul 2019 17:50:29 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x6OHcEMc018710; Wed, 24 Jul 2019 17:50:28 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3030.oracle.com with ESMTP id 2tx60xv7j9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Jul 2019 17:50:28 +0000 Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x6OHoPsb022353; Wed, 24 Jul 2019 17:50:25 GMT Received: from monkey.oracle.com (/71.63.128.209) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 24 Jul 2019 10:50:25 -0700 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Hillf Danton , Michal Hocko , Mel Gorman , Vlastimil Babka , Johannes Weiner , Andrew Morton , Mike Kravetz Subject: [RFC PATCH 1/3] mm, reclaim: make should_continue_reclaim perform dryrun detection Date: Wed, 24 Jul 2019 10:50:12 -0700 Message-Id: <20190724175014.9935-2-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724175014.9935-1-mike.kravetz@oracle.com> References: <20190724175014.9935-1-mike.kravetz@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9328 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907240191 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9328 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907240191 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Hillf Danton Address the issue of should_continue_reclaim continuing true too often for __GFP_RETRY_MAYFAIL attempts when !nr_reclaimed and nr_scanned. This could happen during hugetlb page allocation causing stalls for minutes or hours. Restructure code so that false will be returned in this case even if there are plenty of inactive pages. Need better description from Hillf Danton Need SOB from Hillf Danton Acked-by: Mel Gorman --- mm/vmscan.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index f4fd02ae233e..484b6b1a954e 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2673,18 +2673,6 @@ static inline bool should_continue_reclaim(struct pglist_data *pgdat, return false; } - /* - * If we have not reclaimed enough pages for compaction and the - * inactive lists are large enough, continue reclaiming - */ - pages_for_compaction = compact_gap(sc->order); - inactive_lru_pages = node_page_state(pgdat, NR_INACTIVE_FILE); - if (get_nr_swap_pages() > 0) - inactive_lru_pages += node_page_state(pgdat, NR_INACTIVE_ANON); - if (sc->nr_reclaimed < pages_for_compaction && - inactive_lru_pages > pages_for_compaction) - return true; - /* If compaction would go ahead or the allocation would succeed, stop */ for (z = 0; z <= sc->reclaim_idx; z++) { struct zone *zone = &pgdat->node_zones[z]; @@ -2700,7 +2688,21 @@ static inline bool should_continue_reclaim(struct pglist_data *pgdat, ; } } - return true; + + /* + * If we have not reclaimed enough pages for compaction and the + * inactive lists are large enough, continue reclaiming + */ + pages_for_compaction = compact_gap(sc->order); + inactive_lru_pages = node_page_state(pgdat, NR_INACTIVE_FILE); + if (get_nr_swap_pages() > 0) + inactive_lru_pages += node_page_state(pgdat, NR_INACTIVE_ANON); + + return inactive_lru_pages > pages_for_compaction && + /* + * avoid dryrun with plenty of inactive pages + */ + nr_scanned && nr_reclaimed; } static bool pgdat_memcg_congested(pg_data_t *pgdat, struct mem_cgroup *memcg) From patchwork Wed Jul 24 17:50:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 11057467 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F210F13B1 for ; Wed, 24 Jul 2019 17:50:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DE379208C2 for ; Wed, 24 Jul 2019 17:50:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CC7A62875F; Wed, 24 Jul 2019 17:50:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99EB8208C2 for ; Wed, 24 Jul 2019 17:50:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68EF58E0008; Wed, 24 Jul 2019 13:50:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6193E8E0005; Wed, 24 Jul 2019 13:50:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52E618E0008; Wed, 24 Jul 2019 13:50:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 316E58E0005 for ; Wed, 24 Jul 2019 13:50:34 -0400 (EDT) Received: by mail-qt1-f200.google.com with SMTP id s22so42029175qtb.22 for ; Wed, 24 Jul 2019 10:50:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=xjcDfF8Fr9jyitkNJdHzu+KgDQ97o2yz26S0HowRWGs=; b=UKeUCimo2gF5Jtk+6hNU8qcg/dNaVM9Vjr3P6LsqZGAsI2h6EdR8t05bFNImBCCOcR eRb+vucOQQl3pXjcbGtuoUnCSIW9M7p0yGvwREOanq2IiWiZWvvJsicL8q5EojXQO9MP aYHUeYQVhNz34vNN/gpzZ2+/6rVFkukeeJH+wM9HLZKoHWLDCcFvv7G+XBYc/LRThsLr j+WE9EtqKv2fay/3FFDe3z6bK2gyVKwrl3HJbfRpi6ztqwNGZQn6PRkvpPrJ6VE8jivw fVhmbYP/DnINc+q+W8HPWjighBHVvc4QcQvwKCiCMtpF3unQdS+7aoFWxzSD6VmXt6rh m1jw== X-Gm-Message-State: APjAAAW+kltpuCoo60vRZq8tzClZSP8WezkVaxPh5GHZyywYuA0usl9A lB/zmMn1VYMQD3/fTYXyrJTltOIPOSzUEcLohq4R057yxtTldg3mwdqHwWZJYofdqgzm/b8rTyH 2kt04z36+sHrxH0GCqoDyKbAAubDlQ7UElEtdhMonoy49VdVBzcVhFzBsKSRJ3o7dvg== X-Received: by 2002:a0c:b758:: with SMTP id q24mr59714795qve.45.1563990633856; Wed, 24 Jul 2019 10:50:33 -0700 (PDT) X-Google-Smtp-Source: APXvYqyjXb5NzIk3WvlgThaOqwiGSiBLU3Ce6rR3O+MnJoK5IjWcExByBTpKORJCHBaDM1hh5xRB X-Received: by 2002:a0c:b758:: with SMTP id q24mr59714764qve.45.1563990633168; Wed, 24 Jul 2019 10:50:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1563990633; cv=none; d=google.com; s=arc-20160816; b=SwOLIeJQKTU4yb8VzA2vNMLXaW7gJ0DplZUDruPyzf2CwUY1IIhiPgmGcygUM1n3Tq rowRuBeT7wS332bCL8kMYeSXG6nuL4bYpee7O1OW8iH9lGjEoflQ5VnJ8GQXu1Om/l8E 0suiI35xzyGMhXbSq17dcLpWCUesyDnKS+kshDPsR9Z7RL5uO/xceqSz1Eg0oN8wJz73 uxgdkA5Z5wEvV2B4nBGhvn4iiUYPvth6Ft4VeTt58ckqss0/vNHqESbwMIUWNkjr9fap W+KHIhLihQ00/ZhMKF/wFIMxEoC4BXEvk6KccriMGaykPEv4cxTpKc7f0im0mUwAA/IH OFcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=xjcDfF8Fr9jyitkNJdHzu+KgDQ97o2yz26S0HowRWGs=; b=ZLHLrPjERSBTeBf0Of69Hzg3ueLzPQXMHjqSIuzoAqV5sGhyhT5ONanunbV+3AZCbC qHmScVskTSzdelzA3Sgj7MvhDYBKpZRzEAyuCSPYdRmKJV/n7FDGGBdy8FxZ3MbiRuu6 xXXCDiMXmfaxjjau4/VWkWr7GbBVW9Su5q6XBU6ZWBjA6ymlW1+JgktvrEzhkvc8ioIJ K7aOCE5HEMvC5sd15xNtiKDljR4NPNaYg/GPLUyuDc262pjPoQ30zdYMwk3d5RhT+dTS Ye4iWfJ8OcszuJWpbeoCTjlS1GZXpDBRfSSRL+BE6+SDf47xjyOy3sjMavTWEWf4Hu9v Em7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=nLcnZKWZ; spf=pass (google.com: domain of mike.kravetz@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id q2si31326548qtj.247.2019.07.24.10.50.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2019 10:50:33 -0700 (PDT) Received-SPF: pass (google.com: domain of mike.kravetz@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=nLcnZKWZ; spf=pass (google.com: domain of mike.kravetz@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x6OHdh19049727; Wed, 24 Jul 2019 17:50:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=xjcDfF8Fr9jyitkNJdHzu+KgDQ97o2yz26S0HowRWGs=; b=nLcnZKWZmIDZkJO16y1rVlECVuxXnpR7rtJtjGqGPqJ2Qg/lJjudSK3kHaSvvH/9tyum x3e1DdU7/YablO/0JGCSCoeO4c+JYMTdssHb9oTEE87kVSmJWlTz3/UBjVUM4NmAie59 3jJ3t5gL+10Iv1TcFHdG0j8ui5+ZL6wVfGNkR3DXw7ipvRWNkRdEI436Arvz/MGFQeGb Zx2x/yjd0MdD1kloFtVfjxUARVLdS2xlmvx+88n7WnxFp5Ix1giYemD4PGpdkXNzhZCM R/oNLLxifaajAOQkUlG5tHiMaWJng58PAMCXH0jKnU8fTlzUwMgTSVFdqrjwCWjVJg3y aQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2120.oracle.com with ESMTP id 2tx61by0ug-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Jul 2019 17:50:28 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x6OHcB6L188598; Wed, 24 Jul 2019 17:50:28 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 2tx60y698f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Jul 2019 17:50:28 +0000 Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x6OHoRTh022364; Wed, 24 Jul 2019 17:50:27 GMT Received: from monkey.oracle.com (/71.63.128.209) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 24 Jul 2019 10:50:27 -0700 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Hillf Danton , Michal Hocko , Mel Gorman , Vlastimil Babka , Johannes Weiner , Andrew Morton , Mike Kravetz Subject: [RFC PATCH 2/3] mm, compaction: use MIN_COMPACT_COSTLY_PRIORITY everywhere for costly orders Date: Wed, 24 Jul 2019 10:50:13 -0700 Message-Id: <20190724175014.9935-3-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724175014.9935-1-mike.kravetz@oracle.com> References: <20190724175014.9935-1-mike.kravetz@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9328 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907240191 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9328 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907240191 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP For PAGE_ALLOC_COSTLY_ORDER allocations, MIN_COMPACT_COSTLY_PRIORITY is minimum (highest priority). Other places in the compaction code key off of MIN_COMPACT_PRIORITY. Costly order allocations will never get to MIN_COMPACT_PRIORITY. Therefore, some conditions will never be met for costly order allocations. This was observed when hugetlb allocations could stall for minutes or hours when should_compact_retry() would return true more often then it should. Specifically, this was in the case where compact_result was COMPACT_DEFERRED and COMPACT_PARTIAL_SKIPPED and no progress was being made. Signed-off-by: Mike Kravetz Acked-by: Mel Gorman Reported-by: Mike Kravetz Signed-off-by: Vlastimil Babka --- mm/compaction.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 952dc2fb24e5..325b746068d1 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -2294,9 +2294,15 @@ static enum compact_result compact_zone_order(struct zone *zone, int order, .alloc_flags = alloc_flags, .classzone_idx = classzone_idx, .direct_compaction = true, - .whole_zone = (prio == MIN_COMPACT_PRIORITY), - .ignore_skip_hint = (prio == MIN_COMPACT_PRIORITY), - .ignore_block_suitable = (prio == MIN_COMPACT_PRIORITY) + .whole_zone = ((order > PAGE_ALLOC_COSTLY_ORDER) ? + (prio == MIN_COMPACT_COSTLY_PRIORITY) : + (prio == MIN_COMPACT_PRIORITY)), + .ignore_skip_hint = ((order > PAGE_ALLOC_COSTLY_ORDER) ? + (prio == MIN_COMPACT_COSTLY_PRIORITY) : + (prio == MIN_COMPACT_PRIORITY)), + .ignore_block_suitable = ((order > PAGE_ALLOC_COSTLY_ORDER) ? + (prio == MIN_COMPACT_COSTLY_PRIORITY) : + (prio == MIN_COMPACT_PRIORITY)) }; struct capture_control capc = { .cc = &cc, @@ -2338,6 +2344,7 @@ enum compact_result try_to_compact_pages(gfp_t gfp_mask, unsigned int order, int may_perform_io = gfp_mask & __GFP_IO; struct zoneref *z; struct zone *zone; + int min_priority; enum compact_result rc = COMPACT_SKIPPED; /* @@ -2350,12 +2357,13 @@ enum compact_result try_to_compact_pages(gfp_t gfp_mask, unsigned int order, trace_mm_compaction_try_to_compact_pages(order, gfp_mask, prio); /* Compact each zone in the list */ + min_priority = (order > PAGE_ALLOC_COSTLY_ORDER) ? + MIN_COMPACT_COSTLY_PRIORITY : MIN_COMPACT_PRIORITY; for_each_zone_zonelist_nodemask(zone, z, ac->zonelist, ac->high_zoneidx, ac->nodemask) { enum compact_result status; - if (prio > MIN_COMPACT_PRIORITY - && compaction_deferred(zone, order)) { + if (prio > min_priority && compaction_deferred(zone, order)) { rc = max_t(enum compact_result, COMPACT_DEFERRED, rc); continue; } From patchwork Wed Jul 24 17:50:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 11057473 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE301138D for ; Wed, 24 Jul 2019 17:50:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD595208C2 for ; Wed, 24 Jul 2019 17:50:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A13B1285EC; Wed, 24 Jul 2019 17:50:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC2C1208C2 for ; Wed, 24 Jul 2019 17:50:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 530BC8E000C; Wed, 24 Jul 2019 13:50:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4E0778E0005; Wed, 24 Jul 2019 13:50:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D1138E000C; Wed, 24 Jul 2019 13:50:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 100448E0005 for ; Wed, 24 Jul 2019 13:50:35 -0400 (EDT) Received: by mail-qt1-f200.google.com with SMTP id t5so42063909qtd.21 for ; Wed, 24 Jul 2019 10:50:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=pckymHbnRHu+rafvqysstMGQoahE6dDQ06s/gXDYrBc=; b=bwRoKiPtkkRrg2xcx9ltLEHQwox5bGj+oK+OUtJopu4RO+22H2JBNM61MQ8aaYO3bp QcUjzgD7KLkRijYriHEITcxK8elLHULEzwJb9hn4nBnh7UaEhNcLSvu/Oh4H0cX2HVJg jRB10d5TkvfYVYLaabDNWGOLkMmOO8N/HCX0po/VjEb7mM2UFdoJg2YKnhA2lSGuUjnw NYDB3uuJg3lZ9jXB2/1fsMhyL6JuBfGZxnspzVrw+hutggqnpX+WjUsQSSg0/zlmiqKY ZcD8VUII1KEUkK7GI4tkseswyMCAqF+2Lb22/eEkuib7AqL3/cRq2zpffcWAGVJCoT+j OtAA== X-Gm-Message-State: APjAAAV72s3eo8OwluDsmqYsYELyxV841LJsRCBFqEGgnSn51NMpYzdV M8QNUjx0wj9bVnHpECzTwtdZBNmV1lzhQXDfERsbOvAE9NBE7xFJppPPAxQco8+7nDdQWtZUZpt eccAKD/f9MUt09SqhGbTE4Lt1Cn0fX7Nr4g/BoQlvyft+J+4fXuQ6XQi8QHL6CRkwnA== X-Received: by 2002:a0c:ae31:: with SMTP id y46mr60418559qvc.172.1563990634783; Wed, 24 Jul 2019 10:50:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqxF1uWMTzZQ6DNuhGzgShF7FgF4YzzVroocsMRo4K0003nfbDvkkN8qzbCKlGm9uHpHZ0Yt X-Received: by 2002:a0c:ae31:: with SMTP id y46mr60418507qvc.172.1563990633925; Wed, 24 Jul 2019 10:50:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1563990633; cv=none; d=google.com; s=arc-20160816; b=bIe2uifrsFL/lYHhNOJPZGNuGvchwm1+pbnGdZz2Hv7o7FHM01Nt5quzzShRMNBmvB mCU4vEmpnHht2vIanVoQohIJtZnIrzEesNF0qtc5LEIXHr1SwF+rzHrdkQJx6PPOC0vV wdOKHP05S+ium4+2BUYX9QQB5n/+Ai+wgZI7WTRq/Iwhlb8CdNodMTOIAHwOmfQIrDDD zcPfUUf+lcorLDqM2ATLFn4aC9bO6p/R6LDVeFFkid3xBH93cGFkx/lColPFv7o+VAr3 mOnN+bayKH2iQ3PK0G09RjTDRCQUhxPDEAkoYro3ctPy60+BqmVYeXlhrsDQ+U6mOZXP 8EnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=pckymHbnRHu+rafvqysstMGQoahE6dDQ06s/gXDYrBc=; b=oivxPdI3P97zAHt/hyVYkwX44KrNWj4HFZjs5OXce3mVOOfIATzPTEV3jyKXd8cogb y5JWFhHThOR5cmol3TxCvtPG6HdoTTMpA7atwBvPeQQRecpjTa1Wb5OiP311UtGaoefR pU6ceI8CU925KwVuAywClMeI2I7giRuySfjBhTI/ttLqpqmDYQCpjjGbJXn4WY/td+5G pIS2uQMvwbPX8IGMvLaul3au692W59dm27UVqV+SNV+op0Qw3fO8nG0u/6tmNDUehu7b c4LVQsHhtXXh2i/6l8AV3OxmCQ9iXn10FLN+X4hIIFczd2140v0JTIsm0nyfZEfnd2tc urVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=LyqW3tnX; spf=pass (google.com: domain of mike.kravetz@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id n49si31566888qtf.49.2019.07.24.10.50.33 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2019 10:50:33 -0700 (PDT) Received-SPF: pass (google.com: domain of mike.kravetz@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=LyqW3tnX; spf=pass (google.com: domain of mike.kravetz@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x6OHdv8A050046; Wed, 24 Jul 2019 17:50:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=pckymHbnRHu+rafvqysstMGQoahE6dDQ06s/gXDYrBc=; b=LyqW3tnXiUeCTD+OMYyOSyntNW7Oow0QdHShbM20ewy7LYaVgE4PMeTNIpvjuyZ15caD hPxkaqYeG04m9CEi7QtXOE5KpwMEv4M3Jq7b0c+JP5RWhNaVUcOrt5c8IODDciMT5psW gDxMsdLQZQDCn/mAv1Dsb/eV4uZDzv98dvxlp62wO4nuoIFfnV7/5XG7ks35LsH4QQrA 2sBjEQbze+uv+5o0AsuixJHizx7SwmJGo+qLG+u7O9XgyNAdeeAsGkNpQGXozlmUxzdp +8sSckHg0Bgz0jkpb9AZg3/wkBffjkmvIVtoq0E9DjdQpxA9fQwfx18xpdb+pX1ZeTTF QA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 2tx61by0ur-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Jul 2019 17:50:30 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x6OHcAF9185415; Wed, 24 Jul 2019 17:50:29 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3020.oracle.com with ESMTP id 2tx60y960j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Jul 2019 17:50:29 +0000 Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x6OHoTE7022375; Wed, 24 Jul 2019 17:50:29 GMT Received: from monkey.oracle.com (/71.63.128.209) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 24 Jul 2019 10:50:29 -0700 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Hillf Danton , Michal Hocko , Mel Gorman , Vlastimil Babka , Johannes Weiner , Andrew Morton , Mike Kravetz Subject: [RFC PATCH 3/3] hugetlbfs: don't retry when pool page allocations start to fail Date: Wed, 24 Jul 2019 10:50:14 -0700 Message-Id: <20190724175014.9935-4-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724175014.9935-1-mike.kravetz@oracle.com> References: <20190724175014.9935-1-mike.kravetz@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9328 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907240191 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9328 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907240191 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When allocating hugetlbfs pool pages via /proc/sys/vm/nr_hugepages, the pages will be interleaved between all nodes of the system. If nodes are not equal, it is quite possible for one node to fill up before the others. When this happens, the code still attempts to allocate pages from the full node. This results in calls to direct reclaim and compaction which slow things down considerably. When allocating pool pages, note the state of the previous allocation for each node. If previous allocation failed, do not use the aggressive retry algorithm on successive attempts. The allocation will still succeed if there is memory available, but it will not try as hard to free up memory. Signed-off-by: Mike Kravetz Acked-by: Mel Gorman --- mm/hugetlb.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 77 insertions(+), 10 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index ede7e7f5d1ab..f3c50344a9b4 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1405,12 +1405,27 @@ pgoff_t __basepage_index(struct page *page) } static struct page *alloc_buddy_huge_page(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + nodemask_t *node_alloc_noretry) { int order = huge_page_order(h); struct page *page; + bool alloc_try_hard = true; - gfp_mask |= __GFP_COMP|__GFP_RETRY_MAYFAIL|__GFP_NOWARN; + /* + * By default we always try hard to allocate the page with + * __GFP_RETRY_MAYFAIL flag. However, if we are allocating pages in + * a loop (to adjust global huge page counts) and previous allocation + * failed, do not continue to try hard on the same node. Use the + * node_alloc_noretry bitmap to manage this state information. + */ + if (node_alloc_noretry && node_isset(nid, *node_alloc_noretry)) + alloc_try_hard = false; + gfp_mask |= __GFP_COMP|__GFP_NOWARN; + if (alloc_try_hard) + gfp_mask |= __GFP_RETRY_MAYFAIL; + else + gfp_mask |= __GFP_NORETRY; if (nid == NUMA_NO_NODE) nid = numa_mem_id(); page = __alloc_pages_nodemask(gfp_mask, order, nid, nmask); @@ -1419,6 +1434,22 @@ static struct page *alloc_buddy_huge_page(struct hstate *h, else __count_vm_event(HTLB_BUDDY_PGALLOC_FAIL); + /* + * If we did not specify __GFP_RETRY_MAYFAIL, but still got a page this + * indicates an overall state change. Clear bit so that we resume + * normal 'try hard' allocations. + */ + if (node_alloc_noretry && page && !alloc_try_hard) + node_clear(nid, *node_alloc_noretry); + + /* + * If we tried hard to get a page but failed, set bit so that + * subsequent attempts will not try as hard until there is an + * overall state change. + */ + if (node_alloc_noretry && !page && alloc_try_hard) + node_set(nid, *node_alloc_noretry); + return page; } @@ -1427,7 +1458,8 @@ static struct page *alloc_buddy_huge_page(struct hstate *h, * should use this function to get new hugetlb pages */ static struct page *alloc_fresh_huge_page(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + nodemask_t *node_alloc_noretry) { struct page *page; @@ -1435,7 +1467,7 @@ static struct page *alloc_fresh_huge_page(struct hstate *h, page = alloc_gigantic_page(h, gfp_mask, nid, nmask); else page = alloc_buddy_huge_page(h, gfp_mask, - nid, nmask); + nid, nmask, node_alloc_noretry); if (!page) return NULL; @@ -1450,14 +1482,16 @@ static struct page *alloc_fresh_huge_page(struct hstate *h, * Allocates a fresh page to the hugetlb allocator pool in the node interleaved * manner. */ -static int alloc_pool_huge_page(struct hstate *h, nodemask_t *nodes_allowed) +static int alloc_pool_huge_page(struct hstate *h, nodemask_t *nodes_allowed, + nodemask_t *node_alloc_noretry) { struct page *page; int nr_nodes, node; gfp_t gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE; for_each_node_mask_to_alloc(h, nr_nodes, node, nodes_allowed) { - page = alloc_fresh_huge_page(h, gfp_mask, node, nodes_allowed); + page = alloc_fresh_huge_page(h, gfp_mask, node, nodes_allowed, + node_alloc_noretry); if (page) break; } @@ -1601,7 +1635,7 @@ static struct page *alloc_surplus_huge_page(struct hstate *h, gfp_t gfp_mask, goto out_unlock; spin_unlock(&hugetlb_lock); - page = alloc_fresh_huge_page(h, gfp_mask, nid, nmask); + page = alloc_fresh_huge_page(h, gfp_mask, nid, nmask, NULL); if (!page) return NULL; @@ -1637,7 +1671,7 @@ struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, if (hstate_is_gigantic(h)) return NULL; - page = alloc_fresh_huge_page(h, gfp_mask, nid, nmask); + page = alloc_fresh_huge_page(h, gfp_mask, nid, nmask, NULL); if (!page) return NULL; @@ -2207,13 +2241,31 @@ static void __init gather_bootmem_prealloc(void) static void __init hugetlb_hstate_alloc_pages(struct hstate *h) { unsigned long i; + nodemask_t *node_alloc_noretry; + + if (!hstate_is_gigantic(h)) { + /* + * bit mask controlling how hard we retry per-node + * allocations. + */ + node_alloc_noretry = kmalloc(sizeof(*node_alloc_noretry), + GFP_KERNEL | __GFP_NORETRY); + } else { + /* allocations done at boot time */ + node_alloc_noretry = NULL; + } + + /* bit mask controlling how hard we retry per-node allocations */ + if (node_alloc_noretry) + nodes_clear(*node_alloc_noretry); for (i = 0; i < h->max_huge_pages; ++i) { if (hstate_is_gigantic(h)) { if (!alloc_bootmem_huge_page(h)) break; } else if (!alloc_pool_huge_page(h, - &node_states[N_MEMORY])) + &node_states[N_MEMORY], + node_alloc_noretry)) break; cond_resched(); } @@ -2225,6 +2277,9 @@ static void __init hugetlb_hstate_alloc_pages(struct hstate *h) h->max_huge_pages, buf, i); h->max_huge_pages = i; } + + if (node_alloc_noretry) + kfree(node_alloc_noretry); } static void __init hugetlb_init_hstates(void) @@ -2323,6 +2378,12 @@ static int set_max_huge_pages(struct hstate *h, unsigned long count, int nid, nodemask_t *nodes_allowed) { unsigned long min_count, ret; + NODEMASK_ALLOC(nodemask_t, node_alloc_noretry, + GFP_KERNEL | __GFP_NORETRY); + + /* bit mask controlling how hard we retry per-node allocations */ + if (node_alloc_noretry) + nodes_clear(*node_alloc_noretry); spin_lock(&hugetlb_lock); @@ -2356,6 +2417,8 @@ static int set_max_huge_pages(struct hstate *h, unsigned long count, int nid, if (hstate_is_gigantic(h) && !IS_ENABLED(CONFIG_CONTIG_ALLOC)) { if (count > persistent_huge_pages(h)) { spin_unlock(&hugetlb_lock); + if (node_alloc_noretry) + NODEMASK_FREE(node_alloc_noretry); return -EINVAL; } /* Fall through to decrease pool */ @@ -2388,7 +2451,8 @@ static int set_max_huge_pages(struct hstate *h, unsigned long count, int nid, /* yield cpu to avoid soft lockup */ cond_resched(); - ret = alloc_pool_huge_page(h, nodes_allowed); + ret = alloc_pool_huge_page(h, nodes_allowed, + node_alloc_noretry); spin_lock(&hugetlb_lock); if (!ret) goto out; @@ -2429,6 +2493,9 @@ static int set_max_huge_pages(struct hstate *h, unsigned long count, int nid, h->max_huge_pages = persistent_huge_pages(h); spin_unlock(&hugetlb_lock); + if (node_alloc_noretry) + NODEMASK_FREE(node_alloc_noretry); + return 0; }