From patchwork Thu Nov 7 20:53:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 11233757 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 761EB139A for ; Thu, 7 Nov 2019 20:53:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3584221D79 for ; Thu, 7 Nov 2019 20:53:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=cmpxchg-org.20150623.gappssmtp.com header.i=@cmpxchg-org.20150623.gappssmtp.com header.b="OzWbKP3Q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3584221D79 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=cmpxchg.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5EFC36B0006; Thu, 7 Nov 2019 15:53:43 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5010F6B0007; Thu, 7 Nov 2019 15:53:43 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 415896B0008; Thu, 7 Nov 2019 15:53:43 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id 2AEC76B0006 for ; Thu, 7 Nov 2019 15:53:43 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id C8F704DA5 for ; Thu, 7 Nov 2019 20:53:42 +0000 (UTC) X-FDA: 76130682684.23.cloud96_335511251930 X-Spam-Summary: 2,0,0,4c5e42ca4acc97ef,d41d8cd98f00b204,hannes@cmpxchg.org,:akpm@linux-foundation.org:aryabinin@virtuozzo.com:surenb@google.com:shakeelb@google.com:riel@surriel.com:mhocko@suse.com::cgroups@vger.kernel.org:linux-kernel@vger.kernel.org:kernel-team@fb.com,RULES_HIT:41:69:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1605:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:2692:2693:2731:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4118:4250:4321:4385:5007:6119:6261:6653:7903:7904:8603:9036:9592:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12517:12519:12555:12683:12895:12986:13161:13229:13894:14181:14394:14721:14722:21080:21444:21451:21627:21939:30012:30054:30070,0,RBL:209.85.215.193:@cmpxchg.org:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:26,LUA_SUMMAR Y:none X-HE-Tag: cloud96_335511251930 X-Filterd-Recvd-Size: 7494 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Thu, 7 Nov 2019 20:53:41 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id w11so2728095pga.12 for ; Thu, 07 Nov 2019 12:53:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GnXcKvw24Mkpxrnsn9Jk/2NsGqvskVrKtfbs3H323nA=; b=OzWbKP3QiAn5Cn8rqW3do83+lh2GNKnq0FS6MLFOgzuNg9dowb58c23NoQzbB4lOdp W/UmmwJIvWubOW8e8AJP+Ab0S8N1ndrzXmVCVkPfewcm1xbU9rkvqPvwCOU8QjJ0TCz1 2XA8J6K9+5TBaC2uSnBomFE2jescOZki3s/+b7Zs8yd34ZfQTpzAi0FICpNiOTlz2+iG aepA0Eqnm1gTv2wpj5IcSHjnJhNPZMhqlHSPShHOjL5/j8blN+6ibf/WtsHncdXvrzo1 XbshrsG+fEWB6/dMhCk8in4SmLpDYy0wWZ0Rcw2WEvEW4Pno2tVgkBqADGtysU6hrNC2 1Ngg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GnXcKvw24Mkpxrnsn9Jk/2NsGqvskVrKtfbs3H323nA=; b=j53AQAPGbfF01jkcGUstUqS4BfxCL/CVx8O5l2dO46OzQq7zpHccvflkAIjyG+Npgg nTkxIs8T4K7VFQ+oUpAjS/itEzqJC54StCNe3JSwwrdoLgl8ZDADwyfqUiC/uGzxk5cf TM7L/s+PF7braTEbmMs/S0oiQej/pZtop8grR5w2o27hW8Iqu0Ht65C45HjPBPy6jebd mJf7BVhhzU4Q/MZLI/VHXo0q8cdTzFKesj4jhhVQCkaJQ7+5JviuFmj1ceIa7Gub/b8e OTPQuDNcr90hPsCewttsLvWlC2eN0//LvMgMPHMlmjRcjsHAMx20sjSG78pDmtIxejxz C39g== X-Gm-Message-State: APjAAAWjtEgd/n+DGhlfmdCeleLQK8rVCHi9/7+W938N8q3O5zSAziLU tQnSAcMZ7VitDtAYMykExKa/8g== X-Google-Smtp-Source: APXvYqxk4ogczT9UUDe68wHpQWglnBtvKn9ruyhCd3jhJInvgSAjwsTSTQtXb9gPlZ2UM0VJE49c9w== X-Received: by 2002:a62:53:: with SMTP id 80mr7030693pfa.192.1573160020255; Thu, 07 Nov 2019 12:53:40 -0800 (PST) Received: from localhost ([2620:10d:c090:200::3:3792]) by smtp.gmail.com with ESMTPSA id r11sm2952834pjp.14.2019.11.07.12.53.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Nov 2019 12:53:39 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Andrey Ryabinin , Suren Baghdasaryan , Shakeel Butt , Rik van Riel , Michal Hocko , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 1/3] mm: vmscan: move file exhaustion detection to the node level Date: Thu, 7 Nov 2019 12:53:32 -0800 Message-Id: <20191107205334.158354-2-hannes@cmpxchg.org> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191107205334.158354-1-hannes@cmpxchg.org> References: <20191107205334.158354-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When file pages are lower than the watermark on a node, we try to force scan anonymous pages to counter-act the balancing algorithms preference for new file pages when they are likely thrashing. This is a node-level decision, but it's currently made each time we look at an lruvec. This is unnecessarily expensive and also a layering violation that makes the code harder to understand. Clean this up by making the check once per node and setting a flag in the scan_control. Signed-off-by: Johannes Weiner Reviewed-by: Shakeel Butt Reviewed-by: Suren Baghdasaryan --- mm/vmscan.c | 80 ++++++++++++++++++++++++++++------------------------- 1 file changed, 42 insertions(+), 38 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index d97985262dda..e8dd601e1fad 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -101,6 +101,9 @@ struct scan_control { /* One of the zones is ready for compaction */ unsigned int compaction_ready:1; + /* The file pages on the current node are dangerously low */ + unsigned int file_is_tiny:1; + /* Allocation order */ s8 order; @@ -2289,45 +2292,16 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc, } /* - * Prevent the reclaimer from falling into the cache trap: as - * cache pages start out inactive, every cache fault will tip - * the scan balance towards the file LRU. And as the file LRU - * shrinks, so does the window for rotation from references. - * This means we have a runaway feedback loop where a tiny - * thrashing file LRU becomes infinitely more attractive than - * anon pages. Try to detect this based on file LRU size. + * If the system is almost out of file pages, force-scan anon. + * But only if there are enough inactive anonymous pages on + * the LRU. Otherwise, the small LRU gets thrashed. */ - if (!cgroup_reclaim(sc)) { - unsigned long pgdatfile; - unsigned long pgdatfree; - int z; - unsigned long total_high_wmark = 0; - - pgdatfree = sum_zone_node_page_state(pgdat->node_id, NR_FREE_PAGES); - pgdatfile = node_page_state(pgdat, NR_ACTIVE_FILE) + - node_page_state(pgdat, NR_INACTIVE_FILE); - - for (z = 0; z < MAX_NR_ZONES; z++) { - struct zone *zone = &pgdat->node_zones[z]; - if (!managed_zone(zone)) - continue; - - total_high_wmark += high_wmark_pages(zone); - } - - if (unlikely(pgdatfile + pgdatfree <= total_high_wmark)) { - /* - * Force SCAN_ANON if there are enough inactive - * anonymous pages on the LRU in eligible zones. - * Otherwise, the small LRU gets thrashed. - */ - if (!inactive_list_is_low(lruvec, false, sc, false) && - lruvec_lru_size(lruvec, LRU_INACTIVE_ANON, sc->reclaim_idx) - >> sc->priority) { - scan_balance = SCAN_ANON; - goto out; - } - } + if (sc->file_is_tiny && + !inactive_list_is_low(lruvec, false, sc, false) && + lruvec_lru_size(lruvec, LRU_INACTIVE_ANON, + sc->reclaim_idx) >> sc->priority) { + scan_balance = SCAN_ANON; + goto out; } /* @@ -2754,6 +2728,36 @@ static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc) nr_reclaimed = sc->nr_reclaimed; nr_scanned = sc->nr_scanned; + /* + * Prevent the reclaimer from falling into the cache trap: as + * cache pages start out inactive, every cache fault will tip + * the scan balance towards the file LRU. And as the file LRU + * shrinks, so does the window for rotation from references. + * This means we have a runaway feedback loop where a tiny + * thrashing file LRU becomes infinitely more attractive than + * anon pages. Try to detect this based on file LRU size. + */ + if (!cgroup_reclaim(sc)) { + unsigned long file; + unsigned long free; + int z; + unsigned long total_high_wmark = 0; + + free = sum_zone_node_page_state(pgdat->node_id, NR_FREE_PAGES); + file = node_page_state(pgdat, NR_ACTIVE_FILE) + + node_page_state(pgdat, NR_INACTIVE_FILE); + + for (z = 0; z < MAX_NR_ZONES; z++) { + struct zone *zone = &pgdat->node_zones[z]; + if (!managed_zone(zone)) + continue; + + total_high_wmark += high_wmark_pages(zone); + } + + sc->file_is_tiny = file + free <= total_high_wmark; + } + shrink_node_memcgs(pgdat, sc); if (reclaim_state) {