From patchwork Thu May 23 02:27:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10956843 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB1C176 for ; Thu, 23 May 2019 02:28:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A4D5B1FF2D for ; Thu, 23 May 2019 02:28:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 98AC0262AE; Thu, 23 May 2019 02:28:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E3BE1FF2D for ; Thu, 23 May 2019 02:28:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B71666B0003; Wed, 22 May 2019 22:28:03 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B1E1F6B0006; Wed, 22 May 2019 22:28:03 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 997BF6B000A; Wed, 22 May 2019 22:28:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ot1-f70.google.com (mail-ot1-f70.google.com [209.85.210.70]) by kanga.kvack.org (Postfix) with ESMTP id 6FFB06B0003 for ; Wed, 22 May 2019 22:28:03 -0400 (EDT) Received: by mail-ot1-f70.google.com with SMTP id n21so2208692otq.16 for ; Wed, 22 May 2019 19:28:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=fTV9+QJ3Qlr+eWA8hqTAepwEMa3PO/pXGof/Ta/9BdY=; b=lQ7RBjw12gpOvketQBIX1VoFPgRLIciy31Ke+Q/kAoYLxJVyKLjtibHaZEl09vTWnq m7dERu9wcoS/L3zQ19FrzrOgC0KBMFUHgNcFLAcQq+2f46YSFcgL8YUuHGEuwBEPhNcY zMRddksY1CiVZA4zlNL8FCma+RCiVQKX72rvFbMeanpXF4sF3GwcWF1ZxBBztyN2B8QY WfdZxZ3Xn5zXXEdpuUyPZNAUO1d8osQy9cmfDeP6Utr9YmvREgiSxmrzSGBabcMBmyVQ 9L46bjJr+lGyy7i7StllwiMCSUngqDwsWpFQuI8YfUaOJkKlIbXOEUdJ+Qnj5waeGbwH DU2w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.45 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: APjAAAWigepJkCCtObak4tR9NIYwYgXBwCv9H5plamOOSdWuIqTIFmUt FFV/GxTP+pnHXANwXKsAmsIrWea6Hbb9IV1IAo3fPHj1JFc0M3bUy+Dw3DZALiw7mWFH2St77Mo 1ZtRS/j5HwJ8ztu7yZIGYy3AanbqNVxbwGGqHaIGvjcwMfcxP305jLBPJmmhy9ZJ4YQ== X-Received: by 2002:aca:f007:: with SMTP id o7mr1256983oih.59.1558578483099; Wed, 22 May 2019 19:28:03 -0700 (PDT) X-Google-Smtp-Source: APXvYqw1LPpR0uL6WwUfuGpgHxSB4uBSXErmczlUkMR06teM1igg/Br+JEGiTni1UlE/kEaOehXJ X-Received: by 2002:aca:f007:: with SMTP id o7mr1256947oih.59.1558578482104; Wed, 22 May 2019 19:28:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558578482; cv=none; d=google.com; s=arc-20160816; b=wOp0dBMN7HA40O/KAerferbmtSAOZsV1FM7114zw7r58wpX00EshMJ3XHHmkNDvrNs yG3WUgqFduEFM8QMKSm2StycGeTRwuoAfYOa8xZD9YFp/SCc3ve2WT8E+ydeh5RNsp8z PHXNg0+0SgO0JCErJBsL9jtWxXo8/n5tGgknNmCvdZhYP3AR1hfC+nfYeXLP2zxRYmTl L05tPRBgcu4ctAL3shhF8EJkjP9cwV6FUORssz5Rjjanw0+MQmgIBxoQJMJkmyk8qqL+ 1ttxS6USG4pGWORbcu5pQBXsMDkgxvAMoCvpAEf+Gte7hYWR6OFZBcI7m+QlPBbqAF4m Rfhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from; bh=fTV9+QJ3Qlr+eWA8hqTAepwEMa3PO/pXGof/Ta/9BdY=; b=AwFknq1/n8Mabl7jwpf6KAC95p/MU8i4tJS3JFBxAUYDipHV4swfcXM5XFOAhhG4o8 PiFloDWsSTdvrD2bC7AZSW0Ruo2V5oNyMle84uln/oYUcJwSjmWaLWoGrsl5vqFkG0bM Gb8KnKjHqWQZfv6pq3wqIQNlWrnbroWL3iNmCFTVYCU69v0GYhYImvBRMOY/2sRCyF1k lWfvNMEHXNp8KzH0zNf9M2nG+Vd6n+gz7zEAcpZG4Gt5MTOvI3TGZ+hGs+17RnqXCO1D 4NuVlXyudc2zX+XN4qQZNfn8dUjtofcpwKK/8IzlmLzwhrgkGNvR2PVBtZ2JMkZRmfB+ g4PA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.45 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-45.freemail.mail.aliyun.com (out30-45.freemail.mail.aliyun.com. [115.124.30.45]) by mx.google.com with ESMTPS id k29si15960294otf.306.2019.05.22.19.28.01 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 May 2019 19:28:02 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.45 as permitted sender) client-ip=115.124.30.45; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.45 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01422;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=12;SR=0;TI=SMTPD_---0TSQws5W_1558578458; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0TSQws5W_1558578458) by smtp.aliyun-inc.com(127.0.0.1); Thu, 23 May 2019 10:27:45 +0800 From: Yang Shi To: ying.huang@intel.com, hannes@cmpxchg.org, mhocko@suse.com, mgorman@techsingularity.net, kirill.shutemov@linux.intel.com, josef@toxicpanda.com, hughd@google.com, shakeelb@google.com, akpm@linux-foundation.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v4 PATCH 1/2] mm: vmscan: remove double slab pressure by inc'ing sc->nr_scanned Date: Thu, 23 May 2019 10:27:37 +0800 Message-Id: <1558578458-83807-1-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The commit 9092c71bb724 ("mm: use sc->priority for slab shrink targets") has broken up the relationship between sc->nr_scanned and slab pressure. The sc->nr_scanned can't double slab pressure anymore. So, it sounds no sense to still keep sc->nr_scanned inc'ed. Actually, it would prevent from adding pressure on slab shrink since excessive sc->nr_scanned would prevent from scan->priority raise. The bonnie test doesn't show this would change the behavior of slab shrinkers. w/ w/o /sec %CP /sec %CP Sequential delete: 3960.6 94.6 3997.6 96.2 Random delete: 2518 63.8 2561.6 64.6 The slight increase of "/sec" without the patch would be caused by the slight increase of CPU usage. Cc: Josef Bacik Cc: Michal Hocko Acked-by: Johannes Weiner Signed-off-by: Yang Shi --- v4: Added Johannes's ack mm/vmscan.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 7acd0af..b65bc50 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1137,11 +1137,6 @@ static unsigned long shrink_page_list(struct list_head *page_list, if (!sc->may_unmap && page_mapped(page)) goto keep_locked; - /* Double the slab pressure for mapped and swapcache pages */ - if ((page_mapped(page) || PageSwapCache(page)) && - !(PageAnon(page) && !PageSwapBacked(page))) - sc->nr_scanned++; - may_enter_fs = (sc->gfp_mask & __GFP_FS) || (PageSwapCache(page) && (sc->gfp_mask & __GFP_IO)); From patchwork Thu May 23 02:27:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10956841 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31DBF1390 for ; Thu, 23 May 2019 02:28:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 105D21FF2D for ; Thu, 23 May 2019 02:28:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 03CFB262AE; Thu, 23 May 2019 02:28:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CDE6A1FF2D for ; Thu, 23 May 2019 02:28:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 70B496B0007; Wed, 22 May 2019 22:28:03 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6B0EF6B0006; Wed, 22 May 2019 22:28:03 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 578736B0007; Wed, 22 May 2019 22:28:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id 187186B0003 for ; Wed, 22 May 2019 22:28:03 -0400 (EDT) Received: by mail-pf1-f197.google.com with SMTP id c12so3088254pfb.2 for ; Wed, 22 May 2019 19:28:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=pxbR2tROxGdscNPq0FHUMQrz/OcrNWf6ZKt9Si6kHuk=; b=AbXU56TKVXr+MLPOTVYobZyOa5hFuVncQtUjXrgo3JfnW2O25HdZOkbBg5Xph76bvM X63i6ztI7UUxnXibIoSsdjuMV+zJzREAmf3swTcSCeFkWRsfsy86QPI8ACi4IHG31YMB XIfi+5jgXC8uiWQp9RUm2tZRQDnXGm6hhTr/G5XPut+MqeZKwsAtDemRh+RGYjJc365K Zz/5HYNOsanhomjKH9MKdhA+vYrx13ZpQyg7N65xtfnXzb5INDk5YCWb29Da36CWxj8Q Y8e6OKGAvoGgK7Ng8pMHwjZYjsTbJD6QabhpBKDKvPqkk/bOxjfU69qB5nIcxqs9KOlX gt0Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 47.88.44.37 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: APjAAAWgikQJeHgfrRiu8KAGu1FoRXJkpd6VT08d2hLKN7YMPBtX3aW4 vjo40pmOBMZas1ceLoAdm8FVY1NL3y7fTqOIC+CXSaOR35yIr8q4+0J15DqgZozrz7y3nT+pMAz Xj6MPHq325t3cagEV8Z52+d1cA09lpzYqHm9jWa52+LrVlAP5QD2NBK0N79zKY1YNQA== X-Received: by 2002:a63:9d83:: with SMTP id i125mr1255477pgd.229.1558578482736; Wed, 22 May 2019 19:28:02 -0700 (PDT) X-Google-Smtp-Source: APXvYqxuy9Ci0eWOmGqdVxJI9iEzZjf4Bh4kzWl5VGu1kiESRi5JpSYB3YBWGk61jBXT663S2K8q X-Received: by 2002:a63:9d83:: with SMTP id i125mr1255361pgd.229.1558578481216; Wed, 22 May 2019 19:28:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558578481; cv=none; d=google.com; s=arc-20160816; b=cHQgjCnZRqz7WYrcs9/twBZLtX7dd2erj9V3EVYeWIOUWqxXhr/vQRoIogik6gfeqy 9K/ZanDYNvvJ7fdPRg1VZPVVkePEid/FHvcAXsrhnePseXWAl3xFEC7gq8e6G5PTKutY F0xreF2X/mglzdbTpKrAf5gaNHlBAbY7z5886B6ZqMh6Vuvs97yKYRy5dZM3r4VyPO4F 3qWXkC3Shw0oy/jgZqjrHsQ6cXLcJzPjycmyKUR6fOmkDBOYQwRkP7PlsNRpLppbDxxO mebgNcFHK4clPbmtLdQ/0YobGV1BgviMklKA5HA5oEX7kw9E/NERd6r2hY8mLABt6q1c ZILg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=pxbR2tROxGdscNPq0FHUMQrz/OcrNWf6ZKt9Si6kHuk=; b=hQfALXZcf/5dZUUhBzwDel1XZIUVCsbap1kMPtwK69Dp3RcTEwAYO6H9hS5EDf+6aP CNZCIPHCFl52/3i79yAg+MH0+pIVfEFKmm7ML2uJamRBfLMs9zf4XkSH6c9KlFN3+sLc ovDHqErA9cZtJIvMJxtRrNCVDb1vfYaH5gh2j3+kYLQPwwbq/Xi7GQau26C78U2D9++O J66jKQpTtp4n4ZzgFmxCw+UM4/kElIJn/l2kmD9SCbQxGu4dNAhO7ak55TbzO+MH4Lzm 8oU0B434vpGfztX5Ythu8N9N/++BH3H5K6f4grNEitI2ecAH/VVcKncwWBCbOsmSx26P 9KsQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 47.88.44.37 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out4437.biz.mail.alibaba.com (out4437.biz.mail.alibaba.com. [47.88.44.37]) by mx.google.com with ESMTPS id g16si29623972pfo.191.2019.05.22.19.27.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 May 2019 19:28:01 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 47.88.44.37 as permitted sender) client-ip=47.88.44.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 47.88.44.37 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R721e4;CH=green;DM=||false|;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e07417;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=12;SR=0;TI=SMTPD_---0TSQws5W_1558578458; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0TSQws5W_1558578458) by smtp.aliyun-inc.com(127.0.0.1); Thu, 23 May 2019 10:27:45 +0800 From: Yang Shi To: ying.huang@intel.com, hannes@cmpxchg.org, mhocko@suse.com, mgorman@techsingularity.net, kirill.shutemov@linux.intel.com, josef@toxicpanda.com, hughd@google.com, shakeelb@google.com, akpm@linux-foundation.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v4 PATCH 2/2] mm: vmscan: correct some vmscan counters for THP swapout Date: Thu, 23 May 2019 10:27:38 +0800 Message-Id: <1558578458-83807-2-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1558578458-83807-1-git-send-email-yang.shi@linux.alibaba.com> References: <1558578458-83807-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Since commit bd4c82c22c36 ("mm, THP, swap: delay splitting THP after swapped out"), THP can be swapped out in a whole. But, nr_reclaimed and some other vm counters still get inc'ed by one even though a whole THP (512 pages) gets swapped out. This doesn't make too much sense to memory reclaim. For example, direct reclaim may just need reclaim SWAP_CLUSTER_MAX pages, reclaiming one THP could fulfill it. But, if nr_reclaimed is not increased correctly, direct reclaim may just waste time to reclaim more pages, SWAP_CLUSTER_MAX * 512 pages in worst case. And, it may cause pgsteal_{kswapd|direct} is greater than pgscan_{kswapd|direct}, like the below: pgsteal_kswapd 122933 pgsteal_direct 26600225 pgscan_kswapd 174153 pgscan_direct 14678312 nr_reclaimed and nr_scanned must be fixed in parallel otherwise it would break some page reclaim logic, e.g. vmpressure: this looks at the scanned/reclaimed ratio so it won't change semantics as long as scanned & reclaimed are fixed in parallel. compaction/reclaim: compaction wants a certain number of physical pages freed up before going back to compacting. kswapd priority raising: kswapd raises priority if we scan fewer pages than the reclaim target (which itself is obviously expressed in order-0 pages). As a result, kswapd can falsely raise its aggressiveness even when it's making great progress. Other than nr_scanned and nr_reclaimed, some other counters, e.g. pgactivate, nr_skipped, nr_ref_keep and nr_unmap_fail need to be fixed too since they are user visible via cgroup, /proc/vmstat or trace points, otherwise they would be underreported. When isolating pages from LRUs, nr_taken has been accounted in base page, but nr_scanned and nr_skipped are still accounted in THP. It doesn't make too much sense too since this may cause trace point underreport the numbers as well. So accounting those counters in base page instead of accounting THP as one page. nr_dirty, nr_unqueued_dirty, nr_congested and nr_writeback are used by file cache, so they are not impacted by THP swap. This change may result in lower steal/scan ratio in some cases since THP may get split during page reclaim, then a part of tail pages get reclaimed instead of the whole 512 pages, but nr_scanned is accounted by 512, particularly for direct reclaim. But, this should be not a significant issue. Cc: "Huang, Ying" Cc: Johannes Weiner Cc: Michal Hocko Cc: Mel Gorman Cc: "Kirill A . Shutemov" Cc: Hugh Dickins Cc: Shakeel Butt Signed-off-by: Yang Shi --- v4: Fixed the comments from Johannes and Huang Ying v3: Removed Shakeel's Reviewed-by since the patch has been changed significantly Switched back to use compound_order per Matthew Fixed more counters per Johannes v2: Added Shakeel's Reviewed-by Use hpage_nr_pages instead of compound_order per Huang Ying and William Kucharski mm/vmscan.c | 34 ++++++++++++++++++++++------------ 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index b65bc50..1b35a7a 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1118,6 +1118,7 @@ static unsigned long shrink_page_list(struct list_head *page_list, int may_enter_fs; enum page_references references = PAGEREF_RECLAIM_CLEAN; bool dirty, writeback; + unsigned int nr_pages; cond_resched(); @@ -1129,7 +1130,9 @@ static unsigned long shrink_page_list(struct list_head *page_list, VM_BUG_ON_PAGE(PageActive(page), page); - sc->nr_scanned++; + /* Account the number of base pages evne though THP */ + nr_pages = 1 << compound_order(page); + sc->nr_scanned += nr_pages; if (unlikely(!page_evictable(page))) goto activate_locked; @@ -1250,7 +1253,7 @@ static unsigned long shrink_page_list(struct list_head *page_list, case PAGEREF_ACTIVATE: goto activate_locked; case PAGEREF_KEEP: - stat->nr_ref_keep++; + stat->nr_ref_keep += nr_pages; goto keep_locked; case PAGEREF_RECLAIM: case PAGEREF_RECLAIM_CLEAN: @@ -1315,7 +1318,7 @@ static unsigned long shrink_page_list(struct list_head *page_list, if (unlikely(PageTransHuge(page))) flags |= TTU_SPLIT_HUGE_PMD; if (!try_to_unmap(page, flags)) { - stat->nr_unmap_fail++; + stat->nr_unmap_fail += nr_pages; goto activate_locked; } } @@ -1442,7 +1445,11 @@ static unsigned long shrink_page_list(struct list_head *page_list, unlock_page(page); free_it: - nr_reclaimed++; + /* + * THP may get swapped out in a whole, need account + * all base pages. + */ + nr_reclaimed += (1 << compound_order(page)); /* * Is there need to periodically free_page_list? It would @@ -1464,7 +1471,6 @@ static unsigned long shrink_page_list(struct list_head *page_list, if (!PageMlocked(page)) { int type = page_is_file_cache(page); SetPageActive(page); - pgactivate++; stat->nr_activate[type] += hpage_nr_pages(page); count_memcg_page_event(page, PGACTIVATE); } @@ -1475,6 +1481,8 @@ static unsigned long shrink_page_list(struct list_head *page_list, VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page), page); } + pgactivate = stat->nr_activate[0] + stat->nr_activate[1]; + mem_cgroup_uncharge_list(&free_pages); try_to_unmap_flush(); free_unref_page_list(&free_pages); @@ -1642,14 +1650,14 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan, unsigned long nr_zone_taken[MAX_NR_ZONES] = { 0 }; unsigned long nr_skipped[MAX_NR_ZONES] = { 0, }; unsigned long skipped = 0; - unsigned long scan, total_scan, nr_pages; + unsigned long scan, total_scan; + unsigned long nr_pages; LIST_HEAD(pages_skipped); isolate_mode_t mode = (sc->may_unmap ? 0 : ISOLATE_UNMAPPED); + total_scan = 0; scan = 0; - for (total_scan = 0; - scan < nr_to_scan && nr_taken < nr_to_scan && !list_empty(src); - total_scan++) { + while (scan < nr_to_scan && !list_empty(src)) { struct page *page; page = lru_to_page(src); @@ -1657,9 +1665,12 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan, VM_BUG_ON_PAGE(!PageLRU(page), page); + nr_pages = 1 << compound_order(page); + total_scan += nr_pages; + if (page_zonenum(page) > sc->reclaim_idx) { list_move(&page->lru, &pages_skipped); - nr_skipped[page_zonenum(page)]++; + nr_skipped[page_zonenum(page)] += nr_pages; continue; } @@ -1669,10 +1680,9 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan, * ineligible pages. This causes the VM to not reclaim any * pages, triggering a premature OOM. */ - scan++; + scan += nr_pages; switch (__isolate_lru_page(page, mode)) { case 0: - nr_pages = hpage_nr_pages(page); nr_taken += nr_pages; nr_zone_taken[page_zonenum(page)] += nr_pages; list_move(&page->lru, dst);