From patchwork Tue Jan 22 20:09:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10776233 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06C6D6C2 for ; Tue, 22 Jan 2019 20:10:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E971A2B70F for ; Tue, 22 Jan 2019 20:10:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D7D332B80F; Tue, 22 Jan 2019 20:10:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D9802B70F for ; Tue, 22 Jan 2019 20:10:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 904AE8E0003; Tue, 22 Jan 2019 15:10:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8B1C88E0001; Tue, 22 Jan 2019 15:10:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C7DE8E0003; Tue, 22 Jan 2019 15:10:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f200.google.com (mail-it1-f200.google.com [209.85.166.200]) by kanga.kvack.org (Postfix) with ESMTP id 54B438E0001 for ; Tue, 22 Jan 2019 15:10:07 -0500 (EST) Received: by mail-it1-f200.google.com with SMTP id b14so14530633itd.1 for ; Tue, 22 Jan 2019 12:10:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=vy6YMvOtWIcfPF2rk1qQQgjNqNcxwbmlfk1ObKoCvmk=; b=TOq1lun0s3iiQUYO2GStwe2nSZ2AzePxdO9YS8Lf1cYJsvJmycyDwvoRV9WVsVJbwj mzS2a0wFuoCKHuhr6M8inerb7cGVt70JPaviXEJtgaa26VHXfu/VdafMdSFvaH80kJ+F ZVTQg+zVYyWtqoKFS6P8STNpjgcdPp2QdUC7Nb7e4I6CkrloyHWQFGakthu6gHHyR2HT 3TC8UU6aTBlpc13NIPXL1bKUYRDhTaQ+/gI3BYbGhuHQENGhgQtPNLyvzHn2O5P8tchZ yQKHv3brQI6SEZq/dq9sbgqEevPNE18zG4Znsz6bxghfw4sEBBH6nRKVaexzQmjsUHE9 eSaQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: AJcUuken0oXNm66ahGhcbO/WeoDmg6TJPYP6IYjEP56CJbsRpz80XNHM GsbETe+uBpEsY68P7Os2lM3nLeBJjVTcWCr0oz2MpGuUeVzu/H0L/79PqxU6dWDTsPOaW+rc2J0 AV1M+VDE8pTECi48beTirf39kqvf9EXva79meH0YqTOPUkzIFRlGcOz9trQry3aWppQ== X-Received: by 2002:a24:e04a:: with SMTP id c71mr3066901ith.127.1548187806946; Tue, 22 Jan 2019 12:10:06 -0800 (PST) X-Google-Smtp-Source: AHgI3IYZ2qeBhHjGRjq5PcCvBhL9JwirpY/gBmR4cv/Wre/Qvm5Y9bQdxelGTXAk75VhoYIHtTCR X-Received: by 2002:a24:e04a:: with SMTP id c71mr3066824ith.127.1548187805272; Tue, 22 Jan 2019 12:10:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548187805; cv=none; d=google.com; s=arc-20160816; b=cfgMCtI3/DPeiZ1QlQ0/X7hqHsQRnGy/5gwWmDbd+tZ7QCvOtC9R6AdFmuLks+7h7x 68WTqKGqSjU7fNTLopd76D5Rzasv4Nm2o5sl4YhyEAh/jm76SyKW6Gg1jRI/bXlAlPgu yuUpoESjtaoAAb/UytAaZshf6Xh9xsQg1crhp3A06yZM5VsHe9IQrJONt7Mrf9sLIc4R kZUV+JwMj5DVkT+XjQVyRUl+OgqeuaaeITN9Yu4GXbTR4CwT5X5yz5oBL0BJuZE+zRkz +rVLuBjw+gYiY2jUFeKh3PVo5ERvAwOU1JE3URijtFFAt2hYvUagF2oJBOOdiJHBIhd4 jJHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from; bh=vy6YMvOtWIcfPF2rk1qQQgjNqNcxwbmlfk1ObKoCvmk=; b=tiQy6PCWeUB6LFis35wCTYJia416MfhliUhypJiHANtqnwQh5rHefjLExJY5FDvU6+ 2MYxG3OddidnWY7y2dNMqEtwZt2jzDkzsc+46vSvYPYKhaDZSoM9Dfer0IZwFPUubMLP 12tea45XaQLb+edZGnkC9JvPg8tt+DrqAkNia74l7AgbRY+GWx3RG4kcmmSDJRPo3aj6 lUcBMccPFyxOaJE0nhjTB/R959Xb+HEqRab8S75gUka4YyW62zd922N1p6E7yC99aY8P LOVMwXORpKPAbzOsje/kOAFdHXZRrpbRjEgUA9P2O93UF6wUjcLNJ7ZdqkTx8iSJXrBm CaAw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com. [115.124.30.131]) by mx.google.com with ESMTPS id p6si9322150itm.58.2019.01.22.12.10.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 Jan 2019 12:10:04 -0800 (PST) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) client-ip=115.124.30.131; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01353;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0TInKYlb_1548187782; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0TInKYlb_1548187782) by smtp.aliyun-inc.com(127.0.0.1); Wed, 23 Jan 2019 04:09:51 +0800 From: Yang Shi To: mhocko@suse.com, hannes@cmpxchg.org, akpm@linux-foundation.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH] mm: vmscan: do not iterate all mem cgroups for global direct reclaim Date: Wed, 23 Jan 2019 04:09:42 +0800 Message-Id: <1548187782-108454-1-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In current implementation, both kswapd and direct reclaim has to iterate all mem cgroups. It is not a problem before offline mem cgroups could be iterated. But, currently with iterating offline mem cgroups, it could be very time consuming. In our workloads, we saw over 400K mem cgroups accumulated in some cases, only a few hundred are online memcgs. Although kswapd could help out to reduce the number of memcgs, direct reclaim still get hit with iterating a number of offline memcgs in some cases. We experienced the responsiveness problems due to this occassionally. Here just break the iteration once it reclaims enough pages as what memcg direct reclaim does. This may hurt the fairness among memcgs since direct reclaim may awlays do reclaim from same memcgs. But, it sounds ok since direct reclaim just tries to reclaim SWAP_CLUSTER_MAX pages and memcgs can be protected by min/low. Cc: Johannes Weiner Cc: Michal Hocko Signed-off-by: Yang Shi --- mm/vmscan.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index a714c4f..ced5a16 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2764,16 +2764,15 @@ static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc) sc->nr_reclaimed - reclaimed); /* - * Direct reclaim and kswapd have to scan all memory - * cgroups to fulfill the overall scan target for the - * node. + * Kswapd have to scan all memory cgroups to fulfill + * the overall scan target for the node. * * Limit reclaim, on the other hand, only cares about * nr_to_reclaim pages to be reclaimed and it will * retry with decreasing priority if one round over the * whole hierarchy is not sufficient. */ - if (!global_reclaim(sc) && + if ((!global_reclaim(sc) || !current_is_kswapd()) && sc->nr_reclaimed >= sc->nr_to_reclaim) { mem_cgroup_iter_break(root, memcg); break;