From patchwork Mon Apr 27 07:02:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 11511273 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BA6F613B2 for ; Mon, 27 Apr 2020 07:04:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 87F9A206B6 for ; Mon, 27 Apr 2020 07:04:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 87F9A206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BF3C28E000C; Mon, 27 Apr 2020 03:03:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B7E0C8E0001; Mon, 27 Apr 2020 03:03:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FEBE8E000E; Mon, 27 Apr 2020 03:03:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0118.hostedemail.com [216.40.44.118]) by kanga.kvack.org (Postfix) with ESMTP id 661628E000C for ; Mon, 27 Apr 2020 03:03:48 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1D8B15851 for ; Mon, 27 Apr 2020 07:03:48 +0000 (UTC) X-FDA: 76752744936.19.fowl27_3fc0fc75262a X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,alex.shi@linux.alibaba.com,,RULES_HIT:30012:30036:30054:30055:30056:30070:30083:30090,0,RBL:47.88.44.36:@linux.alibaba.com:.lbl8.mailshell.net-64.10.201.10 62.18.0.100;47.88.44.36-irl.urbl.hostedemail.com-127.0.0.175,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: fowl27_3fc0fc75262a X-Filterd-Recvd-Size: 5972 Received: from out4436.biz.mail.alibaba.com (out4436.biz.mail.alibaba.com [47.88.44.36]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Mon, 27 Apr 2020 07:03:47 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01358;MF=alex.shi@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0Twm27rp_1587971012; Received: from localhost(mailfrom:alex.shi@linux.alibaba.com fp:SMTPD_---0Twm27rp_1587971012) by smtp.aliyun-inc.com(127.0.0.1); Mon, 27 Apr 2020 15:03:32 +0800 From: Alex Shi To: akpm@linux-foundation.org, mgorman@techsingularity.net, tj@kernel.org, hughd@google.com, khlebnikov@yandex-team.ru, daniel.m.jordan@oracle.com, yang.shi@linux.alibaba.com, willy@infradead.org, hannes@cmpxchg.org, lkp@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, shakeelb@google.com, iamjoonsoo.kim@lge.com, richard.weiyang@gmail.com Cc: Alex Shi Subject: [PATCH v10 00/15] per memcg lru lock Date: Mon, 27 Apr 2020 15:02:49 +0800 Message-Id: <1587970985-21629-1-git-send-email-alex.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a new version which bases on Johannes new patchset "mm: memcontrol: charge swapin pages on instantiation" https://lkml.org/lkml/2020/4/21/266 Johannes Weiner has suggested: "So here is a crazy idea that may be worth exploring: Right now, pgdat->lru_lock protects both PageLRU *and* the lruvec's linked list. Can we make PageLRU atomic and use it to stabilize the lru_lock instead, and then use the lru_lock only serialize list operations? ..." With the cleaning memcg charge path and this suggestion, we could isolate LRU pages to exclusive visit them in compaction, page migration, reclaim, memcg move_accunt, huge page split etc scenarios while keeping pages' memcg stable. Then possible to change per node lru locking to per memcg lru locking. As to pagevec_lru_move_fn funcs, it would be safe to let pages remain on lru list, lru lock could guard them for list integrity. This is version safely pass Hugh Dickins's swapping kernel building testcase, Thanks for the great case! I want to send out a bit early for more testing and review while people's memory is still hot with Johannes new memcg charge patch. :) I will do more testing beside. The patchset includes 3 parts: 1, some code cleanup and minimum optimization as a preparation. 2, use TestCleanPageLRU as page isolation's precondition 3, replace per node lru_lock with per memcg per node lru_lock The 3rd part moves per node lru_lock into lruvec, thus bring a lru_lock for each of memcg per node. So on a large machine, each of memcg don't have to suffer from per node pgdat->lru_lock competition. They could go fast with their self lru_lock Following Daniel Jordan's suggestion, I have run 208 'dd' with on 104 containers on a 2s * 26cores * HT box with a modefied case: https://git.kernel.org/pub/scm/linux/kernel/git/wfg/vm-scalability.git/tree/case-lru-file-readtwice With this patchset, the readtwice performance increased about 80% in concurrent containers. Thanks Hugh Dickins and Konstantin Khlebnikov, they both brought this idea 8 years ago, and others who give comments as well: Daniel Jordan, Mel Gorman, Shakeel Butt, Matthew Wilcox etc. Thanks for Testing support from Intel 0day and Rong Chen, Fengguang Wu, and Yun Wang. Alex Shi (13): mm/swap: use vmf clean up swapin funcs parameters mm/vmscan: remove unnecessary lruvec adding mm/page_idle: no unlikely double check for idle page counting mm/thp: move lru_add_page_tail func to huge_memory.c mm/thp: clean up lru_add_page_tail mm/thp: narrow lru locking mm/memcg: add debug checking in lock_page_memcg mm/lru: introduce TestClearPageLRU mm/compaction: do page isolation first in compaction mm/mlock: ClearPageLRU before get lru lock in munlock page isolation mm/lru: replace pgdat lru_lock with lruvec lock mm/lru: introduce the relock_page_lruvec function mm/pgdat: remove pgdat lru_lock Hugh Dickins (2): mm/vmscan: use relock for move_pages_to_lru mm/lru: revise the comments of lru_lock Documentation/admin-guide/cgroup-v1/memcg_test.rst | 15 +- Documentation/admin-guide/cgroup-v1/memory.rst | 8 +- Documentation/trace/events-kmem.rst | 2 +- Documentation/vm/unevictable-lru.rst | 22 +-- include/linux/memcontrol.h | 92 +++++++++++ include/linux/mm_types.h | 2 +- include/linux/mmzone.h | 5 +- include/linux/page-flags.h | 1 + include/linux/swap.h | 12 +- mm/compaction.c | 85 +++++++---- mm/filemap.c | 4 +- mm/huge_memory.c | 55 +++++-- mm/madvise.c | 11 +- mm/memcontrol.c | 87 ++++++++++- mm/mlock.c | 93 ++++++------ mm/mmzone.c | 1 + mm/page_alloc.c | 1 - mm/page_idle.c | 8 - mm/rmap.c | 2 +- mm/swap.c | 119 ++++----------- mm/swap_state.c | 23 ++- mm/swapfile.c | 8 +- mm/vmscan.c | 168 +++++++++++---------- mm/zswap.c | 3 +- 24 files changed, 497 insertions(+), 330 deletions(-)