From patchwork Mon May 8 09:33:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "zhaoyang.huang" X-Patchwork-Id: 13234267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70148C77B7F for ; Mon, 8 May 2023 09:34:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBBD86B007D; Mon, 8 May 2023 05:34:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C6F0D6B007E; Mon, 8 May 2023 05:34:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B34CB6B0080; Mon, 8 May 2023 05:34:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8C9316B007E for ; Mon, 8 May 2023 05:34:12 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4F64BA018A for ; Mon, 8 May 2023 09:34:12 +0000 (UTC) X-FDA: 80766576744.10.B4AD626 Received: from SHSQR01.spreadtrum.com (mx1.unisoc.com [222.66.158.135]) by imf12.hostedemail.com (Postfix) with ESMTP id 1296440008 for ; Mon, 8 May 2023 09:34:09 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of zhaoyang.huang@unisoc.com designates 222.66.158.135 as permitted sender) smtp.mailfrom=zhaoyang.huang@unisoc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683538450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=05B9HLEVIXOyu73vSo8T/gNbsWk1TAFnDf+P1UFDg/w=; b=GRHIfcHspP7FFWBczjc7EcR4i4Wm8HlQ9yNzexwZBx9BjORGLcDY3jNHz9HK16ywtYm0uL GtkdLaXXUZpqI+NQooeyhBKTJ6qcER9Wb7aGCAy1o6iMdYZP8UQOfBZmLHbvtfLf/5apq4 FTqG7a3ENpZ1nk0dLReVC0NsIN0Dz6M= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of zhaoyang.huang@unisoc.com designates 222.66.158.135 as permitted sender) smtp.mailfrom=zhaoyang.huang@unisoc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683538450; a=rsa-sha256; cv=none; b=B/Y6sSIMmo7ETLLcN5MttyOSqeqXeBEODEFdtBw7I79E7+2NBwsQPrbvMMArUUKMkgFaCX tRU3nWCd368M74KqMEBegSIQjxqlL/ktwuwqPZ+AjPNqqtk1WDqFkveKPZenNKXWrKQaGC vRkPOoVFpv0z6f/Yltf6XmWAGIPB5hM= Received: from SHSend.spreadtrum.com (bjmbx01.spreadtrum.com [10.0.64.7]) by SHSQR01.spreadtrum.com with ESMTP id 3489XTVE033277; Mon, 8 May 2023 17:33:29 +0800 (+08) (envelope-from zhaoyang.huang@unisoc.com) Received: from bj03382pcu.spreadtrum.com (10.0.74.65) by BJMBX01.spreadtrum.com (10.0.64.7) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Mon, 8 May 2023 17:33:25 +0800 From: "zhaoyang.huang" To: Andrew Morton , Roman Gushchin , Minchan Kim , Roman Gushchin , Joonsoo Kim , , , Zhaoyang Huang , Subject: [PATCH 1/2] mm: optimization on page allocation when CMA enabled Date: Mon, 8 May 2023 17:33:02 +0800 Message-ID: <1683538383-19685-2-git-send-email-zhaoyang.huang@unisoc.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1683538383-19685-1-git-send-email-zhaoyang.huang@unisoc.com> References: <1683538383-19685-1-git-send-email-zhaoyang.huang@unisoc.com> MIME-Version: 1.0 X-Originating-IP: [10.0.74.65] X-ClientProxiedBy: SHCAS03.spreadtrum.com (10.0.1.207) To BJMBX01.spreadtrum.com (10.0.64.7) X-MAIL: SHSQR01.spreadtrum.com 3489XTVE033277 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 1296440008 X-Stat-Signature: dde591hsdsy4twofhu4ky8qtn7bkh9p6 X-HE-Tag: 1683538449-763790 X-HE-Meta: U2FsdGVkX1+K84S2F2IsW70ImyPSYrXFr6M90a++qyl2MntnecuPevWc/wQRLksQUvsxEY/UZsoqfzw0eRZvK03uiC1CuZ4COacaKekEs4PRJU0+dlH9xWIkApUViHfS0Nj32KKOZr4KL37zbUxgawNWx9lnFNGFQ1R3Uf+PlHsliMMhBtoLUAkjIK5LqFMLVttKEE43QLA7bqsdUmndxz9F2PSRt6si8ULjd2vaNI5Ysmj5wdFWstyHXEAkruBmleABeaTuPaLNSLXS/m835q5FGcSWZYS10X/bxz0lXikNZvtWcV4Iiljq2P3OwHvxVpyPlhBuJSWuEji/YALqlVrPq2e9yY8BWKenNwx4/FkaRREEuphal45hwuvut41soNGH/QxTO0OGRXcSUx+CK6b6t0ruGhGjE2OiGo6iUNAhuBRbW9MXkkQSC9vWIpTnOSp71jV0PFrp0Gls+odqLtToOyJ0vNG5hxKDhLbmi2+Q+J7SfjllOZ3JdKiVpB4MPHdbjMPaSPYou5xq+7cDMq1AZ0hIEzj8gEPcE6dLL/05IIWPugqg2JoACp7I80Zilz9iw5isFhlBgaFBWCWnSMVUsv2/D87idA+XCMIGDEacLSsxZv49kl+MGl000PZhv+Au/xdrkf/w59XEBOS/RNrmia8wVTOQzpQQbg1U56YuPdq8r6p7gQo1BJPBxUHP4EmUdQZ2APwg4mWNlL6M1GTpeP49S4RK8v9hkwpLP7IYW3vNBbG5gOyH+hAdHr/QdK9C/5DJb9DuhQqttkuv0Lmmy3Up+PtRmeDkbhoEuBlAukG7Z3Ua03JEyTGrTNj5w6DLSotUCW2FcfzO6VMb06+beLf66LWJfqWu5mngbJmCFH/SqVj4ZRpqPK59o6VWILRVllEscCTl9ovVhLc2F0nUnmiBA3Fr9l3p2oSD04XPzYNoffZLjS0mE3txVvv9zmchkGl5ewhCib86ptE WBSlHmRb og6aurBH89AB+1Plfsntnjz+wOLZm2ovNqV4suSyI5RSZDRCF/xBrkYKKZiyuzv3SBD0KS1nHo46WvUWpjIxtFHFbq5nurvtKxiA93mxYpTeupGE07HexPy3XrtcjMFzGlewn5pcAT1nTd5bRvkMJVZlV73MTvcMypJ8NOD7Uypq/rS+LheYkZKcB9keV1yXOolkTXnz0XxYbJXXPCh+DzkYB4NuvuGqA9F/6kZxEd67jpo8ddXaUYvPSNmZtuKCUznm4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhaoyang Huang Let us look at the series of scenarios below with WMARK_LOW=25MB,WMARK_MIN=5MB (managed pages 1.9GB). We can know that current 'fixed 1/2 ratio' start to use CMA since C which actually has caused U&R lower than WMARK_LOW (this should be deemed as against current memory policy, that is, UNMOVABLE & RECLAIMABLE should either stay around WATERMARK_LOW when no allocation or do reclaim via entering slowpath) -- Free_pages | | -- WMARK_LOW | -- Free_CMA | | --- Free_CMA/Free_pages(MB) A(12/30) B(12/25) C(12/20) fixed 1/2 ratio N N Y this commit Y Y Y Signed-off-by: Zhaoyang Huang --- v2: do proportion check when zone_watermark_ok, update commit message v3: update coding style and simplify the logic when zone_watermark_ok --- --- mm/page_alloc.c | 46 ++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 40 insertions(+), 6 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0745aed..7aca49d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3071,6 +3071,41 @@ static bool unreserve_highatomic_pageblock(const struct alloc_context *ac, } +#ifdef CONFIG_CMA +/* + * GFP_MOVABLE allocation could drain UNMOVABLE & RECLAIMABLE page blocks via + * the help of CMA which makes GFP_KERNEL failed. Checking if zone_watermark_ok + * again without ALLOC_CMA to see if to use CMA first. + */ +static bool __if_use_cma_first(struct zone *zone, unsigned int order, unsigned int alloc_flags) +{ + unsigned long watermark; + bool cma_first = false; + + watermark = wmark_pages(zone, alloc_flags & ALLOC_WMARK_MASK); + /* check if GFP_MOVABLE pass previous zone_watermark_ok via the help of CMA */ + if (!zone_watermark_ok(zone, order, watermark, 0, alloc_flags & (~ALLOC_CMA))) + /* + * watermark failed means UNMOVABLE & RECLAIMBLE is not enough + * now, we should use cma first to keep them stay around the + * corresponding watermark + */ + cma_first = true; + else + /* + * remain previous fixed 1/2 logic when watermark ok as we have + * above protection now + */ + cma_first = (zone_page_state(zone, NR_FREE_CMA_PAGES) > + zone_page_state(zone, NR_FREE_PAGES) / 2); + return cma_first; +} +#else +static bool __if_use_cma_first(struct zone *zone, unsigned int order, unsigned int alloc_flags) +{ + return false; +} +#endif /* * Do the hard work of removing an element from the buddy allocator. * Call me with the zone->lock already held. @@ -3084,13 +3119,12 @@ static bool unreserve_highatomic_pageblock(const struct alloc_context *ac, if (IS_ENABLED(CONFIG_CMA)) { /* * Balance movable allocations between regular and CMA areas by - * allocating from CMA when over half of the zone's free memory - * is in the CMA area. + * allocating from CMA base on judging zone_watermark_ok again + * to see if the latest check got pass via the help of CMA */ - if (alloc_flags & ALLOC_CMA && - zone_page_state(zone, NR_FREE_CMA_PAGES) > - zone_page_state(zone, NR_FREE_PAGES) / 2) { - page = __rmqueue_cma_fallback(zone, order); + if (migratetype == MIGRATE_MOVABLE) { + page = __if_use_cma_first(zone, order, alloc_flags) ? + __rmqueue_cma_fallback(zone, order) : NULL; if (page) return page; } From patchwork Mon May 8 09:33:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "zhaoyang.huang" X-Patchwork-Id: 13234268 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15895C7EE26 for ; Mon, 8 May 2023 09:34:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 088BA6B0080; Mon, 8 May 2023 05:34:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F05C26B0082; Mon, 8 May 2023 05:34:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D57CC6B0081; Mon, 8 May 2023 05:34:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C73EA6B007E for ; Mon, 8 May 2023 05:34:13 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8F28B80199 for ; Mon, 8 May 2023 09:34:13 +0000 (UTC) X-FDA: 80766576786.01.C5A9247 Received: from SHSQR01.spreadtrum.com (mx1.unisoc.com [222.66.158.135]) by imf05.hostedemail.com (Postfix) with ESMTP id 3670A100003 for ; Mon, 8 May 2023 09:34:10 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of zhaoyang.huang@unisoc.com designates 222.66.158.135 as permitted sender) smtp.mailfrom=zhaoyang.huang@unisoc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683538451; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yuzrgfVROjDmFctozl9v2S47wJtS4TpyhBk41fQ/7Ls=; b=MJBTyrQeP9xGD5J+nnJvJBkP3m9j6R/TBIiK6fUW7GlASpm6wdCQXmkzQh6v87PX0S3Y0s /LuHQjHn3uQlebrdSOksLcioxXe9x/2sJ9X/zzmMIMYdPABZOMrmPeaBeN+Fq2dRWLXlA9 HRMbjijJd3IfctCw0Lzd3TkIOahz3mQ= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of zhaoyang.huang@unisoc.com designates 222.66.158.135 as permitted sender) smtp.mailfrom=zhaoyang.huang@unisoc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683538451; a=rsa-sha256; cv=none; b=Dxxa9wYmnonIx6VnofRV98cRxc5RflthmBSDKXL7GWfW0oIWzqrVN0mLeqrqNsgzuz4m0v S2Up/kPeL12ipZ93Q93OfBViy0MhYs3BZ9Fk+D0qhbFQrnm+sPtB9mnjVGhHRZfB2KQJlV ZO2j1tMp6dux6b/6trTcz0xdkfYNShs= Received: from SHSend.spreadtrum.com (bjmbx01.spreadtrum.com [10.0.64.7]) by SHSQR01.spreadtrum.com with ESMTP id 3489XYUC033919; Mon, 8 May 2023 17:33:35 +0800 (+08) (envelope-from zhaoyang.huang@unisoc.com) Received: from bj03382pcu.spreadtrum.com (10.0.74.65) by BJMBX01.spreadtrum.com (10.0.64.7) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Mon, 8 May 2023 17:33:29 +0800 From: "zhaoyang.huang" To: Andrew Morton , Roman Gushchin , Minchan Kim , Roman Gushchin , Joonsoo Kim , , , Zhaoyang Huang , Subject: [PATCH 2/2] mm: skip CMA pages when they are not available Date: Mon, 8 May 2023 17:33:03 +0800 Message-ID: <1683538383-19685-3-git-send-email-zhaoyang.huang@unisoc.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1683538383-19685-1-git-send-email-zhaoyang.huang@unisoc.com> References: <1683538383-19685-1-git-send-email-zhaoyang.huang@unisoc.com> MIME-Version: 1.0 X-Originating-IP: [10.0.74.65] X-ClientProxiedBy: SHCAS03.spreadtrum.com (10.0.1.207) To BJMBX01.spreadtrum.com (10.0.64.7) X-MAIL: SHSQR01.spreadtrum.com 3489XYUC033919 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3670A100003 X-Stat-Signature: 495g5ukbch77wzn6y63pz3sghzb4z7yj X-HE-Tag: 1683538450-814429 X-HE-Meta: U2FsdGVkX18AvGmabV2xvl4cDKMGbAUol5PJ7GM+Zrsq8M2Z0Z8Xu68cfknblcsghomcQEzlR3dSlTsZ2/jj66fVZR9oTvF+Kp/K4jj13A/Dh54RRCNp3Sz4+gjdpFw+RNFApmo96YPc0EbgUJulUrub9MlPwqRCfukuLXSwQUpXhwkKrMG9veQZ01m9P1WRwOXrB6ZTA+rQWqdA6AM2v+FKEwaJqalFnbufDx5x/qc32WMsQViTaFX/rZP/86IDeP6Wgd8WVkd+tKeckUEZi4buGSypVaRRqZAE+ivvNRwsKPkp7hpR36ydNbVy50HTw8jIsIWM2/FVQqp1v6A8rF1w1PAKl4BCXGJO4N2dckZheGHrfp7cAgJfA5XSzDpCOwijmud07NQHWxwhsbqMXmKl9VFRIOMbIkieCJzXkJgGmEh2u6c257mDeGXp6l1GniIVJlDXU5HNAMOzc3MEKFFF2k8YmnTkCSXtpYQnq9ZVPzFIZs8mpYo0vIxfT49XXMw9S3QqON3vip5FgX+e1czU2afdFzxNI4fphIDeokx03qROZ4EYPfx9MDMh5QVje9dqSeu0BqnaiDQcaZ67N5z87iFWy0eIlS/v96Rplr/2kjlJhojkBAL9tj2JjRz5ybwAsHHW6JF6aBOHLwQMNWo2v33RQfkOwAXZnEnDQ3bwRfmwK64juJE+7bzEm+kqhdXg7rlQGiczCG1y0OrgZc4azuA48lJ0ueu4mTQKGNWLsgum5RxSinrPK7z5HHZDDno2Za131wQBSMOEW8Aq1UQYDF7QmOheD514jmW+UmpMZYqswvVEadOPf0dmu0siYY4cjp4iWVCmKweGcAUurnmWCOPXBBZyTNJ+9Z06KOoABtA6L6wM8CBJmnm0l5qHCodmjNur5uJOPB94jwRS3o7XCB5LrhQUhZ1jnoexpkWSyoV3ZzVwna+KVNdmoOB0jryyMroIxm6u8ahek6Y RTDbd7Fq nm8HmA0jpRD9jqW40m27D2+9HlmZeNZ2G4RXkidbQIs63AjC0z6lxhVbPaher1TUJjuKakyT2tH/XhjclSnOKvINKVRzAFbVcEZ0ZQiUFId4aVMaUKB1UpO0s8pbXMos1vNtsK9X9mlEINqSLCZzv4ty7jRQdNlUHcejvYehMBGWEksjsB5rhA44K36vsGW0e7y37ypfcWcJ7lNwZLnM9ZJ10DtZzmWFBkwVR7Y1SwfJbqYlCc8gANWX7TCT5wAQEN3bi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhaoyang Huang This patch fixes unproductive reclaiming of CMA pages by skipping them when they are not available for current context. It is arise from bellowing OOM issue, which caused by large proportion of MIGRATE_CMA pages among free pages. There has been commit(168676649) to fix it by trying CMA pages first instead of fallback in rmqueue. 04166 < 4> [ 36.172486] [03-19 10:05:52.172] ActivityManager: page allocation failure: order:0, mode:0xc00(GFP_NOIO), nodemask=(null),cpuset=foreground,mems_allowed=0 0419C < 4> [ 36.189447] [03-19 10:05:52.189] DMA32: 0*4kB 447*8kB (C) 217*16kB (C) 124*32kB (C) 136*64kB (C) 70*128kB (C) 22*256kB (C) 3*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 35848kB 0419D < 4> [ 36.193125] [03-19 10:05:52.193] Normal: 231*4kB (UMEH) 49*8kB (MEH) 14*16kB (H) 13*32kB (H) 8*64kB (H) 2*128kB (H) 0*256kB 1*512kB (H) 0*1024kB 0*2048kB 0*4096kB = 3236kB ...... 041EA < 4> [ 36.234447] [03-19 10:05:52.234] SLUB: Unable to allocate memory on node -1, gfp=0xa20(GFP_ATOMIC) 041EB < 4> [ 36.234455] [03-19 10:05:52.234] cache: ext4_io_end, object size: 64, buffer size: 64, default order: 0, min order: 0 041EC < 4> [ 36.234459] [03-19 10:05:52.234] node 0: slabs: 53,objs: 3392, free: 0 Signed-off-by: Zhaoyang Huang --- mm/vmscan.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index bd6637f..19fb445 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2225,10 +2225,16 @@ static unsigned long isolate_lru_folios(unsigned long nr_to_scan, unsigned long nr_skipped[MAX_NR_ZONES] = { 0, }; unsigned long skipped = 0; unsigned long scan, total_scan, nr_pages; + bool cma_cap = true; + struct page *page; LIST_HEAD(folios_skipped); total_scan = 0; scan = 0; + if ((IS_ENABLED(CONFIG_CMA)) && !current_is_kswapd() + && (gfp_migratetype(sc->gfp_mask) != MIGRATE_MOVABLE)) + cma_cap = false; + while (scan < nr_to_scan && !list_empty(src)) { struct list_head *move_to = src; struct folio *folio; @@ -2239,12 +2245,17 @@ static unsigned long isolate_lru_folios(unsigned long nr_to_scan, nr_pages = folio_nr_pages(folio); total_scan += nr_pages; - if (folio_zonenum(folio) > sc->reclaim_idx) { + page = &folio->page; + + if ((folio_zonenum(folio) > sc->reclaim_idx) +#ifdef CONFIG_CMA + || (get_pageblock_migratetype(page) == MIGRATE_CMA && !cma_cap) +#endif + ) { nr_skipped[folio_zonenum(folio)] += nr_pages; move_to = &folios_skipped; goto move; } - /* * Do not count skipped folios because that makes the function * return with no isolated folios if the LRU mostly contains