From patchwork Fri Mar 8 08:56:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13586584 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F4214C54E4A for ; Fri, 8 Mar 2024 08:57:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 594F96B0352; Fri, 8 Mar 2024 03:57:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F5DC6B0355; Fri, 8 Mar 2024 03:57:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 34A236B0357; Fri, 8 Mar 2024 03:57:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 199896B0352 for ; Fri, 8 Mar 2024 03:57:11 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B2C9740768 for ; Fri, 8 Mar 2024 08:57:10 +0000 (UTC) X-FDA: 81873267420.08.3A4DFB7 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf28.hostedemail.com (Postfix) with ESMTP id EB5C7C0006 for ; Fri, 8 Mar 2024 08:57:07 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=U2H9wCtO; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709888228; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=HnD0cp4DmVlF6OCB17KJ6M5uy70a7zM7SvCAKQx/WIc=; b=AgQ++XTAHHWdHADJ2QAmY1YhyY8MXPfzmECMfSZcpy0WrQSpLXrxgSeSeSW6Qy39lkLMuY n9yiu9Dvz6Iay7x8lnMqfy/WIbWZ5M1pzeUUGqe4ut1c+BEGAbyiACZdyxb7gpPthh0mzb dfNTtqGoQ1QaR2A1XX4m0IJDaEC/jrU= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=U2H9wCtO; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709888228; a=rsa-sha256; cv=none; b=upKO2pxJ6+xvK8S21ERSMfYfEwzKTQzb2TcUHlBdEPrZlUWWRynBsvOKE+eovJMo6W/foF 4oRH9/l6I47XtNG3XbONJUXUtS7UGI4FX4ehstjuR2kjy6BA+BJ2kKn2JNHuZ5RdCKhtPb ugt9N3fyBqP+kUkQNXLk2xqBkwV+uGQ= Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1dc96f64c10so14537715ad.1 for ; Fri, 08 Mar 2024 00:57:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709888227; x=1710493027; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=HnD0cp4DmVlF6OCB17KJ6M5uy70a7zM7SvCAKQx/WIc=; b=U2H9wCtO81xFRCEqWldEckyxqIRBTlh8a1Ox7KMK+h3vEIA39hWB7Ft/IJw+2T7XPp NrIRycbB2jFRUDI77ykNE0gRlDdvJrxNx6vJOJ4b7O80qRe+2v+dqp8kuIBulS8Ofx84 EXnPG44WvP8Jwu2Da+aZCPSIfV+Sbqm3q8vtxdCfJD+zmfECZZ0rZ4na4UKMZjrjrdgp BfciqMesTnCsuvYGgOf0K713kNMFSmpsax54V4mafGhAupT/Z7UD5PyycW2ElX43Op4L pK37e/mBDHi1El7bpYzJElsnG7oBmUb6sO3L7fON/3UzBt97Tqp7PnSYu9X4f9K5DZ+v DyGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709888227; x=1710493027; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HnD0cp4DmVlF6OCB17KJ6M5uy70a7zM7SvCAKQx/WIc=; b=UPs3hAbQgwDVHPtQWdHYesByitSN8Y9h5RAMuvfsERXsX/RbTc76PGWwJH56438jiy iqj/VSh0lTQiGmpjh/pQysNYznwv2IgaQK+gyzY0n9gffjEEWFVN82dYSvr1f/L502Gw hCg9bvRmvgvYdM/o5AuJi1sxXFW4dR3PCGA3+1taLJfbAJUzckFuUkQwNPd9PAQFVMGi Ab04bHkLTgAjQYZ1W93HZqv2rAl1Y+yNsfNOAa/4J/0tLnDwB5yFw2G23rclc/LeCtB4 5ijKwzDOa+DDwQMRTX+EvDkeHb2jH5yPi9z6ocXkHcaBA8Rl72KIjM1g/V3MZRt2SODC umKA== X-Forwarded-Encrypted: i=1; AJvYcCVcPxJ/GhiivT91/sQzWl7cNLS4FKC+dpfC3YysWLRQeCEHrxhlqh+5VaAxM5zcKpscp5Y2muXILmKWAmWpYJwM8Lg= X-Gm-Message-State: AOJu0YyX4kpAdABRBWQa5+xWWG66r4MBaU7vitdlCKmrsxebKhqliXyo hENb7wPfg49OU/KpVKVoirNonQ5EvJE1jOeSLs8i7D8VW7IlwOo7 X-Google-Smtp-Source: AGHT+IHnYnGsKBEXeRr6fqKiSG7wnA+FFNHNLSUhzlakvHjpSLfIXu5EBe7jE+OUdYg937Eosmm8uw== X-Received: by 2002:a17:902:eec6:b0:1dd:6211:e26 with SMTP id h6-20020a170902eec600b001dd62110e26mr2748179plb.15.1709888226610; Fri, 08 Mar 2024 00:57:06 -0800 (PST) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id z9-20020a170903018900b001dba2e99a9esm15867890plg.90.2024.03.08.00.57.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Mar 2024 00:57:06 -0800 (PST) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: minchan@kernel.org, fengwei.yin@intel.com, linux-kernel@vger.kernel.org, mhocko@suse.com, peterx@redhat.com, ryan.roberts@arm.com, shy828301@gmail.com, songmuchun@bytedance.com, wangkefeng.wang@huawei.com, xiehuan09@gmail.com, zokeefe@google.com, chrisl@kernel.org, yuzhao@google.com, Barry Song , David Hildenbrand , Lance Yang Subject: [PATCH] mm: prohibit the last subpage from reusing the entire large folio Date: Fri, 8 Mar 2024 21:56:53 +1300 Message-Id: <20240308085653.124180-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: EB5C7C0006 X-Stat-Signature: wf95p6kab7z4wem16pg9udo17n1p35s1 X-Rspam-User: X-HE-Tag: 1709888227-163453 X-HE-Meta: U2FsdGVkX1/XD01xhkSUrr73qjH+1+tlG3yOy1crI4g1Ji9xs5fPOV+YLo0bAa4WnAl6RVUrZVLZW2Qzp0ZcfOtyxfZboUeuhFjAybOXMY2nUphBnlk2v5fkJTuHq3Z1iyby59x4VYRIYE9vfGqH3K2JxIN6ER/mEwo8/IyseH8WeX00PcZv+joKZsKkt/EqsNQ957NOguEhh1FNo6neKPKMa6kczis1Zl9lLYHMauvh28rxFqBhffbN3J+QCacgF9IxRosj4B6V3eDxDIz9x41uJ6JVYd/OLPAbw26bbzHjoSrg9xhO12eRZKG6ZUNMEShDwv4sI8etBdGns+5tBzWiGcGf10BUwikEL+mGqyw4pfVwg8TbMGnT0VAW4oTtPVvUN/PBK9+Nq5MHHB12gCDIhqh0qzktQvF5oj83GqdFATKSucwPznLJt09xcnvV0dMJ9QLSafmI6wXoGNkYJCtKZHOTL3grIGrLvH9mW7feCnaqNftOKaMmFgo8HwD0K0jvRt+JWkLnCZ2ZPyNWW34BCvWgTuoUOPOJdNeVcWj/oPemmXSPFax7WScHqmFMFfVSbN1/5VHoqqLiGVFvrxhbj+qY3nY0MhQWA2LIJ8iTG2/a9ZKLDJM2UoFmmSaecVqZ1eKrxMR20jHDO6JPTVVyJajkEcxiAmnu7L+azp7D7SWmq93tUeVnKT/KDqDnOn98Z9wbWh2pWWx7H9DSIfwCdHHKOtzDXw8A5boM9qX22DswQjQaUiBxnYx/YXo/ybihZK5jCcR0/iqRGKBR9jEx2FUS9B9fhOnfomdoY5rcDkAsh4sbsIR6h+fMByHZ5WrwdE+1fWwCqK0LGMtyEMLEfxVdvMSeQliizzMzNhrWaSc/UEFgl/sN2pj8tF2fdgq8ZRmu3Sc3HHxTCz1xTv2InU690lWCtw9Ep5cyU/CJbSh3mlb7wBmmRla+kxy26qqxdeCVi8m+JbkOIJJ FTxtWJyv N8cBGhOPP0+rxqz2nAW2ukamWdMTll8BTlET85cglv2kkmExMQAKuxeeoaQ1cwSxvrlexSrreN92AbyFdagmPfQ9L0PlCQtSayOdpsIoIfvPI3hDegKIoxZuA2b5m5vLc4B/ROqlGCxh/TR1kXHkfG2tEtpA+9R2/6g3BL7Kz+ewJB0OsKiA2IerEuCCkv3i/ipoTAenbahcLWijP7EFOusodq00e+85wghkaqCP+NtPnyI5Wcw7XYI9bawYROh2z058wUUXGUIAot1aioRGq7t9j2NqdtA/MIvi7NIegAqzogxgh7K6Wx/yQ2h75ASK6PlWUIxdKpJ8HDlVim1hNKQ9SLZa7I91hK+ahJwmlRBQatjpsc+VJuGzTQvPzLB5W6b17hvZxcb3Vw4G/CGJdF+rwdZR8gS6Jn46pEQEpObJ7IyL/K1XndziHm+CEf2MDByYtAPpVv2Xn+Ff5EVlkPuzWzeZVokq95KobyYo0P3Jx72tFW6ZoP9mqOJiUPHCfqsMafRXf8FlgOWgUNHBsfju0hpgulrtcUrdG/eqrheuNvfj6UgBWsQuWBvBvCIpCEpLo1SHNEmi2hwADGJ8fT2g2Dw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song In a Copy-on-Write (CoW) scenario, the last subpage will reuse the entire large folio, resulting in the waste of (nr_pages - 1) pages. This wasted memory remains allocated until it is either unmapped or memory reclamation occurs. The following small program can serve as evidence of this behavior main() { #define SIZE 1024 * 1024 * 1024UL void *p = malloc(SIZE); memset(p, 0x11, SIZE); if (fork() == 0) _exit(0); memset(p, 0x12, SIZE); printf("done\n"); while(1); } For example, using a 1024KiB mTHP by: echo always > /sys/kernel/mm/transparent_hugepage/hugepages-1024kB/enabled (1) w/o the patch, it takes 2GiB, Before running the test program, / # free -m total used free shared buff/cache available Mem: 5754 84 5692 0 17 5669 Swap: 0 0 0 / # /a.out & / # done After running the test program, / # free -m total used free shared buff/cache available Mem: 5754 2149 3627 0 19 3605 Swap: 0 0 0 (2) w/ the patch, it takes 1GiB only, Before running the test program, / # free -m total used free shared buff/cache available Mem: 5754 89 5687 0 17 5664 Swap: 0 0 0 / # /a.out & / # done After running the test program, / # free -m total used free shared buff/cache available Mem: 5754 1122 4655 0 17 4632 Swap: 0 0 0 This patch migrates the last subpage to a small folio and immediately returns the large folio to the system. It benefits both memory availability and anti-fragmentation. Cc: David Hildenbrand Cc: Ryan Roberts Cc: Lance Yang Signed-off-by: Barry Song --- mm/memory.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index e17669d4f72f..0200bfc15f94 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3523,6 +3523,14 @@ static bool wp_can_reuse_anon_folio(struct folio *folio, folio_unlock(folio); return false; } + /* + * If the last subpage reuses the entire large folio, it would + * result in a waste of (nr_pages - 1) pages + */ + if (folio_ref_count(folio) == 1 && folio_test_large(folio)) { + folio_unlock(folio); + return false; + } /* * Ok, we've got the only folio reference from our mapping * and the folio is locked, it's dark out, and we're wearing