From patchwork Mon Aug 19 02:30:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13767728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8F5DC5321D for ; Mon, 19 Aug 2024 02:31:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0CE626B0082; Sun, 18 Aug 2024 22:31:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 07F1C6B0083; Sun, 18 Aug 2024 22:31:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E13306B0085; Sun, 18 Aug 2024 22:31:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C73506B0082 for ; Sun, 18 Aug 2024 22:31:54 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 76749140F30 for ; Mon, 19 Aug 2024 02:31:54 +0000 (UTC) X-FDA: 82467419748.01.355EA9A Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by imf03.hostedemail.com (Postfix) with ESMTP id AB09E20023 for ; Mon, 19 Aug 2024 02:31:52 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Ia0A1eKE; spf=pass (imf03.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724034673; a=rsa-sha256; cv=none; b=S72+k/ErOxjYqqv6vfvCEJkuWhaTlnGLH6d/eGjIY7m1B72tECpri1v2wDC7aeN+ZPPI7/ 0nIgsduPH5uRtq99ZZVvNNmLiMiDSKiuJ5sXA2n2PZqWJcSOBEkoASCSG31kn1tXHaImPT 5f6tVX5S3WYurV+t3hylzNe2JR6JmIQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Ia0A1eKE; spf=pass (imf03.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724034673; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qHbAGWI5YhvG8+e8FC6gBqfnjsq6DXdxHGGYZ2mM3LI=; b=ZXs1D1gw86kvC0rv2rjDR1s0nESu9lqNOmNHIGONyBnO2MR/RPSN6THY2BIDz8G8A8LJvc bzj612p4xgVRa2rqi0kEtQl/Qs5VSVYk7qbWaPaHnBWxmeeIqjVbId4eMTumNXq4UWpRbZ y/MJgfi1hA0fhrh8qNFSDdteqYHH8II= Received: by mail-qt1-f181.google.com with SMTP id d75a77b69052e-451a0b04f6bso25730171cf.2 for ; Sun, 18 Aug 2024 19:31:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724034712; x=1724639512; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qHbAGWI5YhvG8+e8FC6gBqfnjsq6DXdxHGGYZ2mM3LI=; b=Ia0A1eKEnz+MlUZAXT4x9qIb/0CqcgfIBfhHoD+DrDsI/CCV5Xt7nufIElSN/UwX/e UosUNduuSedQtCS1uZWYf5kw1WPspiB4WBehXnaCc+8wBuXhbYEry+Cks39ZcPeFPPBA 86Q4RNCOYJeLzGSH6sgRBME4pplbgJcTyylj8jTKnSP4g1f0NoI0/rsfAYzez+T4RgUm AXrvFpPC7RL1uQCM0RSttaMbLktmsEW4iOYeKx4Oyxz57a0Gh2PxFfNabOORXUK3urJE ZlDekp7fPrDdvU1o+X4VtCGlSU4nFwUOA5uo8U1QfyZ17OEVSq4mopWp7iRHGys9HLqb tWCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724034712; x=1724639512; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qHbAGWI5YhvG8+e8FC6gBqfnjsq6DXdxHGGYZ2mM3LI=; b=ozC8+88qZtN8JHF0MxtSwyPQpdC/L2taPeKSd6mSq/u+hE3GPQRegHbNL3GmE7NOrT sRb7b8JBeZ2HhUuCsthM5UHWv2Xsyg/s2K7sRj/7MVSRuH6V0RxKCS8s/wBJ8/R0ubeB WIh1XXBGhnvsx3NoZgx254nOvfIQgGvfGCyYnBy9Cfj0/V39aRmGkbblIRYiNQ9FuQ0D 676JoG+04G5Y0Fl/Gmte8JIu4i7TJLojxrIpuf508E9lQPhPCuhdN2lwbr3HJJ0dY5rT V7qUadAfoSolZ+TOWVCWOsZ9UJyaJz/W750IPgQjYr0zVRtHsKttu96NNpM3GlifW8du TG+w== X-Forwarded-Encrypted: i=1; AJvYcCXf2iQf6f0yZs3tSPu949Sj3IziQDhI04Y+0vYIGO/K+ZmrikGFcg8K1emxYLxxUCoEhHiUZGxmGgpugio3b+aZmSQ= X-Gm-Message-State: AOJu0YxR5R5uPOYAVToZcv/oRumC5iZcXBifhIq1TB3t1jSxJzXYzGlF utuXYaGG1X/le+kuZnci2AATvyLyOzI/VHKxmA8Ya5OsF6fS5vBP X-Google-Smtp-Source: AGHT+IEpaTMKPf3VW76AVnS5nT7RUI9JcYK6qI1yzTo5GJ8U9EnjvGJNpHkQcNL620aPLntPx4HfJA== X-Received: by 2002:a05:622a:988:b0:44f:e132:14df with SMTP id d75a77b69052e-4537425375emr117719541cf.21.1724034711670; Sun, 18 Aug 2024 19:31:51 -0700 (PDT) Received: from localhost (fwdproxy-ash-003.fbsv.net. [2a03:2880:20ff:3::face:b00c]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45369ff4bc2sm36832461cf.37.2024.08.18.19.31.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 18 Aug 2024 19:31:50 -0700 (PDT) From: Usama Arif To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: hannes@cmpxchg.org, riel@surriel.com, shakeel.butt@linux.dev, roman.gushchin@linux.dev, yuzhao@google.com, david@redhat.com, baohua@kernel.org, ryan.roberts@arm.com, rppt@kernel.org, willy@infradead.org, cerasuolodomenico@gmail.com, ryncsn@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kernel-team@meta.com, Shuang Zhai , Usama Arif Subject: [PATCH v4 1/6] mm: free zapped tail pages when splitting isolated thp Date: Mon, 19 Aug 2024 03:30:54 +0100 Message-ID: <20240819023145.2415299-2-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240819023145.2415299-1-usamaarif642@gmail.com> References: <20240819023145.2415299-1-usamaarif642@gmail.com> MIME-Version: 1.0 X-Stat-Signature: wn9mcoyabbbj97hzrd1j5ur4dp4153qq X-Rspamd-Queue-Id: AB09E20023 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724034712-482736 X-HE-Meta: U2FsdGVkX19tHRkGBSWy+TxYFDgjN3rSDC1TA4pKj6EPhJkdgvmwvZ4ra0DcE+oxsWhYopLDPt30AecJIpaoKBo1sruxv5PHwGUmSw7zZzuAYwv1kf2EnB9ck+/gfrXqarLHB/brkWNWFDrkjEbgiQnpLw1tzFNjYJQtO0pMFyzwlpRG8GfTlL1hZrtICVF09V+0LpChf1XMyswfDl3HdUKd8oweX+DBjpV0BW8ygM4oRllQtluJwiFCt/yFuFhorKAwor/95QrNElbOUbLvJ4F/xs2vGnNXuUJgxMVPT2uKENr2J6KKeOgilnf3gNUG+sxEKZRyb9MNjF1Gn9Jee73wUaORwsqPdHCrR7JMb9TyxiGIo1ps97Vm7ZpWk49eH0jmo7UdUviK6hD7tWEi29jlu60dyAizcQOuBTYF7dVeSGdVFYu06F+tokFRcC2nCCn8/0pWkS5rLn2S1aJIhkpPXmPJ7AviLVUq6FMDgwsyZsOSoVzeXH8QyDIzFkG9f2nWmzK1XKQG90woHVHMpG9hhCh3LlAZumfPFWxPXJcH8xLYqv4f1V+Ikd3BaQ1gnyhGe9Xjv87eUkwL9Z9cH+NcgAmxbLUdTXmpi2gBJ5D+NpeDvU9lkk9ZM956qW2XpBMLueGDMgxHDKdX7OPt61dsLTYyJKrApmIDBBMKwH6iwlWccfv4kBJ5Y5PndtF0O/SR3qWiMyXHu3Bjws5fc0BBJhUlUQ2F7rtAWtAViW4vNslmvSDqCNsqH4UR7qBroG1PVo0MBg38OogaQ9624JQwsMqgrDuBs3La5xbFcCEVDe3YmXiFH+Fn7LB+V3T/Np1qjfdbDSj3dnHtW9us34g5QpVhaOU4SzzqH4p5I2UltOxrB9jd0QicOentqU6PSZeOOWvtU4/YTk9NlDugd6Cnu+lPvsteWC7xDsTQBRjG6cN3Mk/xUvwiBMO7EbPtOpNN9cqw9wY7r6JPQda wiGI0gAx vhZZrCopxSMq3qgIKqOfQ8tK6rsB4lycItq2c4CgdX8vjO7faK9NKwaGwejz74ySztymNbYs7RRADJrXuWsPcUtwUmhC+UV1qBUJpOQDzDOA44Svw02KLXqIr4RLyQG5KXKuwWpQ3J8IvQQG547yi+qHe2i4LJ4wqySnga0Rt5sVkicvINAJvehZLXGN9J37iriH2Yk01XrUjBA1H0u1JJ1ctl3gN573Bje/E/rG/9ywxHL3x+6GlTFb9q9hAx5au6hrOSz7Fw0Zt1Ty/nT2EJEBvcWKc9WXq7ZWCyylIXIqf4TfaqxAAOfilwFs/dyZ+/Jm8LvIOuScJp2sO7ykU4+cCXqG5llN2aUFRZ5rqVHedni5BJ7O7kqlksAR+N9rK2oyWzagFP5lHh8fMkJUV53DNvZv0ZljBhHF+yHPB3yYzrHBsgmfDJcQqWKovIvA/tJY051nNRncdbuKP3Xofy6YsiY1AfJkQQVW7fhIS4PA1wCLD3oqDOFa+2g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Yu Zhao If a tail page has only two references left, one inherited from the isolation of its head and the other from lru_add_page_tail() which we are about to drop, it means this tail page was concurrently zapped. Then we can safely free it and save page reclaim or migration the trouble of trying it. Signed-off-by: Yu Zhao Tested-by: Shuang Zhai Signed-off-by: Usama Arif Acked-by: Johannes Weiner --- mm/huge_memory.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 04ee8abd6475..147655821f09 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3059,7 +3059,9 @@ static void __split_huge_page(struct page *page, struct list_head *list, unsigned int new_nr = 1 << new_order; int order = folio_order(folio); unsigned int nr = 1 << order; + struct folio_batch free_folios; + folio_batch_init(&free_folios); /* complete memcg works before add pages to LRU */ split_page_memcg(head, order, new_order); @@ -3143,6 +3145,27 @@ static void __split_huge_page(struct page *page, struct list_head *list, if (subpage == page) continue; folio_unlock(new_folio); + /* + * If a folio has only two references left, one inherited + * from the isolation of its head and the other from + * lru_add_page_tail() which we are about to drop, it means this + * folio was concurrently zapped. Then we can safely free it + * and save page reclaim or migration the trouble of trying it. + */ + if (list && folio_ref_freeze(new_folio, 2)) { + VM_WARN_ON_ONCE_FOLIO(folio_test_lru(new_folio), new_folio); + VM_WARN_ON_ONCE_FOLIO(folio_test_large(new_folio), new_folio); + VM_WARN_ON_ONCE_FOLIO(folio_mapped(new_folio), new_folio); + + folio_clear_active(new_folio); + folio_clear_unevictable(new_folio); + list_del(&new_folio->lru); + if (!folio_batch_add(&free_folios, new_folio)) { + mem_cgroup_uncharge_folios(&free_folios); + free_unref_folios(&free_folios); + } + continue; + } /* * Subpages may be freed if there wasn't any mapping @@ -3153,6 +3176,11 @@ static void __split_huge_page(struct page *page, struct list_head *list, */ free_page_and_swap_cache(subpage); } + + if (free_folios.nr) { + mem_cgroup_uncharge_folios(&free_folios); + free_unref_folios(&free_folios); + } } /* Racy check whether the huge page can be split */