From patchwork Wed May 29 08:28:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13678362 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C88EFC25B75 for ; Wed, 29 May 2024 08:28:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 575B26B009B; Wed, 29 May 2024 04:28:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4FE4D6B009D; Wed, 29 May 2024 04:28:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 378376B009E; Wed, 29 May 2024 04:28:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 16ACB6B009B for ; Wed, 29 May 2024 04:28:54 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B5B3180825 for ; Wed, 29 May 2024 08:28:53 +0000 (UTC) X-FDA: 82170757746.29.DA9D04E Received: from mail-oa1-f49.google.com (mail-oa1-f49.google.com [209.85.160.49]) by imf30.hostedemail.com (Postfix) with ESMTP id E3CF78000A for ; Wed, 29 May 2024 08:28:51 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fRBHjD75; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.160.49 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716971331; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7k7udukJq/5lufC/QvLOUP0N512BAUWebXH46Q8HtFk=; b=bf60o0PaGw4ekw18Kie4yl7PiAfhNTlTQ9IkCvHv8bC42V4GK0BE9U8GT2fS3amxDJF4z+ VQLZg2noWkrCIa2l8tDjm4vTa1c71Bmd1pMR1xWgUXsc7HwekmG+Bgl520B6LbOTKtOpxU J1KtV54EZTIS+pLA2yA+3KRy3ej55KQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716971331; a=rsa-sha256; cv=none; b=l/eR9qAmpa30gxqREtRvzlZ+KBXukBc0mSN2LoIIFdhbGU3IWhmPS0F6hAg02ppLIx2GPj ntokW0EKYKEUaZavZbgjoQwPQk6lum/zAkte1ppZ0D8iNGiccfcbqKKft3Ijj2qjeXYtqi fYv0QmO0iOROOcwWRDNAYgOljegyXxY= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fRBHjD75; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.160.49 as permitted sender) smtp.mailfrom=21cnbao@gmail.com Received: by mail-oa1-f49.google.com with SMTP id 586e51a60fabf-24c9f2b7b19so873730fac.0 for ; Wed, 29 May 2024 01:28:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1716971331; x=1717576131; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7k7udukJq/5lufC/QvLOUP0N512BAUWebXH46Q8HtFk=; b=fRBHjD756wIF8FRj9AacI9sgkaq0awWGKeeIwK//eoi1Ncbe33MFK1buB5Z5xnSb8n fT8B4YBejVYCXT0yqsY6atuK1GB4Trb2gNDb0cJ2PzfrtRuFTRj3EIfx88MFycHHEPUb NgeNuA4KYgL+A5rdfAhgY0GFsNckKcMaBmgoPTSlWVAiRDDKTbuYK/GkSv1Q+i6JpixO vp7HwS91bOVAD4W9NLRtmvqfWd0ZTwpnEZ+Gnp37xIsb8o7AsPtNk7ycCcFI0/p6gtJS t7k98IR5tXP/QMNhmJ3yvknckJzObI49tDpM0V/TRoHSXt5a+Yrg0NarjhdCGzm15acl q8kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716971331; x=1717576131; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7k7udukJq/5lufC/QvLOUP0N512BAUWebXH46Q8HtFk=; b=VOb/w0g8vD3rj16vOLsQzACExYUhfktVEmb7omV6yPYkg5TK20xLf2OvGnjn7r+qzx B69T4mUtjzEggZjlx+FhXb33T/xDTOeUkMRx5UgpYMKnV6fM3NeOxfR++2wlGQcd/TvA upe16Zv2W8RpOCzxuadRH8nEoA7+C/BX07e2eqAqI0k+vkosuF/Psv+0ftCCImHRUFUJ BIc7FL9xXSTTguwYNlNEjBuM3vM440RhPlP3nJciiNEAPUO/306SSF18igQ58/Lsdix+ +X+B8kIVeSiAJgOQf3UZ02aictJtBjCG9Bvw3Rf1SeXKJrKQ0w3g5Azr/MGVUK5m9FwK P0jg== X-Forwarded-Encrypted: i=1; AJvYcCVURSTZjkhhQRCxUgoRpRACXy4W1GnM5OnXkstK5m/BBKfalIJ45v95p+fwx360VuzsBdgt92m5f0NuhE+Ko04PBJ8= X-Gm-Message-State: AOJu0YzJQFYKweZWT3SpIHgLKwASVFKo8hvUxcPp7LTHVsl5a7FYqt7a lv0U/gju/Y2gQmV6teEE4502ygvsRQEXzeCMihnvBdps6a5GHbQQ X-Google-Smtp-Source: AGHT+IFwms1cBvU+f9ojhWDg3nZKfLTfOVGOcCh0o9BJomwXX1OgvRueLY1XlvYRSI7zo2uCYFeifw== X-Received: by 2002:a05:6870:96a2:b0:24c:a414:3eab with SMTP id 586e51a60fabf-24ca41467f3mr15324961fac.32.1716971330911; Wed, 29 May 2024 01:28:50 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-702220d948bsm174916b3a.131.2024.05.29.01.28.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 May 2024 01:28:50 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, hannes@cmpxchg.org, hughd@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com, yuzhao@google.com, ziy@nvidia.com Subject: [PATCH v5 1/6] mm: swap: introduce swap_free_nr() for batched swap_free() Date: Wed, 29 May 2024 20:28:19 +1200 Message-Id: <20240529082824.150954-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529082824.150954-1-21cnbao@gmail.com> References: <20240529082824.150954-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Stat-Signature: sp7o6wsfe4apd8xrsr8b6o4nqzn77f41 X-Rspamd-Queue-Id: E3CF78000A X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1716971331-728022 X-HE-Meta: U2FsdGVkX19035okKhzh6yNI8Osgogn/u4TjoWdPR6upa5naWIzqDPHK7zX7EwhsD8mcyUa41wD2eWYDmGj85QmciYSo7mQPQ8ba3jWPo993fnKShOUZaL71qoX7pOUYnRO2tRyQx8azfI8Mg8Ud3KAn1ZqDk3jpeeiGDXL3QwnqnVFcLGbwjCfE4zoFk+DWEin4V+R5fT+G7iOee2ZGosb4bOgjYsHjFzUICSDX75eBreT3+5pFuAPc4v9ayuiUfilhalwftqMXPhcw4GbL3KytTMOAh93+e4CKgtVgKu0Qqu3hBcRzm+AyC3tD5w6AjG2+D0WWLZnItgYu+ycPFlGf8p2p/isvUVnKwHLjo0s7u3OpoxV2/pNWx+UdxUDljvzuwsPKouk6+M0SvzS5yR4ZOBcUeq0eqNQhy/qHnxjbitGBteaFIyu4t2FJDc/q2MDfv6R7kQkTxmF2V/BnfvCavQTTcdyyiFjAxB8UZufieboJJOHnnLeTfsHBSuExmQhdEo9kyjC3BEzn89ffec7GvvxkYR05fhlS295bg8Ylluje6dcIxigtHAqQ7UJqei/qQjDvY07hd8xecsyPP/g9+Wodn90mjqzxmr+E1+m9n1T136/9JpQO+PVz3vvujhxgNL3RSbLLTxJGl1eMCAHV1WtOSnEb9uW4afX3BTwpxEA6rfim+kki5QYWssPKpKjLOjoMvxGLbhheqXC4p97cswoa+SOrahKhXtw3Y7duKRkwEG1Hlho2WWvsrrr44Y5UkK3p3aWED0u0H7gOVM2LxAToaajxISzZU9GfCIQt31iHs8+eFoaWCyC+dFm3LgDfHHmbrWLzTuAml14aX3V8FiseV4jUTqcU/rlN0dq2Io1NYuXWdiO2fOsvx3ViB3Wg/d5Wk/Zzzd+mqg1ubax86cqof5ammIhChB9f8H46sIxsmUtkdMmnFzb0HSmFkfwz0wOhgt5YltunRjO VKPupbEb 2j53dPkY7oCVwLqFKMfwS1eWI5zyTYQfZ4wSg/xxXjyZoHlnHlbixn1tBtVvrp7gj8//Gq5hFBSXx74mYrNxNW04Cvz8wan3Gzc5arfuliUQUnrZA2Oku65uM6xA3Q4vTuIAmO072jTV/Azv5sJiyQZYDKUwRPYws3Hs/slwB7LH/qrr4QhuzJFzOCVn9c5RJXCUcsuWnCsU2NT/PZyUWSIVBHT3GI13iUlAWMVTP2isRS32hG4vr6BO5Y8jj+mDpy04GvFQg0PisH8bXxViNrOzCWZbQMAshbEllQmzQdW4p7zlQZDMcwCh3cNG/5QUODuo4ukJhwkfs+pLUg9TchzrBZF4E8F7QSyQifvgEU/9Qc/r/d/ts7g6B8Kt+/Gb2gVVZq5Iw4xuMaIYTpUpBf0yKre6mO7ZHQa4diwR4QMDnFUmnzYzmRB/fIcxP4dI/i6vmYEa6Vv0lTZmRhuepM/ptppffQJneZtzWeAvwY1JG6KdCa4k9kYlrxy0BQjj+8RihwoZNgbiiU0L89Ee5MeQ60W0GsB2nL8L9OXDfu8FtT49dTS9zmoqmzLv66h4myEUI8fCnj89A2ddtlJm/ZyJdSYAu7E0H2Hux7PT0AB8XhjQKrR5sc1nIjQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Chuanhua Han While swapping in a large folio, we need to free swaps related to the whole folio. To avoid frequently acquiring and releasing swap locks, it is better to introduce an API for batched free. Furthermore, this new function, swap_free_nr(), is designed to efficiently handle various scenarios for releasing a specified number, nr, of swap entries. Signed-off-by: Chuanhua Han Co-developed-by: Barry Song Signed-off-by: Barry Song Reviewed-by: Ryan Roberts Acked-by: Chris Li Reviewed-by: "Huang, Ying" --- include/linux/swap.h | 5 +++++ mm/swapfile.c | 47 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/include/linux/swap.h b/include/linux/swap.h index a11c75e897ec..45f76dfe29b1 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -481,6 +481,7 @@ extern void swap_shmem_alloc(swp_entry_t); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t); extern void swap_free(swp_entry_t); +extern void swap_free_nr(swp_entry_t entry, int nr_pages); extern void swapcache_free_entries(swp_entry_t *entries, int n); extern void free_swap_and_cache_nr(swp_entry_t entry, int nr); int swap_type_of(dev_t device, sector_t offset); @@ -562,6 +563,10 @@ static inline void swap_free(swp_entry_t swp) { } +static inline void swap_free_nr(swp_entry_t entry, int nr_pages) +{ +} + static inline void put_swap_folio(struct folio *folio, swp_entry_t swp) { } diff --git a/mm/swapfile.c b/mm/swapfile.c index f1e559e216bd..92a045d34a97 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1356,6 +1356,53 @@ void swap_free(swp_entry_t entry) __swap_entry_free(p, entry); } +static void cluster_swap_free_nr(struct swap_info_struct *sis, + unsigned long offset, int nr_pages) +{ + struct swap_cluster_info *ci; + DECLARE_BITMAP(to_free, BITS_PER_LONG) = { 0 }; + int i, nr; + + ci = lock_cluster_or_swap_info(sis, offset); + while (nr_pages) { + nr = min(BITS_PER_LONG, nr_pages); + for (i = 0; i < nr; i++) { + if (!__swap_entry_free_locked(sis, offset + i, 1)) + bitmap_set(to_free, i, 1); + } + if (!bitmap_empty(to_free, BITS_PER_LONG)) { + unlock_cluster_or_swap_info(sis, ci); + for_each_set_bit(i, to_free, BITS_PER_LONG) + free_swap_slot(swp_entry(sis->type, offset + i)); + if (nr == nr_pages) + return; + bitmap_clear(to_free, 0, BITS_PER_LONG); + ci = lock_cluster_or_swap_info(sis, offset); + } + offset += nr; + nr_pages -= nr; + } + unlock_cluster_or_swap_info(sis, ci); +} + +void swap_free_nr(swp_entry_t entry, int nr_pages) +{ + int nr; + struct swap_info_struct *sis; + unsigned long offset = swp_offset(entry); + + sis = _swap_info_get(entry); + if (!sis) + return; + + while (nr_pages) { + nr = min_t(int, nr_pages, SWAPFILE_CLUSTER - offset % SWAPFILE_CLUSTER); + cluster_swap_free_nr(sis, offset, nr); + offset += nr; + nr_pages -= nr; + } +} + /* * Called after dropping swapcache to decrease refcnt to swap entries. */