From patchwork Wed May 8 22:40:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13659211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 749FEC04FFE for ; Wed, 8 May 2024 22:41:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 03C5A6B008C; Wed, 8 May 2024 18:41:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F2EA96B0093; Wed, 8 May 2024 18:41:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA80D6B0095; Wed, 8 May 2024 18:41:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B929B6B008C for ; Wed, 8 May 2024 18:41:53 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 43B371A0CFD for ; Wed, 8 May 2024 22:41:53 +0000 (UTC) X-FDA: 82096702506.04.5BECF39 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf13.hostedemail.com (Postfix) with ESMTP id 61B782000F for ; Wed, 8 May 2024 22:41:51 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TLo3C6sa; spf=pass (imf13.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715208111; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DOx+XLJ6NG229/wS5wOB5lVEjUmMQDyjSrW3ljEjCDo=; b=sktrh8FCKDpSGvVSDAqYEkuQatF6mY+wM5T5fUDx1VndgBazZo4hRMzeBa28llVSMzrfaF Kyw8rOB2yNTEbn4pc2JLBk8CkD4GgBjqRMqUX/I4/SScnhQTtzTjWdXBsKHF0PX7r7bC0y EvCwpa5INgz4EDB6g+olTYnfYhOCkf0= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TLo3C6sa; spf=pass (imf13.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715208111; a=rsa-sha256; cv=none; b=1FVlHtcsJMMuOSdjBBGdaI4zhSDgG+ckxd7VQTMunVWxLuTrVR9yTMItsFZXE61W1k2BIC g+udscXSn3dc/bBXh0cDPEyga/1j6hXuLJyDO9/9VwStJNEAGTmLh6JLufin+21ec4LKq9 +mcr69+9T1+lFGuXn/LR9Zdmh2NLC3Q= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2b3646494a8so241120a91.2 for ; Wed, 08 May 2024 15:41:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715208110; x=1715812910; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DOx+XLJ6NG229/wS5wOB5lVEjUmMQDyjSrW3ljEjCDo=; b=TLo3C6saTOLrnosYOt/1xBmXgWNgMZkLzPAyvenoqq0oAjga8kWqVhD7tpZnTU4KCb Q4+3O1ULQzKWfu0eMJqp6lsLNWVjpICuxUtoISr+nQAVzxFRwdWbZ46tWOH7q+DgzfgZ ni7xzG4uDwSX501nTb6we4dkh4V0rw124ruCxnRVGlOyKrzeN37Lif+4tsjDL7uEI9Cp Ardikaw/YU+drnc4hWf1wcP3sYOu5Tyg9JCI1SaGbPpPnLQWbYpUYn9Dy/+BN9v5RDhU jzlyyu5S9M/5xenRanQwDXLrQ6lavVW9hUkgRTOF9GnKxC+mNYZTWDqZWKIeMohEjtAr 2GAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715208110; x=1715812910; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DOx+XLJ6NG229/wS5wOB5lVEjUmMQDyjSrW3ljEjCDo=; b=vVxQE2wkXORLN9MCHaCL+zcjJNueUSNk/V0yuC7vYdUhvC6xTYjjhAfVnnbZ95RrGO ZTYESpgR2RNREpDsIsst0tdZxUVKAsRGrZcGWwFi8oxb9P8GwkerDh3szXDbKg0HV3Rs 7qJ4l7L7HCjHvcGub/5y/tNhgzXxGRhFXzR+HThyUAdgWpkr3jOqH4rIb9clM5wzd9+O PuukpAzFO5D1euDFvdxoPpytiT1kPyRkYP0imT8Tp/doPiEdwk/pyxA+KkHgRl2gsD01 a/96EjRS5MMWpFEofsc+WsHHa/TyZ/ydMuZg1Ir2oA7fDSuDcVOJUfvyCAlfpCysP5lN 5Rjw== X-Forwarded-Encrypted: i=1; AJvYcCX7PmjhNZh4yaj4ZfyEXO4FieBOcRT0TneO7Hvfr20pJPZU7sGnn4XhlIf+HvNEgAa24A6E4/UgMcNbYUjMqA9m8Io= X-Gm-Message-State: AOJu0YxZVjhsAuqeQvxGF4ZNHISae2Jbw+wSgA1jH6sJxSoQ4bb5Ts/n sphbWmzH+qIiNBWtINEK9Hdk/REOv+AOZT1o03QSDJCPI29k+aHP X-Google-Smtp-Source: AGHT+IEFp0bWFb71EA7kiwqcgB9bekLg3MgzbqqZFw9IyUlHL4NxSW9RrXf+dnY9e3fmsofVV/uA3A== X-Received: by 2002:a17:90a:fa8c:b0:2b5:1f7d:7eae with SMTP id 98e67ed59e1d1-2b6165c4fbbmr3664457a91.24.1715208110215; Wed, 08 May 2024 15:41:50 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2b67126b666sm102082a91.34.2024.05.08.15.41.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 15:41:49 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, hannes@cmpxchg.org, hughd@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com, yuzhao@google.com, ziy@nvidia.com Subject: [PATCH v4 1/6] mm: swap: introduce swap_free_nr() for batched swap_free() Date: Thu, 9 May 2024 10:40:35 +1200 Message-Id: <20240508224040.190469-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240508224040.190469-1-21cnbao@gmail.com> References: <20240508224040.190469-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 61B782000F X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: i8cnm6mxx99wfeogwbaznqt9m9ztdg51 X-HE-Tag: 1715208111-739799 X-HE-Meta: U2FsdGVkX18E6z+l4SB2gjFiM3wO2sXA7GW+NbD9WsPtB19N9IgJFFqQrRtNlcPgf9XV2OXREDAjseq8cp84CHSm4IC9uzFEwl8cRswPJycJ9N95OPPpSabjSRZZVCBWekk8tWh91ny7zWHcItsXHxMJkm4gLunl0viOO6Xv6FO61wN0drs0mtzEy6NaUEC/CM/vQR6dxQLpo/qMjspvTlagU58my1mCIxtMDqVdy9KH8Z0p9zPZrc/njLpQKdHWmkV+spAJR0s6bfl46zlfkr/uXabECyQnpXxEGZ4pV3tdTiiql/wpCeuiFTE1zvMBoULLkwvb6J/Bh55WGB+AHFoEoBM4CyJtTfFSQIUg8Z5TY44txUh/1OEFQ09suxVXNW74YGt9e2U6DlTG3idcA5ah8OjGXZGy9Dw5WJ/NpE4fc3KKUMnyPdConTOsuBDzOowVW/zW8o6S1eV9e7pYKm3gB+aa+X7eJZScL7RC3p1MbSPz7TgajRb13NcC30iyzv9z6PzwYK7j56eGzsObHL6Shp7FjMwx+J4SgxEEUTugkne7biSDJws/svjhHNy9PSFNTgpauW66nnZyRpJjQ1UUtUIM/QEwsrzZGOq9c06EMEXJHAk4mKYfxGc2qnoa4YJsFX0Z9uyAbZJrdZ9x9PbtRiERDqlohM6Wv2SNzExUlZYwZcIAbVWL7pxJCxvjPqWAygLjG/bv980p/Z1cVUkPJUfPi+w7kNP7KAlNScxaZUeSKiiCs9rNSGtcphN6pdixUSz7FvAoL4dQgNpzJaVyEp555ieru2kGT/1KInuOnwSdLOa4o2p/tHTeLVS6rmyAIE+jg9MoQhQ0f7rqpWnG06pN3FoNzIkHsjRl2KroTIZGDTg6AN0XcWZ1YtuabBB7nyu9aEJwwXfnYKGVjEuuTEtNn2tnUDh2FTwe43IzomwQSWKBZgyPydQdxbfYS/UNvqEeJ7TRW8g8/qN OA+3taJh 1MGwrct+AYuPJC+3JnFJp7x8iRmHn8ZzoOpzmJXIm22rKB61s+vEpog/LcEjWw42cleRrd3VEm16wbD0Fb2wBFzadGsK+THDnI4xMfgHOgr4A+YaxGvbpgJdG49ZCLYTEwjvFNGe3IdNCxVX/kwsAsjRqvR6mAsU4QabDDnepMnZJAq3onribj5KUR1m6EDD3qkbyOuyGLV4Y1JeU7evvA+KNKTsZ6XNGSQU4ROSugd8TgcjKUvNvHsvLk9BLDGdepN20dvmEJOiWt1IO5Tj8U50+9IKaZ9ZfXCsS5/7P3sykYJaDZAiL2lwo52UsDADFENOtlONAFXFh4FwELhxNrzz1oqUrG0u6cQ2W+tuqxDsHunfJiiQsHu5w0hSiIyBFhBwhbTJ9YVGOr9l+zbk57D2lf+vcKAvr03l84LcGQsihtppwbTVzgam/hhEkabdmSVE/2AxJ8YzgkYnWfJn6WB/oHem+bAFFft1eeGqzWlsjfipU+ELQ8AU5fy+Mtai2dLaoc3W4ZDqkWZG/dFCcQWfhAUodXKwlv4lvnJM1IFVFBSTC9oqVKUfBkVgTDZxiPM1c+hypw89iPubuAK0M5Tf5YAeQ28/SX/S5m+2D77Snck4JgiWT5emS1JWoPYSi22c4z3YAZZZpdJe5r2vPYI1kdL+dOFi3fNb9AN3eDURnQu/RBI15hCloq3CYDTW8g3eU3qkrAPlrr3UNzb0rqDJ5ig== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Chuanhua Han While swapping in a large folio, we need to free swaps related to the whole folio. To avoid frequently acquiring and releasing swap locks, it is better to introduce an API for batched free. Furthermore, this new function, swap_free_nr(), is designed to efficiently handle various scenarios for releasing a specified number, nr, of swap entries. Signed-off-by: Chuanhua Han Co-developed-by: Barry Song Signed-off-by: Barry Song Reviewed-by: Ryan Roberts Acked-by: Chris Li Reviewed-by: "Huang, Ying" --- include/linux/swap.h | 5 +++++ mm/swapfile.c | 47 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/include/linux/swap.h b/include/linux/swap.h index 11c53692f65f..d1d35e92d7e9 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -483,6 +483,7 @@ extern void swap_shmem_alloc(swp_entry_t); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t); extern void swap_free(swp_entry_t); +extern void swap_free_nr(swp_entry_t entry, int nr_pages); extern void swapcache_free_entries(swp_entry_t *entries, int n); extern void free_swap_and_cache_nr(swp_entry_t entry, int nr); int swap_type_of(dev_t device, sector_t offset); @@ -564,6 +565,10 @@ static inline void swap_free(swp_entry_t swp) { } +static inline void swap_free_nr(swp_entry_t entry, int nr_pages) +{ +} + static inline void put_swap_folio(struct folio *folio, swp_entry_t swp) { } diff --git a/mm/swapfile.c b/mm/swapfile.c index f6ca215fb92f..ec12f2b9d229 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1356,6 +1356,53 @@ void swap_free(swp_entry_t entry) __swap_entry_free(p, entry); } +static void cluster_swap_free_nr(struct swap_info_struct *sis, + unsigned long offset, int nr_pages) +{ + struct swap_cluster_info *ci; + DECLARE_BITMAP(to_free, BITS_PER_LONG) = { 0 }; + int i, nr; + + ci = lock_cluster_or_swap_info(sis, offset); + while (nr_pages) { + nr = min(BITS_PER_LONG, nr_pages); + for (i = 0; i < nr; i++) { + if (!__swap_entry_free_locked(sis, offset + i, 1)) + bitmap_set(to_free, i, 1); + } + if (!bitmap_empty(to_free, BITS_PER_LONG)) { + unlock_cluster_or_swap_info(sis, ci); + for_each_set_bit(i, to_free, BITS_PER_LONG) + free_swap_slot(swp_entry(sis->type, offset + i)); + if (nr == nr_pages) + return; + bitmap_clear(to_free, 0, BITS_PER_LONG); + ci = lock_cluster_or_swap_info(sis, offset); + } + offset += nr; + nr_pages -= nr; + } + unlock_cluster_or_swap_info(sis, ci); +} + +void swap_free_nr(swp_entry_t entry, int nr_pages) +{ + int nr; + struct swap_info_struct *sis; + unsigned long offset = swp_offset(entry); + + sis = _swap_info_get(entry); + if (!sis) + return; + + while (nr_pages) { + nr = min_t(int, nr_pages, SWAPFILE_CLUSTER - offset % SWAPFILE_CLUSTER); + cluster_swap_free_nr(sis, offset, nr); + offset += nr; + nr_pages -= nr; + } +} + /* * Called after dropping swapcache to decrease refcnt to swap entries. */