From patchwork Fri May 3 00:50:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13652246 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19FD8C4345F for ; Fri, 3 May 2024 00:50:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A42876B0088; Thu, 2 May 2024 20:50:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9F3036B008A; Thu, 2 May 2024 20:50:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 894316B008C; Thu, 2 May 2024 20:50:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 692D46B0088 for ; Thu, 2 May 2024 20:50:54 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id D5EEF1A0AF4 for ; Fri, 3 May 2024 00:50:53 +0000 (UTC) X-FDA: 82075254786.06.7F44D37 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf15.hostedemail.com (Postfix) with ESMTP id 1938CA0007 for ; Fri, 3 May 2024 00:50:51 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gg5L8Cr7; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf15.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714697452; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9ZtaFVcTSDB4ul8nNoaO/PcEtn9Rrjc3ITa59z6anjw=; b=7HOGwnd5uO3lZpO/eUTKE9Ms7CpAMZsGIcDgGKWMqpWTNmzs65HX9T3Ocnj+pj4OEWTh8F wPBWSMDtUlWa5EvjbcDV659BpTTYzQlj4UWop+Eiz6bJWi93t1ZAhvcxZdkwMk/tpLlyAV gp83h56BdZaEVsJt/j5rIywXYjnlzgo= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gg5L8Cr7; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf15.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714697452; a=rsa-sha256; cv=none; b=OR0nsZKFLUvOil7ghx4nsiWvCDCdgOTtpOw2pwX/vaQuiisgPgDcw13KhUbEdmuDj0Kf+R zs34gehDvm9JYsHZSnQEp0adOWBnVHEgp8x45z6LkjCBdNhac2FC42KV9+nwd748kgdGyy 2GKLYLVcN2H/RZBx+wzukmTboEsUDwk= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1e83a2a4f2cso58180775ad.1 for ; Thu, 02 May 2024 17:50:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714697451; x=1715302251; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9ZtaFVcTSDB4ul8nNoaO/PcEtn9Rrjc3ITa59z6anjw=; b=gg5L8Cr7QkN8i8I6HM0yRBuK7/xe2ryly4OyrHUABKZ21kS5Yv4urGz2cvrNtSuN9f b9sHifPb2eYYAIPdGjdDwOsBMaZ96unIE/QlNTiFWwfifqYJxoCvZ4Mj3KieKAYxaesL 8A4izSFGHzD3DY8wVtkv3TlODhHdkayW4mqQFlapdehuDEpH8dI/bgidpYcMA6rhPBsO qcQhRMq20jSm+o6LN7xfDpqe6MHHLRcSeOW4w8ZisybwDs399xBFO6KdWzgcFJhsxCeh FkC/mNDDG1ILpDK/zYRVWeLmWkfHlzzS7ftDLq0vp+2J90HihQ/9bMqa6d2gu+npTxbc adCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714697451; x=1715302251; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9ZtaFVcTSDB4ul8nNoaO/PcEtn9Rrjc3ITa59z6anjw=; b=S19alW3JNSaVH/Hf9SC0XGhOBTyaNkGg4/rEbIM3FDHrT2eff+Qi+PXcA7iu6Tw8te 8Vn484Sl2NqBYtOWi6D9UJ3u7LSw3bfr5P92Z6ot1vwM9s3BcSCqHdimaMV+Kg4dpucJ mnex8lfxhvlwavSWWx0012VVdd4wmyvBi7NksVa9fQYp5zUY5GjZ644d+R3UfoSq3CsF Y9FZ90fgvTT91EQiLZeg75BrQZJ3IqERBVVNJoKKVKJI8ADD3tZVBK36G5etCs+/Qnjq p7/RmAg/1+dppVFFpF6Osry6KqwH0evTQD/LumQ+mCRPdMgqVqFerSsAySCc7NpJaN60 GJ9w== X-Forwarded-Encrypted: i=1; AJvYcCWbyxgZ+2BSMJ4lVD8ZtG9jCtr4TR5FJgyO8zmd9BxptaXytAkEsGgTov+I89ElNns+gWXvqBikrJ0jkeqhZGALuSc= X-Gm-Message-State: AOJu0YxbjIQcjN+xIVvivps/zA8sqfVxIiok0jhvrrSXmuY8WBd9SgrS lqS5O0A9Kmn+f+SIjqNTD0NilE44kPt8i+g/srdqNuv8Qv5og2eF X-Google-Smtp-Source: AGHT+IEefQkadUJjt20GqF9jK1qsxgF1ScWbbqdgQAWnvfCb3u+RiScs5Eeaq3V8oWvBSMJbyG0VfA== X-Received: by 2002:a17:902:bb8b:b0:1e8:b768:2c25 with SMTP id m11-20020a170902bb8b00b001e8b7682c25mr1211101pls.15.1714697450969; Thu, 02 May 2024 17:50:50 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d14-20020a170903230e00b001ec48e41d2esm1969175plh.212.2024.05.02.17.50.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 17:50:50 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, hannes@cmpxchg.org, hughd@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com, yuzhao@google.com, ziy@nvidia.com Subject: [PATCH v3 1/6] mm: swap: introduce swap_free_nr() for batched swap_free() Date: Fri, 3 May 2024 12:50:18 +1200 Message-Id: <20240503005023.174597-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240503005023.174597-1-21cnbao@gmail.com> References: <20240503005023.174597-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1938CA0007 X-Stat-Signature: o4dwqd6dzqwaksrypk46dbasamdj81df X-Rspam-User: X-HE-Tag: 1714697451-316649 X-HE-Meta: U2FsdGVkX1/lGDCIq7L7C8K7fJe5KDkkRk5imtVEd+XAJtOqxDuI2seYbghBcYv6rFv9qd/VfEuA39RDNNFNsoAUMfAn0OoM6+TfAsnw8HKFhiHfkVPreRcyGu9iL+a67Na5eF7JjUTncR7XbcuQB2YbsbIBcSfhcxdAgL+FYmuXO7E4Ar4l82Jmu9g6ofLnjUoEXWFGex9as8uwpTmHD+okPfWXEdcszKZIIVmscYAjVqaKniOSkFxrJTo6gifZfZdNJolbS4G4G6LSSz4wjTc8ZvnE8uDiCk9urgwT8XpaJk5dHO6okIlIrsjmQ7jdUuJPeBIvK41DvFdALjsrmk8UeYMUNBRvJ9HTBDUBDXGN6uqeHzo77EcpSETsYmoEwht8JiFSclRNhDJ1GAr4bzkt5brY0n8kWMe0HyoNse7eCB22+rUYdqtwsFl8GGgedrRyETE4wWTu9/jGguPNwdlUPDl8iEa8jS+xKH1xsSs5bam52Q0qi6E3g3wNbcfH8nwCr7CY3pNxLp+pSkBIYAacMeGQ3MR2bGLJq9UA6DSTj47BT2GxyLXCfT9O897nUUns622cjhltYznqRLYdbfM++RqM4Gin3lcu0BEi/A8TtO57nfo9/rUIAZB+fMJHFCzXNRNWouHaw6vbCBYXBEPbZoHWXZaaCaznP6D+wNObHtMslF06JlkZkofkCODGSTU+6NhTR6/4wNM40/ChePKTd/9/orPJzo3CHTnaBJ/qYluYMJb8/wnxYJt4/Ah5oXBcrm2kSdnYQmj+gv0686EDbeJB7sL1BH6o2QE1FXkKtA/WUKHaEDy4AB3hmHCFHAq9hmZ+oHLKStbZruYkwRicIjxNkZrl5rKvqjUg86MP8zZa1zRXx0ypDNP6FCK2imTlFQEMdI7/sXGAPDox4J5682ANZqGE1IaTcQlNGoIdsYO70D1IuAl97cUOXWwnGXwDNQDIJFmF8fmPQ8s c9wYi1nl OAQqPKnkiA3pNOfclwTXR8ZFzXijeXdA7nWX4huDSPdcB/AUamuHihGMqlfPmF0wxUyPzarHXXFJXKgmZlS6MPG6kuijOllgJUvgsMHGDaD8MHhlJB0Tcc6PAC3rVFQPNXca/YD5YBiqcQhrqhMUsPaHcgku42Hp+n2dJl0cIoP/cH7cY3D90pXCUFN0zLzHr7r5fwhwh1eER8nsDDOjck9IzxLCvvBakTnzFcYrvX4Ms0dRf07E+rnXEbV6wte9S7zky+F/0RoU/KPNms37CF5MuTFCX5aZW2BLHuOm2cEDtXDr7ZjRPtkgTbGtXeINmLiNtoc3ofZnWdZpwvoZ6gO5dHqzfnmwNdB69VTlniYb1zlBnk5rWEcVp/Tm/XG+/n/ILYkiUs/zqet1Mlyj3awy4ntf/N33zoadCo0NEyUFE34OtTg4qkuey8mu2rCBuNop+YuSEhMmpujeapvDTuiR0PuSjc1KGmVnsGm4vhQj8yopmEWt3Y8ebZy9x663vvXaw1uJ9xxu39KSch9JIfdqYtLXrTmPEhKWf8DfdU+yh3ipvnqBEnuD2nvZH2IM8LXCc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Chuanhua Han While swapping in a large folio, we need to free swaps related to the whole folio. To avoid frequently acquiring and releasing swap locks, it is better to introduce an API for batched free. Furthermore, this new function, swap_free_nr(), is designed to efficiently handle various scenarios for releasing a specified number, nr, of swap entries. Signed-off-by: Chuanhua Han Co-developed-by: Barry Song Signed-off-by: Barry Song Reviewed-by: Ryan Roberts Acked-by: Chris Li Reviewed-by: "Huang, Ying" --- include/linux/swap.h | 5 +++++ mm/swapfile.c | 47 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/include/linux/swap.h b/include/linux/swap.h index 11c53692f65f..d1d35e92d7e9 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -483,6 +483,7 @@ extern void swap_shmem_alloc(swp_entry_t); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t); extern void swap_free(swp_entry_t); +extern void swap_free_nr(swp_entry_t entry, int nr_pages); extern void swapcache_free_entries(swp_entry_t *entries, int n); extern void free_swap_and_cache_nr(swp_entry_t entry, int nr); int swap_type_of(dev_t device, sector_t offset); @@ -564,6 +565,10 @@ static inline void swap_free(swp_entry_t swp) { } +static inline void swap_free_nr(swp_entry_t entry, int nr_pages) +{ +} + static inline void put_swap_folio(struct folio *folio, swp_entry_t swp) { } diff --git a/mm/swapfile.c b/mm/swapfile.c index f6ca215fb92f..ec12f2b9d229 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1356,6 +1356,53 @@ void swap_free(swp_entry_t entry) __swap_entry_free(p, entry); } +static void cluster_swap_free_nr(struct swap_info_struct *sis, + unsigned long offset, int nr_pages) +{ + struct swap_cluster_info *ci; + DECLARE_BITMAP(to_free, BITS_PER_LONG) = { 0 }; + int i, nr; + + ci = lock_cluster_or_swap_info(sis, offset); + while (nr_pages) { + nr = min(BITS_PER_LONG, nr_pages); + for (i = 0; i < nr; i++) { + if (!__swap_entry_free_locked(sis, offset + i, 1)) + bitmap_set(to_free, i, 1); + } + if (!bitmap_empty(to_free, BITS_PER_LONG)) { + unlock_cluster_or_swap_info(sis, ci); + for_each_set_bit(i, to_free, BITS_PER_LONG) + free_swap_slot(swp_entry(sis->type, offset + i)); + if (nr == nr_pages) + return; + bitmap_clear(to_free, 0, BITS_PER_LONG); + ci = lock_cluster_or_swap_info(sis, offset); + } + offset += nr; + nr_pages -= nr; + } + unlock_cluster_or_swap_info(sis, ci); +} + +void swap_free_nr(swp_entry_t entry, int nr_pages) +{ + int nr; + struct swap_info_struct *sis; + unsigned long offset = swp_offset(entry); + + sis = _swap_info_get(entry); + if (!sis) + return; + + while (nr_pages) { + nr = min_t(int, nr_pages, SWAPFILE_CLUSTER - offset % SWAPFILE_CLUSTER); + cluster_swap_free_nr(sis, offset, nr); + offset += nr; + nr_pages -= nr; + } +} + /* * Called after dropping swapcache to decrease refcnt to swap entries. */