From patchwork Thu Feb 29 00:37:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13576146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 883C0C5478C for ; Thu, 29 Feb 2024 00:38:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 199E36B00A2; Wed, 28 Feb 2024 19:38:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 121B46B00A1; Wed, 28 Feb 2024 19:38:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F04FE6B00A2; Wed, 28 Feb 2024 19:38:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DDF446B007B for ; Wed, 28 Feb 2024 19:38:43 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A4789161202 for ; Thu, 29 Feb 2024 00:38:43 +0000 (UTC) X-FDA: 81842980926.12.0EB1787 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf02.hostedemail.com (Postfix) with ESMTP id 027148000D for ; Thu, 29 Feb 2024 00:38:41 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EbUZVxYX; spf=pass (imf02.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709167122; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yMJ7yXDIGraWBc5cHOK7WDJvzj0gdP9hzucxgbPAd4E=; b=ruyLreJlXAQjyBKf7oJhbl/2sba8j2tpEKyJAnXcRzYPzbVuqhFLgBJsZndM90Xx1HYjyL OahwwXfWBZt9U2zU+RfcPUDAnYmKvQ3dbxiAgPZodOxKtmGC0DrmYL0R7pxuUK0PuCZ8EK H0tjzWHuHypw/ND5H1WPVTSOxV5I3GY= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EbUZVxYX; spf=pass (imf02.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709167122; a=rsa-sha256; cv=none; b=lF+uRNAVBOuxjVrS8zbSxJy7uAo+1JSHq2I+zR/dR3aCqhUWQNLVcFi1GxDjnB2Zm22jt1 LuZobW+HPReqze+gbencz+EBD+UiLqNjHaT0u9A0DOmiJymCPcoyJhPotUxlvjnBCIGiLW LjBmueOiUcTwE9UXc2RqCWV+LC+9Ljk= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1dc3b4b9b62so3111245ad.1 for ; Wed, 28 Feb 2024 16:38:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709167121; x=1709771921; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yMJ7yXDIGraWBc5cHOK7WDJvzj0gdP9hzucxgbPAd4E=; b=EbUZVxYXLULXfmmEAbQsILRvcrOo4u0gJ+WFOGd0SoEf7b6LcOCS4BPKj2aaaxcXAf Q7hoLYGejVGgKTnNpicTHhX0pyuti/ueVH7GxcLlXXKzRfRzhsbAkMbHW+v1jWh3fR7s WEQ3G1XS02kqw8EPvvifl6WzQNZjaSeFXvsUtU2SS10Osq0SkaIIgIKk+knAdKTiw9fC gMtM8E81498Io1qRthVVL5OsmPkuGIiPbUdklCK8z09B5F2YqJJA/pGDJNeB1rDfhU/g 9dow+xnQPj6FYfWQD8WAtqGmgZjypBUX3ZGsr3b30nLDpPrSvdoO3D3VZFTRtAny3B3a 6dgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709167121; x=1709771921; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yMJ7yXDIGraWBc5cHOK7WDJvzj0gdP9hzucxgbPAd4E=; b=n9hjGpiNL2rzgVBmxZCxTjBO37BjUE08var9NLzuUskmiVC50L4+7hAEJdoyUuX56V 3Jc7TD+i4yFzlyYoAT80PY/1gDUrplHqcb6DI+bV96iCaMlY5l7nwmcK9lImXWEJytc5 5OGb25MyqWx4CTXAKbhR696bgZDeC+yT8/wwPtBsaa7H8jVeI3+tPSh8rARhh1NKa6xC sYvzkJo0inHHKVIv6ELAUYMVqK+GRHxomKy+yhkJhFFNtB0Xu4wMXdkn5Jlx6eN23Rul /iSp6PwlO7HGXZZ8fhzR6vBbe4uwYe96BLOICDpXaWvNClORB7cNIHqpF9s4iJFxfu7O cJbg== X-Forwarded-Encrypted: i=1; AJvYcCU0fm6N39u2OwCvkD4mbleYeKCGiLb2pvJtgQixUjU68398XOkNIDuzlHvmqFsH5eugAwYd/Ma8pUZy4JcMo7keo9c= X-Gm-Message-State: AOJu0YxBCi0nslMBDWlGZPZPEYIV1wP8VRa+RGi23pF6WiZpJFuWYPfU 0wKY3bzQVh0mJooxnZt9i9j3ohtvHHvRYtz7m3gkBlrlswFewNCA X-Google-Smtp-Source: AGHT+IHcSLxazfOE73baRJGF/GDzC2Ut3tKmCraQkgILYtM8ozQfrAGo7OOUSaQPjCdA4QL0Tm1Qmw== X-Received: by 2002:a17:902:e5c2:b0:1d9:adc9:2962 with SMTP id u2-20020a170902e5c200b001d9adc92962mr530369plf.20.1709167120889; Wed, 28 Feb 2024 16:38:40 -0800 (PST) Received: from localhost.localdomain ([2407:7000:8942:5500:5158:ed66:78b3:7fda]) by smtp.gmail.com with ESMTPSA id p3-20020a170902780300b001d9641003cfsm62647pll.142.2024.02.28.16.38.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Feb 2024 16:38:40 -0800 (PST) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, ryan.roberts@arm.com, chrisl@kernel.org Cc: 21cnbao@gmail.com, linux-kernel@vger.kernel.org, mhocko@suse.com, shy828301@gmail.com, steven.price@arm.com, surenb@google.com, wangkefeng.wang@huawei.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yuzhao@google.com, kasong@tencent.com, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, hannes@cmpxchg.org, linux-arm-kernel@lists.infradead.org, Chuanhua Han , Barry Song Subject: [PATCH RFC v2 2/5] mm: swap: introduce swap_nr_free() for batched swap_free() Date: Thu, 29 Feb 2024 13:37:50 +1300 Message-Id: <20240229003753.134193-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240229003753.134193-1-21cnbao@gmail.com> References: <20240229003753.134193-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 027148000D X-Rspam-User: X-Stat-Signature: zhkjcaq9biwr3a5zpkqkjdii6q96znpb X-Rspamd-Server: rspam01 X-HE-Tag: 1709167121-947286 X-HE-Meta: U2FsdGVkX1+BUgD99bF+9PbysXs4KqKQ558KPLj77AAVRHH0jSQ2MZo5GuPYqjWdp9fgO9M9V71MpTJkU4TAHYKbnXXCdA5oAZ8fdbEZWPRSpOn7y9pQ5/+EiCo+PKU5tjPeo/bXxO1Y7TYwxrfmAwTAQ5ZL7+kNb/0wr/5e1I85N1ufmXSIxImIlBTkdxfhg+oQWeK/lugBrw10hn33BGC8QZokTJxO776DHtI5lJbZoaQK4xR1JYiE2X8S8F2FMsRoKmr/8dXoyJmZUm12Ha86pIz0HRrDZeMXf0/hoaDCvoilIyW+vBxNffsouhDEV1Jv9t9pWrRIF6Lh2kovTmn4XsD9HU0BsQoLlrahMmiNINRTJE5vePXPcFPeQjCFOOq02yHz+w8JzLzrZ2Qgk5aEuCPI8IjyGnwisV6JJ7JXGbVglNndTwIkHbVjhbFVBtmLf4akTLW/1gYoHSBN+b9oR8W7pCZNsaSQWQmvtTjcO1Ue6v684NQqjrCmLXH+5bCIa4IxOP+CdVe4OCRMpMhlMkRfgv0fVF5DLmH3U1G3S+hQYi27mFidyU4YNGKjHXrZ+zz+O7BcsRGz59YWj5SppCpcMy5C9P9C2ZZViWrivmqDGSMDcmej7P3SNOU/SVKAM7TE0EcOIR4+eMUAFcxeyQ7I0N7U7pI16f+2LXGh5g6aXkFRgMCBic84cK2izXDRU+h9N3xiHHCAZ0XVsPiGhZQ9G3T+NPKR7WzeTFg0zN5ePpMJo/S+71tk9FagGKVIt9peTvYiepzCKWkVzZaxSvBRSxGxVOiu3UEBGBxgaOITcPzgf7ATQRBS9B3zVu2moFJnXqBdlAvF2RHpbj03WDsEfDdjM0X2+WCCJfjDlHJkJ+MPI0XsMTCRkcxF7daqUiwsnxi3qIh4S0hby9F/pbu7/kAoHd3r8Dh9bty6OlmzlRLYDXWcWO4bfy4iFB1D3pMj0XVwwFtcdX5 EE64UqyA wvSzbYlq0HKej7zqF5JfFqGhh6113ybf18EI//bUkDghP/Rpg/wgPvvPP+slIrvpUZU5YlTR4qSFMJpCjr/+jLXWiiH41KtRrHIEmlGqR880OwjeTvg/zTpEoZFKAsjBPOsBiGM5H3yLolTQJkkkSfWArZBdY1teFXCIX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Chuanhua Han While swapping in a large folio, we need to free swaps related to the whole folio. To avoid frequently acquiring and releasing swap locks, it is better to introduce an API for batched free. Signed-off-by: Chuanhua Han Co-developed-by: Barry Song Signed-off-by: Barry Song --- include/linux/swap.h | 6 ++++++ mm/swapfile.c | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/include/linux/swap.h b/include/linux/swap.h index 25f6368be078..b3581c976e5f 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -481,6 +481,7 @@ extern void swap_shmem_alloc(swp_entry_t); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t); extern void swap_free(swp_entry_t); +extern void swap_nr_free(swp_entry_t entry, int nr_pages); extern void swapcache_free_entries(swp_entry_t *entries, int n); extern int free_swap_and_cache(swp_entry_t); int swap_type_of(dev_t device, sector_t offset); @@ -561,6 +562,11 @@ static inline void swap_free(swp_entry_t swp) { } +void swap_nr_free(swp_entry_t entry, int nr_pages) +{ + +} + static inline void put_swap_folio(struct folio *folio, swp_entry_t swp) { } diff --git a/mm/swapfile.c b/mm/swapfile.c index 2b3a2d85e350..c0c058ee7b69 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1340,6 +1340,41 @@ void swap_free(swp_entry_t entry) __swap_entry_free(p, entry); } +/* + * Called after swapping in a large folio, batched free swap entries + * for this large folio, entry should be for the first subpage and + * its offset is aligned with nr_pages + */ +void swap_nr_free(swp_entry_t entry, int nr_pages) +{ + int i; + struct swap_cluster_info *ci; + struct swap_info_struct *p; + unsigned type = swp_type(entry); + unsigned long offset = swp_offset(entry); + DECLARE_BITMAP(usage, SWAPFILE_CLUSTER) = { 0 }; + + /* all swap entries are within a cluster for mTHP */ + VM_BUG_ON(offset % SWAPFILE_CLUSTER + nr_pages > SWAPFILE_CLUSTER); + + if (nr_pages == 1) { + swap_free(entry); + return; + } + + p = _swap_info_get(entry); + + ci = lock_cluster(p, offset); + for (i = 0; i < nr_pages; i++) { + if (__swap_entry_free_locked(p, offset + i, 1)) + __bitmap_set(usage, i, 1); + } + unlock_cluster(ci); + + for_each_clear_bit(i, usage, nr_pages) + free_swap_slot(swp_entry(type, offset + i)); +} + /* * Called after dropping swapcache to decrease refcnt to swap entries. */