From patchwork Thu Sep 26 21:19:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13813673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9BD8CDE03A for ; Thu, 26 Sep 2024 21:20:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28A7F6B0093; Thu, 26 Sep 2024 17:20:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23AC76B0095; Thu, 26 Sep 2024 17:20:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 102E46B0096; Thu, 26 Sep 2024 17:20:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E77EF6B0093 for ; Thu, 26 Sep 2024 17:20:17 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 94CC612052F for ; Thu, 26 Sep 2024 21:20:17 +0000 (UTC) X-FDA: 82608157674.21.A489C8B Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf16.hostedemail.com (Postfix) with ESMTP id CAED4180010 for ; Thu, 26 Sep 2024 21:20:14 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gh1iNGvZ; spf=pass (imf16.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727385492; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=QZ91nBWFZA/cgktqOlSvWqcjoum6mw/iuuqt/q9fLzg=; b=7fwQ7jpkq/TQjuTVcbKvmt+jQNlkYJvZ2QzovwsAZxFdFYsdXSeuDQPdO9ydTW3JOkBp7U 30iKxhv1eMS/m3TYR6wwmxApBhLfEnKgPJsuOfjYl80pRtl/3u2j6aGrWzRqfi6esqrJEv LxFQTCtWHW80TBVehXw5kFRpGqCqLKs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727385492; a=rsa-sha256; cv=none; b=0K55eMx+JdeDFSTbfK18JRb4huCgMglKlLW8TL9WmOUIM00hR8CFEO6KLAhK2y7xM7vAYf kcfOhnVtEDN/P8W8DqV0518Yu+V/qkxXt6S3o12fc9DakTwKzBVFTApsvLxOqh7yGADiLi dgPcOGgJ87J5CplxkdAIkiq/UjnnFok= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gh1iNGvZ; spf=pass (imf16.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-7198de684a7so1097313b3a.2 for ; Thu, 26 Sep 2024 14:20:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727385613; x=1727990413; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=QZ91nBWFZA/cgktqOlSvWqcjoum6mw/iuuqt/q9fLzg=; b=gh1iNGvZUMthbSath5GqtaezogjZEtfke+pgweO09T7xvKw94IStwTPs1Vnj4McWud iWWxwCF7sLmlFp4FfKD0GnyCzBpqAX53z4HG+hWfrEfp8h5zTycxSsQhPgfFrvvxw4Ag iIhsmZZuipZLG1PCC07oC2supwX5Nyi91yidDqz08Wb2ICUXgymhS9B/r5xHOEheY+f9 Rpqs2nMhBfnODdVE0BvEp+pMku3/nekmuBlkQkH1ZrkBWFSqLfnDWJ73qsqS/ZQSfDMy wrdj4Iben9hpgz7swydJyuF8v2ONydHTsizabcg7vN3cq/wjnrFEfAAFwC+g9IjRpvNB d2lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727385613; x=1727990413; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QZ91nBWFZA/cgktqOlSvWqcjoum6mw/iuuqt/q9fLzg=; b=DWKmnREItM9ODRJ8/MpvuGaFhgswG9wtt5Gem5HuwqTimoa1dOu5stKFqFy3wN84// 6uacyfXkGhSIkECA2oHMyhkcyXj7npKhOOKlP8NPVE/u6nUUzl7O5npV4Kj1G3edp6Gf Vd405jNeikNZdi0ZEUdR8afcT5KO8lTNi42qsB/3muoeQ8+RmfdSjabKp6mK6UttmXmd NkSEuXVOr3PI3Zsw+Il03A1XWyEqai46lTOwA1zZp0mXBsd9971dz59a7r9H42uDg1N6 7s4fQrKH7mvrsUozO+8JxrfO3u9x2ufaepMOWzmT0UkN78I5gS6tCKUOSbT/ZUqUlpy6 G/gg== X-Forwarded-Encrypted: i=1; AJvYcCWI89KKrD6AheW+incw8GeuI1D+Y8W0SOqI6hFx7Icf3l7RSh4KKfgUiynJ+NCZOknOlNRYHiF55A==@kvack.org X-Gm-Message-State: AOJu0YwAjeWDPfqsDsOiFUbn8DKQABzlu36exRxXQ8Q5wfy7T5MFtgx+ m18rKuCKn4UweP4DmzyG0Efz/VnhxbnhvRSBolKfuAQvz30wp8Jp X-Google-Smtp-Source: AGHT+IG9ekqH94a7zizhC0UXzN1YBgcZ9BF/8ctXYzeyJmPq6g4DU7IdJeBD+mf3dajOxhMZuIT5bA== X-Received: by 2002:a05:6a21:a247:b0:1cf:4dc3:a89e with SMTP id adf61e73a8af0-1d4fa64d9d3mr1069643637.9.1727385613293; Thu, 26 Sep 2024 14:20:13 -0700 (PDT) Received: from barry-desktop.hub ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71b26516245sm338881b3a.127.2024.09.26.14.20.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2024 14:20:12 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Barry Song , Kairui Song , "Huang, Ying" , Yu Zhao , David Hildenbrand , Chris Li , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , Minchan Kim , Yosry Ahmed , SeongJae Park , Kalesh Singh , Suren Baghdasaryan , stable@vger.kernel.org, Oven Liyang Subject: [PATCH] mm: avoid unconditional one-tick sleep when swapcache_prepare fails Date: Fri, 27 Sep 2024 09:19:36 +1200 Message-Id: <20240926211936.75373-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Stat-Signature: knr79b45pcomxcaeaj8zbfqq5641db8h X-Rspamd-Queue-Id: CAED4180010 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1727385614-73305 X-HE-Meta: U2FsdGVkX18ieWwrMwuNhBZhemHDN5QhQGKgwVMSx0s8tcReGQIb21iUsFonHGITD0A8vF+ZxfkT8nQm/ueLMmX2WwDlOtHDvtSxbU/HaTtOQfUST8NH3IRpjjaerUCqK3e2WLOWX8YsQDDnyrmU6fKAlDjsULoWkWGEujxv8LTg2bOnYvkb44NSjLlEFupoWIbR+71x0oKoIQJTMthAtwWW+GGVzt72bGFMqsXcNjnN54ehu79t0AvOLqMBxNi/1iSv9SwGaAhBkikcUoVFA7rIlwsarDx7FoyhfYAsWani57bwxiiZqH3AZAAmXl7s/uqkOJfKu2y8GdabBi3tF/ZT5dLiMRaoVfhYvlh+G1k1LBsIub2IHB6Q6whnLV4tZPlQ63U3jkTqdkwSXdTHes9bsZWni29G/mIFluBkgQJszvvcS8Ks+uRPsxScAVx1H+YJtzwEZcV4gEkPNCS+A3K0B7XNa3likPASuhhQYXrs5A8gmeOzbkcSgnaTTJivsM8TLff5yBqF23t09f1NLztx9Db3HiU69IG4qac4hkFaUrQOIp1xyeFHRIKUYZzm3GO1C5uMYw3w8dnCdg4TrjudFXoSjWkeaCjYqYjdmeK+K3q+wo6eyWEjrDILjEUpkBl9BE2bADERtlPzEqCGjWeQCtQp956MyRK9th9QqKiDf5f4CBii09APCOHKpTxq2nOMUam6SFpnlJIvCMEqWX0DN0Zdbe8yWzX5+fuhMEP1bhZ/CBr4HT4/mL7QzdZ7qtpNq0G0Yy+4QsSueVT2Dj7sA2pxRL2wiGzF6PaJl63Im9nPNMwReq+xDpltUsGwukbgxIE1D4CHSPqBUpG9SmoOt7iNZhpcf4RUbfM8IBM8Gya2plLkBVS386Q3lIqqNZbGz3cFjh9zj7aHrbYv2fsW8EvPxSLkoY1BDBsfbYutugcpQkzyCAa61pIh0rLTgAlZYnrCTovN9v4Shjt GAyytM00 4nv5HQm2qn5LxD9wjC0bxcibZeUM3CD0zZRlSuurzmsJjWTwsc31UzVMRqv0ekjY+qIqNswOFW7c7vMDnIkNAshUrN/PZrM5UaDB7cWfBHIfq2Dw0RTEpQ76WuTgt9Ytb5wGv473SFN6X1WoseANHkDgyZbh1G9JFoIuTmg9P/HMhY44Q5grIn6jKnzzZjyIS/zeJYK9ijLqpkzs7ayRgLA30nLt1m1lcYlQFpBdMEgFb7kWkhnM7gV2Uheh34kRTvug6te8fYvb+7paZho9XWzWeqrVzIZBaAA63GW2gXJkdhUde5AHoVkoUbpXR1jmyMIJb11j6a/Ad3vjuuR/rKnk3wbkCC47SDapVfwhzq1hGCMlZjyv8Vo2tsDhNg1rpo4wUNJbdOmqAZuJl0aChOWtGDKt8nQyo9YBQNYkn/jLlS4XyGLPjEmHXpWsMJppvxYyefNm8NqL57vFYDMJdQoeo28Fj6DUuihg7ZkNQgCWjWAuHHnizLcrakyR84jEJtk/Vx5WEKt8K/cLvvMnVCDveAKWbdM9n1XuV65ciiHBs03Y/ewbAiU8GPzM5G6RuwQSFlVLXfSQu/mjV9yb5ZYncs4J+S7gGvTIGs73Oq7BoozIkLEztYDrh2tUcyiddn60cdB3A47587YVJjeyO0N+HloIMWZgBsy3k1yGHYHqK/8N7vyYhi8IlI1aFQEsX7BmeKV44td/Iu03HPuifBEYGQ6Fd6S3qvZIMcyTaLh7tttoc7KI33cboAvpiYxL+ukKP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Commit 13ddaf26be32 ("mm/swap: fix race when skipping swapcache") introduced an unconditional one-tick sleep when `swapcache_prepare()` fails, which has led to reports of UI stuttering on latency-sensitive Android devices. To address this, we can use a waitqueue to wake up tasks that fail `swapcache_prepare()` sooner, instead of always sleeping for a full tick. While tasks may occasionally be woken by an unrelated `do_swap_page()`, this method is preferable to two scenarios: rapid re-entry into page faults, which can cause livelocks, and multiple millisecond sleeps, which visibly degrade user experience. Oven's testing shows that a single waitqueue resolves the UI stuttering issue. If a 'thundering herd' problem becomes apparent later, a waitqueue hash similar to `folio_wait_table[PAGE_WAIT_TABLE_SIZE]` for page bit locks can be introduced. Fixes: 13ddaf26be32 ("mm/swap: fix race when skipping swapcache") Cc: Kairui Song Cc: "Huang, Ying" Cc: Yu Zhao Cc: David Hildenbrand Cc: Chris Li Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Minchan Kim Cc: Yosry Ahmed Cc: SeongJae Park Cc: Kalesh Singh Cc: Suren Baghdasaryan Cc: Reported-by: Oven Liyang Tested-by: Oven Liyang Signed-off-by: Barry Song Signed-off-by: Barry Song --- mm/memory.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 2366578015ad..6913174f7f41 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4192,6 +4192,8 @@ static struct folio *alloc_swap_folio(struct vm_fault *vmf) } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +static DECLARE_WAIT_QUEUE_HEAD(swapcache_wq); + /* * We enter with non-exclusive mmap_lock (to exclude vma changes, * but allow concurrent faults), and pte mapped but not yet locked. @@ -4204,6 +4206,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; struct folio *swapcache, *folio = NULL; + DECLARE_WAITQUEUE(wait, current); struct page *page; struct swap_info_struct *si = NULL; rmap_t rmap_flags = RMAP_NONE; @@ -4302,7 +4305,9 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) * Relax a bit to prevent rapid * repeated page faults. */ + add_wait_queue(&swapcache_wq, &wait); schedule_timeout_uninterruptible(1); + remove_wait_queue(&swapcache_wq, &wait); goto out_page; } need_clear_cache = true; @@ -4609,8 +4614,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) pte_unmap_unlock(vmf->pte, vmf->ptl); out: /* Clear the swap cache pin for direct swapin after PTL unlock */ - if (need_clear_cache) + if (need_clear_cache) { swapcache_clear(si, entry, nr_pages); + wake_up(&swapcache_wq); + } if (si) put_swap_device(si); return ret; @@ -4625,8 +4632,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) folio_unlock(swapcache); folio_put(swapcache); } - if (need_clear_cache) + if (need_clear_cache) { swapcache_clear(si, entry, nr_pages); + wake_up(&swapcache_wq); + } if (si) put_swap_device(si); return ret;