From patchwork Tue Oct 10 03:58:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13414799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12D33CD68E4 for ; Tue, 10 Oct 2023 03:58:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F2DB980044; Mon, 9 Oct 2023 23:58:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E908280027; Mon, 9 Oct 2023 23:58:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D084780044; Mon, 9 Oct 2023 23:58:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C0D7E80027 for ; Mon, 9 Oct 2023 23:58:47 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 955761A04F3 for ; Tue, 10 Oct 2023 03:58:47 +0000 (UTC) X-FDA: 81328195494.12.9C43672 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf02.hostedemail.com (Postfix) with ESMTP id 0172D80004 for ; Tue, 10 Oct 2023 03:58:45 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=uK7CoSts; dmarc=none; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696910326; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9931nhg2FnBqt/YdPpANRM0FFFt7P4pS9twc7+d2cis=; b=rLtUca4LRrTlWLmyBoOFyxTAml6vU+Zx0vWP4yspmCnXTP/HN1zAw47kR5iGZ1MKKulb6T k72nu87MOnZYH0xr82cyj1OXie4QExIif2vl2GuiD4fkx8AY3bmz22mePoybnY/77+JUcN baaIfZjpwXCkjeoGAKOrKe/aPrTr/OU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=uK7CoSts; dmarc=none; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696910326; a=rsa-sha256; cv=none; b=lBflNhN2cKUJUnoUPQ5m+RHfnogbJgYdQtmAw+s09mZAf+3ZO3s3wBTjmJ6BCJSZBMy3a9 14AJQUk7yXPGZDRSwzYRWzINn/AdkIRCZe1rfx1GYAEtK4S1BQReRD9BlZwa7QRmuMnziW EgbNsX6OudITG7lNSHLb5fggn8QOg1U= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=9931nhg2FnBqt/YdPpANRM0FFFt7P4pS9twc7+d2cis=; b=uK7CoStskG4DABGGOCMq6fPVs0 V/inaVjtOHuxRfaPTPlQpGI7rMsgnr73KxSsKSiyzTGNH/qkQiWJh6VhXpg5MhSMAwxsJcfAST87r 2N0AMlaE6vVCEB9hObYQYBW/kSz6W+FjI2Rn0tE7GN/GHFZWiKtIDnhAx1y4+9zTGcHLQqY4imPV+ V8u+VpBjKRXqjnUUL9eWvx07zjy66ST4ynzoPXGVjUh6WqFmUzB7iyxeLKesrvc03FMgATpEKoObF dRsBqT9HFZB1izS56K+Z+mldKgEgPvGiAtlVayyYbpPrweMcVxzBEZhQ8s5oWKascFvgotZdufa1a k6GyaV2Q==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qq3tF-002Hak-RS; Tue, 10 Oct 2023 03:58:37 +0000 From: "Matthew Wilcox (Oracle)" To: Bin Lai Cc: "Matthew Wilcox (Oracle)" , mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, akpm@linux-foundation.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 2/2] sched: Remove wait bookmarks Date: Tue, 10 Oct 2023 04:58:29 +0100 Message-Id: <20231010035829.544242-2-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231010035829.544242-1-willy@infradead.org> References: <20231010032833.398033-1-robinlai@tencent.com> <20231010035829.544242-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0172D80004 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: k96fnebesicectz4i6oqgmdue7gf4wmy X-HE-Tag: 1696910325-320268 X-HE-Meta: U2FsdGVkX19GneLWbaiPRVSiuVwShPXvHqj5jqsUzRPIZfriHL0VfdbS7Sa/JzOa0X0GTHWR6mUP7pKGpCQpYGdtXLkzud7gO5gqk78wTAxWj3etOcylB4B1UAd4gXD8EdZZOFmmEnCIHtjyyBe5y/lCQBgjDQethoeZx0UkLipMMe8XAVXfdtVksPxFrQX9yW27Zu6OdQ/kKEgHZofgeJ8p0ljm14hsDqxGxNGlGae3ar90Depybv479RM0Hm4BPtw9ZpYrR0nPAei3+kwNFXkruzzV+V2nE4G3Z4Vp+ZnvJlkxJaCeVcx5HliXGjpDhvUoL3BMwR28munOVpUhgXsO+nz0/s0dE+B2jqtOYmmDf/Z4hzH/vHsPE7BEI7IhoYogucESixfx/EMTGdeUNnlkaTanlWJgfV8W62vFhf4JyfwvqUc+c+b//7Xgp9mWNGKZ63jKhD93bPNGlHDS0+a0U0gLYrDaJ2BlllKBX1tdVVDYCTa3+pBVMkQnZPtmiLn8ZOaHSSnhrgjKPPD/M3ABQTbFLfpTMH9wp0wFbffdTEtw29lHn60Vnh2oK3Im7G6/0DEd2RMQ4FhIhbMWpjORGCHDjGaLjITiapqqMXgS2qo40wIKqXQU8BYI0NfaPZ5lOyxeTQNG32KHHwwtJvibAUF2zi9ldbybATfWhY+5TOWiOjr7HpHG+J3xZk3ij5P18HewKN4uMDmJeOae5ZrLlPaGFefQllvILLbR+gtYYD7tUsYRPqJhyQ/AXVSmAyGOuEdbBb6qmXlbKY6tmK2tauwj0bRe9OowO1T9DRCXgwbUYE+XxQfIrF2hP+H5rw7bTY5ok7+01p1ucc9UNJDXl9jh7yURtiYg/IoaPLdGe7nFZkY4cG/cIGS7DSy46QbYRGqCKY9ejBbXZVKRv/fz14yTD+aiYZvBpnl6H+73JyMYmgYJi10XLA+uDhC4Kc0RuSsJ32rsWzGplQu wOoc4YtT D5REraRUP7GgQTJw9EYVRMVJJUesKuln2ds2iP4k1LCIsXawmAUpEQkoB4ZtzATlu8gC3wVdMm+hNJYD6bZbjmefUPu6sSk8hu6b4O6WmATpXZfgNm6VGvHwCPm2RS0bkbRs4HifNDONgx/BNRrAoZe/zcXwmVY6cliIQ6JmjTpcO+QmvPfBWNFi3bQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are no users of wait bookmarks left, so simplify the wait code by removing them. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Ingo Molnar --- include/linux/wait.h | 9 +++---- kernel/sched/wait.c | 60 ++++++++------------------------------------ 2 files changed, 13 insertions(+), 56 deletions(-) diff --git a/include/linux/wait.h b/include/linux/wait.h index 5ec7739400f4..3473b663176f 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -19,10 +19,9 @@ int default_wake_function(struct wait_queue_entry *wq_entry, unsigned mode, int /* wait_queue_entry::flags */ #define WQ_FLAG_EXCLUSIVE 0x01 #define WQ_FLAG_WOKEN 0x02 -#define WQ_FLAG_BOOKMARK 0x04 -#define WQ_FLAG_CUSTOM 0x08 -#define WQ_FLAG_DONE 0x10 -#define WQ_FLAG_PRIORITY 0x20 +#define WQ_FLAG_CUSTOM 0x04 +#define WQ_FLAG_DONE 0x08 +#define WQ_FLAG_PRIORITY 0x10 /* * A single wait-queue entry structure: @@ -212,8 +211,6 @@ __remove_wait_queue(struct wait_queue_head *wq_head, struct wait_queue_entry *wq int __wake_up(struct wait_queue_head *wq_head, unsigned int mode, int nr, void *key); void __wake_up_on_current_cpu(struct wait_queue_head *wq_head, unsigned int mode, void *key); void __wake_up_locked_key(struct wait_queue_head *wq_head, unsigned int mode, void *key); -void __wake_up_locked_key_bookmark(struct wait_queue_head *wq_head, - unsigned int mode, void *key, wait_queue_entry_t *bookmark); void __wake_up_sync_key(struct wait_queue_head *wq_head, unsigned int mode, void *key); void __wake_up_locked_sync_key(struct wait_queue_head *wq_head, unsigned int mode, void *key); void __wake_up_locked(struct wait_queue_head *wq_head, unsigned int mode, int nr); diff --git a/kernel/sched/wait.c b/kernel/sched/wait.c index 802d98cf2de3..51e38f5f4701 100644 --- a/kernel/sched/wait.c +++ b/kernel/sched/wait.c @@ -57,13 +57,6 @@ void remove_wait_queue(struct wait_queue_head *wq_head, struct wait_queue_entry } EXPORT_SYMBOL(remove_wait_queue); -/* - * Scan threshold to break wait queue walk. - * This allows a waker to take a break from holding the - * wait queue lock during the wait queue walk. - */ -#define WAITQUEUE_WALK_BREAK_CNT 64 - /* * The core wakeup function. Non-exclusive wakeups (nr_exclusive == 0) just * wake everything up. If it's an exclusive wakeup (nr_exclusive == small +ve @@ -78,21 +71,13 @@ EXPORT_SYMBOL(remove_wait_queue); * zero in this (rare) case, and we handle it by continuing to scan the queue. */ static int __wake_up_common(struct wait_queue_head *wq_head, unsigned int mode, - int nr_exclusive, int wake_flags, void *key, - wait_queue_entry_t *bookmark) + int nr_exclusive, int wake_flags, void *key) { wait_queue_entry_t *curr, *next; - int cnt = 0; lockdep_assert_held(&wq_head->lock); - if (bookmark && (bookmark->flags & WQ_FLAG_BOOKMARK)) { - curr = list_next_entry(bookmark, entry); - - list_del(&bookmark->entry); - bookmark->flags = 0; - } else - curr = list_first_entry(&wq_head->head, wait_queue_entry_t, entry); + curr = list_first_entry(&wq_head->head, wait_queue_entry_t, entry); if (&curr->entry == &wq_head->head) return nr_exclusive; @@ -101,21 +86,11 @@ static int __wake_up_common(struct wait_queue_head *wq_head, unsigned int mode, unsigned flags = curr->flags; int ret; - if (flags & WQ_FLAG_BOOKMARK) - continue; - ret = curr->func(curr, mode, wake_flags, key); if (ret < 0) break; if (ret && (flags & WQ_FLAG_EXCLUSIVE) && !--nr_exclusive) break; - - if (bookmark && (++cnt > WAITQUEUE_WALK_BREAK_CNT) && - (&next->entry != &wq_head->head)) { - bookmark->flags = WQ_FLAG_BOOKMARK; - list_add_tail(&bookmark->entry, &next->entry); - break; - } } return nr_exclusive; @@ -125,20 +100,12 @@ static int __wake_up_common_lock(struct wait_queue_head *wq_head, unsigned int m int nr_exclusive, int wake_flags, void *key) { unsigned long flags; - wait_queue_entry_t bookmark; - int remaining = nr_exclusive; + int remaining; - bookmark.flags = 0; - bookmark.private = NULL; - bookmark.func = NULL; - INIT_LIST_HEAD(&bookmark.entry); - - do { - spin_lock_irqsave(&wq_head->lock, flags); - remaining = __wake_up_common(wq_head, mode, remaining, - wake_flags, key, &bookmark); - spin_unlock_irqrestore(&wq_head->lock, flags); - } while (bookmark.flags & WQ_FLAG_BOOKMARK); + spin_lock_irqsave(&wq_head->lock, flags); + remaining = __wake_up_common(wq_head, mode, nr_exclusive, wake_flags, + key); + spin_unlock_irqrestore(&wq_head->lock, flags); return nr_exclusive - remaining; } @@ -171,23 +138,16 @@ void __wake_up_on_current_cpu(struct wait_queue_head *wq_head, unsigned int mode */ void __wake_up_locked(struct wait_queue_head *wq_head, unsigned int mode, int nr) { - __wake_up_common(wq_head, mode, nr, 0, NULL, NULL); + __wake_up_common(wq_head, mode, nr, 0, NULL); } EXPORT_SYMBOL_GPL(__wake_up_locked); void __wake_up_locked_key(struct wait_queue_head *wq_head, unsigned int mode, void *key) { - __wake_up_common(wq_head, mode, 1, 0, key, NULL); + __wake_up_common(wq_head, mode, 1, 0, key); } EXPORT_SYMBOL_GPL(__wake_up_locked_key); -void __wake_up_locked_key_bookmark(struct wait_queue_head *wq_head, - unsigned int mode, void *key, wait_queue_entry_t *bookmark) -{ - __wake_up_common(wq_head, mode, 1, 0, key, bookmark); -} -EXPORT_SYMBOL_GPL(__wake_up_locked_key_bookmark); - /** * __wake_up_sync_key - wake up threads blocked on a waitqueue. * @wq_head: the waitqueue @@ -233,7 +193,7 @@ EXPORT_SYMBOL_GPL(__wake_up_sync_key); void __wake_up_locked_sync_key(struct wait_queue_head *wq_head, unsigned int mode, void *key) { - __wake_up_common(wq_head, mode, 1, WF_SYNC, key, NULL); + __wake_up_common(wq_head, mode, 1, WF_SYNC, key); } EXPORT_SYMBOL_GPL(__wake_up_locked_sync_key);