[v2,6/6] coroutine/rwlock: Avoid thundering herd when unlocking

Commit Message

David Edmondson March 9, 2021, 2:40 p.m. UTC

Given that we know whether the queued coroutines are reader hopefuls
or writer hopefuls, avoid marking all of the queued coroutines as
runnable when unlocking, choosing instead to wake a single queued
writer or all queued readers.

Suggested-by: Paolo Bonzini <[e-mail]>
Signed-off-by: David Edmondson <[e-mail]>
---
 util/qemu-coroutine-lock.c | 22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)

Patch

diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c
index c05c143142..6e399f28c1 100644
--- a/util/qemu-coroutine-lock.c
+++ b/util/qemu-coroutine-lock.c
@@ -356,24 +356,20 @@  void qemu_co_rwlock_unlock(CoRwlock *lock)
     Coroutine *self = qemu_coroutine_self();
 
     assert(qemu_in_coroutine());
-    if (!lock->reader) {
-        /* The critical section started in qemu_co_rwlock_wrlock or
-         * qemu_co_rwlock_upgrade.
-         */
-        qemu_co_queue_restart_all(&lock->wqueue);
-        qemu_co_queue_restart_all(&lock->rqueue);
-    } else {
+    if (lock->reader) {
         self->locks_held--;
 
+        /* Read-side critical sections do not hold lock->mutex. */
         qemu_co_mutex_lock(&lock->mutex);
         lock->reader--;
         assert(lock->reader >= 0);
-        /* If there are no remaining readers wake one waiting writer
-         * or all waiting readers.
-         */
-        if (!lock->reader && !qemu_co_queue_next(&lock->wqueue)) {
-            qemu_co_queue_restart_all(&lock->rqueue);
-        }
+    }
+
+    /* If there are no remaining readers wake one waiting writer or
+     * all waiting readers.
+     */
+    if (!lock->reader && !qemu_co_queue_next(&lock->wqueue)) {
+        qemu_co_queue_restart_all(&lock->rqueue);
     }
     qemu_co_mutex_unlock(&lock->mutex);
 }