From patchwork Wed Jul 27 11:30:37 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 9249697 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 423AC6077C for ; Wed, 27 Jul 2016 11:31:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3363A20855 for ; Wed, 27 Jul 2016 11:31:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 27A29271FD; Wed, 27 Jul 2016 11:31:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A837520855 for ; Wed, 27 Jul 2016 11:31:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755435AbcG0LbG (ORCPT ); Wed, 27 Jul 2016 07:31:06 -0400 Received: from mail-pf0-f193.google.com ([209.85.192.193]:35382 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754377AbcG0La5 (ORCPT ); Wed, 27 Jul 2016 07:30:57 -0400 Received: by mail-pf0-f193.google.com with SMTP id h186so1707097pfg.2; Wed, 27 Jul 2016 04:30:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id; bh=eCtwBsmqgVi1cVFFqTFWpiGm59uNtcrSHYuBKMqI6+M=; b=TjIBiYs19XmeMuDwdHxxpNdIdhBkdZT8+Zuarhmdsq1BYfIKBRox+vRbUTnqVo48IR QgJJm73KeBt+5S8gM30ZJws8O4lhgt9FqZCfoF/ys+gA25nMjm0roCudGbPChgUjBumm sVgnFxb+3tqev1IKSOAtUJAdYB2m9OnlnsoWS65tcEpjTqtZ/N3J5fdHBR0VEzfJDaIU RKWsTulXN2sKpnmF23UgInk7qUQTrotWVxZr0grPEuothHYvaGy3U7GsAO65QZhXF3f4 K3SfbaiwZKsLbApiB6q0LceJIkfx9SVVGyGxEOstl4METM4g4cveyY8M1OG5TmkZ/UbV QLcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=eCtwBsmqgVi1cVFFqTFWpiGm59uNtcrSHYuBKMqI6+M=; b=DW+ejM7RcUZYMzKCtZu8AQ3VUq0FVXdv3PiUT9i2nPHCpzVPVF39XdgL7QlVXOhwN2 yWCRKoe5TfGVdHkKcA/osHoBn253BJxva2K7POY3CFE3W1TTCM8V1N0110U6PFx2HNPl NIhKn1qY/6o6+FLA+8uTdOnulsjPD+ABs2gLWEDyTMOZsUMxAWgERrAgZzgJL5LYGp7u 1+OeCBm2sgTg+mQJv4P1XYkKvZX4sDRG2gMUO5hiYNKW/ulW6R8ggasQQGWfaLj69GW/ B6uB6pV4A7ht+eKoqixtHq514urOk7kOKLY+Ikoc6QnxZPsPxIAN6CVX9s6MMt4XS7bU 1Riw== X-Gm-Message-State: AEkoout50dRgOASV9xqH0YBxxWm4w8rjFL1Vhu/hETzxJATkyqmLdsOQR0gc5o0FF4I5Sw== X-Received: by 10.98.74.201 with SMTP id c70mr48600995pfj.113.1469619057163; Wed, 27 Jul 2016 04:30:57 -0700 (PDT) Received: from kernel.ksyun.com ([114.255.44.132]) by smtp.gmail.com with ESMTPSA id p9sm8635248pfj.3.2016.07.27.04.30.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 27 Jul 2016 04:30:55 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: Ingo Molnar , Peter Zijlstra , linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Wanpeng Li , Waiman Long , Davidlohr Bueso Subject: [PATCH RESEND v4] locking/pvqspinlock: Fix double hash race Date: Wed, 27 Jul 2016 19:30:37 +0800 Message-Id: <1469619037-13826-1-git-send-email-wanpeng.li@hotmail.com> X-Mailer: git-send-email 1.9.1 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Wanpeng Li When the lock holder vCPU is racing with the queue head vCPU: lock holder vCPU queue head vCPU Acked-by: Waiman Long ===================== ================== node->locked = 1; READ_ONCE(node->locked) ... pv_wait_head_or_lock(): SPIN_THRESHOLD loop; pv_hash(); lock->locked = _Q_SLOW_VAL; node->state = vcpu_hashed; pv_kick_node(): cmpxchg(node->state, vcpu_halted, vcpu_hashed); lock->locked = _Q_SLOW_VAL; pv_hash(); With preemption at the right moment, it is possible that both the lock holder and queue head vCPUs can be racing to set node->state which can result in hash entry race. Making sure the state is never set to vcpu_halted will prevent this racing from happening. This patch fix it by setting vcpu_hashed after we did all hash thing. Reviewed-by: Davidlohr Bueso Reviewed-by: Pan Xinhui Cc: Peter Zijlstra (Intel) Cc: Ingo Molnar Cc: Waiman Long Cc: Davidlohr Bueso Signed-off-by: Wanpeng Li --- v3 -> v4: * update patch subject * add code comments v2 -> v3: * fix typo in patch description v1 -> v2: * adjust patch description kernel/locking/qspinlock_paravirt.h | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/kernel/locking/qspinlock_paravirt.h b/kernel/locking/qspinlock_paravirt.h index 21ede57..ca96db4 100644 --- a/kernel/locking/qspinlock_paravirt.h +++ b/kernel/locking/qspinlock_paravirt.h @@ -450,7 +450,28 @@ pv_wait_head_or_lock(struct qspinlock *lock, struct mcs_spinlock *node) goto gotlock; } } - WRITE_ONCE(pn->state, vcpu_halted); + /* + * lock holder vCPU queue head vCPU + * ---------------- --------------- + * node->locked = 1; + * READ_ONCE(node->locked) + * ... pv_wait_head_or_lock(): + * SPIN_THRESHOLD loop; + * pv_hash(); + * lock->locked = _Q_SLOW_VAL; + * node->state = vcpu_hashed; + * pv_kick_node(): + * cmpxchg(node->state, + * vcpu_halted, vcpu_hashed); + * lock->locked = _Q_SLOW_VAL; + * pv_hash(); + * + * With preemption at the right moment, it is possible that both the + * lock holder and queue head vCPUs can be racing to set node->state. + * Making sure the state is never set to vcpu_halted will prevent this + * racing from happening. + */ + WRITE_ONCE(pn->state, vcpu_hashed); qstat_inc(qstat_pv_wait_head, true); qstat_inc(qstat_pv_wait_again, waitcnt); pv_wait(&l->locked, _Q_SLOW_VAL);