From patchwork Mon Mar 14 11:47:35 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noa Osherovich X-Patchwork-Id: 8578681 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id ECBC39F54C for ; Mon, 14 Mar 2016 11:48:23 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id E1EA420263 for ; Mon, 14 Mar 2016 11:48:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D744D203A5 for ; Mon, 14 Mar 2016 11:48:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752307AbcCNLsT (ORCPT ); Mon, 14 Mar 2016 07:48:19 -0400 Received: from [193.47.165.129] ([193.47.165.129]:45282 "EHLO mellanox.co.il" rhost-flags-FAIL-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1752392AbcCNLsS (ORCPT ); Mon, 14 Mar 2016 07:48:18 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from noaos@mellanox.com) with ESMTPS (AES256-SHA encrypted); 14 Mar 2016 13:47:49 +0200 Received: from vnc20.mtl.labs.mlnx (vnc20.mtl.labs.mlnx [10.7.2.20]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id u2EBln7O025467; Mon, 14 Mar 2016 13:47:49 +0200 Received: from vnc20.mtl.labs.mlnx (localhost.localdomain [127.0.0.1]) by vnc20.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id u2EBlnLG002683; Mon, 14 Mar 2016 13:47:49 +0200 Received: (from noaos@localhost) by vnc20.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id u2EBlnjZ002682; Mon, 14 Mar 2016 13:47:49 +0200 From: Noa Osherovich To: yishaih@mellanox.com Cc: noaos@mellanox.com, linux-rdma@vger.kernel.org, Haggai Abramonvsky Subject: [PATCH libmlx4 2/2] Update consumer index after resize CQ operation Date: Mon, 14 Mar 2016 13:47:35 +0200 Message-Id: <1457956055-2593-3-git-send-email-noaos@mellanox.com> X-Mailer: git-send-email 1.7.11.1 In-Reply-To: <1457956055-2593-1-git-send-email-noaos@mellanox.com> References: <1457956055-2593-1-git-send-email-noaos@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Haggai Abramonvsky The resize CQ operation generates a special CQE to indicate that the resize is done. After consuming this CQE, we need to increment the consumer index and update the doorbell record in the hardware as well. Before this patch, we didn't update the doorbell record in the hardware, which could resulted in CQ overrun. Signed-off-by: Haggai Abramovsky Signed-off-by: Yishai Hadas --- src/cq.c | 9 ++------- src/mlx4.h | 6 ++++++ src/verbs.c | 1 + 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/src/cq.c b/src/cq.c index 32c9070..0bfbce4 100644 --- a/src/cq.c +++ b/src/cq.c @@ -116,11 +116,6 @@ static struct mlx4_cqe *next_cqe_sw(struct mlx4_cq *cq) return get_sw_cqe(cq, cq->cons_index); } -static void update_cons_index(struct mlx4_cq *cq) -{ - *cq->set_ci_db = htonl(cq->cons_index & 0xffffff); -} - static void mlx4_handle_error_cqe(struct mlx4_err_cqe *cqe, struct ibv_wc *wc) { if (cqe->syndrome == MLX4_CQE_SYNDROME_LOCAL_QP_OP_ERR) @@ -356,7 +351,7 @@ int mlx4_poll_cq(struct ibv_cq *ibcq, int ne, struct ibv_wc *wc) } if (npolled || err == CQ_POLL_ERR) - update_cons_index(cq); + mlx4_update_cons_index(cq); pthread_spin_unlock(&cq->lock); @@ -448,7 +443,7 @@ void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) * updating consumer index. */ wmb(); - update_cons_index(cq); + mlx4_update_cons_index(cq); } } diff --git a/src/mlx4.h b/src/mlx4.h index f8d2051..53ea4cb 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -35,6 +35,7 @@ #define MLX4_H #include +#include #include #include @@ -378,6 +379,11 @@ static inline struct mlx4_ah *to_mah(struct ibv_ah *ibah) return to_mxxx(ah, ah); } +static inline void mlx4_update_cons_index(struct mlx4_cq *cq) +{ + *cq->set_ci_db = htonl(cq->cons_index & 0xffffff); +} + int mlx4_alloc_buf(struct mlx4_buf *buf, size_t size, int page_size); void mlx4_free_buf(struct mlx4_buf *buf); diff --git a/src/verbs.c b/src/verbs.c index 2cb1f8a..a4f90bf 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -332,6 +332,7 @@ int mlx4_resize_cq(struct ibv_cq *ibcq, int cqe) mlx4_free_buf(&cq->buf); cq->buf = buf; + mlx4_update_cons_index(cq); out: pthread_spin_unlock(&cq->lock);