From patchwork Thu Sep 20 20:31:02 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Or Gerlitz X-Patchwork-Id: 1487711 X-Patchwork-Delegate: roland@digitalvampire.org Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id E33F1DF2D2 for ; Thu, 20 Sep 2012 20:31:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753115Ab2ITUbR (ORCPT ); Thu, 20 Sep 2012 16:31:17 -0400 Received: from eu1sys200aog117.obsmtp.com ([207.126.144.143]:38460 "HELO eu1sys200aog117.obsmtp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1752873Ab2ITUbQ (ORCPT ); Thu, 20 Sep 2012 16:31:16 -0400 Received: from mtlsws123.lab.mtl.com ([82.166.227.17]) (using TLSv1) by eu1sys200aob117.postini.com ([207.126.147.11]) with SMTP ID DSNKUFt9D6S7wykKVw3cgjepddvcxNOZ5bRj@postini.com; Thu, 20 Sep 2012 20:31:15 UTC Received: from r-vnc04.lab.mtl.com (r-vnc04.lab.mtl.com [10.208.0.116]) by mtlsws123.lab.mtl.com (8.13.8/8.13.8) with ESMTP id q8KKV9OO022186; Thu, 20 Sep 2012 23:31:11 +0300 From: Or Gerlitz To: roland@kernel.org Cc: linux-rdma@vger.kernel.org, Sagi Grimberg , Jack Morgenstein , Or Gerlitz Subject: [PATCH libmlx4 3/8] Limit qp resources accepted for ibv_create_qp() Date: Thu, 20 Sep 2012 23:31:02 +0300 Message-Id: <1348173067-12116-4-git-send-email-ogerlitz@mellanox.com> X-Mailer: git-send-email 1.7.8.2 In-Reply-To: <1348173067-12116-1-git-send-email-ogerlitz@mellanox.com> References: <1348173067-12116-1-git-send-email-ogerlitz@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Sagi Grimberg Use the limits reported in ib_query_device(). Make sure that the limits returned to the caller following qp creation also lie within the reported device limits. Signed-off-by: Sagi Grimberg Signed-off-by: Jack Morgenstein Signed-off-by: Or Gerlitz --- src/mlx4.h | 14 ++++++++++++++ src/qp.c | 6 ++++-- src/verbs.c | 18 +++++++++++++----- 3 files changed, 31 insertions(+), 7 deletions(-) diff --git a/src/mlx4.h b/src/mlx4.h index efaa7e9..e1daaf7 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -83,6 +83,20 @@ #define PFX "mlx4: " +#ifndef max +#define max(a, b) \ + ({ typeof(a) _a = (a); \ + typeof(b) _b = (b); \ + _a > _b ? _a : _b; }) +#endif + +#ifndef min +#define min(a, b) \ + ({ typeof(a) _a = (a); \ + typeof(b) _b = (b); \ + _a < _b ? _a : _b; }) +#endif + enum { MLX4_CQ_ENTRY_SIZE = 0x20 }; diff --git a/src/qp.c b/src/qp.c index 90c4e80..0dfe3a7 100644 --- a/src/qp.c +++ b/src/qp.c @@ -628,6 +628,7 @@ void mlx4_set_sq_sizes(struct mlx4_qp *qp, struct ibv_qp_cap *cap, enum ibv_qp_type type) { int wqe_size; + struct mlx4_context *ctx = to_mctx(qp->ibv_qp.context); wqe_size = (1 << qp->sq.wqe_shift) - sizeof (struct mlx4_wqe_ctrl_seg); switch (type) { @@ -645,8 +646,9 @@ void mlx4_set_sq_sizes(struct mlx4_qp *qp, struct ibv_qp_cap *cap, } qp->sq.max_gs = wqe_size / sizeof (struct mlx4_wqe_data_seg); - cap->max_send_sge = qp->sq.max_gs; - qp->sq.max_post = qp->sq.wqe_cnt - qp->sq_spare_wqes; + cap->max_send_sge = min(ctx->max_sge, qp->sq.max_gs); + qp->sq.max_post = min(ctx->max_qp_wr, + qp->sq.wqe_cnt - qp->sq_spare_wqes); cap->max_send_wr = qp->sq.max_post; /* diff --git a/src/verbs.c b/src/verbs.c index 408fc6d..f629275 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -390,12 +390,14 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) struct ibv_create_qp_resp resp; struct mlx4_qp *qp; int ret; + struct mlx4_context *context = to_mctx(pd->context); + /* Sanity check QP size before proceeding */ - if (attr->cap.max_send_wr > 65536 || - attr->cap.max_recv_wr > 65536 || - attr->cap.max_send_sge > 64 || - attr->cap.max_recv_sge > 64 || + if (attr->cap.max_send_wr > context->max_qp_wr || + attr->cap.max_recv_wr > context->max_qp_wr || + attr->cap.max_send_sge > context->max_sge || + attr->cap.max_recv_sge > context->max_sge || attr->cap.max_inline_data > 1024) return NULL; @@ -464,8 +466,14 @@ struct ibv_qp *mlx4_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *attr) goto err_destroy; pthread_mutex_unlock(&to_mctx(pd->context)->qp_table_mutex); - qp->rq.wqe_cnt = qp->rq.max_post = attr->cap.max_recv_wr; + qp->rq.wqe_cnt = attr->cap.max_recv_wr; qp->rq.max_gs = attr->cap.max_recv_sge; + + /* adjust rq maxima to not exceed reported device maxima */ + attr->cap.max_recv_wr = min(context->max_qp_wr, attr->cap.max_recv_wr); + attr->cap.max_recv_sge = min(context->max_sge, attr->cap.max_recv_sge); + + qp->rq.max_post = attr->cap.max_recv_wr; mlx4_set_sq_sizes(qp, &attr->cap, attr->qp_type); qp->doorbell_qpn = htonl(qp->ibv_qp.qp_num << 8);