From patchwork Thu Aug 28 15:58:54 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 4805601 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id CFFFA9F2A9 for ; Thu, 28 Aug 2014 15:59:04 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 093B82012E for ; Thu, 28 Aug 2014 15:59:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E464920123 for ; Thu, 28 Aug 2014 15:59:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751273AbaH1P7B (ORCPT ); Thu, 28 Aug 2014 11:59:01 -0400 Received: from baptiste.telenet-ops.be ([195.130.132.51]:43705 "EHLO baptiste.telenet-ops.be" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751119AbaH1P7A (ORCPT ); Thu, 28 Aug 2014 11:59:00 -0400 Received: from [192.168.1.117] ([178.119.65.67]) by baptiste.telenet-ops.be with bizsmtp id kFyu1o01m1T3uRu01FyuL7; Thu, 28 Aug 2014 17:58:55 +0200 Message-ID: <53FF51BE.1080103@acm.org> Date: Thu, 28 Aug 2014 17:58:54 +0200 From: Bart Van Assche User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.7.0 MIME-Version: 1.0 To: Mark Lehrer CC: Eli Cohen , Sagi Grimberg , "linux-rdma@vger.kernel.org" Subject: Re: mlx5 + SRP: max_qp_sz mismatch References: <53F36420.3060607@dev.mellanox.co.il> <53FCB18F.5030608@dev.mellanox.co.il> <53FDBED2.2060707@acm.org> In-Reply-To: Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 08/27/14 13:28, Eli Cohen wrote: > On 08/26/14 18:10, Sagi Grimberg wrote: >> >> Since I don't know how true send queue size can be computed from the >> device capabilities at the moment -I can suggest a fix to srpt to >> retry with srp_sq_size/2 (ans so on until it succeeds...) >> > The device capabilities provide the maximum number of send work > requests that the device supports but the actual number of work > requests that can be supported in a specific case depends on other > characteristics of the work requests. For example, in the case of > Connect-IB, the actual number depends on the number of s/g entries, > the transport type, etc. This is in compliance with the IB spec: > > 11.2.1.2 QUERY HCA > Description: > Returns the attributes for the specified HCA. > The maximum values defined in this section are guaranteed > not-to-exceed values. It is possible for an implementation to allocate > some HCA resources from the same space. In that case, the maximum > values returned are not guaranteed for all of those resources > simultaneously. > > So, a well written application should try smaller values if it fails > with ENOMEM. Hello Mark, It would help if you could test the patch below. Sorry but I don't have access to a ConnectIB setup myself. Thanks, Bart. Reported-by: Mark Lehrer Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srpt/ib_srpt.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index fe09f27..3ffaf4e 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -2091,6 +2091,7 @@ static int srpt_create_ch_ib(struct srpt_rdma_ch *ch) if (!qp_init) goto out; +retry: ch->cq = ib_create_cq(sdev->device, srpt_completion, NULL, ch, ch->rq_size + srp_sq_size, 0); if (IS_ERR(ch->cq)) { @@ -2114,6 +2115,13 @@ static int srpt_create_ch_ib(struct srpt_rdma_ch *ch) ch->qp = ib_create_qp(sdev->pd, qp_init); if (IS_ERR(ch->qp)) { ret = PTR_ERR(ch->qp); + if (ret == -ENOMEM) { + srp_sq_size /= 2; + if (srp_sq_size >= MIN_SRPT_SQ_SIZE) { + ib_destroy_cq(ch->cq); + goto retry; + } + } printk(KERN_ERR "failed to create_qp ret= %d\n", ret); goto err_destroy_cq; }