From patchwork Wed Feb 15 15:38:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 9574307 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A28FC60493 for ; Wed, 15 Feb 2017 15:38:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 92E6427E5A for ; Wed, 15 Feb 2017 15:38:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 871B7284EE; Wed, 15 Feb 2017 15:38:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2AAFB27E5A for ; Wed, 15 Feb 2017 15:38:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751328AbdBOPiL (ORCPT ); Wed, 15 Feb 2017 10:38:11 -0500 Received: from mail-wr0-f196.google.com ([209.85.128.196]:33092 "EHLO mail-wr0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750999AbdBOPiK (ORCPT ); Wed, 15 Feb 2017 10:38:10 -0500 Received: by mail-wr0-f196.google.com with SMTP id i10so31733981wrb.0; Wed, 15 Feb 2017 07:38:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=+ZFqSzemAwoBLjigBVI9wy/mtyS/HLe1jN53ZQYV7tg=; b=iOYws5EMZsDlbxBT/o65Ol3kKB8eDZB+iruzvTtrSpyBLsrndWhv32JIF9yiHzwI21 7lEUddURd8cJs0cpt/tUNdpFBvQgiWPmisHwZwTl3XlaABvt3NLELzPBmpcEa6/w1fL/ mFbxJXSmBhrlnVRFK5jj9DaZqxY7cZxXxriktG734V+pRXFBKj74At25gZEGxfsfxXuq JPOvp4o3YaVpPcme3IWI3Yu/OxinbeMwOilQSYTJMo7jSzG/viDY0IgejbCOJF3LrlHo qSh253htQVKkhUOYIF6vjdxIVXTtVNRU21Dk2pbFJ3eUwlOWlplO6jcMmaoVLlcu6mGm QAyQ== X-Gm-Message-State: AMke39k7sMY3RGn3yF0489m/dNlPFPKTDyneHErOIcPKFkfregSmm56suX9Fu0xeU/513A== X-Received: by 10.223.166.80 with SMTP id k74mr30488891wrc.171.1487173089396; Wed, 15 Feb 2017 07:38:09 -0800 (PST) Received: from [192.168.64.116] (bzq-82-81-101-184.red.bezeqint.net. [82.81.101.184]) by smtp.gmail.com with ESMTPSA id q12sm8050799wmd.8.2017.02.15.07.38.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Feb 2017 07:38:08 -0800 (PST) Subject: Re: [PATCH v2 1/8] IB/SRP: Avoid using IB_MR_TYPE_SG_GAPS To: Bart Van Assche , Doug Ledford References: <20170214185636.29250-1-bart.vanassche@sandisk.com> <20170214185636.29250-2-bart.vanassche@sandisk.com> Cc: linux-rdma@vger.kernel.org, Israel Rukshin , Max Gurtovoy , Leon Romanovsky , Mark Bloch , Yuval Shaia , "# 4 . 7+" From: Sagi Grimberg Message-ID: Date: Wed, 15 Feb 2017 17:38:06 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <20170214185636.29250-2-bart.vanassche@sandisk.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP > Tests have shown that the following error message is reported when > using SG-GAPS registration with an mlx5 adapter: > > scsi host1: ib_srp: failed RECV status WR flushed (5) for CQE ffff880bd4270eb0 > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 0f007806 2500002a ad9fafd1 > scsi host1: ib_srp: reconnect succeeded > mlx5_0:dump_cqe:262:(pid 7369): dump error cqe > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 0f007806 25000032 00105dd0 > scsi host1: ib_srp: failed FAST REG status memory management operation error (6) for CQE ffff880b92860138 > > Hence avoid using SG-GAPS memory registrations. Additionally, > always configure the blk_queue_virt_boundary() to avoid to trigger > a mapping failure when using adapters that support SG-GAPS (e.g. > mlx5). Hi Guys, Sorry for addressing this late, but has this failure been investigated? Max, Israel, what does this error syndrome map to? Looking at mlx5_ib_sg_to_klms, I think the mr->length is incorrectly incremented. Does the following change fix the problem? --- -- -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 8f608debe141..c21c9eee37f6 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -1832,7 +1832,7 @@ mlx5_ib_sg_to_klms(struct mlx5_ib_mr *mr, klms[i].va = cpu_to_be64(sg_dma_address(sg) + sg_offset); klms[i].bcount = cpu_to_be32(sg_dma_len(sg) - sg_offset); klms[i].key = cpu_to_be32(lkey); - mr->ibmr.length += sg_dma_len(sg); + mr->ibmr.length += sg_dma_len(sg) - sg_offset; sg_offset = 0; }