From patchwork Sun Aug 30 17:26:20 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 7097861 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 001839F1B9 for ; Sun, 30 Aug 2015 17:26:21 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 0B7782075A for ; Sun, 30 Aug 2015 17:26:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EDE5020752 for ; Sun, 30 Aug 2015 17:26:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753537AbbH3R0M (ORCPT ); Sun, 30 Aug 2015 13:26:12 -0400 Received: from mail-wi0-f180.google.com ([209.85.212.180]:35443 "EHLO mail-wi0-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753530AbbH3R0M (ORCPT ); Sun, 30 Aug 2015 13:26:12 -0400 Received: by wicne3 with SMTP id ne3so55651508wic.0 for ; Sun, 30 Aug 2015 10:26:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=UI6f8ONlw+5tTWPw25ScfqMcjEyCeMdpMJqnH7C8hlo=; b=gAaoY6Tg+kYi+g2cmoPGG3+Q2B7SRYZSl4kj4bv0bH02svsUXrXYZMvh9dXd3LxVMg W/iUVYAHVYQNJHmyv9owOmsgRWkGwJxTmLz9avO8PzxSAfN329css/szKET+M9XRMNLo I2pkCR3dAjkrNORGVIPIJEjVJV5jS5d28nLdBhvm7ZtT+P1JOnxv/lURyImaOpyWA7jY a43AF2B8u0scXre7y6dVQ8baUw/aUVLVgD8LBad1RstszkIHHp7eDV5P3zl96bCF6Q/f 9KLOCc5eyxgCp+wgYOr1z/cwE6+koT070B2p64Z6uhjH6tfE8E1CkTcGYJHJ6jeJUtvG UwCg== X-Gm-Message-State: ALoCoQnQL3Kqadvesp/kprpNl/nTC9/85iFDQzh19hXdGzfh2e50fTc3Pfqgqq9ZPSQ0KiFoFvUx X-Received: by 10.180.12.145 with SMTP id y17mr15067895wib.87.1440955570521; Sun, 30 Aug 2015 10:26:10 -0700 (PDT) Received: from [10.223.0.123] ([193.47.165.251]) by smtp.googlemail.com with ESMTPSA id ik8sm18262924wjb.8.2015.08.30.10.26.09 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 30 Aug 2015 10:26:09 -0700 (PDT) Subject: Re: shrink struct ib_send_wr V3 To: Christoph Hellwig , Doug Ledford , Sean Hefty , Eli Cohen References: <1440579639-10684-1-git-send-email-hch@lst.de> <55E321D7.7000209@dev.mellanox.co.il> Cc: linux-rdma@vger.kernel.org From: Sagi Grimberg Message-ID: <55E33CBC.3090000@dev.mellanox.co.il> Date: Sun, 30 Aug 2015 20:26:20 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <55E321D7.7000209@dev.mellanox.co.il> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 8/30/2015 6:31 PM, Sagi Grimberg wrote: >> - patch 2 now explicitly replaces the weird overloading in the mlx5 >> driver with an explicit embedding of struct ib_send_wr, similar >> to what we do for all other MRs. > > That's nice, > > There is one non-trivial spot that was missed in mlx5_ib_post_send > though: > > diff --git a/drivers/infiniband/hw/mlx5/qp.c > b/drivers/infiniband/hw/mlx5/qp.c > index 7ddfb74..35a18d6 100644 > --- a/drivers/infiniband/hw/mlx5/qp.c > +++ b/drivers/infiniband/hw/mlx5/qp.c > @@ -2802,7 +2802,7 @@ int mlx5_ib_post_send(struct ib_qp *ibqp, struct > ib_send_wr *wr, > goto out; > } > qp->sq.wr_data[idx] = MLX5_IB_WR_UMR; > - ctrl->imm = cpu_to_be32(fast_reg_wr(wr)->rkey); > + ctrl->imm = cpu_to_be32(umr_wr(wr)->mkey); > set_reg_umr_segment(seg, wr); > seg += sizeof(struct mlx5_wqe_umr_ctrl_seg); > size += sizeof(struct mlx5_wqe_umr_ctrl_seg) / 16; > > Care to fold this in? > There is another problem I'm seeing in this patch. in reg_umr the local variable is a type ib_send_wr which is passed to prep_umr_reg_wqe() which casts it to mlx5_umr_wr (container_of). It should have probably been mlx5_umr_wr to begin with... I think this needs to be folded in as well: int size; @@ -798,14 +799,14 @@ static struct mlx5_ib_mr *reg_umr(struct ib_pd *pd, struct ib_umem *umem, goto free_pas; } - memset(&wr, 0, sizeof(wr)); - wr.wr_id = (u64)(unsigned long)&umr_context; - prep_umr_reg_wqe(pd, &wr, &sg, dma, npages, mr->mmr.key, page_shift, - virt_addr, len, access_flags); + memset(&umrwr, 0, sizeof(umrwr)); + umrwr.wr.wr_id = (u64)(unsigned long)&umr_context; + prep_umr_reg_wqe(pd, &umrwr.wr, &sg, dma, npages, mr->mmr.key, + page_shift, virt_addr, len, access_flags); mlx5_ib_init_umr_context(&umr_context); down(&umrc->sem); - err = ib_post_send(umrc->qp, &wr, &bad); + err = ib_post_send(umrc->qp, &umrwr.wr, &bad); if (err) { mlx5_ib_warn(dev, "post send failed, err %d\n", err); goto unmap_dma; @@ -1134,16 +1135,17 @@ static int unreg_umr(struct mlx5_ib_dev *dev, struct mlx5_ib_mr *mr) { struct umr_common *umrc = &dev->umrc; struct mlx5_ib_umr_context umr_context; - struct ib_send_wr wr, *bad; + struct mlx5_umr_wr umrwr; + struct ib_send_wr *bad; int err; - memset(&wr, 0, sizeof(wr)); - wr.wr_id = (u64)(unsigned long)&umr_context; - prep_umr_unreg_wqe(dev, &wr, mr->mmr.key); + memset(&umrwr, 0, sizeof(umrwr)); + umrwr.wr.wr_id = (u64)(unsigned long)&umr_context; + prep_umr_unreg_wqe(dev, &umrwr.wr, mr->mmr.key); mlx5_ib_init_umr_context(&umr_context); down(&umrc->sem); - err = ib_post_send(umrc->qp, &wr, &bad); + err = ib_post_send(umrc->qp, &umrwr.wr, &bad); if (err) { up(&umrc->sem); mlx5_ib_dbg(dev, "err %d\n", err); --- -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 9fac2c1..6f8f3d8 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -752,7 +752,8 @@ static struct mlx5_ib_mr *reg_umr(struct ib_pd *pd, struct ib_umem *umem, struct device *ddev = dev->ib_dev.dma_device; struct umr_common *umrc = &dev->umrc; struct mlx5_ib_umr_context umr_context; - struct ib_send_wr wr, *bad; + struct mlx5_umr_wr umrwr; + struct ib_send_wr *bad; struct mlx5_ib_mr *mr; struct ib_sge sg;