From patchwork Mon Feb 8 12:43:20 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 8249371 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id C6DD5BEEE5 for ; Mon, 8 Feb 2016 12:44:36 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id CBBDC203A9 for ; Mon, 8 Feb 2016 12:44:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E4153203A5 for ; Mon, 8 Feb 2016 12:44:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752928AbcBHMod (ORCPT ); Mon, 8 Feb 2016 07:44:33 -0500 Received: from [193.47.165.129] ([193.47.165.129]:59276 "EHLO mellanox.co.il" rhost-flags-FAIL-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1752371AbcBHMod (ORCPT ); Mon, 8 Feb 2016 07:44:33 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 8 Feb 2016 14:43:54 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id u18Chs3h007058; Mon, 8 Feb 2016 14:43:54 +0200 Received: from vnc17.mtl.labs.mlnx (localhost.localdomain [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id u18ChsLU015347; Mon, 8 Feb 2016 14:43:54 +0200 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id u18Chsej015346; Mon, 8 Feb 2016 14:43:54 +0200 From: Yishai Hadas To: dledford@redhat.com Cc: linux-rdma@vger.kernel.org, yishaih@mellanox.com, matanb@mellanox.com, majd@mellanox.com, talal@mellanox.com, ogerlitz@mellanox.com Subject: [PATCH V1 libibverbs 5/7] Add bind/unbind support for Memory Window type two Date: Mon, 8 Feb 2016 14:43:20 +0200 Message-Id: <1454935402-15192-6-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.7.11.3 In-Reply-To: <1454935402-15192-1-git-send-email-yishaih@mellanox.com> References: <1454935402-15192-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-7.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP To bind: Application should directly post bind WR to the QP using ibv_post_send. To unbind/invalidate there are 2 options, local and remote invalidations. Local invalidation: Send a work request where the immediate data contains the MW's R_key and the opcode is IBV_WR_LOCAL_INV. Upon success a completion with opcode of IBV_WC_LOCAL_INV is polled. Remote invalidation: Send with invalidate can be used to invalidate a remote memory key. The invalidation is done by posting a send work request, where the opcode is IBV_WR_SEND_WITH_INV and the immediate data contains the R_key. Upon success, the responder's work completion will contain the invalidated R_key in its immediate data. Signed-off-by: Yishai Hadas --- include/infiniband/verbs.h | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index 81cc3d9..b9102a6 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -344,6 +344,7 @@ enum ibv_wc_opcode { IBV_WC_COMP_SWAP, IBV_WC_FETCH_ADD, IBV_WC_BIND_MW, + IBV_WC_LOCAL_INV, /* * Set value of IBV_WC_RECV so consumers can test if a completion is a * receive by testing (opcode & IBV_WC_RECV). @@ -359,7 +360,8 @@ enum { enum ibv_wc_flags { IBV_WC_GRH = 1 << 0, IBV_WC_WITH_IMM = 1 << 1, - IBV_WC_IP_CSUM_OK = 1 << IBV_WC_IP_CSUM_OK_SHIFT + IBV_WC_IP_CSUM_OK = 1 << IBV_WC_IP_CSUM_OK_SHIFT, + IBV_WC_WITH_INV = 1 << 3 }; struct ibv_wc { @@ -368,7 +370,10 @@ struct ibv_wc { enum ibv_wc_opcode opcode; uint32_t vendor_err; uint32_t byte_len; - uint32_t imm_data; /* in network byte order */ + /* When (wc_flags & IBV_WC_WITH_IMM): Immediate data in network byte order. + * When (wc_flags & IBV_WC_WITH_INV): Stores the invalidated rkey. + */ + uint32_t imm_data; uint32_t qp_num; uint32_t src_qp; int wc_flags; @@ -701,7 +706,10 @@ enum ibv_wr_opcode { IBV_WR_SEND_WITH_IMM, IBV_WR_RDMA_READ, IBV_WR_ATOMIC_CMP_AND_SWP, - IBV_WR_ATOMIC_FETCH_AND_ADD + IBV_WR_ATOMIC_FETCH_AND_ADD, + IBV_WR_LOCAL_INV, + IBV_WR_BIND_MW, + IBV_WR_SEND_WITH_INV, }; enum ibv_send_flags { @@ -748,6 +756,12 @@ struct ibv_send_wr { uint32_t remote_srqn; } xrc; } qp_type; + struct { + struct ibv_mw *mw; + uint32_t rkey; + struct ibv_mw_bind_info bind_info; + } bind_mw; + }; struct ibv_recv_wr {