From patchwork Mon Nov 5 12:45:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuval Shaia X-Patchwork-Id: 10668153 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7DE9A1709 for ; Mon, 5 Nov 2018 12:48:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B6ECE29426 for ; Mon, 5 Nov 2018 12:48:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A6EA529448; Mon, 5 Nov 2018 12:48:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2EFB529426 for ; Mon, 5 Nov 2018 12:48:32 +0000 (UTC) Received: from localhost ([::1]:34842 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJeIw-00024S-Vn for patchwork-qemu-devel@patchwork.kernel.org; Mon, 05 Nov 2018 07:48:31 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35603) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJeHD-0007T1-7x for qemu-devel@nongnu.org; Mon, 05 Nov 2018 07:46:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJeGu-0002Us-Q8 for qemu-devel@nongnu.org; Mon, 05 Nov 2018 07:46:29 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:39446) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gJeGs-0002Ja-OZ for qemu-devel@nongnu.org; Mon, 05 Nov 2018 07:46:24 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wA5CiJj0060626; Mon, 5 Nov 2018 12:46:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id; s=corp-2018-07-02; bh=qjKzD4x17KvK2L32CxukoOC2ySTpjVb9xh1Llsq2rR8=; b=Eqcz292qqgb43GZtvoFbgcY2JinQqX95TVb6rvDIKL2KxwIkCtCtPvJmqEi++hvy0nzu 6A3QCwBT7edKIXAwysCfqvI3bcrmJO0hBLh7MO3sKvxX+gckPBSt8YUBM5IlEKF7ZnCk WFE3Ztr1TR0aYvs2mO+11Fiu2U+e+bUNjsZpQeTfI5em+NQttP1GN5eiseaHFCkwizDa JiyTHxOCKs5e38+HZArggw2blgkQU7wsPVCQiELSZAJDgZOptlY22dJAT+xr22OsZgMt pXx4fYUvkNVk3MnPoAJ3W85WrT1VVWL4mnpl9ZqNJeBa2KGABgPeK+3IW9RxTLhauqpR qA== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2120.oracle.com with ESMTP id 2nh3mpeetj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 05 Nov 2018 12:46:13 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wA5CkCq6032433 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Nov 2018 12:46:12 GMT Received: from abhmp0020.oracle.com (abhmp0020.oracle.com [141.146.116.26]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wA5CkBo3022525; Mon, 5 Nov 2018 12:46:12 GMT Received: from localhost.localdomain (/77.138.186.148) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 05 Nov 2018 04:46:11 -0800 From: Yuval Shaia To: yuval.shaia@oracle.com, marcel.apfelbaum@gmail.com, dmitry.fleytman@gmail.com, jasowang@redhat.com, eblake@redhat.com, armbru@redhat.com, shamir.rabinovitch@oracle.com, qemu-devel@nongnu.org Date: Mon, 5 Nov 2018 14:45:41 +0200 Message-Id: <20181105124559.21561-1-yuval.shaia@oracle.com> X-Mailer: git-send-email 2.17.2 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9067 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1811050118 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 141.146.126.78 Subject: [Qemu-devel] [PATCH 00/18] Add support for RDMA MAD X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Hi all. This is a major enhancement to the pvrdma device to allow it to work with state of the art applications such as MPI. As described in patch #5, MAD packets are management packets that are used for many purposes including but not limited to communication layer above IB verbs API. Patch 1 exposes new external executable (under contrib) that aims to address a specific limitation in the RDMA usrespace MAD stack. This patch-set mainly present MAD enhancement but during the work on it i came across some bugs and enhancement needed to be implemented before doing any MAD coding. This is the role of patches 2 to 4, 7 to 9 and 15 to 17. Patches 6 and 18 are cosmetic changes while not relevant to this patchset still introduce with it since (at least for 6) hard to decouple. Patches 12 to 15 couple pvrdma device with vmxnet3 device as this is the configuration enforced by pvrdma driver in guest - a vmxnet3 device in function 0 and pvrdma device in function 1 in the same PCI slot. Patch 12 moves needed code from vmxnet3 device to a new header file that can be used by pvrdma code while Patches 13 to 15 use of it. Along with this patch-set there is a parallel patch posted to libvirt to apply the change needed there as part of the process implemented in patches 10 and 11. This change is needed so that guest would be able to configure any IP to the Ethernet function of the pvrdma device. https://www.redhat.com/archives/libvir-list/2018-November/msg00135.html Thanks, Yuval Yuval Shaia (18): contrib/rdmacm-mux: Add implementation of RDMA User MAD multiplexer hw/rdma: Add ability to force notification without re-arm hw/rdma: Return qpn 1 if ibqp is NULL hw/rdma: Abort send-op if fail to create addr handler hw/rdma: Add support for MAD packets hw/pvrdma: Make function reset_device return void hw/pvrdma: Make default pkey 0xFFFF hw/pvrdma: Set the correct opcode for recv completion hw/pvrdma: Set the correct opcode for send completion json: Define new QMP message for pvrdma hw/pvrdma: Add support to allow guest to configure GID table vmxnet3: Move some definitions to header file hw/pvrdma: Make sure PCI function 0 is vmxnet3 hw/rdma: Initialize node_guid from vmxnet3 mac address hw/pvrdma: Make device state depend on Ethernet function state hw/pvrdma: Fill all CQE fields hw/pvrdma: Fill error code in command's response hw/rdma: Remove unneeded code that handles more that one port MAINTAINERS | 2 + Makefile | 6 +- Makefile.objs | 5 + contrib/rdmacm-mux/Makefile.objs | 3 + contrib/rdmacm-mux/main.c | 734 +++++++++++++++++++++++++++++++ contrib/rdmacm-mux/rdmacm-mux.h | 47 ++ hw/net/vmxnet3.c | 116 +---- hw/net/vmxnet3_defs.h | 133 ++++++ hw/rdma/rdma_backend.c | 439 +++++++++++++++--- hw/rdma/rdma_backend.h | 28 +- hw/rdma/rdma_backend_defs.h | 13 +- hw/rdma/rdma_rm.c | 111 ++++- hw/rdma/rdma_rm.h | 17 +- hw/rdma/rdma_rm_defs.h | 21 +- hw/rdma/rdma_utils.h | 24 + hw/rdma/vmw/pvrdma.h | 8 +- hw/rdma/vmw/pvrdma_cmd.c | 112 +++-- hw/rdma/vmw/pvrdma_main.c | 37 +- hw/rdma/vmw/pvrdma_qp_ops.c | 62 ++- qapi/qapi-schema.json | 1 + qapi/rdma.json | 34 ++ 21 files changed, 1667 insertions(+), 286 deletions(-) create mode 100644 contrib/rdmacm-mux/Makefile.objs create mode 100644 contrib/rdmacm-mux/main.c create mode 100644 contrib/rdmacm-mux/rdmacm-mux.h create mode 100644 hw/net/vmxnet3_defs.h create mode 100644 qapi/rdma.json