From patchwork Wed Oct 6 01:58:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12538343 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 399D1C433FE for ; Wed, 6 Oct 2021 01:58:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 14CDA610CE for ; Wed, 6 Oct 2021 01:58:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231872AbhJFCAP (ORCPT ); Tue, 5 Oct 2021 22:00:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229908AbhJFCAP (ORCPT ); Tue, 5 Oct 2021 22:00:15 -0400 Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com [IPv6:2607:f8b0:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CD8C3C061749 for ; Tue, 5 Oct 2021 18:58:23 -0700 (PDT) Received: by mail-ot1-x330.google.com with SMTP id g62-20020a9d2dc4000000b0054752cfbc59so1330173otb.1 for ; Tue, 05 Oct 2021 18:58:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pcxAdiImyK4B9yAh9TbVrtYx8EXTONj/TcOyvXcVc/w=; b=hRJBLiy3dVN0BU1JAaW/7WNWqtIwFsk8AFQaAIkwPCCizLaHYjsMCJTll1eCCXTCE/ PBD7RVVW4fR2qDbU0biDpXr9Kta+Oqnh45hrBHcW1UuW1PohMvBc/KBo/Npokr7Wu3PM ZJHk99aDVbCQXOF9o2ExeJuYgCTtJCQt8jAmNRIuFn8WLp+Jeua5EVeD9MOytxr3TDrl 42Gjgy7cKBN210DYo2HMn0fLevGu8YU9LJJexM66cfF/Vyfh0o1jJvLesRJY1jzvTrHY lgB1Dw0FgAvc0JFRP9dIfVvYC2DWYhtDa2xv9kDap0IwNzxYNpGf0p+VJtgsisaaz6g3 vdVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pcxAdiImyK4B9yAh9TbVrtYx8EXTONj/TcOyvXcVc/w=; b=qRlYxii8Gb038htT/CmReqOeXhLA+G3QgGPhJRXb2B/I1iJSo2hGIU2ep6LJ2Ka97S YdkptIO3BxlsnZiHOm4vLUZN6rjVP4Sxh9lWkc+BTBEx5Zrj0b/t28QErRWFL+haxGw/ ocikLafOC/3+/wXyugRrq7yQD6+LF2jGcv2jIj7WSGPvO7RcSvxKU2TfSxafVWY942L0 OnQxAJIfAAgePigO4/43sedgUvLlHLShv+pZr+jorOoNo4/22dginDbaujtM01qiZek1 AHLIqBzNfU/rkl9UAw7Atu2pB8ZjKxAQaAQA3UnRH2JrHaZc8B0aCtcMaMg7pmAq20iX se1g== X-Gm-Message-State: AOAM531rHk9p4/OG2c33WG72AhNtiEh/ib0169mvKkmlvL9iGKulWfw7 H9N/gjszYsOgQeZ/LqCDdI8= X-Google-Smtp-Source: ABdhPJy/zaT7plcB3tvHhl/Hpepe77JRbwCNy/Dr/KHLrPS39wxAwzG7aHd2SV7knXlhbWOjRnA8Gw== X-Received: by 2002:a05:6830:4105:: with SMTP id w5mr3188437ott.36.1633485503240; Tue, 05 Oct 2021 18:58:23 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-2b16-c5f3-6fcc-9065.res6.spectrum.com. [2603:8081:140c:1a00:2b16:c5f3:6fcc:9065]) by smtp.gmail.com with ESMTPSA id e2sm4016057otk.46.2021.10.05.18.58.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Oct 2021 18:58:23 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v5 1/6] RDMA/rxe: Move AV from rxe_send_wqe to rxe_send_wr Date: Tue, 5 Oct 2021 20:58:10 -0500 Message-Id: <20211006015815.28350-2-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211006015815.28350-1-rpearsonhpe@gmail.com> References: <20211006015815.28350-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Move the struct rxe_av av from struct rxe_send_wqe to struct rxe_send_wr placing it in wr.ud at the same offset as it was previously. This has the effect of increasing the size of struct rxe_send_wr while keeping the size of struct rxe_send_wqe the same. This better reflects the use of this field which is only used for UD sends. This change has no effect on ABI compatibility so the modified rxe driver will operate with older versions of rdma-core. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_av.c | 2 +- drivers/infiniband/sw/rxe/rxe_verbs.c | 3 ++- include/uapi/rdma/rdma_user_rxe.h | 4 +++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_av.c b/drivers/infiniband/sw/rxe/rxe_av.c index da2e867a1ed9..85580ea5eed0 100644 --- a/drivers/infiniband/sw/rxe/rxe_av.c +++ b/drivers/infiniband/sw/rxe/rxe_av.c @@ -107,5 +107,5 @@ struct rxe_av *rxe_get_av(struct rxe_pkt_info *pkt) if (qp_type(pkt->qp) == IB_QPT_RC || qp_type(pkt->qp) == IB_QPT_UC) return &pkt->qp->pri_av; - return (pkt->wqe) ? &pkt->wqe->av : NULL; + return (pkt->wqe) ? &pkt->wqe->wr.wr.ud.av : NULL; } diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 9d0bb9aa7514..c09e1c25ce66 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -584,7 +584,8 @@ static void init_send_wqe(struct rxe_qp *qp, const struct ib_send_wr *ibwr, if (qp_type(qp) == IB_QPT_UD || qp_type(qp) == IB_QPT_SMI || qp_type(qp) == IB_QPT_GSI) - memcpy(&wqe->av, &to_rah(ud_wr(ibwr)->ah)->av, sizeof(wqe->av)); + memcpy(&wqe->wr.wr.ud.av, &to_rah(ud_wr(ibwr)->ah)->av, + sizeof(struct rxe_av)); if (unlikely(ibwr->send_flags & IB_SEND_INLINE)) copy_inline_data_to_wqe(wqe, ibwr); diff --git a/include/uapi/rdma/rdma_user_rxe.h b/include/uapi/rdma/rdma_user_rxe.h index e283c2220aba..2f1ebbe96434 100644 --- a/include/uapi/rdma/rdma_user_rxe.h +++ b/include/uapi/rdma/rdma_user_rxe.h @@ -98,6 +98,9 @@ struct rxe_send_wr { __u32 remote_qpn; __u32 remote_qkey; __u16 pkey_index; + __u16 reserved; + __u32 pad[5]; + struct rxe_av av; } ud; struct { __aligned_u64 addr; @@ -148,7 +151,6 @@ struct rxe_dma_info { struct rxe_send_wqe { struct rxe_send_wr wr; - struct rxe_av av; __u32 status; __u32 state; __aligned_u64 iova; From patchwork Wed Oct 6 01:58:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12538349 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45C67C4332F for ; Wed, 6 Oct 2021 01:58:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2345D6113A for ; Wed, 6 Oct 2021 01:58:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229908AbhJFCAQ (ORCPT ); Tue, 5 Oct 2021 22:00:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233994AbhJFCAP (ORCPT ); Tue, 5 Oct 2021 22:00:15 -0400 Received: from mail-ot1-x332.google.com (mail-ot1-x332.google.com [IPv6:2607:f8b0:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 696C9C061749 for ; Tue, 5 Oct 2021 18:58:24 -0700 (PDT) Received: by mail-ot1-x332.google.com with SMTP id c6-20020a9d2786000000b005471981d559so1236950otb.5 for ; Tue, 05 Oct 2021 18:58:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ln2CICo8MaUfdlYX/StcL4ROVFBeevggZ31VqY1hkwM=; b=XGW/8Xkv/uddut++lyRdHr+L2fNwkYIUZQJDvMpBEi81Mp+cD2dum10LEZpadqlB4S 0Kj5odIuN6Iii6Z3LFO9CiY7AK81QLLWLW228ZEGU25VInIZ7M4Sm8YwdbFlC8kHuVIe 6nm1l31XA8slAVK47PRrwGI4XBLrWIVmIjFrSghI0iodgGFGjh0bdOmLy1mbmbLAJEki kNaZezUkmfYtvVZz1RXA9XY2UlG6tj6atpHqA1cA5QyP6mogGT/KdDCu0k8PPMqveCem JpOVShVvCtFD55GPXcSqKdcaMm6QqbP8IUtVnkTn48LOWoWPxyJQvtD9ck1ltBdmFYpR 7GTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ln2CICo8MaUfdlYX/StcL4ROVFBeevggZ31VqY1hkwM=; b=L6ZdXbJuUuHXEjHe+PqlgQf7K9TYWe7t7IGsiXp/BeseaqRCh7ibRIHhgmd/6ibUDy FCKq6h9qasddAC7v02qiDc01jEnjNPzbK6UXJBAzYu8Ys8GDGoBBCrLGHGngToMNTLBv V1lGz7caMpu5HwTDjd+xZbPpLrzRbnykkb9Wsu8lFR982vkCJ93nCAaG9ToSBTE5e98L b2E/dQmO5HosDH4dnoF/SOAGaUlRbDyiyyv9pF3gRLfWYG9nxyrdR4YoCJ69Unykyd9v 8KORK3aBlUoAQUrsZ5sdDSL00mYMZK+8uhQ12Vbxxx1DOn+uGG534nAJSZUp+HHaH90P g8og== X-Gm-Message-State: AOAM531g/7B6KPEbQFQ79KIODrjAH+iM7dF78jtnKSHlWlZR7xh94lgR i+qkzksRIK6yMfJBO5Hzq/U= X-Google-Smtp-Source: ABdhPJyTinWVupIbu+RpFPOc8bvKbhzxSeHCw/+1zQ5Fm5OtjPeOTSv+ctOdUET+1Ef+0XYg7vrHzg== X-Received: by 2002:a9d:bb1:: with SMTP id 46mr17261979oth.243.1633485503842; Tue, 05 Oct 2021 18:58:23 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-2b16-c5f3-6fcc-9065.res6.spectrum.com. [2603:8081:140c:1a00:2b16:c5f3:6fcc:9065]) by smtp.gmail.com with ESMTPSA id e2sm4016057otk.46.2021.10.05.18.58.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Oct 2021 18:58:23 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v5 2/6] RDMA/rxe: Change AH objects to indexed Date: Tue, 5 Oct 2021 20:58:11 -0500 Message-Id: <20211006015815.28350-3-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211006015815.28350-1-rpearsonhpe@gmail.com> References: <20211006015815.28350-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Make changes to rxe_param.h and rxe_pool.c to allow indexing of AH objects. Valid indices are non-zero so older providers can be detected. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_param.h | 4 +++- drivers/infiniband/sw/rxe/rxe_pool.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h index b5a70cbe94aa..d92d7edd712b 100644 --- a/drivers/infiniband/sw/rxe/rxe_param.h +++ b/drivers/infiniband/sw/rxe/rxe_param.h @@ -67,7 +67,9 @@ enum rxe_device_param { RXE_MAX_MCAST_GRP = 8192, RXE_MAX_MCAST_QP_ATTACH = 56, RXE_MAX_TOT_MCAST_QP_ATTACH = 0x70000, - RXE_MAX_AH = 100, + RXE_MAX_AH = 16383, + RXE_MIN_AH_INDEX = 1, + RXE_MAX_AH_INDEX = 16383, RXE_MAX_SRQ_WR = 0x4000, RXE_MIN_SRQ_WR = 1, RXE_MAX_SRQ_SGE = 27, diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index ffa8420b4765..7b4cb46edfd9 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -26,7 +26,9 @@ struct rxe_type_info rxe_type_info[RXE_NUM_TYPES] = { .name = "rxe-ah", .size = sizeof(struct rxe_ah), .elem_offset = offsetof(struct rxe_ah, pelem), - .flags = RXE_POOL_NO_ALLOC, + .flags = RXE_POOL_INDEX | RXE_POOL_NO_ALLOC, + .min_index = RXE_MIN_AH_INDEX, + .max_index = RXE_MAX_AH_INDEX, }, [RXE_TYPE_SRQ] = { .name = "rxe-srq", From patchwork Wed Oct 6 01:58:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12538347 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 803F4C433EF for ; Wed, 6 Oct 2021 01:58:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 62D1E6113A for ; Wed, 6 Oct 2021 01:58:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237059AbhJFCAQ (ORCPT ); Tue, 5 Oct 2021 22:00:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233994AbhJFCAQ (ORCPT ); Tue, 5 Oct 2021 22:00:16 -0400 Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com [IPv6:2607:f8b0:4864:20::22f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D21DC061749 for ; Tue, 5 Oct 2021 18:58:25 -0700 (PDT) Received: by mail-oi1-x22f.google.com with SMTP id a3so1880181oid.6 for ; Tue, 05 Oct 2021 18:58:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lBxkpKC/Ie6Rn+yTC69M869crKSuud+tFzqyCacmKP8=; b=aWRPwrwSLhm03F60jFJKc25Aqn13/FS73db6Od1aFGTyF55SbEn36Op8uq5KAa+uJ5 w73Yp5LdBsme0K7hEAVypcTppUThnVQ9hIWpiHxNmjEW7VB0nokEPKrD2TOfep1g7Vw5 55Wg7gc/BQYsVqXBoCsg9ZVjaO2/+BktkO/aMBfOefX83v3PuGb71pKONFNS+QWedXTm ctJhUkwwG53bZR/8qSQv6Y/D04nVhqwfdeYln8tV86qYeALVH/Lbm2Ub3h1y2LC+inVZ 9NW6SSqMWVagMZy2zPITPppjG0G7lHf2Q8FIprCqE4woUKlN66h3+EdvNqueEQ5M1QHA 6eng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lBxkpKC/Ie6Rn+yTC69M869crKSuud+tFzqyCacmKP8=; b=LCt8CPoqFcvgwMbLLOP7Q6l2TkF0hBIdVvyBY15/AoSSmN3wHdUQtEgYLCKAKJfwOr bRunz7SP5zOR68sM6st4YQaUW+nx9jkqWPIbeYq4tLKxRltHS0Rw+r5PFr4ynXwwaMG3 q0dnDQgx6Wsv+iq+v5gq7emfeEsTp3Croy2VuJoEKp6BJqWTw8F5xqWVZ+5xcUpBlnna LmlvHwaQ6knUR85xWWX1jHXVAyaZF2Jrg4pqwiA2Xm9LAd0UqY8ypWl4eYohpMSOxGkX RFAxXOMapeFImxUt9RDw6dlBYer6eHAMhEmxUhe0pj2wxjIAa0tu4eWr5lKvA8UCSjoB pzUw== X-Gm-Message-State: AOAM530d/VLAuXGNfJ0vpdNp047TriY49t47JBWcchuumX5o9eUV4CYy AHxwP0hj/z+zVJcuapVWAudvDl77eC5Rbw== X-Google-Smtp-Source: ABdhPJxbZSRQPQGCkNwmORQbrRhm4MWo7ZQ+QE5/FcGzE70FjqnFN6V8nlDgKM7wt4M3yeDWNlMD/A== X-Received: by 2002:a05:6808:1522:: with SMTP id u34mr5172916oiw.76.1633485504466; Tue, 05 Oct 2021 18:58:24 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-2b16-c5f3-6fcc-9065.res6.spectrum.com. [2603:8081:140c:1a00:2b16:c5f3:6fcc:9065]) by smtp.gmail.com with ESMTPSA id e2sm4016057otk.46.2021.10.05.18.58.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Oct 2021 18:58:24 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v5 3/6] RDMA/rxe: Create AH index and return to user space Date: Tue, 5 Oct 2021 20:58:12 -0500 Message-Id: <20211006015815.28350-4-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211006015815.28350-1-rpearsonhpe@gmail.com> References: <20211006015815.28350-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Make changes to rdma_user_rxe.h to allow indexing AH objects, passing the index in UD send WRs to the driver and returning the index to the rxe provider. Modify rxe_create_ah() to add an index to AH when created and if called from a new user provider return it to user space. If called from an old provider mark the AH as not having a useful index. Modify rxe_destroy_ah to drop the index before deleting the object. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_verbs.c | 31 ++++++++++++++++++++++++++- drivers/infiniband/sw/rxe/rxe_verbs.h | 2 ++ include/uapi/rdma/rdma_user_rxe.h | 8 ++++++- 3 files changed, 39 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index c09e1c25ce66..8854ace63acd 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -158,9 +158,19 @@ static int rxe_create_ah(struct ib_ah *ibah, struct ib_udata *udata) { - int err; struct rxe_dev *rxe = to_rdev(ibah->device); struct rxe_ah *ah = to_rah(ibah); + struct rxe_create_ah_resp __user *uresp = NULL; + int err; + + if (udata) { + /* test if new user provider */ + if (udata->outlen >= sizeof(*uresp)) + uresp = udata->outbuf; + ah->is_user = true; + } else { + ah->is_user = false; + } err = rxe_av_chk_attr(rxe, init_attr->ah_attr); if (err) @@ -170,6 +180,24 @@ static int rxe_create_ah(struct ib_ah *ibah, if (err) return err; + /* create index > 0 */ + rxe_add_index(ah); + ah->ah_num = ah->pelem.index; + + if (uresp) { + /* only if new user provider */ + err = copy_to_user(&uresp->ah_num, &ah->ah_num, + sizeof(uresp->ah_num)); + if (err) { + rxe_drop_index(ah); + rxe_drop_ref(ah); + return -EFAULT; + } + } else if (ah->is_user) { + /* only if old user provider */ + ah->ah_num = 0; + } + rxe_init_av(init_attr->ah_attr, &ah->av); return 0; } @@ -202,6 +230,7 @@ static int rxe_destroy_ah(struct ib_ah *ibah, u32 flags) { struct rxe_ah *ah = to_rah(ibah); + rxe_drop_index(ah); rxe_drop_ref(ah); return 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index c807639435eb..9cd203f1fa22 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -48,6 +48,8 @@ struct rxe_ah { struct rxe_pool_entry pelem; struct rxe_pd *pd; struct rxe_av av; + bool is_user; + int ah_num; }; struct rxe_cqe { diff --git a/include/uapi/rdma/rdma_user_rxe.h b/include/uapi/rdma/rdma_user_rxe.h index 2f1ebbe96434..dc9f7a5e203a 100644 --- a/include/uapi/rdma/rdma_user_rxe.h +++ b/include/uapi/rdma/rdma_user_rxe.h @@ -99,7 +99,8 @@ struct rxe_send_wr { __u32 remote_qkey; __u16 pkey_index; __u16 reserved; - __u32 pad[5]; + __u32 ah_num; + __u32 pad[4]; struct rxe_av av; } ud; struct { @@ -170,6 +171,11 @@ struct rxe_recv_wqe { struct rxe_dma_info dma; }; +struct rxe_create_ah_resp { + __u32 ah_num; + __u32 reserved; +}; + struct rxe_create_cq_resp { struct mminfo mi; }; From patchwork Wed Oct 6 01:58:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12538345 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D36EC43219 for ; Wed, 6 Oct 2021 01:58:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EE47C610CE for ; Wed, 6 Oct 2021 01:58:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237115AbhJFCAR (ORCPT ); Tue, 5 Oct 2021 22:00:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237102AbhJFCAQ (ORCPT ); Tue, 5 Oct 2021 22:00:16 -0400 Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com [IPv6:2607:f8b0:4864:20::22f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D8ADC061749 for ; Tue, 5 Oct 2021 18:58:25 -0700 (PDT) Received: by mail-oi1-x22f.google.com with SMTP id t4so44447oie.5 for ; Tue, 05 Oct 2021 18:58:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qApN8UnhyEUIjIpM/GmsOjj23Il/VJ1qkJLR9SYk3QA=; b=h1k53aN/IQWv9m++g8Olw7Tm/Yp44TPDDzoOCya7Hc3Y1GDwdqZwFE5PqZ8bmUH4LA KZaTjof1bYo7wr3uesxZWBt3NCadwwDCxy08Cxz+0Ci2GEIo3N0NX+Um3HC+/dA3dzgB ftvYA4RmWba9ETTxOG/hRMoze5dar3IZKjWdv0aJWAkQiE4Nnsb0VlhA+XTyz2wXLGi8 ahQ5bUsyo0LSJhdPSwqAj3QENlj5dTVb9JiQOZi9C/pZQ95v5jBxXUnDtd499XdUKJVp nloelnpoKIl4Q1HxbdQvL3UF2rlCQlJZn38aSzz6uvr8oHeECidB6si5VpIhhQQwWhyN zAYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qApN8UnhyEUIjIpM/GmsOjj23Il/VJ1qkJLR9SYk3QA=; b=TVrlVkyvUwv/N11/28Vd4CD86OX6hpu1LLVi+4G7ScTTPVxbzZp/iSMSj9jCuCkhHK 1oZ6qz0Sm6XD6W5NI5qvTv4cY4swCgOtFvw462TbgyRsTYSoIcimSBU3kBgoB2RKXK9l Ab2ZieC/huxpm/4JspcumJ+Sz2McDCRmXMZ3GVfAJNa8o3BUhuQ1oxARGArJ+9LdwAyZ 9HBYKT6M/n3VuUaPP9BC5uM3nJdSFZm5nl/aqK7Jw4+rx3vj0CWo/AWVzbyRW68BdgKn bzQ7MESdeg3KS4xjyxy81dhRLqYIV9oXhBon+JxclcZI1rilRoJbFj+8WF4leSzKR7l4 T08g== X-Gm-Message-State: AOAM5337NOrG9ekMOCCTlgbFqRg78yZ2wSMf6YjUNFJ8LDilvQ7xnc6v 4BBZX9S+JTznJMQF1VRE5EodFRSBRQ4Xhg== X-Google-Smtp-Source: ABdhPJxRUEnjrmqA3sEDUzjiVt7FqhRH0ZT7YSyDkytZ9kP5XOO8wITjXOZMmHKBwWLm/35jUDnetw== X-Received: by 2002:a05:6808:bcb:: with SMTP id o11mr5128868oik.168.1633485505033; Tue, 05 Oct 2021 18:58:25 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-2b16-c5f3-6fcc-9065.res6.spectrum.com. [2603:8081:140c:1a00:2b16:c5f3:6fcc:9065]) by smtp.gmail.com with ESMTPSA id e2sm4016057otk.46.2021.10.05.18.58.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Oct 2021 18:58:24 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v5 4/6] RDMA/rxe: Replace ah->pd by ah->ibah.pd Date: Tue, 5 Oct 2021 20:58:13 -0500 Message-Id: <20211006015815.28350-5-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211006015815.28350-1-rpearsonhpe@gmail.com> References: <20211006015815.28350-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The pd field in struct rxe_ah is redundant with the pd field in the rdma-core's ib_ah. Eliminate the pd field in rxe_ah and add an inline to extract the pd from the ibah field. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_verbs.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 9cd203f1fa22..881a5159a7d0 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -46,7 +46,6 @@ struct rxe_pd { struct rxe_ah { struct ib_ah ibah; struct rxe_pool_entry pelem; - struct rxe_pd *pd; struct rxe_av av; bool is_user; int ah_num; @@ -471,6 +470,11 @@ static inline struct rxe_mw *to_rmw(struct ib_mw *mw) return mw ? container_of(mw, struct rxe_mw, ibmw) : NULL; } +static inline struct rxe_pd *rxe_ah_pd(struct rxe_ah *ah) +{ + return to_rpd(ah->ibah.pd); +} + static inline struct rxe_pd *mr_pd(struct rxe_mr *mr) { return to_rpd(mr->ibmr.pd); From patchwork Wed Oct 6 01:58:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12538351 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5ACDC433F5 for ; Wed, 6 Oct 2021 01:58:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CB4886113A for ; Wed, 6 Oct 2021 01:58:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237102AbhJFCAR (ORCPT ); Tue, 5 Oct 2021 22:00:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237148AbhJFCAR (ORCPT ); Tue, 5 Oct 2021 22:00:17 -0400 Received: from mail-oi1-x232.google.com (mail-oi1-x232.google.com [IPv6:2607:f8b0:4864:20::232]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49B26C061753 for ; Tue, 5 Oct 2021 18:58:26 -0700 (PDT) Received: by mail-oi1-x232.google.com with SMTP id a3so1880227oid.6 for ; Tue, 05 Oct 2021 18:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R28ci1vfu6y2BAkpjnPtPdKQKjUHlIJGx5SO1oi5Sfc=; b=jqHCOHuYALL+hOLkAqQKzqjyD+iQDZrgpiaXPnhVntAT7zx+tfySr+5PVn5G2o/iX+ ktiarojogUFg37hcyAeN/zLEy0XFMpPyAepMCT3R2j5yLdj9VJZFf59xTX1oanPV/1gd rgK/jAevT7/NSdLO7tAof7fVn8S1dpD8rkvJ3jOnsLW9pD2N71lSSlIurmHgTQanHAj1 lRmBqkPEzo2C/Vt3y1dExb2oQ10Y+o/KAE6jYlGmZKhCnNi2UFqVsoEWA+sVC7nqcNnM +wfB/9jR733yi1DEyDWcsKEYTB491xgg9OXtB04AbIs+yFldOLhAQgCFTghBVJ+WtskK aLyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R28ci1vfu6y2BAkpjnPtPdKQKjUHlIJGx5SO1oi5Sfc=; b=ocfjuM+wrB4wDLiL/H/KRhGWhP55YwbCunmNkMp3K1xjBdHXdw9eumuRCsMQmZfXw+ yt4Z8XpQsJHJVWylajEpd2RgBdfRIU8yiBthv35OxWjlLLLI6ubKkd69RLvuJpsvP9pK 1sGI9gCj1vl194WlGCrcEbbfadroWKt4N2LjQBQZyZ1TAO6IDHEGsp5n+iAKb9UYZhB4 8AdMoOGQEsJory6BZ4ONJ7cO21sQdqoUI9EqtuZ2DJQwlWepTt9VcTT00dHpdm0xq2cz c7+Iktf2KKMheyzVpFlpolIKaGYJbW102uTxSIJvmIgnuGV9enyMNx8RmsSLL7pLTz+d 1sFQ== X-Gm-Message-State: AOAM533gYIHn3tlduyzBJAgq4oISzFnXWE0S9bnMbQpUK2eRZJ0MsuF9 FAy2OlQAn2RdLWutYi8GpSI= X-Google-Smtp-Source: ABdhPJxtROrsVM5K41FDCcPC6KxZsRWRT4lAN0YnWV2geK820cPJvgaZlvnF+uYifB3CpJuaXB70bA== X-Received: by 2002:a05:6808:1441:: with SMTP id x1mr5135455oiv.77.1633485505727; Tue, 05 Oct 2021 18:58:25 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-2b16-c5f3-6fcc-9065.res6.spectrum.com. [2603:8081:140c:1a00:2b16:c5f3:6fcc:9065]) by smtp.gmail.com with ESMTPSA id e2sm4016057otk.46.2021.10.05.18.58.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Oct 2021 18:58:25 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v5 5/6] RDMA/rxe: Lookup kernel AH from ah index in UD WQEs Date: Tue, 5 Oct 2021 20:58:14 -0500 Message-Id: <20211006015815.28350-6-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211006015815.28350-1-rpearsonhpe@gmail.com> References: <20211006015815.28350-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add code to rxe_get_av in rxe_av.c to use the AH index in UD send WQEs to lookup the kernel AH. For old user providers continue to use the AV passed in WQEs. Move setting pkt->rxe to before the call to rxe_get_av() to get access to the AH pool. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_av.c | 20 +++++++++++++++++++- drivers/infiniband/sw/rxe/rxe_req.c | 8 +++++--- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_av.c b/drivers/infiniband/sw/rxe/rxe_av.c index 85580ea5eed0..38c7b6fb39d7 100644 --- a/drivers/infiniband/sw/rxe/rxe_av.c +++ b/drivers/infiniband/sw/rxe/rxe_av.c @@ -101,11 +101,29 @@ void rxe_av_fill_ip_info(struct rxe_av *av, struct rdma_ah_attr *attr) struct rxe_av *rxe_get_av(struct rxe_pkt_info *pkt) { + struct rxe_ah *ah; + u32 ah_num; + if (!pkt || !pkt->qp) return NULL; if (qp_type(pkt->qp) == IB_QPT_RC || qp_type(pkt->qp) == IB_QPT_UC) return &pkt->qp->pri_av; - return (pkt->wqe) ? &pkt->wqe->wr.wr.ud.av : NULL; + if (!pkt->wqe) + return NULL; + + ah_num = pkt->wqe->wr.wr.ud.ah_num; + if (ah_num) { + /* only new user provider or kernel client */ + ah = rxe_pool_get_index(&pkt->rxe->ah_pool, ah_num); + if (!ah || ah->ah_num != ah_num || rxe_ah_pd(ah) != pkt->qp->pd) { + pr_warn("Unable to find AH matching ah_num\n"); + return NULL; + } + return &ah->av; + } + + /* only old user provider for UD sends*/ + return &pkt->wqe->wr.wr.ud.av; } diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index fe275fcaffbd..0c9d2af15f3d 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -379,9 +379,8 @@ static struct sk_buff *init_req_packet(struct rxe_qp *qp, /* length from start of bth to end of icrc */ paylen = rxe_opcode[opcode].length + payload + pad + RXE_ICRC_SIZE; - /* pkt->hdr, rxe, port_num and mask are initialized in ifc - * layer - */ + /* pkt->hdr, port_num and mask are initialized in ifc layer */ + pkt->rxe = rxe; pkt->opcode = opcode; pkt->qp = qp; pkt->psn = qp->req.psn; @@ -391,6 +390,9 @@ static struct sk_buff *init_req_packet(struct rxe_qp *qp, /* init skb */ av = rxe_get_av(pkt); + if (!av) + return NULL; + skb = rxe_init_packet(rxe, av, paylen, pkt); if (unlikely(!skb)) return NULL; From patchwork Wed Oct 6 01:58:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12538353 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74BF5C43217 for ; Wed, 6 Oct 2021 01:58:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5BE33610CE for ; Wed, 6 Oct 2021 01:58:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237163AbhJFCAS (ORCPT ); Tue, 5 Oct 2021 22:00:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237158AbhJFCAS (ORCPT ); Tue, 5 Oct 2021 22:00:18 -0400 Received: from mail-oi1-x236.google.com (mail-oi1-x236.google.com [IPv6:2607:f8b0:4864:20::236]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCACDC061753 for ; Tue, 5 Oct 2021 18:58:26 -0700 (PDT) Received: by mail-oi1-x236.google.com with SMTP id w206so1894560oiw.4 for ; Tue, 05 Oct 2021 18:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Fo+XA5ZhNhrA0pZdOhXylGStZB1nDPPpygrSUz+ZKz8=; b=lZ7BIm/HaWfMSmBZRtKPQDn6e2J43PHrSPtaiqnwFrhGOuRzZcYnDu+UnXVoKi2wcT TLZSx2ebkyYx3Q+JeK2yQytCck1RnXYws/tkd3CsVCGM1MT1ktFFwm477J21ijMbCFFo nVLOYCGn9nmuc0nuP1Mb/PsS4r72n56ksxIjG2W6Eav9AcnRjnIB09O9142h47uEFhz7 F7bD/cUT27dTqgYklzbSTrncDV0Ym3Fi1zgi6YNaTQy+kGMAYrTJLINgeqEoufJG8EJv KBJzZtYwz3oqyfupZEuntlVXnCZyhCsrrNvXSvUsPLYyvh6D1HfkiRPNoIVxWTdPO1CZ yz5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Fo+XA5ZhNhrA0pZdOhXylGStZB1nDPPpygrSUz+ZKz8=; b=TufovchYGe7LUNKU6oJaR+1+sqKMIvpY2uj/gwbXVnoqCOjnmoOJMktrfA4b2OlLe1 RaH3ACyd0AzktQ34TFU8X6f8RCwqLnYwByYofL0lIvmVkAIReRdkSe8L3P6HMRzoMDWK MfRt84QfSg88YyRvVQFBM+O2NBTk44dwz6I6KIreQpxvrrXOurlYI+LGmyx8f5NyqUTC WESyUheoGxJjzA+t1oKNZlgxF1+SSWU0ObDyxdUGRQLyClSXNriRi7clx049PErGC+0t JVJz/H6n316pKyTDVPfPoFG06L1ToMBEoY41HBrBu+aKXk2MzPhP2zJd2qlQE65NdpTw Q7GA== X-Gm-Message-State: AOAM533xxRL8qNXnWOBU/Uu4riQahL0MDQ4Hgq63f1NY2gzsD+otEiRb 6U6gtyZc3cHc1HfQVEAmsHZeqh0yLtqCAQ== X-Google-Smtp-Source: ABdhPJyYthypV8SjEZAP574Z9fSWoJH66y06hI5yVuL5BBb+G1MTtQRT9mfqpJqG00AX02dqGpTD1g== X-Received: by 2002:aca:c6c9:: with SMTP id w192mr575945oif.116.1633485506275; Tue, 05 Oct 2021 18:58:26 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-2b16-c5f3-6fcc-9065.res6.spectrum.com. [2603:8081:140c:1a00:2b16:c5f3:6fcc:9065]) by smtp.gmail.com with ESMTPSA id e2sm4016057otk.46.2021.10.05.18.58.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Oct 2021 18:58:26 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v5 6/6] RDMA/rxe: Convert kernel UD post send to use ah_num Date: Tue, 5 Oct 2021 20:58:15 -0500 Message-Id: <20211006015815.28350-7-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211006015815.28350-1-rpearsonhpe@gmail.com> References: <20211006015815.28350-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Modify ib_post_send for kernel UD sends to put the AH index into the WQE instead of the address vector. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_verbs.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 8854ace63acd..b808777e2221 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -537,8 +537,11 @@ static void init_send_wr(struct rxe_qp *qp, struct rxe_send_wr *wr, if (qp_type(qp) == IB_QPT_UD || qp_type(qp) == IB_QPT_SMI || qp_type(qp) == IB_QPT_GSI) { + struct ib_ah *ibah = ud_wr(ibwr)->ah; + wr->wr.ud.remote_qpn = ud_wr(ibwr)->remote_qpn; wr->wr.ud.remote_qkey = ud_wr(ibwr)->remote_qkey; + wr->wr.ud.ah_num = to_rah(ibah)->ah_num; if (qp_type(qp) == IB_QPT_GSI) wr->wr.ud.pkey_index = ud_wr(ibwr)->pkey_index; if (wr->opcode == IB_WR_SEND_WITH_IMM) @@ -610,12 +613,6 @@ static void init_send_wqe(struct rxe_qp *qp, const struct ib_send_wr *ibwr, return; } - if (qp_type(qp) == IB_QPT_UD || - qp_type(qp) == IB_QPT_SMI || - qp_type(qp) == IB_QPT_GSI) - memcpy(&wqe->wr.wr.ud.av, &to_rah(ud_wr(ibwr)->ah)->av, - sizeof(struct rxe_av)); - if (unlikely(ibwr->send_flags & IB_SEND_INLINE)) copy_inline_data_to_wqe(wqe, ibwr); else