From patchwork Tue Jul 5 23:08:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 12907185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1308C43334 for ; Tue, 5 Jul 2022 23:08:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232470AbiGEXI1 (ORCPT ); Tue, 5 Jul 2022 19:08:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59184 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232479AbiGEXIZ (ORCPT ); Tue, 5 Jul 2022 19:08:25 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DEE4DFC for ; Tue, 5 Jul 2022 16:08:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657062504; x=1688598504; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cR+IU9F7d1rCQ/Sw9dd0AzU5qaIggNiiNV/lo8vAnU8=; b=LFfwkopTfJyjxo9MhM2P8VT6xRhDsgCn+VKzB+NNsqUS/QxTJzdeAZOu a88kGFl6qMM657hUwXTpSeDY5g2mDMjMETOd2dK0+dCFkVrrTw5mttfEd cwYW9D8fvk/M1FGgg04pfaJY6sVLNCHfBPFkVH9W5gMh8eK4WYh7urH8D JmRY5pWz9T2HkjaAQ0ZecNCBiI+bjDvQKeSIK1fEmlznUr893xvgHUyUW qIlik5w/p41/0tvz8wSakSbnrH3jwO0rxyNe4FFIbRFQ/JwZZqDmYXmQc BNmQUQVUEnhrIsk9Dudz9OjQ8SrDfXsq80lMO0ezjeTNW2XU7TgijNX/Z A==; X-IronPort-AV: E=McAfee;i="6400,9594,10399"; a="266588404" X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="266588404" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:24 -0700 X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="620047514" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.212.17.201]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:23 -0700 From: Shiraz Saleem To: jgg@nvidia.com Cc: linux-rdma@vger.kernel.org, Mustafa Ismail , Shiraz Saleem Subject: [PATCH for-next 1/7] RDMA/irdma: Add 2 level PBLE support for FMR Date: Tue, 5 Jul 2022 18:08:09 -0500 Message-Id: <20220705230815.265-2-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.35.2 In-Reply-To: <20220705230815.265-1-shiraz.saleem@intel.com> References: <20220705230815.265-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mustafa Ismail Level 2 Physical Buffer List Entry (PBLE) is currently not supported for Fast MRs which limits memory registrations to 256K pages. Adapt irdma_set_page and irdma_alloc_mr to allow for 2 level PBLEs. Signed-off-by: Mustafa Ismail Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/main.h | 2 +- drivers/infiniband/hw/irdma/verbs.c | 14 +++++++++++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/irdma/main.h b/drivers/infiniband/hw/irdma/main.h index ef862bc..65e966a 100644 --- a/drivers/infiniband/hw/irdma/main.h +++ b/drivers/infiniband/hw/irdma/main.h @@ -85,7 +85,7 @@ #define IRDMA_NO_QSET 0xffff #define IW_CFG_FPM_QP_COUNT 32768 -#define IRDMA_MAX_PAGES_PER_FMR 512 +#define IRDMA_MAX_PAGES_PER_FMR 262144 #define IRDMA_MIN_PAGES_PER_FMR 1 #define IRDMA_CQP_COMPL_RQ_WQE_FLUSHED 2 #define IRDMA_CQP_COMPL_SQ_WQE_FLUSHED 3 diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c index c4412ec..0f5856b 100644 --- a/drivers/infiniband/hw/irdma/verbs.c +++ b/drivers/infiniband/hw/irdma/verbs.c @@ -2605,7 +2605,7 @@ static struct ib_mr *irdma_alloc_mr(struct ib_pd *pd, enum ib_mr_type mr_type, palloc = &iwpbl->pble_alloc; iwmr->page_cnt = max_num_sg; err_code = irdma_get_pble(iwdev->rf->pble_rsrc, palloc, iwmr->page_cnt, - true); + false); if (err_code) goto err_get_pble; @@ -2641,8 +2641,16 @@ static int irdma_set_page(struct ib_mr *ibmr, u64 addr) if (unlikely(iwmr->npages == iwmr->page_cnt)) return -ENOMEM; - pbl = palloc->level1.addr; - pbl[iwmr->npages++] = addr; + if (palloc->level == PBLE_LEVEL_2) { + struct irdma_pble_info *palloc_info = + palloc->level2.leaf + (iwmr->npages >> PBLE_512_SHIFT); + + palloc_info->addr[iwmr->npages & (PBLE_PER_PAGE - 1)] = addr; + } else { + pbl = palloc->level1.addr; + pbl[iwmr->npages] = addr; + } + iwmr->npages++; return 0; } From patchwork Tue Jul 5 23:08:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 12907186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65FB5CCA47C for ; Tue, 5 Jul 2022 23:08:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232270AbiGEXI2 (ORCPT ); Tue, 5 Jul 2022 19:08:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232490AbiGEXIZ (ORCPT ); Tue, 5 Jul 2022 19:08:25 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D95510D9 for ; Tue, 5 Jul 2022 16:08:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657062505; x=1688598505; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Tztno2/cWvplr0h8cF7pth6xAlEiRhzVk91Lq2CJMCo=; b=W+L0RV9zRMcNT2Wycf/j4FTnzP7pT958hu4c6dxdL4901mjmL9ujXINy FDoBo8h+dKRvcV1llw06YKmbR6VGEauCS763lixQ7G4UkoytRkmUUIOH9 as2C2yc/YCaEn+Y+4/CLJU2XeG7e8QEk3zSuk6h55D0eTG5tPq/+rGBS9 VwgbN8jZ3sWKDVU/WD0EugjXgnh38xUXfIUSNMNZjl5shORqwevkHshqO i8Lxo9FLC20ccH1cQmjDtcF2za7r//E8tFyAEiP6/pfqrD2PCowE266Qi /epdHWmVwRz3UYZhhvQF03J2JvJLM7nY8Cud9wSU8PpxlhqOwhg+GX5AI A==; X-IronPort-AV: E=McAfee;i="6400,9594,10399"; a="266588410" X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="266588410" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:24 -0700 X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="620047521" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.212.17.201]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:24 -0700 From: Shiraz Saleem To: jgg@nvidia.com Cc: linux-rdma@vger.kernel.org, Mustafa Ismail , Shiraz Saleem Subject: [PATCH for-next 2/7] RDMA/irdma: Add AE source to error log Date: Tue, 5 Jul 2022 18:08:10 -0500 Message-Id: <20220705230815.265-3-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.35.2 In-Reply-To: <20220705230815.265-1-shiraz.saleem@intel.com> References: <20220705230815.265-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mustafa Ismail To assist with debugging add the Asynchronous Event (AE) source when logging the abnormal AE error log message. Signed-off-by: Mustafa Ismail Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/hw.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c index dd3943d..bb60431 100644 --- a/drivers/infiniband/hw/irdma/hw.c +++ b/drivers/infiniband/hw/irdma/hw.c @@ -376,8 +376,8 @@ static void irdma_process_aeq(struct irdma_pci_f *rf) ctx_info->iwarp_info->err_rq_idx_valid = false; fallthrough; default: - ibdev_err(&iwdev->ibdev, "abnormal ae_id = 0x%x bool qp=%d qp_id = %d\n", - info->ae_id, info->qp, info->qp_cq_id); + ibdev_err(&iwdev->ibdev, "abnormal ae_id = 0x%x bool qp=%d qp_id = %d, ae_src=%d\n", + info->ae_id, info->qp, info->qp_cq_id, info->ae_src); if (rdma_protocol_roce(&iwdev->ibdev, 1)) { if (!info->sq && ctx_info->roce_info->err_rq_idx_valid) { ctx_info->roce_info->err_rq_idx = info->wqe_idx; From patchwork Tue Jul 5 23:08:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 12907187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B5A0CCA47B for ; Tue, 5 Jul 2022 23:08:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232195AbiGEXI3 (ORCPT ); Tue, 5 Jul 2022 19:08:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59218 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232479AbiGEXI2 (ORCPT ); Tue, 5 Jul 2022 19:08:28 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8C53BB0 for ; Tue, 5 Jul 2022 16:08:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657062505; x=1688598505; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=I5uyoWXeZav+Uzlnbhnn+RbzKUufhV7T8lCW28/W/o0=; b=YJP7EEnZ0seAAL+5++Aqb9gYgWf1Fuspv4f+kaAahRfnotWNwvqKsaH3 kLqpzOc5qXZfdwUVJVOE5IokAYqEkia+fq0SBGDrGlHLp3CCBcKm6Ofv0 J2au7JfxGw9PqKyndPu8LIIF74hoNAhiW3nLYQA0QQHkemzMk0Da7WPh/ WR6M97hkwmwcsFhhENJePAkpuWIE9HLnrGgHwfOn74pTpIxjgzliQnNdX EQVrkMq6Z5x9vyWy+cEkaPAL5tLtmjwYe5zHDrCRc7dGW5DIQnqvQ5cgo 014BASDGNM6xC6BO218RHiQNV9hDrjSY76daBgq6/HZwU+0kdrpNshPUB w==; X-IronPort-AV: E=McAfee;i="6400,9594,10399"; a="266588415" X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="266588415" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:25 -0700 X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="620047529" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.212.17.201]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:24 -0700 From: Shiraz Saleem To: jgg@nvidia.com Cc: linux-rdma@vger.kernel.org, Mustafa Ismail , Shiraz Saleem Subject: [PATCH for-next 3/7] RDMA/irdma: Make CQP invalid state error non-critical Date: Tue, 5 Jul 2022 18:08:11 -0500 Message-Id: <20220705230815.265-4-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.35.2 In-Reply-To: <20220705230815.265-1-shiraz.saleem@intel.com> References: <20220705230815.265-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mustafa Ismail The invalid state error returned by the Control Queue-Pair (CQP) is not a critical error. Add it to the irdma_noncrit_err_list and drop reporting it as device error message. Signed-off-by: Mustafa Ismail Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/utils.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c index ab3c520..fdf4cc8 100644 --- a/drivers/infiniband/hw/irdma/utils.c +++ b/drivers/infiniband/hw/irdma/utils.c @@ -652,6 +652,7 @@ static int irdma_wait_event(struct irdma_pci_f *rf, }; static const struct irdma_cqp_err_info irdma_noncrit_err_list[] = { + {0xffff, 0x8002, "Invalid State"}, {0xffff, 0x8006, "Flush No Wqe Pending"}, {0xffff, 0x8007, "Modify QP Bad Close"}, {0xffff, 0x8009, "LLP Closed"}, From patchwork Tue Jul 5 23:08:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 12907189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F685C43334 for ; Tue, 5 Jul 2022 23:08:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229839AbiGEXIa (ORCPT ); Tue, 5 Jul 2022 19:08:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232689AbiGEXI2 (ORCPT ); Tue, 5 Jul 2022 19:08:28 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50DF15FBF for ; Tue, 5 Jul 2022 16:08:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657062506; x=1688598506; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=btFaKB8EN9a59bp0n0+FTL8jAf4Y1q9R+oHzFVj8WC0=; b=FV/dRS8cjILXxe4oXwfRKl89bL1Qr8XyUhICuDvjGj5h8Iiua1KN2Zcn YMKLG3o6CvAyNGyuQBMJKUYH8FIc/7Ya5vDZ/DxATszUru97xqwvnrcab CaShoncCKUdUMGBp2FPO1JNLFHLS8Qoz95dAIdlpXAw9WTUAHRkFCh5dp OXn9PCcpjTiHKDD79lyG1ro4aTHBq2v0NcpxQoKwgzuGvyfRJRNXwOrjT 8A4tNQC2tCqv8Ewcnq21rbUWtgrbl0EIyzlXt9yhxgWBf7Fk2wwVII48e gHZknxD9O/G7I/HDXgNgtMF0ZCEiU5voIPYe7Y3TCckAUZPEvLz4FWw+k A==; X-IronPort-AV: E=McAfee;i="6400,9594,10399"; a="266588420" X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="266588420" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:26 -0700 X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="620047537" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.212.17.201]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:25 -0700 From: Shiraz Saleem To: jgg@nvidia.com Cc: linux-rdma@vger.kernel.org, Nayan Kumar , Shiraz Saleem Subject: [PATCH for-next 4/7] RDMA/irdma: Make resource distribution algorithm more QP oriented Date: Tue, 5 Jul 2022 18:08:12 -0500 Message-Id: <20220705230815.265-5-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.35.2 In-Reply-To: <20220705230815.265-1-shiraz.saleem@intel.com> References: <20220705230815.265-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Nayan Kumar Adapt the resource distribution algorithm in irdma_cfg_fpm_val to be more QP oriented. If the configuration is too big for the available memory, trim the MR and PBLE's first before trimming the QPs. This also avoids having to double QPs requested as input to algorithm for GEN1 devices. Signed-off-by: Nayan Kumar Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/ctrl.c | 8 +++++--- drivers/infiniband/hw/irdma/hw.c | 5 +---- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/hw/irdma/ctrl.c b/drivers/infiniband/hw/irdma/ctrl.c index 58c0e18..a41e0d2 100644 --- a/drivers/infiniband/hw/irdma/ctrl.c +++ b/drivers/infiniband/hw/irdma/ctrl.c @@ -4872,10 +4872,12 @@ int irdma_cfg_fpm_val(struct irdma_sc_dev *dev, u32 qp_count) sd_diff = sd_needed - hmc_fpm_misc->max_sds; if (sd_diff > 128) { - if (qpwanted > 128 && sd_diff > 144) + if (!(loop_count % 2) && qpwanted > 128) { qpwanted /= 2; - mrwanted /= 2; - pblewanted /= 2; + } else { + mrwanted /= 2; + pblewanted /= 2; + } continue; } if (dev->cqp->hmc_profile != IRDMA_HMC_PROFILE_FAVOR_VF && diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c index bb60431..8abbd50 100644 --- a/drivers/infiniband/hw/irdma/hw.c +++ b/drivers/infiniband/hw/irdma/hw.c @@ -1512,10 +1512,7 @@ static int irdma_hmc_setup(struct irdma_pci_f *rf) int status; u32 qpcnt; - if (rf->rdma_ver == IRDMA_GEN_1) - qpcnt = rsrc_limits_table[rf->limits_sel].qplimit * 2; - else - qpcnt = rsrc_limits_table[rf->limits_sel].qplimit; + qpcnt = rsrc_limits_table[rf->limits_sel].qplimit; rf->sd_type = IRDMA_SD_TYPE_DIRECT; status = irdma_cfg_fpm_val(&rf->sc_dev, qpcnt); From patchwork Tue Jul 5 23:08:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 12907188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9FF1C433EF for ; Tue, 5 Jul 2022 23:08:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229505AbiGEXIa (ORCPT ); Tue, 5 Jul 2022 19:08:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232650AbiGEXI2 (ORCPT ); Tue, 5 Jul 2022 19:08:28 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFF3FD85 for ; Tue, 5 Jul 2022 16:08:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657062506; x=1688598506; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a+A8MHDt5TTVXRjklmppYCO9Tycs5HYBURRPwkyuyA8=; b=CPm/OEEl8mvQgW4SjkO7P+ZzA3QgzTU8R2+5xABrTzfVqkm7MotnbQ4Z s0HgkvJeyYVQsCbXz+JDY9Wg+p9a9df97NFE1kkY8dvPmKh/wZ/yZKz3K +/Hbbx8He/QFjyejKRFBjLpuH6rfXqQY3lZqN+VhtqwvHyUI5V9/CV7c8 VZpKtdSaI2nlAuTTe+W2AxYN908OwPg/OMVey84HtGxctRmElOgeGR/3K JIQbAgM4hPfECycCjFlgq/ToSCWoDyNTIprreQp7vzIMXNBIOBIdmUaNA z4rcV/QKcyXJUvwnB3FG8N4LyuhrYvGsnX4uV/EUWnxayUO+ufokO4uFW g==; X-IronPort-AV: E=McAfee;i="6400,9594,10399"; a="266588423" X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="266588423" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:26 -0700 X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="620047547" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.212.17.201]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:26 -0700 From: Shiraz Saleem To: jgg@nvidia.com Cc: linux-rdma@vger.kernel.org, Mustafa Ismail , Shiraz Saleem Subject: [PATCH for-next 5/7] RDMA/irdma: Fix a window for use-after-free Date: Tue, 5 Jul 2022 18:08:13 -0500 Message-Id: <20220705230815.265-6-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.35.2 In-Reply-To: <20220705230815.265-1-shiraz.saleem@intel.com> References: <20220705230815.265-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mustafa Ismail During a destroy CQ an interrupt may cause processing of a CQE after CQ resources are freed by irdma_cq_free_rsrc(). Fix this by moving the call to irdma_cq_free_rsrc() after the irdma_sc_cleanup_ceqes(), which is called under the cq_lock. Fixes: b48c24c2d710 ("RDMA/irdma: Implement device supported verb APIs") Signed-off-by: Bartosz Sobczak Signed-off-by: Mustafa Ismail Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/verbs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c index 0f5856b..d78b11a 100644 --- a/drivers/infiniband/hw/irdma/verbs.c +++ b/drivers/infiniband/hw/irdma/verbs.c @@ -1776,11 +1776,11 @@ static int irdma_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata) spin_unlock_irqrestore(&iwcq->lock, flags); irdma_cq_wq_destroy(iwdev->rf, cq); - irdma_cq_free_rsrc(iwdev->rf, iwcq); spin_lock_irqsave(&iwceq->ce_lock, flags); irdma_sc_cleanup_ceqes(cq, ceq); spin_unlock_irqrestore(&iwceq->ce_lock, flags); + irdma_cq_free_rsrc(iwdev->rf, iwcq); return 0; } From patchwork Tue Jul 5 23:08:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 12907190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B417C43334 for ; Tue, 5 Jul 2022 23:08:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230425AbiGEXIb (ORCPT ); Tue, 5 Jul 2022 19:08:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232746AbiGEXI2 (ORCPT ); Tue, 5 Jul 2022 19:08:28 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A152460FB for ; Tue, 5 Jul 2022 16:08:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657062507; x=1688598507; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xeJ2ovW/sOD29dV2jOp4Qqo8sM/PjKO9uWsriEwhLfg=; b=aAFDhgMPuSuJkgdOLA5OYJ56mAYJjLeLg2Bmkn0N4GgVXMBES626SR0Z HsU4D0pSLxytXaMzraofHfsH0g2TtqmXIzs9srq+XtJM1yhTsIaQpISb7 wfVXVVpjYbj2K4XVWav7lNLnn0DjBA55ndO+ZtMLbHRE5h/kVXmluKDMx lqEWx8yFEH+fOhp8r2k1e7qqm8bOjXKdO/61mRWJE1liDVH1m0jBv1Cuj vT/p7I1VD9gMp8rhYoq0BWTNopMtRTmWgAG/tusEmZ+Uahwos5UStLEFR RkS9Lff2paf3xwj1U8C2CIxkS1ardRAfCmoStpyYQznSJldWMnKPacUSY w==; X-IronPort-AV: E=McAfee;i="6400,9594,10399"; a="266588434" X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="266588434" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:27 -0700 X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="620047556" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.212.17.201]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:26 -0700 From: Shiraz Saleem To: jgg@nvidia.com Cc: linux-rdma@vger.kernel.org, Mustafa Ismail , Shiraz Saleem Subject: [PATCH for-next 6/7] RDMA/irdma: Fix VLAN connection with wildcard address Date: Tue, 5 Jul 2022 18:08:14 -0500 Message-Id: <20220705230815.265-7-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.35.2 In-Reply-To: <20220705230815.265-1-shiraz.saleem@intel.com> References: <20220705230815.265-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mustafa Ismail When an application listens on a wildcard address, and there are VLAN and non-VLAN IP addresses, iWARP connection establishemnt can fail if the listen node VLAN ID does not match. Fix this by checking the vlan_id only if not a wildcard listen node. Fixes: 146b9756f14c ("RDMA/irdma: Add connection manager") Signed-off-by: Mustafa Ismail Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/cm.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/hw/irdma/cm.c b/drivers/infiniband/hw/irdma/cm.c index 638bf4a..e4d837f 100644 --- a/drivers/infiniband/hw/irdma/cm.c +++ b/drivers/infiniband/hw/irdma/cm.c @@ -1477,12 +1477,13 @@ static int irdma_send_fin(struct irdma_cm_node *cm_node) list_for_each_entry (listen_node, &cm_core->listen_list, list) { memcpy(listen_addr, listen_node->loc_addr, sizeof(listen_addr)); listen_port = listen_node->loc_port; + if (listen_port != dst_port || + !(listener_state & listen_node->listener_state)) + continue; /* compare node pair, return node handle if a match */ - if ((!memcmp(listen_addr, dst_addr, sizeof(listen_addr)) || - !memcmp(listen_addr, ip_zero, sizeof(listen_addr))) && - listen_port == dst_port && - vlan_id == listen_node->vlan_id && - (listener_state & listen_node->listener_state)) { + if (!memcmp(listen_addr, ip_zero, sizeof(listen_addr)) || + (!memcmp(listen_addr, dst_addr, sizeof(listen_addr)) && + vlan_id == listen_node->vlan_id)) { refcount_inc(&listen_node->refcnt); spin_unlock_irqrestore(&cm_core->listen_list_lock, flags); From patchwork Tue Jul 5 23:08:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiraz Saleem X-Patchwork-Id: 12907191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD1DFC433EF for ; Tue, 5 Jul 2022 23:08:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232479AbiGEXIc (ORCPT ); Tue, 5 Jul 2022 19:08:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232762AbiGEXI3 (ORCPT ); Tue, 5 Jul 2022 19:08:29 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6324F1007 for ; Tue, 5 Jul 2022 16:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657062508; x=1688598508; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HsX13ASJ1D/BNbI3VCXTybuv+UArrS0tThSSv1xTcBQ=; b=OgGO2XYCbKuvoGOvpv5YX1W5IE++KtmJQ83Z3Y4Z5Q2zUryoHhDqICiZ UKX8n3JWKxBHwKD4/gkCdY0m0FehBnijldDqSxMfEWglFWduq3ReWZfkn jNfy7I16SLPtSWbEZ65kieXaG9j1SYvblMyALeGZL+yx3ASPuAR5TgZZy N3PRhmTuaL9SOz7shfvue70Mp+fbxJjrrdAZ8XwbpIH87VTe3v0tiDpET rYDhngoMhIE9ApW9u3nx9gsfEaeB1kTtyRUkZWDTE908aq470bLatOzA+ X0qfm8cgadthwng3pHP5xfTSbKlPfh0yn2+OOazjGr1fjXfXPOd2/d2MN w==; X-IronPort-AV: E=McAfee;i="6400,9594,10399"; a="266588439" X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="266588439" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:28 -0700 X-IronPort-AV: E=Sophos;i="5.92,248,1650956400"; d="scan'208";a="620047564" Received: from ssaleem-mobl1.amr.corp.intel.com ([10.212.17.201]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2022 16:08:27 -0700 From: Shiraz Saleem To: jgg@nvidia.com Cc: linux-rdma@vger.kernel.org, Mustafa Ismail , Shiraz Saleem Subject: [PATCH for-next 7/7] RDMA/irdma: Fix setting of QP context err_rq_idx_valid field Date: Tue, 5 Jul 2022 18:08:15 -0500 Message-Id: <20220705230815.265-8-shiraz.saleem@intel.com> X-Mailer: git-send-email 2.35.2 In-Reply-To: <20220705230815.265-1-shiraz.saleem@intel.com> References: <20220705230815.265-1-shiraz.saleem@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mustafa Ismail Setting err_rq_idx_valid field in QP context when the AE source of the AEQE is not associated with an RQ causes the firmware flush to fail. Set err_rq_idx_valid field in QP context only if it is associated with an RQ. Additionally, cleanup the redundant setting of this field in irdma_process_aeq. Fixes: 44d9e52977a1 ("RDMA/irdma: Implement device initialization definitions") Signed-off-by: Mustafa Ismail Signed-off-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/hw.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c index 8abbd50..e041ed3 100644 --- a/drivers/infiniband/hw/irdma/hw.c +++ b/drivers/infiniband/hw/irdma/hw.c @@ -257,10 +257,6 @@ static void irdma_process_aeq(struct irdma_pci_f *rf) iwqp->last_aeq = info->ae_id; spin_unlock_irqrestore(&iwqp->lock, flags); ctx_info = &iwqp->ctx_info; - if (rdma_protocol_roce(&iwqp->iwdev->ibdev, 1)) - ctx_info->roce_info->err_rq_idx_valid = true; - else - ctx_info->iwarp_info->err_rq_idx_valid = true; } else { if (info->ae_id != IRDMA_AE_CQ_OPERATION_ERROR) continue; @@ -370,16 +366,12 @@ static void irdma_process_aeq(struct irdma_pci_f *rf) case IRDMA_AE_LCE_FUNCTION_CATASTROPHIC: case IRDMA_AE_LCE_CQ_CATASTROPHIC: case IRDMA_AE_UDA_XMIT_DGRAM_TOO_LONG: - if (rdma_protocol_roce(&iwdev->ibdev, 1)) - ctx_info->roce_info->err_rq_idx_valid = false; - else - ctx_info->iwarp_info->err_rq_idx_valid = false; - fallthrough; default: ibdev_err(&iwdev->ibdev, "abnormal ae_id = 0x%x bool qp=%d qp_id = %d, ae_src=%d\n", info->ae_id, info->qp, info->qp_cq_id, info->ae_src); if (rdma_protocol_roce(&iwdev->ibdev, 1)) { - if (!info->sq && ctx_info->roce_info->err_rq_idx_valid) { + ctx_info->roce_info->err_rq_idx_valid = info->rq; + if (info->rq) { ctx_info->roce_info->err_rq_idx = info->wqe_idx; irdma_sc_qp_setctx_roce(&iwqp->sc_qp, iwqp->host_ctx.va, ctx_info); @@ -388,7 +380,8 @@ static void irdma_process_aeq(struct irdma_pci_f *rf) irdma_cm_disconn(iwqp); break; } - if (!info->sq && ctx_info->iwarp_info->err_rq_idx_valid) { + ctx_info->iwarp_info->err_rq_idx_valid = info->rq; + if (info->rq) { ctx_info->iwarp_info->err_rq_idx = info->wqe_idx; ctx_info->tcp_info_valid = false; ctx_info->iwarp_info_valid = true;