From patchwork Thu Feb 20 18:34:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13984411 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8D4B262D2E for ; Thu, 20 Feb 2025 18:55:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077754; cv=none; b=tQdmxx7PBufa5Ts61l2i4DH78x+98/LWkeXfgXbTEmaHQ1cco6wWkdD2ZjgMDFQ/QWuWvoJEK95/Law4igA0fwrbS+mhKEuA0lQO4GqV/GwpLC5i/ZosDgtZgcnfKpNhROLMkqHPkcRhVM6x2A4wY6gqQoaGHSHiRHVkDWVXqZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077754; c=relaxed/simple; bh=DRyXIrtbadlAOTbSf4eaomN59p5ysVYTfj15XxsNg3g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=inM7eBGlbpJt8EwAGYc25REokbYscAixKtyGg/04/YQhD2ZnqS5SnJnIBop1UNPNP8CqURqZiJYmWy+JyX0KfQ8FaZOqnAJ5QHXGerzvR7x9gW39lwI1tgVMz7bCq2vTiXsbO6D0gVlabEh/Dw8tpPKF79w83Qm020t7LTJB2mU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=RZYB1PR6; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="RZYB1PR6" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-22114b800f7so24989315ad.2 for ; Thu, 20 Feb 2025 10:55:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1740077752; x=1740682552; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=10BDGIGdjI+2I+pAbbcrEY6QirgZ2x3nU1sHIH+QS5U=; b=RZYB1PR6ijthAtsMhmqlMzLE83mJx0DGZRroLbuD73yvXyF0mar2TdIGBlB3lx1ynh k1K0yiK5inm9H6NKUc0pBSdBBHPeDuRyCLh2dIauccOpSPbQrCkmmF99qO/KAfMLCjwh c8LSkOgvlmIiDQoOnX0eKXGyJBuZsTRgmOv3I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740077752; x=1740682552; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=10BDGIGdjI+2I+pAbbcrEY6QirgZ2x3nU1sHIH+QS5U=; b=ubrDjUqa2DOMJXsU1tFv4WpLop/SgeUHo3fTuRjRXwiyZP93b777tse2+l2xSZafS4 Om9yPqMS1Ym4fEAFAukT3JHPT4Ptrbf9YVSL5xcrUTuR7aESrTdMBIbcXga08KpE2omP Q37KjEZDg70xBvSVXg7FZRrYlhS+p9NeXF8XllEYULEjSuMrhkpbJwDrVExlC1j1tWoY Ick0I53IogxuR5ApU5XtgOA+znRTmaMo0t99fLcDEtZ++oz/bK9xhucS8Id0lvUHS1A7 bB6J65gH9rpqk/qJZBJzAPKOIgUn52UKecok38Oz45swt+DWeh4Kv5LP0gRmWXzPo22f 9XgA== X-Gm-Message-State: AOJu0Yw+AIOSKSwDRbG5GkAA4DVp5OvPy8q+CCo85ScJfp5Rp04/N9sI TqOJmJXRzG/WRbKf5yvCVKqhx5NmPMTmsarUFYjVN2A12pLcBVCpUCrIfbtwsQ== X-Gm-Gg: ASbGnctxPmYnvRKldIK9GLumuS7dCQmOPaIYgOmixtEw7Halt4um1ZRc7YV6DuP8yfZ bVpviGzx/m0l8wc73D2UIlBx/k37WgyM7I70jPwNN1339rifxNruZ9PO6/F+xq9DKaCNn/x6ezL PMLOlfxkqpwfUe2NawxfC7SkoGEyL1uCtRnVZcTP//NJe39PmAfUmIR9vk4TzsUpb3Rspx26YiB 1B1wjMERiMx0TDqYHAnxongeok1imNM7W2eSdT0oWJSs8hSmkJo4gQ6lonDNlzfMhAPLSIf3tkR 2cpLnKT5eVQ9D79ADczWufJXNSiVENtA2fWMmGfjvfZ4UK1AboG/CMatcktjlxGocfg/lOU= X-Google-Smtp-Source: AGHT+IFjAlHdQjg7jtPWPxhoIoMWi6AdSAqqq3IVW4vbmaqpvymP+c1FKAid6KJD2t7F+UXIzoE+KQ== X-Received: by 2002:a05:6a20:9184:b0:1ee:ce5b:853d with SMTP id adf61e73a8af0-1eef3dcc063mr381560637.39.1740077752042; Thu, 20 Feb 2025 10:55:52 -0800 (PST) Received: from sxavier-dev.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-addee79a984sm9572262a12.32.2025.02.20.10.55.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:55:51 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, kalesh-anakkur.purayil@broadcom.com, netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, abeni@redhat.com, horms@kernel.org, michael.chan@broadcom.com, Saravanan Vajravel , Selvin Xavier Subject: [PATCH rdma-next 1/9] RDMA/bnxt_re: Add support for collecting the Queue dumps Date: Thu, 20 Feb 2025 10:34:48 -0800 Message-Id: <1740076496-14227-2-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> References: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Saravanan Vajravel As part of enhancing the debug data collection, allocate few data structures to hold the resources after the queues are destroyed. Initialize the data structures to capture the data. By default, driver will cache the info of the QPs that are in error state. The dump levels can be changed from debugfs hook in a later patch. Driver caches the info of the last 1024 entries only. Signed-off-by: Saravanan Vajravel Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 64 +++++++++++++++++++++++++++++++++ drivers/infiniband/hw/bnxt_re/main.c | 18 ++++++++++ 2 files changed, 82 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index b33b04e..5818db1 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -173,6 +173,67 @@ static inline bool bnxt_re_chip_gen_p7(u16 chip_num) chip_num == CHIP_NUM_57608); } +#define BNXT_RE_MAX_QDUMP_ENTRIES 1024 + +struct qdump_qpinfo { + u32 id; + u32 dest_qpid; + u64 qp_handle; + u32 mtu; + u8 type; + u8 wqe_mode; + u8 state; + u8 is_user; + u64 scq_handle; + u64 rcq_handle; + u32 scq_id; + u32 rcq_id; +}; + +struct qdump_mrinfo { + int type; + u32 lkey; + u32 rkey; + u64 total_size; + u64 mr_handle; +}; + +struct qdump_element { + struct bnxt_qplib_pbl pbl[PBL_LVL_MAX]; + enum bnxt_qplib_pbl_lvl level; + struct bnxt_qplib_hwq *hwq; + struct bnxt_re_dev *rdev; + struct ib_umem *umem; + bool is_user_qp; + char des[32]; + char *buf; + size_t len; + u16 stride; + u32 prod; + u32 cons; +}; + +struct qdump_array { + struct qdump_qpinfo qpinfo; + struct qdump_mrinfo mrinfo; + bool valid; + bool is_mr; +}; + +struct bnxt_re_qdump_head { + struct qdump_array *qdump; + u32 max_elements; + struct mutex lock; /* lock qdump array elements */ + u32 index; +}; + +enum { + BNXT_RE_SNAPDUMP_NONE = 0, + BNXT_RE_SNAPDUMP_ERR, + /* Add new entry before this */ + BNXT_RE_SNAPDUMP_ALL +}; + struct bnxt_re_dev { struct ib_device ibdev; struct list_head list; @@ -232,6 +293,9 @@ struct bnxt_re_dev { unsigned long event_bitmap; struct bnxt_qplib_cc_param cc_param; struct workqueue_struct *dcb_wq; + /* Head to track all QP dump */ + struct bnxt_re_qdump_head qdump_head; + u8 snapdump_dbg_lvl; struct dentry *cc_config; struct bnxt_re_dbg_cc_config_params *cc_config_params; }; diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index e9e4da4..87fdf69 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -2008,6 +2008,11 @@ static void bnxt_re_free_nqr_mem(struct bnxt_re_dev *rdev) rdev->nqr = NULL; } +static void bnxt_re_clean_qdump(struct bnxt_re_dev *rdev) +{ + vfree(rdev->qdump_head.qdump); +} + static void bnxt_re_dev_uninit(struct bnxt_re_dev *rdev, u8 op_type) { u8 type; @@ -2018,6 +2023,7 @@ static void bnxt_re_dev_uninit(struct bnxt_re_dev *rdev, u8 op_type) bnxt_re_net_unregister_async_event(rdev); bnxt_re_uninit_dcb_wq(rdev); + bnxt_re_clean_qdump(rdev); if (test_and_clear_bit(BNXT_RE_FLAG_QOS_WORK_REG, &rdev->flags)) cancel_delayed_work_sync(&rdev->worker); @@ -2063,6 +2069,16 @@ static void bnxt_re_worker(struct work_struct *work) schedule_delayed_work(&rdev->worker, msecs_to_jiffies(30000)); } +static void bnxt_re_init_qdump(struct bnxt_re_dev *rdev) +{ + rdev->qdump_head.max_elements = BNXT_RE_MAX_QDUMP_ENTRIES; + rdev->qdump_head.index = 0; + rdev->snapdump_dbg_lvl = BNXT_RE_SNAPDUMP_ERR; + mutex_init(&rdev->qdump_head.lock); + rdev->qdump_head.qdump = vzalloc(rdev->qdump_head.max_elements * + sizeof(struct qdump_array)); +} + static int bnxt_re_dev_init(struct bnxt_re_dev *rdev, u8 op_type) { struct bnxt_re_ring_attr rattr = {}; @@ -2235,6 +2251,8 @@ static int bnxt_re_dev_init(struct bnxt_re_dev *rdev, u8 op_type) hash_init(rdev->srq_hash); bnxt_re_debugfs_add_pdev(rdev); + if (bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx)) + bnxt_re_init_qdump(rdev); bnxt_re_init_dcb_wq(rdev); bnxt_re_net_register_async_event(rdev); From patchwork Thu Feb 20 18:34:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13984412 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECB17264F8D for ; Thu, 20 Feb 2025 18:55:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077758; cv=none; b=oRcwtG7LCO7KE3h5AF+p2Gz033dXLSLgzbz1AESQ5WSM+crA29JMfYt2M2mVUpK/OXROAfmNuGt69V2YWzggkSZqRVDQRBSDGjDqYZfWgtQisha2vFkV6wzO3/58rpi0jW8o70zL6xqU86mgqr1fxVl7V5DD+2uBYymqJbqhkLg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077758; c=relaxed/simple; bh=KFiItcYA1+vfCd+hinIriGeUe9fKsH3FKL1ViQr6mGI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Uo+b6LLNWiQ5XyzvUDb1zj/xCv35jRTSI8DWFJUB0NXRgM6emGfSa2ROHG52cQlnw27hz19M4F3NsgL/jclnthD+R+S/oHMUjbwDXguD+M8MfMlJQH+N9Ss5/4nTthPjwaOEjsT5dtlqt2vPH/fxlPEL2zQ69uHcQgNu1pfIPjk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=InWgO/iL; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="InWgO/iL" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-220c665ef4cso21449715ad.3 for ; Thu, 20 Feb 2025 10:55:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1740077756; x=1740682556; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=Zr+FohAS+5s358/XL5DNb3kyfI1vHkhFvhvpgEYGLio=; b=InWgO/iLJemDlouldjP+zHXUSKGAXRN4owAQBVQXKHIwjYo8SjJ2pbZHfqVX2hO/4K v0BDTLPyCt5LqXfKGsg/TWPbIZeBtwuDPNJuTPyPGUXKbS97FkIjs6GkHZxpYpHxH/D9 E9SZu4MUxMjindphCFZgts8EPw4RFDujLyJZY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740077756; x=1740682556; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Zr+FohAS+5s358/XL5DNb3kyfI1vHkhFvhvpgEYGLio=; b=oL6xY93nCPlNAnKYTLxix7bQZ1kX1u9FBmtdwH0rKnWVfbdxs9En0fI0/K97zxM/kg heygGon4UAapsW5SJ9GJ+fT6B6wa1u+b5qaU4ZMSTeavStpLzADcIFpIoQk7P/m+YZud havb7uUwQyn+ohhP3ULQiwW2UP/a0nStXGgfYzbcC3HnczmaH16F3DBbebTA2xAlybvT lh+/moS4sLmxNc3yciDH4K1caD5wpWVYep1yzPwYfTQohPWX1sKF0JyEDvnizaZ4Vv3i E0j57pW3KOPVQiXkHdH6ZduISsFiK1MfXuP4tbddGGZbV1hdQvd1Vo4V4k0PunkIwzkf BLFQ== X-Gm-Message-State: AOJu0YxaL9EiSTWqelZR4FV9aY3jfIvOgMF6+6bohujKmaspYkJnvp5V nDj09h5AzhzDqihy9cW4iBP3yEXWxZH72sHAGFJiNFj3rRcUx3wD1fByv3wbKg== X-Gm-Gg: ASbGncs+7P5GdoZ7R3LfjqaNYPKnWXgMs+Y9ES/qJH8Fvwm6B+mUQ8uAvvZMWsot6om 4JR5AwDAj6STkzHbBxZxzNqotXmcgBNGhwP2WzwUt9F3zjFJR5dJ4ju98Vdo0eoNfZ30Sv25i52 VahpJPqwiTaP/ZMdrWZEsInG/TjOEDp14pBEFCf9Jf3pD76NZSwAe7iyhl/5oQ/pC84yDBq+sN/ ERz2NP9FrgtghtVBtM+ZDcHv4scuOGqVr9GRM4hlqMu2UBakBiWWJzimx8xdkyhARNCgRbuQThA m3LBCFfn9yuPnuGmccRh1sda8tqS/BLc6RKw1HjeB8vKGabydbqldOKVpYH9UaYR/bdKBlI= X-Google-Smtp-Source: AGHT+IG5H60oDdyi7JNgfkxGe8sLuWExbjYsWJu3/e2SJDRMel5HMa3BX5oHa9sFdtx0xPH3Zg+4fA== X-Received: by 2002:a05:6a20:9144:b0:1ee:e24d:8fdc with SMTP id adf61e73a8af0-1eef3c883f7mr379399637.10.1740077756102; Thu, 20 Feb 2025 10:55:56 -0800 (PST) Received: from sxavier-dev.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-addee79a984sm9572262a12.32.2025.02.20.10.55.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:55:55 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, kalesh-anakkur.purayil@broadcom.com, netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, abeni@redhat.com, horms@kernel.org, michael.chan@broadcom.com, Saravanan Vajravel , Selvin Xavier Subject: [PATCH rdma-next 2/9] RDMA/bnxt_re: Cache the QP information Date: Thu, 20 Feb 2025 10:34:49 -0800 Message-Id: <1740076496-14227-3-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> References: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Saravanan Vajravel Add routines to cache the information about the QP before destroying. This can be retrieved and used for further debugging. The default behavior is to capture the QPs that are moved to error. The driver sets the flags to capture the dump from the error asynchronous event. Signed-off-by: Saravanan Vajravel Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 67 ++++++++++++++++++++++++++++++++ drivers/infiniband/hw/bnxt_re/ib_verbs.h | 42 ++++++++++++++++++++ drivers/infiniband/hw/bnxt_re/main.c | 8 +++- drivers/infiniband/hw/bnxt_re/qplib_fp.h | 7 ++++ 4 files changed, 123 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index 2de101d..060143e 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -41,6 +41,7 @@ #include #include #include +#include #include #include @@ -954,6 +955,71 @@ static int bnxt_re_destroy_gsi_sqp(struct bnxt_re_qp *qp) return rc; } +static struct qdump_array *bnxt_re_get_next_qpdump(struct bnxt_re_dev *rdev) +{ + struct qdump_array *qdump; + u32 index; + + index = rdev->qdump_head.index; + qdump = &rdev->qdump_head.qdump[index]; + memset(qdump, 0, sizeof(*qdump)); + + index++; + index %= rdev->qdump_head.max_elements; + rdev->qdump_head.index = index; + + return qdump; +} + +/* + * bnxt_re_capture_qpdump - Capture snapshot of various queues of a QP. + * @qp - Pointer to QP for which data has to be collected + * + * This function will capture info about SQ/RQ/SCQ/RCQ of a QP which + * can be used to debug any issue + * + */ +void bnxt_re_capture_qpdump(struct bnxt_re_qp *qp) +{ + struct bnxt_qplib_qp *qpl = &qp->qplib_qp; + struct bnxt_re_dev *rdev = qp->rdev; + struct qdump_qpinfo *qpinfo; + struct qdump_array *qdump; + bool capture_snapdump; + + if (rdev->snapdump_dbg_lvl == BNXT_RE_SNAPDUMP_NONE) + return; + + capture_snapdump = test_bit(QP_FLAGS_CAPTURE_SNAPDUMP, &qpl->flags); + if (rdev->snapdump_dbg_lvl == BNXT_RE_SNAPDUMP_ERR && + !capture_snapdump) + return; + + if (qp->is_snapdump_captured || !rdev->qdump_head.qdump) + return; + + mutex_lock(&rdev->qdump_head.lock); + qdump = bnxt_re_get_next_qpdump(rdev); + + qpinfo = &qdump->qpinfo; + qpinfo->id = qpl->id; + qpinfo->dest_qpid = qpl->dest_qpn; + qpinfo->is_user = qpl->is_user; + qpinfo->mtu = qpl->mtu; + qpinfo->state = qpl->state; + qpinfo->type = qpl->type; + qpinfo->wqe_mode = qpl->wqe_mode; + qpinfo->qp_handle = qpl->qp_handle; + qpinfo->scq_handle = qp->scq->qplib_cq.cq_handle; + qpinfo->rcq_handle = qp->rcq->qplib_cq.cq_handle; + qpinfo->scq_id = qp->scq->qplib_cq.id; + qpinfo->rcq_id = qp->rcq->qplib_cq.id; + + qdump->valid = true; + qp->is_snapdump_captured = true; + mutex_unlock(&rdev->qdump_head.lock); +} + /* Queue Pairs */ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata) { @@ -965,6 +1031,7 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata) unsigned int flags; int rc; + bnxt_re_capture_qpdump(qp); bnxt_re_debug_rem_qpinfo(rdev, qp); bnxt_qplib_flush_cqn_wq(&qp->qplib_qp); diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h index fbb16a4..8d82066 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h @@ -96,6 +96,7 @@ struct bnxt_re_qp { struct bnxt_re_cq *scq; struct bnxt_re_cq *rcq; struct dentry *dentry; + bool is_snapdump_captured; }; struct bnxt_re_cq { @@ -113,6 +114,7 @@ struct bnxt_re_cq { int resize_cqe; void *uctx_cq_page; struct hlist_node hash_entry; + bool is_snapdump_captured; }; struct bnxt_re_mr { @@ -267,6 +269,46 @@ int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata); void bnxt_re_dealloc_ucontext(struct ib_ucontext *context); int bnxt_re_mmap(struct ib_ucontext *context, struct vm_area_struct *vma); void bnxt_re_mmap_free(struct rdma_user_mmap_entry *rdma_entry); +void bnxt_re_capture_qpdump(struct bnxt_re_qp *qp); + +static inline const char *__to_qp_type_str(u8 type) +{ + switch (type) { + case CMDQ_CREATE_QP1_TYPE_GSI: + case CMDQ_CREATE_QP_TYPE_GSI: + return "GSI"; + case CMDQ_CREATE_QP_TYPE_RC: + return "RC"; + case CMDQ_CREATE_QP_TYPE_UD: + return "UD"; + case CMDQ_CREATE_QP_TYPE_RAW_ETHERTYPE: + return "RAW_ETH"; + default: + return "NotSupp"; + } +} + +static inline const char *__to_qp_state_str(u8 state) +{ + switch (state) { + case CMDQ_MODIFY_QP_NEW_STATE_RESET: + return "RESET"; + case CMDQ_MODIFY_QP_NEW_STATE_INIT: + return "INIT"; + case CMDQ_MODIFY_QP_NEW_STATE_RTR: + return "RTR"; + case CMDQ_MODIFY_QP_NEW_STATE_RTS: + return "RTS"; + case CMDQ_MODIFY_QP_NEW_STATE_SQD: + return "SQD"; + case CMDQ_MODIFY_QP_NEW_STATE_SQE: + return "SQE"; + case CMDQ_MODIFY_QP_NEW_STATE_ERR: + return "ERR"; + default: + return "NotSupp"; + } +} static inline u32 __to_ib_port_num(u16 port_id) { diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 87fdf69..67d2bf0 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -1420,9 +1420,15 @@ static int bnxt_re_handle_qp_async_event(struct creq_qp_event *qp_event, struct ib_event event = {}; unsigned int flags; - if (qp->qplib_qp.srq) + if (qp->qplib_qp.srq) { srq = container_of(qp->qplib_qp.srq, struct bnxt_re_srq, qplib_srq); + set_bit(SRQ_FLAGS_CAPTURE_SNAPDUMP, &srq->qplib_srq.flags); + } + + set_bit(QP_FLAGS_CAPTURE_SNAPDUMP, &qp->qplib_qp.flags); + set_bit(CQ_FLAGS_CAPTURE_SNAPDUMP, &qp->scq->qplib_cq.flags); + set_bit(CQ_FLAGS_CAPTURE_SNAPDUMP, &qp->rcq->qplib_cq.flags); if (qp->qplib_qp.state == CMDQ_MODIFY_QP_NEW_STATE_ERR && rdma_is_kernel_res(&qp->ib_qp.res)) { diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.h b/drivers/infiniband/hw/bnxt_re/qplib_fp.h index 0d9487c..d1acb01 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_fp.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.h @@ -106,6 +106,8 @@ struct bnxt_qplib_srq { u16 eventq_hw_ring_id; spinlock_t lock; /* protect SRQE link list */ u8 toggle; + unsigned long flags; +#define SRQ_FLAGS_CAPTURE_SNAPDUMP 1 }; struct bnxt_qplib_sge { @@ -279,6 +281,8 @@ struct bnxt_qplib_qp { u8 wqe_mode; u8 state; u8 cur_qp_state; + u16 ctx_size_sb; + u8 is_user; u64 modify_flags; u32 max_inline_data; u32 mtu; @@ -344,6 +348,8 @@ struct bnxt_qplib_qp { u32 msn_tbl_sz; bool is_host_msn_tbl; u8 tos_dscp; + unsigned long flags; +#define QP_FLAGS_CAPTURE_SNAPDUMP 1 }; #define BNXT_QPLIB_MAX_CQE_ENTRY_SIZE sizeof(struct cq_base) @@ -448,6 +454,7 @@ struct bnxt_qplib_cq { #define CQ_RESIZE_WAIT_TIME_MS 500 unsigned long flags; #define CQ_FLAGS_RESIZE_IN_PROG 1 +#define CQ_FLAGS_CAPTURE_SNAPDUMP 2 wait_queue_head_t waitq; struct list_head sqf_head, rqf_head; atomic_t arm_state; From patchwork Thu Feb 20 18:34:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13984413 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D737D264F82 for ; Thu, 20 Feb 2025 18:56:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077762; cv=none; b=IBheUOOkOIwdQ+MhoAn6qLrJEQyrOJtwN9SZEv1HkbSg1ftgMwx+PtgjDhG6IMcfyN69oOhRYpTz14jAoAIzSPFGgN0aIuEh5pyX0GJQLuFxCtzbfH4mj5S5M0JFrijYNHDDGLMF0GcWtrPJZKbI7l/uaVBFSsZ/EYQMSFfkaHU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077762; c=relaxed/simple; bh=OO35DOOYWqXdZ4O8hec5kL6h6m3RU0cEIxNWZ1peCks=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=ZYUm2es5s1QtIMF7zRSzSDiiVXbYLS7A/kQAnSmycHKo4FtopQPP/tGLVe2ErrTU5CwFLS3dACHqIS333RElTCo5JrD8QaliukXYjLOzLXGfr8xGjf3Buu2FcAZ/Y9rpRKXOsjexFf5Hv7+v/3Ab0Hcfnel1PYH22CiXF3Wtf5M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=CqV5TGlh; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="CqV5TGlh" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-220bff984a0so24765195ad.3 for ; Thu, 20 Feb 2025 10:56:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1740077760; x=1740682560; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=AmBAXHsxWXsrXYjGGcmVq0b9Wu0WtSKDOr6er0hlrAA=; b=CqV5TGlh4p2MJMYKQNuZ7iLdmkqcP7mzR4vcAaxFq05Z97B3KF271+KIDYJWOVyT73 3f7XzcL9yFUQ85FNeceHxWy2z+sY9OsuFRNRqatR558ITaFFz/j+hxvdT1WO45D5aPWp THyQ4wo2jLnrPft0sAT+7IyoPSdK1grQ9W2EI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740077760; x=1740682560; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AmBAXHsxWXsrXYjGGcmVq0b9Wu0WtSKDOr6er0hlrAA=; b=ZrLQ1lnm6E2ThBDrV8neTGje6wDZKMWpTC7ZdH8wV8+BEt5E1yQxA2k0aB+vy8Z2WJ bvquA121DdjNTWXXefwFNw8yMByac2hiswELzK7pJEHNvEs4Y+dlBcMsKNIaWkKU1wFv jcqsKdujqCxR+2A584BUei5X6bAHPIyLcezzCkd7mN+o8j58okPf74aGUogi83BQ0Oe2 BUHT3mV+bdIpB4oYUcLq58iv3/tHfqtKkGtK622+m9ABOW9l10rBSZGgNCDccghDwiql GXLB/GG0YqPN6rfQYkxZKHP//Yt+4w0eHaLreg5JChsOXP14v/+jifNYReK0B/UqkP/t sRzg== X-Gm-Message-State: AOJu0Yy1RkQRRl/zx3aGhfeNplZofEaRx1592D/hheYjXp1Qpr7BR5OJ k+WTSOe+w4+khTbUHGWSEbNLAL+cwA74WOtEojV+dR7qczD4VtPwDZsaFUWDKQ== X-Gm-Gg: ASbGncvD+hgcsPoKY2nO3QYm8bzVkqOZIyTtTUjatlUs6rcnVWUKkBO54sCfUVLwQQH N7E9K7IwQWGdC9knV6yQZWS6FGcuBvwyy4S5yAJ9OH0CU2AmNj5kXf5VZ7MHu5PXN40Rp4ntIxM BVTIYLyyvjWf+jaxcR7CFeFQn9fXxpUJ8dbFr84A4B+HoxkUF8CX42YxwSv5LGbFgChYNCsfmly 8ol4E8DL3njR+0307kyZ/ZfE2fByLLBu1lqwaXlWCPy8pjbVftKBK1CO3ci9Lvgn5trgOw2xD4P lpYow+ugvP6dQM+d+kaaX4iCSaT58JRfr+zOZk/8K+0pELQ+YcXEnoCBSzXPXncOiKmx3LA= X-Google-Smtp-Source: AGHT+IEvYFGe/Oy8MoaSy7XrmhDwHcku3iPlGrY6y4GGF0RxJLx00BwkgxaeUR0BhRu6J4lU7uyVIg== X-Received: by 2002:a05:6a21:9215:b0:1ee:e4f0:629 with SMTP id adf61e73a8af0-1eef3ca07b3mr368607637.18.1740077760124; Thu, 20 Feb 2025 10:56:00 -0800 (PST) Received: from sxavier-dev.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-addee79a984sm9572262a12.32.2025.02.20.10.55.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:55:59 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, kalesh-anakkur.purayil@broadcom.com, netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, abeni@redhat.com, horms@kernel.org, michael.chan@broadcom.com, Kashyap Desai , Selvin Xavier Subject: [PATCH rdma-next 3/9] RDMA/bnxt_re : Initialize the HW context dump collection Date: Thu, 20 Feb 2025 10:34:50 -0800 Message-Id: <1740076496-14227-4-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> References: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Kashyap Desai HW context of the destroyed resources are returned during the destroy operation of the resource. Initialize the data structures to collect the HW contexts. Check if the FW supports the context dump and initialize the FW to send the dump to host. Signed-off-by: Kashyap Desai Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 32 ++++++++++++++++++++++++++++++ drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 2 ++ drivers/infiniband/hw/bnxt_re/qplib_rcfw.h | 12 +++++++++++ drivers/infiniband/hw/bnxt_re/qplib_res.h | 14 +++++++++++++ drivers/infiniband/hw/bnxt_re/roce_hsi.h | 2 ++ 5 files changed, 62 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 67d2bf0..6b5a169 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -2016,6 +2016,12 @@ static void bnxt_re_free_nqr_mem(struct bnxt_re_dev *rdev) static void bnxt_re_clean_qdump(struct bnxt_re_dev *rdev) { + struct bnxt_qplib_rcfw *rcfw = &rdev->rcfw; + + vfree(rcfw->qp_ctxm_data); + vfree(rcfw->cq_ctxm_data); + vfree(rcfw->srq_ctxm_data); + vfree(rcfw->mrw_ctxm_data); vfree(rdev->qdump_head.qdump); } @@ -2075,14 +2081,40 @@ static void bnxt_re_worker(struct work_struct *work) schedule_delayed_work(&rdev->worker, msecs_to_jiffies(30000)); } +static void bnxt_re_init_ctxm_size(struct bnxt_re_dev *rdev) +{ + struct bnxt_qplib_rcfw *rcfw = &rdev->rcfw; + + if (bnxt_qplib_is_chip_gen_p7(rdev->chip_ctx)) { + rcfw->qp_ctxm_size = BNXT_RE_CONTEXT_TYPE_QPC_SIZE_P7; + rcfw->cq_ctxm_size = BNXT_RE_CONTEXT_TYPE_CQ_SIZE_P7; + rcfw->srq_ctxm_size = BNXT_RE_CONTEXT_TYPE_SRQ_SIZE_P7; + rcfw->mrw_ctxm_size = BNXT_RE_CONTEXT_TYPE_MRW_SIZE_P7; + + } else if (bnxt_qplib_is_chip_gen_p5(rdev->chip_ctx)) { + rcfw->qp_ctxm_size = BNXT_RE_CONTEXT_TYPE_QPC_SIZE_P5; + rcfw->cq_ctxm_size = BNXT_RE_CONTEXT_TYPE_CQ_SIZE_P5; + rcfw->srq_ctxm_size = BNXT_RE_CONTEXT_TYPE_SRQ_SIZE_P5; + rcfw->mrw_ctxm_size = BNXT_RE_CONTEXT_TYPE_MRW_SIZE_P5; + } +} + static void bnxt_re_init_qdump(struct bnxt_re_dev *rdev) { + struct bnxt_qplib_rcfw *rcfw = &rdev->rcfw; + rdev->qdump_head.max_elements = BNXT_RE_MAX_QDUMP_ENTRIES; rdev->qdump_head.index = 0; rdev->snapdump_dbg_lvl = BNXT_RE_SNAPDUMP_ERR; mutex_init(&rdev->qdump_head.lock); rdev->qdump_head.qdump = vzalloc(rdev->qdump_head.max_elements * sizeof(struct qdump_array)); + /* Setup Context cache information */ + bnxt_re_init_ctxm_size(rdev); + rcfw->qp_ctxm_data = vzalloc(BNXT_RE_MAX_QDUMP_ENTRIES * rcfw->qp_ctxm_size); + rcfw->cq_ctxm_data = vzalloc(BNXT_RE_MAX_QDUMP_ENTRIES * rcfw->cq_ctxm_size); + rcfw->srq_ctxm_data = vzalloc(BNXT_RE_MAX_QDUMP_ENTRIES * rcfw->srq_ctxm_size); + rcfw->mrw_ctxm_data = vzalloc(BNXT_RE_MAX_QDUMP_ENTRIES * rcfw->mrw_ctxm_size); } static int bnxt_re_dev_init(struct bnxt_re_dev *rdev, u8 op_type) diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c index 17e62f2..435b01f 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c @@ -903,6 +903,8 @@ int bnxt_qplib_init_rcfw(struct bnxt_qplib_rcfw *rcfw, flags |= CMDQ_INITIALIZE_FW_FLAGS_OPTIMIZE_MODIFY_QP_SUPPORTED; if (rcfw->res->en_dev->flags & BNXT_EN_FLAG_ROCE_VF_RES_MGMT) flags |= CMDQ_INITIALIZE_FW_FLAGS_L2_VF_RESOURCE_MGMT; + if (is_roce_context_destroy_sb_enabled(rcfw->res->dattr->dev_cap_flags2)) + flags |= CMDQ_INITIALIZE_FW_FLAGS_DESTROY_CONTEXT_SB_SUPPORTED; req.flags |= cpu_to_le16(flags); req.stat_ctx_id = cpu_to_le32(ctx->stats.fw_id); bnxt_qplib_fill_cmdqmsg(&msg, &req, &resp, NULL, sizeof(req), sizeof(resp), 0); diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h index 88814cb..155e24f 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h @@ -228,6 +228,18 @@ struct bnxt_qplib_rcfw { struct bnxt_qplib_qp_node *qp_tbl; /* To synchronize the qp-handle hash table */ spinlock_t tbl_lock; + u32 qp_ctxm_data_index; + u32 cq_ctxm_data_index; + u32 mrw_ctxm_data_index; + u32 srq_ctxm_data_index; + u16 qp_ctxm_size; + u16 cq_ctxm_size; + u16 mrw_ctxm_size; + u16 srq_ctxm_size; + void *qp_ctxm_data; + void *cq_ctxm_data; + void *mrw_ctxm_data; + void *srq_ctxm_data; u64 oos_prev; u32 init_oos_stats; u32 cmdq_depth; diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.h b/drivers/infiniband/hw/bnxt_re/qplib_res.h index be5d907..93200ea 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_res.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_res.h @@ -592,4 +592,18 @@ static inline bool _is_cq_coalescing_supported(u16 dev_cap_ext_flags2) return dev_cap_ext_flags2 & CREQ_QUERY_FUNC_RESP_SB_CQ_COALESCING_SUPPORTED; } +static inline bool is_roce_context_destroy_sb_enabled(u16 dev_cap_ext_flags2) +{ + return !!(dev_cap_ext_flags2 & CREQ_QUERY_FUNC_RESP_SB_DESTROY_CONTEXT_SB_SUPPORTED); +} + +#define BNXT_RE_CONTEXT_TYPE_QPC_SIZE_P5 1088 +#define BNXT_RE_CONTEXT_TYPE_CQ_SIZE_P5 128 +#define BNXT_RE_CONTEXT_TYPE_MRW_SIZE_P5 128 +#define BNXT_RE_CONTEXT_TYPE_SRQ_SIZE_P5 192 + +#define BNXT_RE_CONTEXT_TYPE_QPC_SIZE_P7 1088 +#define BNXT_RE_CONTEXT_TYPE_CQ_SIZE_P7 192 +#define BNXT_RE_CONTEXT_TYPE_MRW_SIZE_P7 192 +#define BNXT_RE_CONTEXT_TYPE_SRQ_SIZE_P7 192 #endif /* __BNXT_QPLIB_RES_H__ */ diff --git a/drivers/infiniband/hw/bnxt_re/roce_hsi.h b/drivers/infiniband/hw/bnxt_re/roce_hsi.h index 0ee60fd..041e9ac 100644 --- a/drivers/infiniband/hw/bnxt_re/roce_hsi.h +++ b/drivers/infiniband/hw/bnxt_re/roce_hsi.h @@ -218,6 +218,7 @@ struct cmdq_initialize_fw { #define CMDQ_INITIALIZE_FW_FLAGS_HW_REQUESTER_RETX_SUPPORTED 0x2UL #define CMDQ_INITIALIZE_FW_FLAGS_OPTIMIZE_MODIFY_QP_SUPPORTED 0x8UL #define CMDQ_INITIALIZE_FW_FLAGS_L2_VF_RESOURCE_MGMT 0x10UL + #define CMDQ_INITIALIZE_FW_FLAGS_DESTROY_CONTEXT_SB_SUPPORTED 0x20UL __le16 cookie; u8 resp_size; u8 reserved8; @@ -2215,6 +2216,7 @@ struct creq_query_func_resp_sb { #define CREQ_QUERY_FUNC_RESP_SB_REQ_RETRANSMISSION_SUPPORT_IQM_MSN_TABLE (0x2UL << 4) #define CREQ_QUERY_FUNC_RESP_SB_REQ_RETRANSMISSION_SUPPORT_LAST \ CREQ_QUERY_FUNC_RESP_SB_REQ_RETRANSMISSION_SUPPORT_IQM_MSN_TABLE + #define CREQ_QUERY_FUNC_RESP_SB_DESTROY_CONTEXT_SB_SUPPORTED 0x100UL #define CREQ_QUERY_FUNC_RESP_SB_MIN_RNR_RTR_RTS_OPT_SUPPORTED 0x1000UL __le16 max_xp_qp_size; __le16 create_qp_batch_size; From patchwork Thu Feb 20 18:34:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13984414 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6CF4226388E for ; Thu, 20 Feb 2025 18:56:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077767; cv=none; b=REGGXv9KabcsHIyB7ONrwKaCzuWe2xY/OqR9HuNS1Yn6ZNC+VdnuInB1gR9MIRThMEhfhx1xJKLA8t42HywVb0BGF0wyaZnOvDLL10mVp9WY5bI9T8hUL0vexNLTnVkctH/mM7ZmRDCmGJ2b28Kl0MeijREGoeQ3I7I3Qhi3wMg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077767; c=relaxed/simple; bh=e39Ye7f115z5V+apYQtx/SpSC66HkndM95PkDhQUplA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=CGx6V0Ne9bGGYNnA4DCRcG7CT8rU2qnF5nJz6ZMhLJvCoougf3koRyLEwvMHOX7yerMb/046YK8fuo/MTQ0T1cC4X8RB4ylfhG08L+D5BfDFeQEGE4nVcnDjuesRHYCfzZKtoKtzMfHNOhsL1HeVcZVAomt+9Hrg1bSULo87jyo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=eK5IZSoe; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="eK5IZSoe" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-21c2f1b610dso35644625ad.0 for ; Thu, 20 Feb 2025 10:56:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1740077765; x=1740682565; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=PBW5jpO95lyNQP0CO3WA3VL0tUEtqSN8vPLv1rH3qtg=; b=eK5IZSoemjr4SSUgJaZ+Uc700VsnHd2/2hKolark4FtoClu4vHkE2sQk5fvYGPd5kH hvDdjfhNbv1NvJjYZTFChzORUWSlFESwG0R6Shej8TzBpzDIf3FGHwvwjKYdfl2l8v8T sl+ArTwudajLuYNvUUFoKQq+qHdijVlb/nsj8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740077765; x=1740682565; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PBW5jpO95lyNQP0CO3WA3VL0tUEtqSN8vPLv1rH3qtg=; b=ZfHMjNVZVYta1Vl/8d6aZV/PVg1qOslwQoM6Rlk+hEwig20cJFx8AsAJqYfLvQxxh5 YmDdWYILeDmR8V8vFPFx28911XdtksobocBOny6QlAlMKMzsiVQdUezuyC3QNNaEDiVE +SS4JFu+Q5QSwJbxG6NfO9JsSRRePb88fZ+6n5PdzJ0DGTFMae5jxo8kbbaNLdBnRDe1 ghNGGd2+Yw6NHZi/eN3R9SQbMaZop03O6c6lgrGWbmS3o7hpxRK777lXGwhFPefhN6XI 1ezlBTgD+Nt2xBi/rheVBhV0ozFhtGs5y0cbk+lfbFcHx722DUebWEeDwfOoR1tlVHP1 kRNQ== X-Gm-Message-State: AOJu0Yzor36dmXDUntfyG/Z1CIX0tplWvQojmyCZ17mKu8YX2FQ3zhUO EM7H3j0IQUejF7j79zYK90tlNEU5hiF/nTfVs3Uv3ouFze9PTtZM3eVI8PH9Xg== X-Gm-Gg: ASbGncuDCI3LhayAKby/BgOc/+bvppV06NL+LAQhmC94PR+D+e1XR06SultQX6XogIB ZvLRcD0VDRrnxAC1B8KRRtuZARWYxmg8/6bZ1PaRkMY4Dkt4JYf1C8HMjaV0T2ns6lXJ8wR69wV 8ziaiAvSlf0WIIva/W6rl6zmmFmMQ+rxRfOo2cd1KcMK0GbM9fahphPq9CZ+Gc0yQo/3n+sgSeR m5qSXKNyG2DLp7QHDKPAybZBtQIc6dRcHgirNFNrbqz8Foja8QQIO5Q7f8YI7mTp9GBBxhVgo0d R7pttWyR3Y6Ehxg3cBW9+utn1chXQ6XhVxXMHTVfDy/ImoYoGdlzbQGUUDyNMrEDhxe0U60= X-Google-Smtp-Source: AGHT+IGgxyoRm0rro/+RGU0DQSiNh0KMz3mFvEijqmEYGx7+Kd8cXubMPj5jjq6aQuAQ/+11wvdvFQ== X-Received: by 2002:a05:6a21:3949:b0:1ee:c7c8:cad with SMTP id adf61e73a8af0-1eef3dc67c0mr335619637.28.1740077764264; Thu, 20 Feb 2025 10:56:04 -0800 (PST) Received: from sxavier-dev.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-addee79a984sm9572262a12.32.2025.02.20.10.56.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:56:03 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, kalesh-anakkur.purayil@broadcom.com, netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, abeni@redhat.com, horms@kernel.org, michael.chan@broadcom.com, Kashyap Desai , Selvin Xavier Subject: [PATCH rdma-next 4/9] RDMA/bnxt_re: Get the resource contexts from the HW Date: Thu, 20 Feb 2025 10:34:51 -0800 Message-Id: <1740076496-14227-5-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> References: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Kashyap Desai While destroying the resources like QP/CQ/SRQ/MRs, FW sends the contexts to the host. Driver caches these information to be dumped later for debugging. Driver can hold the information of the last 1024 entries of each resources. Signed-off-by: Kashyap Desai Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 9 +++ drivers/infiniband/hw/bnxt_re/ib_verbs.c | 120 +++++++++++++++++++++++++++---- drivers/infiniband/hw/bnxt_re/qplib_fp.c | 47 +++++++++++- drivers/infiniband/hw/bnxt_re/qplib_fp.h | 9 ++- drivers/infiniband/hw/bnxt_re/qplib_sp.c | 15 +++- drivers/infiniband/hw/bnxt_re/qplib_sp.h | 3 +- 6 files changed, 181 insertions(+), 22 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index 5818db1..45601635 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -175,6 +175,15 @@ static inline bool bnxt_re_chip_gen_p7(u16 chip_num) #define BNXT_RE_MAX_QDUMP_ENTRIES 1024 +enum { + BNXT_RE_RES_TYPE_CQ = 0, + BNXT_RE_RES_TYPE_UCTX, + BNXT_RE_RES_TYPE_QP, + BNXT_RE_RES_TYPE_SRQ, + BNXT_RE_RES_TYPE_MR, + BNXT_RE_RES_TYPE_MAX +}; + struct qdump_qpinfo { u32 id; u32 dest_qpid; diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index 060143e..a1ee6ca 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -537,7 +537,7 @@ static void bnxt_re_destroy_fence_mr(struct bnxt_re_pd *pd) bnxt_qplib_dereg_mrw(&rdev->qplib_res, &mr->qplib_mr, true); if (mr->ib_mr.lkey) - bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr); + bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr, 0, NULL); kfree(mr); fence->mr = NULL; } @@ -886,6 +886,55 @@ int bnxt_re_query_ah(struct ib_ah *ib_ah, struct rdma_ah_attr *ah_attr) return 0; } +static inline void bnxt_re_save_resource_context(struct bnxt_re_dev *rdev, + void *ctx_sb_data, + u8 res_type, bool do_snapdump) +{ + void *drv_ctx_data; + u32 *ctx_index; + u16 ctx_size; + + if (!ctx_sb_data) + return; + + switch (res_type) { + case BNXT_RE_RES_TYPE_QP: + drv_ctx_data = rdev->rcfw.qp_ctxm_data; + ctx_index = &rdev->rcfw.qp_ctxm_data_index; + ctx_size = rdev->rcfw.qp_ctxm_size; + break; + case BNXT_RE_RES_TYPE_CQ: + drv_ctx_data = rdev->rcfw.cq_ctxm_data; + ctx_index = &rdev->rcfw.cq_ctxm_data_index; + ctx_size = rdev->rcfw.cq_ctxm_size; + break; + case BNXT_RE_RES_TYPE_MR: + drv_ctx_data = rdev->rcfw.mrw_ctxm_data; + ctx_index = &rdev->rcfw.mrw_ctxm_data_index; + ctx_size = rdev->rcfw.mrw_ctxm_size; + break; + case BNXT_RE_RES_TYPE_SRQ: + drv_ctx_data = rdev->rcfw.srq_ctxm_data; + ctx_index = &rdev->rcfw.srq_ctxm_data_index; + ctx_size = rdev->rcfw.srq_ctxm_size; + break; + default: + return; + } + + if (rdev->snapdump_dbg_lvl == BNXT_RE_SNAPDUMP_ALL || + (rdev->snapdump_dbg_lvl == BNXT_RE_SNAPDUMP_ERR && do_snapdump)) { + memcpy(drv_ctx_data + (*ctx_index * ctx_size), + ctx_sb_data, ctx_size); + *ctx_index = *ctx_index + 1; + *ctx_index = *ctx_index % BNXT_RE_MAX_QDUMP_ENTRIES; + dev_dbg(rdev_to_dev(rdev), + "%s : res_type %d ctx_index %d 0x%lx\n", __func__, + res_type, *ctx_index, + (unsigned long)drv_ctx_data + (*ctx_index * ctx_size)); + } +} + unsigned long bnxt_re_lock_cqs(struct bnxt_re_qp *qp) __acquires(&qp->scq->cq_lock) __acquires(&qp->rcq->cq_lock) { @@ -930,7 +979,7 @@ static int bnxt_re_destroy_gsi_sqp(struct bnxt_re_qp *qp) bnxt_qplib_clean_qp(&qp->qplib_qp); ibdev_dbg(&rdev->ibdev, "Destroy the shadow QP\n"); - rc = bnxt_qplib_destroy_qp(&rdev->qplib_res, &gsi_sqp->qplib_qp); + rc = bnxt_qplib_destroy_qp(&rdev->qplib_res, &gsi_sqp->qplib_qp, 0, NULL); if (rc) { ibdev_err(&rdev->ibdev, "Destroy Shadow QP failed"); goto fail; @@ -1029,17 +1078,28 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata) struct bnxt_qplib_nq *scq_nq = NULL; struct bnxt_qplib_nq *rcq_nq = NULL; unsigned int flags; + void *ctx_sb_data = NULL; + bool do_snapdump; + u16 ctx_size; int rc; bnxt_re_capture_qpdump(qp); bnxt_re_debug_rem_qpinfo(rdev, qp); bnxt_qplib_flush_cqn_wq(&qp->qplib_qp); + ctx_size = qplib_qp->ctx_size_sb; + if (ctx_size) + ctx_sb_data = vzalloc(ctx_size); + do_snapdump = test_bit(QP_FLAGS_CAPTURE_SNAPDUMP, &qplib_qp->flags); - rc = bnxt_qplib_destroy_qp(&rdev->qplib_res, &qp->qplib_qp); + rc = bnxt_qplib_destroy_qp(&rdev->qplib_res, &qp->qplib_qp, ctx_size, ctx_sb_data); if (rc) ibdev_err(&rdev->ibdev, "Failed to destroy HW QP"); + else + bnxt_re_save_resource_context(rdev, ctx_sb_data, + BNXT_RE_RES_TYPE_QP, do_snapdump); + vfree(ctx_sb_data); if (rdma_is_kernel_res(&qp->ib_qp.res)) { flags = bnxt_re_lock_cqs(qp); bnxt_qplib_clean_qp(&qp->qplib_qp); @@ -1599,7 +1659,7 @@ static int bnxt_re_create_shadow_gsi(struct bnxt_re_qp *qp, &qp->qplib_qp); if (!sah) { bnxt_qplib_destroy_qp(&rdev->qplib_res, - &sqp->qplib_qp); + &sqp->qplib_qp, 0, NULL); rc = -ENODEV; ibdev_err(&rdev->ibdev, "Failed to create AH entry for ShadowQP"); @@ -1747,7 +1807,7 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr, return 0; qp_destroy: - bnxt_qplib_destroy_qp(&rdev->qplib_res, &qp->qplib_qp); + bnxt_qplib_destroy_qp(&rdev->qplib_res, &qp->qplib_qp, 0, NULL); free_umem: ib_umem_release(qp->rumem); ib_umem_release(qp->sumem); @@ -1841,6 +1901,9 @@ int bnxt_re_destroy_srq(struct ib_srq *ib_srq, struct ib_udata *udata) struct bnxt_re_dev *rdev = srq->rdev; struct bnxt_qplib_srq *qplib_srq = &srq->qplib_srq; struct bnxt_qplib_nq *nq = NULL; + void *ctx_sb_data = NULL; + bool do_snapdump; + u16 ctx_size; if (qplib_srq->cq) nq = qplib_srq->cq->nq; @@ -1848,7 +1911,18 @@ int bnxt_re_destroy_srq(struct ib_srq *ib_srq, struct ib_udata *udata) free_page((unsigned long)srq->uctx_srq_page); hash_del(&srq->hash_entry); } - bnxt_qplib_destroy_srq(&rdev->qplib_res, qplib_srq); + ctx_size = rdev->rcfw.srq_ctxm_size; + if (ctx_size) + ctx_sb_data = vzalloc(ctx_size); + + bnxt_qplib_destroy_srq(&rdev->qplib_res, qplib_srq, ctx_size, ctx_sb_data); + + do_snapdump = test_bit(SRQ_FLAGS_CAPTURE_SNAPDUMP, &qplib_srq->flags); + bnxt_re_save_resource_context(rdev, ctx_sb_data, + BNXT_RE_RES_TYPE_SRQ, do_snapdump); + + vfree(ctx_sb_data); + ib_umem_release(srq->umem); atomic_dec(&rdev->stats.res.srq_count); if (nq) @@ -1968,7 +2042,7 @@ int bnxt_re_create_srq(struct ib_srq *ib_srq, if (rc) { ibdev_err(&rdev->ibdev, "SRQ copy to udata failed!"); bnxt_qplib_destroy_srq(&rdev->qplib_res, - &srq->qplib_srq); + &srq->qplib_srq, 0, NULL); goto fail; } } @@ -3127,6 +3201,9 @@ int bnxt_re_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata) struct bnxt_qplib_nq *nq; struct bnxt_re_dev *rdev; struct bnxt_re_cq *cq; + void *ctx_sb_data = NULL; + bool do_snapdump; + u16 ctx_size; cq = container_of(ib_cq, struct bnxt_re_cq, ib_cq); rdev = cq->rdev; @@ -3137,7 +3214,15 @@ int bnxt_re_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata) free_page((unsigned long)cq->uctx_cq_page); hash_del(&cq->hash_entry); } - bnxt_qplib_destroy_cq(&rdev->qplib_res, &cq->qplib_cq); + ctx_size = rdev->rcfw.cq_ctxm_size; + if (ctx_size) + ctx_sb_data = vzalloc(ctx_size); + bnxt_qplib_destroy_cq(&rdev->qplib_res, &cq->qplib_cq, ctx_size, ctx_sb_data); + do_snapdump = test_bit(CQ_FLAGS_CAPTURE_SNAPDUMP, &cq->qplib_cq.flags); + bnxt_re_save_resource_context(rdev, ctx_sb_data, + BNXT_RE_RES_TYPE_CQ, do_snapdump); + + vfree(ctx_sb_data); bnxt_re_put_nq(rdev, nq); ib_umem_release(cq->umem); @@ -3247,7 +3332,7 @@ int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, rc = ib_copy_to_udata(udata, &resp, min(sizeof(resp), udata->outlen)); if (rc) { ibdev_err(&rdev->ibdev, "Failed to copy CQ udata"); - bnxt_qplib_destroy_cq(&rdev->qplib_res, &cq->qplib_cq); + bnxt_qplib_destroy_cq(&rdev->qplib_res, &cq->qplib_cq, 0, NULL); goto free_mem; } } @@ -4070,7 +4155,7 @@ struct ib_mr *bnxt_re_get_dma_mr(struct ib_pd *ib_pd, int mr_access_flags) return &mr->ib_mr; fail_mr: - bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr); + bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr, 0, NULL); fail: kfree(mr); return ERR_PTR(rc); @@ -4080,13 +4165,20 @@ int bnxt_re_dereg_mr(struct ib_mr *ib_mr, struct ib_udata *udata) { struct bnxt_re_mr *mr = container_of(ib_mr, struct bnxt_re_mr, ib_mr); struct bnxt_re_dev *rdev = mr->rdev; + void *ctx_sb_data = NULL; + u16 ctx_size; int rc; - rc = bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr); + ctx_size = rdev->rcfw.mrw_ctxm_size; + if (ctx_size) + ctx_sb_data = vzalloc(ctx_size); + rc = bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr, ctx_size, ctx_sb_data); if (rc) { ibdev_err(&rdev->ibdev, "Dereg MR failed: %#x\n", rc); return rc; } + bnxt_re_save_resource_context(rdev, ctx_sb_data, BNXT_RE_RES_TYPE_MR, 0); + vfree(ctx_sb_data); if (mr->pages) { rc = bnxt_qplib_free_fast_reg_page_list(&rdev->qplib_res, @@ -4175,7 +4267,7 @@ struct ib_mr *bnxt_re_alloc_mr(struct ib_pd *ib_pd, enum ib_mr_type type, fail_mr: kfree(mr->pages); fail: - bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr); + bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr, 0, NULL); bail: kfree(mr); return ERR_PTR(rc); @@ -4222,7 +4314,7 @@ int bnxt_re_dealloc_mw(struct ib_mw *ib_mw) struct bnxt_re_dev *rdev = mw->rdev; int rc; - rc = bnxt_qplib_free_mrw(&rdev->qplib_res, &mw->qplib_mw); + rc = bnxt_qplib_free_mrw(&rdev->qplib_res, &mw->qplib_mw, 0, NULL); if (rc) { ibdev_err(&rdev->ibdev, "Free MW failed: %#x\n", rc); return rc; @@ -4301,7 +4393,7 @@ static struct ib_mr *__bnxt_re_user_reg_mr(struct ib_pd *ib_pd, u64 length, u64 return &mr->ib_mr; free_mrw: - bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr); + bnxt_qplib_free_mrw(&rdev->qplib_res, &mr->qplib_mr, 0, NULL); free_mr: kfree(mr); return ERR_PTR(rc); diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.c b/drivers/infiniband/hw/bnxt_re/qplib_fp.c index 5336f74..fa221a7 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_fp.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.c @@ -616,14 +616,21 @@ int bnxt_qplib_alloc_nq(struct bnxt_qplib_res *res, struct bnxt_qplib_nq *nq) /* SRQ */ void bnxt_qplib_destroy_srq(struct bnxt_qplib_res *res, - struct bnxt_qplib_srq *srq) + struct bnxt_qplib_srq *srq, u16 sb_resp_size, void *sb_resp_va) { struct bnxt_qplib_rcfw *rcfw = res->rcfw; struct creq_destroy_srq_resp resp = {}; + struct bnxt_qplib_rcfw_sbuf sbuf = {}; struct bnxt_qplib_cmdqmsg msg = {}; struct cmdq_destroy_srq req = {}; int rc; + if (sb_resp_size && sb_resp_va) { + sbuf.size = sb_resp_size; + sbuf.sb = dma_alloc_coherent(&rcfw->pdev->dev, sbuf.size, + &sbuf.dma_addr, GFP_KERNEL); + } + bnxt_qplib_rcfw_cmd_prep((struct cmdq_base *)&req, CMDQ_BASE_OPCODE_DESTROY_SRQ, sizeof(req)); @@ -632,7 +639,13 @@ void bnxt_qplib_destroy_srq(struct bnxt_qplib_res *res, req.srq_cid = cpu_to_le32(srq->id); bnxt_qplib_fill_cmdqmsg(&msg, &req, &resp, NULL, sizeof(req), sizeof(resp), 0); + req.resp_addr = cpu_to_le64(sbuf.dma_addr); + req.resp_size = sb_resp_size / BNXT_QPLIB_CMDQE_UNITS; rc = bnxt_qplib_rcfw_send_message(rcfw, &msg); + if (sbuf.sb) { + memcpy(sb_resp_va, sbuf.sb, sb_resp_size); + dma_free_coherent(&rcfw->pdev->dev, sbuf.size, sbuf.sb, sbuf.dma_addr); + } kfree(srq->swq); if (rc) return; @@ -1593,15 +1606,22 @@ static void __clean_cq(struct bnxt_qplib_cq *cq, u64 qp) } int bnxt_qplib_destroy_qp(struct bnxt_qplib_res *res, - struct bnxt_qplib_qp *qp) + struct bnxt_qplib_qp *qp, + u16 sb_resp_size, void *sb_resp_va) { struct bnxt_qplib_rcfw *rcfw = res->rcfw; + struct bnxt_qplib_rcfw_sbuf sbuf = {}; struct creq_destroy_qp_resp resp = {}; struct bnxt_qplib_cmdqmsg msg = {}; struct cmdq_destroy_qp req = {}; u32 tbl_indx; int rc; + if (sb_resp_size && sb_resp_va) { + sbuf.size = sb_resp_size; + sbuf.sb = dma_alloc_coherent(&rcfw->pdev->dev, sbuf.size, + &sbuf.dma_addr, GFP_KERNEL); + } spin_lock_bh(&rcfw->tbl_lock); tbl_indx = map_qp_id_to_tbl_indx(qp->id, rcfw); rcfw->qp_tbl[tbl_indx].qp_id = BNXT_QPLIB_QP_ID_INVALID; @@ -1613,6 +1633,8 @@ int bnxt_qplib_destroy_qp(struct bnxt_qplib_res *res, sizeof(req)); req.qp_cid = cpu_to_le32(qp->id); + req.resp_addr = cpu_to_le64(sbuf.dma_addr); + req.resp_size = sb_resp_size / BNXT_QPLIB_CMDQE_UNITS; bnxt_qplib_fill_cmdqmsg(&msg, &req, &resp, NULL, sizeof(req), sizeof(resp), 0); rc = bnxt_qplib_rcfw_send_message(rcfw, &msg); @@ -1623,6 +1645,10 @@ int bnxt_qplib_destroy_qp(struct bnxt_qplib_res *res, spin_unlock_bh(&rcfw->tbl_lock); return rc; } + if (sbuf.sb) { + memcpy(sb_resp_va, sbuf.sb, sb_resp_size); + dma_free_coherent(&rcfw->pdev->dev, sbuf.size, sbuf.sb, sbuf.dma_addr); + } return 0; } @@ -2355,25 +2381,40 @@ int bnxt_qplib_resize_cq(struct bnxt_qplib_res *res, struct bnxt_qplib_cq *cq, return rc; } -int bnxt_qplib_destroy_cq(struct bnxt_qplib_res *res, struct bnxt_qplib_cq *cq) +int bnxt_qplib_destroy_cq(struct bnxt_qplib_res *res, struct bnxt_qplib_cq *cq, + u16 sb_resp_size, void *sb_resp_va) { struct bnxt_qplib_rcfw *rcfw = res->rcfw; + struct bnxt_qplib_rcfw_sbuf sbuf = {}; struct creq_destroy_cq_resp resp = {}; struct bnxt_qplib_cmdqmsg msg = {}; struct cmdq_destroy_cq req = {}; u16 total_cnq_events; int rc; + if (sb_resp_size && sb_resp_va) { + sbuf.size = sb_resp_size; + sbuf.sb = dma_alloc_coherent(&rcfw->pdev->dev, sbuf.size, + &sbuf.dma_addr, GFP_KERNEL); + } + bnxt_qplib_rcfw_cmd_prep((struct cmdq_base *)&req, CMDQ_BASE_OPCODE_DESTROY_CQ, sizeof(req)); + req.resp_addr = cpu_to_le64(sbuf.dma_addr); + req.resp_size = sb_resp_size / BNXT_QPLIB_CMDQE_UNITS; + req.cq_cid = cpu_to_le32(cq->id); bnxt_qplib_fill_cmdqmsg(&msg, &req, &resp, NULL, sizeof(req), sizeof(resp), 0); rc = bnxt_qplib_rcfw_send_message(rcfw, &msg); if (rc) return rc; + if (sbuf.sb) { + memcpy(sb_resp_va, sbuf.sb, sb_resp_size); + dma_free_coherent(&rcfw->pdev->dev, sbuf.size, sbuf.sb, sbuf.dma_addr); + } total_cnq_events = le16_to_cpu(resp.total_cnq_events); __wait_for_all_nqes(cq, total_cnq_events); bnxt_qplib_free_hwq(res, &cq->hwq); diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.h b/drivers/infiniband/hw/bnxt_re/qplib_fp.h index d1acb01..1710b59 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_fp.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.h @@ -555,14 +555,16 @@ int bnxt_qplib_modify_srq(struct bnxt_qplib_res *res, int bnxt_qplib_query_srq(struct bnxt_qplib_res *res, struct bnxt_qplib_srq *srq); void bnxt_qplib_destroy_srq(struct bnxt_qplib_res *res, - struct bnxt_qplib_srq *srq); + struct bnxt_qplib_srq *srq, + u16 sb_resp_size, void *sb_resp_va); int bnxt_qplib_post_srq_recv(struct bnxt_qplib_srq *srq, struct bnxt_qplib_swqe *wqe); int bnxt_qplib_create_qp1(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp); int bnxt_qplib_create_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp); int bnxt_qplib_modify_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp); int bnxt_qplib_query_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp); -int bnxt_qplib_destroy_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp); +int bnxt_qplib_destroy_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp, + u16 ctx_size, void *ctx_sb_data); void bnxt_qplib_clean_qp(struct bnxt_qplib_qp *qp); void bnxt_qplib_free_qp_res(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp); @@ -584,7 +586,8 @@ int bnxt_qplib_resize_cq(struct bnxt_qplib_res *res, struct bnxt_qplib_cq *cq, int new_cqes); void bnxt_qplib_resize_cq_complete(struct bnxt_qplib_res *res, struct bnxt_qplib_cq *cq); -int bnxt_qplib_destroy_cq(struct bnxt_qplib_res *res, struct bnxt_qplib_cq *cq); +int bnxt_qplib_destroy_cq(struct bnxt_qplib_res *res, struct bnxt_qplib_cq *cq, + u16 sb_resp_size, void *sb_resp_v); int bnxt_qplib_poll_cq(struct bnxt_qplib_cq *cq, struct bnxt_qplib_cqe *cqe, int num, struct bnxt_qplib_qp **qp); bool bnxt_qplib_is_cq_empty(struct bnxt_qplib_cq *cq); diff --git a/drivers/infiniband/hw/bnxt_re/qplib_sp.c b/drivers/infiniband/hw/bnxt_re/qplib_sp.c index 4ccd440..d2c519d 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_sp.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_sp.c @@ -504,10 +504,12 @@ int bnxt_qplib_destroy_ah(struct bnxt_qplib_res *res, struct bnxt_qplib_ah *ah, } /* MRW */ -int bnxt_qplib_free_mrw(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mrw) +int bnxt_qplib_free_mrw(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mrw, + u16 sb_resp_size, void *sb_resp_va) { struct creq_deallocate_key_resp resp = {}; struct bnxt_qplib_rcfw *rcfw = res->rcfw; + struct bnxt_qplib_rcfw_sbuf sbuf = {}; struct cmdq_deallocate_key req = {}; struct bnxt_qplib_cmdqmsg msg = {}; int rc; @@ -517,6 +519,11 @@ int bnxt_qplib_free_mrw(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mrw) return 0; } + if (sb_resp_size && sb_resp_va) { + sbuf.size = sb_resp_size; + sbuf.sb = dma_alloc_coherent(&rcfw->pdev->dev, sbuf.size, + &sbuf.dma_addr, GFP_KERNEL); + } bnxt_qplib_rcfw_cmd_prep((struct cmdq_base *)&req, CMDQ_BASE_OPCODE_DEALLOCATE_KEY, sizeof(req)); @@ -530,12 +537,18 @@ int bnxt_qplib_free_mrw(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mrw) else req.key = cpu_to_le32(mrw->lkey); + req.resp_addr = cpu_to_le64(sbuf.dma_addr); + req.resp_size = sb_resp_size / BNXT_QPLIB_CMDQE_UNITS; bnxt_qplib_fill_cmdqmsg(&msg, &req, &resp, NULL, sizeof(req), sizeof(resp), 0); rc = bnxt_qplib_rcfw_send_message(rcfw, &msg); if (rc) return rc; + if (sbuf.sb) { + memcpy(sb_resp_va, sbuf.sb, sb_resp_size); + dma_free_coherent(&rcfw->pdev->dev, sbuf.size, sbuf.sb, sbuf.dma_addr); + } /* Free the qplib's MRW memory */ if (mrw->hwq.max_elements) bnxt_qplib_free_hwq(res, &mrw->hwq); diff --git a/drivers/infiniband/hw/bnxt_re/qplib_sp.h b/drivers/infiniband/hw/bnxt_re/qplib_sp.h index e626b05..728b1f1 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_sp.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_sp.h @@ -340,7 +340,8 @@ int bnxt_qplib_dereg_mrw(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mrw, bool block); int bnxt_qplib_reg_mr(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mr, struct ib_umem *umem, int num_pbls, u32 buf_pg_size); -int bnxt_qplib_free_mrw(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mr); +int bnxt_qplib_free_mrw(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mr, + u16 ctx_size, void *ctx_sb_data); int bnxt_qplib_alloc_fast_reg_mr(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mr, int max); int bnxt_qplib_alloc_fast_reg_page_list(struct bnxt_qplib_res *res, From patchwork Thu Feb 20 18:34:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13984415 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0900C265623 for ; Thu, 20 Feb 2025 18:56:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077770; cv=none; b=qYrM13yqnc3P4heNu41+RHietY4tJ/H6wRLIA0cHv6Ew4mqULuadE1jjHyVhKGWhYzPY3Mav/z9FLKoADVBA8VklF21ylIL7HeJb1EkdDDrwmZVUb8/pK5EuhfGQiCRNBoIxdXcENAwKnE9D7uB/zHKXERG98SBT/1Y1g3bBQ40= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077770; c=relaxed/simple; bh=D8r3NMs/v/gdOgXckK137771Z7EmXwYtM9x6L549VTk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=AIui9ZRhXfIzsgJO7b0g8+rD6AG7nWDmqNrEPZjHKJZpzrHDwHi3hYUdfG9sFwqzw5u8aNHlGwCzl/4CoAuluVtCleNuq/BboIwgi/sL9dtWVIXRvRlWgazL0TqCOVopy9yPqLY8cxGmTbOoaA0rSrCYMswKqZ3dNByDQ2s1k/Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=c5twu1Ee; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="c5twu1Ee" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-220dc3831e3so37838125ad.0 for ; Thu, 20 Feb 2025 10:56:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1740077768; x=1740682568; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=DDGE3L4+QtgAAh6Ko802g9HGK0Xw8N2Rxx0aARspUp4=; b=c5twu1Ee+b62oEfz88Cq4R02kV0MES30ivKaXpVXLT3NAdG1xyEMmWFAWiiMEmO5WL jVlLE0KOMTbDgb3brE2VvDmzDP1CtXDijhn64CuzYCbRFBIlDl7Jwcf7+NpHAJ6gPiDj w7a85qfRjusiAP5jKkBjUc/9FL2eLSInR+UdU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740077768; x=1740682568; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DDGE3L4+QtgAAh6Ko802g9HGK0Xw8N2Rxx0aARspUp4=; b=Bc7fzwVyLb0Zxoz3DFlmGElpQfzWN+Gzhu8Js41o6Hpe+0ezzNz+HyWtRhc8F1BZeh IvzkG4XAfKAN4kDANhkk37a8Y5TwLA9pz1sDtiHZnbdjlhS843kWry5Hjqu1RLIOW1JT f+oVz5ZI3HOJFT9BxJHx2m/r+vUICib5CZoQoUlBTuEM19RPweSP7E0Ie9U8NuJgn9Fd SfTL1+vXyDIjTlD3fHn5w1O7HF2/2HFRYO+Y+HMHfZXV2A0K2BTbFMy/NajMkB2rUZpo 8GKcpekJSDy0sx80PmZgQr5sPj9jSEMs63rPMzfsgjqfhuYhZf1dLpmOPAe3BUPHW5jT xp6A== X-Gm-Message-State: AOJu0Yw/PuEoa6Lwhz0kxpGw0uNwr05bzIJrZsRjpAEpzpC0NpJaWT/E 6+k8CPtjFXQh9mVDSaS77dZiMV6Z/bd+UuWi7M7FAqfV9kB7rooKg+lPBdMicQ== X-Gm-Gg: ASbGnctKmykh+pqZPpnPSMzdAolb8vJCqejG1FiG/ua35lDSwN/7GnQrVt8usGrRInL 4JX12kjVpR8FpR2OV3aukI3xe1OGsfDVBUOcfgcWncf2YL8kK/cChxYJc9B0377ks76ZJXHFhHA s8qFNZzG56OS2VclKC0jFmGX8sw3BTIFfYjsacabKMwL7rIaU0uqqf6UHdrggNkkd7GCprTN1Bw JhsTiRQrwP8lS62YJTIIxYQ3YKV8l1UCaCNe9Fq9mKauiOnVD0RJmMi6VVillTdEag8eoOE324t eBtma55jrzWlVSfKVZZgbWjMQnyRn3LkZMBAtEoWflTvHDPXFqdvRZ4VZ8gCeIia/jQlC50= X-Google-Smtp-Source: AGHT+IGk3I5xMJOQybBNubZesmP0ZDBZjg0y/axwJ+QSrgEcCl4h1AEI/eEbdqV5zZZFRZPg+G0BBQ== X-Received: by 2002:a05:6a21:6d88:b0:1e1:a789:1b4d with SMTP id adf61e73a8af0-1eef3d9f0cfmr321809637.15.1740077768126; Thu, 20 Feb 2025 10:56:08 -0800 (PST) Received: from sxavier-dev.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-addee79a984sm9572262a12.32.2025.02.20.10.56.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:56:07 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, kalesh-anakkur.purayil@broadcom.com, netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, abeni@redhat.com, horms@kernel.org, michael.chan@broadcom.com, Selvin Xavier Subject: [PATCH rdma-next 5/9] bnxt_en: Introduce ULP coredump callbacks Date: Thu, 20 Feb 2025 10:34:52 -0800 Message-Id: <1740076496-14227-6-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> References: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Michael Chan Add .ulp_get_dump_info() and .ulp_get_dump_data() callbacks to struct bnxt_ulp_ops. When ethtool -w is invoked to get the coredump, these 2 callbacks to the bnxt_re auxbus driver will be called if they are populated by the bnxt_re driver. The first callback gets the number of coredump segments and the size of each coredump segment. The 2nd callback copies the coredump data for each segment. Signed-off-by: Michael Chan Reviewed-by: Andy Gospodarek Reviewed-by: Selvin Xavier Signed-off-by: Selvin Xavier Acked-by: Jakub Kicinski --- drivers/net/ethernet/broadcom/bnxt/bnxt_coredump.c | 12 ++++- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 57 ++++++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 22 +++++++++ 3 files changed, 89 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_coredump.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_coredump.c index 7236d8e..2106d0d 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_coredump.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_coredump.c @@ -14,6 +14,7 @@ #include "bnxt.h" #include "bnxt_hwrm.h" #include "bnxt_coredump.h" +#include "bnxt_ulp.h" static const u16 bnxt_bstore_to_seg_id[] = { [BNXT_CTX_QP] = BNXT_CTX_MEM_SEG_QP, @@ -414,13 +415,20 @@ static int __bnxt_get_coredump(struct bnxt *bp, u16 dump_type, void *buf, } if (dump_type == BNXT_DUMP_DRIVER) { - u32 drv_len, segs = 0; + u32 drv_len, drv_segs, segs = 0; + void *drv_buf = NULL; drv_len = bnxt_get_ctx_coredump(bp, buf, offset, &segs); + drv_segs = segs; + segs = 0; + if (buf) + drv_buf = buf + offset + drv_len; + drv_len += bnxt_get_ulp_dump(bp, dump_type, drv_buf, &segs); + drv_segs += segs; *dump_len += drv_len; offset += drv_len; if (buf) - coredump.total_segs += segs; + coredump.total_segs += drv_segs; goto err; } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index e4a7f37..6f640b2 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -25,6 +25,7 @@ #include "bnxt.h" #include "bnxt_hwrm.h" #include "bnxt_ulp.h" +#include "bnxt_coredump.h" static DEFINE_IDA(bnxt_aux_dev_ids); @@ -393,6 +394,62 @@ void bnxt_register_async_events(struct bnxt_en_dev *edev, } EXPORT_SYMBOL(bnxt_register_async_events); +static void bnxt_ulp_fill_dump_hdr(struct bnxt *bp, void *buf, u32 seg_id, + u32 seg_len) +{ + struct bnxt_coredump_segment_hdr seg_hdr; + + bnxt_fill_coredump_seg_hdr(bp, &seg_hdr, NULL, seg_len, 0, 0, 0, + BNXT_DRV_COMP_ID, seg_id); + memcpy(buf, &seg_hdr, sizeof(seg_hdr)); +} + +u32 bnxt_get_ulp_dump(struct bnxt *bp, u32 dump_flag, void *buf, u32 *segs) +{ + struct bnxt_en_dev *edev = bp->edev; + struct bnxt_ulp_dump *dump; + struct bnxt_ulp_ops *ops; + struct bnxt_ulp *ulp; + u32 i, dump_len = 0; + + *segs = 0; + if (!edev || !bnxt_ulp_registered(edev)) + return 0; + + ulp = edev->ulp_tbl; + ops = rtnl_dereference(ulp->ulp_ops); + if (!ops || !ops->ulp_get_dump_info || !ops->ulp_get_dump_data) + return 0; + + dump = &ulp->ulp_dump; + if (!buf) { + memset(dump, 0, sizeof(*dump)); + ops->ulp_get_dump_info(ulp->handle, dump_flag, dump); + if (dump->segs > BNXT_ULP_MAX_DUMP_SEGS) + return 0; + for (i = 0; i < dump->segs; i++) { + dump_len += dump->seg_tbl[i].seg_len; + dump_len += BNXT_SEG_HDR_LEN; + } + } else { + for (i = 0; i < dump->segs; i++) { + struct bnxt_ulp_dump_tbl *tbl = &dump->seg_tbl[i]; + u32 seg_len = tbl->seg_len; + u32 seg_id = tbl->seg_id; + + bnxt_ulp_fill_dump_hdr(bp, buf, seg_id, seg_len); + buf += BNXT_SEG_HDR_LEN; + dump_len += BNXT_SEG_HDR_LEN; + ops->ulp_get_dump_data(ulp->handle, seg_id, buf, + seg_len); + buf += seg_len; + dump_len += seg_len; + } + } + *segs = dump->segs; + return dump_len; +} + void bnxt_rdma_aux_device_uninit(struct bnxt *bp) { struct bnxt_aux_priv *aux_priv; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index 7fa3b8d..9298589 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -29,11 +29,31 @@ struct bnxt_msix_entry { u32 db_offset; }; +#define BNXT_ULP_MAX_DUMP_SEGS 8 + +/** + * struct bnxt_ulp_dump - bnxt ULP aux device coredump info + * @segs: number of coredump segments with info in the seg_tbl + * @seg_tbl: coredump segment table + * @seg_tbl.seg_id: coredump segment ID + * @seg_tbl.seg_len: coredump segment len + */ +struct bnxt_ulp_dump { + u32 segs; + struct bnxt_ulp_dump_tbl { + u32 seg_id; + u32 seg_len; + } seg_tbl[BNXT_ULP_MAX_DUMP_SEGS]; +}; + struct bnxt_ulp_ops { /* async_notifier() cannot sleep (in BH context) */ void (*ulp_async_notifier)(void *, struct hwrm_async_event_cmpl *); void (*ulp_irq_stop)(void *, bool); void (*ulp_irq_restart)(void *, struct bnxt_msix_entry *); + void (*ulp_get_dump_info)(void *handle, u32 dump_flags, + struct bnxt_ulp_dump *dump); + void (*ulp_get_dump_data)(void *handle, u32 seg_id, void *buf, u32 len); }; struct bnxt_fw_msg { @@ -51,6 +71,7 @@ struct bnxt_ulp { u16 max_async_event_id; u16 msix_requested; atomic_t ref_count; + struct bnxt_ulp_dump ulp_dump; }; struct bnxt_en_dev { @@ -119,6 +140,7 @@ void bnxt_ulp_start(struct bnxt *bp, int err); void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs); void bnxt_ulp_irq_stop(struct bnxt *bp); void bnxt_ulp_irq_restart(struct bnxt *bp, int err); +u32 bnxt_get_ulp_dump(struct bnxt *bp, u32 dump_flag, void *buf, u32 *segs); void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl); void bnxt_rdma_aux_device_uninit(struct bnxt *bp); void bnxt_rdma_aux_device_del(struct bnxt *bp); From patchwork Thu Feb 20 18:34:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13984416 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8C18263F54 for ; Thu, 20 Feb 2025 18:56:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077774; cv=none; b=O/sjxU0qkZ61xQeTVn5efxEND8sjBCqLu4feJ80BGt1cCC/lwWh11M1f//2Fuzh2SIGo+qNkbcT4n0CBbrSQJzx1t/GRu6s1zJuaKFV5O8Dw2dtPOjfO+ZQqNcsJa0O4CyHG59kqCihI4rJIeW3MUhBc1Boe8cASBUi476n3vq0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077774; c=relaxed/simple; bh=TmVHhw+YyfHJXm1Tk11ATr1f5Ybq7UumTRqLxFLOUkE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=jzGgz3QRqgqP83Udyb5C56r8hW3mBLcNOvAqYxren5YTqkWVroJLRp1ZnC/m8/6V6uBHWQBN+5qAFCPYDCJa+d7k7nep27t2s/BcIS6QwQ8F4+CA0Ec4ELs5JjCFqBQU5dmUcpyMwmDfQHjj/Kh0cTdB4rIzURl2hO//6sBhxPc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=A4P0ySsn; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="A4P0ySsn" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-22185cddbffso35410905ad.1 for ; Thu, 20 Feb 2025 10:56:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1740077772; x=1740682572; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=wbEO2ZvYX8S6gkbKt5XzFSL0VmDi/kPQc2XpzR/hKDw=; b=A4P0ySsnPzFxVRD6vE8EOCLWwoe7fKSSyqjuK2KBBM8fHFuJlvZvkCwkT4sLlZhhxW EU2ZjwDDw5SwLnwyhj3fF/AnYVUYH7b6h8NIlzUIGdQ/3h+j8vcNnFXA4+aruqTH1vyU LpU52g45bPbV330Je7L38TcenQp0CKnUMKZ54= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740077772; x=1740682572; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wbEO2ZvYX8S6gkbKt5XzFSL0VmDi/kPQc2XpzR/hKDw=; b=DhMnwZKQZNs7Li0MDmxhsfjzBXUJRnOFL2aDpHfdmELdK1yqk9VNS0gsPjU3DtrC2C /SqTuhvTwFumesLt/fQ+5hwCTjgGGiooM/5dfntThhXL6xW0MrFdVr8eyDE2Bqy3kjEi qSIMaj0+zNXYy29JwwbNEa+q2CTY/mX8t0EK9yDFu03sJ/sZTZpTg/RnM7Nxd6iNIN6C evX/QjKl05mKRyxGUFntRnq8C6LraOxZjTSJUnGLwuL/iShRNp0eNZX0HCor5x5aIQYC MoGLVvm4Df142XKqqxeeeqqd7qpVYTsYl8JinJocIkuy6VuwKaI4bRq3Fle6IHbilffn m+nA== X-Gm-Message-State: AOJu0YywpcFXR1OXiKAHDXwaEa3i3okuwjxcTVCinqG0JQ1cTtaKBM67 +TiS/y85Chn6zC6KPm8yBiOogsDARR+rFBZM9MOX8FTcHcSS0Smu/cme3rXCjvMJkoLb9uFPfRA = X-Gm-Gg: ASbGncsj/k6NS/7wlBWoYELnjqsB7a/CkyuRIFpoRwjlpb2bzkkMe09fTZibgDhSP1a RqNoiLMqcwDXhNcxG7/8DX0ufa4Txw+QJXLxA/3bpyPQZWsnb35IbkSOQxnsG6Jvf8wfDuxBiIp l0gPN5vNXE5sfS5aRHcp/NMxKJtOuwzEMTJXM2N3DYYCVGElUliVFDc2rDkdyvM0ih0OPZ+SGC9 mTx4GOyjY0H3Iijo7fsIVfMmev7pbTMnIbiVM8edI1g8/2nqNhmf8ReR8jvghoCnXU1aFVR7/BX PbYYpYDEYJqyqh7IFZG7h7rKh5lvJFAbd2v1S3tHpMXv0kG5r+XT4m5jTE9TBM6063QRfIA= X-Google-Smtp-Source: AGHT+IHvxObTMqiJriOaCFcBkp4iNsiG0ly710DfyEz61MF6mHwZNyTEkxPtzIfoqhMu9KW7oIb1mA== X-Received: by 2002:a05:6a20:7f9c:b0:1ee:a410:4aa5 with SMTP id adf61e73a8af0-1eef3dd0e8bmr314966637.17.1740077771979; Thu, 20 Feb 2025 10:56:11 -0800 (PST) Received: from sxavier-dev.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-addee79a984sm9572262a12.32.2025.02.20.10.56.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:56:11 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, kalesh-anakkur.purayil@broadcom.com, netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, abeni@redhat.com, horms@kernel.org, michael.chan@broadcom.com, Selvin Xavier Subject: [PATCH rdma-next 6/9] RDMA/bnxt_re: Support the dump infrastructure Date: Thu, 20 Feb 2025 10:34:53 -0800 Message-Id: <1740076496-14227-7-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> References: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Michael Chan Adds the stubs for the debug dump in bnxt_re driver. Each segment of the dump data holds a particalar resource type. get_dump_info returns the number of segments and the size of each segments. Driver currently supports segments for QP/CQ/SRQ/MR and for the generic host logs. While taking the dump, get_dump_data will be invoked for each of the segments and bnxt_re driver returns the data associated with each segment. Signed-off-by: Michael Chan Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 85 +++++++++++++++++++++++++++++++++++- 1 file changed, 84 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 6b5a169..76dd0fa 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -497,10 +497,93 @@ static void bnxt_re_start_irq(void *handle, struct bnxt_msix_entry *ent) } } +#define BNXT_SEGMENT_ROCE 255 +#define BNXT_SEGMENT_QP_CTX 256 +#define BNXT_SEGMENT_SRQ_CTX 257 +#define BNXT_SEGMENT_CQ_CTX 258 +#define BNXT_SEGMENT_MR_CTX 270 + +static void bnxt_re_dump_ctx(struct bnxt_re_dev *rdev, u32 seg_id, void *buf, + u32 buf_len) +{ +} + +/* bnxt_re_snapdump - Collect RoCE debug data for coredump. + * @rdev - rdma device instance + * @buf - Pointer to dump buffer + * @buf_len - Buffer length + * + * This function will dump RoCE debug data to the coredump. + * + * Returns: Nothing + * + */ +static void bnxt_re_snapdump(struct bnxt_re_dev *rdev, void *buf, u32 buf_len) +{ +} + +#define BNXT_RE_TRACE_DUMP_SIZE 0x2000000 + +/* bnxt_re_get_dump_info - ULP callback from L2 driver to collect dump info + * @handle - en_dev information. L2 and RoCE device information + * @dump_flags - ethtool dump flags + * @dump - ulp structure containing all coredump segment info + * + * This function is the callback from the L2 driver to provide the list of + * dump segments for the ethtool coredump. + * + * Returns: Nothing + * + */ +static void bnxt_re_get_dump_info(void *handle, u32 dump_flags, + struct bnxt_ulp_dump *dump) +{ + struct bnxt_re_en_dev_info *en_info = auxiliary_get_drvdata(handle); + struct bnxt_ulp_dump_tbl *tbl = dump->seg_tbl; + struct bnxt_re_dev *rdev = en_info->rdev; + + dump->segs = 5; + tbl[0].seg_id = BNXT_SEGMENT_QP_CTX; + tbl[0].seg_len = rdev->rcfw.qp_ctxm_size * BNXT_RE_MAX_QDUMP_ENTRIES; + tbl[1].seg_id = BNXT_SEGMENT_CQ_CTX; + tbl[1].seg_len = rdev->rcfw.cq_ctxm_size * BNXT_RE_MAX_QDUMP_ENTRIES; + tbl[2].seg_id = BNXT_SEGMENT_MR_CTX; + tbl[2].seg_len = rdev->rcfw.mrw_ctxm_size * BNXT_RE_MAX_QDUMP_ENTRIES; + tbl[3].seg_id = BNXT_SEGMENT_SRQ_CTX; + tbl[3].seg_len = rdev->rcfw.srq_ctxm_size * BNXT_RE_MAX_QDUMP_ENTRIES; + tbl[4].seg_id = BNXT_SEGMENT_ROCE; + tbl[4].seg_len = BNXT_RE_TRACE_DUMP_SIZE; +} + +/* bnxt_re_get_dump_data - ULP callback from L2 driver to collect dump data + * @handle - en_dev information. L2 and RoCE device information + * @seg_id - segment ID of the dump + * @buf - dump buffer pointer + * @len - length of the buffer + * + * This function is the callback from the L2 driver to fill the buffer with + * coredump data for each segment. + * + * Returns: Nothing + * + */ +static void bnxt_re_get_dump_data(void *handle, u32 seg_id, void *buf, u32 len) +{ + struct bnxt_re_en_dev_info *en_info = auxiliary_get_drvdata(handle); + struct bnxt_re_dev *rdev = en_info->rdev; + + if (seg_id == BNXT_SEGMENT_ROCE) + return bnxt_re_snapdump(rdev, buf, len); + + bnxt_re_dump_ctx(rdev, seg_id, buf, len); +} + static struct bnxt_ulp_ops bnxt_re_ulp_ops = { .ulp_async_notifier = bnxt_re_async_notifier, .ulp_irq_stop = bnxt_re_stop_irq, - .ulp_irq_restart = bnxt_re_start_irq + .ulp_irq_restart = bnxt_re_start_irq, + .ulp_get_dump_info = bnxt_re_get_dump_info, + .ulp_get_dump_data = bnxt_re_get_dump_data, }; /* RoCE -> Net driver */ From patchwork Thu Feb 20 18:34:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13984417 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC37F265622 for ; Thu, 20 Feb 2025 18:56:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077778; cv=none; b=KBJWKsXpawrkbT+dbyZy0C5JFq1z9qqnDQbUW6jutvFfK3QEdI51raVtMG0NRpzOV0AQSyLrJf0dLQbP6faTIhHrLijZpZvRuCQi81aFKGB/vtWLSn/7l2kRTQ9fYzFYQTOXWvQ8gBnAoFih+1ooJFozb9qm6Mq6P3dRlOAbUPw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077778; c=relaxed/simple; bh=qQmOt74s8s7ggF+1kRfXXp1O+t46qBBm4OZfmbYtmr0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=HASXP0GgFoTkyKW+3l4OFJaHfebYP3Dt18EeZZ/HBN5hxb1Xp34A7ZqQwLco1HXEg07L+TmEI/WXJT8kKkzUr/C4DvTSGHJAHtwdBSdX4iUcAO82gUP0SBp5yBMxA/OyP1OIS/ciwN0pNHon4hup2QA13oReBHinYnPLFKmTOWU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=g8WdLfBS; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="g8WdLfBS" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-221057b6ac4so24353275ad.2 for ; Thu, 20 Feb 2025 10:56:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1740077776; x=1740682576; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=62gcd1c9eE38mVAGq05LCRcGvvb8fU0AO/xGYZf2u2I=; b=g8WdLfBSnGkXhRa46Gf+ocEWvUmmbxaAwBXO94k4mjRdWnGkklA70e7cTXCqco8ZNq LHveT4YHnR9e3wj6sbBsw8+2mQgfnqNvP8JNC8CLc8XJGw3qh+hJp2t3dQIIef6/tNHy SuQueltiNBy5U6Yf8YaYc6dpsi1fGWlhjdQPs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740077776; x=1740682576; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=62gcd1c9eE38mVAGq05LCRcGvvb8fU0AO/xGYZf2u2I=; b=CnhBc+VpbNjmHQZ69Q6sqiBzlqB+DPgpkRMfE70yKsfR6DlPiSEhSiSqXu3CWxv6BT KN08WjMmtKvtYcBz2vTzME69L6YIz1/6duPt7SvXS4GLFAb9If1l+Lk3jPc1ANolixd3 bNe3q8bIIBw65m8JRAt+ZCh8mTVtr0BTXV4SVUn0UX1OVBni8C6cXtU+/b06I27E3FyV chWiv5/8d6e8g8DQc8pqwAda+U038timS9QQ0Zsg4/pdWYjietdO1j+XUjkCvbqbybI9 dvGS+LAPRP05VSwPZqU1FY3w4VtPRdExdasYev1yhtgmH6Vdf58RZgpMr8sJBAMW/sXY nlDw== X-Gm-Message-State: AOJu0YwW5EpQUv0DD2boUI4xlh2OP6dxrmw98zKeRZnLbL9/YvAyNIhF 9RJJpEa88qBiHiuV2zwOtgnf5FBscmxHeJQyVPp6ahzQ1rVVpyhFFWDkeSEMXA== X-Gm-Gg: ASbGncuoYuVGXuE4yqALhg97SZuXcpIc4g7Z/3zEcRjjelw3Z0RaDjMGmfoqRVdaz4K 7TUPoXjXP6eFpaXp2EBBfqg4Ra4VmllarYo/Nfl7VmYKmou4uLSwXdCAa1KAC70o4uReG3651vp aqWWtEMyNZpHf2YY3n+bJw1yuc00rTjOkA/3Gp3MufvPErO3Yy3JkDELQ8i6simm6v0fNK183aa abdKLLM43eYub4h0PxmSqhWb1VBirLHq7T9xPzK5wZWoXoWy7DngsaKRp4f5LosIgRgAFrPyxCi 4cCupTujlwcq7VayD99TrlFvQ1Jmz+OGhbgS/SpYhjgJE+DQwjG1Zq+4+sPkUk3JNFhMcT8= X-Google-Smtp-Source: AGHT+IEqTR1Tj8xQJb09Um5L3+18Y3HeubsWUM4BCPraVepPaC1Lc0gQGeDiUUcGWKdFr4E2tzR1LQ== X-Received: by 2002:a05:6a21:6b0f:b0:1ee:e24d:8fd2 with SMTP id adf61e73a8af0-1eef3cbdfbdmr519522637.25.1740077776052; Thu, 20 Feb 2025 10:56:16 -0800 (PST) Received: from sxavier-dev.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-addee79a984sm9572262a12.32.2025.02.20.10.56.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:56:15 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, kalesh-anakkur.purayil@broadcom.com, netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, abeni@redhat.com, horms@kernel.org, michael.chan@broadcom.com, Saravanan Vajravel , Selvin Xavier Subject: [PATCH rdma-next 7/9] RDMA/bnxt_re: Dump the debug information in snapdump Date: Thu, 20 Feb 2025 10:34:54 -0800 Message-Id: <1740076496-14227-8-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> References: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Saravanan Vajravel Dump the cached debug information when the L2 driver invokes the get_dump_data hook. Signed-off-by: Saravanan Vajravel Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 109 +++++++++++++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 76dd0fa..afde0ef 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -508,6 +508,106 @@ static void bnxt_re_dump_ctx(struct bnxt_re_dev *rdev, u32 seg_id, void *buf, { } +/* + * bnxt_re_snapdump_qp - Log QP dump to the coredump buffer + * @rdev - Pointer to RoCE device instance + * @buf - Pointer to dump buffer + * @buf_len - Buffer length + * + * This function will invoke ULP logger to capture snapshot of + * SQ/RQ/SCQ/RCQ of a QP in ASCII format. + * + * Returns: Buffer length + */ +static u32 bnxt_re_snapdump_qp(struct bnxt_re_dev *rdev, void *buf, u32 buf_len) +{ + u32 index, len = 0, count = 0; + struct qdump_qpinfo *qpinfo; + struct qdump_array *qdump; + + if (!rdev->qdump_head.qdump) + return 0; + + mutex_lock(&rdev->qdump_head.lock); + index = rdev->qdump_head.index; + while (count < rdev->qdump_head.max_elements) { + count++; + index = (!index) ? (rdev->qdump_head.max_elements - 1) : (index - 1); + qdump = &rdev->qdump_head.qdump[index]; + if (!qdump->valid || qdump->is_mr) + continue; + + qpinfo = &qdump->qpinfo; + len += snprintf(buf + len, buf_len - len, + "qp_handle 0x%.8x 0x%.8x (xid %d) dest_qp %d type %s state %s ", + (u32)(qpinfo->qp_handle >> 32), + (u32)(qpinfo->qp_handle & 0xFFFFFFFF), + qpinfo->id, qpinfo->dest_qpid, + __to_qp_type_str(qpinfo->type), + __to_qp_state_str(qpinfo->state)); + len += snprintf(buf + len, buf_len - len, + "is_usr %d scq_handle 0x%llx rcq_handle 0x%llx", + qpinfo->is_user, qpinfo->scq_handle, qpinfo->rcq_handle); + len += snprintf(buf + len, buf_len - len, + "scq_id %d rcq_id %d\n", + qpinfo->scq_id, qpinfo->rcq_id); + if (len >= buf_len) + goto dump_full; + + if (len >= buf_len) + goto dump_full; + } + mutex_unlock(&rdev->qdump_head.lock); + return len; + +dump_full: + mutex_unlock(&rdev->qdump_head.lock); + return buf_len; +} + +/* + * bnxt_re_snapdump_mr - Log PBL of MR to the coredump buffer + * @rdev - Pointer to RoCE device instance + * @buf - Pointer to dump buffer + * @buf_len - Buffer length + * + * This function will invoke ULP logger to capture PBL list of + * Memory Region in ASCII format. + * + * Returns: Buffer length + */ +static u32 bnxt_re_snapdump_mr(struct bnxt_re_dev *rdev, void *buf, u32 buf_len) +{ + u32 index, count = 0, len = 0; + struct qdump_array *qdump; + + if (!rdev->qdump_head.qdump) + return 0; + + mutex_lock(&rdev->qdump_head.lock); + index = rdev->qdump_head.index; + while (count < rdev->qdump_head.max_elements) { + count++; + index = (!index) ? (rdev->qdump_head.max_elements - 1) : (index - 1); + qdump = &rdev->qdump_head.qdump[index]; + if (!qdump->valid || !qdump->is_mr) + continue; + + len += snprintf(buf + len, buf_len - len, + "**MemoryRegion: type %d lkey 0x%x rkey 0x%x tot_sz %llu **\n", + qdump->mrinfo.type, qdump->mrinfo.lkey, + qdump->mrinfo.rkey, qdump->mrinfo.total_size); + if (len >= buf_len) + goto dump_full; + } + mutex_unlock(&rdev->qdump_head.lock); + return len; + +dump_full: + mutex_unlock(&rdev->qdump_head.lock); + return buf_len; +} + /* bnxt_re_snapdump - Collect RoCE debug data for coredump. * @rdev - rdma device instance * @buf - Pointer to dump buffer @@ -520,6 +620,15 @@ static void bnxt_re_dump_ctx(struct bnxt_re_dev *rdev, u32 seg_id, void *buf, */ static void bnxt_re_snapdump(struct bnxt_re_dev *rdev, void *buf, u32 buf_len) { + u32 len = 0; + + len += bnxt_re_snapdump_qp(rdev, buf + len, buf_len - len); + if (len >= buf_len) + return; + + len += bnxt_re_snapdump_mr(rdev, buf + len, buf_len - len); + if (len >= buf_len) + return; } #define BNXT_RE_TRACE_DUMP_SIZE 0x2000000 From patchwork Thu Feb 20 18:34:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13984418 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCFB32641DE for ; Thu, 20 Feb 2025 18:56:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077782; cv=none; b=e4ISIzbQGqzZ5FG1WbeAveAqAeq46/jaDMbPFuRT0SjOHPVSy19ITbZJ+EqFIE/h144IG8Hb4x7PRBwQ+/VVJvYeGS8JkK3EFo2i+mA0Geqks0mz6pqJ1Zzl92pWXGR3AD4yZfRYehHVOaj+Gbd70rmL8R3IHcAMFghAfFPe950= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077782; c=relaxed/simple; bh=ps9j0Wix5FxOg6J4lOpwhIo0pwUPZtapGIIm3Ker8C8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=XBMASTO+9IiRWbYeVkfbvLwb1toZq7PfZqn2Jw+AGSYAPuR+T9T/9yLvidpZRZHvwblLZyvi+q2vF2xcAxBxoZt1zCUZA2/jXWqaWEoBdTAdxdDbraZrnkIvm3kVUsbHNszX88xyX+0xr5NxEmmrZbMdmvkP+JODJ83BC+jc3E8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=bUGHchtp; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="bUGHchtp" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-220d601886fso20261525ad.1 for ; Thu, 20 Feb 2025 10:56:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1740077780; x=1740682580; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=tif6h+vIQ6/biBxnmUJxR84piPWX45JJ651sYhwJ4MA=; b=bUGHchtplBXLKm8Rk3lJo84LP25Yj4pfjWc7sMUIBtgO8w7ZhoV0/di4HrhBAt6qbD LwE4rcicn4EQnf/tndGKWC3h357DwLjSRyrKKz0LZ/r4XgxrD28jzDLtySI1YL8+/9ha yWqmbC9q7KkVOCIu4wdA91gsxB7gjZsb6Re4w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740077780; x=1740682580; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tif6h+vIQ6/biBxnmUJxR84piPWX45JJ651sYhwJ4MA=; b=Mu75OpQgThfFQH3GBVC9oJgE/7ZTpWRtiVSTWnpao1HuQc2KNac3Ra7WWVTorErf9Q Zxp91U+W4srN7/JCNVLl2FrjacWzZGwMTJTncGsinBuTiSirn+5se849QetadRyqpq4G R2ldcwPnQtk9UFoEn2D9oYKBPtiIe8B/f8IA8Ds8qNLYEUBq9xMxLEiDPFssK5dGqFt/ j2jx3zGGkJJN4VPIMHziS7xP4DRV0gbyfBT+WtPnt59EHfU0aLIKWxZ9dcVwUcz7zisn M0b5eB7/J36v4lJ9Wtcxsa5exDFl5X4l14G1ZEkxLh2ptSsHtN31iDqBaJ3gsszitFcY RxQQ== X-Gm-Message-State: AOJu0Yx8fwJ3rO0NgRDcDq4uqnDAHaV9s26FZGqaKfpY5TWG0Ih0SXXT 1zq/9PDhOErNa21Na/vyQe4Y/5CZtjZyHklq7nEESZ2aOP3gRu1dcAzAGawkiw== X-Gm-Gg: ASbGncsz+OjXn/+uxsJ/PyIIxtspH9t4R04vU9uW9ltLDvJcvrmN707sUsh7WBu4zrK uJ83I71dnrHdvKJKDcWkYFgTilXBgNDYbDHC1gLnk999gWh6SLTKbPvfZDczNWt8UoFnWhX4FEC mq5nLuGK2MgdqUObrb8knpYCHGmKhQuN4owimgTSThLbKFSithxEGL/81mwUVkDaHgYPEySz3Lv Ky8PSnSof4EiAJG/G6eWE8m9qQx38H5nFswmxD6fg6FCSE0RPLcacTGVaeaH2T8tdPsgsxFtBEA UATeGEyaEn+v5Vq6lr451f+NUImNshuBbl9m0mIraXTH4drxflxePx8IrBthrr1ohkbdFuo= X-Google-Smtp-Source: AGHT+IFw4WLUXvlZeVZ7DNjlcwoAVhBgMp/91u56hwsVYXz4XkQ5BCYpHuWdsyyiEkUv1s+GRqSPCw== X-Received: by 2002:a05:6a21:6b0f:b0:1ee:c093:e23c with SMTP id adf61e73a8af0-1eef3db70e6mr504945637.41.1740077780108; Thu, 20 Feb 2025 10:56:20 -0800 (PST) Received: from sxavier-dev.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-addee79a984sm9572262a12.32.2025.02.20.10.56.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:56:19 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, kalesh-anakkur.purayil@broadcom.com, netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, abeni@redhat.com, horms@kernel.org, michael.chan@broadcom.com, Kashyap Desai , Selvin Xavier Subject: [PATCH rdma-next 8/9] RDMA/bnxt_re: Dump the HW context information Date: Thu, 20 Feb 2025 10:34:55 -0800 Message-Id: <1740076496-14227-9-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> References: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Kashyap Desai Dump the cached HW context information for QP/CQ/SRQ/MRs when the L2 driver queries using get_dump_data Signed-off-by: Kashyap Desai Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 10 ++++----- drivers/infiniband/hw/bnxt_re/main.c | 38 ++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index a1ee6ca..dca435e 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -1079,7 +1079,7 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata) struct bnxt_qplib_nq *rcq_nq = NULL; unsigned int flags; void *ctx_sb_data = NULL; - bool do_snapdump; + bool do_snapdump = false; u16 ctx_size; int rc; @@ -1087,17 +1087,15 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata) bnxt_re_debug_rem_qpinfo(rdev, qp); bnxt_qplib_flush_cqn_wq(&qp->qplib_qp); - ctx_size = qplib_qp->ctx_size_sb; + ctx_size = rdev->rcfw.qp_ctxm_size; if (ctx_size) ctx_sb_data = vzalloc(ctx_size); - do_snapdump = test_bit(QP_FLAGS_CAPTURE_SNAPDUMP, &qplib_qp->flags); rc = bnxt_qplib_destroy_qp(&rdev->qplib_res, &qp->qplib_qp, ctx_size, ctx_sb_data); if (rc) ibdev_err(&rdev->ibdev, "Failed to destroy HW QP"); - else - bnxt_re_save_resource_context(rdev, ctx_sb_data, - BNXT_RE_RES_TYPE_QP, do_snapdump); + do_snapdump = test_bit(QP_FLAGS_CAPTURE_SNAPDUMP, &qplib_qp->flags); + bnxt_re_save_resource_context(rdev, ctx_sb_data, BNXT_RE_RES_TYPE_QP, do_snapdump); vfree(ctx_sb_data); if (rdma_is_kernel_res(&qp->ib_qp.res)) { diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index afde0ef..b2bf0d0 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -506,6 +506,44 @@ static void bnxt_re_start_irq(void *handle, struct bnxt_msix_entry *ent) static void bnxt_re_dump_ctx(struct bnxt_re_dev *rdev, u32 seg_id, void *buf, u32 buf_len) { + int ctx_index, i; + void *ctx_data; + u16 ctx_size; + + switch (seg_id) { + case BNXT_SEGMENT_QP_CTX: + ctx_data = rdev->rcfw.qp_ctxm_data; + ctx_size = rdev->rcfw.qp_ctxm_size; + ctx_index = rdev->rcfw.qp_ctxm_data_index; + break; + case BNXT_SEGMENT_CQ_CTX: + ctx_data = rdev->rcfw.cq_ctxm_data; + ctx_size = rdev->rcfw.cq_ctxm_size; + ctx_index = rdev->rcfw.cq_ctxm_data_index; + break; + case BNXT_SEGMENT_MR_CTX: + ctx_data = rdev->rcfw.mrw_ctxm_data; + ctx_size = rdev->rcfw.mrw_ctxm_size; + ctx_index = rdev->rcfw.mrw_ctxm_data_index; + break; + case BNXT_SEGMENT_SRQ_CTX: + ctx_data = rdev->rcfw.srq_ctxm_data; + ctx_size = rdev->rcfw.srq_ctxm_size; + ctx_index = rdev->rcfw.srq_ctxm_data_index; + break; + default: + return; + } + + if (!ctx_data || (ctx_size * BNXT_RE_MAX_QDUMP_ENTRIES) > buf_len) + return; + + for (i = ctx_index; i < BNXT_RE_MAX_QDUMP_ENTRIES + ctx_index; i++) { + memcpy(buf, ctx_data + ((i % BNXT_RE_MAX_QDUMP_ENTRIES) * ctx_size), + ctx_size); + buf += ctx_size; + } + } /* From patchwork Thu Feb 20 18:34:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13984419 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B601A263F34 for ; Thu, 20 Feb 2025 18:56:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077786; cv=none; b=syUA6FDWHfHG6pYwtX39rebuchQoWgx0+43faGWebF5EvY7/qlcC25G7WcvKI5VfiIoWe9ER6AFP9jz+pSmxqzE0sm1Z/y+AWsjSh485e6tSeBacLYlU7l0XrOnjhi2CuM7BC3fmksn5LXnSuUySOMho/8K6ikuK1oefZhkKyJE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740077786; c=relaxed/simple; bh=4pokzCpm/lUAfMKuosqyIEkYPbh7I36XpLT3BcQvRag=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=CNgwsHciAhkUEfSm8hldWIIgQX/He+4F1m1g/AHwla0Kw1RYNZx7tFeIOYrAheuU8b7IAzVB9sOGmrvn21VpOkrCudTQxJtPjh+HTSABtfyhjaOYzql6NTbTo7ttKxGnPYVmJ7n9oc8bFbz/ROITDHKvb0tRs304fWGCAUj9r3U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=EmQo9PuX; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="EmQo9PuX" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-221206dbd7eso25880915ad.2 for ; Thu, 20 Feb 2025 10:56:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1740077784; x=1740682584; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=WE82c+aiEEEUeKpSOXCYU/WI+Y4+5eTNi2kEnvfIaks=; b=EmQo9PuXIak/V3eZp/81+nrMNRvZYjhLMhXsQnRdGJMr2k0PpMfD0wlzOk5KW0I8lV jqemdsPa3M2F83mpCZ2EkCyQKaOpiQwuyFkqqIe5tQcAI2I1lD9cu/L7IA/HFmu7ydBo TxuWo5khV6fsvbP4K3USLEvTYlsNmmSX4wg0Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740077784; x=1740682584; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WE82c+aiEEEUeKpSOXCYU/WI+Y4+5eTNi2kEnvfIaks=; b=Nhfx2fzoPSS96P+o5p/zNiO7HCVE1i25QaKB5pxzrqYp9WKYJKHqM7PSlDyWPfipsI k4fazowZSwt4aql9i6CuwjozAUr+r4tY0nS5Af59ozTEpW3MyyDy8tawoUkJuZ8k5CyP twRQx2vNSdgLjHuqOlretTgfUpUFy+kqzOHohVH9P8qyFCr0BOxdJGTRN6GvYBOz+Fdm Xa5KvOEyE03ySmzDHR8e13uqouZCThLfiDtDHkjwD1HC17Qschs2Mjp+0/1b8iM1103j KcZwwL2CeXaQN0BhKAV7A6W7FqNHaHbvoNkBCaffu4d/UOIxYIoLV4lLRIkKWbGrsQFb KHxQ== X-Gm-Message-State: AOJu0YxrDqxDDtLNwZ/6J6E8bZQJY/MmpGuQzRS51ci4lDxsEttxcr7x 7P6MfHWj7iCRycXpkHt4vjHUpOlj+L1kBlFbT1u2ysGt1ptrkDK3YcpG7AUlkw== X-Gm-Gg: ASbGncvjylsCBdNle9wwHq7iJlzgvj7llXueZxV0io2JcjTigHMpkXY9YHJlH1ytMsy jU+lZcGu7aJfdRTGyG39zQxuHMxwfpDVon/oexKD758oZmhrdfhLS6yHg3ERPiNnU+8YtxHfFFt Df4OwsVIO++qIazMb5xwbBvkFjETDVpGCcDxkd9dKGSYh41aTq13+YB2UhELgwtOpqgdyT3n/+X YjqciklP4ETiKZudDwpuNoJdx4wa7CCQo2FQOmaCKyq/P/vErOOAiZ2D1Svv6FlQQYNjo4QXVFi bq9Fn3nijqtr0r/zp0iN2kJNKTd7k2OKrzfrtDWhEEAZDvX3+hz0utTQ2OtKibAuFPleG80= X-Google-Smtp-Source: AGHT+IGEA8WDnrG+ENLZvMkfcpeufwauXW2db4AOLTcqYxDOdLCVeXrL0ymSpYLCNBgTU7qcybsNdw== X-Received: by 2002:a05:6a20:7f91:b0:1ee:8435:6b69 with SMTP id adf61e73a8af0-1eef3c4908dmr498654637.1.1740077784029; Thu, 20 Feb 2025 10:56:24 -0800 (PST) Received: from sxavier-dev.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-addee79a984sm9572262a12.32.2025.02.20.10.56.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:56:23 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, kalesh-anakkur.purayil@broadcom.com, netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, abeni@redhat.com, horms@kernel.org, michael.chan@broadcom.com, Selvin Xavier Subject: [PATCH rdma-next 9/9] RDMA/bnxt_re: Add support for changing the snap dump level Date: Thu, 20 Feb 2025 10:34:56 -0800 Message-Id: <1740076496-14227-10-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> References: <1740076496-14227-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add support for changing the default debug dump level. Reviewed-by: Saravanan Vajravel Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 2 ++ drivers/infiniband/hw/bnxt_re/debugfs.c | 49 +++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index 45601635..45d62f6 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -304,6 +304,8 @@ struct bnxt_re_dev { struct workqueue_struct *dcb_wq; /* Head to track all QP dump */ struct bnxt_re_qdump_head qdump_head; + struct dentry *tunables; + struct dentry *snapdump_dbg; u8 snapdump_dbg_lvl; struct dentry *cc_config; struct bnxt_re_dbg_cc_config_params *cc_config_params; diff --git a/drivers/infiniband/hw/bnxt_re/debugfs.c b/drivers/infiniband/hw/bnxt_re/debugfs.c index af91d16..e3fba53 100644 --- a/drivers/infiniband/hw/bnxt_re/debugfs.c +++ b/drivers/infiniband/hw/bnxt_re/debugfs.c @@ -306,6 +306,52 @@ static const struct file_operations bnxt_re_cc_config_ops = { .write = bnxt_re_cc_config_set, }; +static ssize_t bnxt_re_snapdump_dbg_lvl_set(struct file *filp, const char __user *buffer, + size_t count, loff_t *ppos) +{ + struct bnxt_re_dev *rdev = filp->private_data; + char buf[16]; + u32 val; + + if (count >= sizeof(buf)) + return -EINVAL; + + if (copy_from_user(buf, buffer, count)) + return -EFAULT; + + buf[count] = '\0'; + if (kstrtou32(buf, 0, &val)) + return -EINVAL; + + if (val > BNXT_RE_SNAPDUMP_ALL) + return -EINVAL; + + rdev->snapdump_dbg_lvl = val; + + return count; +} + +static ssize_t bnxt_re_snapdump_dbg_lvl_get(struct file *filp, char __user *buffer, + size_t usr_buf_len, loff_t *ppos) +{ + struct bnxt_re_dev *rdev = filp->private_data; + char buf[16]; + int rc; + + rc = snprintf(buf, sizeof(buf), "%d\n", rdev->snapdump_dbg_lvl); + if (rc < 0) + return rc; + + return simple_read_from_buffer(buffer, usr_buf_len, ppos, (u8 *)(buf), rc); +} + +static const struct file_operations bnxt_re_snapdump_dbg_lvl = { + .owner = THIS_MODULE, + .open = simple_open, + .read = bnxt_re_snapdump_dbg_lvl_get, + .write = bnxt_re_snapdump_dbg_lvl_set, +}; + void bnxt_re_debugfs_add_pdev(struct bnxt_re_dev *rdev) { struct pci_dev *pdev = rdev->en_dev->pdev; @@ -329,6 +375,9 @@ void bnxt_re_debugfs_add_pdev(struct bnxt_re_dev *rdev) rdev->cc_config, tmp_params, &bnxt_re_cc_config_ops); } + rdev->tunables = debugfs_create_dir("tunables", rdev->dbg_root); + rdev->snapdump_dbg = debugfs_create_file("snapdump_dbg_lvl", 0400, rdev->tunables, rdev, + &bnxt_re_snapdump_dbg_lvl); } void bnxt_re_debugfs_rem_pdev(struct bnxt_re_dev *rdev)