From patchwork Thu May 16 15:20:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ranjan Kumar X-Patchwork-Id: 13666323 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8FBB1E491 for ; Thu, 16 May 2024 15:23:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715872999; cv=none; b=RXCT3mNkrJ+9N++a5YSsiYdX0mMSj7Abytunh0v3/bM9fQSpfe7SUXlbk7oKgJf7x8NCsD6MEUupnBkN9PKNK9e+YOBMC1w87nEKohtTX4VuKLSo9yhdG1gmQvAro9Yt6gW27GkkoG1P3znYTotrEjJwkxP9ywohvTAssAzt3Hw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715872999; c=relaxed/simple; bh=Oirm+I96aBz5GGnAy/zr/FCvINMvUIMEPfHVp287/c8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=AyUaROAq3aHBgZcvGZ6U27XcUm76dMRj8Ob5/GAe7F8g/9N9QarJsC+1HXXQ512n+hZarJXAjKmZLU+16nIvKoNNocU9zY7pv0dvoVY2QTN81DV2QhdJtpj55uqrN5zJjgERoRTtD5xjaw626eM6lTmqYOo4Ri9ojyKzCLwOWvI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=MCnpImwd; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="MCnpImwd" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6f43ee95078so320750b3a.1 for ; Thu, 16 May 2024 08:23:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1715872996; x=1716477796; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=f3Dq3mk1WS5OdXWgkd92Q0QW7IXBjuwPUCl6iAYZ5ZI=; b=MCnpImwdkx0VsUE8//BYkQ63x0emtU7iO91eid1MLqtparOKgp9cvmMmFvZC8qILsA DpbJLsf/ruLUSMbKqo/CjNLNbkMjT5g5/kG9e2dylTdRAQpaismxPpm3C2jjtwrB/RF9 cQLfBCF4VtSmgkutczuHx3p5FvG9od6LLTd/s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715872996; x=1716477796; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=f3Dq3mk1WS5OdXWgkd92Q0QW7IXBjuwPUCl6iAYZ5ZI=; b=MyD0MA+vGzQAbrGnAsdB/ggMB0OyyZIZLmBHPeKZ/GnOo4P3o+TXLksWHrlvb2HDUi qFpR0sYwgBcQFngxQJo6iB68W7A6xdoaxIr/4o5VRnfpXjZPoDKAExNzxzi1OtNAbdB9 H+R4x3HIn6nBLPYPoqKrdS2ieyeCjxztvnq9DXDFgqsoMZl5yOLFDLoTZ9XhslGLB5Zh RDLOb4HkQENVew2sqQkwZFcJU1lIl4IMCLhujHad6Iw9l5BGqLocC6MBwR3VrA2IJsIR WgqAhpS1E/ywYM11YHMxSiW8KmiFn26ulV6a2Wke/0oVoRXrKkxkGL9F4Dboe82wCSkg snYg== X-Gm-Message-State: AOJu0Yxg58BJqpV3xtLk+zp+L0NKWyb+U6TusAbqpRDv7wiGKIi0IHrj JYk80wLyDDpu93SNhzgcmGNEo1NxIsCH2NMlu6jiPoHzxUKAUs3ECuUTfAvb6gucH4VMnyCPt6/ 7qHZ7jkIr6j1ey2A9M/P0eRgPSCko2u5RCNpZzzL6M0GOOs/weEE8k8aZBHOPm9UgcPdkOzkbnd bSyxrSTu8YtNCK6sRwr9xAycheeR+sXOxBgPQXSu9gBpONA/+z X-Google-Smtp-Source: AGHT+IFhMMok3i/KPc9qDM9FkqW93EmjI1SAPuid0yYMNa6ReQJfzhHQ4OrLPt5siMKHOdjmAiwAug== X-Received: by 2002:a05:6a21:9216:b0:1b0:25b6:a75d with SMTP id adf61e73a8af0-1b025b6a9bdmr3325028637.52.1715872996113; Thu, 16 May 2024 08:23:16 -0700 (PDT) Received: from localhost.localdomain ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-6f4d2a83d3dsm13241749b3a.65.2024.05.16.08.23.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 May 2024 08:23:15 -0700 (PDT) From: Ranjan Kumar To: linux-scsi@vger.kernel.org, martin.petersen@oracle.com Cc: rajsekhar.chundru@broadcom.com, sathya.prakash@broadcom.com, sumit.saxena@broadcom.com, chandrakanth.patil@broadcom.com, prayas.patel@broadcom.com, Ranjan Kumar Subject: [PATCH v3 2/6] mpi3mr: Driver buffer allocation and posting Date: Thu, 16 May 2024 20:50:06 +0530 Message-Id: <20240516152010.88227-3-ranjan.kumar@broadcom.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240516152010.88227-1-ranjan.kumar@broadcom.com> References: <20240516152010.88227-1-ranjan.kumar@broadcom.com> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch adds support for allocating a driver diagnostic buffer and posting it to the firmware for capturing the driver logs in the controller for further debugging. Signed-off-by: Sathya Prakash Signed-off-by: Ranjan Kumar --- drivers/scsi/mpi3mr/mpi/mpi30_tool.h | 13 +++ drivers/scsi/mpi3mr/mpi3mr.h | 21 ++++ drivers/scsi/mpi3mr/mpi3mr_fw.c | 154 +++++++++++++++++++++++++++ 3 files changed, 188 insertions(+) diff --git a/drivers/scsi/mpi3mr/mpi/mpi30_tool.h b/drivers/scsi/mpi3mr/mpi/mpi30_tool.h index 3b960893870f..495933856006 100644 --- a/drivers/scsi/mpi3mr/mpi/mpi30_tool.h +++ b/drivers/scsi/mpi3mr/mpi/mpi30_tool.h @@ -7,6 +7,7 @@ #define MPI3_DIAG_BUFFER_TYPE_TRACE (0x01) #define MPI3_DIAG_BUFFER_TYPE_FW (0x02) +#define MPI3_DIAG_BUFFER_TYPE_DRIVER (0x10) #define MPI3_DIAG_BUFFER_ACTION_RELEASE (0x01) struct mpi3_diag_buffer_post_request { @@ -40,5 +41,17 @@ struct mpi3_diag_buffer_manage_request { __le16 reserved0e; }; +struct mpi3_driver_buffer_header { + __le32 signature; + __le16 header_size; + __le16 rtt_file_header_offset; + __le32 flags; + __le32 circular_buffer_size; + __le32 logical_buffer_end; + __le32 logical_buffer_start; + __le32 ioc_use_only18[2]; + __le32 reserved20[760]; + __le32 reserved_rttrace[256]; +}; #endif diff --git a/drivers/scsi/mpi3mr/mpi3mr.h b/drivers/scsi/mpi3mr/mpi3mr.h index 4ef96c39c832..dc7e8f461826 100644 --- a/drivers/scsi/mpi3mr/mpi3mr.h +++ b/drivers/scsi/mpi3mr/mpi3mr.h @@ -37,6 +37,7 @@ #include #include #include +#include #include #include @@ -195,6 +196,13 @@ extern atomic64_t event_counter; #define MPI3MR_HDB_TRIGGER_TYPE_GLOBAL 3 +/* Driver Host Diag Buffer (drv_db) */ +#define MPI3MR_MIN_DIAG_HOST_BUFFER_SZ ((32 * 1024) + \ + sizeof(struct mpi3_driver_buffer_header)) +#define MPI3MR_DEFAULT_DIAG_HOST_BUFFER_SZ ((512 * 1024) + \ + sizeof(struct mpi3_driver_buffer_header)) +#define MPI3MR_UEFI_DIAG_HOST_BUFFER_OFFSET (16 * 1024) + /* SGE Flag definition */ #define MPI3MR_SGEFLAGS_SYSTEM_SIMPLE_END_OF_LIST \ (MPI3_SGE_FLAGS_ELEMENT_TYPE_SIMPLE | MPI3_SGE_FLAGS_DLAS_SYSTEM | \ @@ -218,6 +226,12 @@ extern atomic64_t event_counter; #define MPI3MR_WRITE_SAME_MAX_LEN_256_BLKS 256 #define MPI3MR_WRITE_SAME_MAX_LEN_2048_BLKS 2048 +/* Driver diag buffer levels */ +enum mpi3mr_drv_db_level { + MRIOC_DRV_DB_DISABLED = 0, + MRIOC_DRV_DB_MINI = 1, + MRIOC_DRV_DB_FULL = 2, +}; /** * struct mpi3mr_nvme_pt_sge - Structure to store SGEs for NVMe @@ -1113,6 +1127,10 @@ struct scmd_priv { * @ioctl_chain_sge: DMA buffer descriptor for IOCTL chain * @ioctl_resp_sge: DMA buffer descriptor for Mgmt cmd response * @ioctl_sges_allocated: Flag for IOCTL SGEs allocated or not + * @drv_diag_buffer: Diagnostic host buffer virtual address + * @drv_diag_buffer_dma: Diagnostic host buffer DMA address + * @drv_diag_buffer_sz: Diagnostic host buffer size + * */ struct mpi3mr_ioc { struct list_head list; @@ -1310,6 +1328,9 @@ struct mpi3mr_ioc { struct diag_buffer_desc diag_buffers[MPI3MR_MAX_NUM_HDB]; struct mpi3_driver_page2 *driver_pg2; spinlock_t trigger_lock; + void *drv_diag_buffer; + dma_addr_t drv_diag_buffer_dma; + u32 drv_diag_buffer_sz; }; /** diff --git a/drivers/scsi/mpi3mr/mpi3mr_fw.c b/drivers/scsi/mpi3mr/mpi3mr_fw.c index fbd6f32f79ce..5937054b3cdb 100644 --- a/drivers/scsi/mpi3mr/mpi3mr_fw.c +++ b/drivers/scsi/mpi3mr/mpi3mr_fw.c @@ -22,6 +22,17 @@ static int poll_queues; module_param(poll_queues, int, 0444); MODULE_PARM_DESC(poll_queues, "Number of queues for io_uring poll mode. (Range 1 - 126)"); +int drv_db_level = 1; +module_param(drv_db_level, int, 0444); +MODULE_PARM_DESC(drv_db_level, "Driver diagnostic buffer level(Default=1).\n\t\t" + "options:\n\t\t" + "0 = disabled: Driver diagnostic buffer not captured\n\t\t" + "1 = minidump: Driver diagnostic buffer captures prints\n\t\t" + "related to specific mrioc instance\n\t\t" + "2 = fulldump: Driver diagnostic buffer captures prints\n\t\t" + "related to specific mrioc instance and complete dmesg logs" + ); + #if defined(writeq) && defined(CONFIG_64BIT) static inline void mpi3mr_writeq(__u64 b, volatile void __iomem *addr) { @@ -872,6 +883,31 @@ static int mpi3mr_setup_isr(struct mpi3mr_ioc *mrioc, u8 setup_one) return retval; } +static const struct { + enum mpi3mr_drv_db_level value; + char *name; +} mpi3mr_drv_db[] = { + { MRIOC_DRV_DB_DISABLED, "disabled (uefi dump is enabled)" }, + { MRIOC_DRV_DB_MINI, "minidump" }, + { MRIOC_DRV_DB_FULL, "fulldump" }, +}; +static const char *mpi3mr_drv_db_name(enum mpi3mr_drv_db_level drv_db_level) +{ + int i; + char *name = NULL; + + /* Start with Disabled */ + name = mpi3mr_drv_db[0].name; + + for (i = 0; i < ARRAY_SIZE(mpi3mr_drv_db); i++) { + if (mpi3mr_drv_db[i].value == drv_db_level) { + name = mpi3mr_drv_db[i].name; + break; + } + } + return name; +} + static const struct { enum mpi3mr_iocstate value; char *name; @@ -1238,6 +1274,102 @@ static int mpi3mr_issue_and_process_mur(struct mpi3mr_ioc *mrioc, return retval; } +/** + * mpi3mr_alloc_issue_host_diag_buf - Allocate and send host diag buffer + * @mrioc: Adapter instance reference + * + * Issue diagnostic buffer post (unconditional) MPI request through admin queue + * and wait for the completion of it or time out. + * + * Return: 0 on success non-zero on failure + */ +static int mpi3mr_alloc_issue_host_diag_buf(struct mpi3mr_ioc *mrioc) +{ + struct mpi3_diag_buffer_post_request diag_buf_post_req; + dma_addr_t buf_dma_addr; + u32 buf_sz; + int retval = -1; + + ioc_info(mrioc, "driver diag buffer level = %s.\n", + mpi3mr_drv_db_name(drv_db_level)); + + if (!mrioc->drv_diag_buffer) { + mrioc->drv_diag_buffer_sz = + MPI3MR_DEFAULT_DIAG_HOST_BUFFER_SZ; + mrioc->drv_diag_buffer = + dma_alloc_coherent(&mrioc->pdev->dev, + mrioc->drv_diag_buffer_sz, + &mrioc->drv_diag_buffer_dma, GFP_KERNEL); + if (!mrioc->drv_diag_buffer) { + mrioc->drv_diag_buffer_sz = + MPI3MR_MIN_DIAG_HOST_BUFFER_SZ; + mrioc->drv_diag_buffer = + dma_alloc_coherent(&mrioc->pdev->dev, + mrioc->drv_diag_buffer_sz, + &mrioc->drv_diag_buffer_dma, GFP_KERNEL); + } + if (!mrioc->drv_diag_buffer) { + ioc_warn(mrioc, "%s:%d:failed to allocate buffer\n", + __func__, __LINE__); + mrioc->drv_diag_buffer_sz = 0; + return retval; + } + /* TBD - memset to Zero once feature is stable */ + memset(mrioc->drv_diag_buffer, 0x55, mrioc->drv_diag_buffer_sz); + } + + buf_dma_addr = mrioc->drv_diag_buffer_dma; + buf_sz = mrioc->drv_diag_buffer_sz; + + memset(&diag_buf_post_req, 0, sizeof(diag_buf_post_req)); + mutex_lock(&mrioc->init_cmds.mutex); + if (mrioc->init_cmds.state & MPI3MR_CMD_PENDING) { + ioc_err(mrioc, "sending driver diag buffer post is failed due to command in use\n"); + mutex_unlock(&mrioc->init_cmds.mutex); + return retval; + } + mrioc->init_cmds.state = MPI3MR_CMD_PENDING; + mrioc->init_cmds.is_waiting = 1; + mrioc->init_cmds.callback = NULL; + diag_buf_post_req.host_tag = cpu_to_le16(MPI3MR_HOSTTAG_INITCMDS); + diag_buf_post_req.function = MPI3_FUNCTION_DIAG_BUFFER_POST; + diag_buf_post_req.type = MPI3_DIAG_BUFFER_TYPE_DRIVER; + diag_buf_post_req.address = le64_to_cpu(buf_dma_addr); + diag_buf_post_req.length = le32_to_cpu(buf_sz); + + init_completion(&mrioc->init_cmds.done); + retval = mpi3mr_admin_request_post(mrioc, &diag_buf_post_req, + sizeof(diag_buf_post_req), 1); + if (retval) { + ioc_err(mrioc, "posting driver diag buffer failed\n"); + goto out_unlock; + } + wait_for_completion_timeout(&mrioc->init_cmds.done, + (MPI3MR_INTADMCMD_TIMEOUT * HZ)); + if (!(mrioc->init_cmds.state & MPI3MR_CMD_COMPLETE)) { + ioc_err(mrioc, "posting driver diag buffer timed out\n"); + mpi3mr_check_rh_fault_ioc(mrioc, + MPI3MR_RESET_FROM_DIAG_BUFFER_POST_TIMEOUT); + retval = -1; + goto out_unlock; + } + retval = 0; + if ((mrioc->init_cmds.ioc_status & MPI3_IOCSTATUS_STATUS_MASK) + != MPI3_IOCSTATUS_SUCCESS) + ioc_warn(mrioc, + "driver diag buffer post returned with ioc_status(0x%04x) log_info(0x%08x)\n", + (mrioc->init_cmds.ioc_status & MPI3_IOCSTATUS_STATUS_MASK), + mrioc->init_cmds.ioc_loginfo); + else + ioc_info(mrioc, "driver diag buffer of size %dKB posted successfully\n", + mrioc->drv_diag_buffer_sz / 1024); + +out_unlock: + mrioc->init_cmds.state = MPI3MR_CMD_NOTUSED; + mutex_unlock(&mrioc->init_cmds.mutex); + return retval; +} + /** * mpi3mr_revalidate_factsdata - validate IOCFacts parameters * during reset/resume @@ -4168,6 +4300,13 @@ int mpi3mr_init_ioc(struct mpi3mr_ioc *mrioc) goto out_failed; } + dprint_reset(mrioc, "posting driver diag buffer\n"); + retval = mpi3mr_alloc_issue_host_diag_buf(mrioc); + if (retval) { + ioc_err(mrioc, "failed to post driver diag buffer\n"); + goto out_failed; + } + ioc_info(mrioc, "controller initialization completed successfully\n"); return retval; out_failed: @@ -4358,6 +4497,13 @@ int mpi3mr_reinit_ioc(struct mpi3mr_ioc *mrioc, u8 is_resume) } else ioc_info(mrioc, "port enable completed successfully\n"); + dprint_reset(mrioc, "posting driver diag buffer\n"); + retval = mpi3mr_alloc_issue_host_diag_buf(mrioc); + if (retval) { + ioc_err(mrioc, "failed to post driver diag buffer\n"); + goto out_failed; + } + ioc_info(mrioc, "controller %s completed successfully\n", (is_resume)?"resume":"re-initialization"); return retval; @@ -4669,6 +4815,14 @@ void mpi3mr_free_mem(struct mpi3mr_ioc *mrioc) } } + if (mrioc->drv_diag_buffer) { + dma_free_coherent(&mrioc->pdev->dev, + mrioc->drv_diag_buffer_sz, mrioc->drv_diag_buffer, + mrioc->drv_diag_buffer_dma); + mrioc->drv_diag_buffer = NULL; + mrioc->drv_diag_buffer_sz = 0; + } + kfree(mrioc->throttle_groups); mrioc->throttle_groups = NULL;