From patchwork Mon Apr 17 19:15:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Justin Tee X-Patchwork-Id: 13214456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39E9DC77B76 for ; Mon, 17 Apr 2023 19:05:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230228AbjDQTFu (ORCPT ); Mon, 17 Apr 2023 15:05:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229595AbjDQTFs (ORCPT ); Mon, 17 Apr 2023 15:05:48 -0400 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E796C7682 for ; Mon, 17 Apr 2023 12:05:41 -0700 (PDT) Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-63b7b18a336so572500b3a.1 for ; Mon, 17 Apr 2023 12:05:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681758341; x=1684350341; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OdJwpaLu2tZb3jZIZvv4q18r0V9tzBrb6V2n0Dk1YgU=; b=giL8wGQ5LZhssEUnSjWN4Jno2n0/IHVqbQxQh9QpRJqBXNCYFfu+l6xt5i33120URw yyzE1AofeK+hueZLlabq0BOrSntgGoPxmKiYMhIR7ta0RecmVnQ3e0VO6efVvCvrN9wr yjmRcgyVs0/yI7ntrzR9W4zH3q3XJz+sm4tu8M/GosubbM2c0wfiWG/lJcKPOgEzrnwh CSLBvWV+9HxO+ju9fjuyfj18giqvFvRVNsNASEASaDLQW8Wdqe7l28ntQp4itNolSAra AHO53Ei7mRtZNUkTXlyAof1/iNLfUlz41gr0UaWReEbpvSFHg30eg5998uXoP0I3606s QgpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681758341; x=1684350341; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OdJwpaLu2tZb3jZIZvv4q18r0V9tzBrb6V2n0Dk1YgU=; b=jX9qzMsXEoSYdu+VRc3pEagKMy7b+SE77z5OrrHl2GAaJ0mniOrs5zTsfJ/xjXgWJt OwCLQXOnI53877Bx9PY8JzCEMo1h4e+brGMIowz7EQCN5E54gSOdZZSG4yocdvpwI61E n7KkwvvhJbGcw0y2omMz+rtjVvu0iQE+67DMcUuRt2nAITVAuhzX47hX3ifmd7kOMp+P P3G2xIpCgqr6ywbaAUXveHVGEmyv3cLBU4nENTu4x9SDAJvebzF70bnQSbI5ht3y6OAn x4AbSBME3QWrnUnsAHoHmK/r0PqUq6O9KPdzYT7yo5FFOAY+x3zAzIsNUghuK1G+a7v+ xXRQ== X-Gm-Message-State: AAQBX9eTF4y60PZZ/oBZGGNizPmlNORHqAglU90dngH5DpOH2TL8y9rV nNgAI9o8/KW1FmGqKy5hqcdh2LgZ3bg= X-Google-Smtp-Source: AKy350YJhWjxoJAdLEY5Jo5XEVNHic/FPjkjSQmfgFmqQN2Tb0VLTAsXNLPeMYbXpHXT0rz87elgmg== X-Received: by 2002:a05:6a00:a06:b0:63d:2d6a:47be with SMTP id p6-20020a056a000a0600b0063d2d6a47bemr2204872pfh.2.1681758341088; Mon, 17 Apr 2023 12:05:41 -0700 (PDT) Received: from dhcp-10-231-55-133.dhcp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id x18-20020aa793b2000000b0063b7c42a070sm4291439pff.68.2023.04.17.12.05.40 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Apr 2023 12:05:40 -0700 (PDT) From: Justin Tee To: linux-scsi@vger.kernel.org Cc: jsmart2021@gmail.com, justin.tee@broadcom.com, Justin Tee Subject: [PATCH 1/7] lpfc: Fix verbose logging for scsi commands issued to SES devices Date: Mon, 17 Apr 2023 12:15:52 -0700 Message-Id: <20230417191558.83100-2-justintee8345@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20230417191558.83100-1-justintee8345@gmail.com> References: <20230417191558.83100-1-justintee8345@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org For SES LUNs with scsi_device sector_size member set to zero, there is no point to log an LBA. When verbose FCP driver logging is enabled, sanity check sector_size before calling scsi_get_lba on a scsi_cmnd. Signed-off-by: Justin Tee --- drivers/scsi/lpfc/lpfc_scsi.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c index e989f130434e..49aa86c477c6 100644 --- a/drivers/scsi/lpfc/lpfc_scsi.c +++ b/drivers/scsi/lpfc/lpfc_scsi.c @@ -4273,7 +4273,8 @@ lpfc_fcp_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, "x%x SNS x%x x%x LBA x%llx Data: x%x x%x\n", cmd->device->id, cmd->device->lun, cmd, cmd->result, *lp, *(lp + 3), - (u64)scsi_get_lba(cmd), + (cmd->device->sector_size) ? + (u64)scsi_get_lba(cmd) : 0, cmd->retries, scsi_get_resid(cmd)); } From patchwork Mon Apr 17 19:15:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Justin Tee X-Patchwork-Id: 13214457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6EB4C77B78 for ; Mon, 17 Apr 2023 19:06:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230199AbjDQTGF (ORCPT ); Mon, 17 Apr 2023 15:06:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229966AbjDQTGB (ORCPT ); Mon, 17 Apr 2023 15:06:01 -0400 Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BFF11FF3; Mon, 17 Apr 2023 12:05:57 -0700 (PDT) Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-63d32d21f95so76969b3a.1; Mon, 17 Apr 2023 12:05:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681758357; x=1684350357; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VBueJxAlG6y1fXZLE74AlFUiUcaS4JSfD7jE9xvrGEI=; b=izLBrfl9pU8gPsAnOkFmAv45xPNom1ln9VCR7+zmiNuEJG3nGnE6/GH4K3dVPJCMHd x7wkoVtQkqe6Fyl6kfIdB3OxuxMhtyZOYg3mhaOm+JpRcFJvJIh/n9LqBbme3xIodfyo Gsl46+yvjshwNLpYYf4l9G3J2FSbmu06LfbQXTHpbl0vwPWq6j2BcH7WjO6GqrkZyTrT ty3uNCj530X3dCIuDEpLDte6p9WkPnzzhNsDfoZgaxRq4AZ3eVwQV4uztfpoRdtJ0Ekr zZqNObpNjfRu8oa6TqPnmqNJUjOjvuyYw+jCq7/71iCCK617CPj9xRjKgfDSlB7npKFG 0I8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681758357; x=1684350357; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VBueJxAlG6y1fXZLE74AlFUiUcaS4JSfD7jE9xvrGEI=; b=Vj5gwshp8VNExVpssSB2Nu/Bn4uzbZWV0EGIpGsCQkj8ICK7RpLXeLs0hFy1cvLG+w Kd0V+PejVwHQCYhPbjRLzm3OafdYwZEt/wHnU84eTI800zeDmNGFgdVhej5XF1aJaah9 IiT6PdiZ9Uh9y/np9INsuBlwWDa3rqGlYDIpwr6HabSQhRb4TzR1XeVx2XzqnmoC8Hay vhtDFsdjzAJgCBYsiwU0S1jgb2rHdl2AZPVCjHUCv+1fUUH9ergB990sb4/KZdMlWCOD PCnmvKrNP8O+eJb5N8l4yK98svBgZ4epueVXopGKeZjx0K9QS61PBdPxFVwgJSzMlse8 P1RQ== X-Gm-Message-State: AAQBX9eod1SDi6f6DANbs2KQRIOlodx2Qd7NyF7cTBeK30rwFjQy0g1Q v61wSGkiUUwrEKC+kuONxgM6kTGIjCU= X-Google-Smtp-Source: AKy350YhVt7So0AR1UMzwcgn7grqW9ryFPzViDC+OqXmjiDhB4rjUfCmWDU7kveSBGiyQnPgH15ueA== X-Received: by 2002:a05:6a20:440e:b0:f0:69de:6165 with SMTP id ce14-20020a056a20440e00b000f069de6165mr1633887pzb.4.1681758356745; Mon, 17 Apr 2023 12:05:56 -0700 (PDT) Received: from dhcp-10-231-55-133.dhcp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id x18-20020aa793b2000000b0063b7c42a070sm4291439pff.68.2023.04.17.12.05.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Apr 2023 12:05:56 -0700 (PDT) From: Justin Tee To: linux-scsi@vger.kernel.org Cc: jsmart2021@gmail.com, justin.tee@broadcom.com, Justin Tee , stable@vger.kernel.org, Dan Carpenter Subject: [PATCH 2/7] lpfc: Fix double free in lpfc_cmpl_els_logo_acc caused by lpfc_nlp_not_used Date: Mon, 17 Apr 2023 12:15:53 -0700 Message-Id: <20230417191558.83100-3-justintee8345@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20230417191558.83100-1-justintee8345@gmail.com> References: <20230417191558.83100-1-justintee8345@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Smatch detected a double free path because lpfc_nlp_not_used releases an ndlp object before reaching lpfc_nlp_put at the end of lpfc_cmpl_els_logo_acc. Remove the outdated lpfc_nlp_not_used routine. In lpfc_mbx_cmpl_ns_reg_login, replace the call with lpfc_nlp_put. In lpfc_cmpl_els_logo_acc, replace the call with lpfc_unreg_rpi and keep the lpfc_nlp_put at the end of the routine. If ndlp's rpi was registered, then lpfc_unreg_rpi's completion routine performs the final ndlp clean up after lpfc_nlp_put is called from lpfc_cmpl_els_logo_acc. Otherwise if ndlp has no rpi registered, the lpfc_nlp_put at the end of lpfc_cmpl_els_logo_acc is the final ndlp clean up. Fixes: 4430f7fd09ec ("scsi: lpfc: Rework locations of ndlp reference taking") Cc: # v5.11+ Reported-by: Dan Carpenter Link: https://lore.kernel.org/all/Y3OefhyyJNKH%2Fiaf@kili/ Signed-off-by: Justin Tee --- drivers/scsi/lpfc/lpfc_crtn.h | 1 - drivers/scsi/lpfc/lpfc_els.c | 30 +++++++----------------------- drivers/scsi/lpfc/lpfc_hbadisc.c | 24 +++--------------------- 3 files changed, 10 insertions(+), 45 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h index b833b983e69d..0b9edde26abd 100644 --- a/drivers/scsi/lpfc/lpfc_crtn.h +++ b/drivers/scsi/lpfc/lpfc_crtn.h @@ -134,7 +134,6 @@ void lpfc_check_nlp_post_devloss(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp); void lpfc_ignore_els_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, struct lpfc_iocbq *rspiocb); -int lpfc_nlp_not_used(struct lpfc_nodelist *ndlp); struct lpfc_nodelist *lpfc_setup_disc_node(struct lpfc_vport *, uint32_t); void lpfc_disc_list_loopmap(struct lpfc_vport *); void lpfc_disc_start(struct lpfc_vport *); diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index 6a15f879e517..a3c8550e9985 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -5205,14 +5205,9 @@ lpfc_els_free_iocb(struct lpfc_hba *phba, struct lpfc_iocbq *elsiocb) * * This routine is the completion callback function to the Logout (LOGO) * Accept (ACC) Response ELS command. This routine is invoked to indicate - * the completion of the LOGO process. It invokes the lpfc_nlp_not_used() to - * release the ndlp if it has the last reference remaining (reference count - * is 1). If succeeded (meaning ndlp released), it sets the iocb ndlp - * field to NULL to inform the following lpfc_els_free_iocb() routine no - * ndlp reference count needs to be decremented. Otherwise, the ndlp - * reference use-count shall be decremented by the lpfc_els_free_iocb() - * routine. Finally, the lpfc_els_free_iocb() is invoked to release the - * IOCB data structure. + * the completion of the LOGO process. If the node has transitioned to NPR, + * this routine unregisters the RPI if it is still registered. The + * lpfc_els_free_iocb() is invoked to release the IOCB data structure. **/ static void lpfc_cmpl_els_logo_acc(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, @@ -5253,19 +5248,9 @@ lpfc_cmpl_els_logo_acc(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, (ndlp->nlp_last_elscmd == ELS_CMD_PLOGI)) goto out; - /* NPort Recovery mode or node is just allocated */ - if (!lpfc_nlp_not_used(ndlp)) { - /* A LOGO is completing and the node is in NPR state. - * Just unregister the RPI because the node is still - * required. - */ + if (ndlp->nlp_flag & NLP_RPI_REGISTERED) lpfc_unreg_rpi(vport, ndlp); - } else { - /* Indicate the node has already released, should - * not reference to it from within lpfc_els_free_iocb. - */ - cmdiocb->ndlp = NULL; - } + } out: /* @@ -5285,9 +5270,8 @@ lpfc_cmpl_els_logo_acc(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, * RPI (Remote Port Index) mailbox command to the @phba. It simply releases * the associated lpfc Direct Memory Access (DMA) buffer back to the pool and * decrements the ndlp reference count held for this completion callback - * function. After that, it invokes the lpfc_nlp_not_used() to check - * whether there is only one reference left on the ndlp. If so, it will - * perform one more decrement and trigger the release of the ndlp. + * function. After that, it invokes the lpfc_drop_node to check + * whether it is appropriate to release the node. **/ void lpfc_mbx_cmpl_dflt_rpi(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb) diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c index 5ba3a9ad9501..67bfdddb897c 100644 --- a/drivers/scsi/lpfc/lpfc_hbadisc.c +++ b/drivers/scsi/lpfc/lpfc_hbadisc.c @@ -4333,13 +4333,14 @@ lpfc_mbx_cmpl_ns_reg_login(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb) /* If the node is not registered with the scsi or nvme * transport, remove the fabric node. The failed reg_login - * is terminal. + * is terminal and forces the removal of the last node + * reference. */ if (!(ndlp->fc4_xpt_flags & (SCSI_XPT_REGD | NVME_XPT_REGD))) { spin_lock_irq(&ndlp->lock); ndlp->nlp_flag &= ~NLP_NPR_2B_DISC; spin_unlock_irq(&ndlp->lock); - lpfc_nlp_not_used(ndlp); + lpfc_nlp_put(ndlp); } if (phba->fc_topology == LPFC_TOPOLOGY_LOOP) { @@ -6704,25 +6705,6 @@ lpfc_nlp_put(struct lpfc_nodelist *ndlp) return ndlp ? kref_put(&ndlp->kref, lpfc_nlp_release) : 0; } -/* This routine free's the specified nodelist if it is not in use - * by any other discovery thread. This routine returns 1 if the - * ndlp has been freed. A return value of 0 indicates the ndlp is - * not yet been released. - */ -int -lpfc_nlp_not_used(struct lpfc_nodelist *ndlp) -{ - lpfc_debugfs_disc_trc(ndlp->vport, LPFC_DISC_TRC_NODE, - "node not used: did:x%x flg:x%x refcnt:x%x", - ndlp->nlp_DID, ndlp->nlp_flag, - kref_read(&ndlp->kref)); - - if (kref_read(&ndlp->kref) == 1) - if (lpfc_nlp_put(ndlp)) - return 1; - return 0; -} - /** * lpfc_fcf_inuse - Check if FCF can be unregistered. * @phba: Pointer to hba context object. From patchwork Mon Apr 17 19:15:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Justin Tee X-Patchwork-Id: 13214458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71780C77B76 for ; Mon, 17 Apr 2023 19:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229681AbjDQTGM (ORCPT ); Mon, 17 Apr 2023 15:06:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32940 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230186AbjDQTGK (ORCPT ); Mon, 17 Apr 2023 15:06:10 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61B5D76A6 for ; Mon, 17 Apr 2023 12:06:00 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-63b78525ac5so840550b3a.0 for ; Mon, 17 Apr 2023 12:06:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681758360; x=1684350360; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=y3HBwzcR3cBD4MiYbP/IRrqSm0Yf6zhUBT3XLShYEg8=; b=OO+f4opA6I7iEKRXltSNaoiuvNNefTVsTHpafBCSJbU70u7iyCu4eAVbYECP79OQcX qlKPDfbvWpkNBcdmLRhAgfn38bFmmGUsf4aI6GzoBtqVGaSPpRDVbq/b1nPKkLto7GMD hPQxp3K+LG2zE46NaTOYjwrzxtVQRYrQMVsi4Snf42F09/bBo9GCpnAdQ2wBQpqToM7I Is1ROZccq4asuo08BOilUTJPknSTBULIIbPrJObqQpoJmWnkoZiQzqM+N+1N9gcE1DYG qeAGOt4tizrvwPKr/inLjgPlvxc9mOxr+zI17GHD3wNTCrLG0MBzoNOz45+AHheprM6V V2Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681758360; x=1684350360; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y3HBwzcR3cBD4MiYbP/IRrqSm0Yf6zhUBT3XLShYEg8=; b=GWAwxfDmyDAs4/u6d6xb+XOidxiXCwXvqdmDl6J1Bm7fC0OFyinTBK7cLGGkQ5pEL1 vzIRaATADW/cAp8/gEnEZEUQvJlR9koOQrrQHg36rqh6QHFRBKcLNr4/CRWEHqplgmDE J+yEhFJlBnVeCj8W+Zyr2aETMJXBisAkF25kLQdIJWFcQNsS3heD7wMR/eYLkn/DzsNc WVCEC3dEuFX4nmtOCqQX/CE0LSG3jYMLFSM6ekNd+hqJHRM+7hmQNlPdwNfPlRXLV5x3 K/5gvQ4jcSkOrOL4e8CsD1FBnXqdEGbISI/g8q/PBYAMS3qvBR0ZikfvTIu5PoUXiMzi K80g== X-Gm-Message-State: AAQBX9e/GMKxM+kXkk3wyJdl+VLVLFmOhesVzMV1n4jqXEyzQqYmkVex 8O0GzPaQrmr+UoeNY9zAzm8ZlI4+LY4= X-Google-Smtp-Source: AKy350Zjd+Hac7Hr433Ic1K5Q4PT68S+lAGB5ZoSNv8nayMdkojY6AxrIPXd4UsfkzTMkvy8KB6sDg== X-Received: by 2002:a05:6a00:4106:b0:635:4f6:2f38 with SMTP id bu6-20020a056a00410600b0063504f62f38mr12578004pfb.2.1681758359775; Mon, 17 Apr 2023 12:05:59 -0700 (PDT) Received: from dhcp-10-231-55-133.dhcp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id x18-20020aa793b2000000b0063b7c42a070sm4291439pff.68.2023.04.17.12.05.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Apr 2023 12:05:59 -0700 (PDT) From: Justin Tee To: linux-scsi@vger.kernel.org Cc: jsmart2021@gmail.com, justin.tee@broadcom.com, Justin Tee Subject: [PATCH 3/7] lpfc: Match lock ordering of lpfc_cmd->buf_lock and hbalock for abort paths Date: Mon, 17 Apr 2023 12:15:54 -0700 Message-Id: <20230417191558.83100-4-justintee8345@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20230417191558.83100-1-justintee8345@gmail.com> References: <20230417191558.83100-1-justintee8345@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The SCSI version of the abort handler routine, lpfc_abort_handler, takes the lpfc_cmd->buf_lock and then phba->hbalock. Make the same change for the NVME abort path, lpfc_nvme_fcp_abort, to have consistent lock ordering logic between the two abort paths. Signed-off-by: Justin Tee --- drivers/scsi/lpfc/lpfc_nvme.c | 44 +++++++++++++++++------------------ 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index adda70423c77..82730a89ecb5 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -1893,38 +1893,38 @@ lpfc_nvme_fcp_abort(struct nvme_fc_local_port *pnvme_lport, pnvme_rport->port_id, pnvme_fcreq); - /* If the hba is getting reset, this flag is set. It is - * cleared when the reset is complete and rings reestablished. - */ - spin_lock_irqsave(&phba->hbalock, flags); - /* driver queued commands are in process of being flushed */ - if (phba->hba_flag & HBA_IOQ_FLUSH) { - spin_unlock_irqrestore(&phba->hbalock, flags); - lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT, - "6139 Driver in reset cleanup - flushing " - "NVME Req now. hba_flag x%x\n", - phba->hba_flag); - return; - } - lpfc_nbuf = freqpriv->nvme_buf; if (!lpfc_nbuf) { - spin_unlock_irqrestore(&phba->hbalock, flags); lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT, "6140 NVME IO req has no matching lpfc nvme " "io buffer. Skipping abort req.\n"); return; } else if (!lpfc_nbuf->nvmeCmd) { - spin_unlock_irqrestore(&phba->hbalock, flags); lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT, "6141 lpfc NVME IO req has no nvme_fcreq " "io buffer. Skipping abort req.\n"); return; } - nvmereq_wqe = &lpfc_nbuf->cur_iocbq; /* Guard against IO completion being called at same time */ - spin_lock(&lpfc_nbuf->buf_lock); + spin_lock_irqsave(&lpfc_nbuf->buf_lock, flags); + + /* If the hba is getting reset, this flag is set. It is + * cleared when the reset is complete and rings reestablished. + */ + spin_lock(&phba->hbalock); + /* driver queued commands are in process of being flushed */ + if (phba->hba_flag & HBA_IOQ_FLUSH) { + spin_unlock(&phba->hbalock); + spin_unlock_irqrestore(&lpfc_nbuf->buf_lock, flags); + lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT, + "6139 Driver in reset cleanup - flushing " + "NVME Req now. hba_flag x%x\n", + phba->hba_flag); + return; + } + + nvmereq_wqe = &lpfc_nbuf->cur_iocbq; /* * The lpfc_nbuf and the mapped nvme_fcreq in the driver's @@ -1971,8 +1971,8 @@ lpfc_nvme_fcp_abort(struct nvme_fc_local_port *pnvme_lport, ret_val = lpfc_sli4_issue_abort_iotag(phba, nvmereq_wqe, lpfc_nvme_abort_fcreq_cmpl); - spin_unlock(&lpfc_nbuf->buf_lock); - spin_unlock_irqrestore(&phba->hbalock, flags); + spin_unlock(&phba->hbalock); + spin_unlock_irqrestore(&lpfc_nbuf->buf_lock, flags); /* Make sure HBA is alive */ lpfc_issue_hb_tmo(phba); @@ -1998,8 +1998,8 @@ lpfc_nvme_fcp_abort(struct nvme_fc_local_port *pnvme_lport, return; out_unlock: - spin_unlock(&lpfc_nbuf->buf_lock); - spin_unlock_irqrestore(&phba->hbalock, flags); + spin_unlock(&phba->hbalock); + spin_unlock_irqrestore(&lpfc_nbuf->buf_lock, flags); return; } From patchwork Mon Apr 17 19:15:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Justin Tee X-Patchwork-Id: 13214459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD226C77B78 for ; Mon, 17 Apr 2023 19:06:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230093AbjDQTGN (ORCPT ); Mon, 17 Apr 2023 15:06:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32940 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229769AbjDQTGM (ORCPT ); Mon, 17 Apr 2023 15:06:12 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 006B665A9 for ; Mon, 17 Apr 2023 12:06:02 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-63b621b1dabso995665b3a.0 for ; Mon, 17 Apr 2023 12:06:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681758362; x=1684350362; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2j9J060vLAlQowpKGJFGHePq4VIe6y0ksKckHdUIZBE=; b=R4V1J6WH+TVeWUAFdSsmADu094z1PhtxcO9B4jEcyWD1Bp0EtFeyilV5DwjJxeTxOX 6RDpxtPn5dLTwbOekln7ejF4pucoxmCEfeG0L9c64C3DGHbkyuwz/J7RUpAGxT9pDFaU OjkoCxhE6HO0evkQeoX33weNxzMJiTap89OPY/Qs87PSfdWE6U1AFHg8FAb8Qij/in3t yHkrQ6OaTy9MkLMavyfVLvaJqGNAkQrc4MnJ8OSd+fAkON5MkD4HelCez0B8bF6xzAJ4 waVb/7/yKy8KgsBd5XYrcpvL8drTYaz/6/dlCn/Pp8eAjSsT3/pBDKHUc5Djj6QR10Ge E26Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681758362; x=1684350362; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2j9J060vLAlQowpKGJFGHePq4VIe6y0ksKckHdUIZBE=; b=hAr4KmCSPE7otfg/Xq6XNEl+K7H3w79JSDE7lLV0+ey5hsEPA4zJG8xq2Ut+E8J2oc /7ntO7uiaWB0SXv8PNYz1HpLQbpMZ4lgmg6mPzlWCX0oW7OZYm2HmvLnAd+Ksna1GKNs ucVCcqqahfvYO/+/Lyb251JqyMX9MbGLUzpXlBkSsaW0/ApWEYQfqmHO/XDv9FCLnR0w znxncBhskwwZ4jD/vsLDTEmhOJYk3cgkf4lJMc+Jo4ymoGAuLwQ5DSH7aH3mHzwilXwN IACTejEsYCn2F4foa1DhtlAZZDYjYBko3kW9gFbBqH1CcQfQEUzV0Eg2Lt3EfBx7X9SJ 7MfQ== X-Gm-Message-State: AAQBX9fw4iERccpWH4rr5jhe9f6y78psT0iBJoIjYpRAt7O5NrcNlRUH u7VQDRoDUXoIrS+DREzOF4SQEKKlPOI= X-Google-Smtp-Source: AKy350Zgf2/3h9i4nbhKfCE96PqM1v57z2j7x6RJb5m1iymssxCMwWo05kZ9r8P+v6bfutjZVHYUgQ== X-Received: by 2002:a05:6a00:310c:b0:624:bf7e:9d8c with SMTP id bi12-20020a056a00310c00b00624bf7e9d8cmr14091152pfb.1.1681758361979; Mon, 17 Apr 2023 12:06:01 -0700 (PDT) Received: from dhcp-10-231-55-133.dhcp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id x18-20020aa793b2000000b0063b7c42a070sm4291439pff.68.2023.04.17.12.06.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Apr 2023 12:06:01 -0700 (PDT) From: Justin Tee To: linux-scsi@vger.kernel.org Cc: jsmart2021@gmail.com, justin.tee@broadcom.com, Justin Tee Subject: [PATCH 4/7] lpfc: Update congestion warning notification period Date: Mon, 17 Apr 2023 12:15:55 -0700 Message-Id: <20230417191558.83100-5-justintee8345@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20230417191558.83100-1-justintee8345@gmail.com> References: <20230417191558.83100-1-justintee8345@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The CMF_SYNC_WQE command is updated to use an 8 bit field sync period. All related variables used to calculate congestion warning notifications are updated to 8 bit fields accordingly. Signed-off-by: Justin Tee --- drivers/scsi/lpfc/lpfc_attr.c | 4 ++-- drivers/scsi/lpfc/lpfc_crtn.h | 2 +- drivers/scsi/lpfc/lpfc_hw4.h | 4 ++-- drivers/scsi/lpfc/lpfc_sli.c | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c index 3863a5341782..21c7ecd3ede5 100644 --- a/drivers/scsi/lpfc/lpfc_attr.c +++ b/drivers/scsi/lpfc/lpfc_attr.c @@ -5858,8 +5858,8 @@ int lpfc_fabric_cgn_frequency = 100; /* 100 ms default */ module_param(lpfc_fabric_cgn_frequency, int, 0444); MODULE_PARM_DESC(lpfc_fabric_cgn_frequency, "Congestion signaling fabric freq"); -int lpfc_acqe_cgn_frequency = 10; /* 10 sec default */ -module_param(lpfc_acqe_cgn_frequency, int, 0444); +unsigned char lpfc_acqe_cgn_frequency = 10; /* 10 sec default */ +module_param(lpfc_acqe_cgn_frequency, byte, 0444); MODULE_PARM_DESC(lpfc_acqe_cgn_frequency, "Congestion signaling ACQE freq"); int lpfc_use_cgn_signal = 1; /* 0 - only use FPINs, 1 - Use signals if avail */ diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h index 0b9edde26abd..f42fb6ebe448 100644 --- a/drivers/scsi/lpfc/lpfc_crtn.h +++ b/drivers/scsi/lpfc/lpfc_crtn.h @@ -663,7 +663,7 @@ extern int lpfc_enable_nvmet_cnt; extern unsigned long long lpfc_enable_nvmet[]; extern int lpfc_no_hba_reset_cnt; extern unsigned long lpfc_no_hba_reset[]; -extern int lpfc_acqe_cgn_frequency; +extern unsigned char lpfc_acqe_cgn_frequency; extern int lpfc_fabric_cgn_frequency; extern int lpfc_use_cgn_signal; diff --git a/drivers/scsi/lpfc/lpfc_hw4.h b/drivers/scsi/lpfc/lpfc_hw4.h index 58fa39c403a0..a42811682ac7 100644 --- a/drivers/scsi/lpfc/lpfc_hw4.h +++ b/drivers/scsi/lpfc/lpfc_hw4.h @@ -4813,8 +4813,8 @@ struct cmf_sync_wqe { #define cmf_sync_cqid_WORD word11 uint32_t read_bytes; uint32_t word13; -#define cmf_sync_period_SHIFT 16 -#define cmf_sync_period_MASK 0x0000ffff +#define cmf_sync_period_SHIFT 24 +#define cmf_sync_period_MASK 0x000000ff #define cmf_sync_period_WORD word13 uint32_t word14; uint32_t word15; diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 8693578888f1..35b1d5d4079f 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -1931,7 +1931,7 @@ lpfc_issue_cmf_sync_wqe(struct lpfc_hba *phba, u32 ms, u64 total) unsigned long iflags; u32 ret_val; u32 atot, wtot, max; - u16 warn_sync_period = 0; + u8 warn_sync_period = 0; /* First address any alarm / warning activity */ atot = atomic_xchg(&phba->cgn_sync_alarm_cnt, 0); From patchwork Mon Apr 17 19:15:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Justin Tee X-Patchwork-Id: 13214460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE622C77B76 for ; Mon, 17 Apr 2023 19:06:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229769AbjDQTGV (ORCPT ); Mon, 17 Apr 2023 15:06:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230301AbjDQTGS (ORCPT ); Mon, 17 Apr 2023 15:06:18 -0400 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4171F5BA7 for ; Mon, 17 Apr 2023 12:06:05 -0700 (PDT) Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-63b7b18a336so572544b3a.1 for ; Mon, 17 Apr 2023 12:06:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681758365; x=1684350365; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FxCTlqIflmSxJx0SNE+LeDybU9chqbGt7VlLZobaq3w=; b=r77SXbILOsBQ/t1MhN4McAyh3i8nW4JqhnYLiTI3FiVYZ69gsaSp+KfnP3hCqpeeQM JfMbCAuOV70bE6K6P6KtMrwpS+ydfsngjM29jjGcs1/j1GZNqGZQlGajU3J2tCZH5Iu3 6vIJ09t0PKrh0sMsFPZe6AJ3tiGdl4SZg0ll/YfCI5qMg7y2AHNCVseHyUBup4T7Fdhq 7myvEN6iXiQo4rcukA3BQAUnywilkkgFkucsA/TAi5PstEWPbBOg0z4CCjSQBp63lHbC OhWOMfivP4G9L2zO2ZzWpcdiHQOLI2wT2gwIokN0vsDOAigha10j14uBpm8ssjtwT48B gcNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681758365; x=1684350365; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FxCTlqIflmSxJx0SNE+LeDybU9chqbGt7VlLZobaq3w=; b=F3zbHh1kD0Q3BE6iORpAw0jSiHX8W+Ut73uLeiRPxZlLZKVNz3QX9/brg/4fcYEaix uxJ/smc43MGHMt1bxluV7Qe1SieSTEdVO5xBOA9bVO4ZFczxfx2t63JZ+J79W8G3x1na 5sEN/Xue3nr2Vg9uidwsD81wKFth4xGelzkfiIFFdfpEmBIzoVYNioGFVg5bx61MAT/2 QOJqE20OBjSAxkt+jek5FSlVmni4nkp8bfvVENHSe1FWjEezwVzEW3PxKj+5ASHNeqb8 G3dLrZGXTmJPVXLjPfqb6OSubNvBUzgSUhh1qrCXWmAY1cLdDuTnQbB7n7PGc5noKkZ+ 7MQA== X-Gm-Message-State: AAQBX9dghSelKj4uwMw3YoZQSVsEwYLtApCuCMWrXdlHtYeu1sbT87tg rWfmFd1bTGE9un40vW/vE9FQOP0X79E= X-Google-Smtp-Source: AKy350aq/MKOGQo5ASGTrw8tsBsbedQhDHFPNCqcvXsN+37lTSz+6t5V9YiBZdUxbCPnXk/XRN8WqA== X-Received: by 2002:a05:6a00:338f:b0:5e4:f141:568b with SMTP id cm15-20020a056a00338f00b005e4f141568bmr13298805pfb.3.1681758364788; Mon, 17 Apr 2023 12:06:04 -0700 (PDT) Received: from dhcp-10-231-55-133.dhcp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id x18-20020aa793b2000000b0063b7c42a070sm4291439pff.68.2023.04.17.12.06.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Apr 2023 12:06:04 -0700 (PDT) From: Justin Tee To: linux-scsi@vger.kernel.org Cc: jsmart2021@gmail.com, justin.tee@broadcom.com, Justin Tee Subject: [PATCH 5/7] lpfc: Add new RCQE status for handling DMA failures Date: Mon, 17 Apr 2023 12:15:56 -0700 Message-Id: <20230417191558.83100-6-justintee8345@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20230417191558.83100-1-justintee8345@gmail.com> References: <20230417191558.83100-1-justintee8345@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org A new RCQE status value indicating DMA failure when transferring asynchronously received data to an RQE is introduced. Such errors are unexpected and handlers are updated to log KERN_ERR and dump lpfc's debug trace buffer to kmsg. Signed-off-by: Justin Tee --- drivers/scsi/lpfc/lpfc_hw4.h | 7 ++-- drivers/scsi/lpfc/lpfc_sli.c | 64 ++++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_hw4.h b/drivers/scsi/lpfc/lpfc_hw4.h index a42811682ac7..082f8a109e55 100644 --- a/drivers/scsi/lpfc/lpfc_hw4.h +++ b/drivers/scsi/lpfc/lpfc_hw4.h @@ -536,9 +536,9 @@ struct sli4_wcqe_xri_aborted { /* completion queue entry structure for rqe completion */ struct lpfc_rcqe { uint32_t word0; -#define lpfc_rcqe_bindex_SHIFT 16 -#define lpfc_rcqe_bindex_MASK 0x0000FFF -#define lpfc_rcqe_bindex_WORD word0 +#define lpfc_rcqe_iv_SHIFT 31 +#define lpfc_rcqe_iv_MASK 0x00000001 +#define lpfc_rcqe_iv_WORD word0 #define lpfc_rcqe_status_SHIFT 8 #define lpfc_rcqe_status_MASK 0x000000FF #define lpfc_rcqe_status_WORD word0 @@ -546,6 +546,7 @@ struct lpfc_rcqe { #define FC_STATUS_RQ_BUF_LEN_EXCEEDED 0x11 /* payload truncated */ #define FC_STATUS_INSUFF_BUF_NEED_BUF 0x12 /* Insufficient buffers */ #define FC_STATUS_INSUFF_BUF_FRM_DISC 0x13 /* Frame Discard */ +#define FC_STATUS_RQ_DMA_FAILURE 0x14 /* DMA failure */ uint32_t word1; #define lpfc_rcqe_fcf_id_v1_SHIFT 0 #define lpfc_rcqe_fcf_id_v1_MASK 0x0000003F diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 35b1d5d4079f..5f979daae9fc 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -14682,6 +14682,38 @@ lpfc_sli4_sp_handle_rcqe(struct lpfc_hba *phba, struct lpfc_rcqe *rcqe) spin_unlock_irqrestore(&phba->hbalock, iflags); workposted = true; break; + case FC_STATUS_RQ_DMA_FAILURE: + lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, + "2564 RQE DMA Error x%x, x%08x x%08x x%08x " + "x%08x\n", + status, rcqe->word0, rcqe->word1, + rcqe->word2, rcqe->word3); + + /* If IV set, no further recovery */ + if (bf_get(lpfc_rcqe_iv, rcqe)) + break; + + /* recycle consumed resource */ + spin_lock_irqsave(&phba->hbalock, iflags); + lpfc_sli4_rq_release(hrq, drq); + dma_buf = lpfc_sli_hbqbuf_get(&phba->hbqs[0].hbq_buffer_list); + if (!dma_buf) { + hrq->RQ_no_buf_found++; + spin_unlock_irqrestore(&phba->hbalock, iflags); + break; + } + hrq->RQ_rcv_buf++; + hrq->RQ_buf_posted--; + spin_unlock_irqrestore(&phba->hbalock, iflags); + lpfc_in_buf_free(phba, &dma_buf->dbuf); + break; + default: + lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, + "2565 Unexpected RQE Status x%x, w0-3 x%08x " + "x%08x x%08x x%08x\n", + status, rcqe->word0, rcqe->word1, + rcqe->word2, rcqe->word3); + break; } out: return workposted; @@ -15203,6 +15235,38 @@ lpfc_sli4_nvmet_handle_rcqe(struct lpfc_hba *phba, struct lpfc_queue *cq, hrq->RQ_no_posted_buf++; /* Post more buffers if possible */ break; + case FC_STATUS_RQ_DMA_FAILURE: + lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, + "2575 RQE DMA Error x%x, x%08x x%08x x%08x " + "x%08x\n", + status, rcqe->word0, rcqe->word1, + rcqe->word2, rcqe->word3); + + /* If IV set, no further recovery */ + if (bf_get(lpfc_rcqe_iv, rcqe)) + break; + + /* recycle consumed resource */ + spin_lock_irqsave(&phba->hbalock, iflags); + lpfc_sli4_rq_release(hrq, drq); + dma_buf = lpfc_sli_rqbuf_get(phba, hrq); + if (!dma_buf) { + hrq->RQ_no_buf_found++; + spin_unlock_irqrestore(&phba->hbalock, iflags); + break; + } + hrq->RQ_rcv_buf++; + hrq->RQ_buf_posted--; + spin_unlock_irqrestore(&phba->hbalock, iflags); + lpfc_rq_buf_free(phba, &dma_buf->hbuf); + break; + default: + lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, + "2576 Unexpected RQE Status x%x, w0-3 x%08x " + "x%08x x%08x x%08x\n", + status, rcqe->word0, rcqe->word1, + rcqe->word2, rcqe->word3); + break; } out: return workposted; From patchwork Mon Apr 17 19:15:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Justin Tee X-Patchwork-Id: 13214462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF6FDC77B78 for ; Mon, 17 Apr 2023 19:06:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230338AbjDQTGX (ORCPT ); Mon, 17 Apr 2023 15:06:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33114 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230344AbjDQTGT (ORCPT ); Mon, 17 Apr 2023 15:06:19 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D96037D8B for ; Mon, 17 Apr 2023 12:06:07 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-63b78525ac5so840565b3a.0 for ; Mon, 17 Apr 2023 12:06:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681758367; x=1684350367; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AauAbRrkKzAkOoMI3uqtBrx3L29jJdgotFLv+mT6OQ8=; b=rf5by25ejE9yCO4h03V2RYCCMsoYEGJ6unqD4r6rDdZnF2Mo51X72vZN5hHwofQHvl 5PWs7LP+OCc21pXW0pIgE65Q2hlNLT4qLcMaqar3Ky0//mWFapxx+nu17jbcWTDioZOU yKkmyLQ5suzDCFPsNeJsxPjpPRfN6q6+lM3T4TIVhjTiURQLqB5r10SSCFfyoVTo5YoF tx9jrQ5TtdpCxHuoE32nxkZqHof8Pa0BinjuO/0CGoFgTpsZVXr6GU/BySJxIGQBhiYj jDjG4b+nqVZdAf8SO7goQV3pPPQQC4CHXOjiaf1FCik2VEQpF1sINEaA9Vkrc7lkcu7R z5og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681758367; x=1684350367; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AauAbRrkKzAkOoMI3uqtBrx3L29jJdgotFLv+mT6OQ8=; b=FfliO2Gybu/MZuZbXw9EaXgnTFv5upFm6BNgK/KQQ6DI4w5Fgs/hwdIujcnvkVrhsE YtnJGpTKE5aahN2SmDk+wnnBmMb7UTZwKW+esYnbB0HGSLaoAj6r7GB+8fCilQYlAD8S v5VgqTNkOia4xSfvxxm2xtR4bUa6PmQ5xR7fZ9lALpZroyk0BxZGPze/Ju2e+w5LMBxz 3ap5icU/qtf+ilYBDhKZGhP1aeeLuebp63WcSy6LfsCPay5M3GO5jwqQ8iEQPqlgcfM1 q7BgOeni4izEbL+GkfLQJ7G+046+EPEBiskq2HwsY3nCQJleSPM3Dhn3VDKLOzZ+jgXg 9APQ== X-Gm-Message-State: AAQBX9d2vocaoLm9Vn81XxTTxPXhW3JfLRNj7li17qmzxfpUSz5vApfN T2s1TACUCJc9hwIwisFniwwabspxZiE= X-Google-Smtp-Source: AKy350YMZaBD5NLCVbRVjY6IEBEJ/ywdpiHkxuapAT8yT9tfcJn4wx0RMF8OS36Pqkf1ZjNB2o7kMg== X-Received: by 2002:a05:6a00:1c9e:b0:63d:344c:f123 with SMTP id y30-20020a056a001c9e00b0063d344cf123mr704820pfw.1.1681758366809; Mon, 17 Apr 2023 12:06:06 -0700 (PDT) Received: from dhcp-10-231-55-133.dhcp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id x18-20020aa793b2000000b0063b7c42a070sm4291439pff.68.2023.04.17.12.06.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Apr 2023 12:06:06 -0700 (PDT) From: Justin Tee To: linux-scsi@vger.kernel.org Cc: jsmart2021@gmail.com, justin.tee@broadcom.com, Justin Tee Subject: [PATCH 6/7] lpfc: Replace blk_irq_poll intr handler with threaded irq Date: Mon, 17 Apr 2023 12:15:57 -0700 Message-Id: <20230417191558.83100-7-justintee8345@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20230417191558.83100-1-justintee8345@gmail.com> References: <20230417191558.83100-1-justintee8345@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org It has been determined that the threaded irq API accomplishes effectively the same performance metrics as blk_irq_poll. As blk_irq_poll is mostly scheduled by the softirqd and handled in softirq context, this is not entirely desired from a fibre channel driver context. A threaded irq model fits cleaner. This patch replaces the blk_irq_poll logic with threaded irq. Signed-off-by: Justin Tee --- drivers/scsi/lpfc/lpfc_crtn.h | 1 + drivers/scsi/lpfc/lpfc_init.c | 26 +-- drivers/scsi/lpfc/lpfc_sli.c | 326 +++++++++++++++++++--------------- drivers/scsi/lpfc/lpfc_sli4.h | 4 +- 4 files changed, 200 insertions(+), 157 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h index f42fb6ebe448..d4e46a08f94d 100644 --- a/drivers/scsi/lpfc/lpfc_crtn.h +++ b/drivers/scsi/lpfc/lpfc_crtn.h @@ -247,6 +247,7 @@ irqreturn_t lpfc_sli_sp_intr_handler(int, void *); irqreturn_t lpfc_sli_fp_intr_handler(int, void *); irqreturn_t lpfc_sli4_intr_handler(int, void *); irqreturn_t lpfc_sli4_hba_intr_handler(int, void *); +irqreturn_t lpfc_sli4_hba_intr_handler_th(int irq, void *dev_id); int lpfc_read_object(struct lpfc_hba *phba, char *s, uint32_t *datap, uint32_t len); diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index a9e36c73cfc5..91bc6527ca84 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -1279,7 +1279,7 @@ lpfc_hb_mbox_cmpl(struct lpfc_hba * phba, LPFC_MBOXQ_t * pmboxq) /* * lpfc_idle_stat_delay_work - idle_stat tracking * - * This routine tracks per-cq idle_stat and determines polling decisions. + * This routine tracks per-eq idle_stat and determines polling decisions. * * Return codes: * None @@ -1290,7 +1290,7 @@ lpfc_idle_stat_delay_work(struct work_struct *work) struct lpfc_hba *phba = container_of(to_delayed_work(work), struct lpfc_hba, idle_stat_delay_work); - struct lpfc_queue *cq; + struct lpfc_queue *eq; struct lpfc_sli4_hdw_queue *hdwq; struct lpfc_idle_stat *idle_stat; u32 i, idle_percent; @@ -1306,10 +1306,10 @@ lpfc_idle_stat_delay_work(struct work_struct *work) for_each_present_cpu(i) { hdwq = &phba->sli4_hba.hdwq[phba->sli4_hba.cpu_map[i].hdwq]; - cq = hdwq->io_cq; + eq = hdwq->hba_eq; - /* Skip if we've already handled this cq's primary CPU */ - if (cq->chann != i) + /* Skip if we've already handled this eq's primary CPU */ + if (eq->chann != i) continue; idle_stat = &phba->sli4_hba.idle_stat[i]; @@ -1333,9 +1333,9 @@ lpfc_idle_stat_delay_work(struct work_struct *work) idle_percent = 100 - idle_percent; if (idle_percent < 15) - cq->poll_mode = LPFC_QUEUE_WORK; + eq->poll_mode = LPFC_QUEUE_WORK; else - cq->poll_mode = LPFC_IRQ_POLL; + eq->poll_mode = LPFC_THREADED_IRQ; idle_stat->prev_idle = wall_idle; idle_stat->prev_wall = wall; @@ -4357,6 +4357,7 @@ lpfc_io_buf_replenish(struct lpfc_hba *phba, struct list_head *cbuf) struct lpfc_sli4_hdw_queue *qp; struct lpfc_io_buf *lpfc_cmd; int idx, cnt; + unsigned long iflags; qp = phba->sli4_hba.hdwq; cnt = 0; @@ -4371,12 +4372,13 @@ lpfc_io_buf_replenish(struct lpfc_hba *phba, struct list_head *cbuf) lpfc_cmd->hdwq_no = idx; lpfc_cmd->hdwq = qp; lpfc_cmd->cur_iocbq.cmd_cmpl = NULL; - spin_lock(&qp->io_buf_list_put_lock); + spin_lock_irqsave(&qp->io_buf_list_put_lock, iflags); list_add_tail(&lpfc_cmd->list, &qp->lpfc_io_buf_list_put); qp->put_io_bufs++; qp->total_io_bufs++; - spin_unlock(&qp->io_buf_list_put_lock); + spin_unlock_irqrestore(&qp->io_buf_list_put_lock, + iflags); } } return cnt; @@ -13114,8 +13116,10 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba) } eqhdl->irq = rc; - rc = request_irq(eqhdl->irq, &lpfc_sli4_hba_intr_handler, 0, - name, eqhdl); + rc = request_threaded_irq(eqhdl->irq, + &lpfc_sli4_hba_intr_handler, + &lpfc_sli4_hba_intr_handler_th, + IRQF_ONESHOT, name, eqhdl); if (rc) { lpfc_printf_log(phba, KERN_WARNING, LOG_INIT, "0486 MSI-X fast-path (%d) " diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 5f979daae9fc..22708f66be64 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -82,7 +82,8 @@ static int lpfc_sli4_post_sgl_list(struct lpfc_hba *, struct list_head *, int); static void lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_queue *eq, - struct lpfc_eqe *eqe); + struct lpfc_eqe *eqe, + enum lpfc_poll_mode poll_mode); static bool lpfc_sli4_mbox_completions_pending(struct lpfc_hba *phba); static bool lpfc_sli4_process_missed_mbox_completions(struct lpfc_hba *phba); static struct lpfc_cqe *lpfc_sli4_cq_get(struct lpfc_queue *q); @@ -629,7 +630,7 @@ lpfc_sli4_eqcq_flush(struct lpfc_hba *phba, struct lpfc_queue *eq) static int lpfc_sli4_process_eq(struct lpfc_hba *phba, struct lpfc_queue *eq, - uint8_t rearm) + u8 rearm, enum lpfc_poll_mode poll_mode) { struct lpfc_eqe *eqe; int count = 0, consumed = 0; @@ -639,7 +640,7 @@ lpfc_sli4_process_eq(struct lpfc_hba *phba, struct lpfc_queue *eq, eqe = lpfc_sli4_eq_get(eq); while (eqe) { - lpfc_sli4_hba_handle_eqe(phba, eq, eqe); + lpfc_sli4_hba_handle_eqe(phba, eq, eqe, poll_mode); __lpfc_sli4_consume_eqe(phba, eq, eqe); consumed++; @@ -7957,7 +7958,7 @@ lpfc_config_cgn_signal(struct lpfc_hba *phba) * lpfc_init_idle_stat_hb - Initialize idle_stat tracking * @phba: pointer to lpfc hba data structure. * - * This routine initializes the per-cq idle_stat to dynamically dictate + * This routine initializes the per-eq idle_stat to dynamically dictate * polling decisions. * * Return codes: @@ -7967,16 +7968,16 @@ static void lpfc_init_idle_stat_hb(struct lpfc_hba *phba) { int i; struct lpfc_sli4_hdw_queue *hdwq; - struct lpfc_queue *cq; + struct lpfc_queue *eq; struct lpfc_idle_stat *idle_stat; u64 wall; for_each_present_cpu(i) { hdwq = &phba->sli4_hba.hdwq[phba->sli4_hba.cpu_map[i].hdwq]; - cq = hdwq->io_cq; + eq = hdwq->hba_eq; - /* Skip if we've already handled this cq's primary CPU */ - if (cq->chann != i) + /* Skip if we've already handled this eq's primary CPU */ + if (eq->chann != i) continue; idle_stat = &phba->sli4_hba.idle_stat[i]; @@ -7985,13 +7986,14 @@ static void lpfc_init_idle_stat_hb(struct lpfc_hba *phba) idle_stat->prev_wall = wall; if (phba->nvmet_support || - phba->cmf_active_mode != LPFC_CFG_OFF) - cq->poll_mode = LPFC_QUEUE_WORK; + phba->cmf_active_mode != LPFC_CFG_OFF || + phba->intr_type != MSIX) + eq->poll_mode = LPFC_QUEUE_WORK; else - cq->poll_mode = LPFC_IRQ_POLL; + eq->poll_mode = LPFC_THREADED_IRQ; } - if (!phba->nvmet_support) + if (!phba->nvmet_support && phba->intr_type == MSIX) schedule_delayed_work(&phba->idle_stat_delay_work, msecs_to_jiffies(LPFC_IDLE_STAT_DELAY)); } @@ -9218,7 +9220,8 @@ lpfc_sli4_process_missed_mbox_completions(struct lpfc_hba *phba) if (mbox_pending) /* process and rearm the EQ */ - lpfc_sli4_process_eq(phba, fpeq, LPFC_QUEUE_REARM); + lpfc_sli4_process_eq(phba, fpeq, LPFC_QUEUE_REARM, + LPFC_QUEUE_WORK); else /* Always clear and re-arm the EQ */ sli4_hba->sli4_write_eq_db(phba, fpeq, 0, LPFC_QUEUE_REARM); @@ -11254,7 +11257,8 @@ inline void lpfc_sli4_poll_eq(struct lpfc_queue *eq) * will be handled through a sched from polling timer * function which is currently triggered every 1msec. */ - lpfc_sli4_process_eq(phba, eq, LPFC_QUEUE_NOARM); + lpfc_sli4_process_eq(phba, eq, LPFC_QUEUE_NOARM, + LPFC_QUEUE_WORK); } /** @@ -14835,7 +14839,6 @@ lpfc_sli4_sp_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe, * @cq: Pointer to CQ to be processed * @handler: Routine to process each cqe * @delay: Pointer to usdelay to set in case of rescheduling of the handler - * @poll_mode: Polling mode we were called from * * This routine processes completion queue entries in a CQ. While a valid * queue element is found, the handler is called. During processing checks @@ -14853,8 +14856,7 @@ lpfc_sli4_sp_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe, static bool __lpfc_sli4_process_cq(struct lpfc_hba *phba, struct lpfc_queue *cq, bool (*handler)(struct lpfc_hba *, struct lpfc_queue *, - struct lpfc_cqe *), unsigned long *delay, - enum lpfc_poll_mode poll_mode) + struct lpfc_cqe *), unsigned long *delay) { struct lpfc_cqe *cqe; bool workposted = false; @@ -14895,10 +14897,6 @@ __lpfc_sli4_process_cq(struct lpfc_hba *phba, struct lpfc_queue *cq, arm = false; } - /* Note: complete the irq_poll softirq before rearming CQ */ - if (poll_mode == LPFC_IRQ_POLL) - irq_poll_complete(&cq->iop); - /* Track the max number of CQEs processed in 1 EQ */ if (count > cq->CQ_max_cqe) cq->CQ_max_cqe = count; @@ -14948,17 +14946,17 @@ __lpfc_sli4_sp_process_cq(struct lpfc_queue *cq) case LPFC_MCQ: workposted |= __lpfc_sli4_process_cq(phba, cq, lpfc_sli4_sp_handle_mcqe, - &delay, LPFC_QUEUE_WORK); + &delay); break; case LPFC_WCQ: if (cq->subtype == LPFC_IO) workposted |= __lpfc_sli4_process_cq(phba, cq, lpfc_sli4_fp_handle_cqe, - &delay, LPFC_QUEUE_WORK); + &delay); else workposted |= __lpfc_sli4_process_cq(phba, cq, lpfc_sli4_sp_handle_cqe, - &delay, LPFC_QUEUE_WORK); + &delay); break; default: lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, @@ -15335,45 +15333,64 @@ lpfc_sli4_fp_handle_cqe(struct lpfc_hba *phba, struct lpfc_queue *cq, } /** - * lpfc_sli4_sched_cq_work - Schedules cq work - * @phba: Pointer to HBA context object. - * @cq: Pointer to CQ - * @cqid: CQ ID - * - * This routine checks the poll mode of the CQ corresponding to - * cq->chann, then either schedules a softirq or queue_work to complete - * cq work. + * __lpfc_sli4_hba_process_cq - Process a fast-path event queue entry + * @cq: Pointer to CQ to be processed * - * queue_work path is taken if in NVMET mode, or if poll_mode is in - * LPFC_QUEUE_WORK mode. Otherwise, softirq path is taken. + * This routine calls the cq processing routine with the handler for + * fast path CQEs. * + * The CQ routine returns two values: the first is the calling status, + * which indicates whether work was queued to the background discovery + * thread. If true, the routine should wakeup the discovery thread; + * the second is the delay parameter. If non-zero, rather than rearming + * the CQ and yet another interrupt, the CQ handler should be queued so + * that it is processed in a subsequent polling action. The value of + * the delay indicates when to reschedule it. **/ -static void lpfc_sli4_sched_cq_work(struct lpfc_hba *phba, - struct lpfc_queue *cq, uint16_t cqid) +static void +__lpfc_sli4_hba_process_cq(struct lpfc_queue *cq) { - int ret = 0; + struct lpfc_hba *phba = cq->phba; + unsigned long delay; + bool workposted = false; + int ret; - switch (cq->poll_mode) { - case LPFC_IRQ_POLL: - /* CGN mgmt is mutually exclusive from softirq processing */ - if (phba->cmf_active_mode == LPFC_CFG_OFF) { - irq_poll_sched(&cq->iop); - break; - } - fallthrough; - case LPFC_QUEUE_WORK: - default: + /* process and rearm the CQ */ + workposted |= __lpfc_sli4_process_cq(phba, cq, lpfc_sli4_fp_handle_cqe, + &delay); + + if (delay) { if (is_kdump_kernel()) - ret = queue_work(phba->wq, &cq->irqwork); + ret = queue_delayed_work(phba->wq, &cq->sched_irqwork, + delay); else - ret = queue_work_on(cq->chann, phba->wq, &cq->irqwork); + ret = queue_delayed_work_on(cq->chann, phba->wq, + &cq->sched_irqwork, delay); if (!ret) lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, - "0383 Cannot schedule queue work " - "for CQ eqcqid=%d, cqid=%d on CPU %d\n", - cqid, cq->queue_id, - raw_smp_processor_id()); + "0367 Cannot schedule queue work " + "for cqid=%d on CPU %d\n", + cq->queue_id, cq->chann); } + + /* wake up worker thread if there are works to be done */ + if (workposted) + lpfc_worker_wake_up(phba); +} + +/** + * lpfc_sli4_hba_process_cq - fast-path work handler when started by + * interrupt + * @work: pointer to work element + * + * translates from the work handler and calls the fast-path handler. + **/ +static void +lpfc_sli4_hba_process_cq(struct work_struct *work) +{ + struct lpfc_queue *cq = container_of(work, struct lpfc_queue, irqwork); + + __lpfc_sli4_hba_process_cq(cq); } /** @@ -15381,6 +15398,7 @@ static void lpfc_sli4_sched_cq_work(struct lpfc_hba *phba, * @phba: Pointer to HBA context object. * @eq: Pointer to the queue structure. * @eqe: Pointer to fast-path event queue entry. + * @poll_mode: poll_mode to execute processing the cq. * * This routine process a event queue entry from the fast-path event queue. * It will check the MajorCode and MinorCode to determine this is for a @@ -15391,11 +15409,12 @@ static void lpfc_sli4_sched_cq_work(struct lpfc_hba *phba, **/ static void lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_queue *eq, - struct lpfc_eqe *eqe) + struct lpfc_eqe *eqe, enum lpfc_poll_mode poll_mode) { struct lpfc_queue *cq = NULL; uint32_t qidx = eq->hdwq; uint16_t cqid, id; + int ret; if (unlikely(bf_get_le32(lpfc_eqe_major_code, eqe) != 0)) { lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, @@ -15455,70 +15474,25 @@ lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_queue *eq, else cq->isr_timestamp = 0; #endif - lpfc_sli4_sched_cq_work(phba, cq, cqid); -} -/** - * __lpfc_sli4_hba_process_cq - Process a fast-path event queue entry - * @cq: Pointer to CQ to be processed - * @poll_mode: Enum lpfc_poll_state to determine poll mode - * - * This routine calls the cq processing routine with the handler for - * fast path CQEs. - * - * The CQ routine returns two values: the first is the calling status, - * which indicates whether work was queued to the background discovery - * thread. If true, the routine should wakeup the discovery thread; - * the second is the delay parameter. If non-zero, rather than rearming - * the CQ and yet another interrupt, the CQ handler should be queued so - * that it is processed in a subsequent polling action. The value of - * the delay indicates when to reschedule it. - **/ -static void -__lpfc_sli4_hba_process_cq(struct lpfc_queue *cq, - enum lpfc_poll_mode poll_mode) -{ - struct lpfc_hba *phba = cq->phba; - unsigned long delay; - bool workposted = false; - int ret = 0; - - /* process and rearm the CQ */ - workposted |= __lpfc_sli4_process_cq(phba, cq, lpfc_sli4_fp_handle_cqe, - &delay, poll_mode); - - if (delay) { + switch (poll_mode) { + case LPFC_THREADED_IRQ: + __lpfc_sli4_hba_process_cq(cq); + break; + case LPFC_QUEUE_WORK: + default: if (is_kdump_kernel()) - ret = queue_delayed_work(phba->wq, &cq->sched_irqwork, - delay); + ret = queue_work(phba->wq, &cq->irqwork); else - ret = queue_delayed_work_on(cq->chann, phba->wq, - &cq->sched_irqwork, delay); + ret = queue_work_on(cq->chann, phba->wq, &cq->irqwork); if (!ret) lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, - "0367 Cannot schedule queue work " - "for cqid=%d on CPU %d\n", - cq->queue_id, cq->chann); + "0383 Cannot schedule queue work " + "for CQ eqcqid=%d, cqid=%d on CPU %d\n", + cqid, cq->queue_id, + raw_smp_processor_id()); + break; } - - /* wake up worker thread if there are works to be done */ - if (workposted) - lpfc_worker_wake_up(phba); -} - -/** - * lpfc_sli4_hba_process_cq - fast-path work handler when started by - * interrupt - * @work: pointer to work element - * - * translates from the work handler and calls the fast-path handler. - **/ -static void -lpfc_sli4_hba_process_cq(struct work_struct *work) -{ - struct lpfc_queue *cq = container_of(work, struct lpfc_queue, irqwork); - - __lpfc_sli4_hba_process_cq(cq, LPFC_QUEUE_WORK); } /** @@ -15533,7 +15507,7 @@ lpfc_sli4_dly_hba_process_cq(struct work_struct *work) struct lpfc_queue *cq = container_of(to_delayed_work(work), struct lpfc_queue, sched_irqwork); - __lpfc_sli4_hba_process_cq(cq, LPFC_QUEUE_WORK); + __lpfc_sli4_hba_process_cq(cq); } /** @@ -15559,8 +15533,9 @@ lpfc_sli4_dly_hba_process_cq(struct work_struct *work) * and returns for these events. This function is called without any lock * held. It gets the hbalock to access and update SLI data structures. * - * This function returns IRQ_HANDLED when interrupt is handled else it - * returns IRQ_NONE. + * This function returns IRQ_HANDLED when interrupt is handled, IRQ_WAKE_THREAD + * when interrupt is scheduled to be handled from a threaded irq context, or + * else returns IRQ_NONE. **/ irqreturn_t lpfc_sli4_hba_intr_handler(int irq, void *dev_id) @@ -15569,8 +15544,8 @@ lpfc_sli4_hba_intr_handler(int irq, void *dev_id) struct lpfc_hba_eq_hdl *hba_eq_hdl; struct lpfc_queue *fpeq; unsigned long iflag; - int ecount = 0; int hba_eqidx; + int ecount = 0; struct lpfc_eq_intr_info *eqi; /* Get the driver's phba structure from the dev_id */ @@ -15599,30 +15574,41 @@ lpfc_sli4_hba_intr_handler(int irq, void *dev_id) return IRQ_NONE; } - eqi = this_cpu_ptr(phba->sli4_hba.eq_info); - eqi->icnt++; - - fpeq->last_cpu = raw_smp_processor_id(); + switch (fpeq->poll_mode) { + case LPFC_THREADED_IRQ: + /* CGN mgmt is mutually exclusive from irq processing */ + if (phba->cmf_active_mode == LPFC_CFG_OFF) + return IRQ_WAKE_THREAD; + fallthrough; + case LPFC_QUEUE_WORK: + default: + eqi = this_cpu_ptr(phba->sli4_hba.eq_info); + eqi->icnt++; - if (eqi->icnt > LPFC_EQD_ISR_TRIGGER && - fpeq->q_flag & HBA_EQ_DELAY_CHK && - phba->cfg_auto_imax && - fpeq->q_mode != LPFC_MAX_AUTO_EQ_DELAY && - phba->sli.sli_flag & LPFC_SLI_USE_EQDR) - lpfc_sli4_mod_hba_eq_delay(phba, fpeq, LPFC_MAX_AUTO_EQ_DELAY); + fpeq->last_cpu = raw_smp_processor_id(); - /* process and rearm the EQ */ - ecount = lpfc_sli4_process_eq(phba, fpeq, LPFC_QUEUE_REARM); + if (eqi->icnt > LPFC_EQD_ISR_TRIGGER && + fpeq->q_flag & HBA_EQ_DELAY_CHK && + phba->cfg_auto_imax && + fpeq->q_mode != LPFC_MAX_AUTO_EQ_DELAY && + phba->sli.sli_flag & LPFC_SLI_USE_EQDR) + lpfc_sli4_mod_hba_eq_delay(phba, fpeq, + LPFC_MAX_AUTO_EQ_DELAY); - if (unlikely(ecount == 0)) { - fpeq->EQ_no_entry++; - if (phba->intr_type == MSIX) - /* MSI-X treated interrupt served as no EQ share INT */ - lpfc_printf_log(phba, KERN_WARNING, LOG_SLI, - "0358 MSI-X interrupt with no EQE\n"); - else - /* Non MSI-X treated on interrupt as EQ share INT */ - return IRQ_NONE; + /* process and rearm the EQ */ + ecount = lpfc_sli4_process_eq(phba, fpeq, LPFC_QUEUE_REARM, + LPFC_QUEUE_WORK); + + if (unlikely(ecount == 0)) { + fpeq->EQ_no_entry++; + if (phba->intr_type == MSIX) + /* MSI-X treated interrupt served as no EQ share INT */ + lpfc_printf_log(phba, KERN_WARNING, LOG_SLI, + "0358 MSI-X interrupt with no EQE\n"); + else + /* Non MSI-X treated on interrupt as EQ share INT */ + return IRQ_NONE; + } } return IRQ_HANDLED; @@ -16179,13 +16165,69 @@ lpfc_eq_create(struct lpfc_hba *phba, struct lpfc_queue *eq, uint32_t imax) return status; } -static int lpfc_cq_poll_hdler(struct irq_poll *iop, int budget) +/** + * lpfc_sli4_hba_intr_handler_th - SLI4 HBA threaded interrupt handler + * @irq: Interrupt number. + * @dev_id: The device context pointer. + * + * This routine is a mirror of lpfc_sli4_hba_intr_handler, but executed within + * threaded irq context. + * + * Returns + * IRQ_HANDLED - interrupt is handled + * IRQ_NONE - otherwise + **/ +irqreturn_t lpfc_sli4_hba_intr_handler_th(int irq, void *dev_id) { - struct lpfc_queue *cq = container_of(iop, struct lpfc_queue, iop); + struct lpfc_hba *phba; + struct lpfc_hba_eq_hdl *hba_eq_hdl; + struct lpfc_queue *fpeq; + int ecount = 0; + int hba_eqidx; + struct lpfc_eq_intr_info *eqi; + + /* Get the driver's phba structure from the dev_id */ + hba_eq_hdl = (struct lpfc_hba_eq_hdl *)dev_id; + phba = hba_eq_hdl->phba; + hba_eqidx = hba_eq_hdl->idx; - __lpfc_sli4_hba_process_cq(cq, LPFC_IRQ_POLL); + if (unlikely(!phba)) + return IRQ_NONE; + if (unlikely(!phba->sli4_hba.hdwq)) + return IRQ_NONE; - return 1; + /* Get to the EQ struct associated with this vector */ + fpeq = phba->sli4_hba.hba_eq_hdl[hba_eqidx].eq; + if (unlikely(!fpeq)) + return IRQ_NONE; + + eqi = per_cpu_ptr(phba->sli4_hba.eq_info, raw_smp_processor_id()); + eqi->icnt++; + + fpeq->last_cpu = raw_smp_processor_id(); + + if (eqi->icnt > LPFC_EQD_ISR_TRIGGER && + fpeq->q_flag & HBA_EQ_DELAY_CHK && + phba->cfg_auto_imax && + fpeq->q_mode != LPFC_MAX_AUTO_EQ_DELAY && + phba->sli.sli_flag & LPFC_SLI_USE_EQDR) + lpfc_sli4_mod_hba_eq_delay(phba, fpeq, LPFC_MAX_AUTO_EQ_DELAY); + + /* process and rearm the EQ */ + ecount = lpfc_sli4_process_eq(phba, fpeq, LPFC_QUEUE_REARM, + LPFC_THREADED_IRQ); + + if (unlikely(ecount == 0)) { + fpeq->EQ_no_entry++; + if (phba->intr_type == MSIX) + /* MSI-X treated interrupt served as no EQ share INT */ + lpfc_printf_log(phba, KERN_WARNING, LOG_SLI, + "3358 MSI-X interrupt with no EQE\n"); + else + /* Non MSI-X treated on interrupt as EQ share INT */ + return IRQ_NONE; + } + return IRQ_HANDLED; } /** @@ -16329,8 +16371,6 @@ lpfc_cq_create(struct lpfc_hba *phba, struct lpfc_queue *cq, if (cq->queue_id > phba->sli4_hba.cq_max) phba->sli4_hba.cq_max = cq->queue_id; - - irq_poll_init(&cq->iop, LPFC_IRQ_POLL_WEIGHT, lpfc_cq_poll_hdler); out: mempool_free(mbox, phba->mbox_mem_pool); return status; diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h index 2a0864e6d7cd..2541a8fba093 100644 --- a/drivers/scsi/lpfc/lpfc_sli4.h +++ b/drivers/scsi/lpfc/lpfc_sli4.h @@ -140,7 +140,7 @@ struct lpfc_rqb { enum lpfc_poll_mode { LPFC_QUEUE_WORK, - LPFC_IRQ_POLL + LPFC_THREADED_IRQ, }; struct lpfc_idle_stat { @@ -279,8 +279,6 @@ struct lpfc_queue { struct list_head _poll_list; void **q_pgs; /* array to index entries per page */ -#define LPFC_IRQ_POLL_WEIGHT 256 - struct irq_poll iop; enum lpfc_poll_mode poll_mode; }; From patchwork Mon Apr 17 19:15:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Justin Tee X-Patchwork-Id: 13214461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C959C77B72 for ; Mon, 17 Apr 2023 19:06:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230323AbjDQTGW (ORCPT ); Mon, 17 Apr 2023 15:06:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230338AbjDQTGT (ORCPT ); Mon, 17 Apr 2023 15:06:19 -0400 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59C135BB3 for ; Mon, 17 Apr 2023 12:06:09 -0700 (PDT) Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-63b875d0027so587681b3a.1 for ; Mon, 17 Apr 2023 12:06:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681758369; x=1684350369; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yKhXi+iyEvxzRcipT/f2A4QIc1t/0KyNVutF7WgeQJM=; b=fLw7dASH0J2UnDGrydIxRsad5YviDEoOdmUmJoP3IcNkhL29R8a1fDu81Nd+TPehro XbBvngRtHwq+9qHjl3ftR2y4RmwMDIxEivV7b1tCOqy+ol5FMmQnVKLuw/X3nhArpXm8 pxJUZIz0KSy/y32kRD1kCSrarOzUrp5sT5tT48EFVQsAU2cMC7h3G2p60RTPsCzLD4je 8ru6gXiiqAQzNOjLwtngE8aPnywicYX+lQGnnznYBlEffeab5OlXvLPu6g1kouLrgCWp V7+LHWRuCHQpqtsHAYIieC5P45Ss7m8M+3kREBEh0jTBxJ9FvluM7KsDItvjICmlyR8o 8i3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681758369; x=1684350369; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yKhXi+iyEvxzRcipT/f2A4QIc1t/0KyNVutF7WgeQJM=; b=hHUOlLduNMziF5HRTpdZCtQSVPccp0n36uDa7h8fe5TUTr9NDDxUlz+8YWjfbWMiBt K93Abt3rjBRvInuR1NPiFd4Be1e+hRxc2VGEp9a90Efnevp7FNxUvkbzVAERdefbPbjW VIFGuxR03RBpG9suqJ5Fd4h5zylIjpHXMIy88TkgaZdxJSRqhVBqf7nvO8qv6dxmpPnI JT5iKKiEr2gQVjziQ+nKSgUKfsaQLm3WXP1SgvBqMRiEMpyCLY+Hmz+JPNOEyf8sXsgo lIbc0nBi2Q7QF4vTRVGhW1PaWlM8TGMADBof7Rxfr5CUaAKvgUiI0aTxtLrTaDUPM+lG BoNg== X-Gm-Message-State: AAQBX9efbusKTAFj7wlg3Yohhh9xagTaeRwKyo5KLyrsKbssaUKKnhoF bK+3RWuaVZCIdtIaE25kPc4dk9WI4eE= X-Google-Smtp-Source: AKy350bfTQfFHCXRqUWB28Q7HVOPmAomMJXvr50EL9fdZrXFkQAvxujW9Usu/fWcNsFB3Oa2JJN+VA== X-Received: by 2002:a05:6a00:1c9e:b0:63d:344c:f123 with SMTP id y30-20020a056a001c9e00b0063d344cf123mr704898pfw.1.1681758368972; Mon, 17 Apr 2023 12:06:08 -0700 (PDT) Received: from dhcp-10-231-55-133.dhcp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id x18-20020aa793b2000000b0063b7c42a070sm4291439pff.68.2023.04.17.12.06.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Apr 2023 12:06:08 -0700 (PDT) From: Justin Tee To: linux-scsi@vger.kernel.org Cc: jsmart2021@gmail.com, justin.tee@broadcom.com, Justin Tee Subject: [PATCH 7/7] lpfc: Update lpfc version to 14.2.0.12 Date: Mon, 17 Apr 2023 12:15:58 -0700 Message-Id: <20230417191558.83100-8-justintee8345@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20230417191558.83100-1-justintee8345@gmail.com> References: <20230417191558.83100-1-justintee8345@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Update lpfc version to 14.2.0.12 Signed-off-by: Justin Tee --- drivers/scsi/lpfc/lpfc_version.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_version.h b/drivers/scsi/lpfc/lpfc_version.h index c97411b0992e..5fda8ac6b883 100644 --- a/drivers/scsi/lpfc/lpfc_version.h +++ b/drivers/scsi/lpfc/lpfc_version.h @@ -20,7 +20,7 @@ * included with this package. * *******************************************************************/ -#define LPFC_DRIVER_VERSION "14.2.0.11" +#define LPFC_DRIVER_VERSION "14.2.0.12" #define LPFC_DRIVER_NAME "lpfc" /* Used for SLI 2/3 */