From patchwork Sat Dec 9 01:18:04 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10103481 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1A99760223 for ; Sat, 9 Dec 2017 01:18:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0EB1828FFE for ; Sat, 9 Dec 2017 01:18:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 038892900D; Sat, 9 Dec 2017 01:18:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F89828FFE for ; Sat, 9 Dec 2017 01:18:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752682AbdLIBS1 (ORCPT ); Fri, 8 Dec 2017 20:18:27 -0500 Received: from mail-qk0-f196.google.com ([209.85.220.196]:38289 "EHLO mail-qk0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752620AbdLIBSZ (ORCPT ); Fri, 8 Dec 2017 20:18:25 -0500 Received: by mail-qk0-f196.google.com with SMTP id z203so1027924qkb.5 for ; Fri, 08 Dec 2017 17:18:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=G0fPES7BEXljPZfeeiGfxX2TnMBLgNzjcQYa0Aj2gwU=; b=nYRaBgk3CBx8wKZ6LLgjbhAGpKTmNsqEgdv+mqHu9sG9r2PdGnA4wy2SmLHslsLSXy e7oVGbeO0nCqCITcIMFnd0IukOvzot6o+qG4uLGSgtCygSKGejw6R6OoCiroaKkkykiK 5rowyALPajKtS6CYOL5BwGLOI/5pY/6dh4ifg0kz0ve+eWxGbsANfm7uXB/k/ej30UGX RbKur66fhAR51ZJFypiMBaiN2cB5ewpSVBvcdTzTZWf1MhouMSZTD+DO+mGhv6kSwCXj 5IczLjJSWcxLum3OLgH9ZvZWPP8OmU0MifoPBHv6xbglyPV5S2Hg/CZyY9ZlJGfNP5xs aWAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=G0fPES7BEXljPZfeeiGfxX2TnMBLgNzjcQYa0Aj2gwU=; b=YOGNP2EUEvS0MbG/vYP4oERlAPkeetnwuunqLJ/dj8eXkCDRcI6MLm9jDZ3viXbzcj gJ5IZYvyIUzggBt3eL0/hMGUwDQT0IdboGrcPPxHReu44KOvaplx9aCxSDgigTprqAP9 +ep7N01AbdPyWGR6q0rTOsx9wXG0xRPbI86y8uxSMaZfz8ygLwn5W4isUFDycgixVj8J YUfWbFyPXrZKh8KOT1RWKpppWqxcuv6EXQFsM0QCQPDaExUKhA48pbEDTpHr2R4AznFd GpPN1Jq3hcCGxLktGnJ/rJFaiX0FtExeJIfR+87dAJEkXRPICHQemXQlDr/F7AOddXHO 5pxw== X-Gm-Message-State: AKGB3mKqnGBTbNY192tMlK/85YW7/17uhfcn+6U1ZZbjQICu4N1C9Np1 FcOc6EOZvzLb9qUG9v821SoLcA== X-Google-Smtp-Source: AGs4zMaaTDho47x2p7kn3C47h66FehTv8tBxH1oy04wlhoFSUIZcMJKPJIiRfmR9CFxq32vM74LlqA== X-Received: by 10.55.188.71 with SMTP id m68mr39716169qkf.44.1512782304482; Fri, 08 Dec 2017 17:18:24 -0800 (PST) Received: from pallmd1.broadcom.com ([192.19.228.250]) by smtp.gmail.com with ESMTPSA id k1sm922511qtf.11.2017.12.08.17.18.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 08 Dec 2017 17:18:24 -0800 (PST) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy , James Smart Subject: [PATCH 2/9] lpfc: Fix -EOVERFLOW behavior for NVMET and defer_rcv Date: Fri, 8 Dec 2017 17:18:04 -0800 Message-Id: <20171209011811.23421-3-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.1 In-Reply-To: <20171209011811.23421-1-jsmart2021@gmail.com> References: <20171209011811.23421-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The driver is all set to handle the defer_rcv api for the nvmet_fc transport, yet didn't properly recognize the return status when the defer_rcv occurred. The driver treated it simply as an error and aborted the io. Several residual issues occurred at that point. Finish the defer_rcv support: recognize the return status when the io request is being handled in a deferred style. This stops the rogue aborts; Replenish the async cmd rcv buffer in the deferred receive if needed. Signed-off-by: Dick Kennedy Signed-off-by: James Smart Reviewed-by: Hannes Reinecke --- drivers/scsi/lpfc/lpfc_nvmet.c | 24 ++++++++++++++++++++++-- drivers/scsi/lpfc/lpfc_nvmet.h | 1 + drivers/scsi/lpfc/lpfc_sli.c | 24 +++++++++++++----------- 3 files changed, 36 insertions(+), 13 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index d80cd1def3b9..02a1cfa10f72 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -38,6 +38,7 @@ #include <../drivers/nvme/host/nvme.h> #include +#include #include "lpfc_version.h" #include "lpfc_hw4.h" @@ -218,6 +219,7 @@ lpfc_nvmet_ctxbuf_post(struct lpfc_hba *phba, struct lpfc_nvmet_ctxbuf *ctx_buf) ctxp->entry_cnt = 1; ctxp->flag = 0; ctxp->ctxbuf = ctx_buf; + ctxp->rqb_buffer = (void *)nvmebuf; spin_lock_init(&ctxp->ctxlock); #ifdef CONFIG_SCSI_LPFC_DEBUG_FS @@ -253,6 +255,17 @@ lpfc_nvmet_ctxbuf_post(struct lpfc_hba *phba, struct lpfc_nvmet_ctxbuf *ctx_buf) return; } + /* Processing of FCP command is deferred */ + if (rc == -EOVERFLOW) { + lpfc_nvmeio_data(phba, + "NVMET RCV BUSY: xri x%x sz %d " + "from %06x\n", + oxid, size, sid); + /* defer repost rcv buffer till .defer_rcv callback */ + ctxp->flag &= ~LPFC_NVMET_DEFER_RCV_REPOST; + atomic_inc(&tgtp->rcv_fcp_cmd_out); + return; + } atomic_inc(&tgtp->rcv_fcp_cmd_drop); lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, "2582 FCP Drop IO x%x: err x%x: x%x x%x x%x\n", @@ -921,7 +934,11 @@ lpfc_nvmet_defer_rcv(struct nvmet_fc_target_port *tgtport, tgtp = phba->targetport->private; atomic_inc(&tgtp->rcv_fcp_cmd_defer); - lpfc_rq_buf_free(phba, &nvmebuf->hbuf); /* repost */ + if (ctxp->flag & LPFC_NVMET_DEFER_RCV_REPOST) + lpfc_rq_buf_free(phba, &nvmebuf->hbuf); /* repost */ + else + nvmebuf->hrq->rqbp->rqb_free_buffer(phba, nvmebuf); + ctxp->flag &= ~LPFC_NVMET_DEFER_RCV_REPOST; } static struct nvmet_fc_target_template lpfc_tgttemplate = { @@ -1693,6 +1710,7 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba, ctxp->entry_cnt = 1; ctxp->flag = 0; ctxp->ctxbuf = ctx_buf; + ctxp->rqb_buffer = (void *)nvmebuf; spin_lock_init(&ctxp->ctxlock); #ifdef CONFIG_SCSI_LPFC_DEBUG_FS @@ -1726,6 +1744,7 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba, /* Process FCP command */ if (rc == 0) { + ctxp->rqb_buffer = NULL; atomic_inc(&tgtp->rcv_fcp_cmd_out); lpfc_rq_buf_free(phba, &nvmebuf->hbuf); /* repost */ return; @@ -1737,10 +1756,11 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba, "NVMET RCV BUSY: xri x%x sz %d from %06x\n", oxid, size, sid); /* defer reposting rcv buffer till .defer_rcv callback */ - ctxp->rqb_buffer = nvmebuf; + ctxp->flag |= LPFC_NVMET_DEFER_RCV_REPOST; atomic_inc(&tgtp->rcv_fcp_cmd_out); return; } + ctxp->rqb_buffer = nvmebuf; atomic_inc(&tgtp->rcv_fcp_cmd_drop); lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, diff --git a/drivers/scsi/lpfc/lpfc_nvmet.h b/drivers/scsi/lpfc/lpfc_nvmet.h index 6723e7b81946..03096024e073 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.h +++ b/drivers/scsi/lpfc/lpfc_nvmet.h @@ -126,6 +126,7 @@ struct lpfc_nvmet_rcv_ctx { #define LPFC_NVMET_XBUSY 0x4 /* XB bit set on IO cmpl */ #define LPFC_NVMET_CTX_RLS 0x8 /* ctx free requested */ #define LPFC_NVMET_ABTS_RCV 0x10 /* ABTS received on exchange */ +#define LPFC_NVMET_DEFER_RCV_REPOST 0x20 /* repost to RQ on defer rcv */ struct rqb_dmabuf *rqb_buffer; struct lpfc_nvmet_ctxbuf *ctxbuf; diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 1d489b89954e..5f5528a12308 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -475,28 +475,30 @@ lpfc_sli4_rq_put(struct lpfc_queue *hq, struct lpfc_queue *dq, struct lpfc_rqe *temp_hrqe; struct lpfc_rqe *temp_drqe; struct lpfc_register doorbell; - int put_index; + int hq_put_index; + int dq_put_index; /* sanity check on queue memory */ if (unlikely(!hq) || unlikely(!dq)) return -ENOMEM; - put_index = hq->host_index; - temp_hrqe = hq->qe[put_index].rqe; - temp_drqe = dq->qe[dq->host_index].rqe; + hq_put_index = hq->host_index; + dq_put_index = dq->host_index; + temp_hrqe = hq->qe[hq_put_index].rqe; + temp_drqe = dq->qe[dq_put_index].rqe; if (hq->type != LPFC_HRQ || dq->type != LPFC_DRQ) return -EINVAL; - if (put_index != dq->host_index) + if (hq_put_index != dq_put_index) return -EINVAL; /* If the host has not yet processed the next entry then we are done */ - if (((put_index + 1) % hq->entry_count) == hq->hba_index) + if (((hq_put_index + 1) % hq->entry_count) == hq->hba_index) return -EBUSY; lpfc_sli_pcimem_bcopy(hrqe, temp_hrqe, hq->entry_size); lpfc_sli_pcimem_bcopy(drqe, temp_drqe, dq->entry_size); /* Update the host index to point to the next slot */ - hq->host_index = ((put_index + 1) % hq->entry_count); - dq->host_index = ((dq->host_index + 1) % dq->entry_count); + hq->host_index = ((hq_put_index + 1) % hq->entry_count); + dq->host_index = ((dq_put_index + 1) % dq->entry_count); hq->RQ_buf_posted++; /* Ring The Header Receive Queue Doorbell */ @@ -517,7 +519,7 @@ lpfc_sli4_rq_put(struct lpfc_queue *hq, struct lpfc_queue *dq, } writel(doorbell.word0, hq->db_regaddr); } - return put_index; + return hq_put_index; } /** @@ -12887,8 +12889,8 @@ lpfc_sli4_sp_handle_rcqe(struct lpfc_hba *phba, struct lpfc_rcqe *rcqe) lpfc_printf_log(phba, KERN_ERR, LOG_SLI, "2537 Receive Frame Truncated!!\n"); case FC_STATUS_RQ_SUCCESS: - lpfc_sli4_rq_release(hrq, drq); spin_lock_irqsave(&phba->hbalock, iflags); + lpfc_sli4_rq_release(hrq, drq); dma_buf = lpfc_sli_hbqbuf_get(&phba->hbqs[0].hbq_buffer_list); if (!dma_buf) { hrq->RQ_no_buf_found++; @@ -13290,8 +13292,8 @@ lpfc_sli4_nvmet_handle_rcqe(struct lpfc_hba *phba, struct lpfc_queue *cq, "6126 Receive Frame Truncated!!\n"); /* Drop thru */ case FC_STATUS_RQ_SUCCESS: - lpfc_sli4_rq_release(hrq, drq); spin_lock_irqsave(&phba->hbalock, iflags); + lpfc_sli4_rq_release(hrq, drq); dma_buf = lpfc_sli_rqbuf_get(phba, hrq); if (!dma_buf) { hrq->RQ_no_buf_found++;