From patchwork Thu Jun 22 02:14:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uma Krishnan X-Patchwork-Id: 9803325 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6C30F60234 for ; Thu, 22 Jun 2017 02:15:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F21B28531 for ; Thu, 22 Jun 2017 02:15:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 53E0B28539; Thu, 22 Jun 2017 02:15:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D784A28531 for ; Thu, 22 Jun 2017 02:15:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752140AbdFVCPG (ORCPT ); Wed, 21 Jun 2017 22:15:06 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:37401 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751140AbdFVCPF (ORCPT ); Wed, 21 Jun 2017 22:15:05 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v5M2EZjY140591 for ; Wed, 21 Jun 2017 22:15:05 -0400 Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.149]) by mx0b-001b2d01.pphosted.com with ESMTP id 2b7t205y6f-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Wed, 21 Jun 2017 22:15:04 -0400 Received: from localhost by e31.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 21 Jun 2017 20:15:04 -0600 Received: from b03cxnp08025.gho.boulder.ibm.com (9.17.130.17) by e31.co.us.ibm.com (192.168.1.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 21 Jun 2017 20:15:01 -0600 Received: from b03ledav003.gho.boulder.ibm.com (b03ledav003.gho.boulder.ibm.com [9.17.130.234]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v5M2Ex2Z1048890; Wed, 21 Jun 2017 19:14:59 -0700 Received: from b03ledav003.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 202DB6A03D; Wed, 21 Jun 2017 20:14:59 -0600 (MDT) Received: from p8tul1-build.aus.stglabs.ibm.com (unknown [9.3.141.206]) by b03ledav003.gho.boulder.ibm.com (Postfix) with ESMTP id 7893F6A048; Wed, 21 Jun 2017 20:14:58 -0600 (MDT) From: Uma Krishnan To: linux-scsi@vger.kernel.org, James Bottomley , "Martin K. Petersen" , "Matthew R. Ochs" , "Manoj N. Kumar" Cc: linuxppc-dev@lists.ozlabs.org, Ian Munsie , Andrew Donnellan , Frederic Barrat , Christophe Lombard Subject: [PATCH 07/17] cxlflash: Flush pending commands in cleanup path Date: Wed, 21 Jun 2017 21:14:56 -0500 X-Mailer: git-send-email 2.1.0 In-Reply-To: <1498097563-8680-1-git-send-email-ukrishn@linux.vnet.ibm.com> References: <1498097563-8680-1-git-send-email-ukrishn@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 17062202-8235-0000-0000-00000BC5248E X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007269; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000214; SDB=6.00878072; UDB=6.00437482; IPR=6.00658213; BA=6.00005434; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015913; XFM=3.00000015; UTC=2017-06-22 02:15:02 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17062202-8236-0000-0000-00003C64F868 Message-Id: <1498097696-8934-1-git-send-email-ukrishn@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-22_01:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706220037 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When the AFU is reset in an error path, pending scsi commands can be silently dropped without completion or a formal abort. This puts the onus on the cxlflash driver to notify mid-layer and indicating that the command can be retried. Once the card has been quiesced, the hardware send queue lock is acquired to prevent any data movement while the pending commands are processed. Signed-off-by: Uma Krishnan Acked-by: Matthew R. Ochs --- drivers/scsi/cxlflash/common.h | 5 +++- drivers/scsi/cxlflash/main.c | 57 +++++++++++++++++++++++++++++++++++++++--- 2 files changed, 57 insertions(+), 5 deletions(-) diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h index 3eaa3be..11a5b0a 100644 --- a/drivers/scsi/cxlflash/common.h +++ b/drivers/scsi/cxlflash/common.h @@ -157,7 +157,9 @@ struct afu_cmd { struct list_head queue; u32 hwq_index; - u8 cmd_tmf:1; + u8 cmd_tmf:1, + cmd_aborted:1; + struct list_head list; /* Pending commands link */ /* As per the SISLITE spec the IOARCB EA has to be 16-byte aligned. @@ -176,6 +178,7 @@ static inline struct afu_cmd *sc_to_afucz(struct scsi_cmnd *sc) struct afu_cmd *afuc = sc_to_afuc(sc); memset(afuc, 0, sizeof(*afuc)); + INIT_LIST_HEAD(&afuc->queue); return afuc; } diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c index 1446fab..0a3de42 100644 --- a/drivers/scsi/cxlflash/main.c +++ b/drivers/scsi/cxlflash/main.c @@ -194,6 +194,36 @@ static void cmd_complete(struct afu_cmd *cmd) } /** + * flush_pending_cmds() - flush all pending commands on this hardware queue + * @hwq: Hardware queue to flush. + * + * The hardware send queue lock associated with this hardware queue must be + * held when calling this routine. + */ +static void flush_pending_cmds(struct hwq *hwq) +{ + struct afu_cmd *cmd, *tmp; + struct scsi_cmnd *scp; + + list_for_each_entry_safe(cmd, tmp, &hwq->pending_cmds, list) { + /* Bypass command when on a doneq, cmd_complete() will handle */ + if (!list_empty(&cmd->queue)) + continue; + + list_del(&cmd->list); + + if (cmd->scp) { + scp = cmd->scp; + scp->result = (DID_IMM_RETRY << 16); + scp->scsi_done(scp); + } else { + cmd->cmd_aborted = true; + complete(&cmd->cevent); + } + } +} + +/** * context_reset() - reset context via specified register * @hwq: Hardware queue owning the context to be reset. * @reset_reg: MMIO register to perform reset. @@ -357,6 +387,9 @@ static int wait_resp(struct afu *afu, struct afu_cmd *cmd) if (!timeout) rc = -ETIMEDOUT; + if (cmd->cmd_aborted) + rc = -EAGAIN; + if (unlikely(cmd->sa.ioasc != 0)) { dev_err(dev, "%s: cmd %02x failed, ioasc=%08x\n", __func__, cmd->rcb.cdb[0], cmd->sa.ioasc); @@ -702,6 +735,7 @@ static void term_mc(struct cxlflash_cfg *cfg, u32 index) struct afu *afu = cfg->afu; struct device *dev = &cfg->dev->dev; struct hwq *hwq; + ulong lock_flags; if (!afu) { dev_err(dev, "%s: returning with NULL afu\n", __func__); @@ -719,6 +753,10 @@ static void term_mc(struct cxlflash_cfg *cfg, u32 index) if (index != PRIMARY_HWQ) WARN_ON(cxl_release_context(hwq->ctx)); hwq->ctx = NULL; + + spin_lock_irqsave(&hwq->hsq_slock, lock_flags); + flush_pending_cmds(hwq); + spin_unlock_irqrestore(&hwq->hsq_slock, lock_flags); } /** @@ -2155,7 +2193,7 @@ int cxlflash_afu_sync(struct afu *afu, ctx_hndl_t ctx_hndl_u, mutex_lock(&sync_active); atomic_inc(&afu->cmds_active); - buf = kzalloc(sizeof(*cmd) + __alignof__(*cmd) - 1, GFP_KERNEL); + buf = kmalloc(sizeof(*cmd) + __alignof__(*cmd) - 1, GFP_KERNEL); if (unlikely(!buf)) { dev_err(dev, "%s: no memory for command\n", __func__); rc = -ENOMEM; @@ -2165,6 +2203,8 @@ int cxlflash_afu_sync(struct afu *afu, ctx_hndl_t ctx_hndl_u, cmd = (struct afu_cmd *)PTR_ALIGN(buf, __alignof__(*cmd)); retry: + memset(cmd, 0, sizeof(*cmd)); + INIT_LIST_HEAD(&cmd->queue); init_completion(&cmd->cevent); cmd->parent = afu; cmd->hwq_index = hwq->index; @@ -2191,11 +2231,20 @@ int cxlflash_afu_sync(struct afu *afu, ctx_hndl_t ctx_hndl_u, } rc = wait_resp(afu, cmd); - if (rc == -ETIMEDOUT) { + switch (rc) { + case -ETIMEDOUT: rc = afu->context_reset(hwq); - if (!rc && ++nretry < 2) + if (rc) { + cxlflash_schedule_async_reset(cfg); + break; + } + /* fall through to retry */ + case -EAGAIN: + if (++nretry < 2) goto retry; - cxlflash_schedule_async_reset(cfg); + /* fall through to exit */ + default: + break; } out: