From patchwork Tue Oct 23 18:00:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bill Kuzeja X-Patchwork-Id: 10653457 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DCB8F14DE for ; Tue, 23 Oct 2018 18:00:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC3EA2A255 for ; Tue, 23 Oct 2018 18:00:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CA8C02A258; Tue, 23 Oct 2018 18:00:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6BD052A255 for ; Tue, 23 Oct 2018 18:00:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727758AbeJXCYe convert rfc822-to-8bit (ORCPT ); Tue, 23 Oct 2018 22:24:34 -0400 Received: from us-smtp-delivery-131.mimecast.com ([63.128.21.131]:58759 "EHLO us-smtp-delivery-131.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725997AbeJXCYe (ORCPT ); Tue, 23 Oct 2018 22:24:34 -0400 Received: from mailhub4.stratus.com (mailhub4.stratus.com [134.111.1.17]) by us-smtp-1.mimecast.com with ESMTP id us-mta-132-ElYq7D4gOHicOIJp_-EdOw-1; Tue, 23 Oct 2018 14:00:04 -0400 Received: from EXHQ1.corp.stratus.com (exhq1.corp.stratus.com [134.111.200.125]) by mailhub4.stratus.com (8.12.11/8.12.11) with ESMTP id w9NI040N001552; Tue, 23 Oct 2018 14:00:04 -0400 Received: from linuxdev.lnx.eng.stratus.com (134.111.220.63) by EXHQ1.corp.stratus.com (134.111.200.125) with Microsoft SMTP Server (TLS) id 14.3.279.2; Tue, 23 Oct 2018 13:59:40 -0400 From: Bill Kuzeja To: CC: , Subject: [PATCH RESEND] Timeouts occur on QLogic adapter surprise removal Date: Tue, 23 Oct 2018 14:00:03 -0400 Message-ID: <1540317603-6329-1-git-send-email-William.Kuzeja@stratus.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 X-MC-Unique: ElYq7D4gOHicOIJp_-EdOw-1 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When doing a surprise removal of an adapter, some in flight I/Os can get stuck and take a while to complete (they actually timeout and are retried). We are not handling an early error exit from qla2xxx_eh_abort properly. Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip") --- Note #1: (Reworked ACKed patch to cleanly apply to 4.20/scsi-queue. The main explanation has not been reworked. Note #2: I also see the following outstanding patch which removes a variable used in this patch) [PATCH 5/7] qla2xxx: Remove a set-but-not-used variable Obviously, that patch is no longer needed as I am now using that variable... Patch explanation: After a hot remove of a Qlogic adapter, the driver's remove function gets called and we end up aborting all in progress I/Os. Here is the code flow: qla2x00_remove_one qla2x00_abort_isp_cleanup qla2x00_abort_all_cmds __qla2x00_abort_all_cmds qla2xxx_eh_abort At the start of qla2xxx_eh_abort, some sanity checks are done before actually sending the abort. One of these checks is a call to fc_block_scsi_eh. In the case of a hot remove, it turns out that this routine can exit with FAST_IO_FAIL. When this occurs, we return back to __qla2x00_abort_all_cmds with an extra reference on sp (because the abort never gets sent). Originally, this was addressed with another fix: commit 4cd3b6ebff85 scsi: qla2xxx: Fix extraneous ref on sp's after adapter break But this later this added change complicated matters: commit 45235022da99 scsi: qla2xxx: Fix driver unload by shutting down chip Because the abort is now being done earlier in the teardown (through qla2x00_abort_isp_cleanup), in qla2xxx_eh_abort we make it past the first check because qla2x00_isp_reg_stat(ha) returns zero. When we fail a few lines later in fc_block_scsi_eh, this error is not handled properly in __qla2x00_abort_all_cmds and the I/O ends up hanging and timing out because of the extra reference. For this fix, a check for FAST_IO_FAIL is added to __qla2x00_abort_all_cmds where we check to see if qla2xxx_eh_abort succeeded or not. This removes the extra reference in this additional early exit case. In my testing (hw surprise removals and also adapter remove via sysfs), this eliminates the timeouts and delays and the remove proceeds smoothly. drivers/scsi/qla2xxx/qla_os.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c index dba672f..af57f05 100644 --- a/drivers/scsi/qla2xxx/qla_os.c +++ b/drivers/scsi/qla2xxx/qla_os.c @@ -1804,6 +1804,12 @@ uint32_t qla2x00_isp_reg_stat(struct qla_hw_data *ha) spin_lock_irqsave (qp->qp_lock_ptr, flags); } + /* + * Get rid of extra reference caused by early + * exit from qla2xxx_eh_abort + */ + if (status == FAST_IO_FAIL) + atomic_dec(&sp->ref_count); } sp->done(sp, res); break;