From patchwork Thu Jan 17 16:14:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ewan Milne X-Patchwork-Id: 10768485 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C683713A4 for ; Thu, 17 Jan 2019 16:14:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B89E43017F for ; Thu, 17 Jan 2019 16:14:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AD404301AD; Thu, 17 Jan 2019 16:14:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F2183017F for ; Thu, 17 Jan 2019 16:14:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728671AbfAQQOr (ORCPT ); Thu, 17 Jan 2019 11:14:47 -0500 Received: from mx1.redhat.com ([209.132.183.28]:36216 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728488AbfAQQOr (ORCPT ); Thu, 17 Jan 2019 11:14:47 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 21EDB5D67E; Thu, 17 Jan 2019 16:14:47 +0000 (UTC) Received: from emilne.bos.redhat.com (unknown [10.18.25.205]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8DAC15C22E; Thu, 17 Jan 2019 16:14:46 +0000 (UTC) From: "Ewan D. Milne" To: linux-scsi@vger.kernel.org, linux-nvme@lists.infradead.org, james.smart@broadcom.com, dick.kennedy@broadcom.com Subject: [PATCH 1/2] lpfc: nvme: avoid hang / use-after-free when destroying localport Date: Thu, 17 Jan 2019 11:14:44 -0500 Message-Id: <20190117161445.25671-2-emilne@redhat.com> In-Reply-To: <20190117161445.25671-1-emilne@redhat.com> References: <20190117161445.25671-1-emilne@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Thu, 17 Jan 2019 16:14:47 +0000 (UTC) Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We cannot wait on a completion object in the lpfc_nvme_lport structure in the _destroy_localport() code path because the NVMe/fc transport will free that structure immediately after the .localport_delete() callback. This results in a use-after-free, and a hang if slub_debug=FZPU is enabled. Fix this by putting the completion on the stack. Signed-off-by: Ewan D. Milne Reviewed-by:   James Smart  --- drivers/scsi/lpfc/lpfc_nvme.c | 16 +++++++++------- drivers/scsi/lpfc/lpfc_nvme.h | 2 +- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index 4c66b19..8c9f790 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -297,7 +297,8 @@ lpfc_nvme_localport_delete(struct nvme_fc_local_port *localport) lport); /* release any threads waiting for the unreg to complete */ - complete(&lport->lport_unreg_done); + if (lport->vport->localport) + complete(lport->lport_unreg_cmp); } /* lpfc_nvme_remoteport_delete @@ -2545,7 +2546,8 @@ lpfc_nvme_create_localport(struct lpfc_vport *vport) */ void lpfc_nvme_lport_unreg_wait(struct lpfc_vport *vport, - struct lpfc_nvme_lport *lport) + struct lpfc_nvme_lport *lport, + struct completion *lport_unreg_cmp) { #if (IS_ENABLED(CONFIG_NVME_FC)) u32 wait_tmo; @@ -2557,8 +2559,7 @@ lpfc_nvme_lport_unreg_wait(struct lpfc_vport *vport, */ wait_tmo = msecs_to_jiffies(LPFC_NVME_WAIT_TMO * 1000); while (true) { - ret = wait_for_completion_timeout(&lport->lport_unreg_done, - wait_tmo); + ret = wait_for_completion_timeout(lport_unreg_cmp, wait_tmo); if (unlikely(!ret)) { lpfc_printf_vlog(vport, KERN_ERR, LOG_NVME_IOERR, "6176 Lport %p Localport %p wait " @@ -2592,12 +2593,12 @@ lpfc_nvme_destroy_localport(struct lpfc_vport *vport) struct lpfc_nvme_lport *lport; struct lpfc_nvme_ctrl_stat *cstat; int ret; + DECLARE_COMPLETION_ONSTACK(lport_unreg_cmp); if (vport->nvmei_support == 0) return; localport = vport->localport; - vport->localport = NULL; lport = (struct lpfc_nvme_lport *)localport->private; cstat = lport->cstat; @@ -2608,13 +2609,14 @@ lpfc_nvme_destroy_localport(struct lpfc_vport *vport) /* lport's rport list is clear. Unregister * lport and release resources. */ - init_completion(&lport->lport_unreg_done); + lport->lport_unreg_cmp = &lport_unreg_cmp; ret = nvme_fc_unregister_localport(localport); /* Wait for completion. This either blocks * indefinitely or succeeds */ - lpfc_nvme_lport_unreg_wait(vport, lport); + lpfc_nvme_lport_unreg_wait(vport, lport, &lport_unreg_cmp); + vport->localport = NULL; kfree(cstat); /* Regardless of the unregister upcall response, clear diff --git a/drivers/scsi/lpfc/lpfc_nvme.h b/drivers/scsi/lpfc/lpfc_nvme.h index cfd4719..b234d02 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.h +++ b/drivers/scsi/lpfc/lpfc_nvme.h @@ -50,7 +50,7 @@ struct lpfc_nvme_ctrl_stat { /* Declare nvme-based local and remote port definitions. */ struct lpfc_nvme_lport { struct lpfc_vport *vport; - struct completion lport_unreg_done; + struct completion *lport_unreg_cmp; /* Add stats counters here */ struct lpfc_nvme_ctrl_stat *cstat; atomic_t fc4NvmeLsRequests; From patchwork Thu Jan 17 16:14:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ewan Milne X-Patchwork-Id: 10768487 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 105E614E5 for ; Thu, 17 Jan 2019 16:14:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 018883017F for ; Thu, 17 Jan 2019 16:14:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E9BED301A9; Thu, 17 Jan 2019 16:14:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7728D301A7 for ; Thu, 17 Jan 2019 16:14:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728675AbfAQQOs (ORCPT ); Thu, 17 Jan 2019 11:14:48 -0500 Received: from mx1.redhat.com ([209.132.183.28]:41757 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728655AbfAQQOr (ORCPT ); Thu, 17 Jan 2019 11:14:47 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C2ABF64385; Thu, 17 Jan 2019 16:14:47 +0000 (UTC) Received: from emilne.bos.redhat.com (unknown [10.18.25.205]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3E1EE5C22E; Thu, 17 Jan 2019 16:14:47 +0000 (UTC) From: "Ewan D. Milne" To: linux-scsi@vger.kernel.org, linux-nvme@lists.infradead.org, james.smart@broadcom.com, dick.kennedy@broadcom.com Subject: [PATCH 2/2] lpfc: nvmet: avoid hang / use-after-free when destroying targetport Date: Thu, 17 Jan 2019 11:14:45 -0500 Message-Id: <20190117161445.25671-3-emilne@redhat.com> In-Reply-To: <20190117161445.25671-1-emilne@redhat.com> References: <20190117161445.25671-1-emilne@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Thu, 17 Jan 2019 16:14:47 +0000 (UTC) Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We cannot wait on a completion object in the lpfc_nvme_targetport structure in the _destroy_targetport() code path because the NVMe/fc transport will free that structure immediately after the .targetport_delete() callback. This results in a use-after-free, and a hang if slub_debug=FZPU is enabled. Fix this by putting the completion on the stack. Signed-off-by: Ewan D. Milne Reviewed-by:   James Smart  --- drivers/scsi/lpfc/lpfc_nvmet.c | 8 +++++--- drivers/scsi/lpfc/lpfc_nvmet.h | 2 +- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index 6245f44..95fee83 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -1003,7 +1003,8 @@ lpfc_nvmet_targetport_delete(struct nvmet_fc_target_port *targetport) struct lpfc_nvmet_tgtport *tport = targetport->private; /* release any threads waiting for the unreg to complete */ - complete(&tport->tport_unreg_done); + if (tport->phba->targetport) + complete(tport->tport_unreg_cmp); } static void @@ -1692,6 +1693,7 @@ lpfc_nvmet_destroy_targetport(struct lpfc_hba *phba) struct lpfc_nvmet_tgtport *tgtp; struct lpfc_queue *wq; uint32_t qidx; + DECLARE_COMPLETION_ONSTACK(tport_unreg_cmp); if (phba->nvmet_support == 0) return; @@ -1701,9 +1703,9 @@ lpfc_nvmet_destroy_targetport(struct lpfc_hba *phba) wq = phba->sli4_hba.nvme_wq[qidx]; lpfc_nvmet_wqfull_flush(phba, wq, NULL); } - init_completion(&tgtp->tport_unreg_done); + tgtp->tport_unreg_cmp = &tport_unreg_cmp; nvmet_fc_unregister_targetport(phba->targetport); - wait_for_completion_timeout(&tgtp->tport_unreg_done, 5); + wait_for_completion_timeout(&tport_unreg_cmp, 5); lpfc_nvmet_cleanup_io_context(phba); } phba->targetport = NULL; diff --git a/drivers/scsi/lpfc/lpfc_nvmet.h b/drivers/scsi/lpfc/lpfc_nvmet.h index 1aaff63..0ec1082 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.h +++ b/drivers/scsi/lpfc/lpfc_nvmet.h @@ -34,7 +34,7 @@ /* Used for NVME Target */ struct lpfc_nvmet_tgtport { struct lpfc_hba *phba; - struct completion tport_unreg_done; + struct completion *tport_unreg_cmp; /* Stats counters - lpfc_nvmet_unsol_ls_buffer */ atomic_t rcv_ls_req_in;