From patchwork Tue Aug 29 15:00:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian King X-Patchwork-Id: 9927627 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 49EC160383 for ; Tue, 29 Aug 2017 15:00:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 38350289B9 for ; Tue, 29 Aug 2017 15:00:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 341E4289D0; Tue, 29 Aug 2017 15:00:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7C43289B9 for ; Tue, 29 Aug 2017 15:00:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752218AbdH2PAg (ORCPT ); Tue, 29 Aug 2017 11:00:36 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:45808 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751974AbdH2PAf (ORCPT ); Tue, 29 Aug 2017 11:00:35 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v7TF0KZE045293 for ; Tue, 29 Aug 2017 11:00:35 -0400 Received: from e13.ny.us.ibm.com (e13.ny.us.ibm.com [129.33.205.203]) by mx0a-001b2d01.pphosted.com with ESMTP id 2cna461bfa-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 29 Aug 2017 11:00:34 -0400 Received: from localhost by e13.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 29 Aug 2017 11:00:33 -0400 Received: from b01cxnp22034.gho.pok.ibm.com (9.57.198.24) by e13.ny.us.ibm.com (146.89.104.200) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 29 Aug 2017 11:00:30 -0400 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v7TF0T1q21757964; Tue, 29 Aug 2017 15:00:29 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8EC97AE057; Tue, 29 Aug 2017 11:00:53 -0400 (EDT) Received: from [9.10.86.49] (unknown [9.10.86.49]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP id 51845AE043; Tue, 29 Aug 2017 11:00:53 -0400 (EDT) To: aacraid@microsemi.com, linux-scsi Cc: Douglas Miller , Wendy Xiong , RaghavaAditya.Renukunta@microsemi.com, stable@kernel.org Subject: [PATCH] aacraid: Fix command send race condition From: Brian King Date: Tue, 29 Aug 2017 10:00:29 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 Content-Language: en-US X-TM-AS-GCONF: 00 x-cbid: 17082915-0008-0000-0000-0000027622A0 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007633; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000226; SDB=6.00909289; UDB=6.00456026; IPR=6.00689574; BA=6.00005560; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016918; XFM=3.00000015; UTC=2017-08-29 15:00:32 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17082915-0009-0000-0000-0000368716D9 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-08-29_04:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=25 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1708290226 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This fixes a potential race condition observed on Power systems. Several places throughout the aacraid driver call aac_fib_send or similar to send a command to the aacraid adapter, then check the return code to determine if the command was actually sent to the adapter, then update the phase field in the scsi command scratch pad area to track that the firmware now owns this command. However, there is nothing that ensures that by the time the aac_fib_send function returns and we go to write to the scsi command, that the command hasn't already completed and the scsi command has been freed. This was causing random crashes in the TCP stack which was tracked down to be caused by memory that had been a struct request + scsi_cmnd being now used for an skbuff. Memory poisoning was enabled in the kernel to debug this which showed that the last owner of the memory that had been freed was aacraid and that it was a struct request. The memory that was corrupted was the exact data pattern of AAC_OWNER_FIRMWARE and it was at the same offset that aacraid writes, which is scsicmd->SCp.phase. The patch below resolves this issue. Cc: stable Signed-off-by: Brian King Reviewed-by: Dave Carroll Index: linux.git/drivers/scsi/aacraid/aachba.c =================================================================== --- linux.git.orig/drivers/scsi/aacraid/aachba.c +++ linux.git/drivers/scsi/aacraid/aachba.c @@ -594,6 +594,7 @@ static int aac_get_container_name(struct aac_fib_init(cmd_fibcontext); dinfo = (struct aac_get_name *) fib_data(cmd_fibcontext); + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; dinfo->command = cpu_to_le32(VM_ContainerConfig); dinfo->type = cpu_to_le32(CT_READ_NAME); @@ -611,10 +612,8 @@ static int aac_get_container_name(struct /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) { - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; + if (status == -EINPROGRESS) return 0; - } printk(KERN_WARNING "aac_get_container_name: aac_fib_send failed with status: %d.\n", status); aac_fib_complete(cmd_fibcontext); @@ -725,6 +724,7 @@ static void _aac_probe_container1(void * dinfo->count = cpu_to_le32(scmd_id(scsicmd)); dinfo->type = cpu_to_le32(FT_FILESYS); + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; status = aac_fib_send(ContainerCommand, fibptr, @@ -736,9 +736,7 @@ static void _aac_probe_container1(void * /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; - else if (status < 0) { + if (status < 0 && status != -EINPROGRESS) { /* Inherit results from VM_NameServe, if any */ dresp->status = cpu_to_le32(ST_OK); _aac_probe_container2(context, fibptr); @@ -766,6 +764,7 @@ static int _aac_probe_container(struct s dinfo->count = cpu_to_le32(scmd_id(scsicmd)); dinfo->type = cpu_to_le32(FT_FILESYS); scsicmd->SCp.ptr = (char *)callback; + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; status = aac_fib_send(ContainerCommand, fibptr, @@ -777,10 +776,9 @@ static int _aac_probe_container(struct s /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) { - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; + if (status == -EINPROGRESS) return 0; - } + if (status < 0) { scsicmd->SCp.ptr = NULL; aac_fib_complete(fibptr); @@ -1126,6 +1124,7 @@ static int aac_get_container_serial(stru dinfo->command = cpu_to_le32(VM_ContainerConfig); dinfo->type = cpu_to_le32(CT_CID_TO_32BITS_UID); dinfo->cid = cpu_to_le32(scmd_id(scsicmd)); + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; status = aac_fib_send(ContainerCommand, cmd_fibcontext, @@ -1138,10 +1137,8 @@ static int aac_get_container_serial(stru /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) { - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; + if (status == -EINPROGRESS) return 0; - } printk(KERN_WARNING "aac_get_container_serial: aac_fib_send failed with status: %d.\n", status); aac_fib_complete(cmd_fibcontext); @@ -2335,16 +2332,14 @@ static int aac_read(struct scsi_cmnd * s * Alocate and initialize a Fib */ cmd_fibcontext = aac_fib_alloc_tag(dev, scsicmd); - + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; status = aac_adapter_read(cmd_fibcontext, scsicmd, lba, count); /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) { - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; + if (status == -EINPROGRESS) return 0; - } printk(KERN_WARNING "aac_read: aac_fib_send failed with status: %d.\n", status); /* @@ -2429,16 +2424,14 @@ static int aac_write(struct scsi_cmnd * * Allocate and initialize a Fib then setup a BlockWrite command */ cmd_fibcontext = aac_fib_alloc_tag(dev, scsicmd); - + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; status = aac_adapter_write(cmd_fibcontext, scsicmd, lba, count, fua); /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) { - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; + if (status == -EINPROGRESS) return 0; - } printk(KERN_WARNING "aac_write: aac_fib_send failed with status: %d\n", status); /* @@ -2588,6 +2581,7 @@ static int aac_synchronize(struct scsi_c synchronizecmd->cid = cpu_to_le32(scmd_id(scsicmd)); synchronizecmd->count = cpu_to_le32(sizeof(((struct aac_synchronize_reply *)NULL)->data)); + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; /* * Now send the Fib to the adapter @@ -2603,10 +2597,8 @@ static int aac_synchronize(struct scsi_c /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) { - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; + if (status == -EINPROGRESS) return 0; - } printk(KERN_WARNING "aac_synchronize: aac_fib_send failed with status: %d.\n", status); @@ -2666,6 +2658,7 @@ static int aac_start_stop(struct scsi_cm pmcmd->cid = cpu_to_le32(sdev_id(sdev)); pmcmd->parm = (scsicmd->cmnd[1] & 1) ? cpu_to_le32(CT_PM_UNIT_IMMEDIATE) : 0; + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; /* * Now send the Fib to the adapter @@ -2681,10 +2674,8 @@ static int aac_start_stop(struct scsi_cm /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) { - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; + if (status == -EINPROGRESS) return 0; - } aac_fib_complete(cmd_fibcontext); aac_fib_free(cmd_fibcontext); @@ -3692,16 +3683,14 @@ static int aac_send_srb_fib(struct scsi_ * Allocate and initialize a Fib then setup a BlockWrite command */ cmd_fibcontext = aac_fib_alloc_tag(dev, scsicmd); - + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; status = aac_adapter_scsi(cmd_fibcontext, scsicmd); /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) { - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; + if (status == -EINPROGRESS) return 0; - } printk(KERN_WARNING "aac_srb: aac_fib_send failed with status: %d\n", status); aac_fib_complete(cmd_fibcontext); @@ -3739,15 +3728,14 @@ static int aac_send_hba_fib(struct scsi_ if (!cmd_fibcontext) return -1; + scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; status = aac_adapter_hba(cmd_fibcontext, scsicmd); /* * Check that the command queued to the controller */ - if (status == -EINPROGRESS) { - scsicmd->SCp.phase = AAC_OWNER_FIRMWARE; + if (status == -EINPROGRESS) return 0; - } pr_warn("aac_hba_cmd_req: aac_fib_send failed with status: %d\n", status);