From patchwork Tue Dec 1 12:39:40 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Raghava Aditya Renukunta X-Patchwork-Id: 7740261 Return-Path: X-Original-To: patchwork-linux-scsi@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id DE948BEEE1 for ; Tue, 1 Dec 2015 19:34:25 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id E2F4C2068E for ; Tue, 1 Dec 2015 19:34:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E4BC220697 for ; Tue, 1 Dec 2015 19:34:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754104AbbLATeT (ORCPT ); Tue, 1 Dec 2015 14:34:19 -0500 Received: from bby1mta03.pmc-sierra.com ([216.241.235.118]:55955 "EHLO bby1mta03.pmc-sierra.bc.ca" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753760AbbLATeR (ORCPT ); Tue, 1 Dec 2015 14:34:17 -0500 Received: from bby1mta03.pmc-sierra.bc.ca (localhost.pmc-sierra.bc.ca [127.0.0.1]) by localhost (Postfix) with SMTP id 913601070908; Tue, 1 Dec 2015 11:34:16 -0800 (PST) Received: from smtp.pmcs.com (bby1cas04.pmc-sierra.internal [216.241.227.182]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by bby1mta03.pmc-sierra.bc.ca (Postfix) with ESMTP id 76F081070714; Tue, 1 Dec 2015 11:34:16 -0800 (PST) Received: from localhost (216.241.227.4) by bby1cas04.pmc-sierra.internal (216.241.227.182) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 1 Dec 2015 11:34:16 -0800 From: Raghava Aditya Renukunta To: , CC: , , , , , , Subject: [PATCH 02/10] aacraid: Fix RRQ overload Date: Tue, 1 Dec 2015 04:39:40 -0800 Message-ID: <1448973589-9216-3-git-send-email-RaghavaAditya.Renukunta@pmcs.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1448973589-9216-1-git-send-email-RaghavaAditya.Renukunta@pmcs.com> References: <1448973589-9216-1-git-send-email-RaghavaAditya.Renukunta@pmcs.com> MIME-Version: 1.0 X-Originating-IP: [216.241.227.4] DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pmcs.com; h=from:to:cc:subject:date:message-id:in-reply-to:references:mime-version:content-type; s=default; bh=jYoXvfMkfmLH9Nc5LFo5TNxw52jy+NjCUYbowt14lyE=; b=J2LBAOp5xXgu/ws1myfqIWVLvK+v3AIGX9RwQtFDO28xUf+pE2Bt4jH90m8PUEA/+mzWlNJflT02j0IO2EfirHR4N8ojmR3E0CXhNdfsIkTm2lFKRM1jOdGFJrTby+oZTI4VzgwnDQxqF4bKkYL/X/31O2uT2benaG66dG8eLII= Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Spam-Status: No, score=-5.3 required=5.0 tests=BAYES_00, DATE_IN_PAST_06_12, DKIM_SIGNED,RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Raghava Aditya Renukunta The driver utilizes an array of atomic variables to keep track of IO submissions to each vector. To submit an IO multiple threads iterate through the array to find a vector which has empty slots to send an IO. The reading and updating of the variable is not atomic, causing race conditions when a thread uses a full vector to submit an IO. Fixed by mapping each FIB to a vector, the submission path then uses said vector to submit IO thereby removing the possibly of a race condition.The vector assignment is started from 1 since vector 0 is reserved for the use of AIF management FIBS.If the number of MSIx vectors is 1 (MSI or INTx mode) then all the fibs are allocated to vector 0. Signed-off-by: Raghava Aditya Renukunta Reviewed-by: Johannes Thumshirn Reviewed-by: Tomas Henzl --- drivers/scsi/aacraid/aacraid.h | 1 + drivers/scsi/aacraid/commsup.c | 12 ++++++++++++ drivers/scsi/aacraid/src.c | 30 +++++++----------------------- 3 files changed, 20 insertions(+), 23 deletions(-) diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h index da227e8..d133c4a 100644 --- a/drivers/scsi/aacraid/aacraid.h +++ b/drivers/scsi/aacraid/aacraid.h @@ -944,6 +944,7 @@ struct fib { */ struct list_head fiblink; void *data; + u32 vector_no; struct hw_fib *hw_fib_va; /* Actual shared object */ dma_addr_t hw_fib_pa; /* physical address of hw_fib*/ }; diff --git a/drivers/scsi/aacraid/commsup.c b/drivers/scsi/aacraid/commsup.c index b5b653c..b257d3b 100644 --- a/drivers/scsi/aacraid/commsup.c +++ b/drivers/scsi/aacraid/commsup.c @@ -104,6 +104,7 @@ int aac_fib_setup(struct aac_dev * dev) struct hw_fib *hw_fib; dma_addr_t hw_fib_pa; int i; + u32 vector = 1; while (((i = fib_map_alloc(dev)) == -ENOMEM) && (dev->scsi_host_ptr->can_queue > (64 - AAC_NUM_MGT_FIB))) { @@ -150,6 +151,17 @@ int aac_fib_setup(struct aac_dev * dev) dev->max_fib_size + sizeof(struct aac_fib_xporthdr)); hw_fib_pa = hw_fib_pa + dev->max_fib_size + sizeof(struct aac_fib_xporthdr); + + if ((dev->max_msix == 1) || + (i > ((dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB - 1) + - dev->vector_cap))) { + fibptr->vector_no = 0; + } else { + fibptr->vector_no = vector; + vector++; + if (vector == dev->max_msix) + vector = 1; + } } /* * Add the fib chain to the free list diff --git a/drivers/scsi/aacraid/src.c b/drivers/scsi/aacraid/src.c index 2aa34ea..bc0203f 100644 --- a/drivers/scsi/aacraid/src.c +++ b/drivers/scsi/aacraid/src.c @@ -156,8 +156,8 @@ static irqreturn_t aac_src_intr_message(int irq, void *dev_id) break; if (dev->msi_enabled && dev->max_msix > 1) atomic_dec(&dev->rrq_outstanding[vector_no]); - aac_intr_normal(dev, handle-1, 0, isFastResponse, NULL); dev->host_rrq[index++] = 0; + aac_intr_normal(dev, handle-1, 0, isFastResponse, NULL); if (index == (vector_no + 1) * dev->vector_cap) index = vector_no * dev->vector_cap; dev->host_rrq_idx[vector_no] = index; @@ -452,36 +452,20 @@ static int aac_src_deliver_message(struct fib *fib) #endif u16 hdr_size = le16_to_cpu(fib->hw_fib_va->header.Size); + u16 vector_no; atomic_inc(&q->numpending); if (dev->msi_enabled && fib->hw_fib_va->header.Command != AifRequest && dev->max_msix > 1) { - u_int16_t vector_no, first_choice = 0xffff; - - vector_no = dev->fibs_pushed_no % dev->max_msix; - do { - vector_no += 1; - if (vector_no == dev->max_msix) - vector_no = 1; - if (atomic_read(&dev->rrq_outstanding[vector_no]) < - dev->vector_cap) - break; - if (0xffff == first_choice) - first_choice = vector_no; - else if (vector_no == first_choice) - break; - } while (1); - if (vector_no == first_choice) - vector_no = 0; - atomic_inc(&dev->rrq_outstanding[vector_no]); - if (dev->fibs_pushed_no == 0xffffffff) - dev->fibs_pushed_no = 0; - else - dev->fibs_pushed_no++; + vector_no = fib->vector_no; fib->hw_fib_va->header.Handle += (vector_no << 16); + } else { + vector_no = 0; } + atomic_inc(&dev->rrq_outstanding[vector_no]); + if (dev->comm_interface == AAC_COMM_MESSAGE_TYPE2) { /* Calculate the amount to the fibsize bits */ fibsize = (hdr_size + 127) / 128 - 1;