From patchwork Wed Feb 3 00:58:00 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Raghava Aditya Renukunta X-Patchwork-Id: 8196591 Return-Path: X-Original-To: patchwork-linux-scsi@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 39F3BBEEE5 for ; Wed, 3 Feb 2016 00:48:06 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 29A6720251 for ; Wed, 3 Feb 2016 00:48:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 20AEE201EF for ; Wed, 3 Feb 2016 00:48:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754860AbcBCAsA (ORCPT ); Tue, 2 Feb 2016 19:48:00 -0500 Received: from bby1mta02.pmc-sierra.com ([216.241.235.117]:40789 "EHLO bby1mta02.pmc-sierra.bc.ca" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754791AbcBCAr7 (ORCPT ); Tue, 2 Feb 2016 19:47:59 -0500 Received: from bby1mta02.pmc-sierra.bc.ca (localhost.pmc-sierra.bc.ca [127.0.0.1]) by localhost (Postfix) with SMTP id A0B2E8E030A; Tue, 2 Feb 2016 16:47:58 -0800 (PST) Received: from smtp.pmcs.com (bby1cas03.pmc-sierra.internal [216.241.227.144]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by bby1mta02.pmc-sierra.bc.ca (Postfix) with ESMTP id 80C6E8E0189; Tue, 2 Feb 2016 16:47:58 -0800 (PST) Received: from localhost (216.241.227.4) by bby1cas03.pmc-sierra.internal (216.241.227.144) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 2 Feb 2016 16:47:58 -0800 From: Raghava Aditya Renukunta To: , , CC: , , , , , , , , Subject: [PATCH V5 02/10] aacraid: Fix RRQ overload Date: Tue, 2 Feb 2016 16:58:00 -0800 Message-ID: <1454461088-32482-3-git-send-email-RaghavaAditya.Renukunta@pmcs.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1454461088-32482-1-git-send-email-RaghavaAditya.Renukunta@pmcs.com> References: <1454461088-32482-1-git-send-email-RaghavaAditya.Renukunta@pmcs.com> MIME-Version: 1.0 X-Originating-IP: [216.241.227.4] DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pmcs.com; h=from:to:cc:subject:date:message-id:in-reply-to:references:mime-version:content-type; s=default; bh=zfih1Qkvgm0VMuKit+/p4G8K6gLcxMxF+zVCI5UUN88=; b=hU8N5nf5BGMjigHuM6cBF6KPLFY8mXESxGILIUn2TOT9bKt5eefZSTXKHMfIdDuefWZANHR8BRhlJa81n3eAEplxpwPujuVXDzHswOBpTTsfAWoTnYZhi0bqU6uKUNgPvRWotfYMkhfN6QU6SIpUju3FUh3w3xnnPUYFxsGSv1k= Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Spam-Status: No, score=-7.2 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Raghava Aditya Renukunta The driver utilizes an array of atomic variables to keep track of IO submissions to each vector. To submit an IO multiple threads iterate through the array to find a vector which has empty slots to send an IO. The reading and updating of the variable is not atomic, causing race conditions when a thread uses a full vector to submit an IO. Fixed by mapping each FIB to a vector, the submission path then uses said vector to submit IO thereby removing the possibly of a race condition.The vector assignment is started from 1 since vector 0 is reserved for the use of AIF management FIBS.If the number of MSIx vectors is 1 (MSI or INTx mode) then all the fibs are allocated to vector 0. Changes in V2: Created function aac_fib_vector_assign to assign vector numbers to fib and to prevent code duplication Changes in V3: None Changes in V4: None Changes in V5: None Fixes: 495c0217 "aacraid: MSI-x support" Cc: stable@vger.kernel.org # v4.1 Signed-off-by: Raghava Aditya Renukunta Reviewed-by: Johannes Thumshirn Reviewed-by: Tomas Henzl --- drivers/scsi/aacraid/aacraid.h | 2 ++ drivers/scsi/aacraid/commsup.c | 28 ++++++++++++++++++++++++++++ drivers/scsi/aacraid/src.c | 30 +++++++----------------------- 3 files changed, 37 insertions(+), 23 deletions(-) diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h index f51f0a0..fff1306 100644 --- a/drivers/scsi/aacraid/aacraid.h +++ b/drivers/scsi/aacraid/aacraid.h @@ -944,6 +944,7 @@ struct fib { */ struct list_head fiblink; void *data; + u32 vector_no; struct hw_fib *hw_fib_va; /* Actual shared object */ dma_addr_t hw_fib_pa; /* physical address of hw_fib*/ }; @@ -2113,6 +2114,7 @@ static inline unsigned int cap_to_cyls(sector_t capacity, unsigned divisor) int aac_acquire_irq(struct aac_dev *dev); void aac_free_irq(struct aac_dev *dev); const char *aac_driverinfo(struct Scsi_Host *); +void aac_fib_vector_assign(struct aac_dev *dev); struct fib *aac_fib_alloc(struct aac_dev *dev); struct fib *aac_fib_alloc_tag(struct aac_dev *dev, struct scsi_cmnd *scmd); int aac_fib_setup(struct aac_dev *dev); diff --git a/drivers/scsi/aacraid/commsup.c b/drivers/scsi/aacraid/commsup.c index 46a2a2f..07a42a3 100644 --- a/drivers/scsi/aacraid/commsup.c +++ b/drivers/scsi/aacraid/commsup.c @@ -90,6 +90,28 @@ void aac_fib_map_free(struct aac_dev *dev) dev->hw_fib_pa = 0; } +void aac_fib_vector_assign(struct aac_dev *dev) +{ + u32 i = 0; + u32 vector = 1; + struct fib *fibptr = NULL; + + for (i = 0, fibptr = &dev->fibs[i]; + i < (dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB); + i++, fibptr++) { + if ((dev->max_msix == 1) || + (i > ((dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB - 1) + - dev->vector_cap))) { + fibptr->vector_no = 0; + } else { + fibptr->vector_no = vector; + vector++; + if (vector == dev->max_msix) + vector = 1; + } + } +} + /** * aac_fib_setup - setup the fibs * @dev: Adapter to set up @@ -152,6 +174,12 @@ int aac_fib_setup(struct aac_dev * dev) hw_fib_pa = hw_fib_pa + dev->max_fib_size + sizeof(struct aac_fib_xporthdr); } + + /* + *Assign vector numbers to fibs + */ + aac_fib_vector_assign(dev); + /* * Add the fib chain to the free list */ diff --git a/drivers/scsi/aacraid/src.c b/drivers/scsi/aacraid/src.c index 2aa34ea..bc0203f 100644 --- a/drivers/scsi/aacraid/src.c +++ b/drivers/scsi/aacraid/src.c @@ -156,8 +156,8 @@ static irqreturn_t aac_src_intr_message(int irq, void *dev_id) break; if (dev->msi_enabled && dev->max_msix > 1) atomic_dec(&dev->rrq_outstanding[vector_no]); - aac_intr_normal(dev, handle-1, 0, isFastResponse, NULL); dev->host_rrq[index++] = 0; + aac_intr_normal(dev, handle-1, 0, isFastResponse, NULL); if (index == (vector_no + 1) * dev->vector_cap) index = vector_no * dev->vector_cap; dev->host_rrq_idx[vector_no] = index; @@ -452,36 +452,20 @@ static int aac_src_deliver_message(struct fib *fib) #endif u16 hdr_size = le16_to_cpu(fib->hw_fib_va->header.Size); + u16 vector_no; atomic_inc(&q->numpending); if (dev->msi_enabled && fib->hw_fib_va->header.Command != AifRequest && dev->max_msix > 1) { - u_int16_t vector_no, first_choice = 0xffff; - - vector_no = dev->fibs_pushed_no % dev->max_msix; - do { - vector_no += 1; - if (vector_no == dev->max_msix) - vector_no = 1; - if (atomic_read(&dev->rrq_outstanding[vector_no]) < - dev->vector_cap) - break; - if (0xffff == first_choice) - first_choice = vector_no; - else if (vector_no == first_choice) - break; - } while (1); - if (vector_no == first_choice) - vector_no = 0; - atomic_inc(&dev->rrq_outstanding[vector_no]); - if (dev->fibs_pushed_no == 0xffffffff) - dev->fibs_pushed_no = 0; - else - dev->fibs_pushed_no++; + vector_no = fib->vector_no; fib->hw_fib_va->header.Handle += (vector_no << 16); + } else { + vector_no = 0; } + atomic_inc(&dev->rrq_outstanding[vector_no]); + if (dev->comm_interface == AAC_COMM_MESSAGE_TYPE2) { /* Calculate the amount to the fibsize bits */ fibsize = (hdr_size + 127) / 128 - 1;