From patchwork Tue Apr 24 09:28:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaitra P B X-Patchwork-Id: 10359073 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 20D4C60225 for ; Tue, 24 Apr 2018 09:29:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14DBD28D04 for ; Tue, 24 Apr 2018 09:29:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 09F0728D3B; Tue, 24 Apr 2018 09:29:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 66EA328D04 for ; Tue, 24 Apr 2018 09:29:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753551AbeDXJ3N (ORCPT ); Tue, 24 Apr 2018 05:29:13 -0400 Received: from mail-qt0-f194.google.com ([209.85.216.194]:44445 "EHLO mail-qt0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753422AbeDXJ3K (ORCPT ); Tue, 24 Apr 2018 05:29:10 -0400 Received: by mail-qt0-f194.google.com with SMTP id j26-v6so21078738qtl.11 for ; Tue, 24 Apr 2018 02:29:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=67WMQg4X6NRq0fkPTN7gQQvihRhGKmfEzAJ2p7vDWgo=; b=fNKaL58crQm1gDwUXAN0E7PEXRgkgk47rm9L6ZAx0oaCC7JIOUUwftqJWgQGaCaLE8 6XaUnl8rjKwX2w1DayCDxjZsFomzXTZRFVe7zZytokMyB9K/53k5MPRlzEfpqaJ8N4UA EQMScrG5eGIdCkSCP1oCMPh4nir7S8sIuCzLk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=67WMQg4X6NRq0fkPTN7gQQvihRhGKmfEzAJ2p7vDWgo=; b=GLIP7lLwI0Vgjl9xXyEyT+olbkfM6VpXk/6S9YApv5/x83vPa5aGT28AsQd8N+7jXP ApmnUv1RSmRFm90Zyl6q2QKA20nS93ewN+2eeXMmFr1YjxYrVycyamFLu1XNvSNrFD+a 72B3Ec5lyb+94GNRwYMBV+/TBuR/cLswKVndcz1s3NT2xkc9fLLRkQ9TrQANcYJ8FEtJ Ud8paTEnwGra834rWwvM/AjpmJUhRpsix+zhfmNNDjZJ5rJztg9eBfpo+B8pQxmKu1St GuJ+HSgADKFPaZPJ9No8hnSXC00bQ1sG2104NTw6hiuGVMVX5yLaGQmYYBAvBPk+tHLQ tURg== X-Gm-Message-State: ALQs6tDjECXYC0U6WoDQwpkwILxe1UzQbknuazwDoFClJl66sCrJlbm8 EQrPZkJIyTTNXMwBjOIxOxG/VadYfeo= X-Google-Smtp-Source: AB8JxZra56tKPL4bQtn/Xr5zHp/gw6VNnOtKLp2i60gU3564bbRxxQQkABasS/5bj3CDbLwYMgJRGw== X-Received: by 10.12.228.133 with SMTP id n5mr1817207qvl.74.1524562149650; Tue, 24 Apr 2018 02:29:09 -0700 (PDT) Received: from localhost.localdomain.localdomain ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 96sm12615184qkx.71.2018.04.24.02.29.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Apr 2018 02:29:09 -0700 (PDT) From: Chaitra P B To: linux-scsi@vger.kernel.org Cc: Sathya.Prakash@broadcom.com, sreekanth.reddy@broadcom.com, suganath-prabu.subramani@broadcom.com, Chaitra P B Subject: [PATCH v3 03/14] mpt3sas: Lockless access for chain buffers. Date: Tue, 24 Apr 2018 05:28:32 -0400 Message-Id: <1524562123-33229-4-git-send-email-chaitra.basappa@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1524562123-33229-1-git-send-email-chaitra.basappa@broadcom.com> References: <1524562123-33229-1-git-send-email-chaitra.basappa@broadcom.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Introduces Chain lookup table/tracker and implements accessing chain buffer using smid. Removed link list based access of chain buffer which requires lock and allocated as many chains needed. Signed-off-by: Chaitra P B Signed-off-by: Suganath Prabu S --- drivers/scsi/mpt3sas/mpt3sas_base.c | 111 +++++++++++++++++++----------------- drivers/scsi/mpt3sas/mpt3sas_base.h | 8 ++- 2 files changed, 65 insertions(+), 54 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index b23a2eb..438da7c 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -297,12 +297,15 @@ static void * _base_get_chain_buffer_dma_to_chain_buffer(struct MPT3SAS_ADAPTER *ioc, dma_addr_t chain_buffer_dma) { - u16 index; - - for (index = 0; index < ioc->chain_depth; index++) { - if (ioc->chain_lookup[index].chain_buffer_dma == - chain_buffer_dma) - return ioc->chain_lookup[index].chain_buffer; + u16 index, j; + struct chain_tracker *ct; + + for (index = 0; index < ioc->scsiio_depth; index++) { + for (j = 0; j < ioc->chains_needed_per_io; j++) { + ct = &ioc->chain_lookup[index].chains_per_smid[j]; + if (ct && ct->chain_buffer_dma == chain_buffer_dma) + return ct->chain_buffer; + } } pr_info(MPT3SAS_FMT "Provided chain_buffer_dma address is not in the lookup list\n", @@ -1679,7 +1682,8 @@ _base_add_sg_single_64(void *paddr, u32 flags_length, dma_addr_t dma_addr) * @ioc: per adapter object * @scmd: SCSI commands of the IO request * - * Returns chain tracker(from ioc->free_chain_list) + * Returns chain tracker from chain_lookup table using key as + * smid and smid's chain_offset. */ static struct chain_tracker * _base_get_chain_buffer_tracker(struct MPT3SAS_ADAPTER *ioc, @@ -1687,20 +1691,15 @@ _base_get_chain_buffer_tracker(struct MPT3SAS_ADAPTER *ioc, { struct chain_tracker *chain_req; struct scsiio_tracker *st = scsi_cmd_priv(scmd); - unsigned long flags; + u16 smid = st->smid; + u8 chain_offset = + atomic_read(&ioc->chain_lookup[smid - 1].chain_offset); - spin_lock_irqsave(&ioc->scsi_lookup_lock, flags); - if (list_empty(&ioc->free_chain_list)) { - spin_unlock_irqrestore(&ioc->scsi_lookup_lock, flags); - dfailprintk(ioc, pr_warn(MPT3SAS_FMT - "chain buffers not available\n", ioc->name)); + if (chain_offset == ioc->chains_needed_per_io) return NULL; - } - chain_req = list_entry(ioc->free_chain_list.next, - struct chain_tracker, tracker_list); - list_del_init(&chain_req->tracker_list); - list_add_tail(&chain_req->tracker_list, &st->chain_list); - spin_unlock_irqrestore(&ioc->scsi_lookup_lock, flags); + + chain_req = &ioc->chain_lookup[smid - 1].chains_per_smid[chain_offset]; + atomic_inc(&ioc->chain_lookup[smid - 1].chain_offset); return chain_req; } @@ -3278,13 +3277,7 @@ void mpt3sas_base_clear_st(struct MPT3SAS_ADAPTER *ioc, return; st->cb_idx = 0xFF; st->direct_io = 0; - if (!list_empty(&st->chain_list)) { - unsigned long flags; - - spin_lock_irqsave(&ioc->scsi_lookup_lock, flags); - list_splice_init(&st->chain_list, &ioc->free_chain_list); - spin_unlock_irqrestore(&ioc->scsi_lookup_lock, flags); - } + atomic_set(&ioc->chain_lookup[st->smid - 1].chain_offset, 0); } /** @@ -4102,6 +4095,8 @@ static void _base_release_memory_pools(struct MPT3SAS_ADAPTER *ioc) { int i = 0; + int j = 0; + struct chain_tracker *ct; struct reply_post_struct *rps; dexitprintk(ioc, pr_info(MPT3SAS_FMT "%s\n", ioc->name, @@ -4192,14 +4187,18 @@ _base_release_memory_pools(struct MPT3SAS_ADAPTER *ioc) kfree(ioc->hpr_lookup); kfree(ioc->internal_lookup); if (ioc->chain_lookup) { - for (i = 0; i < ioc->chain_depth; i++) { - if (ioc->chain_lookup[i].chain_buffer) - dma_pool_free(ioc->chain_dma_pool, - ioc->chain_lookup[i].chain_buffer, - ioc->chain_lookup[i].chain_buffer_dma); + for (i = 0; i < ioc->scsiio_depth; i++) { + for (j = 0; j < ioc->chains_needed_per_io; j++) { + ct = &ioc->chain_lookup[i].chains_per_smid[j]; + if (ct && ct->chain_buffer) + dma_pool_free(ioc->chain_dma_pool, + ct->chain_buffer, + ct->chain_buffer_dma); + } + kfree(ioc->chain_lookup[i].chains_per_smid); } dma_pool_destroy(ioc->chain_dma_pool); - free_pages((ulong)ioc->chain_lookup, ioc->chain_pages); + kfree(ioc->chain_lookup); ioc->chain_lookup = NULL; } } @@ -4221,7 +4220,8 @@ _base_allocate_memory_pools(struct MPT3SAS_ADAPTER *ioc) u16 max_request_credit, nvme_blocks_needed; unsigned short sg_tablesize; u16 sge_size; - int i; + int i, j; + struct chain_tracker *ct; dinitprintk(ioc, pr_info(MPT3SAS_FMT "%s\n", ioc->name, __func__)); @@ -4502,15 +4502,24 @@ _base_allocate_memory_pools(struct MPT3SAS_ADAPTER *ioc) ioc->name, ioc->request, ioc->scsiio_depth)); ioc->chain_depth = min_t(u32, ioc->chain_depth, MAX_CHAIN_DEPTH); - sz = ioc->chain_depth * sizeof(struct chain_tracker); - ioc->chain_pages = get_order(sz); - ioc->chain_lookup = (struct chain_tracker *)__get_free_pages( - GFP_KERNEL, ioc->chain_pages); + sz = ioc->scsiio_depth * sizeof(struct chain_lookup); + ioc->chain_lookup = kzalloc(sz, GFP_KERNEL); if (!ioc->chain_lookup) { - pr_err(MPT3SAS_FMT "chain_lookup: __get_free_pages failed\n", - ioc->name); + pr_err(MPT3SAS_FMT "chain_lookup: __get_free_pages " + "failed\n", ioc->name); goto out; } + + sz = ioc->chains_needed_per_io * sizeof(struct chain_tracker); + for (i = 0; i < ioc->scsiio_depth; i++) { + ioc->chain_lookup[i].chains_per_smid = kzalloc(sz, GFP_KERNEL); + if (!ioc->chain_lookup[i].chains_per_smid) { + pr_err(MPT3SAS_FMT "chain_lookup: " + " kzalloc failed\n", ioc->name); + goto out; + } + } + ioc->chain_dma_pool = dma_pool_create("chain pool", &ioc->pdev->dev, ioc->chain_segment_sz, 16, 0); if (!ioc->chain_dma_pool) { @@ -4518,17 +4527,21 @@ _base_allocate_memory_pools(struct MPT3SAS_ADAPTER *ioc) ioc->name); goto out; } - for (i = 0; i < ioc->chain_depth; i++) { - ioc->chain_lookup[i].chain_buffer = dma_pool_alloc( + for (i = 0; i < ioc->scsiio_depth; i++) { + for (j = 0; j < ioc->chains_needed_per_io; j++) { + ct = &ioc->chain_lookup[i].chains_per_smid[j]; + ct->chain_buffer = dma_pool_alloc( ioc->chain_dma_pool , GFP_KERNEL, - &ioc->chain_lookup[i].chain_buffer_dma); - if (!ioc->chain_lookup[i].chain_buffer) { - ioc->chain_depth = i; - goto chain_done; + &ct->chain_buffer_dma); + if (!ct->chain_buffer) { + pr_err(MPT3SAS_FMT "chain_lookup: " + " pci_pool_alloc failed\n", ioc->name); + goto out; + } } total_sz += ioc->chain_segment_sz; } - chain_done: + dinitprintk(ioc, pr_info(MPT3SAS_FMT "chain pool depth(%d), frame_size(%d), pool_size(%d kB)\n", ioc->name, ioc->chain_depth, ioc->chain_segment_sz, @@ -6176,12 +6189,6 @@ _base_make_ioc_operational(struct MPT3SAS_ADAPTER *ioc) &ioc->internal_free_list); } - /* chain pool */ - INIT_LIST_HEAD(&ioc->free_chain_list); - for (i = 0; i < ioc->chain_depth; i++) - list_add_tail(&ioc->chain_lookup[i].tracker_list, - &ioc->free_chain_list); - spin_unlock_irqrestore(&ioc->scsi_lookup_lock, flags); /* initialize Reply Free Queue */ diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h index f997aad..01eb910 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.h +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h @@ -770,7 +770,11 @@ struct pcie_sg_list { struct chain_tracker { void *chain_buffer; dma_addr_t chain_buffer_dma; - struct list_head tracker_list; +}; + +struct chain_lookup { + struct chain_tracker *chains_per_smid; + atomic_t chain_offset; }; /** @@ -1261,7 +1265,7 @@ struct MPT3SAS_ADAPTER { u32 page_size; /* chain */ - struct chain_tracker *chain_lookup; + struct chain_lookup *chain_lookup; struct list_head free_chain_list; struct dma_pool *chain_dma_pool; ulong chain_pages;