From patchwork Wed Feb 7 10:51:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suganath Prabu S X-Patchwork-Id: 10204973 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3498F60247 for ; Wed, 7 Feb 2018 10:52:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8826C27F8E for ; Wed, 7 Feb 2018 10:52:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7B283283F9; Wed, 7 Feb 2018 10:52:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A68EC27F8E for ; Wed, 7 Feb 2018 10:52:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753788AbeBGKwY (ORCPT ); Wed, 7 Feb 2018 05:52:24 -0500 Received: from mail-qt0-f196.google.com ([209.85.216.196]:39304 "EHLO mail-qt0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753913AbeBGKwU (ORCPT ); Wed, 7 Feb 2018 05:52:20 -0500 Received: by mail-qt0-f196.google.com with SMTP id f4so599884qtj.6 for ; Wed, 07 Feb 2018 02:52:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lFCfmX5MRhKdaELxp0JK4uGV8s7d8G6eBX+r1oEMpfs=; b=YHX5zvEIrP8LOYP/N4b9VghHrTAA5n+VNVmrFdbKZ2y0yunybcAPD9ppCAU2GPY6/N 2IwK/QdU/nZEHDaUcgXsV/KTtZZzmeWKjo4qxPghAgLjOzuHE32gR2YSc604jVvyesco zItupbT3rFRehKjWlNi1NbEfywUXDZv5J1e/g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lFCfmX5MRhKdaELxp0JK4uGV8s7d8G6eBX+r1oEMpfs=; b=BuwSuUd5eSMc0gQ16FczYCaethFOIEZweRqkpNwwfv2a/LnZPp69jbilI5S9YJDR1L hH7evUpuyp6iwKZhfjeZD8YesY9dR4T+JmDu2FYvP2+ls4lL/9qzYYZV74PVen5sI0kQ rMi0i/qSDvcpiCxcDnk6LTw4ccakrZN84LS0cw76rQv3to8EHItyNAOjVt1SVVcKdhIu HxOYiQqkiEbFGAxTbKSDTJOPaZPkzp8z+zaC1539jn4SF6T6ttKbNJY/Ok6Fjij9gUc7 cGMDwlsi0IwuvtORsrUtg/iRmLvfkRylL+5yE7iIF6EDpr2jYphg8MHh3HiFBQVkM8vE KcVQ== X-Gm-Message-State: APf1xPAjOJx8IJ4SvNYx2rf7iiEMQ9vlfEkN42e9w5e1kNBF7CRQqkIZ LQBz8T8YUcKcntH3WEpn1P32x+GP X-Google-Smtp-Source: AH8x225M0LROi8dncHwBovlXUE5iOOY5gvfXoCDHyOEEewBMLJkjNKqA5aBn39ijC9TYNAv7s1FGhA== X-Received: by 10.237.35.38 with SMTP id h35mr8742807qtc.300.1518000739320; Wed, 07 Feb 2018 02:52:19 -0800 (PST) Received: from dhcp-135-24-192-103.dhcp.broadcom.net ([192.19.252.250]) by smtp.googlemail.com with ESMTPSA id a63sm834005qkj.70.2018.02.07.02.52.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 07 Feb 2018 02:52:18 -0800 (PST) From: Suganath Prabu S To: linux-scsi@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Sathya.Prakash@broadcom.com, sreekanth.reddy@broadcom.com, chaitra.basappa@broadcom.com, Suganath Prabu S Subject: [V1 4/6] mpt3sas: Introduce Base function for cloning. Date: Wed, 7 Feb 2018 02:51:48 -0800 Message-Id: <1518000710-17256-5-git-send-email-suganath-prabu.subramani@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1518000710-17256-1-git-send-email-suganath-prabu.subramani@broadcom.com> References: <1518000710-17256-1-git-send-email-suganath-prabu.subramani@broadcom.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP All scsi IO's and config requests data buffer and sgl are cloned to system memory in _clone_sg_entries before submitting it to Firmware. Signed-off-by: Suganath Prabu S --- drivers/scsi/mpt3sas/mpt3sas_base.c | 215 +++++++++++++++++++++++++++++++++- drivers/scsi/mpt3sas/mpt3sas_base.h | 3 + drivers/scsi/mpt3sas/mpt3sas_config.c | 1 + 3 files changed, 218 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 36f1242..c41c65b 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -126,6 +126,24 @@ module_param_call(mpt3sas_fwfault_debug, _scsih_set_fwfault_debug, param_get_int, &mpt3sas_fwfault_debug, 0644); /** + * _base_clone_to_sys_mem - Writes/copies data to system/BAR0 region + * + * @dst_iomem: Pointer to the destinaltion location in BAR0 space. + * @src: Pointer to the Source data. + * @size: Size of data to be copied. + */ +static void +_base_clone_to_sys_mem(void __iomem *dst_iomem, void *src, u32 size) +{ + int i; + u32 *src_virt_mem = (u32 *)(src); + + for (i = 0; i < size/4; i++) + writel((u32)src_virt_mem[i], + (void __iomem *)dst_iomem + (i * 4)); +} + +/** * _base_get_chain - Calculates and Returns virtual chain address * for the provided smid in BAR0 space. * @@ -219,6 +237,201 @@ _base_get_buffer_phys_bar0(struct MPT3SAS_ADAPTER *ioc, u16 smid) } /** + * _base_get_chain_buffer_dma_to_chain_buffer - Iterates chain + * lookup list and Provides chain_buffer + * address for the matching dma address. + * (Each smid can have 64K starts from 17024) + * + * @ioc: per adapter object + * @chain_buffer_dma: Chain buffer dma address. + * + * @Returns - Pointer to chain buffer. Or Null on Failure. + */ +static void * +_base_get_chain_buffer_dma_to_chain_buffer(struct MPT3SAS_ADAPTER *ioc, + dma_addr_t chain_buffer_dma) +{ + u16 index; + + for (index = 0; index < ioc->chain_depth; index++) { + if (ioc->chain_lookup[index].chain_buffer_dma == + chain_buffer_dma) + return ioc->chain_lookup[index].chain_buffer; + } + pr_info(MPT3SAS_FMT + "Provided chain_buffer_dma address is not in the lookup list\n", + ioc->name); + return NULL; +} + +/** + * _clone_sg_entries - MPI EP's scsiio and config requests + * are handled here. Base function for + * double buffering, before submitting + * the requests. + * + * @ioc: per adapter object. + * @mpi_request: mf request pointer. + * @smid: system request message index. + * + * @Returns: Nothing. + */ +static void _clone_sg_entries(struct MPT3SAS_ADAPTER *ioc, + void *mpi_request, u16 smid) +{ + Mpi2SGESimple32_t *sgel, *sgel_next; + u32 sgl_flags, sge_chain_count = 0; + bool is_write = 0; + u16 i = 0; + void __iomem *buffer_iomem; + void *buffer_iomem_phys; + void __iomem *buff_ptr; + void *buff_ptr_phys; + void __iomem *dst_chain_addr[MCPU_MAX_CHAINS_PER_IO]; + void *src_chain_addr[MCPU_MAX_CHAINS_PER_IO], *dst_addr_phys; + MPI2RequestHeader_t *request_hdr; + struct scsi_cmnd *scmd; + struct scatterlist *sg_scmd = NULL; + int is_scsiio_req = 0; + + request_hdr = (MPI2RequestHeader_t *) mpi_request; + + if (request_hdr->Function == MPI2_FUNCTION_SCSI_IO_REQUEST) { + Mpi25SCSIIORequest_t *scsiio_request = + (Mpi25SCSIIORequest_t *)mpi_request; + sgel = (Mpi2SGESimple32_t *) &scsiio_request->SGL; + is_scsiio_req = 1; + } else if (request_hdr->Function == MPI2_FUNCTION_CONFIG) { + Mpi2ConfigRequest_t *config_req = + (Mpi2ConfigRequest_t *)mpi_request; + sgel = (Mpi2SGESimple32_t *) &config_req->PageBufferSGE; + } else + return; + + /* From smid we can get scsi_cmd, once we have sg_scmd, + * we just need to get sg_virt and sg_next to get virual + * address associated with sgel->Address. + */ + + if (is_scsiio_req) { + /* Get scsi_cmd using smid */ + scmd = mpt3sas_scsih_scsi_lookup_get(ioc, smid); + if (scmd == NULL) { + pr_err(MPT3SAS_FMT "scmd is NULL\n", ioc->name); + return; + } + + /* Get sg_scmd from scmd provided */ + sg_scmd = scsi_sglist(scmd); + } + + /* + * 0 - 255 System register + * 256 - 4352 MPI Frame. (This is based on maxCredit 32) + * 4352 - 4864 Reply_free pool (512 byte is reserved + * considering maxCredit 32. Reply need extra + * room, for mCPU case kept four times of + * maxCredit). + * 4864 - 17152 SGE chain element. (32cmd * 3 chain of + * 128 byte size = 12288) + * 17152 - x Host buffer mapped with smid. + * (Each smid can have 64K Max IO.) + * BAR0+Last 1K MSIX Addr and Data + * Total size in use 2113664 bytes of 4MB BAR0 + */ + + buffer_iomem = _base_get_buffer_bar0(ioc, smid); + buffer_iomem_phys = _base_get_buffer_phys_bar0(ioc, smid); + + buff_ptr = buffer_iomem; + buff_ptr_phys = buffer_iomem_phys; + + if (sgel->FlagsLength & + (MPI2_SGE_FLAGS_HOST_TO_IOC << MPI2_SGE_FLAGS_SHIFT)) + is_write = 1; + + for (i = 0; i < MPT_MIN_PHYS_SEGMENTS + ioc->facts.MaxChainDepth; i++) { + + sgl_flags = (sgel->FlagsLength >> MPI2_SGE_FLAGS_SHIFT); + + switch (sgl_flags & MPI2_SGE_FLAGS_ELEMENT_MASK) { + case MPI2_SGE_FLAGS_CHAIN_ELEMENT: + /* + * Helper function which on passing + * chain_buffer_dma returns chain_buffer. Get + * the virtual address for sgel->Address + */ + sgel_next = + _base_get_chain_buffer_dma_to_chain_buffer(ioc, + sgel->Address); + if (sgel_next == NULL) + return; + /* + * This is coping 128 byte chain + * frame (not a host buffer) + */ + dst_chain_addr[sge_chain_count] = + _base_get_chain(ioc, + smid, sge_chain_count); + src_chain_addr[sge_chain_count] = + (void *) sgel_next; + dst_addr_phys = + _base_get_chain_phys(ioc, + smid, sge_chain_count); + sgel->Address = (dma_addr_t)dst_addr_phys; + sgel = sgel_next; + sge_chain_count++; + break; + case MPI2_SGE_FLAGS_SIMPLE_ELEMENT: + if (is_write) { + if (is_scsiio_req) { + _base_clone_to_sys_mem(buff_ptr, + sg_virt(sg_scmd), + (sgel->FlagsLength & 0x00ffffff)); + sgel->Address = + (dma_addr_t)buff_ptr_phys; + } else { + _base_clone_to_sys_mem(buff_ptr, + ioc->config_vaddr, + (sgel->FlagsLength & 0x00ffffff)); + sgel->Address = + (dma_addr_t)buff_ptr_phys; + } + } + buff_ptr += (sgel->FlagsLength & 0x00ffffff); + buff_ptr_phys += (sgel->FlagsLength & 0x00ffffff); + if ((sgel->FlagsLength & + (MPI2_SGE_FLAGS_END_OF_BUFFER + << MPI2_SGE_FLAGS_SHIFT))) + goto eob_clone_chain; + else { + /* + * Every single element in MPT will have + * associated sg_next. Better to sanity that + * sg_next is not NULL, but it will be a bug + * if it is null. + */ + if (is_scsiio_req) { + sg_scmd = sg_next(sg_scmd); + if (sg_scmd) + sgel++; + else + goto eob_clone_chain; + } + } + break; + } + } + +eob_clone_chain: + for (i = 0; i < sge_chain_count; i++) { + if (is_scsiio_req) + _base_clone_to_sys_mem(dst_chain_addr[i], + src_chain_addr[i], ioc->request_sz); + } +} + +/** * mpt3sas_remove_dead_ioc_func - kthread context to remove dead ioc * @arg: input argument, used to derive ioc * @@ -3295,7 +3508,7 @@ _base_put_smid_nvme_encap_atomic(struct MPT3SAS_ADAPTER *ioc, u16 smid) /** * _base_put_smid_default - Default, primarily used for config pages - * use Atomic Request Descriptor + * use Atomic Request Descriptor * @ioc: per adapter object * @smid: system request message index * diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h index 2529d25..4fd582b 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.h +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h @@ -95,6 +95,8 @@ #define MPT_MIN_PHYS_SEGMENTS 16 #define MPT_KDUMP_MIN_PHYS_SEGMENTS 32 +#define MCPU_MAX_CHAINS_PER_IO 3 + #ifdef CONFIG_SCSI_MPT3SAS_MAX_SGE #define MPT3SAS_SG_DEPTH CONFIG_SCSI_MPT3SAS_MAX_SGE #else @@ -1238,6 +1240,7 @@ struct MPT3SAS_ADAPTER { u16 config_page_sz; void *config_page; dma_addr_t config_page_dma; + void *config_vaddr; /* scsiio request */ u16 hba_queue_depth; diff --git a/drivers/scsi/mpt3sas/mpt3sas_config.c b/drivers/scsi/mpt3sas/mpt3sas_config.c index 1c747cf..0dba3c4 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_config.c +++ b/drivers/scsi/mpt3sas/mpt3sas_config.c @@ -219,6 +219,7 @@ _config_alloc_config_dma_memory(struct MPT3SAS_ADAPTER *ioc, mem->page = ioc->config_page; mem->page_dma = ioc->config_page_dma; } + ioc->config_vaddr = mem->page; return r; }