From patchwork Fri Nov 10 01:08:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10052227 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 47C9E6032D for ; Fri, 10 Nov 2017 01:09:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3AABC2B225 for ; Fri, 10 Nov 2017 01:09:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2F8272B228; Fri, 10 Nov 2017 01:09:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AB3C22B225 for ; Fri, 10 Nov 2017 01:09:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755549AbdKJBJE (ORCPT ); Thu, 9 Nov 2017 20:09:04 -0500 Received: from mail-qt0-f169.google.com ([209.85.216.169]:49506 "EHLO mail-qt0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755540AbdKJBI7 (ORCPT ); Thu, 9 Nov 2017 20:08:59 -0500 Received: by mail-qt0-f169.google.com with SMTP id y45so3921002qty.6 for ; Thu, 09 Nov 2017 17:08:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=OG5HHcUdu7oZts8Cl+1s+xsZ8uEXP8nvRMOskhwDX2I=; b=tD18vfdmEMokhPeljo/Kqq4dmH/4D0GaRtddoClkQ3OnPgJaarJMlOIZXhmZc+9WlR vFLZUzojXN3BndK2QvUxdbwcEgzbZA61vS1qIY+5RRPS9ybKX3TVFNH2UHLDGbiMXGlE lf6Z8QeG4ybfyPx8UrpLgMcy6vc8viNCBmujEFHrDKHBHwo/3MStWXMelIzSaWN1cdwV 28G6dQ4Jtl5zm+egt54/AmM7xyaUl0MBGF5atvO1l423yuNdyzMWg51cFv6dPaC81zZ9 V3cWvInVe78bqSkXZRHncQUqDkbwedGT/3YSl2uATlJ6jUV5dhnqNWCNgpXVhzC49Zcs yzmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=OG5HHcUdu7oZts8Cl+1s+xsZ8uEXP8nvRMOskhwDX2I=; b=jrHG+IRFs2Z0z4j9g3qYcj9uBJJz6CUsL4/pEZRJNc8yRs4aLqqmI/fAPTje/ofGMo hCuUf3kYsuoyaIpcWsQY5sGGQZM8P5cXChy9bicck4eGetCM+YrHK7kAn6rXQcwKSvWU 9UfadcEDw2Ag7mKgZRWCtAaGDMVfX4Kc/FkLUxx6uvXwqzAINmhoMnRIb+rxhwUMoam1 TmifWs/sH47BUGTBVqoucg34cOsHOmI+vpSjGaEbpFLtfLkN04AiB+GXKKDwybuxroRu ofnUY8BZDoHJxNZyWb5WyylWli1zkPE9ncuTxqElBrDqO6lxwGsndq0y0Vikclcns+VR aUGQ== X-Gm-Message-State: AJaThX7Wo60DWw9WIoeTAg70nwqv+3Kwo4HPXdxC3ERWKU3vazeopICd eERC3oC+OZCzDM4JzS5aYDEP0I9V X-Google-Smtp-Source: AGs4zMazUkLj/dF3k/pdoiyibOK1KBkNLXhnXDU42d5nDUdnwyPcAzE3WSVRVjbXh3DncrSgPAt9Bw== X-Received: by 10.237.61.49 with SMTP id g46mr4103644qtf.233.1510276138458; Thu, 09 Nov 2017 17:08:58 -0800 (PST) Received: from pallmd1.broadcom.com ([192.19.255.250]) by smtp.gmail.com with ESMTPSA id j48sm5661969qta.96.2017.11.09.17.08.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 09 Nov 2017 17:08:58 -0800 (PST) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy , James Smart Subject: [PATCH v2 16/17] lpfc: small sg cnt cleanup Date: Thu, 9 Nov 2017 17:08:33 -0800 Message-Id: <20171110010834.4542-17-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.1 In-Reply-To: <20171110010834.4542-1-jsmart2021@gmail.com> References: <20171110010834.4542-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The logic for sg_seg_cnt is a bit convoluted. This patch tries to clean up a couple of areas, especially around the +2 and +1 logic. This patch: - cleans up the lpfc_sg_seg_cnt attribute to specify a real minimum rather than making the minimum be whatever the default is. - Remove the hardcoding of +2 (for the number of elements we use in a sgl for cmd iu and rsp iu) and +1 (an additional entry to compensate for nvme's reduction of io size based on a possible partial page) logic in sg list initialization. In the case where the +1 logic is referenced in host and target io checks, use the values set in the transport template as that value was properly set. There can certainly be more done in this area and it will be addressed in combined host/target driver effort. Signed-off-by: Dick Kennedy Signed-off-by: James Smart Reviewed-by: Hannes Reinecke --- drivers/scsi/lpfc/lpfc.h | 1 + drivers/scsi/lpfc/lpfc_attr.c | 2 +- drivers/scsi/lpfc/lpfc_init.c | 19 ++++++++++++++----- drivers/scsi/lpfc/lpfc_nvme.c | 3 ++- drivers/scsi/lpfc/lpfc_nvmet.c | 2 +- 5 files changed, 19 insertions(+), 8 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h index 46a89bdff8e4..dd2191c83052 100644 --- a/drivers/scsi/lpfc/lpfc.h +++ b/drivers/scsi/lpfc/lpfc.h @@ -55,6 +55,7 @@ struct lpfc_sli2_slim; #define LPFC_MAX_SG_SLI4_SEG_CNT_DIF 128 /* sg element count per scsi cmnd */ #define LPFC_MAX_SG_SEG_CNT_DIF 512 /* sg element count per scsi cmnd */ #define LPFC_MAX_SG_SEG_CNT 4096 /* sg element count per scsi cmnd */ +#define LPFC_MIN_SG_SEG_CNT 32 /* sg element count per scsi cmnd */ #define LPFC_MAX_SGL_SEG_CNT 512 /* SGL element count per scsi cmnd */ #define LPFC_MAX_BPL_SEG_CNT 4096 /* BPL element count per scsi cmnd */ #define LPFC_MAX_NVME_SEG_CNT 256 /* max SGL element cnt per NVME cmnd */ diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c index 598e07f43912..74d6fe984df4 100644 --- a/drivers/scsi/lpfc/lpfc_attr.c +++ b/drivers/scsi/lpfc/lpfc_attr.c @@ -5135,7 +5135,7 @@ LPFC_ATTR(delay_discovery, 0, 0, 1, * this parameter will be limited to 128 if BlockGuard is enabled under SLI4 * and will be limited to 512 if BlockGuard is enabled under SLI3. */ -LPFC_ATTR_R(sg_seg_cnt, LPFC_DEFAULT_SG_SEG_CNT, LPFC_DEFAULT_SG_SEG_CNT, +LPFC_ATTR_R(sg_seg_cnt, LPFC_MIN_SG_SEG_CNT, LPFC_DEFAULT_SG_SEG_CNT, LPFC_MAX_SG_SEG_CNT, "Max Scatter Gather Segment Count"); /* diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index c466ceb43bc9..92dc865ca52c 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -5812,6 +5812,7 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) struct lpfc_mqe *mqe; int longs; int fof_vectors = 0; + int extra; uint64_t wwn; phba->sli4_hba.num_online_cpu = num_online_cpus(); @@ -5867,13 +5868,21 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) */ /* + * 1 for cmd, 1 for rsp, NVME adds an extra one + * for boundary conditions in its max_sgl_segment template. + */ + extra = 2; + if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) + extra++; + + /* * It doesn't matter what family our adapter is in, we are * limited to 2 Pages, 512 SGEs, for our SGL. * There are going to be 2 reserved SGEs: 1 FCP cmnd + 1 FCP rsp */ max_buf_size = (2 * SLI4_PAGE_SIZE); - if (phba->cfg_sg_seg_cnt > LPFC_MAX_SGL_SEG_CNT - 2) - phba->cfg_sg_seg_cnt = LPFC_MAX_SGL_SEG_CNT - 2; + if (phba->cfg_sg_seg_cnt > LPFC_MAX_SGL_SEG_CNT - extra) + phba->cfg_sg_seg_cnt = LPFC_MAX_SGL_SEG_CNT - extra; /* * Since lpfc_sg_seg_cnt is module param, the sg_dma_buf_size @@ -5906,14 +5915,14 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) */ phba->cfg_sg_dma_buf_size = sizeof(struct fcp_cmnd) + sizeof(struct fcp_rsp) + - ((phba->cfg_sg_seg_cnt + 2) * + ((phba->cfg_sg_seg_cnt + extra) * sizeof(struct sli4_sge)); /* Total SGEs for scsi_sg_list */ - phba->cfg_total_seg_cnt = phba->cfg_sg_seg_cnt + 2; + phba->cfg_total_seg_cnt = phba->cfg_sg_seg_cnt + extra; /* - * NOTE: if (phba->cfg_sg_seg_cnt + 2) <= 256 we only + * NOTE: if (phba->cfg_sg_seg_cnt + extra) <= 256 we only * need to post 1 page for the SGL. */ } diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index 67bd0ff299d1..0a42956058e9 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -62,6 +62,7 @@ lpfc_get_nvme_buf(struct lpfc_hba *phba, struct lpfc_nodelist *ndlp); static void lpfc_release_nvme_buf(struct lpfc_hba *, struct lpfc_nvme_buf *); +static struct nvme_fc_port_template lpfc_nvme_template; /** * lpfc_nvme_create_queue - @@ -1174,7 +1175,7 @@ lpfc_nvme_prep_io_dma(struct lpfc_vport *vport, first_data_sgl = sgl; lpfc_ncmd->seg_cnt = nCmd->sg_cnt; - if (lpfc_ncmd->seg_cnt > phba->cfg_nvme_seg_cnt + 1) { + if (lpfc_ncmd->seg_cnt > lpfc_nvme_template.max_sgl_segments) { lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, "6058 Too many sg segments from " "NVME Transport. Max %d, " diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index 2b50aecc2722..d80cd1def3b9 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -2003,7 +2003,7 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba, return NULL; } - if (rsp->sg_cnt > phba->cfg_nvme_seg_cnt) { + if (rsp->sg_cnt > lpfc_tgttemplate.max_sgl_segments) { lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, "6109 NVMET prep FCP wqe: seg cnt err: " "NPORT x%x oxid x%x ste %d cnt %d\n",