From patchwork Tue Mar 26 07:38:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saurav Kashyap X-Patchwork-Id: 10870507 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DC8B115AC for ; Tue, 26 Mar 2019 07:39:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C585D28C2B for ; Tue, 26 Mar 2019 07:39:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B803228DD5; Tue, 26 Mar 2019 07:39:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E3AE28C2B for ; Tue, 26 Mar 2019 07:39:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730935AbfCZHjh (ORCPT ); Tue, 26 Mar 2019 03:39:37 -0400 Received: from mail-eopbgr800048.outbound.protection.outlook.com ([40.107.80.48]:9088 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726266AbfCZHjh (ORCPT ); Tue, 26 Mar 2019 03:39:37 -0400 Received: from DM6PR07CA0040.namprd07.prod.outlook.com (2603:10b6:5:74::17) by SN2PR07MB2541.namprd07.prod.outlook.com (2603:10b6:804:7::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.16; Tue, 26 Mar 2019 07:39:30 +0000 Received: from DM3NAM05FT049.eop-nam05.prod.protection.outlook.com (2a01:111:f400:7e51::207) by DM6PR07CA0040.outlook.office365.com (2603:10b6:5:74::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1750.15 via Frontend Transport; Tue, 26 Mar 2019 07:39:30 +0000 Authentication-Results: spf=fail (sender IP is 199.233.58.38) smtp.mailfrom=marvell.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=fail action=none header.from=marvell.com; Received-SPF: Fail (protection.outlook.com: domain of marvell.com does not designate 199.233.58.38 as permitted sender) receiver=protection.outlook.com; client-ip=199.233.58.38; helo=CAEXCH02.caveonetworks.com; Received: from CAEXCH02.caveonetworks.com (199.233.58.38) by DM3NAM05FT049.mail.protection.outlook.com (10.152.98.163) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.20.1750.4 via Frontend Transport; Tue, 26 Mar 2019 07:39:30 +0000 Received: from dut1171.mv.qlogic.com (10.112.88.18) by CAEXCH02.caveonetworks.com (10.67.98.110) with Microsoft SMTP Server (TLS) id 14.2.347.0; Tue, 26 Mar 2019 00:39:12 -0700 Received: from dut1171.mv.qlogic.com (localhost [127.0.0.1]) by dut1171.mv.qlogic.com (8.14.7/8.14.7) with ESMTP id x2Q7dBrx026843; Tue, 26 Mar 2019 00:39:11 -0700 Received: (from root@localhost) by dut1171.mv.qlogic.com (8.14.7/8.14.7/Submit) id x2Q7dB4L026842; Tue, 26 Mar 2019 00:39:11 -0700 From: Saurav Kashyap To: CC: , Subject: [PATCH v2 04/26] qedf: Simplify s/g list mapping. Date: Tue, 26 Mar 2019 00:38:36 -0700 Message-ID: <20190326073858.26792-5-skashyap@marvell.com> X-Mailer: git-send-email 2.12.0 In-Reply-To: <20190326073858.26792-1-skashyap@marvell.com> References: <20190326073858.26792-1-skashyap@marvell.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Matching-Connectors: 131980595706188909;(abac79dc-c90b-41ba-8033-08d666125e47);(abac79dc-c90b-41ba-8033-08d666125e47) X-Forefront-Antispam-Report: CIP:199.233.58.38;IPV:CAL;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10009020)(136003)(396003)(346002)(39860400002)(376002)(2980300002)(1109001)(1110001)(339900001)(189003)(199004)(2351001)(76176011)(36756003)(6862004)(53936002)(356004)(4326008)(6666004)(11346002)(476003)(2616005)(486006)(50466002)(51416003)(446003)(14444005)(126002)(48376002)(336012)(5660300002)(106466001)(86362001)(2906002)(105606002)(81166006)(47776003)(1076003)(26826003)(81156014)(97736004)(54906003)(305945005)(16586007)(80596001)(85426001)(87636003)(36906005)(26005)(42186006)(316002)(8936002)(50226002)(68736007)(498600001)(8676002)(69596002);DIR:OUT;SFP:1101;SCL:1;SRVR:SN2PR07MB2541;H:CAEXCH02.caveonetworks.com;FPR:;SPF:Fail;LANG:en;PTR:InfoDomainNonexistent;MX:1;A:1; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 80253259-98ad-4180-863e-08d6b1be2ec6 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(5600127)(711020)(4605104)(2017052603328);SRVR:SN2PR07MB2541; X-MS-TrafficTypeDiagnostic: SN2PR07MB2541: X-Microsoft-Antispam-PRVS: X-Forefront-PRVS: 09888BC01D X-Microsoft-Antispam-Message-Info: M6clexeUaUPMj8hz2yfcSYwOcTmXD//ojrpoWripepEv4e16DQ5Ctuj5FlF8Rvoa1u218er2WVOV975nqP/LcScJAWQH6StOGxt2uYSIOS/yuytB5M1zG4TSMhtVEtmEcDPbsZfzzGMz+O0Qg3a2dtlhAO4hPEbN12xURO2bwwwjh1YZ89jpmlyG0fpihV71O10yjYUPxJT6S2sCHNntwWxcejo2LFZiWvSRviovULrkEFnQtXzd4d3IsALqlp+MUzNBazFzzslx29fhOE9hG34ZdaYwSQ0uXoSXk33UBKiBcsASqvfUU4Ht3wl5Lqbj8kuMZq+DSNK2TTtppEJtpOAJuEE9FVIyp8AJipMmxCkGBoUBx2j+kfoinLI0uRv+9xuOuxq1Sg9iWwM8DMtGRd3gNIfzPZtXtkofk29WbrU= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Mar 2019 07:39:30.2716 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 80253259-98ad-4180-863e-08d6b1be2ec6 X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e;Ip=[199.233.58.38];Helo=[CAEXCH02.caveonetworks.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR07MB2541 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Chad Dupuis When mapping the pages from a scatter/gather list from the SCSI layer we only need to follow these rules: o Max SGEs for each I/O request is 256 o No size limit on each SGE o No need to split OS provided SGEs to 4K before sending to firmware. o Slow SGE is applicable only when: - There are > 8 SGEs and any middle SGE is less than a page size (4K) Make necessary changes so that driver follows these rules. applicable only for Write requests (not for Read requests) No need to check SGE address alignment requirements (first, middle or last) before declaring slow SGE Signed-off-by: Chad Dupuis Signed-off-by: Saurav Kashyap --- drivers/scsi/qedf/qedf.h | 8 +-- drivers/scsi/qedf/qedf_debugfs.c | 2 - drivers/scsi/qedf/qedf_io.c | 124 ++++++++++++--------------------------- 3 files changed, 40 insertions(+), 94 deletions(-) diff --git a/drivers/scsi/qedf/qedf.h b/drivers/scsi/qedf/qedf.h index 8e75c21..4080af8 100644 --- a/drivers/scsi/qedf/qedf.h +++ b/drivers/scsi/qedf/qedf.h @@ -102,7 +102,6 @@ struct qedf_ioreq { struct list_head link; uint16_t xid; struct scsi_cmnd *sc_cmd; - bool use_slowpath; /* Use slow SGL for this I/O */ #define QEDF_SCSI_CMD 1 #define QEDF_TASK_MGMT_CMD 2 #define QEDF_ABTS 3 @@ -152,9 +151,9 @@ struct qedf_ioreq { int fp_idx; unsigned int cpu; unsigned int int_cpu; -#define QEDF_IOREQ_SLOW_SGE 0 -#define QEDF_IOREQ_SINGLE_SGE 1 -#define QEDF_IOREQ_FAST_SGE 2 +#define QEDF_IOREQ_UNKNOWN_SGE 1 +#define QEDF_IOREQ_SLOW_SGE 2 +#define QEDF_IOREQ_FAST_SGE 3 u8 sge_type; struct delayed_work rrq_work; @@ -366,7 +365,6 @@ struct qedf_ctx { u32 slow_sge_ios; u32 fast_sge_ios; - u32 single_sge_ios; uint8_t *grcdump; uint32_t grcdump_size; diff --git a/drivers/scsi/qedf/qedf_debugfs.c b/drivers/scsi/qedf/qedf_debugfs.c index a32d8ee..6ae78dd 100644 --- a/drivers/scsi/qedf/qedf_debugfs.c +++ b/drivers/scsi/qedf/qedf_debugfs.c @@ -303,7 +303,6 @@ seq_printf(s, "cmg_mgr free io_reqs: %d\n", atomic_read(&qedf->cmd_mgr->free_list_cnt)); seq_printf(s, "slow SGEs: %d\n", qedf->slow_sge_ios); - seq_printf(s, "single SGEs: %d\n", qedf->single_sge_ios); seq_printf(s, "fast SGEs: %d\n\n", qedf->fast_sge_ios); seq_puts(s, "Offloaded ports:\n\n"); @@ -361,7 +360,6 @@ /* Clear stat counters exposed by 'stats' node */ qedf->slow_sge_ios = 0; - qedf->single_sge_ios = 0; qedf->fast_sge_ios = 0; return count; diff --git a/drivers/scsi/qedf/qedf_io.c b/drivers/scsi/qedf/qedf_io.c index da728d2..ed38df9 100644 --- a/drivers/scsi/qedf/qedf_io.c +++ b/drivers/scsi/qedf/qedf_io.c @@ -428,29 +428,6 @@ void qedf_release_cmd(struct kref *ref) clear_bit(QEDF_CMD_OUTSTANDING, &io_req->flags); } -static int qedf_split_bd(struct qedf_ioreq *io_req, u64 addr, int sg_len, - int bd_index) -{ - struct scsi_sge *bd = io_req->bd_tbl->bd_tbl; - int frag_size, sg_frags; - - sg_frags = 0; - while (sg_len) { - if (sg_len > QEDF_BD_SPLIT_SZ) - frag_size = QEDF_BD_SPLIT_SZ; - else - frag_size = sg_len; - bd[bd_index + sg_frags].sge_addr.lo = U64_LO(addr); - bd[bd_index + sg_frags].sge_addr.hi = U64_HI(addr); - bd[bd_index + sg_frags].sge_len = (uint16_t)frag_size; - - addr += (u64)frag_size; - sg_frags++; - sg_len -= frag_size; - } - return sg_frags; -} - static int qedf_map_sg(struct qedf_ioreq *io_req) { struct scsi_cmnd *sc = io_req->sc_cmd; @@ -462,75 +439,45 @@ static int qedf_map_sg(struct qedf_ioreq *io_req) int byte_count = 0; int sg_count = 0; int bd_count = 0; - int sg_frags; - unsigned int sg_len; + u32 sg_len; u64 addr, end_addr; - int i; + int i = 0; sg_count = dma_map_sg(&qedf->pdev->dev, scsi_sglist(sc), scsi_sg_count(sc), sc->sc_data_direction); - sg = scsi_sglist(sc); - /* - * New condition to send single SGE as cached-SGL with length less - * than 64k. - */ - if ((sg_count == 1) && (sg_dma_len(sg) <= - QEDF_MAX_SGLEN_FOR_CACHESGL)) { - sg_len = sg_dma_len(sg); - addr = (u64)sg_dma_address(sg); - - bd[bd_count].sge_addr.lo = (addr & 0xffffffff); - bd[bd_count].sge_addr.hi = (addr >> 32); - bd[bd_count].sge_len = (u16)sg_len; + io_req->sge_type = QEDF_IOREQ_UNKNOWN_SGE; - return ++bd_count; - } + if (sg_count <= 8 || io_req->io_req_flags == QEDF_READ) + io_req->sge_type = QEDF_IOREQ_FAST_SGE; scsi_for_each_sg(sc, sg, sg_count, i) { - sg_len = sg_dma_len(sg); + sg_len = (u32)sg_dma_len(sg); addr = (u64)sg_dma_address(sg); end_addr = (u64)(addr + sg_len); /* - * First s/g element in the list so check if the end_addr - * is paged aligned. Also check to make sure the length is - * at least page size. - */ - if ((i == 0) && (sg_count > 1) && - ((end_addr % QEDF_PAGE_SIZE) || - sg_len < QEDF_PAGE_SIZE)) - io_req->use_slowpath = true; - /* - * Last s/g element so check if the start address is paged - * aligned. - */ - else if ((i == (sg_count - 1)) && (sg_count > 1) && - (addr % QEDF_PAGE_SIZE)) - io_req->use_slowpath = true; - /* * Intermediate s/g element so check if start and end address - * is page aligned. + * is page aligned. Only required for writes and only if the + * number of scatter/gather elements is 8 or more. */ - else if ((i != 0) && (i != (sg_count - 1)) && - ((addr % QEDF_PAGE_SIZE) || (end_addr % QEDF_PAGE_SIZE))) - io_req->use_slowpath = true; + if (io_req->sge_type == QEDF_IOREQ_UNKNOWN_SGE && (i) && + (i != (sg_count - 1)) && sg_len < QEDF_PAGE_SIZE) + io_req->sge_type = QEDF_IOREQ_SLOW_SGE; - if (sg_len > QEDF_MAX_BD_LEN) { - sg_frags = qedf_split_bd(io_req, addr, sg_len, - bd_count); - } else { - sg_frags = 1; - bd[bd_count].sge_addr.lo = U64_LO(addr); - bd[bd_count].sge_addr.hi = U64_HI(addr); - bd[bd_count].sge_len = (uint16_t)sg_len; - } + bd[bd_count].sge_addr.lo = cpu_to_le32(U64_LO(addr)); + bd[bd_count].sge_addr.hi = cpu_to_le32(U64_HI(addr)); + bd[bd_count].sge_len = cpu_to_le32(sg_len); - bd_count += sg_frags; + bd_count++; byte_count += sg_len; } + /* To catch a case where FAST and SLOW nothing is set, set FAST */ + if (io_req->sge_type == QEDF_IOREQ_UNKNOWN_SGE) + io_req->sge_type = QEDF_IOREQ_FAST_SGE; + if (byte_count != scsi_bufflen(sc)) QEDF_ERR(&(qedf->dbg_ctx), "byte_count = %d != " "scsi_bufflen = %d, task_id = 0x%x.\n", byte_count, @@ -655,8 +602,10 @@ static void qedf_init_task(struct qedf_rport *fcport, struct fc_lport *lport, io_req->sgl_task_params->num_sges = bd_count; io_req->sgl_task_params->total_buffer_size = scsi_bufflen(io_req->sc_cmd); - io_req->sgl_task_params->small_mid_sge = - io_req->use_slowpath; + if (io_req->sge_type == QEDF_IOREQ_SLOW_SGE) + io_req->sgl_task_params->small_mid_sge = 1; + else + io_req->sgl_task_params->small_mid_sge = 0; } /* Fill in physical address of sense buffer */ @@ -679,16 +628,10 @@ static void qedf_init_task(struct qedf_rport *fcport, struct fc_lport *lport, io_req->task_retry_identifier, fcp_cmnd); /* Increment SGL type counters */ - if (bd_count == 1) { - qedf->single_sge_ios++; - io_req->sge_type = QEDF_IOREQ_SINGLE_SGE; - } else if (io_req->use_slowpath) { + if (io_req->sge_type == QEDF_IOREQ_SLOW_SGE) qedf->slow_sge_ios++; - io_req->sge_type = QEDF_IOREQ_SLOW_SGE; - } else { + else qedf->fast_sge_ios++; - io_req->sge_type = QEDF_IOREQ_FAST_SGE; - } } void qedf_init_mp_task(struct qedf_ioreq *io_req, @@ -770,9 +713,6 @@ void qedf_init_mp_task(struct qedf_ioreq *io_req, &task_fc_hdr, &tx_sgl_task_params, &rx_sgl_task_params, 0); - - /* Midpath requests always consume 1 SGE */ - qedf->single_sge_ios++; } /* Presumed that fcport->rport_lock is held */ @@ -872,7 +812,7 @@ int qedf_post_io_req(struct qedf_rport *fcport, struct qedf_ioreq *io_req) /* Initialize rest of io_req fileds */ io_req->data_xfer_len = scsi_bufflen(sc_cmd); sc_cmd->SCp.ptr = (char *)io_req; - io_req->use_slowpath = false; /* Assume fast SGL by default */ + io_req->sge_type = QEDF_IOREQ_FAST_SGE; /* Assume fast SGL by default */ /* Record which cpu this request is associated with */ io_req->cpu = smp_processor_id(); @@ -942,7 +882,17 @@ int qedf_post_io_req(struct qedf_rport *fcport, struct qedf_ioreq *io_req) int rc = 0; int rval; unsigned long flags = 0; - + int num_sgs = 0; + + num_sgs = scsi_sg_count(sc_cmd); + if (scsi_sg_count(sc_cmd) > QEDF_MAX_BDS_PER_CMD) { + QEDF_ERR(&qedf->dbg_ctx, + "Number of SG elements %d exceeds what hardware limitation of %d.\n", + num_sgs, QEDF_MAX_BDS_PER_CMD); + sc_cmd->result = DID_ERROR; + sc_cmd->scsi_done(sc_cmd); + return 0; + } if (test_bit(QEDF_UNLOADING, &qedf->flags) || test_bit(QEDF_DBG_STOP_IO, &qedf->flags)) {