From patchwork Tue Oct 3 08:54:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kalderon, Michal" X-Patchwork-Id: 9981943 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6441C6029B for ; Tue, 3 Oct 2017 08:55:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 564471FFCA for ; Tue, 3 Oct 2017 08:55:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4B22B286FD; Tue, 3 Oct 2017 08:55:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DCC01FFCA for ; Tue, 3 Oct 2017 08:55:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751200AbdJCIzh (ORCPT ); Tue, 3 Oct 2017 04:55:37 -0400 Received: from mail-dm3nam03on0070.outbound.protection.outlook.com ([104.47.41.70]:39053 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751156AbdJCIzf (ORCPT ); Tue, 3 Oct 2017 04:55:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=T8OZ1n9xL/fNtGaTA4zcfz6jXrpA/dxGPv5/bNjOPWM=; b=eFOlWGzlXD1uqPWTwmPk8V6EK2zVdD86edQo5sbWhqqy3uFMtmRpBmhapYGHNjZw0DcwWnH8tKVlyaGkd3IC4DBTN4bixM4MQxalhQ5kOqLx2kQAsdwa0ug94qx0xKaDMPCt5g/d4vvxnmu/ZxTOsk09mq5pchoGoDLS2r494gc= Received: from SN1PR0701CA0054.namprd07.prod.outlook.com (2a01:111:e400:52fd::22) by CO1PR07MB894.namprd07.prod.outlook.com (2a01:111:e400:101f::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Tue, 3 Oct 2017 08:55:30 +0000 Received: from BY2FFO11FD042.protection.gbl (2a01:111:f400:7c0c::151) by SN1PR0701CA0054.outlook.office365.com (2a01:111:e400:52fd::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.8 via Frontend Transport; Tue, 3 Oct 2017 08:55:30 +0000 Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) Received: from CAEXCH02.caveonetworks.com (50.232.66.26) by BY2FFO11FD042.mail.protection.outlook.com (10.1.14.227) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.20.77.10 via Frontend Transport; Tue, 3 Oct 2017 08:55:29 +0000 Received: from lb-tlvb-michal.il.qlogic.com.com (10.185.6.89) by CAEXCH02.caveonetworks.com (10.17.4.29) with Microsoft SMTP Server id 14.2.347.0; Tue, 3 Oct 2017 01:55:27 -0700 From: Michal Kalderon To: CC: , , , Michal Kalderon , "Ariel Elior" Subject: [PATCH v2 net-next 01/12] qed: Add ll2 option to limit the number of bds per packet Date: Tue, 3 Oct 2017 11:54:51 +0300 Message-ID: <1507020902-4952-2-git-send-email-Michal.Kalderon@cavium.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1507020902-4952-1-git-send-email-Michal.Kalderon@cavium.com> References: <1507020902-4952-1-git-send-email-Michal.Kalderon@cavium.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:50.232.66.26; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(376002)(346002)(2980300002)(428002)(199003)(189002)(2906002)(6666003)(478600001)(16586007)(77096006)(106466001)(36756003)(316002)(5003940100001)(105586002)(104016004)(72206003)(107886003)(50226002)(69596002)(54906003)(2351001)(4326008)(6916009)(81166006)(8676002)(81156014)(8936002)(47776003)(101416001)(189998001)(2950100002)(305945005)(48376002)(86362001)(356003)(5660300001)(50986999)(76176999)(50466002); DIR:OUT; SFP:1101; SCL:1; SRVR:CO1PR07MB894; H:CAEXCH02.caveonetworks.com; FPR:; SPF:None; PTR:50-232-66-26-static.hfc.comcastbusiness.net; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD042; 1:AT1plsVJIuqG6NNOHklT9SgP7f+l4ek7s2CFVQ/Q1D88diK+51q7Zd5Rig9A7bZTAH2zrVzrzv8KBlWWMEwXyE9tBEFEXvJArj8IONaBUuKbazCLPmL+ruuRPg37WXhN X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c6fb1968-ac40-462e-9c55-08d50a3c7fd6 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:CO1PR07MB894; X-Microsoft-Exchange-Diagnostics: 1; CO1PR07MB894; 3:vzCm3W4JeNPJAcm+a5IxrqQ5VAoh/SZKjPdXD279Vo8kwaCjwNyX//U058TeA4G7FkCCJNQLofT2qj8x6FTKMgpguAIZCM/ByLsJkhC5VM/g4Gld8lghScPKdYS4VGC7aPXSua0eS4EpkwtkAqC2ZKZdmFz5kRPFSBSeMYkgqVAdROdDE+ZGNeSCzg8+vyHb7QV5v4AeyDRRbXwjm9LSLpGHTKwCBjycFZ9HCJIf5u5ihvs3xYVE/yR1e1mBLH1nB80iNu1hkO9OxUBD1MuHQro2FmunftrToHOiX8Ksr/cXJyFBqaN/uCFSnLeJWDClcqDtE9J3mcApy3ia8mQvBw==; 25:jOQqq3jeIfwsHD8gKX4QNHXgUY/iMhhivF7vA3a4pYtRIM0INBfQtNzsPT2WKTB5NNG/B0kR7R1Q6/75qch47sjOwYt7egNYs4wsZe+Rxn1Fn11LiDXySQH7ofjel72X72U6SJPRHqW1wdaKT2pVdB1A5otDoA//kK8u6s3Zs7ZCMi6d5PnaGHXe8C4JLE2/weCcW/8fqkLLeFCC6Ebe15i7eEZ29eXpsmjCU4GdAbqdpfURlJxbI86j4CD+67iMdo8fSyEUD5ScOOYP+4ZjwHgWBEHtBkKxlz8DqRrfgu3SSI5G0aNddHEPGgBT9rdlO++K7EqhF4+A252y+qp1MA==; 31:YkYzkz4J1Q3XyIkgnHu1LQYOgECfajl0dYcmBer9YmkCKldVTXFS7otqZhHOJFYROlNIzDAkDgWNPV3yDbYTj9gygn7j/8Y+xU7C1suMGYd63ONGjaRq1CqZ1RLeAkAEfwQIjkLEsDpa1gw3AXAoYWJ6UZf7bx4tT/K0POmaMq2ya27Ee5JuPatkceOedN7lXt6crio/hXxk1LoIRZdZ+5HS8ThGN0ZDpcIA/Nuvim0= X-MS-TrafficTypeDiagnostic: CO1PR07MB894: X-Microsoft-Exchange-Diagnostics: 1; CO1PR07MB894; 20:oQGGuL+zztzIa/+pyW4LTo1mlZXtoZQXaa1xbwgkbyuFPqQzSh+3sF5j2b3uBnD706az8YuQJf3mMbKD7xsI6NeoPUIXQkErUnC72xIp1W1HuBcskVlPT+iR/9bGZlAkjnhufohTiS5A+8lnaJRI7FtJdkBxGkbd5BPC8Mcab99SPzbOCG/Qqd+wYyN3kksJHNNHSw5LZ/W1wEJBkexIQK+URnU40V76JgGkbUZncvA8ExiLFJ0NPdWnmDaszX+XYIMRFuwS1C8dpgnDKfILldIOLDpRUKcuJnhFsM0oM/qCKMVk52CKij4EY8ZAEWHnjEqekSllQvq/cypYk0XSJw0nASVCtxPLRis9yz0EZQfI5MDy5U+AQod1vwtL5aedapMCdnuc4G6KITjA0/+R4k3nhu7PYgEUgtfdUrDxrhtQJaA2ywllrVXJf3uvHwfoAiG4dDqRc743X7QVK85sjgwnCrBSsLR8nfLjpt0QkMaAJXfhXqsYT+W6OZ4ckQxV; 4:iKIcyrUrqzZOr2zvgw0/0xYXFpbTSxPKLB7KDl11RNlmcphteLk8T1hx+nb8bKMV93gBow0WdSvLiw2KvhTYRgpsW4IeB7BuJPl6Vk9AiMJNERwyYbyThWmH7xEil+SdKMzHDqmeRf6hczrM4sCkbCY/GYjyFlluDJOheaNRkLVtcWihEEmGr69iOQ8fP9y870qA+zWxR2J7yU7L0r0F+u4ZjdcgoDIZoAVSkL5kJc2M+KA9zQMuoj7Ef3kzEE6P X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(100000703101)(100105400095)(3002001)(6041248)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CO1PR07MB894; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CO1PR07MB894; X-Forefront-PRVS: 044968D9E1 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CO1PR07MB894; 23:pzCQ0ioorTfEAxsK0pEEry7hF+7extPWGUJr+EVMrY?= =?us-ascii?Q?wVqWX4XjPZ5M6ZD1zwkU7fbQowkk1XQAxXnfZ1OQKXLClUEr7c0ymtIvjwWL?= =?us-ascii?Q?Azx2RnwAIlUx6nqqS9ISQHFNGLi5cw+qOodLiTwL4Q9aqUbXeIPtS9vAmmzX?= =?us-ascii?Q?T4BQX0EjMfK4xgvF13lZZhwtBL8ZcUR0n6rqdUwQQ1Tigv7o+2IFXp8Eafyk?= =?us-ascii?Q?qfi+S2bKCFU8F+2a9/TdniXMQAEozezARaNRmNibBl68RwuaNK7l3s5n2Wdm?= =?us-ascii?Q?ZXVYuTr/q6ODjsigFrR435c6vKxSfmCJoUmVJyEQGxV+8ZBNGzoBGcrc0YYI?= =?us-ascii?Q?qzC2ZSDCO4qPMpFCaZ/m9QRp4hZOLEEUTISXEK0NKVckVJCPuI270K+BLCP5?= =?us-ascii?Q?2CFGmyueTMY8bmumU+nHbR1VTwTCBHOcYNJzNGZpD9DGwB6Xq8jwDg7/VXBe?= =?us-ascii?Q?b/b8DH96T04V8SGyJ8Eqrd9O0LZykY7Ylm473+7JQauNrxUodSNBBV0qhIbt?= =?us-ascii?Q?Q7ReHvtbugsL0FBPvUPSq8gs4je3YaL8AYQ0w8ScF5avMKE/yrp7sxGLj2JH?= =?us-ascii?Q?fyqZLbQJrfANecZhle39z5pe3LQf6I7eccir6LxW6IYetkhga9o/PreDtg/y?= =?us-ascii?Q?/x6D9Lm/s2NoqcjDjskSNF2icy/oDlOa0bhn4uIOMgD3FP+NyZv1xyCMU7Lp?= =?us-ascii?Q?BHcn/tjzfN0q6UUDw95RGcxv60sma7WLe6Nuz1j5DFcI2USdVG8HGbVJC4aZ?= =?us-ascii?Q?CuGWaL3egPL0mHvAEgwOI4IxW7EtlP5rTqxZCFLpIdHuAxpXDAy/JGnK2qKQ?= =?us-ascii?Q?VEDf5faSQvumdWrYV+QR4nEmBs8t2RMmAkvK7Yi1G/HWE0EM9BHgA2JQf7d8?= =?us-ascii?Q?kbMe4pc5xLt8Q76WUVwqu3jSiyO455nDGcilPuXSd0/SSXYRdmXdCylD6dDm?= =?us-ascii?Q?F2BA3cuI4YXBYM5olsGpPIu8zkz643yg4z71sB+UtsPWaKX1r521tk7ubVVJ?= =?us-ascii?Q?6bY0p9bjxpbBTmU0GIk7/U?= X-Microsoft-Exchange-Diagnostics: 1; CO1PR07MB894; 6:zFekA7jUUbgk3PCiMzM9oLbA9a43AmlCRX/UOeWduO64+kxFhXfeGjJzEtqPabny9PEeX6pJW00OQbW4YIp1R4OgqKs3D6Doa/HCzOmSxfKzvx9yUAZUYNpPwJHaTo7nJYKMcKs72R/RGh+T5fjLVrXL0TxsW6vAfvthjoThbkXcHRxgswdtteJU1VOH3OhcmC4XClZYCqVLPxIuhml9WYtbgsyf9+2RMMg3m2EfV48NRVnQOf3d97p1b84uj9pxU9YyjwytreikdBjguyYPPNFkSrWQ65j4sU1sCNU3p/EhlZzhijW/AFvXNxOKyp06UqF0EnqCRYd8GoOKYpj+2w==; 5:XhCWOc9k9LYWvnOfieA7UuI27VIts8fSemD6SvASliEVeem+ZKPSuFTncxHYGAGW7qSqkaLiQcSIAyUJcg5uSIhWZmbkV9SvAfYJrhwEDzt/oMNKF1fETA3TYpVCBFIOk8omzWUsOWs+DiQvVagJ8g==; 24:/AZQvI2CAS0kZf22VIEBT56QMKP+pG0X7MTQnAHh0i6Gz31DUnduwfU8nob6iseKNuTZEdIdIxFXnp197nJK2uZ/l0Q1kCQ6qoUiaTqtELY=; 7:IyfvgSDHIgIxZyUXqF2ZGZZPcirZZcrWvwMfwxta4Rkx30KKkGYWRlU2d5rbu+Go0DSFtkITga9MKRazIZmoneLOoevwPrWOe/IwjR1XflNxbULTdvix7zPEzFf2wlKdF4N5w14Gpbqx/n8N+eAzm7xme7cDg9oR+VWxVpuCqicYlLVODca1Yq2ePBm44ot+w81DeYOMg/QPHtS0VG2czrXFOdIqEzNXcYmpYUo1l+I= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Oct 2017 08:55:29.9695 (UTC) X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=711e4ccf-2e9b-4bcf-a551-4094005b6194; Ip=[50.232.66.26]; Helo=[CAEXCH02.caveonetworks.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR07MB894 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP iWARP uses 3 ll2 connections, the maximum number of bds is known during connection setup. This patch modifies the static array in the ll2_tx_packet descriptor to be a flexible array and significantlly reduces memory size. In addition, some redundant fields in the ll2_tx_packet were removed, which also contributed to decreasing the descriptor size. Signed-off-by: Michal Kalderon Signed-off-by: Ariel Elior --- drivers/net/ethernet/qlogic/qed/qed_ll2.c | 29 +++++++++++++++++++++-------- drivers/net/ethernet/qlogic/qed/qed_ll2.h | 9 +++------ 2 files changed, 24 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/qlogic/qed/qed_ll2.c b/drivers/net/ethernet/qlogic/qed/qed_ll2.c index 250afa5..75af40a 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_ll2.c +++ b/drivers/net/ethernet/qlogic/qed/qed_ll2.c @@ -1105,6 +1105,7 @@ static int qed_ll2_acquire_connection_tx(struct qed_hwfn *p_hwfn, struct qed_ll2_info *p_ll2_info) { struct qed_ll2_tx_packet *p_descq; + u32 desc_size; u32 capacity; int rc = 0; @@ -1122,13 +1123,17 @@ static int qed_ll2_acquire_connection_tx(struct qed_hwfn *p_hwfn, goto out; capacity = qed_chain_get_capacity(&p_ll2_info->tx_queue.txq_chain); - p_descq = kcalloc(capacity, sizeof(struct qed_ll2_tx_packet), - GFP_KERNEL); + /* First element is part of the packet, rest are flexibly added */ + desc_size = (sizeof(*p_descq) + + (p_ll2_info->input.tx_max_bds_per_packet - 1) * + sizeof(p_descq->bds_set)); + + p_descq = kcalloc(capacity, desc_size, GFP_KERNEL); if (!p_descq) { rc = -ENOMEM; goto out; } - p_ll2_info->tx_queue.descq_array = p_descq; + p_ll2_info->tx_queue.descq_mem = p_descq; DP_VERBOSE(p_hwfn, QED_MSG_LL2, "Allocated LL2 Txq [Type %08x] with 0x%08x buffers\n", @@ -1359,11 +1364,13 @@ int qed_ll2_establish_connection(void *cxt, u8 connection_handle) { struct qed_hwfn *p_hwfn = cxt; struct qed_ll2_info *p_ll2_conn; + struct qed_ll2_tx_packet *p_pkt; struct qed_ll2_rx_queue *p_rx; struct qed_ll2_tx_queue *p_tx; struct qed_ptt *p_ptt; int rc = -EINVAL; u32 i, capacity; + u32 desc_size; u8 qid; p_ptt = qed_ptt_acquire(p_hwfn); @@ -1397,9 +1404,15 @@ int qed_ll2_establish_connection(void *cxt, u8 connection_handle) INIT_LIST_HEAD(&p_tx->sending_descq); spin_lock_init(&p_tx->lock); capacity = qed_chain_get_capacity(&p_tx->txq_chain); - for (i = 0; i < capacity; i++) - list_add_tail(&p_tx->descq_array[i].list_entry, - &p_tx->free_descq); + /* First element is part of the packet, rest are flexibly added */ + desc_size = (sizeof(*p_pkt) + + (p_ll2_conn->input.tx_max_bds_per_packet - 1) * + sizeof(p_pkt->bds_set)); + + for (i = 0; i < capacity; i++) { + p_pkt = p_tx->descq_mem + desc_size * i; + list_add_tail(&p_pkt->list_entry, &p_tx->free_descq); + } p_tx->cur_completing_bd_idx = 0; p_tx->bds_idx = 0; p_tx->b_completing_packet = false; @@ -1698,7 +1711,7 @@ int qed_ll2_prepare_tx_packet(void *cxt, p_tx = &p_ll2_conn->tx_queue; p_tx_chain = &p_tx->txq_chain; - if (pkt->num_of_bds > CORE_LL2_TX_MAX_BDS_PER_PACKET) + if (pkt->num_of_bds > p_ll2_conn->input.tx_max_bds_per_packet) return -EIO; spin_lock_irqsave(&p_tx->lock, flags); @@ -1858,7 +1871,7 @@ void qed_ll2_release_connection(void *cxt, u8 connection_handle) qed_int_unregister_cb(p_hwfn, p_ll2_conn->tx_queue.tx_sb_index); } - kfree(p_ll2_conn->tx_queue.descq_array); + kfree(p_ll2_conn->tx_queue.descq_mem); qed_chain_free(p_hwfn->cdev, &p_ll2_conn->tx_queue.txq_chain); kfree(p_ll2_conn->rx_queue.descq_array); diff --git a/drivers/net/ethernet/qlogic/qed/qed_ll2.h b/drivers/net/ethernet/qlogic/qed/qed_ll2.h index a822528..9bdd08f 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_ll2.h +++ b/drivers/net/ethernet/qlogic/qed/qed_ll2.h @@ -63,17 +63,14 @@ struct qed_ll2_rx_packet { struct qed_ll2_tx_packet { struct list_head list_entry; u16 bd_used; - u16 vlan; - u16 l4_hdr_offset_w; - u8 bd_flags; bool notify_fw; void *cookie; - + /* Flexible Array of bds_set determined by max_bds_per_packet */ struct { struct core_tx_bd *txq_bd; dma_addr_t tx_frag; u16 frag_len; - } bds_set[ETH_TX_MAX_BDS_PER_NON_LSO_PACKET]; + } bds_set[1]; }; struct qed_ll2_rx_queue { @@ -101,7 +98,7 @@ struct qed_ll2_tx_queue { struct list_head active_descq; struct list_head free_descq; struct list_head sending_descq; - struct qed_ll2_tx_packet *descq_array; + void *descq_mem; /* memory for variable sized qed_ll2_tx_packet*/ struct qed_ll2_tx_packet *cur_send_packet; struct qed_ll2_tx_packet cur_completing_packet; u16 cur_completing_bd_idx;