From patchwork Mon May 6 07:04:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Arinzon, David" X-Patchwork-Id: 13655001 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp-fw-9106.amazon.com (smtp-fw-9106.amazon.com [207.171.188.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 820C814039E for ; Mon, 6 May 2024 07:05:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=207.171.188.206 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979114; cv=none; b=MM2uEtE+fmfWg0ZwvwdwEwbkqX9Hit+sbrCd2qwE0wRSAshZVmOVmV6UkDK8uGTHstiNtjVdjWYlmErKw72aW+PYyeA36FD6qanH79d3eBeL20Jbrj9BlJgeqOwy7mTSRGgjbDQaXP8Wt4UDv7j0jx5sm4Wi4r7dyfVslatldWA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979114; c=relaxed/simple; bh=3sjiuCuEVgzWKmn6V7by1aqXmRfcSV/TWODtIWQw2Tc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=imTDVB0TUYXLYAKUPYirnLSww9YlbJbFzUGXOhvTqrZBGvGYSyexkQ6VYGVPlyq/glhJ4qAx/OnpFPQBkCJpot2IzfpMXJbngLrmriMT0dbAp7wV55vEAXc3/J5vg5A9xVO4w1Vnd5iO94V2Uw1HpRolydV13B9wGjeCTtLEhWw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=Z6r2yDRy; arc=none smtp.client-ip=207.171.188.206 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="Z6r2yDRy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1714979113; x=1746515113; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hqK7YjX30RcyYWudkzqCLxv1PJIQ5kQZobD1wSDzwYI=; b=Z6r2yDRyZKGi4HQ8GO+Hw/j9u+NG2PRw50S2F9pc/8gYkcn4lF7kNJLx viMiUKWeI1WPatsXX1nhqAEa4G6Py74VRGwd3znMPF++Ja7GnGD9yCmns ibGMhxJ1N37AmGnEuB92CJSH0JWaokqZT/XoAYrldooiZdJzxzVhmL3OV E=; X-IronPort-AV: E=Sophos;i="6.07,257,1708387200"; d="scan'208";a="723871357" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210]) by smtp-border-fw-9106.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 07:05:06 +0000 Received: from EX19MTAUEA002.ant.amazon.com [10.0.0.204:50748] by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.59.248:2525] with esmtp (Farcaster) id 49c5451d-6ef1-4df8-8ea0-a260677864b1; Mon, 6 May 2024 07:05:05 +0000 (UTC) X-Farcaster-Flow-ID: 49c5451d-6ef1-4df8-8ea0-a260677864b1 Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by EX19MTAUEA002.ant.amazon.com (10.252.134.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:04 +0000 Received: from EX19MTAUEA001.ant.amazon.com (10.252.134.203) by EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:04 +0000 Received: from dev-dsk-darinzon-1c-05962a8d.eu-west-1.amazon.com (172.19.80.187) by mail-relay.amazon.com (10.252.134.102) with Microsoft SMTP Server id 15.2.1258.28 via Frontend Transport; Mon, 6 May 2024 07:05:02 +0000 From: To: David Miller , Jakub Kicinski , CC: David Arinzon , "Woodhouse, David" , "Machulsky, Zorik" , "Matushevsky, Alexander" , Saeed Bshara , "Wilson, Matt" , "Liguori, Anthony" , "Bshara, Nafea" , "Belgazal, Netanel" , "Saidi, Ali" , "Herrenschmidt, Benjamin" , "Kiyanovski, Arthur" , "Dagan, Noam" , "Agroskin, Shay" , "Itzko, Shahar" , "Abboud, Osama" , "Ostrovsky, Evgeny" , "Tabachnik, Ofir" Subject: [PATCH v1 net-next 1/6] net: ena: Add a counter for driver's reset failures Date: Mon, 6 May 2024 07:04:48 +0000 Message-ID: <20240506070453.17054-2-darinzon@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240506070453.17054-1-darinzon@amazon.com> References: <20240506070453.17054-1-darinzon@amazon.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: David Arinzon This patch adds a counter to the ena_adapter struct in order to keep track of reset failures. The counter is incremented every time either ena_restore_device() or ena_destroy_device() fail. Signed-off-by: Osama Abboud Signed-off-by: David Arinzon --- drivers/net/ethernet/amazon/ena/ena_ethtool.c | 1 + drivers/net/ethernet/amazon/ena/ena_netdev.c | 18 ++++++++++++------ drivers/net/ethernet/amazon/ena/ena_netdev.h | 1 + 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_ethtool.c b/drivers/net/ethernet/amazon/ena/ena_ethtool.c index 0cb6cc1c..28583db8 100644 --- a/drivers/net/ethernet/amazon/ena/ena_ethtool.c +++ b/drivers/net/ethernet/amazon/ena/ena_ethtool.c @@ -49,6 +49,7 @@ static const struct ena_stats ena_stats_global_strings[] = { ENA_STAT_GLOBAL_ENTRY(interface_up), ENA_STAT_GLOBAL_ENTRY(interface_down), ENA_STAT_GLOBAL_ENTRY(admin_q_pause), + ENA_STAT_GLOBAL_ENTRY(reset_fail), }; static const struct ena_stats ena_stats_eni_strings[] = { diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index be5acfa4..683088af 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -42,7 +42,7 @@ MODULE_DEVICE_TABLE(pci, ena_pci_tbl); static int ena_rss_init_default(struct ena_adapter *adapter); static void check_for_admin_com_state(struct ena_adapter *adapter); -static void ena_destroy_device(struct ena_adapter *adapter, bool graceful); +static int ena_destroy_device(struct ena_adapter *adapter, bool graceful); static int ena_restore_device(struct ena_adapter *adapter); static void ena_tx_timeout(struct net_device *dev, unsigned int txqueue) @@ -3235,14 +3235,15 @@ err_disable_msix: return rc; } -static void ena_destroy_device(struct ena_adapter *adapter, bool graceful) +static int ena_destroy_device(struct ena_adapter *adapter, bool graceful) { struct net_device *netdev = adapter->netdev; struct ena_com_dev *ena_dev = adapter->ena_dev; bool dev_up; + int rc = 0; if (!test_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags)) - return; + return 0; netif_carrier_off(netdev); @@ -3260,7 +3261,7 @@ static void ena_destroy_device(struct ena_adapter *adapter, bool graceful) * and device is up, ena_down() already reset the device. */ if (!(test_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags) && dev_up)) - ena_com_dev_reset(adapter->ena_dev, adapter->reset_reason); + rc = ena_com_dev_reset(adapter->ena_dev, adapter->reset_reason); ena_free_mgmnt_irq(adapter); @@ -3279,6 +3280,8 @@ static void ena_destroy_device(struct ena_adapter *adapter, bool graceful) clear_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags); clear_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags); + + return rc; } static int ena_restore_device(struct ena_adapter *adapter) @@ -3355,14 +3358,17 @@ err: static void ena_fw_reset_device(struct work_struct *work) { + int rc = 0; + struct ena_adapter *adapter = container_of(work, struct ena_adapter, reset_task); rtnl_lock(); if (likely(test_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags))) { - ena_destroy_device(adapter, false); - ena_restore_device(adapter); + rc |= ena_destroy_device(adapter, false); + rc |= ena_restore_device(adapter); + adapter->dev_stats.reset_fail += !!rc; dev_err(&adapter->pdev->dev, "Device reset completed successfully\n"); } diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.h b/drivers/net/ethernet/amazon/ena/ena_netdev.h index 6d2cc202..d5950974 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.h +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.h @@ -290,6 +290,7 @@ struct ena_stats_dev { u64 admin_q_pause; u64 rx_drops; u64 tx_drops; + u64 reset_fail; }; enum ena_flags_t { From patchwork Mon May 6 07:04:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Arinzon, David" X-Patchwork-Id: 13654999 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53B8213C8E8 for ; Mon, 6 May 2024 07:05:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.95.48.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979111; cv=none; b=lZOE/tf56bIRrFMOxc3DXVE8J9AmvM8LHz5ex3rpi7LGKf+26/OJWnYrnFSCxXxip7kH+uWeHv6sdXbsPH9/KXUlatjTqx3nyXAWu9zJxZ2HkOJz7QOowupJ/V47qNi/E4ZyWQ9TJplPguow56lx20sOjS/5FfwIZwS+jpoQhEU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979111; c=relaxed/simple; bh=jBzPvio5m+I9R+VPZoRZOYN95Od+s6uzAuY01QLwUO4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FBD+mZtEftYMwv5Sl1k6dqnG0LhdTVbT9gOjiYgdxqWEAreJ+qzIak6+t8N4eVJD7keJV6O67wTAq09haZ7RoulTS0JFGto8vyDp4yiupi/ZcjMI5tbLgSApw9SgcIOeCMpz9JGrpPvNkmG9RA/LWsfrXy16Uj3VB650cQvcb04= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=HHycgs2d; arc=none smtp.client-ip=52.95.48.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="HHycgs2d" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1714979111; x=1746515111; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gh1ofUqgre/duj2ag7tkXXoA0sIVAh5XO/Bo5Xwuqmw=; b=HHycgs2dz6ENgkypJDHy0jvG1F2WJ1Tv3DzpvmjjwbYLU8Coly/74dIL Sg8Fh3OMASTBUVKMKjez9Cvvmq3cbiqY5/LPnGiRlkJA4ToK7fm1Vyq56 woYjtrUpBZdeuWAB8D59wMAmAG9QsJLkYDyMGtlWyDxa+TubwEen5YozO 0=; X-IronPort-AV: E=Sophos;i="6.07,257,1708387200"; d="scan'208";a="394633717" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.2]) by smtp-border-fw-6001.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 07:05:08 +0000 Received: from EX19MTAUEA001.ant.amazon.com [10.0.29.78:57982] by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.16.208:2525] with esmtp (Farcaster) id b44b1178-56cb-4552-a920-58e7d4e92a8e; Mon, 6 May 2024 07:05:07 +0000 (UTC) X-Farcaster-Flow-ID: b44b1178-56cb-4552-a920-58e7d4e92a8e Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:07 +0000 Received: from EX19MTAUEA001.ant.amazon.com (10.252.134.203) by EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:07 +0000 Received: from dev-dsk-darinzon-1c-05962a8d.eu-west-1.amazon.com (172.19.80.187) by mail-relay.amazon.com (10.252.134.102) with Microsoft SMTP Server id 15.2.1258.28 via Frontend Transport; Mon, 6 May 2024 07:05:05 +0000 From: To: David Miller , Jakub Kicinski , CC: David Arinzon , "Woodhouse, David" , "Machulsky, Zorik" , "Matushevsky, Alexander" , Saeed Bshara , "Wilson, Matt" , "Liguori, Anthony" , "Bshara, Nafea" , "Belgazal, Netanel" , "Saidi, Ali" , "Herrenschmidt, Benjamin" , "Kiyanovski, Arthur" , "Dagan, Noam" , "Agroskin, Shay" , "Itzko, Shahar" , "Abboud, Osama" , "Ostrovsky, Evgeny" , "Tabachnik, Ofir" Subject: [PATCH v1 net-next 2/6] net: ena: Reduce holes in ena_com structures Date: Mon, 6 May 2024 07:04:49 +0000 Message-ID: <20240506070453.17054-3-darinzon@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240506070453.17054-1-darinzon@amazon.com> References: <20240506070453.17054-1-darinzon@amazon.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: David Arinzon This patch makes two changes in order to fill holes and reduce ther overall size of the structures ena_com_dev and ena_com_rx_ctx. Signed-off-by: Shahar Itzko Signed-off-by: David Arinzon --- drivers/net/ethernet/amazon/ena/ena_com.h | 4 ++-- drivers/net/ethernet/amazon/ena/ena_eth_com.h | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_com.h b/drivers/net/ethernet/amazon/ena/ena_com.h index fea57eb8..fdae0cc9 100644 --- a/drivers/net/ethernet/amazon/ena/ena_com.h +++ b/drivers/net/ethernet/amazon/ena/ena_com.h @@ -305,6 +305,8 @@ struct ena_com_dev { u16 stats_func; /* Selected function for extended statistic dump */ u16 stats_queue; /* Selected queue for extended statistic dump */ + u32 ena_min_poll_delay_us; + struct ena_com_mmio_read mmio_read; struct ena_rss rss; @@ -325,8 +327,6 @@ struct ena_com_dev { struct ena_intr_moder_entry *intr_moder_tbl; struct ena_com_llq_info llq_info; - - u32 ena_min_poll_delay_us; }; struct ena_com_dev_get_features_ctx { diff --git a/drivers/net/ethernet/amazon/ena/ena_eth_com.h b/drivers/net/ethernet/amazon/ena/ena_eth_com.h index 72b01975..449bc496 100644 --- a/drivers/net/ethernet/amazon/ena/ena_eth_com.h +++ b/drivers/net/ethernet/amazon/ena/ena_eth_com.h @@ -47,7 +47,7 @@ struct ena_com_rx_ctx { bool frag; u32 hash; u16 descs; - int max_bufs; + u16 max_bufs; u8 pkt_offset; }; From patchwork Mon May 6 07:04:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Arinzon, David" X-Patchwork-Id: 13655000 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp-fw-80007.amazon.com (smtp-fw-80007.amazon.com [99.78.197.218]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4298714038F for ; Mon, 6 May 2024 07:05:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=99.78.197.218 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979112; cv=none; b=B8pmTeQ3p4Pg83dDS9n3t7SBWEhKMMCeFfqzfgcXWCetXYUB59RoLT99e7bYUZhREiw69o2lkpo2fKMz3eMehXUQmHNR2zsnu4hPEmuum7oGKZs09crGM0Lw4PRQ2WmDqidfJwoVpW8U2dhvR2RWBRcB9ZbcB7Z17+YLmllzkHM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979112; c=relaxed/simple; bh=GYBUNIwYl58jUDE8404WrXzFoQbpEPnMd8EOj/smreU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=aCh0IQG9X0Ask+bIb9LOTa6CUBjWkxqLw5J68W8x+Zb7cI5Z+R5Ls2mA5RnkhDl1CtUxgK8kD8C9paLwx1/E60mQqzCbhUywQZ7Z8gcSNGUTagJLpN77F/UGbbR3cEhdc484KWnFbLjeq3Ryf/AOsHfHDySukkV4Ga+GKOTR/r8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=kTP+Sk47; arc=none smtp.client-ip=99.78.197.218 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="kTP+Sk47" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1714979111; x=1746515111; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kygTiuspGmVIG3GLz1pB4Tk9SDGdw1+uPcC568kSBgA=; b=kTP+Sk47cojbn9NoYHNnhJdC/6sUWtCLZPn9d0eBrh/PK4MFrOvGTJV6 L+hd5fammqI1rfk+u5egt+qdbcwwKEW8QP1zE1Z50/Kq+fTZkiCBI/4z0 3w9UzgvcUAm31s8YjDrWA+FVrwRIZm/181P+K2SX5FIajyMaYqRD5P7V6 w=; X-IronPort-AV: E=Sophos;i="6.07,257,1708387200"; d="scan'208";a="294210513" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210]) by smtp-border-fw-80007.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 07:05:11 +0000 Received: from EX19MTAUEC002.ant.amazon.com [10.0.29.78:20305] by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.83.228:2525] with esmtp (Farcaster) id 0182a925-be39-492b-aed1-3ec0926e429f; Mon, 6 May 2024 07:05:10 +0000 (UTC) X-Farcaster-Flow-ID: 0182a925-be39-492b-aed1-3ec0926e429f Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by EX19MTAUEC002.ant.amazon.com (10.252.135.253) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:10 +0000 Received: from EX19MTAUEA001.ant.amazon.com (10.252.134.203) by EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:09 +0000 Received: from dev-dsk-darinzon-1c-05962a8d.eu-west-1.amazon.com (172.19.80.187) by mail-relay.amazon.com (10.252.134.102) with Microsoft SMTP Server id 15.2.1258.28 via Frontend Transport; Mon, 6 May 2024 07:05:08 +0000 From: To: David Miller , Jakub Kicinski , CC: David Arinzon , "Woodhouse, David" , "Machulsky, Zorik" , "Matushevsky, Alexander" , Saeed Bshara , "Wilson, Matt" , "Liguori, Anthony" , "Bshara, Nafea" , "Belgazal, Netanel" , "Saidi, Ali" , "Herrenschmidt, Benjamin" , "Kiyanovski, Arthur" , "Dagan, Noam" , "Agroskin, Shay" , "Itzko, Shahar" , "Abboud, Osama" , "Ostrovsky, Evgeny" , "Tabachnik, Ofir" Subject: [PATCH v1 net-next 3/6] net: ena: Add validation for completion descriptors consistency Date: Mon, 6 May 2024 07:04:50 +0000 Message-ID: <20240506070453.17054-4-darinzon@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240506070453.17054-1-darinzon@amazon.com> References: <20240506070453.17054-1-darinzon@amazon.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: David Arinzon Validate that `first` flag is set only for the first descriptor in multi-buffer packets. In case of an invalid descriptor, a reset will occur. A new reset reason for RX data corruption has been added. Signed-off-by: Shahar Itzko Signed-off-by: David Arinzon --- drivers/net/ethernet/amazon/ena/ena_eth_com.c | 37 ++++++++++++++----- drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 + .../net/ethernet/amazon/ena/ena_regs_defs.h | 1 + 3 files changed, 30 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_eth_com.c b/drivers/net/ethernet/amazon/ena/ena_eth_com.c index 933e619b..4c6e07aa 100644 --- a/drivers/net/ethernet/amazon/ena/ena_eth_com.c +++ b/drivers/net/ethernet/amazon/ena/ena_eth_com.c @@ -229,30 +229,43 @@ static struct ena_eth_io_rx_cdesc_base * idx * io_cq->cdesc_entry_size_in_bytes); } -static u16 ena_com_cdesc_rx_pkt_get(struct ena_com_io_cq *io_cq, - u16 *first_cdesc_idx) +static int ena_com_cdesc_rx_pkt_get(struct ena_com_io_cq *io_cq, + u16 *first_cdesc_idx, + u16 *num_descs) { + u16 count = io_cq->cur_rx_pkt_cdesc_count, head_masked; struct ena_eth_io_rx_cdesc_base *cdesc; - u16 count = 0, head_masked; u32 last = 0; do { + u32 status; + cdesc = ena_com_get_next_rx_cdesc(io_cq); if (!cdesc) break; + status = READ_ONCE(cdesc->status); ena_com_cq_inc_head(io_cq); + if (unlikely((status & ENA_ETH_IO_RX_CDESC_BASE_FIRST_MASK) >> + ENA_ETH_IO_RX_CDESC_BASE_FIRST_SHIFT && count != 0)) { + struct ena_com_dev *dev = ena_com_io_cq_to_ena_dev(io_cq); + + netdev_err(dev->net_device, + "First bit is on in descriptor #%d on q_id: %d, req_id: %u\n", + count, io_cq->qid, cdesc->req_id); + return -EFAULT; + } count++; - last = (READ_ONCE(cdesc->status) & ENA_ETH_IO_RX_CDESC_BASE_LAST_MASK) >> - ENA_ETH_IO_RX_CDESC_BASE_LAST_SHIFT; + last = (status & ENA_ETH_IO_RX_CDESC_BASE_LAST_MASK) >> + ENA_ETH_IO_RX_CDESC_BASE_LAST_SHIFT; } while (!last); if (last) { *first_cdesc_idx = io_cq->cur_rx_pkt_cdesc_start_idx; - count += io_cq->cur_rx_pkt_cdesc_count; head_masked = io_cq->head & (io_cq->q_depth - 1); + *num_descs = count; io_cq->cur_rx_pkt_cdesc_count = 0; io_cq->cur_rx_pkt_cdesc_start_idx = head_masked; @@ -260,11 +273,11 @@ static u16 ena_com_cdesc_rx_pkt_get(struct ena_com_io_cq *io_cq, "ENA q_id: %d packets were completed. first desc idx %u descs# %d\n", io_cq->qid, *first_cdesc_idx, count); } else { - io_cq->cur_rx_pkt_cdesc_count += count; - count = 0; + io_cq->cur_rx_pkt_cdesc_count = count; + *num_descs = 0; } - return count; + return 0; } static int ena_com_create_meta(struct ena_com_io_sq *io_sq, @@ -539,10 +552,14 @@ int ena_com_rx_pkt(struct ena_com_io_cq *io_cq, u16 cdesc_idx = 0; u16 nb_hw_desc; u16 i = 0; + int rc; WARN(io_cq->direction != ENA_COM_IO_QUEUE_DIRECTION_RX, "wrong Q type"); - nb_hw_desc = ena_com_cdesc_rx_pkt_get(io_cq, &cdesc_idx); + rc = ena_com_cdesc_rx_pkt_get(io_cq, &cdesc_idx, &nb_hw_desc); + if (unlikely(rc != 0)) + return -EFAULT; + if (nb_hw_desc == 0) { ena_rx_ctx->descs = nb_hw_desc; return 0; diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index 683088af..c17a9833 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -1347,6 +1347,8 @@ error: if (rc == -ENOSPC) { ena_increase_stat(&rx_ring->rx_stats.bad_desc_num, 1, &rx_ring->syncp); ena_reset_device(adapter, ENA_REGS_RESET_TOO_MANY_RX_DESCS); + } else if (rc == -EFAULT) { + ena_reset_device(adapter, ENA_REGS_RESET_RX_DESCRIPTOR_MALFORMED); } else { ena_increase_stat(&rx_ring->rx_stats.bad_req_id, 1, &rx_ring->syncp); diff --git a/drivers/net/ethernet/amazon/ena/ena_regs_defs.h b/drivers/net/ethernet/amazon/ena/ena_regs_defs.h index 2c3d6a77..a2efebaf 100644 --- a/drivers/net/ethernet/amazon/ena/ena_regs_defs.h +++ b/drivers/net/ethernet/amazon/ena/ena_regs_defs.h @@ -22,6 +22,7 @@ enum ena_regs_reset_reason_types { ENA_REGS_RESET_GENERIC = 13, ENA_REGS_RESET_MISS_INTERRUPT = 14, ENA_REGS_RESET_SUSPECTED_POLL_STARVATION = 15, + ENA_REGS_RESET_RX_DESCRIPTOR_MALFORMED = 16, }; /* ena_registers offsets */ From patchwork Mon May 6 07:04:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Arinzon, David" X-Patchwork-Id: 13655002 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com [99.78.197.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2068F5338C for ; Mon, 6 May 2024 07:05:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=99.78.197.217 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979118; cv=none; b=mk5M7SbvIx0pLVTVmxLAUGprhSYngGGJggGvVnocxEEc5l4AzGgaYXFdiQbiNZCqgdVZeCdtszRcH08NZHamHlwqXBcRAPEzQH7RydYtjOLkzZEzIJ1AwUKohf7qxhxCMtzwhpkFLsWJJsCQxSy0uECOhYJvJ0av3UBCLa0jPcQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979118; c=relaxed/simple; bh=y8TijsN5cZcRlAjXenEn4D+F2JxysTKIm+kt1OX8wxU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UAWlRr/fK55hDI5UY+C+wXfivmzd8LBMhX2YAr4bSrh2s8TNv4WFL2evG7JTBgUkPt5V8XkS78QA0s9R4AtOcbY8I1ZSJKQ4tw4wetWoocIOtrgwSlvR9bSs41Usqi5UqC/bdsigITRPPmJKbfwTS/JeUlmrlk5jzXIpe4aoFcE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=e1I8KKHA; arc=none smtp.client-ip=99.78.197.217 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="e1I8KKHA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1714979118; x=1746515118; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=K2XKbUuKFSNnaDX3EZvj69QBngT9O2/joPD7nMFpejg=; b=e1I8KKHAH4kza3lP0I+WTBBkVxJPqUptnp65FHBa7ezG4yUQULV0zsi5 oWbl/terdXQKGRt1zjqDorI2xdslXfxhMus4nt231k3ujKs13FpCQuixw wZB2ku+SOti2yHQuw5VkLbEmYwjHjIKqzkfM0jn+Rjj+g4H1oPKX/sCIN k=; X-IronPort-AV: E=Sophos;i="6.07,257,1708387200"; d="scan'208";a="293116609" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.25.36.214]) by smtp-border-fw-80006.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 07:05:15 +0000 Received: from EX19MTAUWA001.ant.amazon.com [10.0.21.151:58251] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.61.254:2525] with esmtp (Farcaster) id 0ddae137-5ef9-4e7a-8329-1ca18dda79e3; Mon, 6 May 2024 07:05:14 +0000 (UTC) X-Farcaster-Flow-ID: 0ddae137-5ef9-4e7a-8329-1ca18dda79e3 Received: from EX19D002UWA004.ant.amazon.com (10.13.138.230) by EX19MTAUWA001.ant.amazon.com (10.250.64.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:14 +0000 Received: from EX19MTAUWC001.ant.amazon.com (10.250.64.145) by EX19D002UWA004.ant.amazon.com (10.13.138.230) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:14 +0000 Received: from dev-dsk-darinzon-1c-05962a8d.eu-west-1.amazon.com (172.19.80.187) by mail-relay.amazon.com (10.250.64.145) with Microsoft SMTP Server id 15.2.1258.28 via Frontend Transport; Mon, 6 May 2024 07:05:11 +0000 From: To: David Miller , Jakub Kicinski , CC: David Arinzon , "Woodhouse, David" , "Machulsky, Zorik" , "Matushevsky, Alexander" , Saeed Bshara , "Wilson, Matt" , "Liguori, Anthony" , "Bshara, Nafea" , "Belgazal, Netanel" , "Saidi, Ali" , "Herrenschmidt, Benjamin" , "Kiyanovski, Arthur" , "Dagan, Noam" , "Agroskin, Shay" , "Itzko, Shahar" , "Abboud, Osama" , "Ostrovsky, Evgeny" , "Tabachnik, Ofir" Subject: [PATCH v1 net-next 4/6] net: ena: Changes around strscpy calls Date: Mon, 6 May 2024 07:04:51 +0000 Message-ID: <20240506070453.17054-5-darinzon@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240506070453.17054-1-darinzon@amazon.com> References: <20240506070453.17054-1-darinzon@amazon.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: David Arinzon strscpy copies as much of the string as possible, meaning that the destination string will be truncated in case of no space. As this is a non-critical error in our case, adding a debug level print for indication. This patch also removes a -1 which was added to ensure enough space for NUL, but strscpy destination string is guaranteed to be NUL-terminted, therefore, the -1 is not needed. Signed-off-by: David Arinzon --- drivers/net/ethernet/amazon/ena/ena_ethtool.c | 16 ++++++++++++---- drivers/net/ethernet/amazon/ena/ena_netdev.c | 17 +++++++++++++---- 2 files changed, 25 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_ethtool.c b/drivers/net/ethernet/amazon/ena/ena_ethtool.c index 28583db8..b24cc3f0 100644 --- a/drivers/net/ethernet/amazon/ena/ena_ethtool.c +++ b/drivers/net/ethernet/amazon/ena/ena_ethtool.c @@ -460,10 +460,18 @@ static void ena_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct ena_adapter *adapter = netdev_priv(dev); - - strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); - strscpy(info->bus_info, pci_name(adapter->pdev), - sizeof(info->bus_info)); + ssize_t ret = 0; + + ret = strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + if (ret < 0) + netif_dbg(adapter, drv, dev, + "module name will be truncated, status = %zd\n", ret); + + ret = strscpy(info->bus_info, pci_name(adapter->pdev), + sizeof(info->bus_info)); + if (ret < 0) + netif_dbg(adapter, drv, dev, + "bus info will be truncated, status = %zd\n", ret); } static void ena_get_ringparam(struct net_device *netdev, diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index c17a9833..53f1000f 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -2703,6 +2703,7 @@ static void ena_config_host_info(struct ena_com_dev *ena_dev, struct pci_dev *pd { struct device *dev = &pdev->dev; struct ena_admin_host_info *host_info; + ssize_t ret; int rc; /* Allocate only the host info */ @@ -2717,11 +2718,19 @@ static void ena_config_host_info(struct ena_com_dev *ena_dev, struct pci_dev *pd host_info->bdf = pci_dev_id(pdev); host_info->os_type = ENA_ADMIN_OS_LINUX; host_info->kernel_ver = LINUX_VERSION_CODE; - strscpy(host_info->kernel_ver_str, utsname()->version, - sizeof(host_info->kernel_ver_str) - 1); + ret = strscpy(host_info->kernel_ver_str, utsname()->version, + sizeof(host_info->kernel_ver_str)); + if (ret < 0) + dev_dbg(dev, + "kernel version string will be truncated, status = %zd\n", ret); + host_info->os_dist = 0; - strscpy(host_info->os_dist_str, utsname()->release, - sizeof(host_info->os_dist_str)); + ret = strscpy(host_info->os_dist_str, utsname()->release, + sizeof(host_info->os_dist_str)); + if (ret < 0) + dev_dbg(dev, + "OS distribution string will be truncated, status = %zd\n", ret); + host_info->driver_version = (DRV_MODULE_GEN_MAJOR) | (DRV_MODULE_GEN_MINOR << ENA_ADMIN_HOST_INFO_MINOR_SHIFT) | From patchwork Mon May 6 07:04:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Arinzon, David" X-Patchwork-Id: 13655003 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEFC95337A for ; Mon, 6 May 2024 07:05:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.95.48.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979122; cv=none; b=HNwOP4ey7qOn2RK4boprw+Lojs5xwUMk0RKvCEKr2bXHkpPvCVhNN7YkqOcDPXnNx1IuCf6VBJPhWEpCAMMSoFGGvRP2U2MhwWpFJ+Bwk393nT1Iz3f9JjlbsQqCJyUco1nLOth2Pdv5Kqq4YmnMp0pyGCiLsp/RDenaqwEgSVo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979122; c=relaxed/simple; bh=v1SW2LD1kKZnEBcIlJ8oI3sqvyUjqR2fDIZIkhd7Xzg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=kS4vk0nEwc5yT0N0LG7ayLocir62Lvo/fNc7aKY4m1bK3SWHznYgXjFcSlufLK4k9WMTRkju4GfejX+hd2LWK+XZ69IYaWvhIueXwn6AeBOcBIkiXirOH6bDcJ7gaDZIWz+PIh0nz4/UhxKxMEhEjhKuRdlc/NBpa8hZDOri3X0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=ibxIv+q2; arc=none smtp.client-ip=52.95.48.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="ibxIv+q2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1714979121; x=1746515121; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=w+ZtxqWVAmNISxxyTCRftYBBDFRc+t3aZY2luzMzLXM=; b=ibxIv+q2X823jUAlgUULl/uZbO0sEROdzk4pJY6a0xSOjLkDdqOcqAcT l4KWBpJbojrEu0kBA91Okm7h3H8WHL0wGKNEdeox4d5K5iqX6Em2hWzPN qCAIPJlutUnftYlBNXZEwd03lvZw8BTtScX6CH1UlLoFJePVoJBOxvcTF 0=; X-IronPort-AV: E=Sophos;i="6.07,257,1708387200"; d="scan'208";a="394633732" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.43.8.2]) by smtp-border-fw-6001.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 07:05:20 +0000 Received: from EX19MTAUWA002.ant.amazon.com [10.0.38.20:2249] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.8.38:2525] with esmtp (Farcaster) id edfab993-9d86-49eb-a4fa-f5e88534080d; Mon, 6 May 2024 07:05:18 +0000 (UTC) X-Farcaster-Flow-ID: edfab993-9d86-49eb-a4fa-f5e88534080d Received: from EX19D010UWA003.ant.amazon.com (10.13.138.199) by EX19MTAUWA002.ant.amazon.com (10.250.64.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:18 +0000 Received: from EX19MTAUWC001.ant.amazon.com (10.250.64.145) by EX19D010UWA003.ant.amazon.com (10.13.138.199) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:18 +0000 Received: from dev-dsk-darinzon-1c-05962a8d.eu-west-1.amazon.com (172.19.80.187) by mail-relay.amazon.com (10.250.64.145) with Microsoft SMTP Server id 15.2.1258.28 via Frontend Transport; Mon, 6 May 2024 07:05:15 +0000 From: To: David Miller , Jakub Kicinski , CC: David Arinzon , "Woodhouse, David" , "Machulsky, Zorik" , "Matushevsky, Alexander" , Saeed Bshara , "Wilson, Matt" , "Liguori, Anthony" , "Bshara, Nafea" , "Belgazal, Netanel" , "Saidi, Ali" , "Herrenschmidt, Benjamin" , "Kiyanovski, Arthur" , "Dagan, Noam" , "Agroskin, Shay" , "Itzko, Shahar" , "Abboud, Osama" , "Ostrovsky, Evgeny" , "Tabachnik, Ofir" Subject: [PATCH v1 net-next 5/6] net: ena: Change initial rx_usec interval Date: Mon, 6 May 2024 07:04:52 +0000 Message-ID: <20240506070453.17054-6-darinzon@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240506070453.17054-1-darinzon@amazon.com> References: <20240506070453.17054-1-darinzon@amazon.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: David Arinzon For the purpose of obtaining better CPU utilization, minimum rx moderation interval is set to 20 usec. Signed-off-by: Osama Abboud Signed-off-by: David Arinzon --- drivers/net/ethernet/amazon/ena/ena_com.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_com.h b/drivers/net/ethernet/amazon/ena/ena_com.h index fdae0cc9..924f03f5 100644 --- a/drivers/net/ethernet/amazon/ena/ena_com.h +++ b/drivers/net/ethernet/amazon/ena/ena_com.h @@ -47,7 +47,7 @@ /* ENA adaptive interrupt moderation settings */ #define ENA_INTR_INITIAL_TX_INTERVAL_USECS 64 -#define ENA_INTR_INITIAL_RX_INTERVAL_USECS 0 +#define ENA_INTR_INITIAL_RX_INTERVAL_USECS 20 #define ENA_DEFAULT_INTR_DELAY_RESOLUTION 1 #define ENA_HASH_KEY_SIZE 40 From patchwork Mon May 6 07:04:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Arinzon, David" X-Patchwork-Id: 13655004 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3806D5337A for ; Mon, 6 May 2024 07:05:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.95.48.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979126; cv=none; b=K7U7Q5ZE8x9rOnkcxe7j4fXENOyiozVLpebW44WSyAUAHTK9CrIXaxRm1xDDaUwUix2h8JYoR4VZsCSWqQYiraLSc8ZJqClOnIUnoTyiDRoFnXSdJk/u8aJQ63vfiR2Pb+6KsH/0TpCasv8GdI3TmGOjN4kTERHm04FB1VFsaho= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714979126; c=relaxed/simple; bh=bEnalVJ/sBAM8HVo9cUffBA/gTYUkCrx0WZTpHdZoc8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Fpq8l36HHNf5/PNM7+cIPXgLGSdjzKiinbR/lSGRdnih17N3dHay5rOFbAjT2BoRQ5T+DJ1fVpHCN4sL4sIQbIEuYEgAKcz7r1auERhwx/qr5qmnRfCMhpiMPYEER5VXBfa+oRdL9e968k4sTbSX6DFjcjlZs1fAmpTel5Dx4aU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=GmFqGz+U; arc=none smtp.client-ip=52.95.48.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="GmFqGz+U" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1714979125; x=1746515125; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0XtFZI0bNXU9eVOCOmoZdIQycsMMpyx/PKmOu9jgRsQ=; b=GmFqGz+UUIFb0fCP5Baga0t4hoyhFk7p+Y5MTVEu95bMdTHKtUFfeQ9m OgAitvhs+zkb4AFBuRmO0lilhz9MZshjgF7oeIB4dFQS7pZQMaMS3CynP hnFGjlO4VuPHJTbZjefZzhu8+YHOM6fR609botSn5rL2mg0wqaiFOaOxR E=; X-IronPort-AV: E=Sophos;i="6.07,257,1708387200"; d="scan'208";a="394633739" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.43.8.2]) by smtp-border-fw-6001.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 07:05:24 +0000 Received: from EX19MTAUWB001.ant.amazon.com [10.0.38.20:31247] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.34.73:2525] with esmtp (Farcaster) id 3d7d49e0-d92c-4031-95ae-acfe3c5e4eb2; Mon, 6 May 2024 07:05:22 +0000 (UTC) X-Farcaster-Flow-ID: 3d7d49e0-d92c-4031-95ae-acfe3c5e4eb2 Received: from EX19D010UWA003.ant.amazon.com (10.13.138.199) by EX19MTAUWB001.ant.amazon.com (10.250.64.248) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:22 +0000 Received: from EX19MTAUWC001.ant.amazon.com (10.250.64.145) by EX19D010UWA003.ant.amazon.com (10.13.138.199) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 6 May 2024 07:05:22 +0000 Received: from dev-dsk-darinzon-1c-05962a8d.eu-west-1.amazon.com (172.19.80.187) by mail-relay.amazon.com (10.250.64.145) with Microsoft SMTP Server id 15.2.1258.28 via Frontend Transport; Mon, 6 May 2024 07:05:19 +0000 From: To: David Miller , Jakub Kicinski , CC: David Arinzon , "Woodhouse, David" , "Machulsky, Zorik" , "Matushevsky, Alexander" , Saeed Bshara , "Wilson, Matt" , "Liguori, Anthony" , "Bshara, Nafea" , "Belgazal, Netanel" , "Saidi, Ali" , "Herrenschmidt, Benjamin" , "Kiyanovski, Arthur" , "Dagan, Noam" , "Agroskin, Shay" , "Itzko, Shahar" , "Abboud, Osama" , "Ostrovsky, Evgeny" , "Tabachnik, Ofir" Subject: [PATCH v1 net-next 6/6] net: ena: Add a field for no interrupt moderation update action Date: Mon, 6 May 2024 07:04:53 +0000 Message-ID: <20240506070453.17054-7-darinzon@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240506070453.17054-1-darinzon@amazon.com> References: <20240506070453.17054-1-darinzon@amazon.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: David Arinzon Changes in the interrupt moderation values for TX/RX are infrequent from host side. When asking to unmask interrupts, the driver always provides the requested interrupt moderation values, regardless of whether they've changed or not. A new mechanism has been developed in new devices, which allows selectively updating the relevant HW components based on whether interrupt moderation values have changed from the previous request or not (and by that, saving the processing on the device side). When a change in the interrupt moderation value is made (either by DIM or through an ethtool command), a field is updated to reflect this change. When asking to unmask interrupts, the driver checks the above mentioned field to see if any of the interrupt moderation values (TX or RX) for the particular queue whose interrupts are now being unmasked has changed, and updates the device accordingly. The mentioned field is being reset (set to false) during the procedure which triggers the interrupt unmask. Signed-off-by: David Arinzon --- drivers/net/ethernet/amazon/ena/ena_com.h | 9 +++++- .../net/ethernet/amazon/ena/ena_eth_io_defs.h | 5 ++- drivers/net/ethernet/amazon/ena/ena_ethtool.c | 14 +++++--- drivers/net/ethernet/amazon/ena/ena_netdev.c | 32 ++++++++++++++----- drivers/net/ethernet/amazon/ena/ena_netdev.h | 9 ++++-- 5 files changed, 53 insertions(+), 16 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_com.h b/drivers/net/ethernet/amazon/ena/ena_com.h index 924f03f5..adce2d7f 100644 --- a/drivers/net/ethernet/amazon/ena/ena_com.h +++ b/drivers/net/ethernet/amazon/ena/ena_com.h @@ -980,13 +980,16 @@ static inline bool ena_com_get_cap(struct ena_com_dev *ena_dev, * @rx_delay_interval: Rx interval in usecs * @tx_delay_interval: Tx interval in usecs * @unmask: unmask enable/disable + * @no_moderation_update: 0 - Indicates that any of the TX/RX intervals was + * updated, 1 - otherwise * * Prepare interrupt update register with the supplied parameters. */ static inline void ena_com_update_intr_reg(struct ena_eth_io_intr_reg *intr_reg, u32 rx_delay_interval, u32 tx_delay_interval, - bool unmask) + bool unmask, + bool no_moderation_update) { intr_reg->intr_control = 0; intr_reg->intr_control |= rx_delay_interval & @@ -998,6 +1001,10 @@ static inline void ena_com_update_intr_reg(struct ena_eth_io_intr_reg *intr_reg, if (unmask) intr_reg->intr_control |= ENA_ETH_IO_INTR_REG_INTR_UNMASK_MASK; + + intr_reg->intr_control |= + (((u32)no_moderation_update) << ENA_ETH_IO_INTR_REG_NO_MODERATION_UPDATE_SHIFT) & + ENA_ETH_IO_INTR_REG_NO_MODERATION_UPDATE_MASK; } static inline u8 *ena_com_get_next_bounce_buffer(struct ena_com_io_bounce_buffer_control *bounce_buf_ctrl) diff --git a/drivers/net/ethernet/amazon/ena/ena_eth_io_defs.h b/drivers/net/ethernet/amazon/ena/ena_eth_io_defs.h index 332ac0d2..a4d6d0ee 100644 --- a/drivers/net/ethernet/amazon/ena/ena_eth_io_defs.h +++ b/drivers/net/ethernet/amazon/ena/ena_eth_io_defs.h @@ -261,7 +261,8 @@ struct ena_eth_io_intr_reg { /* 14:0 : rx_intr_delay * 29:15 : tx_intr_delay * 30 : intr_unmask - * 31 : reserved + * 31 : no_moderation_update - 0 - moderation + * updated, 1 - moderation not updated */ u32 intr_control; }; @@ -381,6 +382,8 @@ struct ena_eth_io_numa_node_cfg_reg { #define ENA_ETH_IO_INTR_REG_TX_INTR_DELAY_MASK GENMASK(29, 15) #define ENA_ETH_IO_INTR_REG_INTR_UNMASK_SHIFT 30 #define ENA_ETH_IO_INTR_REG_INTR_UNMASK_MASK BIT(30) +#define ENA_ETH_IO_INTR_REG_NO_MODERATION_UPDATE_SHIFT 31 +#define ENA_ETH_IO_INTR_REG_NO_MODERATION_UPDATE_MASK BIT(31) /* numa_node_cfg_reg */ #define ENA_ETH_IO_NUMA_NODE_CFG_REG_NUMA_MASK GENMASK(7, 0) diff --git a/drivers/net/ethernet/amazon/ena/ena_ethtool.c b/drivers/net/ethernet/amazon/ena/ena_ethtool.c index b24cc3f0..d7a343ce 100644 --- a/drivers/net/ethernet/amazon/ena/ena_ethtool.c +++ b/drivers/net/ethernet/amazon/ena/ena_ethtool.c @@ -390,8 +390,11 @@ static void ena_update_tx_rings_nonadaptive_intr_moderation(struct ena_adapter * val = ena_com_get_nonadaptive_moderation_interval_tx(adapter->ena_dev); - for (i = 0; i < adapter->num_io_queues; i++) - adapter->tx_ring[i].smoothed_interval = val; + for (i = 0; i < adapter->num_io_queues; i++) { + adapter->tx_ring[i].interrupt_interval_changed = + adapter->tx_ring[i].interrupt_interval != val; + adapter->tx_ring[i].interrupt_interval = val; + } } static void ena_update_rx_rings_nonadaptive_intr_moderation(struct ena_adapter *adapter) @@ -401,8 +404,11 @@ static void ena_update_rx_rings_nonadaptive_intr_moderation(struct ena_adapter * val = ena_com_get_nonadaptive_moderation_interval_rx(adapter->ena_dev); - for (i = 0; i < adapter->num_io_queues; i++) - adapter->rx_ring[i].smoothed_interval = val; + for (i = 0; i < adapter->num_io_queues; i++) { + adapter->rx_ring[i].interrupt_interval_changed = + adapter->rx_ring[i].interrupt_interval != val; + adapter->rx_ring[i].interrupt_interval = val; + } } static int ena_set_coalesce(struct net_device *net_dev, diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index 53f1000f..d5bac233 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -224,8 +224,10 @@ void ena_init_io_rings(struct ena_adapter *adapter, txr->tx_max_header_size = ena_dev->tx_max_header_size; txr->tx_mem_queue_type = ena_dev->tx_mem_queue_type; txr->sgl_size = adapter->max_tx_sgl_size; - txr->smoothed_interval = + txr->interrupt_interval = ena_com_get_nonadaptive_moderation_interval_tx(ena_dev); + /* Initial value, mark as true */ + txr->interrupt_interval_changed = true; txr->disable_meta_caching = adapter->disable_meta_caching; spin_lock_init(&txr->xdp_tx_lock); @@ -238,8 +240,10 @@ void ena_init_io_rings(struct ena_adapter *adapter, rxr->ring_size = adapter->requested_rx_ring_size; rxr->rx_copybreak = adapter->rx_copybreak; rxr->sgl_size = adapter->max_rx_sgl_size; - rxr->smoothed_interval = + rxr->interrupt_interval = ena_com_get_nonadaptive_moderation_interval_rx(ena_dev); + /* Initial value, mark as true */ + rxr->interrupt_interval_changed = true; rxr->empty_rx_queue = 0; rxr->rx_headroom = NET_SKB_PAD; adapter->ena_napi[i].dim.mode = DIM_CQ_PERIOD_MODE_START_FROM_EQE; @@ -1364,7 +1368,10 @@ static void ena_dim_work(struct work_struct *w) net_dim_get_rx_moderation(dim->mode, dim->profile_ix); struct ena_napi *ena_napi = container_of(dim, struct ena_napi, dim); - ena_napi->rx_ring->smoothed_interval = cur_moder.usec; + ena_napi->rx_ring->interrupt_interval = cur_moder.usec; + /* DIM will schedule the work in case there was a change in the profile. */ + ena_napi->rx_ring->interrupt_interval_changed = true; + dim->state = DIM_START_MEASURE; } @@ -1391,24 +1398,33 @@ static void ena_adjust_adaptive_rx_intr_moderation(struct ena_napi *ena_napi) void ena_unmask_interrupt(struct ena_ring *tx_ring, struct ena_ring *rx_ring) { - u32 rx_interval = tx_ring->smoothed_interval; + u32 rx_interval = tx_ring->interrupt_interval; struct ena_eth_io_intr_reg intr_reg; + bool no_moderation_update = true; /* Rx ring can be NULL when for XDP tx queues which don't have an * accompanying rx_ring pair. */ - if (rx_ring) + if (rx_ring) { rx_interval = ena_com_get_adaptive_moderation_enabled(rx_ring->ena_dev) ? - rx_ring->smoothed_interval : + rx_ring->interrupt_interval : ena_com_get_nonadaptive_moderation_interval_rx(rx_ring->ena_dev); + no_moderation_update &= !rx_ring->interrupt_interval_changed; + rx_ring->interrupt_interval_changed = false; + } + + no_moderation_update &= !tx_ring->interrupt_interval_changed; + tx_ring->interrupt_interval_changed = false; + /* Update intr register: rx intr delay, * tx intr delay and interrupt unmask */ ena_com_update_intr_reg(&intr_reg, rx_interval, - tx_ring->smoothed_interval, - true); + tx_ring->interrupt_interval, + true, + no_moderation_update); ena_increase_stat(&tx_ring->tx_stats.unmask_interrupt, 1, &tx_ring->syncp); diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.h b/drivers/net/ethernet/amazon/ena/ena_netdev.h index d5950974..b5a501eb 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.h +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.h @@ -267,8 +267,13 @@ struct ena_ring { enum ena_admin_placement_policy_type tx_mem_queue_type; struct ena_com_rx_buf_info ena_bufs[ENA_PKT_MAX_BUFS]; - u32 smoothed_interval; - u32 per_napi_packets; + u32 interrupt_interval; + /* Indicates whether interrupt interval has changed since previous set. + * This flag will be kept up, until cleared by the routine which updates + * the device with the modified interrupt interval value. + */ + bool interrupt_interval_changed; + u32 per_napi_packets; u16 non_empty_napi_events; struct u64_stats_sync syncp; union {