diff mbox series

[V2,net,1/9] net: hns3: default enable tx bounce buffer when smmu enabled

Message ID 20241018101059.1718375-2-shaojijie@huawei.com (mailing list archive)
State Superseded
Delegated to: Netdev Maintainers
Headers show
Series There are some bugfix for the HNS3 ethernet driver | expand

Checks

Context Check Description
netdev/series_format success Posting correctly formatted
netdev/tree_selection success Clearly marked for net, async
netdev/ynl success Generated files up to date; no warnings/errors; no diff in generated;
netdev/fixes_present success Fixes tag present in non-next series
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit success Errors and warnings before: 5 this patch: 5
netdev/build_tools success No tools touched, skip
netdev/cc_maintainers warning 1 maintainers not CCed: andrew+netdev@lunn.ch
netdev/build_clang success Errors and warnings before: 3 this patch: 3
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/deprecated_api success None detected
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success No Fixes tag
netdev/build_allmodconfig_warn success Errors and warnings before: 4 this patch: 4
netdev/checkpatch success total: 0 errors, 0 warnings, 0 checks, 126 lines checked
netdev/build_clang_rust success No Rust files in patch. Skipping build
netdev/kdoc success Errors and warnings before: 5 this patch: 5
netdev/source_inline success Was 0 now: 0
netdev/contest success net-next-2024-10-19--00-00 (tests: 777)

Commit Message

Jijie Shao Oct. 18, 2024, 10:10 a.m. UTC 
From: Peiyang Wang <wangpeiyang1@huawei.com>

The SMMU engine on HIP09 chip has a hardware issue.
SMMU pagetable prefetch features may prefetch and use a invalid PTE
even the PTE is valid at that time. This will cause the device trigger
fake pagefaults. The solution is to avoid prefetching by adding a
SYNC command when smmu mapping a iova. But the performance of nic has a
sharp drop. Then we do this workaround, always enable tx bounce buffer,
avoid mapping/unmapping on TX path.

This issue only affects HNS3, so we always enable
tx bounce buffer when smmu enabled to improve performance.

Signed-off-by: Peiyang Wang <wangpeiyang1@huawei.com>
Signed-off-by: Jian Shen <shenjian15@huawei.com>
Signed-off-by: Jijie Shao <shaojijie@huawei.com>
---
 .../net/ethernet/hisilicon/hns3/hns3_enet.c   | 31 +++++++++++++++++
 .../net/ethernet/hisilicon/hns3/hns3_enet.h   |  2 ++
 .../ethernet/hisilicon/hns3/hns3_ethtool.c    | 33 +++++++++++++++++++
 3 files changed, 66 insertions(+)

Comments

Paolo Abeni Oct. 24, 2024, 8:26 a.m. UTC | #1 
On 10/18/24 12:10, Jijie Shao wrote:
> From: Peiyang Wang <wangpeiyang1@huawei.com>
> 
> The SMMU engine on HIP09 chip has a hardware issue.
> SMMU pagetable prefetch features may prefetch and use a invalid PTE
> even the PTE is valid at that time. This will cause the device trigger
> fake pagefaults. The solution is to avoid prefetching by adding a
> SYNC command when smmu mapping a iova. But the performance of nic has a
> sharp drop. Then we do this workaround, always enable tx bounce buffer,
> avoid mapping/unmapping on TX path.
> 
> This issue only affects HNS3, so we always enable
> tx bounce buffer when smmu enabled to improve performance.
> 
> Signed-off-by: Peiyang Wang <wangpeiyang1@huawei.com>
> Signed-off-by: Jian Shen <shenjian15@huawei.com>
> Signed-off-by: Jijie Shao <shaojijie@huawei.com>

I'm sorry to nick pick on somewhat small details, but we really need a
fixes tag here to make 110% clear is a bugfix. I guess it could be the
commit introducing the support for the buggy H/W.

Thanks,

Paolo
Jijie Shao Oct. 24, 2024, 8:31 a.m. UTC | #2 
on 2024/10/24 16:26, Paolo Abeni wrote:
> On 10/18/24 12:10, Jijie Shao wrote:
>> From: Peiyang Wang <wangpeiyang1@huawei.com>
>>
>> The SMMU engine on HIP09 chip has a hardware issue.
>> SMMU pagetable prefetch features may prefetch and use a invalid PTE
>> even the PTE is valid at that time. This will cause the device trigger
>> fake pagefaults. The solution is to avoid prefetching by adding a
>> SYNC command when smmu mapping a iova. But the performance of nic has a
>> sharp drop. Then we do this workaround, always enable tx bounce buffer,
>> avoid mapping/unmapping on TX path.
>>
>> This issue only affects HNS3, so we always enable
>> tx bounce buffer when smmu enabled to improve performance.
>>
>> Signed-off-by: Peiyang Wang <wangpeiyang1@huawei.com>
>> Signed-off-by: Jian Shen <shenjian15@huawei.com>
>> Signed-off-by: Jijie Shao <shaojijie@huawei.com>
> I'm sorry to nick pick on somewhat small details, but we really need a
> fixes tag here to make 110% clear is a bugfix. I guess it could be the
> commit introducing the support for the buggy H/W.
>
> Thanks,
>
> Paolo

I have a little doubt that this patch is about H/W problem,
so how can we write the the fixes tag?

Thanks,
Jijie Shao
Simon Horman Oct. 24, 2024, 4:04 p.m. UTC | #3 
On Thu, Oct 24, 2024 at 04:31:46PM +0800, Jijie Shao wrote:
> 
> on 2024/10/24 16:26, Paolo Abeni wrote:
> > On 10/18/24 12:10, Jijie Shao wrote:
> > > From: Peiyang Wang <wangpeiyang1@huawei.com>
> > > 
> > > The SMMU engine on HIP09 chip has a hardware issue.
> > > SMMU pagetable prefetch features may prefetch and use a invalid PTE
> > > even the PTE is valid at that time. This will cause the device trigger
> > > fake pagefaults. The solution is to avoid prefetching by adding a
> > > SYNC command when smmu mapping a iova. But the performance of nic has a
> > > sharp drop. Then we do this workaround, always enable tx bounce buffer,
> > > avoid mapping/unmapping on TX path.
> > > 
> > > This issue only affects HNS3, so we always enable
> > > tx bounce buffer when smmu enabled to improve performance.
> > > 
> > > Signed-off-by: Peiyang Wang <wangpeiyang1@huawei.com>
> > > Signed-off-by: Jian Shen <shenjian15@huawei.com>
> > > Signed-off-by: Jijie Shao <shaojijie@huawei.com>
> > I'm sorry to nick pick on somewhat small details, but we really need a
> > fixes tag here to make 110% clear is a bugfix. I guess it could be the
> > commit introducing the support for the buggy H/W.
> > 
> > Thanks,
> > 
> > Paolo
> 
> I have a little doubt that this patch is about H/W problem,
> so how can we write the the fixes tag?

Hi Jijie,

That is a good point. But the much point of the Fixes tag is to indicate how
far back the fix should be backported. So I would say the ID of the patch
where the user would have first seen this problem - possibly the patch that
added the driver.
Jijie Shao Oct. 25, 2024, 1:10 a.m. UTC | #4 
on 2024/10/25 0:04, Simon Horman wrote:
> On Thu, Oct 24, 2024 at 04:31:46PM +0800, Jijie Shao wrote:
>> on 2024/10/24 16:26, Paolo Abeni wrote:
>>> On 10/18/24 12:10, Jijie Shao wrote:
>>>> From: Peiyang Wang <wangpeiyang1@huawei.com>
>>>>
>>>> The SMMU engine on HIP09 chip has a hardware issue.
>>>> SMMU pagetable prefetch features may prefetch and use a invalid PTE
>>>> even the PTE is valid at that time. This will cause the device trigger
>>>> fake pagefaults. The solution is to avoid prefetching by adding a
>>>> SYNC command when smmu mapping a iova. But the performance of nic has a
>>>> sharp drop. Then we do this workaround, always enable tx bounce buffer,
>>>> avoid mapping/unmapping on TX path.
>>>>
>>>> This issue only affects HNS3, so we always enable
>>>> tx bounce buffer when smmu enabled to improve performance.
>>>>
>>>> Signed-off-by: Peiyang Wang <wangpeiyang1@huawei.com>
>>>> Signed-off-by: Jian Shen <shenjian15@huawei.com>
>>>> Signed-off-by: Jijie Shao <shaojijie@huawei.com>
>>> I'm sorry to nick pick on somewhat small details, but we really need a
>>> fixes tag here to make 110% clear is a bugfix. I guess it could be the
>>> commit introducing the support for the buggy H/W.
>>>
>>> Thanks,
>>>
>>> Paolo
>> I have a little doubt that this patch is about H/W problem,
>> so how can we write the the fixes tag?
> Hi Jijie,
>
> That is a good point. But the much point of the Fixes tag is to indicate how
> far back the fix should be backported. So I would say the ID of the patch
> where the user would have first seen this problem - possibly the patch that
> added the driver.

That's a good idea. Thank you.
diff mbox series

Patch

diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
index 4cbc4d069a1f..ac88e301f221 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
@@ -11,6 +11,7 @@ 
 #include <linux/irq.h>
 #include <linux/ip.h>
 #include <linux/ipv6.h>
+#include <linux/iommu.h>
 #include <linux/module.h>
 #include <linux/pci.h>
 #include <linux/skbuff.h>
@@ -1032,6 +1033,8 @@  static bool hns3_can_use_tx_sgl(struct hns3_enet_ring *ring,
 static void hns3_init_tx_spare_buffer(struct hns3_enet_ring *ring)
 {
 	u32 alloc_size = ring->tqp->handle->kinfo.tx_spare_buf_size;
+	struct net_device *netdev = ring_to_netdev(ring);
+	struct hns3_nic_priv *priv = netdev_priv(netdev);
 	struct hns3_tx_spare *tx_spare;
 	struct page *page;
 	dma_addr_t dma;
@@ -1073,6 +1076,7 @@  static void hns3_init_tx_spare_buffer(struct hns3_enet_ring *ring)
 	tx_spare->buf = page_address(page);
 	tx_spare->len = PAGE_SIZE << order;
 	ring->tx_spare = tx_spare;
+	ring->tx_copybreak = priv->tx_copybreak;
 	return;
 
 dma_mapping_error:
@@ -4868,6 +4872,30 @@  static void hns3_nic_dealloc_vector_data(struct hns3_nic_priv *priv)
 	devm_kfree(&pdev->dev, priv->tqp_vector);
 }
 
+static void hns3_update_tx_spare_buf_config(struct hns3_nic_priv *priv)
+{
+#define HNS3_MIN_SPARE_BUF_SIZE (2 * 1024 * 1024)
+#define HNS3_MAX_PACKET_SIZE (64 * 1024)
+
+	struct iommu_domain *domain = iommu_get_domain_for_dev(priv->dev);
+	struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(priv->ae_handle);
+	struct hnae3_handle *handle = priv->ae_handle;
+
+	if (ae_dev->dev_version < HNAE3_DEVICE_VERSION_V3)
+		return;
+
+	if (!(domain && iommu_is_dma_domain(domain)))
+		return;
+
+	priv->min_tx_copybreak = HNS3_MAX_PACKET_SIZE;
+	priv->min_tx_spare_buf_size = HNS3_MIN_SPARE_BUF_SIZE;
+
+	if (priv->tx_copybreak < priv->min_tx_copybreak)
+		priv->tx_copybreak = priv->min_tx_copybreak;
+	if (handle->kinfo.tx_spare_buf_size < priv->min_tx_spare_buf_size)
+		handle->kinfo.tx_spare_buf_size = priv->min_tx_spare_buf_size;
+}
+
 static void hns3_ring_get_cfg(struct hnae3_queue *q, struct hns3_nic_priv *priv,
 			      unsigned int ring_type)
 {
@@ -5101,6 +5129,7 @@  int hns3_init_all_ring(struct hns3_nic_priv *priv)
 	int i, j;
 	int ret;
 
+	hns3_update_tx_spare_buf_config(priv);
 	for (i = 0; i < ring_num; i++) {
 		ret = hns3_alloc_ring_memory(&priv->ring[i]);
 		if (ret) {
@@ -5305,6 +5334,8 @@  static int hns3_client_init(struct hnae3_handle *handle)
 	priv->ae_handle = handle;
 	priv->tx_timeout_count = 0;
 	priv->max_non_tso_bd_num = ae_dev->dev_specs.max_non_tso_bd_num;
+	priv->min_tx_copybreak = 0;
+	priv->min_tx_spare_buf_size = 0;
 	set_bit(HNS3_NIC_STATE_DOWN, &priv->state);
 
 	handle->msg_enable = netif_msg_init(debug, DEFAULT_MSG_LEVEL);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
index d36c4ed16d8d..caf7a4df8585 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
@@ -596,6 +596,8 @@  struct hns3_nic_priv {
 	struct hns3_enet_coalesce rx_coal;
 	u32 tx_copybreak;
 	u32 rx_copybreak;
+	u32 min_tx_copybreak;
+	u32 min_tx_spare_buf_size;
 };
 
 union l3_hdr_info {
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
index b1e988347347..97eaeec1952b 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
@@ -1933,6 +1933,31 @@  static int hns3_set_tx_spare_buf_size(struct net_device *netdev,
 	return ret;
 }
 
+static int hns3_check_tx_copybreak(struct net_device *netdev, u32 copybreak)
+{
+	struct hns3_nic_priv *priv = netdev_priv(netdev);
+
+	if (copybreak < priv->min_tx_copybreak) {
+		netdev_err(netdev, "tx copybreak %u should be no less than %u!\n",
+			   copybreak, priv->min_tx_copybreak);
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int hns3_check_tx_spare_buf_size(struct net_device *netdev, u32 buf_size)
+{
+	struct hns3_nic_priv *priv = netdev_priv(netdev);
+
+	if (buf_size < priv->min_tx_spare_buf_size) {
+		netdev_err(netdev,
+			   "tx spare buf size %u should be no less than %u!\n",
+			   buf_size, priv->min_tx_spare_buf_size);
+		return -EINVAL;
+	}
+	return 0;
+}
+
 static int hns3_set_tunable(struct net_device *netdev,
 			    const struct ethtool_tunable *tuna,
 			    const void *data)
@@ -1949,6 +1974,10 @@  static int hns3_set_tunable(struct net_device *netdev,
 
 	switch (tuna->id) {
 	case ETHTOOL_TX_COPYBREAK:
+		ret = hns3_check_tx_copybreak(netdev, *(u32 *)data);
+		if (ret)
+			return ret;
+
 		priv->tx_copybreak = *(u32 *)data;
 
 		for (i = 0; i < h->kinfo.num_tqps; i++)
@@ -1963,6 +1992,10 @@  static int hns3_set_tunable(struct net_device *netdev,
 
 		break;
 	case ETHTOOL_TX_COPYBREAK_BUF_SIZE:
+		ret = hns3_check_tx_spare_buf_size(netdev, *(u32 *)data);
+		if (ret)
+			return ret;
+
 		old_tx_spare_buf_size = h->kinfo.tx_spare_buf_size;
 		new_tx_spare_buf_size = *(u32 *)data;
 		netdev_info(netdev, "request to set tx spare buf size from %u to %u\n",