| Message ID | 20211125060547.11961-1-jasowang@redhat.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net] virtio-net: enable big mode correctly | expand |
On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > large max_mtu. In this case, using small packet mode is not correct > since it may breaks the networking when MTU is grater than > ETH_DATA_LEN. > > To have a quick fix, simply enable the big packet mode when > VIRTIO_NET_F_MTU is not negotiated. This will slow down dpdk hosts which disable mergeable buffers and send standard MTU sized packets. > We can do optimization on top. I don't think it works like this, increasing mtu from guest >4k never worked, we can't regress everyone's performance with a promise to maybe sometime bring it back. > Reported-by: Eli Cohen <elic@nvidia.com> > Cc: Eli Cohen <elic@nvidia.com> > Signed-off-by: Jason Wang <jasowang@redhat.com> > > --- > drivers/net/virtio_net.c | 7 ++++--- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > index 7c43bfc1ce44..83ae3ef5eb11 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > dev->mtu = mtu; > dev->max_mtu = mtu; > > - /* TODO: size buffers correctly in this case. */ > - if (dev->mtu > ETH_DATA_LEN) > - vi->big_packets = true; > } > > + /* TODO: size buffers correctly in this case. */ > + if (dev->max_mtu > ETH_DATA_LEN) > + vi->big_packets = true; > + > if (vi->any_header_sg) > dev->needed_headroom = vi->hdr_len; > > -- > 2.25.1
On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > large max_mtu. In this case, using small packet mode is not correct > since it may breaks the networking when MTU is grater than > ETH_DATA_LEN. > > To have a quick fix, simply enable the big packet mode when > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top. > > Reported-by: Eli Cohen <elic@nvidia.com> > Cc: Eli Cohen <elic@nvidia.com> > Signed-off-by: Jason Wang <jasowang@redhat.com> > --- > drivers/net/virtio_net.c | 7 ++++--- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > index 7c43bfc1ce44..83ae3ef5eb11 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > dev->mtu = mtu; > dev->max_mtu = mtu; > > - /* TODO: size buffers correctly in this case. */ > - if (dev->mtu > ETH_DATA_LEN) > - vi->big_packets = true; > } > > + /* TODO: size buffers correctly in this case. */ > + if (dev->max_mtu > ETH_DATA_LEN) > + vi->big_packets = true; > + If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set big_packets to true. > if (vi->any_header_sg) > dev->needed_headroom = vi->hdr_len; > > -- > 2.25.1 >
On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > large max_mtu. In this case, using small packet mode is not correct > > since it may breaks the networking when MTU is grater than > > ETH_DATA_LEN. > > > > To have a quick fix, simply enable the big packet mode when > > VIRTIO_NET_F_MTU is not negotiated. > > This will slow down dpdk hosts which disable mergeable buffers > and send standard MTU sized packets. > > > We can do optimization on top. > > I don't think it works like this, increasing mtu > from guest >4k never worked, Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong. > we can't regress everyone's > performance with a promise to maybe sometime bring it back. So consider it never work before I wonder if we can assume a 1500 as max_mtu value instead of simply using MAX_MTU? Thanks > > > Reported-by: Eli Cohen <elic@nvidia.com> > > Cc: Eli Cohen <elic@nvidia.com> > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > > --- > > drivers/net/virtio_net.c | 7 ++++--- > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > --- a/drivers/net/virtio_net.c > > +++ b/drivers/net/virtio_net.c > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > dev->mtu = mtu; > > dev->max_mtu = mtu; > > > > - /* TODO: size buffers correctly in this case. */ > > - if (dev->mtu > ETH_DATA_LEN) > > - vi->big_packets = true; > > } > > > > + /* TODO: size buffers correctly in this case. */ > > + if (dev->max_mtu > ETH_DATA_LEN) > > + vi->big_packets = true; > > + > > if (vi->any_header_sg) > > dev->needed_headroom = vi->hdr_len; > > > > -- > > 2.25.1 >
On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote: > On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > > large max_mtu. In this case, using small packet mode is not correct > > > since it may breaks the networking when MTU is grater than > > > ETH_DATA_LEN. > > > > > > To have a quick fix, simply enable the big packet mode when > > > VIRTIO_NET_F_MTU is not negotiated. > > > > This will slow down dpdk hosts which disable mergeable buffers > > and send standard MTU sized packets. > > > > > We can do optimization on top. > > > > I don't think it works like this, increasing mtu > > from guest >4k never worked, > > Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong. OK, even more so then. > > we can't regress everyone's > > performance with a promise to maybe sometime bring it back. > > So consider it never work before I wonder if we can assume a 1500 as > max_mtu value instead of simply using MAX_MTU? > > Thanks You want to block guests from setting MTU to a value >GOOD_PACKET_LEN? Maybe ... it will prevent sending large packets which did work ... I'd tread carefully here, and I don't think this kind of thing is net material. > > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > > Cc: Eli Cohen <elic@nvidia.com> > > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > > > > --- > > > drivers/net/virtio_net.c | 7 ++++--- > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > > --- a/drivers/net/virtio_net.c > > > +++ b/drivers/net/virtio_net.c > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > > dev->mtu = mtu; > > > dev->max_mtu = mtu; > > > > > > - /* TODO: size buffers correctly in this case. */ > > > - if (dev->mtu > ETH_DATA_LEN) > > > - vi->big_packets = true; > > > } > > > > > > + /* TODO: size buffers correctly in this case. */ > > > + if (dev->max_mtu > ETH_DATA_LEN) > > > + vi->big_packets = true; > > > + > > > if (vi->any_header_sg) > > > dev->needed_headroom = vi->hdr_len; > > > > > > -- > > > 2.25.1 > >
On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote: > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > large max_mtu. In this case, using small packet mode is not correct > > since it may breaks the networking when MTU is grater than > > ETH_DATA_LEN. > > > > To have a quick fix, simply enable the big packet mode when > > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top. > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > Cc: Eli Cohen <elic@nvidia.com> > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > --- > > drivers/net/virtio_net.c | 7 ++++--- > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > --- a/drivers/net/virtio_net.c > > +++ b/drivers/net/virtio_net.c > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > dev->mtu = mtu; > > dev->max_mtu = mtu; > > > > - /* TODO: size buffers correctly in this case. */ > > - if (dev->mtu > ETH_DATA_LEN) > > - vi->big_packets = true; > > } > > > > + /* TODO: size buffers correctly in this case. */ > > + if (dev->max_mtu > ETH_DATA_LEN) > > + vi->big_packets = true; > > + > > If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal > ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set > big_packets to true. I may miss something, the dev->max_mtu is just assigned to the mtu value read from the config space in the code block above (inside the feature check of VIRTIO_NET_F_MTU). Thanks > > > > if (vi->any_header_sg) > > dev->needed_headroom = vi->hdr_len; > > > > -- > > 2.25.1 > > >
On Thu, Nov 25, 2021 at 3:15 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote: > > On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > > > large max_mtu. In this case, using small packet mode is not correct > > > > since it may breaks the networking when MTU is grater than > > > > ETH_DATA_LEN. > > > > > > > > To have a quick fix, simply enable the big packet mode when > > > > VIRTIO_NET_F_MTU is not negotiated. > > > > > > This will slow down dpdk hosts which disable mergeable buffers > > > and send standard MTU sized packets. > > > > > > > We can do optimization on top. > > > > > > I don't think it works like this, increasing mtu > > > from guest >4k never worked, > > > > Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong. > > OK, even more so then. > > > > we can't regress everyone's > > > performance with a promise to maybe sometime bring it back. > > > > So consider it never work before I wonder if we can assume a 1500 as > > max_mtu value instead of simply using MAX_MTU? > > > > Thanks > > You want to block guests from setting MTU to a value >GOOD_PACKET_LEN? Yes, or fix the issue to let large packets on RX work (e.g as the TODO said, size the buffer: for <=4K mtu continue to work as add_recvbuf_small(), for >= 4K switch to use big). > Maybe ... it will prevent sending large packets which did work ... Yes, but it's strange to allow TX but not RX > I'd tread carefully here, and I don't think this kind of thing is net > material. I agree consider it can't be fixed easily. Thanks > > > > > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > > > Cc: Eli Cohen <elic@nvidia.com> > > > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > > > > > > --- > > > > drivers/net/virtio_net.c | 7 ++++--- > > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > > > --- a/drivers/net/virtio_net.c > > > > +++ b/drivers/net/virtio_net.c > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > > > dev->mtu = mtu; > > > > dev->max_mtu = mtu; > > > > > > > > - /* TODO: size buffers correctly in this case. */ > > > > - if (dev->mtu > ETH_DATA_LEN) > > > > - vi->big_packets = true; > > > > } > > > > > > > > + /* TODO: size buffers correctly in this case. */ > > > > + if (dev->max_mtu > ETH_DATA_LEN) > > > > + vi->big_packets = true; > > > > + > > > > if (vi->any_header_sg) > > > > dev->needed_headroom = vi->hdr_len; > > > > > > > > -- > > > > 2.25.1 > > > >
On Thu, Nov 25, 2021 at 03:15:33PM +0800, Jason Wang wrote: > On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote: > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > > large max_mtu. In this case, using small packet mode is not correct > > > since it may breaks the networking when MTU is grater than > > > ETH_DATA_LEN. > > > > > > To have a quick fix, simply enable the big packet mode when > > > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top. > > > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > > Cc: Eli Cohen <elic@nvidia.com> > > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > --- > > > drivers/net/virtio_net.c | 7 ++++--- > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > > --- a/drivers/net/virtio_net.c > > > +++ b/drivers/net/virtio_net.c > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > > dev->mtu = mtu; > > > dev->max_mtu = mtu; > > > > > > - /* TODO: size buffers correctly in this case. */ > > > - if (dev->mtu > ETH_DATA_LEN) > > > - vi->big_packets = true; > > > } > > > > > > + /* TODO: size buffers correctly in this case. */ > > > + if (dev->max_mtu > ETH_DATA_LEN) > > > + vi->big_packets = true; > > > + > > > > If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal > > ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set > > big_packets to true. > > I may miss something, the dev->max_mtu is just assigned to the mtu > value read from the config space in the code block above (inside the > feature check of VIRTIO_NET_F_MTU). Sorry, I meant "If VIRTIO_NET_F_MTU is ***NOT*** provided". In that case dev->max_mtu eauals ETH_DATA_LEN so you won't set vi->big_packets to true. > > Thanks > > > > > > > > if (vi->any_header_sg) > > > dev->needed_headroom = vi->hdr_len; > > > > > > -- > > > 2.25.1 > > > > > >
On Thu, Nov 25, 2021 at 03:20:07PM +0800, Jason Wang wrote: > On Thu, Nov 25, 2021 at 3:15 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote: > > > On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > > > > large max_mtu. In this case, using small packet mode is not correct > > > > > since it may breaks the networking when MTU is grater than > > > > > ETH_DATA_LEN. > > > > > > > > > > To have a quick fix, simply enable the big packet mode when > > > > > VIRTIO_NET_F_MTU is not negotiated. > > > > > > > > This will slow down dpdk hosts which disable mergeable buffers > > > > and send standard MTU sized packets. > > > > > > > > > We can do optimization on top. > > > > > > > > I don't think it works like this, increasing mtu > > > > from guest >4k never worked, > > > > > > Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong. > > > > OK, even more so then. > > > > > > we can't regress everyone's > > > > performance with a promise to maybe sometime bring it back. > > > > > > So consider it never work before I wonder if we can assume a 1500 as > > > max_mtu value instead of simply using MAX_MTU? > > > > > > Thanks > > > > You want to block guests from setting MTU to a value >GOOD_PACKET_LEN? > > Yes, or fix the issue to let large packets on RX work (e.g as the TODO > said, size the buffer: for <=4K mtu continue to work as > add_recvbuf_small(), for >= 4K switch to use big). Right. The difficulty is with changing modes, current code isn't designed for it. > > Maybe ... it will prevent sending large packets which did work ... > > Yes, but it's strange to allow TX but not RX > > > I'd tread carefully here, and I don't think this kind of thing is net > > material. > > I agree consider it can't be fixed easily. > > Thanks > > > > > > > > > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > > > > Cc: Eli Cohen <elic@nvidia.com> > > > > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > > > > > > > > --- > > > > > drivers/net/virtio_net.c | 7 ++++--- > > > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > > > > --- a/drivers/net/virtio_net.c > > > > > +++ b/drivers/net/virtio_net.c > > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > > > > dev->mtu = mtu; > > > > > dev->max_mtu = mtu; > > > > > > > > > > - /* TODO: size buffers correctly in this case. */ > > > > > - if (dev->mtu > ETH_DATA_LEN) > > > > > - vi->big_packets = true; > > > > > } > > > > > > > > > > + /* TODO: size buffers correctly in this case. */ > > > > > + if (dev->max_mtu > ETH_DATA_LEN) > > > > > + vi->big_packets = true; > > > > > + > > > > > if (vi->any_header_sg) > > > > > dev->needed_headroom = vi->hdr_len; > > > > > > > > > > -- > > > > > 2.25.1 > > > > > >
On Thu, Nov 25, 2021 at 3:20 PM Eli Cohen <elic@nvidia.com> wrote: > > On Thu, Nov 25, 2021 at 03:15:33PM +0800, Jason Wang wrote: > > On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote: > > > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > > > large max_mtu. In this case, using small packet mode is not correct > > > > since it may breaks the networking when MTU is grater than > > > > ETH_DATA_LEN. > > > > > > > > To have a quick fix, simply enable the big packet mode when > > > > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top. > > > > > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > > > Cc: Eli Cohen <elic@nvidia.com> > > > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > > --- > > > > drivers/net/virtio_net.c | 7 ++++--- > > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > > > --- a/drivers/net/virtio_net.c > > > > +++ b/drivers/net/virtio_net.c > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > > > dev->mtu = mtu; > > > > dev->max_mtu = mtu; > > > > > > > > - /* TODO: size buffers correctly in this case. */ > > > > - if (dev->mtu > ETH_DATA_LEN) > > > > - vi->big_packets = true; > > > > } > > > > > > > > + /* TODO: size buffers correctly in this case. */ > > > > + if (dev->max_mtu > ETH_DATA_LEN) > > > > + vi->big_packets = true; > > > > + > > > > > > If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal > > > ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set > > > big_packets to true. > > > > I may miss something, the dev->max_mtu is just assigned to the mtu > > value read from the config space in the code block above (inside the > > feature check of VIRTIO_NET_F_MTU). > > Sorry, I meant "If VIRTIO_NET_F_MTU is ***NOT*** provided". In that case > dev->max_mtu eauals ETH_DATA_LEN so you won't set vi->big_packets to > true. I see but in this case, the above assignment: /* MTU range: 68 - 65535 */ dev->min_mtu = MIN_MTU; dev->max_mtu = MAX_MTU; happens after alloc_etherdev_mq() which calls ether_setup(), so we are probably fine here. Thanks > > > > > Thanks > > > > > > > > > > > > if (vi->any_header_sg) > > > > dev->needed_headroom = vi->hdr_len; > > > > > > > > -- > > > > 2.25.1 > > > > > > > > > >
On Thu, Nov 25, 2021 at 3:26 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Thu, Nov 25, 2021 at 03:20:07PM +0800, Jason Wang wrote: > > On Thu, Nov 25, 2021 at 3:15 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote: > > > > On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > > > > > large max_mtu. In this case, using small packet mode is not correct > > > > > > since it may breaks the networking when MTU is grater than > > > > > > ETH_DATA_LEN. > > > > > > > > > > > > To have a quick fix, simply enable the big packet mode when > > > > > > VIRTIO_NET_F_MTU is not negotiated. > > > > > > > > > > This will slow down dpdk hosts which disable mergeable buffers > > > > > and send standard MTU sized packets. > > > > > > > > > > > We can do optimization on top. > > > > > > > > > > I don't think it works like this, increasing mtu > > > > > from guest >4k never worked, > > > > > > > > Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong. > > > > > > OK, even more so then. > > > > > > > > we can't regress everyone's > > > > > performance with a promise to maybe sometime bring it back. > > > > > > > > So consider it never work before I wonder if we can assume a 1500 as > > > > max_mtu value instead of simply using MAX_MTU? > > > > > > > > Thanks > > > > > > You want to block guests from setting MTU to a value >GOOD_PACKET_LEN? > > > > Yes, or fix the issue to let large packets on RX work (e.g as the TODO > > said, size the buffer: for <=4K mtu continue to work as > > add_recvbuf_small(), for >= 4K switch to use big). > > Right. The difficulty is with changing modes, current code isn't > designed for it. I think it might work if we reset the device during the mode change. Thanks > > > > Maybe ... it will prevent sending large packets which did work ... > > > > Yes, but it's strange to allow TX but not RX > > > > > I'd tread carefully here, and I don't think this kind of thing is net > > > material. > > > > I agree consider it can't be fixed easily. > > > > Thanks > > > > > > > > > > > > > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > > > > > Cc: Eli Cohen <elic@nvidia.com> > > > > > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > > > > > > > > > > --- > > > > > > drivers/net/virtio_net.c | 7 ++++--- > > > > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > > > > > --- a/drivers/net/virtio_net.c > > > > > > +++ b/drivers/net/virtio_net.c > > > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > > > > > dev->mtu = mtu; > > > > > > dev->max_mtu = mtu; > > > > > > > > > > > > - /* TODO: size buffers correctly in this case. */ > > > > > > - if (dev->mtu > ETH_DATA_LEN) > > > > > > - vi->big_packets = true; > > > > > > } > > > > > > > > > > > > + /* TODO: size buffers correctly in this case. */ > > > > > > + if (dev->max_mtu > ETH_DATA_LEN) > > > > > > + vi->big_packets = true; > > > > > > + > > > > > > if (vi->any_header_sg) > > > > > > dev->needed_headroom = vi->hdr_len; > > > > > > > > > > > > -- > > > > > > 2.25.1 > > > > > > > > >
On Thu, Nov 25, 2021 at 03:26:22PM +0800, Jason Wang wrote: > On Thu, Nov 25, 2021 at 3:20 PM Eli Cohen <elic@nvidia.com> wrote: > > > > On Thu, Nov 25, 2021 at 03:15:33PM +0800, Jason Wang wrote: > > > On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote: > > > > > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > > > > large max_mtu. In this case, using small packet mode is not correct > > > > > since it may breaks the networking when MTU is grater than > > > > > ETH_DATA_LEN. > > > > > > > > > > To have a quick fix, simply enable the big packet mode when > > > > > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top. > > > > > > > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > > > > Cc: Eli Cohen <elic@nvidia.com> > > > > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > > > --- > > > > > drivers/net/virtio_net.c | 7 ++++--- > > > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > > > > --- a/drivers/net/virtio_net.c > > > > > +++ b/drivers/net/virtio_net.c > > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > > > > dev->mtu = mtu; > > > > > dev->max_mtu = mtu; > > > > > > > > > > - /* TODO: size buffers correctly in this case. */ > > > > > - if (dev->mtu > ETH_DATA_LEN) > > > > > - vi->big_packets = true; > > > > > } > > > > > > > > > > + /* TODO: size buffers correctly in this case. */ > > > > > + if (dev->max_mtu > ETH_DATA_LEN) > > > > > + vi->big_packets = true; > > > > > + > > > > > > > > If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal > > > > ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set > > > > big_packets to true. > > > > > > I may miss something, the dev->max_mtu is just assigned to the mtu > > > value read from the config space in the code block above (inside the > > > feature check of VIRTIO_NET_F_MTU). > > > > Sorry, I meant "If VIRTIO_NET_F_MTU is ***NOT*** provided". In that case > > dev->max_mtu eauals ETH_DATA_LEN so you won't set vi->big_packets to > > true. > > I see but in this case, the above assignment: > > /* MTU range: 68 - 65535 */ > dev->min_mtu = MIN_MTU; > dev->max_mtu = MAX_MTU; > > happens after alloc_etherdev_mq() which calls ether_setup(), so we are > probably fine here. > > Thanks Actually the issue with VIRTIO_NET_F_MTU is that devices might be tempted to expose 9k here simply because they support jumbo frames, if they also don't support mergeable buffers this will force big packet mode. > > > > > > > > Thanks > > > > > > > > > > > > > > > > if (vi->any_header_sg) > > > > > dev->needed_headroom = vi->hdr_len; > > > > > > > > > > -- > > > > > 2.25.1 > > > > > > > > > > > > > >
On Thu, Nov 25, 2021 at 03:26:22PM +0800, Jason Wang wrote: > On Thu, Nov 25, 2021 at 3:20 PM Eli Cohen <elic@nvidia.com> wrote: > > > > On Thu, Nov 25, 2021 at 03:15:33PM +0800, Jason Wang wrote: > > > On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote: > > > > > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > > > > large max_mtu. In this case, using small packet mode is not correct > > > > > since it may breaks the networking when MTU is grater than > > > > > ETH_DATA_LEN. > > > > > > > > > > To have a quick fix, simply enable the big packet mode when > > > > > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top. > > > > > > > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > > > > Cc: Eli Cohen <elic@nvidia.com> > > > > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > > > --- > > > > > drivers/net/virtio_net.c | 7 ++++--- > > > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > > > > --- a/drivers/net/virtio_net.c > > > > > +++ b/drivers/net/virtio_net.c > > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > > > > dev->mtu = mtu; > > > > > dev->max_mtu = mtu; > > > > > > > > > > - /* TODO: size buffers correctly in this case. */ > > > > > - if (dev->mtu > ETH_DATA_LEN) > > > > > - vi->big_packets = true; > > > > > } > > > > > > > > > > + /* TODO: size buffers correctly in this case. */ > > > > > + if (dev->max_mtu > ETH_DATA_LEN) > > > > > + vi->big_packets = true; > > > > > + > > > > > > > > If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal > > > > ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set > > > > big_packets to true. > > > > > > I may miss something, the dev->max_mtu is just assigned to the mtu > > > value read from the config space in the code block above (inside the > > > feature check of VIRTIO_NET_F_MTU). > > > > Sorry, I meant "If VIRTIO_NET_F_MTU is ***NOT*** provided". In that case > > dev->max_mtu eauals ETH_DATA_LEN so you won't set vi->big_packets to > > true. > > I see but in this case, the above assignment: > > /* MTU range: 68 - 65535 */ > dev->min_mtu = MIN_MTU; > dev->max_mtu = MAX_MTU; > > happens after alloc_etherdev_mq() which calls ether_setup(), so we are > probably fine here. > I see, thanks. > Thanks > > > > > > > > > Thanks > > > > > > > > > > > > > > > > if (vi->any_header_sg) > > > > > dev->needed_headroom = vi->hdr_len; > > > > > > > > > > -- > > > > > 2.25.1 > > > > > > > > > > > > > > >
On Thu, Nov 25, 2021 at 03:28:31PM +0800, Jason Wang wrote: > On Thu, Nov 25, 2021 at 3:26 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Thu, Nov 25, 2021 at 03:20:07PM +0800, Jason Wang wrote: > > > On Thu, Nov 25, 2021 at 3:15 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote: > > > > > On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote: > > > > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very > > > > > > > large max_mtu. In this case, using small packet mode is not correct > > > > > > > since it may breaks the networking when MTU is grater than > > > > > > > ETH_DATA_LEN. > > > > > > > > > > > > > > To have a quick fix, simply enable the big packet mode when > > > > > > > VIRTIO_NET_F_MTU is not negotiated. > > > > > > > > > > > > This will slow down dpdk hosts which disable mergeable buffers > > > > > > and send standard MTU sized packets. > > > > > > > > > > > > > We can do optimization on top. > > > > > > > > > > > > I don't think it works like this, increasing mtu > > > > > > from guest >4k never worked, > > > > > > > > > > Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong. > > > > > > > > OK, even more so then. > > > > > > > > > > we can't regress everyone's > > > > > > performance with a promise to maybe sometime bring it back. > > > > > > > > > > So consider it never work before I wonder if we can assume a 1500 as > > > > > max_mtu value instead of simply using MAX_MTU? > > > > > > > > > > Thanks > > > > > > > > You want to block guests from setting MTU to a value >GOOD_PACKET_LEN? > > > > > > Yes, or fix the issue to let large packets on RX work (e.g as the TODO > > > said, size the buffer: for <=4K mtu continue to work as > > > add_recvbuf_small(), for >= 4K switch to use big). > > > > Right. The difficulty is with changing modes, current code isn't > > designed for it. > > I think it might work if we reset the device during the mode change. > > Thanks For sure. It's hard to do without races though, and we need to carefully restore all the programming done so far. Maybe it will be easier if we do something like disable_irq to reliably suppress interrupts from hardware. > > > > > > Maybe ... it will prevent sending large packets which did work ... > > > > > > Yes, but it's strange to allow TX but not RX > > > > > > > I'd tread carefully here, and I don't think this kind of thing is net > > > > material. > > > > > > I agree consider it can't be fixed easily. > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > Reported-by: Eli Cohen <elic@nvidia.com> > > > > > > > Cc: Eli Cohen <elic@nvidia.com> > > > > > > > Signed-off-by: Jason Wang <jasowang@redhat.com> > > > > > > > > > > > > > > --- > > > > > > > drivers/net/virtio_net.c | 7 ++++--- > > > > > > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > > > > > > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644 > > > > > > > --- a/drivers/net/virtio_net.c > > > > > > > +++ b/drivers/net/virtio_net.c > > > > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) > > > > > > > dev->mtu = mtu; > > > > > > > dev->max_mtu = mtu; > > > > > > > > > > > > > > - /* TODO: size buffers correctly in this case. */ > > > > > > > - if (dev->mtu > ETH_DATA_LEN) > > > > > > > - vi->big_packets = true; > > > > > > > } > > > > > > > > > > > > > > + /* TODO: size buffers correctly in this case. */ > > > > > > > + if (dev->max_mtu > ETH_DATA_LEN) > > > > > > > + vi->big_packets = true; > > > > > > > + > > > > > > > if (vi->any_header_sg) > > > > > > > dev->needed_headroom = vi->hdr_len; > > > > > > > > > > > > > > -- > > > > > > > 2.25.1 > > > > > > > > > > > >
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 7c43bfc1ce44..83ae3ef5eb11 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev) dev->mtu = mtu; dev->max_mtu = mtu; - /* TODO: size buffers correctly in this case. */ - if (dev->mtu > ETH_DATA_LEN) - vi->big_packets = true; } + /* TODO: size buffers correctly in this case. */ + if (dev->max_mtu > ETH_DATA_LEN) + vi->big_packets = true; + if (vi->any_header_sg) dev->needed_headroom = vi->hdr_len;
When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very large max_mtu. In this case, using small packet mode is not correct since it may breaks the networking when MTU is grater than ETH_DATA_LEN. To have a quick fix, simply enable the big packet mode when VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top. Reported-by: Eli Cohen <elic@nvidia.com> Cc: Eli Cohen <elic@nvidia.com> Signed-off-by: Jason Wang <jasowang@redhat.com> --- drivers/net/virtio_net.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)