From patchwork Thu Jul 29 19:19:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Vivier X-Patchwork-Id: 12409535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8373EC432BE for ; Thu, 29 Jul 2021 19:20:20 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E44736044F for ; Thu, 29 Jul 2021 19:20:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E44736044F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:56070 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m9BZq-0006wW-QV for qemu-devel@archiver.kernel.org; Thu, 29 Jul 2021 15:20:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41600) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m9BZ2-0006Db-Ej for qemu-devel@nongnu.org; Thu, 29 Jul 2021 15:19:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:26608) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m9BYy-0005YD-Uc for qemu-devel@nongnu.org; Thu, 29 Jul 2021 15:19:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1627586363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=HoAmKHh4GhnppbcZkpyMJ1vZFCdirrkU6MeQ3ilyTdc=; b=Er7ZOBM5PcfJHU5YHmUnuxfbDDrjxtJT/kbg9Lw3g4EnJPY0cARbod8lPNbIfw2mtMK2uJ 1BYlTqWSDb7vWZUnonAaAJK9hlcdcdY8afRwaSDWoGJstBbz7Gntw0DTq3WPMXIWVPjB6B TPo7xU/rHnn0eXO15yTkXzz8IQtLi0Q= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-430-oOaX4TOTOkqRdsbqWcF1IQ-1; Thu, 29 Jul 2021 15:19:21 -0400 X-MC-Unique: oOaX4TOTOkqRdsbqWcF1IQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 357C418C9F40 for ; Thu, 29 Jul 2021 19:19:20 +0000 (UTC) Received: from thinkpad.redhat.com (unknown [10.39.192.222]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2B58269CB7; Thu, 29 Jul 2021 19:19:10 +0000 (UTC) From: Laurent Vivier To: qemu-devel@nongnu.org Subject: [PATCH 0/2] virtio: failover: allow to keep the VFIO device rather than the virtio-net one Date: Thu, 29 Jul 2021 21:19:08 +0200 Message-Id: <20210729191910.317114-1-lvivier@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=lvivier@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=lvivier@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.717, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , Jason Wang , Jens Freimann , "Michael S. Tsirkin" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" With failover, when the guest virtio-net driver doesn't support the STANDBY feature, the primary device is not plugged and only the virtio-net device is kept. Doing like that we can migrate the machine and keep the network connection. But in some cases, when performance is more important than availability we would prefer to keep the VFIO device rather than the virtio-net one, even if it means to lose the network connection during the migration of the machine. To do that we can't simply unplug the virtio-net device and plug the VFIO one because for the migration the initial state must be kept (virtio-net plugged, VFIO unplugged) but we can try to disable the virtio-net driver and plug the VFIO card, so the initial state is correct (the virtio-net card is plugged, but disabled in guest, and the VFIO card is unplugged before migration). A way to disable the virtio-net driver at startup is to trigger an error in the kernel probing function. We can do that by disabling the RX queues. I tried to add a function to disable the queue that does that by setting the queue vring "num" value to 0 (and re-enable the queue by setting back to the default value). This change doesn't impact the case when guest and host support the STANDBY feature. I've introduced the "failover-default" property to virtio-net device to set which device to keep (failover-default=true keeps the virtio-net device, =off the other one). For example, with a guest driver that doesn't support STANDBY: ... -device virtio-net-pci,id=virtio0,failover=on,failover-default=on \ -device vfio-pci,host=$PCI,id=hostdev0,failover_pair_id=virtio0 \ ... [root@localhost ~]# ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc pfifo_fast state UP q0 link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff inet 192.168.20.2/24 brd 192.168.20.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::2428:c5ff:fe7f:1424/64 scope link valid_lft forever preferred_lft forever # ethtool -i eth0 driver: virtio_net version: 1.0.0 firmware-version: expansion-rom-version: bus-info: 0000:04:00.0 supports-statistics: no supports-test: no supports-eeprom-access: no supports-register-dump: no supports-priv-flags: no ... -device virtio-net-pci,id=virtio0,failover=on,failover-default=off \ -device vfio-pci,host=$PCI,id=hostdev0,failover_pair_id=virtio0 \ ... [root@localhost ~]# ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp2s0: mtu 1500 qdisc mq state UP qlen 100 link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff inet 192.168.20.2/24 brd 192.168.20.255 scope global enp2s0 valid_lft forever preferred_lft forever inet6 fe80::2428:c5ff:fe7f:1424/64 scope link valid_lft forever preferred_lft forever [root@localhost ~]# ethtool -i enp2s0 driver: i40evf version: 1.6.27-k firmware-version: N/A expansion-rom-version: bus-info: 0000:02:00.0 supports-statistics: yes supports-test: no supports-eeprom-access: no supports-register-dump: no supports-priv-flags: no With guest driver that supports STANDBY, we would always have: [root@localhost ~]# ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group defau0 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp4s0: mtu 1500 qdisc noqueue state UP gr0 link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff inet 192.168.20.2/24 brd 192.168.20.255 scope global noprefixroute enp4s0 valid_lft forever preferred_lft forever inet6 fe80::2428:c5ff:fe7f:1424/64 scope link valid_lft forever preferred_lft forever 3: enp4s0nsby: mtu 1500 qdisc fq_codel master0 link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff 4: enp2s0: mtu 1500 qdisc mq master enp4s0 st0 link/ether 26:28:c5:7f:14:24 brd ff:ff:ff:ff:ff:ff [root@localhost ~]# ethtool -i enp4s0 driver: net_failover version: 0.1 firmware-version: expansion-rom-version: bus-info: supports-statistics: no supports-test: no supports-eeprom-access: no supports-register-dump: no supports-priv-flags: no [root@localhost ~]# ethtool -i enp4s0nsby driver: virtio_net version: 1.0.0 firmware-version: expansion-rom-version: bus-info: 0000:04:00.0 supports-statistics: yes supports-test: no supports-eeprom-access: no supports-register-dump: no supports-priv-flags: no [root@localhost ~]# ethtool -i enp2s0 driver: iavf version: 4.18.0-310.el8.x86_64 firmware-version: N/A expansion-rom-version: bus-info: 0000:02:00.0 supports-statistics: yes supports-test: no supports-eeprom-access: no supports-register-dump: no supports-priv-flags: yes Laurent Vivier (2): virtio: add a way to disable a queue virtio: failover: define the default device to use in case of error include/hw/virtio/virtio-net.h | 1 + include/hw/virtio/virtio.h | 2 ++ hw/net/virtio-net.c | 34 ++++++++++++++++++++++++++++++++++ hw/virtio/virtio.c | 10 ++++++++++ 4 files changed, 47 insertions(+)