mbox series

[iwl-next,v3,00/13] Add E800 live migration driver

Message ID 20230918062546.40419-1-yahui.cao@intel.com (mailing list archive)
Headers show
Series Add E800 live migration driver | expand

Message

Cao, Yahui Sept. 18, 2023, 6:25 a.m. UTC 
This series adds vfio live migration support for Intel E810 VF devices
based on the v2 migration protocol definition series discussed here[0].

Steps to test:
1. Bind one or more E810 VF devices to the module ice_vfio_pci.ko
2. Assign the VFs to the virtual machine and enable device live migration
3. Run a workload using IAVF inside the VM, for example, iperf.
4. Migrate the VM from the source node to a destination node.

Thanks,
Yahui
[0] https://lore.kernel.org/kvm/20220224142024.147653-1-yishaih@nvidia.com/

Change log:

v2 --> v3: link [2]
 - Add P2P support in vfio driver (Jason)
 - Remove source/destination check in vfio driver (Jason)
 - Restructure PF exported API with proper types and layering (Jason)
 - Change patchset email sender.
 - Reword commit message and comments to be more reviewer-friendly (Kevin)
 - Add s-o-b for Patch01 (Kevin)
 - Merge Patch08 into Patch04 and merge Patch13 into Patch06 (Kevin)
 - Remove uninit() in VF destroy stage for Patch 05 (Kevin)
 - change migration_active to migration_enabled (Kevin)
 - Add total_size in devstate to greatly simplify the various checks for
   Patch07 (Kevin)
 - Add magic and version in device state for Patch07 (Kevin)
 - Fix rx head init issue in Patch10 (Kevin)
 - Remove DMA access for Guest Memory at device resume stage and deprecate
   the approach to restore TX head in VF space, instead restore TX head in
   PF space and then switch context back to VF space which is transparent
   to Guest for Patch11 (Jason, Kevin)
 - Use non-interrupt mode instead of VF MSIX vector to restore TX head for
   Patch11 (Kevin)
 - Move VF pci mmio save/restore from vfio driver into PF driver
 - Add configuration match check at device resume stage (Kevin)
 - Remove sleep before stopping queue at device suspend stage (Kevin)
 - Let PF respond failure to VF if virtual channel messages logging failed (Kevin)
 - Add migration setup and description in cover letter

[2] https://lore.kernel.org/intel-wired-lan/20230621091112.44945-1-lingyu.liu@intel.com/

v1 --> v2: link [1]
 - clarified comments and commit message

[1] https://lore.kernel.org/intel-wired-lan/20230620100001.5331-1-lingyu.liu@intel.com/

---

Lingyu Liu (9):
  ice: Introduce VF state ICE_VF_STATE_REPLAYING_VC for migration
  ice: Add fundamental migration init and exit function
  ice: Log virtual channel messages in PF
  ice: Add device state save/restore function for migration
  ice: Fix VSI id in virtual channel message for migration
  ice: Save and restore RX Queue head
  ice: Save and restore TX Queue head
  ice: Add device suspend function for migration
  vfio/ice: Implement vfio_pci driver for E800 devices

Xu Ting (1):
  ice: Fix missing legacy 32byte RXDID in the supported bitmap

Yahui Cao (3):
  ice: Add function to get RX queue context
  ice: Add function to get and set TX queue context
  ice: Save and restore mmio registers

 MAINTAINERS                                   |    7 +
 drivers/net/ethernet/intel/ice/Makefile       |    3 +-
 drivers/net/ethernet/intel/ice/ice.h          |    3 +
 drivers/net/ethernet/intel/ice/ice_common.c   |  484 +++++-
 drivers/net/ethernet/intel/ice/ice_common.h   |   11 +
 .../net/ethernet/intel/ice/ice_hw_autogen.h   |   23 +
 .../net/ethernet/intel/ice/ice_lan_tx_rx.h    |    3 +
 drivers/net/ethernet/intel/ice/ice_main.c     |   15 +
 .../net/ethernet/intel/ice/ice_migration.c    | 1344 +++++++++++++++++
 .../intel/ice/ice_migration_private.h         |   44 +
 drivers/net/ethernet/intel/ice/ice_vf_lib.c   |    4 +
 drivers/net/ethernet/intel/ice/ice_vf_lib.h   |   10 +
 drivers/net/ethernet/intel/ice/ice_virtchnl.c |  267 ++--
 drivers/net/ethernet/intel/ice/ice_virtchnl.h |   15 +-
 .../ethernet/intel/ice/ice_virtchnl_fdir.c    |   28 +-
 drivers/vfio/pci/Kconfig                      |    2 +
 drivers/vfio/pci/Makefile                     |    2 +
 drivers/vfio/pci/ice/Kconfig                  |   10 +
 drivers/vfio/pci/ice/Makefile                 |    4 +
 drivers/vfio/pci/ice/ice_vfio_pci.c           |  707 +++++++++
 include/linux/net/intel/ice_migration.h       |   42 +
 21 files changed, 2916 insertions(+), 112 deletions(-)
 create mode 100644 drivers/net/ethernet/intel/ice/ice_migration.c
 create mode 100644 drivers/net/ethernet/intel/ice/ice_migration_private.h
 create mode 100644 drivers/vfio/pci/ice/Kconfig
 create mode 100644 drivers/vfio/pci/ice/Makefile
 create mode 100644 drivers/vfio/pci/ice/ice_vfio_pci.c
 create mode 100644 include/linux/net/intel/ice_migration.h

Comments

Jason Gunthorpe Oct. 16, 2023, 4:51 p.m. UTC | #1 
On Mon, Sep 18, 2023 at 06:25:33AM +0000, Yahui Cao wrote:
> This series adds vfio live migration support for Intel E810 VF devices
> based on the v2 migration protocol definition series discussed here[0].
> 
> Steps to test:
> 1. Bind one or more E810 VF devices to the module ice_vfio_pci.ko
> 2. Assign the VFs to the virtual machine and enable device live migration
> 3. Run a workload using IAVF inside the VM, for example, iperf.
> 4. Migrate the VM from the source node to a destination node.

It looked better that the previous versions, I wanted to take a closer
look but the series didn't apply to v6.6-rc6. When you send v4 with
the compilation fixed include a git link please

Jason
Cao, Yahui Oct. 20, 2023, 6:49 a.m. UTC | #2 
On 10/17/2023 12:51 AM, Jason Gunthorpe wrote:
> On Mon, Sep 18, 2023 at 06:25:33AM +0000, Yahui Cao wrote:
>> This series adds vfio live migration support for Intel E810 VF devices
>> based on the v2 migration protocol definition series discussed here[0].
>>
>> Steps to test:
>> 1. Bind one or more E810 VF devices to the module ice_vfio_pci.ko
>> 2. Assign the VFs to the virtual machine and enable device live migration
>> 3. Run a workload using IAVF inside the VM, for example, iperf.
>> 4. Migrate the VM from the source node to a destination node.
> It looked better that the previous versions, I wanted to take a closer
> look but the series didn't apply to v6.6-rc6. When you send v4 with
> the compilation fixed include a git link please
>
> Jason


Sure. I'll send next version with compilation fixed and a git link.

Thanks.
Yahui.