mbox series

[0/3] Introduce CXL type-2 device emulation

Message ID 20241212130422.69380-1-zhiw@nvidia.com (mailing list archive)
Headers show
Series Introduce CXL type-2 device emulation | expand

Message

Zhi Wang Dec. 12, 2024, 1:04 p.m. UTC
Hi folks:

Per the discussion with Ira/Jonathan in the LPC 2024 and in the CXL
discord channel, we are trying to introduce a CXL type-2 device emulation
in QEMU, as there are work currently on supporting CXL type-2 device [1]
in Linux kernel and CXL type-2 device virtualization [2].

It provides a bare minimum base for folks who would like to:

- Contribute and test the CXL type-2 device support in the linux kernel
  and CXL type-2 virtualization without having an actual HW.
- Introduce more emulated features to prototype the kernel CXL type-2
  device features and CXL type-2 virtualization.

To test this patchset, please refer to steps in [3]. Use this patcheset
with the latest QEMU repo to be the QEMU host. It achieves the same output
as in the demo video [4]: The VFIO CXL core and VFIO CXL sample variant
driver can be attached to the emulated device in the L1 guest and assigned
to the L2 guest. The sample driver in the L2 guest can attach to the
pass-thrued device and create the CXL region.

Tested on the CXL type-2 virtualization RFC patches [3] with an extra
fix [5].

[1] https://lore.kernel.org/linux-cxl/20241209185429.54054-1-alejandro.lucero-palau@amd.com/T/#t
[2] https://www.youtube.com/watch?v=e5OW1pR84Zs
[3] https://lore.kernel.org/kvm/20240920223446.1908673-3-zhiw@nvidia.com/T/
[4] https://youtu.be/zlk_ecX9bxs?si=pf9CttcGT5KwUgiH
[5] https://lore.kernel.org/linux-cxl/20241212123959.68514-1-zhiw@nvidia.com/T/#u

Zhi Wang (3):
  hw/cxl: factor out cxl_host_addr_to_dpa()
  hw/cxl: introduce cxl_component_update_dvsec()
  hw/cxl: introduce CXL type-2 device emulation

 MAINTAINERS                    |   1 +
 docs/system/devices/cxl.rst    |  11 ++
 hw/cxl/cxl-component-utils.c   | 103 ++++++++++-
 hw/cxl/cxl-host.c              |  19 +-
 hw/mem/Kconfig                 |   5 +
 hw/mem/cxl_accel.c             | 319 +++++++++++++++++++++++++++++++++
 hw/mem/cxl_type3.c             |  61 +------
 hw/mem/meson.build             |   1 +
 include/hw/cxl/cxl_component.h |   7 +
 include/hw/cxl/cxl_device.h    |  25 +++
 include/hw/pci/pci_ids.h       |   1 +
 11 files changed, 484 insertions(+), 69 deletions(-)
 create mode 100644 hw/mem/cxl_accel.c

Comments

Alejandro Lucero Palau Dec. 12, 2024, 4:49 p.m. UTC | #1
On 12/12/24 13:04, Zhi Wang wrote:
> Hi folks:
>
> Per the discussion with Ira/Jonathan in the LPC 2024 and in the CXL
> discord channel, we are trying to introduce a CXL type-2 device emulation
> in QEMU, as there are work currently on supporting CXL type-2 device [1]
> in Linux kernel and CXL type-2 device virtualization [2].
>
> It provides a bare minimum base for folks who would like to:
>
> - Contribute and test the CXL type-2 device support in the linux kernel
>    and CXL type-2 virtualization without having an actual HW.
> - Introduce more emulated features to prototype the kernel CXL type-2
>    device features and CXL type-2 virtualization.
>
> To test this patchset, please refer to steps in [3]. Use this patcheset
> with the latest QEMU repo to be the QEMU host. It achieves the same output
> as in the demo video [4]: The VFIO CXL core and VFIO CXL sample variant
> driver can be attached to the emulated device in the L1 guest and assigned
> to the L2 guest. The sample driver in the L2 guest can attach to the
> pass-thrued device and create the CXL region.
>
> Tested on the CXL type-2 virtualization RFC patches [3] with an extra
> fix [5].
>
> [1] https://lore.kernel.org/linux-cxl/20241209185429.54054-1-alejandro.lucero-palau@amd.com/T/#t
> [2] https://www.youtube.com/watch?v=e5OW1pR84Zs
> [3] https://lore.kernel.org/kvm/20240920223446.1908673-3-zhiw@nvidia.com/T/
> [4] https://youtu.be/zlk_ecX9bxs?si=pf9CttcGT5KwUgiH
> [5] https://lore.kernel.org/linux-cxl/20241212123959.68514-1-zhiw@nvidia.com/T/#u
>
> Zhi Wang (3):
>    hw/cxl: factor out cxl_host_addr_to_dpa()
>    hw/cxl: introduce cxl_component_update_dvsec()
>    hw/cxl: introduce CXL type-2 device emulation
>
>   MAINTAINERS                    |   1 +
>   docs/system/devices/cxl.rst    |  11 ++
>   hw/cxl/cxl-component-utils.c   | 103 ++++++++++-
>   hw/cxl/cxl-host.c              |  19 +-
>   hw/mem/Kconfig                 |   5 +
>   hw/mem/cxl_accel.c             | 319 +++++++++++++++++++++++++++++++++
>   hw/mem/cxl_type3.c             |  61 +------
>   hw/mem/meson.build             |   1 +
>   include/hw/cxl/cxl_component.h |   7 +
>   include/hw/cxl/cxl_device.h    |  25 +++
>   include/hw/pci/pci_ids.h       |   1 +
>   11 files changed, 484 insertions(+), 69 deletions(-)
>   create mode 100644 hw/mem/cxl_accel.c
>

Hi Zhi,


Thank you for this patchset.


I  have a similar work done for helping in the Type2 support work, but 
it is all quick-and-dirty changes.


My main concern here is with the optional features for Type2: how to 
create an easy way for configuring Type2 devices using some qemu cxl 
param. I'm afraid I did not work on that so no suggestions at all!


Thank you
Zhi Wang Dec. 12, 2024, 6:10 p.m. UTC | #2
On 12/12/2024 18.49, Alejandro Lucero Palau wrote:
> 
> On 12/12/24 13:04, Zhi Wang wrote:
>> Hi folks:
>>
>> Per the discussion with Ira/Jonathan in the LPC 2024 and in the CXL
>> discord channel, we are trying to introduce a CXL type-2 device emulation
>> in QEMU, as there are work currently on supporting CXL type-2 device [1]
>> in Linux kernel and CXL type-2 device virtualization [2].
>>
>> It provides a bare minimum base for folks who would like to:
>>
>> - Contribute and test the CXL type-2 device support in the linux kernel
>>    and CXL type-2 virtualization without having an actual HW.
>> - Introduce more emulated features to prototype the kernel CXL type-2
>>    device features and CXL type-2 virtualization.
>>
>> To test this patchset, please refer to steps in [3]. Use this patcheset
>> with the latest QEMU repo to be the QEMU host. It achieves the same 
>> output
>> as in the demo video [4]: The VFIO CXL core and VFIO CXL sample variant
>> driver can be attached to the emulated device in the L1 guest and 
>> assigned
>> to the L2 guest. The sample driver in the L2 guest can attach to the
>> pass-thrued device and create the CXL region.
>>
>> Tested on the CXL type-2 virtualization RFC patches [3] with an extra
>> fix [5].
>>
>> [1] https://nam11.safelinks.protection.outlook.com/? 
>> url=https%3A%2F%2Flore.kernel.org%2Flinux- 
>> cxl%2F20241209185429.54054-1-alejandro.lucero- 
>> palau%40amd.com%2FT%2F%23t&data=05%7C02%7Czhiw%40nvidia.com%7C3a61139bf3554f4f38f408dd1accf1b9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638696189761390919%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6WziKnwMlZJQ4yxT2jLn7W1So0OfqYss78fOosuLiwA%3D&reserved=0
>> [2] https://nam11.safelinks.protection.outlook.com/? 
>> url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3De5OW1pR84Zs&data=05%7C02%7Czhiw%40nvidia.com%7C3a61139bf3554f4f38f408dd1accf1b9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638696189761413039%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=hTF%2F1I%2B4fYPQeCz7NhM0uvWd%2FrWfIzaKdcteD5%2BrcZ0%3D&reserved=0
>> [3] https://nam11.safelinks.protection.outlook.com/? 
>> url=https%3A%2F%2Flore.kernel.org%2Fkvm%2F20240920223446.1908673-3- 
>> zhiw%40nvidia.com%2FT%2F&data=05%7C02%7Czhiw%40nvidia.com%7C3a61139bf3554f4f38f408dd1accf1b9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638696189761425646%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wq3mr0mXZCbG3cXRKlibq%2BksTuwL8RGqiUS9jBFDfDY%3D&reserved=0
>> [4] https://nam11.safelinks.protection.outlook.com/? 
>> url=https%3A%2F%2Fyoutu.be%2Fzlk_ecX9bxs%3Fsi%3Dpf9CttcGT5KwUgiH&data=05%7C02%7Czhiw%40nvidia.com%7C3a61139bf3554f4f38f408dd1accf1b9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638696189761437780%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=SReTnBUC1bIhBwC%2BvASCXX%2F0ltIYcfWAkHXMmi%2FTRRg%3D&reserved=0
>> [5] https://nam11.safelinks.protection.outlook.com/? 
>> url=https%3A%2F%2Flore.kernel.org%2Flinux- 
>> cxl%2F20241212123959.68514-1- 
>> zhiw%40nvidia.com%2FT%2F%23u&data=05%7C02%7Czhiw%40nvidia.com%7C3a61139bf3554f4f38f408dd1accf1b9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638696189761449589%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=pmZ8JNctUlcLFwQLivNMkHj7fMt2PR24e%2BuHY%2Bk7bNA%3D&reserved=0
>>
>> Zhi Wang (3):
>>    hw/cxl: factor out cxl_host_addr_to_dpa()
>>    hw/cxl: introduce cxl_component_update_dvsec()
>>    hw/cxl: introduce CXL type-2 device emulation
>>
>>   MAINTAINERS                    |   1 +
>>   docs/system/devices/cxl.rst    |  11 ++
>>   hw/cxl/cxl-component-utils.c   | 103 ++++++++++-
>>   hw/cxl/cxl-host.c              |  19 +-
>>   hw/mem/Kconfig                 |   5 +
>>   hw/mem/cxl_accel.c             | 319 +++++++++++++++++++++++++++++++++
>>   hw/mem/cxl_type3.c             |  61 +------
>>   hw/mem/meson.build             |   1 +
>>   include/hw/cxl/cxl_component.h |   7 +
>>   include/hw/cxl/cxl_device.h    |  25 +++
>>   include/hw/pci/pci_ids.h       |   1 +
>>   11 files changed, 484 insertions(+), 69 deletions(-)
>>   create mode 100644 hw/mem/cxl_accel.c
>>
> 
> Hi Zhi,
> 
> 
> Thank you for this patchset.
> 
> 
> I  have a similar work done for helping in the Type2 support work, but 
> it is all quick-and-dirty changes.
> 
> 
> My main concern here is with the optional features for Type2: how to 
> create an easy way for configuring Type2 devices using some qemu cxl 
> param. I'm afraid I did not work on that so no suggestions at all!
> 

Hi Alejandro:

No worries. The work is to provide a minimum base for CXL folks and CXL 
type-2 folks to start with, e.g. introducing more emulated features. As 
the type-3 emulation has been quite complicated and I was thinking maybe 
having a clean start would help. For re-factoring, I was mostly thinking 
of a step by step style: E.g. when both emulation of devices are 
reaching a point to have the common routines, then we re-factor them or 
draw a glue layer.

Also, the patchset is good enough for people to test our works.

If folks are OK on this minimum emulation, I think the next thing would 
be meaningful for us is aligning the plan for what features that we want 
to plug into this, so that we can share the efforts.

The items on my list are:

- Locked HDM decoder
- CDAT and DOE

I remembered you were talking about the configuration params, I think it 
can be very helpful on prototyping different features in the kernel as 
well. Feel free to reach out for discussions.

Z.

> 
> Thank you
>