| Message ID | 20200403091406.22381-1-guennadi.liakhovetski@linux.intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | ASoC: SOF DSP virtualisation | expand |
On Fri, Apr 03, 2020 at 11:13:54AM +0200, Guennadi Liakhovetski wrote: > This patch series extends the SOF driver to add support for DSP > virtualisation to ASoC. It is built on top of VirtIO, contains a > guest driver and a vhost in-kernel guest driver. This version > supports a single playback and a single capture interface on the > guest. The specific guest audio topology is supplied by the host I've asked a couple of times for documentation of the protocol here but don't think I've seen anything yet?
Hi Mark, On Fri, Apr 03, 2020 at 10:28:42AM +0100, Mark Brown wrote: > On Fri, Apr 03, 2020 at 11:13:54AM +0200, Guennadi Liakhovetski wrote: > > This patch series extends the SOF driver to add support for DSP > > virtualisation to ASoC. It is built on top of VirtIO, contains a > > guest driver and a vhost in-kernel guest driver. This version > > supports a single playback and a single capture interface on the > > guest. The specific guest audio topology is supplied by the host > > I've asked a couple of times for documentation of the protocol here but > don't think I've seen anything yet? Sorry, we were thinking about the best way to reply. I think Liam will provide more procise information. The API we are using is the same as what is already used by SOF to communicate with the DSP firmware. With VirtIO we re-use the same IPC messages as those, used with the DSP, they are just transmitted over Virtual Queues. Additionally to existing IPC messages the SOF VirtIO implementation defines 3 more message types for topology loading and DSP power management. IPC message types are defined in include/sound/sof/header.h, IPC overview is available in https://thesofproject.github.io/latest/developer_guides/linux_driver/architecture/sof_driver_arch.html#ipc-processing but I'm not aware of a formal SOF IPC protocol documentation. Thanks Guennadi
On Fri, Apr 03, 2020 at 01:04:05PM +0200, Guennadi Liakhovetski wrote: > On Fri, Apr 03, 2020 at 10:28:42AM +0100, Mark Brown wrote: > > I've asked a couple of times for documentation of the protocol here but > > don't think I've seen anything yet? > Sorry, we were thinking about the best way to reply. I think Liam will > provide more procise information. The API we are using is the same as > what is already used by SOF to communicate with the DSP firmware. With > VirtIO we re-use the same IPC messages as those, used with the DSP, > they are just transmitted over Virtual Queues. Additionally to existing Quickly scanning through the code it's explicitly talking about things like back ends and front ends... In any case talking to virtualization people they strongly recommend that any new virtio stuff should be documented up front.
On Fri, 2020-04-03 at 13:04 +0200, Guennadi Liakhovetski wrote: > Hi Mark, > > On Fri, Apr 03, 2020 at 10:28:42AM +0100, Mark Brown wrote: > > On Fri, Apr 03, 2020 at 11:13:54AM +0200, Guennadi Liakhovetski > > wrote: > > > This patch series extends the SOF driver to add support for DSP > > > virtualisation to ASoC. It is built on top of VirtIO, contains a > > > guest driver and a vhost in-kernel guest driver. This version > > > supports a single playback and a single capture interface on the > > > guest. The specific guest audio topology is supplied by the host > > > > I've asked a couple of times for documentation of the protocol here > > but > > don't think I've seen anything yet? Sorry Mark, the whole series should not have been sent since we are still pending on some OASIS standards being ratified. Guennadi, please just send the patches that add the protocol independent dependencies to ASoC and SOF driver only for review atm (patches 1,2,3,5,6 & 7). The full series is blocking on 1) the virto-snd patches being merged. This will then allow the code (when modified) to run HDA like audio on SOF DSPs. 2) virtio DMA buffer sharing being concluded for "zero copy" usage. 3) rpmsg integration. The SOF IPC will use rpmsg virtio transport between host and guests. This series does not mandate a DSP IPC standard, since this may differ between DSP vendors, but it will use OASIS standards for virtio-snd, DMA buffers and rpmsg. I would say that parts of the "SOF protocol" may be useful for other vendors in the DAPM/topology areas where users want to connect guest topologies to host topologies (connected internally with DAPM). Thanks Liam
On Fri, Apr 03, 2020 at 05:10:20PM +0100, Liam Girdwood wrote: > 3) rpmsg integration. The SOF IPC will use rpmsg virtio transport > between host and guests. > This series does not mandate a DSP IPC standard, since this may differ > between DSP vendors, but it will use OASIS standards for virtio-snd, > DMA buffers and rpmsg. This sounds good, I think rpmsg bit was the initial part of this that I was missing (but that will then lead into requiring the others) - this is essentially VFIO for a software defined component so it's clear that there's value in standardization on the discovery and transport parts of things but I agree that the actual messages are just a vendor thing and don't need an OASIS standard.
Hi Liam, Thanks for the clarifications. On Fri, Apr 03, 2020 at 05:10:20PM +0100, Liam Girdwood wrote: > On Fri, 2020-04-03 at 13:04 +0200, Guennadi Liakhovetski wrote: > > Hi Mark, > > > > On Fri, Apr 03, 2020 at 10:28:42AM +0100, Mark Brown wrote: > > > On Fri, Apr 03, 2020 at 11:13:54AM +0200, Guennadi Liakhovetski > > > wrote: > > > > This patch series extends the SOF driver to add support for DSP > > > > virtualisation to ASoC. It is built on top of VirtIO, contains a > > > > guest driver and a vhost in-kernel guest driver. This version > > > > supports a single playback and a single capture interface on the > > > > guest. The specific guest audio topology is supplied by the host > > > > > > I've asked a couple of times for documentation of the protocol here > > > but > > > don't think I've seen anything yet? > > Sorry Mark, the whole series should not have been sent since we are > still pending on some OASIS standards being ratified. Guennadi, please > just send the patches that add the protocol independent dependencies to > ASoC and SOF driver only for review atm (patches 1,2,3,5,6 & 7). > > The full series is blocking on > > 1) the virto-snd patches being merged. This will then allow the code > (when modified) to run HDA like audio on SOF DSPs. > > 2) virtio DMA buffer sharing being concluded for "zero copy" usage. I don't think we're blocked by this. If I understand correctly it is our intention to first upstream the present copying solution and then implement zero-copy as a next step. > 3) rpmsg integration. The SOF IPC will use rpmsg virtio transport > between host and guests. We started discussing this on github, unfortunately this didn't come to a conclusion. From what I've read in the kernel, RPMSG is currencly used there in scenarios, that are very different from ours. Typically you have a Linux host, that uses RPMSG to communicate with an "embedded" counterpart, where that communication includes boooting ELF firmware on that counterpart, and then using RPMSG on top of Virtual Queues to communicate with it. Our case is quite different. I'm not saying, that it is impossible to use the Linux RPMSG subsystem for use-cases like ours, but it seems to me, that this would require a significant effort on the Linux RPMSG core implementation, and we would be the first use-case for this. The rest has to be discussed. Thanks Guennadi > This series does not mandate a DSP IPC standard, since this may differ > between DSP vendors, but it will use OASIS standards for virtio-snd, > DMA buffers and rpmsg. > > I would say that parts of the "SOF protocol" may be useful for other > vendors in the DAPM/topology areas where users want to connect guest > topologies to host topologies (connected internally with DAPM). > > Thanks > > Liam > > >
On Fri, Apr 03, 2020 at 08:09:19PM +0200, Guennadi Liakhovetski wrote: > Hi Liam, > > Thanks for the clarifications. > > On Fri, Apr 03, 2020 at 05:10:20PM +0100, Liam Girdwood wrote: > > On Fri, 2020-04-03 at 13:04 +0200, Guennadi Liakhovetski wrote: > > > Hi Mark, > > > > > > On Fri, Apr 03, 2020 at 10:28:42AM +0100, Mark Brown wrote: > > > > On Fri, Apr 03, 2020 at 11:13:54AM +0200, Guennadi Liakhovetski > > > > wrote: > > > > > This patch series extends the SOF driver to add support for DSP > > > > > virtualisation to ASoC. It is built on top of VirtIO, contains a > > > > > guest driver and a vhost in-kernel guest driver. This version > > > > > supports a single playback and a single capture interface on the > > > > > guest. The specific guest audio topology is supplied by the host > > > > > > > > I've asked a couple of times for documentation of the protocol here > > > > but > > > > don't think I've seen anything yet? > > > > Sorry Mark, the whole series should not have been sent since we are > > still pending on some OASIS standards being ratified. Guennadi, please > > just send the patches that add the protocol independent dependencies to > > ASoC and SOF driver only for review atm (patches 1,2,3,5,6 & 7). > > > > The full series is blocking on > > > > 1) the virto-snd patches being merged. This will then allow the code > > (when modified) to run HDA like audio on SOF DSPs. > > > > 2) virtio DMA buffer sharing being concluded for "zero copy" usage. > > I don't think we're blocked by this. If I understand correctly it is our > intention to first upstream the present copying solution and then > implement zero-copy as a next step. > > > 3) rpmsg integration. The SOF IPC will use rpmsg virtio transport > > between host and guests. > > We started discussing this on github, unfortunately this didn't come to a > conclusion. From what I've read in the kernel, RPMSG is currencly used > there in scenarios, that are very different from ours. Typically you have > a Linux host, that uses RPMSG to communicate with an "embedded" > counterpart, where that communication includes boooting ELF firmware on > that counterpart, and then using RPMSG on top of Virtual Queues to > communicate with it. Our case is quite different. I'm not saying, that it > is impossible to use the Linux RPMSG subsystem for use-cases like ours, > but it seems to me, that this would require a significant effort on the > Linux RPMSG core implementation, and we would be the first use-case for > this. After a discussion we agreed, that we shall try to port SOF VirtIO support on top of RPMSG, even though this would be the first ever such use of RPMSG. And since this likely will take a while and be a big change we suspend this review process for now. We'll be back. Thanks Guennadi > The rest has to be discussed. > > Thanks > Guennadi > > > This series does not mandate a DSP IPC standard, since this may differ > > between DSP vendors, but it will use OASIS standards for virtio-snd, > > DMA buffers and rpmsg. > > > > I would say that parts of the "SOF protocol" may be useful for other > > vendors in the DAPM/topology areas where users want to connect guest > > topologies to host topologies (connected internally with DAPM). > > > > Thanks > > > > Liam > > > > > >
On Tue, Apr 07, 2020 at 01:24:11PM +0200, Guennadi Liakhovetski wrote: > After a discussion we agreed, that we shall try to port SOF VirtIO support on > top of RPMSG, even though this would be the first ever such use of RPMSG. And > since this likely will take a while and be a big change we suspend this > review process for now. OK... the generic patches (IIRC it was about the list that Liam identified) looked OK so if you want to send them by themselves then I can apply them and it's less diff for you to carry.
Hi Mark, On Tue, Apr 07, 2020 at 12:30:26PM +0100, Mark Brown wrote: > On Tue, Apr 07, 2020 at 01:24:11PM +0200, Guennadi Liakhovetski wrote: > > > After a discussion we agreed, that we shall try to port SOF VirtIO support on > > top of RPMSG, even though this would be the first ever such use of RPMSG. And > > since this likely will take a while and be a big change we suspend this > > review process for now. > > OK... the generic patches (IIRC it was about the list that Liam > identified) looked OK so if you want to send them by themselves then I > can apply them and it's less diff for you to carry. Ok, I'll route them via Pierre then, because they include a new SOF IPC, which probably requires an ABI version update, even though it will never be sent to the DSP. Thanks Guennadi