| Message ID | 20230824213312.1258499-1-cujomalainey@chromium.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [1/2] ucm: docs: Add EchoReferenceDev | expand |
On 24. 08. 23 23:33, cujomalainey@chromium.org wrote: > From: Curtis Malainey <cujomalainey@chromium.org> > > Sometimes userspace may want to use a reference channel to cancel echos > when using video chat, this value identifies the device which carries > that channel. The UCM modifier should be used for this - see "Echo Reference" comments in use-case.h. Note that this allows additional setup (in Sequences) for this stream. Jaroslav
On Sat, Aug 26, 2023 at 4:28 AM Jaroslav Kysela <perex@perex.cz> wrote: > > On 24. 08. 23 23:33, cujomalainey@chromium.org wrote: > > From: Curtis Malainey <cujomalainey@chromium.org> > > > > Sometimes userspace may want to use a reference channel to cancel echos > > when using video chat, this value identifies the device which carries > > that channel. > > The UCM modifier should be used for this - see "Echo Reference" comments in > use-case.h. > > Note that this allows additional setup (in Sequences) for this stream. > > Jaroslav I was under the impression modifiers were state manipulators that acted upon existing devices/pcms and did not designate their own PCM. That is at least how we use them in CRAS. Are there any examples of how to designate a PCM? I don't see any modifiers at all in ucm-conf repo. Curtis > > -- > Jaroslav Kysela <perex@perex.cz> > Linux Sound Maintainer; ALSA Project; Red Hat, Inc. >
On 8/28/23 12:59, Curtis Malainey wrote: > On Sat, Aug 26, 2023 at 4:28 AM Jaroslav Kysela <perex@perex.cz> wrote: >> >> On 24. 08. 23 23:33, cujomalainey@chromium.org wrote: >>> From: Curtis Malainey <cujomalainey@chromium.org> >>> >>> Sometimes userspace may want to use a reference channel to cancel echos >>> when using video chat, this value identifies the device which carries >>> that channel. >> >> The UCM modifier should be used for this - see "Echo Reference" comments in >> use-case.h. >> >> Note that this allows additional setup (in Sequences) for this stream. >> >> Jaroslav > > I was under the impression modifiers were state manipulators that > acted upon existing devices/pcms and did not designate their own PCM. > That is at least how we use them in CRAS. > > Are there any examples of how to designate a PCM? I don't see any > modifiers at all in ucm-conf repo. I will second Curtis' request for clarifications. I naively thought that modifiers would be used to e.g. select a 'Deep Buffer' output for low-power playback, or different capture streams based on the needs of the applications. It's not uncommon for capture applications to request different PCM streams for raw, AEC processed, AEC+NS processed data. Echo reference is not really something that varies based on any qualifiers. And specifically in the Chrome case we want userspace to open the PCM echo reference device whenever the playback is used. So it's not really a use-case dependent thing but more a way to express a dependency between PCM devices.
I would triple this for the steamOS case, which has an inversion of this problem (a playback device that has a coupled capture device for speaker management). Coupling PCM devices statically in the UCM file avoids a class of side effects that could occur if one tried to implement the same behavior with exec calls in Enable/DisableSequence. On Mon, Aug 28, 2023 at 11:35 AM Pierre-Louis Bossart < pierre-louis.bossart@linux.intel.com> wrote: > > > On 8/28/23 12:59, Curtis Malainey wrote: > > On Sat, Aug 26, 2023 at 4:28 AM Jaroslav Kysela <perex@perex.cz> wrote: > >> > >> On 24. 08. 23 23:33, cujomalainey@chromium.org wrote: > >>> From: Curtis Malainey <cujomalainey@chromium.org> > >>> > >>> Sometimes userspace may want to use a reference channel to cancel echos > >>> when using video chat, this value identifies the device which carries > >>> that channel. > >> > >> The UCM modifier should be used for this - see "Echo Reference" > comments in > >> use-case.h. > >> > >> Note that this allows additional setup (in Sequences) for this stream. > >> > >> Jaroslav > > > > I was under the impression modifiers were state manipulators that > > acted upon existing devices/pcms and did not designate their own PCM. > > That is at least how we use them in CRAS. > > > > Are there any examples of how to designate a PCM? I don't see any > > modifiers at all in ucm-conf repo. > > I will second Curtis' request for clarifications. > > I naively thought that modifiers would be used to e.g. select a 'Deep > Buffer' output for low-power playback, or different capture streams > based on the needs of the applications. It's not uncommon for capture > applications to request different PCM streams for raw, AEC processed, > AEC+NS processed data. > > Echo reference is not really something that varies based on any > qualifiers. And specifically in the Chrome case we want userspace to > open the PCM echo reference device whenever the playback is used. So > it's not really a use-case dependent thing but more a way to express a > dependency between PCM devices. >
On 28. 08. 23 20:35, Pierre-Louis Bossart wrote: > > > On 8/28/23 12:59, Curtis Malainey wrote: >> On Sat, Aug 26, 2023 at 4:28 AM Jaroslav Kysela <perex@perex.cz> wrote: >>> >>> On 24. 08. 23 23:33, cujomalainey@chromium.org wrote: >>>> From: Curtis Malainey <cujomalainey@chromium.org> >>>> >>>> Sometimes userspace may want to use a reference channel to cancel echos >>>> when using video chat, this value identifies the device which carries >>>> that channel. >>> >>> The UCM modifier should be used for this - see "Echo Reference" comments in >>> use-case.h. >>> >>> Note that this allows additional setup (in Sequences) for this stream. >>> >>> Jaroslav >> >> I was under the impression modifiers were state manipulators that >> acted upon existing devices/pcms and did not designate their own PCM. >> That is at least how we use them in CRAS. >> >> Are there any examples of how to designate a PCM? I don't see any >> modifiers at all in ucm-conf repo. > > I will second Curtis' request for clarifications. > > I naively thought that modifiers would be used to e.g. select a 'Deep > Buffer' output for low-power playback, or different capture streams > based on the needs of the applications. It's not uncommon for capture > applications to request different PCM streams for raw, AEC processed, > AEC+NS processed data. > > Echo reference is not really something that varies based on any > qualifiers. And specifically in the Chrome case we want userspace to > open the PCM echo reference device whenever the playback is used. So > it's not really a use-case dependent thing but more a way to express a > dependency between PCM devices. It seems that there are some assumptions. I will try to address some things: You can enable/use multiple modifiers per device. The modifiers may define extra PCM streams in the standard Value section - you can use CapturePCM value for the modifier like "Echo Reference". Modifiers may alter the characteristics of the original UCM device stream (using command sequences), too. Modifiers are an extra layer on top of devices. I don't think that we have any default relation between the modifier PCM device and the original PCM device (from the UCM device definition). A new value to describe this (like "ReplacePlaybackPCM 1") may be introduced. Another issue is when multiple modifiers with this description are active - they conflict, so it should be described somewhere, too. Perhaps, "ConflictingModifier" array may be added to API. But those extensions are not required for the "Echo Reference" modifier. Jaroslav
> It seems that there are some assumptions. I will try to address some > things: > > You can enable/use multiple modifiers per device. > > The modifiers may define extra PCM streams in the standard Value section > - you can use CapturePCM value for the modifier like "Echo Reference". > Modifiers may alter the characteristics of the original UCM device > stream (using command sequences), too. Sorry, not following. The 'modifier' has to be selected by userspace, isn't it? "someone" has to tell UCM that the 'echo reference' is on. And that's precisely the conceptual issue I have. The echo reference in our cases is ALWAYS available when the speaker output is selected. In other words, we are asking userspace to select something that is always present and available. Or put differently, a modifier that's always applicable is not really a modifier, is it? And last, the whole story of the echo reference is that it needs to be opened when the speaker output is opened. How would we model this with the modifier concept? > Modifiers are an extra layer on top of devices. I don't think that we > have any default relation between the modifier PCM device and the > original PCM device (from the UCM device definition). A new value to > describe this (like "ReplacePlaybackPCM 1") may be introduced. Another > issue is when multiple modifiers with this description are active - they > conflict, so it should be described somewhere, too. Perhaps, > "ConflictingModifier" array may be added to API. But those extensions > are not required for the "Echo Reference" modifier. Right, the main issue here is that the PlaybackPCM and Echo reference PCM are joined and need to be handled at the same time. It's not a conflict, they are a bundle or a set of devices that cannot be used independently.
On 29. 08. 23 18:30, Pierre-Louis Bossart wrote: > > > >> It seems that there are some assumptions. I will try to address some >> things: >> >> You can enable/use multiple modifiers per device. >> >> The modifiers may define extra PCM streams in the standard Value section >> - you can use CapturePCM value for the modifier like "Echo Reference". >> Modifiers may alter the characteristics of the original UCM device >> stream (using command sequences), too. > > Sorry, not following. > > The 'modifier' has to be selected by userspace, isn't it? "someone" has > to tell UCM that the 'echo reference' is on. > > And that's precisely the conceptual issue I have. The echo reference in > our cases is ALWAYS available when the speaker output is selected. The function of modifier is selected by it's name, so an app should know, how to handle the "Echo Reference". And this use is optional. > In other words, we are asking userspace to select something that is > always present and available. Or put differently, a modifier that's > always applicable is not really a modifier, is it? Yes, in this special case, only joined PCM will be provided. But do not forget, that we have command sequences for modifiers, if we need to initialize something else like related controls for this stream in future. It's just more universal than to hardcode this to key/value in the UCM device definition. > And last, the whole story of the echo reference is that it needs to be > opened when the speaker output is opened. How would we model this with > the modifier concept? The modifiers are allowed to be activated only for the active devices. We can refine the use of the "Echo Reference" for applications and define that the playback PCM should be opened at first. If we need to alter this default behaviour in future, we may put a hint to configuration values. Jaroslav
On Tue, Aug 29, 2023 at 8:37 AM Jaroslav Kysela <perex@perex.cz> wrote: > > > The modifiers may define extra PCM streams in the standard Value section - you > can use CapturePCM value for the modifier like "Echo Reference". Modifiers may > alter the characteristics of the original UCM device stream (using command > sequences), too. Thanks for the clarification, that indeed is where the assumption I think for most of us is not as clear. I would have never thought a modifier would carry a PCM. I will trigger some work internally to convert our value to the modifier syntax. Curtis > > Modifiers are an extra layer on top of devices. I don't think that we have any > default relation between the modifier PCM device and the original PCM device > (from the UCM device definition). A new value to describe this (like > "ReplacePlaybackPCM 1") may be introduced. Another issue is when multiple > modifiers with this description are active - they conflict, so it should be > described somewhere, too. Perhaps, "ConflictingModifier" array may be added to > API. But those extensions are not required for the "Echo Reference" modifier. > > Jaroslav > > -- > Jaroslav Kysela <perex@perex.cz> > Linux Sound Maintainer; ALSA Project; Red Hat, Inc. >
On Tue, Aug 29, 2023 at 12:17 PM Curtis Malainey <cujomalainey@google.com> wrote: > > On Tue, Aug 29, 2023 at 8:37 AM Jaroslav Kysela <perex@perex.cz> wrote: > > > > Modifiers are an extra layer on top of devices. I don't think that we have any > > default relation between the modifier PCM device and the original PCM device > > (from the UCM device definition). A new value to describe this (like > > "ReplacePlaybackPCM 1") may be introduced. Another issue is when multiple > > modifiers with this description are active - they conflict, so it should be > > described somewhere, too. Perhaps, "ConflictingModifier" array may be added to > > API. But those extensions are not required for the "Echo Reference" modifier. What is the expectation for naming and relationship to the devices? Is the Modifier a suffix to the device name to associate it? Curtis
On 30. 08. 23 0:57, Curtis Malainey wrote: > On Tue, Aug 29, 2023 at 12:17 PM Curtis Malainey > <cujomalainey@google.com> wrote: >> >> On Tue, Aug 29, 2023 at 8:37 AM Jaroslav Kysela <perex@perex.cz> wrote: >>> >>> Modifiers are an extra layer on top of devices. I don't think that we have any >>> default relation between the modifier PCM device and the original PCM device >>> (from the UCM device definition). A new value to describe this (like >>> "ReplacePlaybackPCM 1") may be introduced. Another issue is when multiple >>> modifiers with this description are active - they conflict, so it should be >>> described somewhere, too. Perhaps, "ConflictingModifier" array may be added to >>> API. But those extensions are not required for the "Echo Reference" modifier. > > What is the expectation for naming and relationship to the devices? Is > the Modifier a suffix to the device name to associate it? Verbs, devices and modifiers should be named according use-case.h: https://www.alsa-project.org/alsa-doc/alsa-lib/group__ucm.html The relation is defined using SupportedDevices or ConflictingDevices array. Those arrays are available in modifiers, too. Jaroslav
diff --git a/include/use-case.h b/include/use-case.h index e5d4fd68..3c7c0a26 100644 --- a/include/use-case.h +++ b/include/use-case.h @@ -376,6 +376,15 @@ int snd_use_case_get_list(snd_use_case_mgr_t *uc_mgr, * - CaptureMicInfoFile * - json file with the microphone array placement and type description * (e.g. output from nhlt-dmic-info) + * - EchoReferenceDev + * - The name of a section device that provides some sort of reference + * to the current section device. This can be a echo taken as a + * reference or as IV data from a smart amp. The device should be + * opened immediately after this device and closed right before this + * device. This is meant to support hardware features such as smart + * amps or hardware offloaded processing where the host still needs + * to open the channel. The audio server should open the reference + * device with the same parameters it opens the source device with. * - EDIDFile * - Path to EDID file for HDMI devices * - JackCTL