| Message ID | 1705581128-4604-9-git-send-email-shengjiu.wang@nxp.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Add audio support in v4l2 framework | expand |
Em Thu, 18 Jan 2024 20:32:01 +0800 Shengjiu Wang <shengjiu.wang@nxp.com> escreveu: > The audio sample format definition is from alsa, > the header file is include/uapi/sound/asound.h, but > don't include this header file directly, because in > user space, there is another copy in alsa-lib. > There will be conflict in userspace for include > videodev2.h & asound.h and asoundlib.h > > Here still use the fourcc format. > > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > --- > .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ > .../userspace-api/media/v4l/pixfmt.rst | 1 + > drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ > include/uapi/linux/videodev2.h | 23 +++++ > 4 files changed, 124 insertions(+) > create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > new file mode 100644 > index 000000000000..04b4a7fbd8f4 > --- /dev/null > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > @@ -0,0 +1,87 @@ > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > + > +.. _pixfmt-audio: > + > +************* > +Audio Formats > +************* > + > +These formats are used for :ref:`audiomem2mem` interface only. > + > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| > + > +.. cssclass:: longtable > + > +.. flat-table:: Audio Format > + :header-rows: 1 > + :stub-columns: 0 > + :widths: 3 1 4 > + > + * - Identifier > + - Code > + - Details > + * .. _V4L2-AUDIO-FMT-S8: > + > + - ``V4L2_AUDIO_FMT_S8`` > + - 'S8' > + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA > + * .. _V4L2-AUDIO-FMT-S16-LE: Hmm... why can't we just use SNDRV_*_FORMAT_*? Those are already part of an uAPI header. No need to add any abstraction here and/or redefine what is there already at include/uapi/sound/asound.h. Thanks, Mauro
Hi Mauro On Sat, Feb 17, 2024 at 5:19 PM Mauro Carvalho Chehab <mchehab@kernel.org> wrote: > > Em Thu, 18 Jan 2024 20:32:01 +0800 > Shengjiu Wang <shengjiu.wang@nxp.com> escreveu: > > > The audio sample format definition is from alsa, > > the header file is include/uapi/sound/asound.h, but > > don't include this header file directly, because in > > user space, there is another copy in alsa-lib. > > There will be conflict in userspace for include > > videodev2.h & asound.h and asoundlib.h > > > > Here still use the fourcc format. > > > > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > > --- > > .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ > > .../userspace-api/media/v4l/pixfmt.rst | 1 + > > drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ > > include/uapi/linux/videodev2.h | 23 +++++ > > 4 files changed, 124 insertions(+) > > create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > new file mode 100644 > > index 000000000000..04b4a7fbd8f4 > > --- /dev/null > > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > @@ -0,0 +1,87 @@ > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > > + > > +.. _pixfmt-audio: > > + > > +************* > > +Audio Formats > > +************* > > + > > +These formats are used for :ref:`audiomem2mem` interface only. > > + > > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: Audio Format > > + :header-rows: 1 > > + :stub-columns: 0 > > + :widths: 3 1 4 > > + > > + * - Identifier > > + - Code > > + - Details > > + * .. _V4L2-AUDIO-FMT-S8: > > + > > + - ``V4L2_AUDIO_FMT_S8`` > > + - 'S8' > > + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA > > + * .. _V4L2-AUDIO-FMT-S16-LE: > > Hmm... why can't we just use SNDRV_*_FORMAT_*? Those are already part of > an uAPI header. No need to add any abstraction here and/or redefine > what is there already at include/uapi/sound/asound.h. > Actually I try to avoid including the include/uapi/sound/asound.h. Because in user space, there is another copy in alsa-lib (asoundlib.h). There will be conflict in userspace when including videodev2.h and asoundlib.h. And in the V4l framework, the fourcc type is commonly used in other cases (video, radio, touch, meta....), to avoid changing common code a lot, so I think using fourcc definition for audio may be simpler. Best regards Shengjiu Wang
Em Mon, 19 Feb 2024 12:05:02 +0800 Shengjiu Wang <shengjiu.wang@gmail.com> escreveu: > Hi Mauro > > On Sat, Feb 17, 2024 at 5:19 PM Mauro Carvalho Chehab > <mchehab@kernel.org> wrote: > > > > Em Thu, 18 Jan 2024 20:32:01 +0800 > > Shengjiu Wang <shengjiu.wang@nxp.com> escreveu: > > > > > The audio sample format definition is from alsa, > > > the header file is include/uapi/sound/asound.h, but > > > don't include this header file directly, because in > > > user space, there is another copy in alsa-lib. > > > There will be conflict in userspace for include > > > videodev2.h & asound.h and asoundlib.h > > > > > > Here still use the fourcc format. > > > > > > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > > > --- > > > .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ > > > .../userspace-api/media/v4l/pixfmt.rst | 1 + > > > drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ > > > include/uapi/linux/videodev2.h | 23 +++++ > > > 4 files changed, 124 insertions(+) > > > create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > > > > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > > new file mode 100644 > > > index 000000000000..04b4a7fbd8f4 > > > --- /dev/null > > > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > > @@ -0,0 +1,87 @@ > > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > > > + > > > +.. _pixfmt-audio: > > > + > > > +************* > > > +Audio Formats > > > +************* > > > + > > > +These formats are used for :ref:`audiomem2mem` interface only. > > > + > > > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| > > > + > > > +.. cssclass:: longtable > > > + > > > +.. flat-table:: Audio Format > > > + :header-rows: 1 > > > + :stub-columns: 0 > > > + :widths: 3 1 4 > > > + > > > + * - Identifier > > > + - Code > > > + - Details > > > + * .. _V4L2-AUDIO-FMT-S8: > > > + > > > + - ``V4L2_AUDIO_FMT_S8`` > > > + - 'S8' > > > + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA > > > + * .. _V4L2-AUDIO-FMT-S16-LE: > > > > Hmm... why can't we just use SNDRV_*_FORMAT_*? Those are already part of > > an uAPI header. No need to add any abstraction here and/or redefine > > what is there already at include/uapi/sound/asound.h. > > > Actually I try to avoid including the include/uapi/sound/asound.h. > Because in user space, there is another copy in alsa-lib (asoundlib.h). > There will be conflict in userspace when including videodev2.h and > asoundlib.h. Well, alsasoundlib.h seems to be using the same definitions: https://github.com/michaelwu/alsa-lib/blob/master/include/pcm.h So, I can't see what would be the actual issue, as both userspace library and ALSA internal headers use the same magic numbers. You can still do things like: #ifdef __KERNEL__ # include <sound/asound.h> #else # include <asoundlib.h> #endif To avoid such kind of conflicts, if you need to have it included on some header file. Yet, I can't see why you would need that. IMO, at uAPI headers, you just need to declare the uAPI audiofmt field to be either __u32 or __u64, pointing it to where this value comes from (on both userspace and Kernelspace. E. g.: /** * struct v4l2_audio_format - audio data format definition * @audioformat: * an integer number matching the fields inside * enum snd_pcm_format_t (e. g. `SNDRV_PCM_FORMAT_*`), as defined * in include/uapi/sound/asound.h and * https://www.alsa-project.org/alsa-doc/alsa-lib/group___p_c_m.html#gaa14b7f26877a812acbb39811364177f8. * @channels: channel numbers * @buffersize: maximum size in bytes required for data */ struct v4l2_audio_format { __u32 audioformat; __u32 channels; __u32 buffersize; } __attribute__ ((packed)); Then, at documentation you just need to point to where the possible values for SNDRV_PCM_FORMAT_ are defined. No need to document them one by one. With such definition, you'll only need to include sound/asound.h within the kAPI scope. > > And in the V4l framework, the fourcc type is commonly used in other > cases (video, radio, touch, meta....), to avoid changing common code > a lot, so I think using fourcc definition for audio may be simpler. Those are real video streams (or a video-related streams, in the case of metadata) where fourcc is widely used. There, it makes sense. However, ALSA format definitions are already being used for a long time. There's no sense on trying to reinvent it - or having an abstract layer to convert from/to fourcc <==> enum snd_pcm_format_t. Just use what is there already. Thanks, Mauro
On 19/02/2024 13:56, Mauro Carvalho Chehab wrote: > Em Mon, 19 Feb 2024 12:05:02 +0800 > Shengjiu Wang <shengjiu.wang@gmail.com> escreveu: > >> Hi Mauro >> >> On Sat, Feb 17, 2024 at 5:19 PM Mauro Carvalho Chehab >> <mchehab@kernel.org> wrote: >>> >>> Em Thu, 18 Jan 2024 20:32:01 +0800 >>> Shengjiu Wang <shengjiu.wang@nxp.com> escreveu: >>> >>>> The audio sample format definition is from alsa, >>>> the header file is include/uapi/sound/asound.h, but >>>> don't include this header file directly, because in >>>> user space, there is another copy in alsa-lib. >>>> There will be conflict in userspace for include >>>> videodev2.h & asound.h and asoundlib.h >>>> >>>> Here still use the fourcc format. >>>> >>>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> >>>> --- >>>> .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ >>>> .../userspace-api/media/v4l/pixfmt.rst | 1 + >>>> drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ >>>> include/uapi/linux/videodev2.h | 23 +++++ >>>> 4 files changed, 124 insertions(+) >>>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>> >>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>> new file mode 100644 >>>> index 000000000000..04b4a7fbd8f4 >>>> --- /dev/null >>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>> @@ -0,0 +1,87 @@ >>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later >>>> + >>>> +.. _pixfmt-audio: >>>> + >>>> +************* >>>> +Audio Formats >>>> +************* >>>> + >>>> +These formats are used for :ref:`audiomem2mem` interface only. >>>> + >>>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| >>>> + >>>> +.. cssclass:: longtable >>>> + >>>> +.. flat-table:: Audio Format >>>> + :header-rows: 1 >>>> + :stub-columns: 0 >>>> + :widths: 3 1 4 >>>> + >>>> + * - Identifier >>>> + - Code >>>> + - Details >>>> + * .. _V4L2-AUDIO-FMT-S8: >>>> + >>>> + - ``V4L2_AUDIO_FMT_S8`` >>>> + - 'S8' >>>> + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA >>>> + * .. _V4L2-AUDIO-FMT-S16-LE: >>> >>> Hmm... why can't we just use SNDRV_*_FORMAT_*? Those are already part of >>> an uAPI header. No need to add any abstraction here and/or redefine >>> what is there already at include/uapi/sound/asound.h. >>> >> Actually I try to avoid including the include/uapi/sound/asound.h. >> Because in user space, there is another copy in alsa-lib (asoundlib.h). >> There will be conflict in userspace when including videodev2.h and >> asoundlib.h. > > Well, alsasoundlib.h seems to be using the same definitions: > https://github.com/michaelwu/alsa-lib/blob/master/include/pcm.h > > So, I can't see what would be the actual issue, as both userspace library > and ALSA internal headers use the same magic numbers. > > You can still do things like: > > #ifdef __KERNEL__ > # include <sound/asound.h> > #else > # include <asoundlib.h> > #endif > > To avoid such kind of conflicts, if you need to have it included on > some header file. Yet, I can't see why you would need that. > > IMO, at uAPI headers, you just need to declare the uAPI audiofmt field > to be either __u32 or __u64, pointing it to where this value comes from > (on both userspace and Kernelspace. E. g.: > > /** > * struct v4l2_audio_format - audio data format definition > * @audioformat: > * an integer number matching the fields inside > * enum snd_pcm_format_t (e. g. `SNDRV_PCM_FORMAT_*`), as defined > * in include/uapi/sound/asound.h and > * https://www.alsa-project.org/alsa-doc/alsa-lib/group___p_c_m.html#gaa14b7f26877a812acbb39811364177f8. > * @channels: channel numbers > * @buffersize: maximum size in bytes required for data > */ > struct v4l2_audio_format { > __u32 audioformat; > __u32 channels; > __u32 buffersize; > } __attribute__ ((packed)); > > Then, at documentation you just need to point to where the > possible values for SNDRV_PCM_FORMAT_ are defined. No need to > document them one by one. > > With such definition, you'll only need to include sound/asound.h > within the kAPI scope. > >> >> And in the V4l framework, the fourcc type is commonly used in other >> cases (video, radio, touch, meta....), to avoid changing common code >> a lot, so I think using fourcc definition for audio may be simpler. > > Those are real video streams (or a video-related streams, in the case > of metadata) where fourcc is widely used. There, it makes sense. > However, ALSA format definitions are already being used for a long time. > There's no sense on trying to reinvent it - or having an abstract layer > to convert from/to fourcc <==> enum snd_pcm_format_t. Just use what is > there already. The problem is that within V4L2 we use fourcc consistently to describe a format, including in VIDIOC_ENUM_FMT. And the expectation is that the fourcc can be printed to a human readable string (there is even a printk format for that these days). But the pcm values are all small integers (and can even be 0!), and printing the fourcc will give garbage. It doesn't work well at all with the V4L2 API. But by having a straightforward conversion between the pcm identifier and a fourcc it was really easy to deal with this. There might even be applications today that call VIDIOC_ENUM_FMT to see what is supported and fail if it is not a proper fourcc is returned. It will certainly report nonsense in v4l_print_fmtdesc() (v4l2-ioctl.c). One of the early versions of this patch series did precisely what you request, but it just doesn't work well within the V4L2 uAPI. Regards, Hans > > Thanks, > Mauro
On Wed, Feb 21, 2024 at 7:10 PM Hans Verkuil <hverkuil@xs4all.nl> wrote: > > On 19/02/2024 13:56, Mauro Carvalho Chehab wrote: > > Em Mon, 19 Feb 2024 12:05:02 +0800 > > Shengjiu Wang <shengjiu.wang@gmail.com> escreveu: > > > >> Hi Mauro > >> > >> On Sat, Feb 17, 2024 at 5:19 PM Mauro Carvalho Chehab > >> <mchehab@kernel.org> wrote: > >>> > >>> Em Thu, 18 Jan 2024 20:32:01 +0800 > >>> Shengjiu Wang <shengjiu.wang@nxp.com> escreveu: > >>> > >>>> The audio sample format definition is from alsa, > >>>> the header file is include/uapi/sound/asound.h, but > >>>> don't include this header file directly, because in > >>>> user space, there is another copy in alsa-lib. > >>>> There will be conflict in userspace for include > >>>> videodev2.h & asound.h and asoundlib.h > >>>> > >>>> Here still use the fourcc format. > >>>> > >>>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > >>>> --- > >>>> .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ > >>>> .../userspace-api/media/v4l/pixfmt.rst | 1 + > >>>> drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ > >>>> include/uapi/linux/videodev2.h | 23 +++++ > >>>> 4 files changed, 124 insertions(+) > >>>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst > >>>> > >>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > >>>> new file mode 100644 > >>>> index 000000000000..04b4a7fbd8f4 > >>>> --- /dev/null > >>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > >>>> @@ -0,0 +1,87 @@ > >>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > >>>> + > >>>> +.. _pixfmt-audio: > >>>> + > >>>> +************* > >>>> +Audio Formats > >>>> +************* > >>>> + > >>>> +These formats are used for :ref:`audiomem2mem` interface only. > >>>> + > >>>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| > >>>> + > >>>> +.. cssclass:: longtable > >>>> + > >>>> +.. flat-table:: Audio Format > >>>> + :header-rows: 1 > >>>> + :stub-columns: 0 > >>>> + :widths: 3 1 4 > >>>> + > >>>> + * - Identifier > >>>> + - Code > >>>> + - Details > >>>> + * .. _V4L2-AUDIO-FMT-S8: > >>>> + > >>>> + - ``V4L2_AUDIO_FMT_S8`` > >>>> + - 'S8' > >>>> + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA > >>>> + * .. _V4L2-AUDIO-FMT-S16-LE: > >>> > >>> Hmm... why can't we just use SNDRV_*_FORMAT_*? Those are already part of > >>> an uAPI header. No need to add any abstraction here and/or redefine > >>> what is there already at include/uapi/sound/asound.h. > >>> > >> Actually I try to avoid including the include/uapi/sound/asound.h. > >> Because in user space, there is another copy in alsa-lib (asoundlib.h). > >> There will be conflict in userspace when including videodev2.h and > >> asoundlib.h. > > > > Well, alsasoundlib.h seems to be using the same definitions: > > https://github.com/michaelwu/alsa-lib/blob/master/include/pcm.h > > > > So, I can't see what would be the actual issue, as both userspace library > > and ALSA internal headers use the same magic numbers. > > > > You can still do things like: > > > > #ifdef __KERNEL__ > > # include <sound/asound.h> > > #else > > # include <asoundlib.h> > > #endif > > > > To avoid such kind of conflicts, if you need to have it included on > > some header file. Yet, I can't see why you would need that. > > > > IMO, at uAPI headers, you just need to declare the uAPI audiofmt field > > to be either __u32 or __u64, pointing it to where this value comes from > > (on both userspace and Kernelspace. E. g.: > > > > /** > > * struct v4l2_audio_format - audio data format definition > > * @audioformat: > > * an integer number matching the fields inside > > * enum snd_pcm_format_t (e. g. `SNDRV_PCM_FORMAT_*`), as defined > > * in include/uapi/sound/asound.h and > > * https://www.alsa-project.org/alsa-doc/alsa-lib/group___p_c_m.html#gaa14b7f26877a812acbb39811364177f8. > > * @channels: channel numbers > > * @buffersize: maximum size in bytes required for data > > */ > > struct v4l2_audio_format { > > __u32 audioformat; > > __u32 channels; > > __u32 buffersize; > > } __attribute__ ((packed)); > > > > Then, at documentation you just need to point to where the > > possible values for SNDRV_PCM_FORMAT_ are defined. No need to > > document them one by one. > > > > With such definition, you'll only need to include sound/asound.h > > within the kAPI scope. > > > >> > >> And in the V4l framework, the fourcc type is commonly used in other > >> cases (video, radio, touch, meta....), to avoid changing common code > >> a lot, so I think using fourcc definition for audio may be simpler. > > > > Those are real video streams (or a video-related streams, in the case > > of metadata) where fourcc is widely used. There, it makes sense. > > However, ALSA format definitions are already being used for a long time. > > There's no sense on trying to reinvent it - or having an abstract layer > > to convert from/to fourcc <==> enum snd_pcm_format_t. Just use what is > > there already. > > The problem is that within V4L2 we use fourcc consistently to describe a > format, including in VIDIOC_ENUM_FMT. And the expectation is that the fourcc > can be printed to a human readable string (there is even a printk format for > that these days). > > But the pcm values are all small integers (and can even be 0!), and > printing the fourcc will give garbage. It doesn't work well at all > with the V4L2 API. But by having a straightforward conversion between the > pcm identifier and a fourcc it was really easy to deal with this. > > There might even be applications today that call VIDIOC_ENUM_FMT to see > what is supported and fail if it is not a proper fourcc is returned. > > It will certainly report nonsense in v4l_print_fmtdesc() (v4l2-ioctl.c). > > One of the early versions of this patch series did precisely what you request, > but it just doesn't work well within the V4L2 uAPI. > Thanks all. So can I reach the conclusion below? 1. Keep using the fourcc definition for v4l2_audio_format.audioformat. 2. Keep the name V4L2_BUF_TYPE_AUDIO_CAPTURE and V4L2_BUF_TYPE_AUDIO_OUTPUT. 3. Fix Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst to change 'pixelformat' to 'audioformat'. 4. Update Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst to add more description for clock drift. Best Regards Shengjiu Wang > Regards, > > Hans > > > > > Thanks, > > Mauro >
On 22/02/2024 04:50, Shengjiu Wang wrote: > On Wed, Feb 21, 2024 at 7:10 PM Hans Verkuil <hverkuil@xs4all.nl> wrote: >> >> On 19/02/2024 13:56, Mauro Carvalho Chehab wrote: >>> Em Mon, 19 Feb 2024 12:05:02 +0800 >>> Shengjiu Wang <shengjiu.wang@gmail.com> escreveu: >>> >>>> Hi Mauro >>>> >>>> On Sat, Feb 17, 2024 at 5:19 PM Mauro Carvalho Chehab >>>> <mchehab@kernel.org> wrote: >>>>> >>>>> Em Thu, 18 Jan 2024 20:32:01 +0800 >>>>> Shengjiu Wang <shengjiu.wang@nxp.com> escreveu: >>>>> >>>>>> The audio sample format definition is from alsa, >>>>>> the header file is include/uapi/sound/asound.h, but >>>>>> don't include this header file directly, because in >>>>>> user space, there is another copy in alsa-lib. >>>>>> There will be conflict in userspace for include >>>>>> videodev2.h & asound.h and asoundlib.h >>>>>> >>>>>> Here still use the fourcc format. >>>>>> >>>>>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> >>>>>> --- >>>>>> .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ >>>>>> .../userspace-api/media/v4l/pixfmt.rst | 1 + >>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ >>>>>> include/uapi/linux/videodev2.h | 23 +++++ >>>>>> 4 files changed, 124 insertions(+) >>>>>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>>>> >>>>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>>>> new file mode 100644 >>>>>> index 000000000000..04b4a7fbd8f4 >>>>>> --- /dev/null >>>>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>>>> @@ -0,0 +1,87 @@ >>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later >>>>>> + >>>>>> +.. _pixfmt-audio: >>>>>> + >>>>>> +************* >>>>>> +Audio Formats >>>>>> +************* >>>>>> + >>>>>> +These formats are used for :ref:`audiomem2mem` interface only. >>>>>> + >>>>>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| >>>>>> + >>>>>> +.. cssclass:: longtable >>>>>> + >>>>>> +.. flat-table:: Audio Format >>>>>> + :header-rows: 1 >>>>>> + :stub-columns: 0 >>>>>> + :widths: 3 1 4 >>>>>> + >>>>>> + * - Identifier >>>>>> + - Code >>>>>> + - Details >>>>>> + * .. _V4L2-AUDIO-FMT-S8: >>>>>> + >>>>>> + - ``V4L2_AUDIO_FMT_S8`` >>>>>> + - 'S8' >>>>>> + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA >>>>>> + * .. _V4L2-AUDIO-FMT-S16-LE: >>>>> >>>>> Hmm... why can't we just use SNDRV_*_FORMAT_*? Those are already part of >>>>> an uAPI header. No need to add any abstraction here and/or redefine >>>>> what is there already at include/uapi/sound/asound.h. >>>>> >>>> Actually I try to avoid including the include/uapi/sound/asound.h. >>>> Because in user space, there is another copy in alsa-lib (asoundlib.h). >>>> There will be conflict in userspace when including videodev2.h and >>>> asoundlib.h. >>> >>> Well, alsasoundlib.h seems to be using the same definitions: >>> https://github.com/michaelwu/alsa-lib/blob/master/include/pcm.h >>> >>> So, I can't see what would be the actual issue, as both userspace library >>> and ALSA internal headers use the same magic numbers. >>> >>> You can still do things like: >>> >>> #ifdef __KERNEL__ >>> # include <sound/asound.h> >>> #else >>> # include <asoundlib.h> >>> #endif >>> >>> To avoid such kind of conflicts, if you need to have it included on >>> some header file. Yet, I can't see why you would need that. >>> >>> IMO, at uAPI headers, you just need to declare the uAPI audiofmt field >>> to be either __u32 or __u64, pointing it to where this value comes from >>> (on both userspace and Kernelspace. E. g.: >>> >>> /** >>> * struct v4l2_audio_format - audio data format definition >>> * @audioformat: >>> * an integer number matching the fields inside >>> * enum snd_pcm_format_t (e. g. `SNDRV_PCM_FORMAT_*`), as defined >>> * in include/uapi/sound/asound.h and >>> * https://www.alsa-project.org/alsa-doc/alsa-lib/group___p_c_m.html#gaa14b7f26877a812acbb39811364177f8. >>> * @channels: channel numbers >>> * @buffersize: maximum size in bytes required for data >>> */ >>> struct v4l2_audio_format { >>> __u32 audioformat; >>> __u32 channels; >>> __u32 buffersize; >>> } __attribute__ ((packed)); >>> >>> Then, at documentation you just need to point to where the >>> possible values for SNDRV_PCM_FORMAT_ are defined. No need to >>> document them one by one. >>> >>> With such definition, you'll only need to include sound/asound.h >>> within the kAPI scope. >>> >>>> >>>> And in the V4l framework, the fourcc type is commonly used in other >>>> cases (video, radio, touch, meta....), to avoid changing common code >>>> a lot, so I think using fourcc definition for audio may be simpler. >>> >>> Those are real video streams (or a video-related streams, in the case >>> of metadata) where fourcc is widely used. There, it makes sense. >>> However, ALSA format definitions are already being used for a long time. >>> There's no sense on trying to reinvent it - or having an abstract layer >>> to convert from/to fourcc <==> enum snd_pcm_format_t. Just use what is >>> there already. >> >> The problem is that within V4L2 we use fourcc consistently to describe a >> format, including in VIDIOC_ENUM_FMT. And the expectation is that the fourcc >> can be printed to a human readable string (there is even a printk format for >> that these days). >> >> But the pcm values are all small integers (and can even be 0!), and >> printing the fourcc will give garbage. It doesn't work well at all >> with the V4L2 API. But by having a straightforward conversion between the >> pcm identifier and a fourcc it was really easy to deal with this. >> >> There might even be applications today that call VIDIOC_ENUM_FMT to see >> what is supported and fail if it is not a proper fourcc is returned. >> >> It will certainly report nonsense in v4l_print_fmtdesc() (v4l2-ioctl.c). >> >> One of the early versions of this patch series did precisely what you request, >> but it just doesn't work well within the V4L2 uAPI. >> > > Thanks all. > > So can I reach the conclusion below? > 1. Keep using the fourcc definition for v4l2_audio_format.audioformat. > 2. Keep the name V4L2_BUF_TYPE_AUDIO_CAPTURE and > V4L2_BUF_TYPE_AUDIO_OUTPUT. > 3. Fix Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst > to change 'pixelformat' to 'audioformat'. > 4. Update Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > to add more description for clock drift. Yes, let's go with that. Regards, Hans > > Best Regards > Shengjiu Wang > >> Regards, >> >> Hans >> >>> >>> Thanks, >>> Mauro >>
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst new file mode 100644 index 000000000000..04b4a7fbd8f4 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst @@ -0,0 +1,87 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _pixfmt-audio: + +************* +Audio Formats +************* + +These formats are used for :ref:`audiomem2mem` interface only. + +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| + +.. cssclass:: longtable + +.. flat-table:: Audio Format + :header-rows: 1 + :stub-columns: 0 + :widths: 3 1 4 + + * - Identifier + - Code + - Details + * .. _V4L2-AUDIO-FMT-S8: + + - ``V4L2_AUDIO_FMT_S8`` + - 'S8' + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA + * .. _V4L2-AUDIO-FMT-S16-LE: + + - ``V4L2_AUDIO_FMT_S16_LE`` + - 'S16_LE' + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA + * .. _V4L2-AUDIO-FMT-U16-LE: + + - ``V4L2_AUDIO_FMT_U16_LE`` + - 'U16_LE' + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA + * .. _V4L2-AUDIO-FMT-S24-LE: + + - ``V4L2_AUDIO_FMT_S24_LE`` + - 'S24_LE' + - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA + * .. _V4L2-AUDIO-FMT-U24-LE: + + - ``V4L2_AUDIO_FMT_U24_LE`` + - 'U24_LE' + - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA + * .. _V4L2-AUDIO-FMT-S32-LE: + + - ``V4L2_AUDIO_FMT_S32_LE`` + - 'S32_LE' + - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA + * .. _V4L2-AUDIO-FMT-U32-LE: + + - ``V4L2_AUDIO_FMT_U32_LE`` + - 'U32_LE' + - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA + * .. _V4L2-AUDIO-FMT-FLOAT-LE: + + - ``V4L2_AUDIO_FMT_FLOAT_LE`` + - 'FLOAT_LE' + - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA + * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE: + + - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE`` + - 'IEC958_SUBFRAME_LE' + - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA + * .. _V4L2-AUDIO-FMT-S24-3LE: + + - ``V4L2_AUDIO_FMT_S24_3LE`` + - 'S24_3LE' + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA + * .. _V4L2-AUDIO-FMT-U24-3LE: + + - ``V4L2_AUDIO_FMT_U24_3LE`` + - 'U24_3LE' + - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA + * .. _V4L2-AUDIO-FMT-S20-3LE: + + - ``V4L2_AUDIO_FMT_S20_3LE`` + - 'S20_3LE' + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA + * .. _V4L2-AUDIO-FMT-U20-3LE: + + - ``V4L2_AUDIO_FMT_U20_3LE`` + - 'U20_3LE' + - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst index 11dab4a90630..2eb6fdd3b43d 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.) colorspaces colorspaces-defs colorspaces-details + pixfmt-audio diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index e7be7c2f302d..e5094f0e75c1 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; + case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break; + case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit Signed LE"; break; + case V4L2_AUDIO_FMT_U16_LE: descr = "16-bit Unsigned LE"; break; + case V4L2_AUDIO_FMT_S24_LE: descr = "24(32)-bit Signed LE"; break; + case V4L2_AUDIO_FMT_U24_LE: descr = "24(32)-bit Unsigned LE"; break; + case V4L2_AUDIO_FMT_S32_LE: descr = "32-bit Signed LE"; break; + case V4L2_AUDIO_FMT_U32_LE: descr = "32-bit Unsigned LE"; break; + case V4L2_AUDIO_FMT_FLOAT_LE: descr = "32-bit Float LE"; break; + case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break; + case V4L2_AUDIO_FMT_S24_3LE: descr = "24(24)-bit Signed LE"; break; + case V4L2_AUDIO_FMT_U24_3LE: descr = "24(24)-bit Unsigned LE"; break; + case V4L2_AUDIO_FMT_S20_3LE: descr = "20(24)-bit Signed LE"; break; + case V4L2_AUDIO_FMT_U20_3LE: descr = "20(24)-bit Unsigned LE"; break; default: /* Compressed formats */ diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 57c82eb158f1..7d0b33e7d854 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -843,6 +843,29 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ +/* + * Audio-data formats + * All these audio formats use a fourcc starting with 'AU' + * followed by the SNDRV_PCM_FORMAT_ value from asound.h. + */ +#define V4L2_AUDIO_FMT_S8 v4l2_fourcc('A', 'U', '0', '0') +#define V4L2_AUDIO_FMT_S16_LE v4l2_fourcc('A', 'U', '0', '2') +#define V4L2_AUDIO_FMT_U16_LE v4l2_fourcc('A', 'U', '0', '4') +#define V4L2_AUDIO_FMT_S24_LE v4l2_fourcc('A', 'U', '0', '6') +#define V4L2_AUDIO_FMT_U24_LE v4l2_fourcc('A', 'U', '0', '8') +#define V4L2_AUDIO_FMT_S32_LE v4l2_fourcc('A', 'U', '1', '0') +#define V4L2_AUDIO_FMT_U32_LE v4l2_fourcc('A', 'U', '1', '2') +#define V4L2_AUDIO_FMT_FLOAT_LE v4l2_fourcc('A', 'U', '1', '4') +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE v4l2_fourcc('A', 'U', '1', '8') +#define V4L2_AUDIO_FMT_S24_3LE v4l2_fourcc('A', 'U', '3', '2') +#define V4L2_AUDIO_FMT_U24_3LE v4l2_fourcc('A', 'U', '3', '4') +#define V4L2_AUDIO_FMT_S20_3LE v4l2_fourcc('A', 'U', '3', '6') +#define V4L2_AUDIO_FMT_U20_3LE v4l2_fourcc('A', 'U', '3', '8') + +#define v4l2_fourcc_to_audfmt(fourcc) \ + (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10 \ + + ((((fourcc) >> 24) & 0xff) - '0')) + /* priv field value to indicates that subsequent fields are valid. */ #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
The audio sample format definition is from alsa, the header file is include/uapi/sound/asound.h, but don't include this header file directly, because in user space, there is another copy in alsa-lib. There will be conflict in userspace for include videodev2.h & asound.h and asoundlib.h Here still use the fourcc format. Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> --- .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ .../userspace-api/media/v4l/pixfmt.rst | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ include/uapi/linux/videodev2.h | 23 +++++ 4 files changed, 124 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst