| Message ID | 86a39343d33f0f75079407d8b36202a1de4c58de.1347023744.git.hans.verkuil@cisco.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 09/07/2012 03:29 PM, Hans Verkuil wrote: > From: Hans Verkuil<hans.verkuil@cisco.com> > > Add a new flag that tells userspace that the monotonic clock is used > for timestamps and update the documentation accordingly. > > We decided on this new flag during the 2012 Media Workshop. > > Signed-off-by: Hans Verkuil<hans.verkuil@cisco.com> Reviewed-by: Sylwester Nawrocki <s.nawrocki@samsung.com> -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Friday 07 September 2012 15:29:12 Hans Verkuil wrote: > From: Hans Verkuil <hans.verkuil@cisco.com> > > Add a new flag that tells userspace that the monotonic clock is used > for timestamps and update the documentation accordingly. > > We decided on this new flag during the 2012 Media Workshop. > > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Hi Hans, Thanks for the patch! On Fri, Sep 07, 2012 at 03:29:12PM +0200, Hans Verkuil wrote: > From: Hans Verkuil <hans.verkuil@cisco.com> > > Add a new flag that tells userspace that the monotonic clock is used > for timestamps and update the documentation accordingly. > > We decided on this new flag during the 2012 Media Workshop. > > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> > --- > Documentation/DocBook/media/v4l/io.xml | 10 +++++++--- > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 3 ++- > Documentation/DocBook/media/v4l/vidioc-querycap.xml | 7 +++++++ > include/linux/videodev2.h | 1 + > 4 files changed, 17 insertions(+), 4 deletions(-) > > diff --git a/Documentation/DocBook/media/v4l/io.xml b/Documentation/DocBook/media/v4l/io.xml > index 2dc39d8..b680d66 100644 > --- a/Documentation/DocBook/media/v4l/io.xml > +++ b/Documentation/DocBook/media/v4l/io.xml > @@ -582,10 +582,14 @@ applications when an output stream.</entry> > <entry>struct timeval</entry> > <entry><structfield>timestamp</structfield></entry> > <entry></entry> > - <entry><para>For input streams this is the > + <entry><para>This is either the > system time (as returned by the <function>gettimeofday()</function> > -function) when the first data byte was captured. For output streams > -the data will not be displayed before this time, secondary to the > +function) or a monotonic timestamp (as returned by the > +<function>clock_gettime(CLOCK_MONOTONIC, &ts)</function> function). > +A monotonic timestamp is used if the <constant>V4L2_CAP_MONOTONIC_TS</constant> > +capability is set, otherwise the system time is used. > +For input streams this is the timestamp when the first data byte was captured. > +For output streams the data will not be displayed before this time, secondary to the I have an alternative proposal. The type of the desired timestamps depend on the use case, not the driver used to capture the buffers. Thus we could also give the choice to the user by means of e.g. a control. If we'd make it configurable (applications would still get the wallclock time unless they ask for monotonic time), we'd have a chance to add more precision by using struct timespec (ns precision) instead of struct timeval (us precision). struct timespec is also used by V4L2 events. Adding support for CLOCK_MONOTONIC_RAW would also be a non-issue. Additional helper function could be used to generate the timestamp of the desired type. What do you think? Cheers,
Hi Sakari, On Thursday 13 September 2012 23:38:14 Sakari Ailus wrote: > On Fri, Sep 07, 2012 at 03:29:12PM +0200, Hans Verkuil wrote: > > From: Hans Verkuil <hans.verkuil@cisco.com> > > > > Add a new flag that tells userspace that the monotonic clock is used > > for timestamps and update the documentation accordingly. > > > > We decided on this new flag during the 2012 Media Workshop. > > > > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> > > --- > > > > Documentation/DocBook/media/v4l/io.xml | 10 +++++++--- > > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 3 ++- > > Documentation/DocBook/media/v4l/vidioc-querycap.xml | 7 +++++++ > > include/linux/videodev2.h | 1 + > > 4 files changed, 17 insertions(+), 4 deletions(-) > > > > diff --git a/Documentation/DocBook/media/v4l/io.xml > > b/Documentation/DocBook/media/v4l/io.xml index 2dc39d8..b680d66 100644 > > --- a/Documentation/DocBook/media/v4l/io.xml > > +++ b/Documentation/DocBook/media/v4l/io.xml > > @@ -582,10 +582,14 @@ applications when an output stream.</entry> > > > > <entry>struct timeval</entry> > > <entry><structfield>timestamp</structfield></entry> > > <entry></entry> > > > > - <entry><para>For input streams this is the > > + <entry><para>This is either the > > > > system time (as returned by the <function>gettimeofday()</function> > > > > -function) when the first data byte was captured. For output streams > > -the data will not be displayed before this time, secondary to the > > +function) or a monotonic timestamp (as returned by the > > +<function>clock_gettime(CLOCK_MONOTONIC, &ts)</function> function). > > +A monotonic timestamp is used if the > > <constant>V4L2_CAP_MONOTONIC_TS</constant> +capability is set, otherwise > > the system time is used. > > +For input streams this is the timestamp when the first data byte was > > captured. +For output streams the data will not be displayed before this > > time, secondary to the > I have an alternative proposal. > > The type of the desired timestamps depend on the use case, not the driver > used to capture the buffers. Thus we could also give the choice to the user > by means of e.g. a control. Or a buffer flag. I will need something similar to select device-specific timestamps. However, for wall clock vs. monotonic clock, I don't think there's a reason to let applications decide to use the wall clock. It would be a broken use case. I don't think we should let applications decide in this case. On the other hand, reporting a timespec instead of a timeval would be a good idea. I'm tempted. > If we'd make it configurable (applications would still get the wallclock > time unless they ask for monotonic time), we'd have a chance to add more > precision by using struct timespec (ns precision) instead of struct timeval > (us precision). struct timespec is also used by V4L2 events. > > Adding support for CLOCK_MONOTONIC_RAW would also be a non-issue. > > Additional helper function could be used to generate the timestamp of the > desired type. > > What do you think?
On Thu September 13 2012 22:50:32 Laurent Pinchart wrote: > Hi Sakari, > > On Thursday 13 September 2012 23:38:14 Sakari Ailus wrote: > > On Fri, Sep 07, 2012 at 03:29:12PM +0200, Hans Verkuil wrote: > > > From: Hans Verkuil <hans.verkuil@cisco.com> > > > > > > Add a new flag that tells userspace that the monotonic clock is used > > > for timestamps and update the documentation accordingly. > > > > > > We decided on this new flag during the 2012 Media Workshop. > > > > > > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> > > > --- > > > > > > Documentation/DocBook/media/v4l/io.xml | 10 +++++++--- > > > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 3 ++- > > > Documentation/DocBook/media/v4l/vidioc-querycap.xml | 7 +++++++ > > > include/linux/videodev2.h | 1 + > > > 4 files changed, 17 insertions(+), 4 deletions(-) > > > > > > diff --git a/Documentation/DocBook/media/v4l/io.xml > > > b/Documentation/DocBook/media/v4l/io.xml index 2dc39d8..b680d66 100644 > > > --- a/Documentation/DocBook/media/v4l/io.xml > > > +++ b/Documentation/DocBook/media/v4l/io.xml > > > @@ -582,10 +582,14 @@ applications when an output stream.</entry> > > > > > > <entry>struct timeval</entry> > > > <entry><structfield>timestamp</structfield></entry> > > > <entry></entry> > > > > > > - <entry><para>For input streams this is the > > > + <entry><para>This is either the > > > > > > system time (as returned by the <function>gettimeofday()</function> > > > > > > -function) when the first data byte was captured. For output streams > > > -the data will not be displayed before this time, secondary to the > > > +function) or a monotonic timestamp (as returned by the > > > +<function>clock_gettime(CLOCK_MONOTONIC, &ts)</function> function). > > > +A monotonic timestamp is used if the > > > <constant>V4L2_CAP_MONOTONIC_TS</constant> +capability is set, otherwise > > > the system time is used. > > > +For input streams this is the timestamp when the first data byte was > > > captured. +For output streams the data will not be displayed before this > > > time, secondary to the > > I have an alternative proposal. > > > > The type of the desired timestamps depend on the use case, not the driver > > used to capture the buffers. Thus we could also give the choice to the user > > by means of e.g. a control. > > Or a buffer flag. I will need something similar to select device-specific > timestamps. > > However, for wall clock vs. monotonic clock, I don't think there's a reason to > let applications decide to use the wall clock. It would be a broken use case. > I don't think we should let applications decide in this case. I agree. > On the other hand, reporting a timespec instead of a timeval would be a good > idea. I'm tempted. Microsecond precision seems more than sufficient to me for video frames. I see no good reason for messing around with the v4l2_buffer struct just to get a timespec in. Regards, Hans -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Sep 13, 2012 at 10:56:41PM +0200, Hans Verkuil wrote: > On Thu September 13 2012 22:50:32 Laurent Pinchart wrote: > > Hi Sakari, > > > > On Thursday 13 September 2012 23:38:14 Sakari Ailus wrote: > > > On Fri, Sep 07, 2012 at 03:29:12PM +0200, Hans Verkuil wrote: > > > > From: Hans Verkuil <hans.verkuil@cisco.com> > > > > > > > > Add a new flag that tells userspace that the monotonic clock is used > > > > for timestamps and update the documentation accordingly. > > > > > > > > We decided on this new flag during the 2012 Media Workshop. > > > > > > > > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> > > > > --- > > > > > > > > Documentation/DocBook/media/v4l/io.xml | 10 +++++++--- > > > > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 3 ++- > > > > Documentation/DocBook/media/v4l/vidioc-querycap.xml | 7 +++++++ > > > > include/linux/videodev2.h | 1 + > > > > 4 files changed, 17 insertions(+), 4 deletions(-) > > > > > > > > diff --git a/Documentation/DocBook/media/v4l/io.xml > > > > b/Documentation/DocBook/media/v4l/io.xml index 2dc39d8..b680d66 100644 > > > > --- a/Documentation/DocBook/media/v4l/io.xml > > > > +++ b/Documentation/DocBook/media/v4l/io.xml > > > > @@ -582,10 +582,14 @@ applications when an output stream.</entry> > > > > > > > > <entry>struct timeval</entry> > > > > <entry><structfield>timestamp</structfield></entry> > > > > <entry></entry> > > > > > > > > - <entry><para>For input streams this is the > > > > + <entry><para>This is either the > > > > > > > > system time (as returned by the <function>gettimeofday()</function> > > > > > > > > -function) when the first data byte was captured. For output streams > > > > -the data will not be displayed before this time, secondary to the > > > > +function) or a monotonic timestamp (as returned by the > > > > +<function>clock_gettime(CLOCK_MONOTONIC, &ts)</function> function). > > > > +A monotonic timestamp is used if the > > > > <constant>V4L2_CAP_MONOTONIC_TS</constant> +capability is set, otherwise > > > > the system time is used. > > > > +For input streams this is the timestamp when the first data byte was > > > > captured. +For output streams the data will not be displayed before this > > > > time, secondary to the > > > I have an alternative proposal. > > > > > > The type of the desired timestamps depend on the use case, not the driver > > > used to capture the buffers. Thus we could also give the choice to the user > > > by means of e.g. a control. > > > > Or a buffer flag. I will need something similar to select device-specific > > timestamps. > > > > However, for wall clock vs. monotonic clock, I don't think there's a reason to > > let applications decide to use the wall clock. It would be a broken use case. > > I don't think we should let applications decide in this case. > > I agree. How about the raw monotonic clock then? You can also tell clock_gettime() what kind of timestamp you're interested in. It's also true monotonic timestamps are being used elsewhere, so the selection should apply there as well. > > On the other hand, reporting a timespec instead of a timeval would be a good > > idea. I'm tempted. > > Microsecond precision seems more than sufficient to me for video frames. I see > no good reason for messing around with the v4l2_buffer struct just to get a > timespec in. The extra precision could be confusing for some existing users. Another thing, however not related to this patch, is that the spec mentions the timestamp is taken when the "first data byte is captured". In practice many (if not most) drivers produce the timestamp when the buffer is ready. The reason is that some hardware simply does not produce interrupts at when the reception of the frame starts. What kind of timestamps should be used in that case? The frame start event can be used to get information on when the frame starts. Regards,
On Fri 14 September 2012 11:02:22 Sakari Ailus wrote: > On Thu, Sep 13, 2012 at 10:56:41PM +0200, Hans Verkuil wrote: > > On Thu September 13 2012 22:50:32 Laurent Pinchart wrote: > > > Hi Sakari, > > > > > > On Thursday 13 September 2012 23:38:14 Sakari Ailus wrote: > > > > On Fri, Sep 07, 2012 at 03:29:12PM +0200, Hans Verkuil wrote: > > > > > From: Hans Verkuil <hans.verkuil@cisco.com> > > > > > > > > > > Add a new flag that tells userspace that the monotonic clock is used > > > > > for timestamps and update the documentation accordingly. > > > > > > > > > > We decided on this new flag during the 2012 Media Workshop. > > > > > > > > > > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> > > > > > --- > > > > > > > > > > Documentation/DocBook/media/v4l/io.xml | 10 +++++++--- > > > > > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 3 ++- > > > > > Documentation/DocBook/media/v4l/vidioc-querycap.xml | 7 +++++++ > > > > > include/linux/videodev2.h | 1 + > > > > > 4 files changed, 17 insertions(+), 4 deletions(-) > > > > > > > > > > diff --git a/Documentation/DocBook/media/v4l/io.xml > > > > > b/Documentation/DocBook/media/v4l/io.xml index 2dc39d8..b680d66 100644 > > > > > --- a/Documentation/DocBook/media/v4l/io.xml > > > > > +++ b/Documentation/DocBook/media/v4l/io.xml > > > > > @@ -582,10 +582,14 @@ applications when an output stream.</entry> > > > > > > > > > > <entry>struct timeval</entry> > > > > > <entry><structfield>timestamp</structfield></entry> > > > > > <entry></entry> > > > > > > > > > > - <entry><para>For input streams this is the > > > > > + <entry><para>This is either the > > > > > > > > > > system time (as returned by the <function>gettimeofday()</function> > > > > > > > > > > -function) when the first data byte was captured. For output streams > > > > > -the data will not be displayed before this time, secondary to the > > > > > +function) or a monotonic timestamp (as returned by the > > > > > +<function>clock_gettime(CLOCK_MONOTONIC, &ts)</function> function). > > > > > +A monotonic timestamp is used if the > > > > > <constant>V4L2_CAP_MONOTONIC_TS</constant> +capability is set, otherwise > > > > > the system time is used. > > > > > +For input streams this is the timestamp when the first data byte was > > > > > captured. +For output streams the data will not be displayed before this > > > > > time, secondary to the > > > > I have an alternative proposal. > > > > > > > > The type of the desired timestamps depend on the use case, not the driver > > > > used to capture the buffers. Thus we could also give the choice to the user > > > > by means of e.g. a control. > > > > > > Or a buffer flag. I will need something similar to select device-specific > > > timestamps. > > > > > > However, for wall clock vs. monotonic clock, I don't think there's a reason to > > > let applications decide to use the wall clock. It would be a broken use case. > > > I don't think we should let applications decide in this case. > > > > I agree. > > How about the raw monotonic clock then? You can also tell clock_gettime() > what kind of timestamp you're interested in. It's also true monotonic > timestamps are being used elsewhere, so the selection should apply there as > well. ALSA only has wallclock time and monotonic clock, not raw monotonic. The important thing right now is that apps can tell that a driver uses a monotonic clock. In the future we might want to refine that, but such efforts should be done together with ALSA. In other words, let's not add stuff that does not have any users. Regards, Hans > > > > On the other hand, reporting a timespec instead of a timeval would be a good > > > idea. I'm tempted. > > > > Microsecond precision seems more than sufficient to me for video frames. I see > > no good reason for messing around with the v4l2_buffer struct just to get a > > timespec in. > > The extra precision could be confusing for some existing users. > > Another thing, however not related to this patch, is that the spec mentions > the timestamp is taken when the "first data byte is captured". In practice > many (if not most) drivers produce the timestamp when the buffer is ready. > The reason is that some hardware simply does not produce interrupts at when > the reception of the frame starts. What kind of timestamps should be used in > that case? The frame start event can be used to get information on when the > frame starts. > > Regards, > > -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/Documentation/DocBook/media/v4l/io.xml b/Documentation/DocBook/media/v4l/io.xml index 2dc39d8..b680d66 100644 --- a/Documentation/DocBook/media/v4l/io.xml +++ b/Documentation/DocBook/media/v4l/io.xml @@ -582,10 +582,14 @@ applications when an output stream.</entry> <entry>struct timeval</entry> <entry><structfield>timestamp</structfield></entry> <entry></entry> - <entry><para>For input streams this is the + <entry><para>This is either the system time (as returned by the <function>gettimeofday()</function> -function) when the first data byte was captured. For output streams -the data will not be displayed before this time, secondary to the +function) or a monotonic timestamp (as returned by the +<function>clock_gettime(CLOCK_MONOTONIC, &ts)</function> function). +A monotonic timestamp is used if the <constant>V4L2_CAP_MONOTONIC_TS</constant> +capability is set, otherwise the system time is used. +For input streams this is the timestamp when the first data byte was captured. +For output streams the data will not be displayed before this time, secondary to the nominal frame rate determined by the current video standard in enqueued order. Applications can for example zero this field to display frames as soon as possible. The driver stores the time at diff --git a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml index 98a856f..00757d4 100644 --- a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml +++ b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml @@ -120,7 +120,8 @@ <entry>struct timespec</entry> <entry><structfield>timestamp</structfield></entry> <entry></entry> - <entry>Event timestamp.</entry> + <entry>Event timestamp using the monotonic clock as returned by the + <function>clock_gettime(CLOCK_MONOTONIC, &ts)</function> function.</entry> </row> <row> <entry>u32</entry> diff --git a/Documentation/DocBook/media/v4l/vidioc-querycap.xml b/Documentation/DocBook/media/v4l/vidioc-querycap.xml index d5b1248..48aa7ac 100644 --- a/Documentation/DocBook/media/v4l/vidioc-querycap.xml +++ b/Documentation/DocBook/media/v4l/vidioc-querycap.xml @@ -319,6 +319,13 @@ linkend="async">asynchronous</link> I/O methods.</entry> linkend="mmap">streaming</link> I/O method.</entry> </row> <row> + <entry><constant>V4L2_CAP_MONOTONIC_TS</constant></entry> + <entry>0x40000000</entry> + <entry>The driver uses a monotonic timestamp instead of wallclock time for the + &v4l2-buffer; <structfield>timestamp</structfield> field. + </entry> + </row> + <row> <entry><constant>V4L2_CAP_DEVICE_CAPS</constant></entry> <entry>0x80000000</entry> <entry>The driver fills the <structfield>device_caps</structfield> diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 47d58ed..00f464d 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -290,6 +290,7 @@ struct v4l2_capability { #define V4L2_CAP_ASYNCIO 0x02000000 /* async I/O */ #define V4L2_CAP_STREAMING 0x04000000 /* streaming I/O ioctls */ +#define V4L2_CAP_MONOTONIC_TS 0x40000000 /* uses monotonic timestamps */ #define V4L2_CAP_DEVICE_CAPS 0x80000000 /* sets device capabilities field */ /*