diff mbox series

[v8,1/6] drm/fourcc: Add modifier definitions for describing Amlogic Video Framebuffer Compression

Message ID 20200702074759.32356-2-narmstrong@baylibre.com (mailing list archive)
State Superseded
Headers show
Series drm/meson: add support for Amlogic Video FBC | expand

Commit Message

Neil Armstrong July 2, 2020, 7:47 a.m. UTC 
Amlogic uses a proprietary lossless image compression protocol and format
for their hardware video codec accelerators, either video decoders or
video input encoders.

It considerably reduces memory bandwidth while writing and reading
frames in memory.

The underlying storage is considered to be 3 components, 8bit or 10-bit
per component, YCbCr 420, single plane :
- DRM_FORMAT_YUV420_8BIT
- DRM_FORMAT_YUV420_10BIT

This modifier will be notably added to DMA-BUF frames imported from the V4L2
Amlogic VDEC decoder.

This introduces the basic layout composed of:
- a body content organized in 64x32 superblocks with 4096 bytes per
  superblock in default mode.
- a 32 bytes per 128x64 header block

This layout is tranferrable between Amlogic SoCs supporting this modifier.

The Memory Saving option exist changing the layout superblock size to save memory when
using 8bit components pixels size.

Finally is also adds the Scatter Memory layout, meaning the header contains IOMMU
references to the compressed frames content to optimize memory access
and layout.

In this mode, only the header memory address is needed, thus the content
memory organization is tied to the current producer execution and cannot
be saved/dumped neither transferrable between Amlogic SoCs supporting this
modifier.

Tested-by: Kevin Hilman <khilman@baylibre.com>
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
---
 include/uapi/drm/drm_fourcc.h | 74 +++++++++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)

Comments

Simon Ser July 2, 2020, 9:23 a.m. UTC | #1 
On Thursday, July 2, 2020 9:47 AM, Neil Armstrong <narmstrong@baylibre.com> wrote:

> Finally is also adds the Scatter Memory layout, meaning the header contains IOMMU
> references to the compressed frames content to optimize memory access
> and layout.
>
> In this mode, only the header memory address is needed, thus the content
> memory organization is tied to the current producer execution and cannot
> be saved/dumped neither transferrable between Amlogic SoCs supporting this
> modifier.

Still not sure how to handle this one, since this breaks fundamental
assumptions about modifiers.
Daniel Vetter July 2, 2020, 1:18 p.m. UTC | #2 
On Thu, Jul 02, 2020 at 09:23:11AM +0000, Simon Ser wrote:
> On Thursday, July 2, 2020 9:47 AM, Neil Armstrong <narmstrong@baylibre.com> wrote:
> 
> > Finally is also adds the Scatter Memory layout, meaning the header contains IOMMU
> > references to the compressed frames content to optimize memory access
> > and layout.
> >
> > In this mode, only the header memory address is needed, thus the content
> > memory organization is tied to the current producer execution and cannot
> > be saved/dumped neither transferrable between Amlogic SoCs supporting this
> > modifier.
> 
> Still not sure how to handle this one, since this breaks fundamental
> assumptions about modifiers.

I wonder whether we should require special allocations for these, and then
just outright reject mmap on these buffers. mmap on dma-buf isn't a
required feature.

That would make sure that userspace cannot look at them.

Also I'm kinda suspecting that there's not unlimited amounts of this magic
invisible storage available anyway.
-Daniel
Neil Armstrong July 2, 2020, 1:34 p.m. UTC | #3 
On 02/07/2020 15:18, Daniel Vetter wrote:
> On Thu, Jul 02, 2020 at 09:23:11AM +0000, Simon Ser wrote:
>> On Thursday, July 2, 2020 9:47 AM, Neil Armstrong <narmstrong@baylibre.com> wrote:
>>
>>> Finally is also adds the Scatter Memory layout, meaning the header contains IOMMU
>>> references to the compressed frames content to optimize memory access
>>> and layout.
>>>
>>> In this mode, only the header memory address is needed, thus the content
>>> memory organization is tied to the current producer execution and cannot
>>> be saved/dumped neither transferrable between Amlogic SoCs supporting this
>>> modifier.
>>
>> Still not sure how to handle this one, since this breaks fundamental
>> assumptions about modifiers.
> 
> I wonder whether we should require special allocations for these, and then
> just outright reject mmap on these buffers. mmap on dma-buf isn't a
> required feature.

Yes, it's the plan to reject mmap on these buffers, but it can't be explained
in the modifiers description and it's a requirement of the producer, not the
consumer.

> 
> That would make sure that userspace cannot look at them.
> 
> Also I'm kinda suspecting that there's not unlimited amounts of this magic
> invisible storage available anyway.
> -Daniel
>
Daniel Vetter July 2, 2020, 2:15 p.m. UTC | #4 
On Thu, Jul 2, 2020 at 3:34 PM Neil Armstrong <narmstrong@baylibre.com> wrote:
>
> On 02/07/2020 15:18, Daniel Vetter wrote:
> > On Thu, Jul 02, 2020 at 09:23:11AM +0000, Simon Ser wrote:
> >> On Thursday, July 2, 2020 9:47 AM, Neil Armstrong <narmstrong@baylibre.com> wrote:
> >>
> >>> Finally is also adds the Scatter Memory layout, meaning the header contains IOMMU
> >>> references to the compressed frames content to optimize memory access
> >>> and layout.
> >>>
> >>> In this mode, only the header memory address is needed, thus the content
> >>> memory organization is tied to the current producer execution and cannot
> >>> be saved/dumped neither transferrable between Amlogic SoCs supporting this
> >>> modifier.
> >>
> >> Still not sure how to handle this one, since this breaks fundamental
> >> assumptions about modifiers.
> >
> > I wonder whether we should require special allocations for these, and then
> > just outright reject mmap on these buffers. mmap on dma-buf isn't a
> > required feature.
>
> Yes, it's the plan to reject mmap on these buffers, but it can't be explained
> in the modifiers description and it's a requirement of the producer, not the
> consumer.

Hm I think worth to add that as a note to the modifier. Just to make
sure. And avoids questions like the one from Simon.
-Daniel

>
> >
> > That would make sure that userspace cannot look at them.
> >
> > Also I'm kinda suspecting that there's not unlimited amounts of this magic
> > invisible storage available anyway.
> > -Daniel
> >
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
Neil Armstrong July 2, 2020, 3:11 p.m. UTC | #5 
Hi,

On 02/07/2020 16:15, Daniel Vetter wrote:
> On Thu, Jul 2, 2020 at 3:34 PM Neil Armstrong <narmstrong@baylibre.com> wrote:
>>
>> On 02/07/2020 15:18, Daniel Vetter wrote:
>>> On Thu, Jul 02, 2020 at 09:23:11AM +0000, Simon Ser wrote:
>>>> On Thursday, July 2, 2020 9:47 AM, Neil Armstrong <narmstrong@baylibre.com> wrote:
>>>>
>>>>> Finally is also adds the Scatter Memory layout, meaning the header contains IOMMU
>>>>> references to the compressed frames content to optimize memory access
>>>>> and layout.
>>>>>
>>>>> In this mode, only the header memory address is needed, thus the content
>>>>> memory organization is tied to the current producer execution and cannot
>>>>> be saved/dumped neither transferrable between Amlogic SoCs supporting this
>>>>> modifier.
>>>>
>>>> Still not sure how to handle this one, since this breaks fundamental
>>>> assumptions about modifiers.
>>>
>>> I wonder whether we should require special allocations for these, and then
>>> just outright reject mmap on these buffers. mmap on dma-buf isn't a
>>> required feature.
>>
>> Yes, it's the plan to reject mmap on these buffers, but it can't be explained
>> in the modifiers description and it's a requirement of the producer, not the
>> consumer.
> 
> Hm I think worth to add that as a note to the modifier. Just to make
> sure. And avoids questions like the one from Simon.

Something like:

 /*
  * Amlogic FBC Scatter Memory layout
  *
  * Indicates the header contains IOMMU references to the compressed
  * frames content to optimize memory access and layout.
  *
  * In this mode, only the header memory address is needed, thus the
  * content memory organization is tied to the current producer
  * execution and cannot be saved/dumped neither transferrable between
  * Amlogic SoCs supporting this modifier.
+ *
+ * Due to the nature of the layout, these buffers are not expected to
+ * be accessible by the user-space clients but only accessible by the
+ * hardware producers and consumers.
+ *
+ * The user-space clients should expect a failure while trying to mmap
+ * the DMA-BUF handle returned by the producer.
  */

Thanks,
Neil

> -Daniel
> 
>>
>>>
>>> That would make sure that userspace cannot look at them.
>>>
>>> Also I'm kinda suspecting that there's not unlimited amounts of this magic
>>> invisible storage available anyway.
>>> -Daniel
>>>
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> 
> 
>
Daniel Vetter July 2, 2020, 6:03 p.m. UTC | #6 
On Thu, Jul 02, 2020 at 05:11:25PM +0200, Neil Armstrong wrote:
> Hi,
> 
> On 02/07/2020 16:15, Daniel Vetter wrote:
> > On Thu, Jul 2, 2020 at 3:34 PM Neil Armstrong <narmstrong@baylibre.com> wrote:
> >>
> >> On 02/07/2020 15:18, Daniel Vetter wrote:
> >>> On Thu, Jul 02, 2020 at 09:23:11AM +0000, Simon Ser wrote:
> >>>> On Thursday, July 2, 2020 9:47 AM, Neil Armstrong <narmstrong@baylibre.com> wrote:
> >>>>
> >>>>> Finally is also adds the Scatter Memory layout, meaning the header contains IOMMU
> >>>>> references to the compressed frames content to optimize memory access
> >>>>> and layout.
> >>>>>
> >>>>> In this mode, only the header memory address is needed, thus the content
> >>>>> memory organization is tied to the current producer execution and cannot
> >>>>> be saved/dumped neither transferrable between Amlogic SoCs supporting this
> >>>>> modifier.
> >>>>
> >>>> Still not sure how to handle this one, since this breaks fundamental
> >>>> assumptions about modifiers.
> >>>
> >>> I wonder whether we should require special allocations for these, and then
> >>> just outright reject mmap on these buffers. mmap on dma-buf isn't a
> >>> required feature.
> >>
> >> Yes, it's the plan to reject mmap on these buffers, but it can't be explained
> >> in the modifiers description and it's a requirement of the producer, not the
> >> consumer.
> > 
> > Hm I think worth to add that as a note to the modifier. Just to make
> > sure. And avoids questions like the one from Simon.
> 
> Something like:
> 
>  /*
>   * Amlogic FBC Scatter Memory layout
>   *
>   * Indicates the header contains IOMMU references to the compressed
>   * frames content to optimize memory access and layout.
>   *
>   * In this mode, only the header memory address is needed, thus the
>   * content memory organization is tied to the current producer
>   * execution and cannot be saved/dumped neither transferrable between
>   * Amlogic SoCs supporting this modifier.
> + *
> + * Due to the nature of the layout, these buffers are not expected to
> + * be accessible by the user-space clients but only accessible by the
> + * hardware producers and consumers.
> + *
> + * The user-space clients should expect a failure while trying to mmap
> + * the DMA-BUF handle returned by the producer.
>   */

lgtm, Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> 
> Thanks,
> Neil
> 
> > -Daniel
> > 
> >>
> >>>
> >>> That would make sure that userspace cannot look at them.
> >>>
> >>> Also I'm kinda suspecting that there's not unlimited amounts of this magic
> >>> invisible storage available anyway.
> >>> -Daniel
> >>>
> >>
> >> _______________________________________________
> >> dri-devel mailing list
> >> dri-devel@lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > 
> > 
> > 
>
diff mbox series

Patch

diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index 993c1b342315..54e8935f0e4a 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -331,6 +331,7 @@  extern "C" {
 #define DRM_FORMAT_MOD_VENDOR_BROADCOM 0x07
 #define DRM_FORMAT_MOD_VENDOR_ARM     0x08
 #define DRM_FORMAT_MOD_VENDOR_ALLWINNER 0x09
+#define DRM_FORMAT_MOD_VENDOR_AMLOGIC 0x0a
 
 /* add more to the end as needed */
 
@@ -950,6 +951,79 @@  drm_fourcc_canonicalize_nvidia_format_mod(__u64 modifier)
  */
 #define DRM_FORMAT_MOD_ALLWINNER_TILED fourcc_mod_code(ALLWINNER, 1)
 
+/*
+ * Amlogic Video Framebuffer Compression modifiers
+ *
+ * Amlogic uses a proprietary lossless image compression protocol and format
+ * for their hardware video codec accelerators, either video decoders or
+ * video input encoders.
+ *
+ * It considerably reduces memory bandwidth while writing and reading
+ * frames in memory.
+ *
+ * The underlying storage is considered to be 3 components, 8bit or 10-bit
+ * per component YCbCr 420, single plane :
+ * - DRM_FORMAT_YUV420_8BIT
+ * - DRM_FORMAT_YUV420_10BIT
+ *
+ * The first 8 bits of the mode defines the layout, then the following 8 bits
+ * defines the options changing the layout.
+ *
+ * Not all combinations are valid, and different SoCs may support different
+ * combinations of layout and options.
+ */
+#define __fourcc_mod_amlogic_layout_mask 0xf
+#define __fourcc_mod_amlogic_options_shift 8
+#define __fourcc_mod_amlogic_options_mask 0xf
+
+#define DRM_FORMAT_MOD_AMLOGIC_FBC(__layout, __options) \
+	fourcc_mod_code(AMLOGIC, \
+			((__layout) & __fourcc_mod_amlogic_layout_mask) | \
+			((__options) & __fourcc_mod_amlogic_options_mask \
+			 << __fourcc_mod_amlogic_options_shift))
+
+/* Amlogic FBC Layouts */
+
+/*
+ * Amlogic FBC Basic Layout
+ *
+ * The basic layout is composed of:
+ * - a body content organized in 64x32 superblocks with 4096 bytes per
+ *   superblock in default mode.
+ * - a 32 bytes per 128x64 header block
+ *
+ * This layout is transferrable between Amlogic SoCs supporting this modifier.
+ */
+#define AMLOGIC_FBC_LAYOUT_BASIC		(1ULL)
+
+/*
+ * Amlogic FBC Scatter Memory layout
+ *
+ * Indicates the header contains IOMMU references to the compressed
+ * frames content to optimize memory access and layout.
+ *
+ * In this mode, only the header memory address is needed, thus the
+ * content memory organization is tied to the current producer
+ * execution and cannot be saved/dumped neither transferrable between
+ * Amlogic SoCs supporting this modifier.
+ */
+#define AMLOGIC_FBC_LAYOUT_SCATTER		(2ULL)
+
+/* Amlogic FBC Layout Options Bit Mask */
+
+/*
+ * Amlogic FBC Memory Saving mode
+ *
+ * Indicates the storage is packed when pixel size is multiple of word
+ * boudaries, i.e. 8bit should be stored in this mode to save allocation
+ * memory.
+ *
+ * This mode reduces body layout to 3072 bytes per 64x32 superblock with
+ * the basic layout and 3200 bytes per 64x32 superblock combined with
+ * the scatter layout.
+ */
+#define AMLOGIC_FBC_OPTION_MEM_SAVING		(1ULL << 0)
+
 #if defined(__cplusplus)
 }
 #endif