diff mbox series

[v6,3/3] drm: Add GUD USB Display driver

Message ID 20210219121702.50964-4-noralf@tronnes.org (mailing list archive)
State New, archived
Headers show
Series GUD USB Display driver | expand

Commit Message

Noralf Trønnes Feb. 19, 2021, 12:17 p.m. UTC
This adds a USB display driver with the intention that it can be
used with future USB interfaced low end displays/adapters. The Linux
gadget device driver will serve as the canonical device implementation.

The following DRM properties are supported:
- Plane rotation
- Connector TV properties

There is also support for backlight brightness exposed as a backlight
device.

Display modes can be made available to the host driver either as DRM
display modes or through EDID. If both are present, EDID is just passed
on to userspace.

Performance is preferred over color depth, so if the device supports
RGB565, DRM_CAP_DUMB_PREFERRED_DEPTH will return 16.

If the device transfer buffer can't fit an uncompressed framebuffer
update, the update is split up into parts that do fit.

Optimal user experience is achieved by providing damage reports either by
setting FB_DAMAGE_CLIPS on pageflips or calling DRM_IOCTL_MODE_DIRTYFB.

LZ4 compression is used if the device supports it.

The driver supports a one bit monochrome transfer format: R1. This is not
implemented in the gadget driver. It is added in preparation for future
monochrome e-ink displays.

The driver is MIT licensed to smooth the path for any BSD port of the
driver.

v2:
- Use devm_drm_dev_alloc() and drmm_mode_config_init()
- drm_fbdev_generic_setup: Use preferred_bpp=0, 16 was a copy paste error
- The drm_backlight_helper is dropped, copy in the code
- Support protocol version backwards compatibility for device

v3:
- Use donated Openmoko USB pid
- Use direct compression from framebuffer when pitch matches, not only on
  full frames, so split updates can benefit
- Use __le16 in struct gud_drm_req_get_connector_status
- Set edid property when the device only provides edid
- Clear compression fields in struct gud_drm_req_set_buffer
- Fix protocol version negotiation
- Remove mode->vrefresh, it's calculated

v4:
- Drop the status req polling which was a workaround for something that
  turned out to be a dwc2 udc driver problem
- Add a flag for the Linux gadget to require a status request on
  SET operations. Other devices will only get status req on STALL errors
- Use protocol specific error codes (Peter)
- Add a flag for devices that want to receive the entire framebuffer on
  each flush (Lubomir)
- Retry a failed framebuffer flush
- If mode has changed wait for worker and clear pending damage before
  queuing up new damage, fb width/height might have changed
- Increase error counter on bulk transfer failures
- Use DRM_MODE_CONNECTOR_USB
- Handle R1 kmalloc error (Peter)
- Don't try and replicate the USB get descriptor request standard for the
  display descriptor (Peter)
- Make max_buffer_size optional (Peter), drop the pow2 requirement since
  it's not necessary anymore.
- Don't pre-alloc a control request buffer, it was only 4k
- Let gud.h describe the whole protocol explicitly and don't let DRM
  leak into it (Peter)
- Drop display mode .hskew and .vscan from the protocol
- Shorten names: s/GUD_DRM_/GUD_/ s/gud_drm_/gud_/ (Peter)
- Fix gud_pipe_check() connector picking when switching connector
- Drop gud_drm_driver_gem_create_object() cached is default now
- Retrieve USB device from struct drm_device.dev instead of keeping a
  pointer
- Honour fb->offsets[0]
- Fix mode fetching when connector status is forced
- Check EDID length reported by the device
- Use drm_do_get_edid() so userspace can overrride EDID
- Set epoch counter to signal connector status change
- gud_drm_driver can be const now

v5:
- GUD_DRM_FORMAT_R1: Use non-human ascii values (Daniel)
- Change name to: GUD USB Display (Thomas, Simon)
- Change one __u32 -> __le32 in protocol header
- Always log fb flush errors, unless the previous one failed
- Run backlight update in a worker to avoid upsetting lockdep (Daniel)
- Drop backlight_ops.get_brightness, there's no readback from the device
  so it doesn't really add anything.
- Set dma mask, needed by dma-buf importers

v6:
- Use obj-y in Makefile (Peter)
- Fix missing le32_to_cpu() when using GUD_DISPLAY_MAGIC (Peter)
- Set initial brightness on backlight device

Cc: Peter Stuge <peter@stuge.se>
Tested-by: Lubomir Rintel <lkundrak@v3.sk>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Noralf Trønnes <noralf@tronnes.org>
---
 MAINTAINERS                         |   8 +
 drivers/gpu/drm/Kconfig             |   2 +
 drivers/gpu/drm/Makefile            |   1 +
 drivers/gpu/drm/gud/Kconfig         |  14 +
 drivers/gpu/drm/gud/Makefile        |   4 +
 drivers/gpu/drm/gud/gud_connector.c | 738 ++++++++++++++++++++++++++++
 drivers/gpu/drm/gud/gud_drv.c       | 625 +++++++++++++++++++++++
 drivers/gpu/drm/gud/gud_internal.h  | 149 ++++++
 drivers/gpu/drm/gud/gud_pipe.c      | 475 ++++++++++++++++++
 include/drm/gud.h                   | 356 ++++++++++++++
 10 files changed, 2372 insertions(+)
 create mode 100644 drivers/gpu/drm/gud/Kconfig
 create mode 100644 drivers/gpu/drm/gud/Makefile
 create mode 100644 drivers/gpu/drm/gud/gud_connector.c
 create mode 100644 drivers/gpu/drm/gud/gud_drv.c
 create mode 100644 drivers/gpu/drm/gud/gud_internal.h
 create mode 100644 drivers/gpu/drm/gud/gud_pipe.c
 create mode 100644 include/drm/gud.h

Comments

Peter Stuge Feb. 19, 2021, 9:42 p.m. UTC | #1
Hi Noralf,

Noralf Trønnes wrote:
> +++ b/drivers/gpu/drm/gud/gud_connector.c
..
> +static int gud_connector_get_edid_block(void *data, u8 *buf, unsigned int block, size_t len)
..
> +	struct gud_connector *gconn = ctx->gconn;
> +	size_t start = block * EDID_LENGTH;
> +
> +	if (start + len > gconn->edid_len)
> +		return -1;
> +
> +	if (!block) {
> +		struct gud_device *gdrm = to_gud_device(gconn->connector.dev);
> +		int ret;
> +
> +		/* Check because drm_do_get_edid() will retry on failure */
> +		if (!ctx->buf)
> +			ctx->buf = kmalloc(gconn->edid_len, GFP_KERNEL);
> +		if (!ctx->buf)
> +			return -1;
> +
> +		ret = gud_usb_get(gdrm, GUD_REQ_GET_CONNECTOR_EDID, gconn->connector.index,
> +				  ctx->buf, gconn->edid_len);
..
> +	memcpy(buf, ctx->buf + start, len);

Danger, danger?

gconn->edid_len in this call to gud_usb_get() comes from the device in
gud_connector_status_request() where the only validation is that
edid_len % EDID_LENGTH == 0, so a device could write past the buffer
if drm_do_get_edid() passes a buffer smaller than edid_len.

I guess the buffer passed is just 128, EDID_LENGTH, so a malicious
or buggy device could overwrite 64k-128 kernel memory? Ouch!


More generally it's not very typical in USB to report the data size
separately from the data itself, if reporting size explicitly at all.

Sizes can be part of the data structure itself (like in descriptors) but
on the application layer (like here) it's convenient to just decide a
sensible fixed maximum size and let the host try to always transfer
that size while accepting short transfers. Unlike read() a short
transfer only ever happens if and when a device intends for it,
so that's like an in-band handshake but "for free".

Oh, and does/should the GUD EDID change if the panel "behind" the device
CPU on a hotpluggable connector changes? It wouldn't be great to require
GUD driver reprobe in that case. But maybe DRM requires that anyway?


I'm sorry I didn't spot this pattern earlier, I understand that it's late
in the game and that changing it needs the gadget to change as well, but I
do really think this is a worthwhile change throughout the protocol.

And I think it applies to more than EDID, e.g. both GUD and connector
properties, maybe formats, something else?


Unfortunately, the gud_usb_control_msg() check (ret != len) creates a
requirement to know in advance how much data will be transfered.

That could be revised at least for the general case, even if not used
everywhere; maybe something like adding a size_t required_min_len
parameter to gud_usb_control_msg()?


> +static int gud_connector_get_modes(struct drm_connector *connector)
> +{
> +	struct gud_connector *gconn = to_gud_connector(connector);
> +	struct gud_device *gdrm = to_gud_device(connector->dev);
> +	struct gud_connector_get_edid_ctx edid_ctx = {
> +		.gconn = gconn,
> +	};
> +	struct gud_display_mode_req *reqmodes = NULL;
> +	unsigned int i, num_modes = 0;

The error path of this function executes "return num_modes" with num_modes
unmodified; ie. 0. Is that intentional?


> +static int gud_connector_add_tv_mode(struct gud_device *gdrm,
..
> +	buf_len = num_modes * GUD_CONNECTOR_TV_MODE_NAME_LEN;
> +	modes = kmalloc_array(num_modes, sizeof(*modes), GFP_KERNEL);
> +	buf = kmalloc(buf_len, GFP_KERNEL);

Maybe moving the buf assignment immediately following the buf_len assignment
would help readability? This is quite minor.


> +static int gud_connector_add_properties(struct gud_device *gdrm, struct gud_connector *gconn,
> +					unsigned int num_properties)
> +{
> +	struct drm_device *drm = &gdrm->drm;
> +	struct drm_connector *connector = &gconn->connector;
> +	struct gud_property_req *properties;
> +	unsigned int i;
> +	int ret;
> +
> +	gconn->properties = kcalloc(num_properties, sizeof(*gconn->properties), GFP_KERNEL);
> +	if (!gconn->properties)
> +		return -ENOMEM;
> +
> +	properties = kcalloc(num_properties, sizeof(*properties), GFP_KERNEL);
> +	if (!properties)
> +		return -ENOMEM;

I think this error path leaks gconn->properties?


> +int gud_connector_create(struct gud_device *gdrm, unsigned int index)

Most error paths in this function seem to leak both gconn and connector?


> +++ b/drivers/gpu/drm/gud/gud_drv.c
..
> +static int gud_usb_get_status(struct usb_device *usb, u8 ifnum, u8 *status)
..
> +	ret = gud_usb_control_msg(usb, ifnum, true, GUD_REQ_GET_STATUS, 0, buf, sizeof(*buf));
> +	*status = *buf;

Maybe make this conditional on 0 == ret.


> +static int gud_set_version(struct usb_device *usb, u8 ifnum, u32 flags, u8 version)
..
> +	if (ret == -EPIPE)
> +		return -EPROTONOSUPPORT;

So yeah, this isn't typical, devices usually describe optional things that
the driver may need to know about, unless it's something that can change
during operation.

Arguably mildly contradictory to the short transfer pattern, but one is
capability and the other is "runtime" data.


> +static int gud_get_properties(struct gud_device *gdrm, unsigned int num_properties)
> +{
> +	struct gud_property_req *properties;
> +	unsigned int i;
> +	int ret;
> +
> +	if (!num_properties)
> +		return 0;
> +
> +	gdrm->properties = drmm_kcalloc(&gdrm->drm, num_properties, sizeof(*gdrm->properties),
> +					GFP_KERNEL);
> +	if (!gdrm->properties)
> +		return -ENOMEM;
> +
> +	properties = kcalloc(num_properties, sizeof(*properties), GFP_KERNEL);
> +	if (!properties)
> +		return -ENOMEM;

It looks like this function leaks gdrm->properties in all error paths?


> +		default:
> +			/* New ones might show up in future devices, skip those we don't know. */
> +			drm_dbg(&gdrm->drm, "Unknown property: %u\n", prop);

Maybe "Ignoring unknown property: %u\n" would be a little more clear?


> +static int gud_stats_debugfs(struct seq_file *m, void *data)
..
> +	seq_puts(m, "Compression:      ");
> +	if (gdrm->compression & GUD_COMPRESSION_LZ4)
> +		seq_puts(m, " lz4");
> +	seq_puts(m, "\n");

Maybe an explicit seq_puts(m, " none") if there are none?


> +	if (gdrm->compression) {
> +		u64 remainder;
> +		u64 ratio = div64_u64_rem(gdrm->stats_length, gdrm->stats_actual_length,
> +					  &remainder);
> +		u64 ratio_frac = div64_u64(remainder * 10, gdrm->stats_actual_length);
> +
> +		seq_printf(m, "Compression ratio: %llu.%llu\n", ratio, ratio_frac);
> +	}

Will the fraction ever need zero padding?


> +static int gud_probe(struct usb_interface *interface, const struct usb_device_id *id)

I appreciate very much that GUD works on interface level, so that it
can also be used in composite devices at some point. Thanks a lot! \o/


> +++ b/drivers/gpu/drm/gud/gud_pipe.c
..
> +int gud_pipe_check(struct drm_simple_display_pipe *pipe,
..
> +	req = kzalloc(len, GFP_KERNEL);
> +	if (!req)
> +		return -ENOMEM;
> +
> +	gud_from_display_mode(&req->mode, mode);
> +
> +	req->format = gud_from_fourcc(format->format);
> +	if (WARN_ON_ONCE(!req->format))
> +		return -EINVAL;

req leaks?


> +	req->connector = drm_connector_index(connector);
> +	req->num_properties = num_properties;
> +
> +	num_properties = gud_connector_fill_properties(connector, connector_state,
> +						       req->properties);

Following this new assignment to num_properties the new value is used
to (hopefully!) append at the first req->properties[] index after the
old value was used as index, that doesn't feel great..

I mean, it's harmless as long as gud_connector_fill_properties() is
sure to return the same value, but, well, maybe sometime later it
doesn't, or is that guaranteed if there is no error? Then maybe at
least document that requirement by the function. What do you think?


Thanks a lot

//Peter
Noralf Trønnes Feb. 20, 2021, 5:27 p.m. UTC | #2
Den 19.02.2021 22.42, skrev Peter Stuge:
> Hi Noralf,
> 
> Noralf Trønnes wrote:
>> +++ b/drivers/gpu/drm/gud/gud_connector.c
> ..
>> +static int gud_connector_get_edid_block(void *data, u8 *buf, unsigned int block, size_t len)
> ..
>> +	struct gud_connector *gconn = ctx->gconn;
>> +	size_t start = block * EDID_LENGTH;
>> +
>> +	if (start + len > gconn->edid_len)
>> +		return -1;
>> +
>> +	if (!block) {
>> +		struct gud_device *gdrm = to_gud_device(gconn->connector.dev);
>> +		int ret;
>> +
>> +		/* Check because drm_do_get_edid() will retry on failure */
>> +		if (!ctx->buf)
>> +			ctx->buf = kmalloc(gconn->edid_len, GFP_KERNEL);
>> +		if (!ctx->buf)
>> +			return -1;
>> +
>> +		ret = gud_usb_get(gdrm, GUD_REQ_GET_CONNECTOR_EDID, gconn->connector.index,
>> +				  ctx->buf, gconn->edid_len);
> ..
>> +	memcpy(buf, ctx->buf + start, len);
> 
> Danger, danger?
> 
> gconn->edid_len in this call to gud_usb_get() comes from the device in
> gud_connector_status_request() where the only validation is that
> edid_len % EDID_LENGTH == 0, so a device could write past the buffer
> if drm_do_get_edid() passes a buffer smaller than edid_len.
> 
> I guess the buffer passed is just 128, EDID_LENGTH, so a malicious
> or buggy device could overwrite 64k-128 kernel memory? Ouch!
> 

The result goes into ctx->buf which is big enough. Then each edid block
is copied from that buffer as the parser runs the callback.

Maybe I should add a cap on ->edid_len, but I don't know how big it
should be. There's no danger as such not having a cap, the host will
either fail to allocate memory (max 4MB usually) or do a very big
transfer and error out in the edid parser.

> 
> More generally it's not very typical in USB to report the data size
> separately from the data itself, if reporting size explicitly at all.
> 
> Sizes can be part of the data structure itself (like in descriptors) but
> on the application layer (like here) it's convenient to just decide a
> sensible fixed maximum size and let the host try to always transfer
> that size while accepting short transfers. Unlike read() a short
> transfer only ever happens if and when a device intends for it,
> so that's like an in-band handshake but "for free".
> 
> Oh, and does/should the GUD EDID change if the panel "behind" the device
> CPU on a hotpluggable connector changes? It wouldn't be great to require
> GUD driver reprobe in that case. But maybe DRM requires that anyway?
> 

If gud_connector_status_req.status has changed since last poll or
GUD_CONNECTOR_STATUS_CHANGED is set, DRM will notify userspace which
will reprobe the connector. connector->epoch_counter++ in
gud_connector_status_request() triggers that.

> 
> I'm sorry I didn't spot this pattern earlier, I understand that it's late
> in the game and that changing it needs the gadget to change as well, but I
> do really think this is a worthwhile change throughout the protocol.
> 

I see what you mean, I'll give it a try.

> And I think it applies to more than EDID, e.g. both GUD and connector
> properties, maybe formats, something else?
> 
> 
> Unfortunately, the gud_usb_control_msg() check (ret != len) creates a
> requirement to know in advance how much data will be transfered.
> 
> That could be revised at least for the general case, even if not used
> everywhere; maybe something like adding a size_t required_min_len
> parameter to gud_usb_control_msg()?
> 
> 
>> +static int gud_connector_get_modes(struct drm_connector *connector)
>> +{
>> +	struct gud_connector *gconn = to_gud_connector(connector);
>> +	struct gud_device *gdrm = to_gud_device(connector->dev);
>> +	struct gud_connector_get_edid_ctx edid_ctx = {
>> +		.gconn = gconn,
>> +	};
>> +	struct gud_display_mode_req *reqmodes = NULL;
>> +	unsigned int i, num_modes = 0;
> 
> The error path of this function executes "return num_modes" with num_modes
> unmodified; ie. 0. Is that intentional?
> 

It's not allowed to return an error code so 0 is the only option. The
caller drm_helper_probe_single_connector_modes() adds a fallback
1024x768 mode in that case which probably won't fit for a display panel,
but the device should reject an illegal mode on GUD_REQ_SET_STATE_CHECK
so we're fine.

> 
>> +static int gud_connector_add_tv_mode(struct gud_device *gdrm,
> ..
>> +	buf_len = num_modes * GUD_CONNECTOR_TV_MODE_NAME_LEN;
>> +	modes = kmalloc_array(num_modes, sizeof(*modes), GFP_KERNEL);
>> +	buf = kmalloc(buf_len, GFP_KERNEL);
> 
> Maybe moving the buf assignment immediately following the buf_len assignment
> would help readability? This is quite minor.
> 
> 
>> +static int gud_connector_add_properties(struct gud_device *gdrm, struct gud_connector *gconn,
>> +					unsigned int num_properties)
>> +{
>> +	struct drm_device *drm = &gdrm->drm;
>> +	struct drm_connector *connector = &gconn->connector;
>> +	struct gud_property_req *properties;
>> +	unsigned int i;
>> +	int ret;
>> +
>> +	gconn->properties = kcalloc(num_properties, sizeof(*gconn->properties), GFP_KERNEL);
>> +	if (!gconn->properties)
>> +		return -ENOMEM;
>> +
>> +	properties = kcalloc(num_properties, sizeof(*properties), GFP_KERNEL);
>> +	if (!properties)
>> +		return -ENOMEM;
> 
> I think this error path leaks gconn->properties?
> 

It's freed in gud_connector_destroy() which will be called on error at
this stage.

> 
>> +int gud_connector_create(struct gud_device *gdrm, unsigned int index)
> 
> Most error paths in this function seem to leak both gconn and connector?
> 

Everything that happens after the drm_connector_init() call is cleaned
up automatically when the DRM device is torn down.
devm_drm_dev_alloc() and drmm_mode_config_init() sets this up.

> 
>> +++ b/drivers/gpu/drm/gud/gud_drv.c
> ..
>> +static int gud_usb_get_status(struct usb_device *usb, u8 ifnum, u8 *status)
> ..
>> +	ret = gud_usb_control_msg(usb, ifnum, true, GUD_REQ_GET_STATUS, 0, buf, sizeof(*buf));
>> +	*status = *buf;
> 
> Maybe make this conditional on 0 == ret.
> 
> 
>> +static int gud_set_version(struct usb_device *usb, u8 ifnum, u32 flags, u8 version)
> ..
>> +	if (ret == -EPIPE)
>> +		return -EPROTONOSUPPORT;
> 
> So yeah, this isn't typical, devices usually describe optional things that
> the driver may need to know about, unless it's something that can change
> during operation.
> 
> Arguably mildly contradictory to the short transfer pattern, but one is
> capability and the other is "runtime" data.
> 
> 
>> +static int gud_get_properties(struct gud_device *gdrm, unsigned int num_properties)
>> +{
>> +	struct gud_property_req *properties;
>> +	unsigned int i;
>> +	int ret;
>> +
>> +	if (!num_properties)
>> +		return 0;
>> +
>> +	gdrm->properties = drmm_kcalloc(&gdrm->drm, num_properties, sizeof(*gdrm->properties),
>> +					GFP_KERNEL);
>> +	if (!gdrm->properties)
>> +		return -ENOMEM;
>> +
>> +	properties = kcalloc(num_properties, sizeof(*properties), GFP_KERNEL);
>> +	if (!properties)
>> +		return -ENOMEM;
> 
> It looks like this function leaks gdrm->properties in all error paths?
> 

drmm_kcalloc() is the DRM version of devm_kcalloc() which frees the
memory when the last DRM fd is closed. There might be open fd's when the
device goes away so the devm_ versions can't be used.

> 
>> +		default:
>> +			/* New ones might show up in future devices, skip those we don't know. */
>> +			drm_dbg(&gdrm->drm, "Unknown property: %u\n", prop);
> 
> Maybe "Ignoring unknown property: %u\n" would be a little more clear?
> 

Sure.

> 
>> +static int gud_stats_debugfs(struct seq_file *m, void *data)
> ..
>> +	seq_puts(m, "Compression:      ");
>> +	if (gdrm->compression & GUD_COMPRESSION_LZ4)
>> +		seq_puts(m, " lz4");
>> +	seq_puts(m, "\n");
> 
> Maybe an explicit seq_puts(m, " none") if there are none?
> 

That makes sense.

> 
>> +	if (gdrm->compression) {
>> +		u64 remainder;
>> +		u64 ratio = div64_u64_rem(gdrm->stats_length, gdrm->stats_actual_length,
>> +					  &remainder);
>> +		u64 ratio_frac = div64_u64(remainder * 10, gdrm->stats_actual_length);
>> +
>> +		seq_printf(m, "Compression ratio: %llu.%llu\n", ratio, ratio_frac);
>> +	}
> 
> Will the fraction ever need zero padding?
> 

No, I don't see why.

> 
>> +static int gud_probe(struct usb_interface *interface, const struct usb_device_id *id)
> 
> I appreciate very much that GUD works on interface level, so that it
> can also be used in composite devices at some point. Thanks a lot! \o/
> 

This was a design requirement since I wanted to make room for at least
HID touch and maybe audio. It will also tolerate other vendor class
interfaces and just silently ignore them.

The Pi images on the wiki has the ability to add a serial console USB
interface for debugging.

> 
>> +++ b/drivers/gpu/drm/gud/gud_pipe.c
> ..
>> +int gud_pipe_check(struct drm_simple_display_pipe *pipe,
> ..
>> +	req = kzalloc(len, GFP_KERNEL);
>> +	if (!req)
>> +		return -ENOMEM;
>> +
>> +	gud_from_display_mode(&req->mode, mode);
>> +
>> +	req->format = gud_from_fourcc(format->format);
>> +	if (WARN_ON_ONCE(!req->format))
>> +		return -EINVAL;
> 
> req leaks?
> 

Yep, you're right.

> 
>> +	req->connector = drm_connector_index(connector);
>> +	req->num_properties = num_properties;
>> +
>> +	num_properties = gud_connector_fill_properties(connector, connector_state,
>> +						       req->properties);
> 
> Following this new assignment to num_properties the new value is used
> to (hopefully!) append at the first req->properties[] index after the
> old value was used as index, that doesn't feel great..
> 
> I mean, it's harmless as long as gud_connector_fill_properties() is
> sure to return the same value, but, well, maybe sometime later it
> doesn't, or is that guaranteed if there is no error? Then maybe at
> least document that requirement by the function. What do you think?
> 

The number of properties doesn't change, it is returned by
GUD_REQ_GET_CONNECTOR during probe.
There is a small comment in gud_connector_fill_properties() but I can
expand on that and put it before the function definition.

Thanks for your scrutiny.

Noralf.
Peter Stuge Feb. 25, 2021, 9:58 a.m. UTC | #3
Hi Noralf,

Noralf Trønnes wrote:
> The driver supports a one bit monochrome transfer format: R1. This is not
> implemented in the gadget driver. It is added in preparation for future
> monochrome e-ink displays.

I forgot, but I have a two-tone (black/red) e-ink display here, and I
also have a 3-bpp RGB TFT display.

Should we add maybe R2 and R3? (or R3/R8 for number of colours?)

I'm particularly considering the 3-bpp RGB panel for GUD use now, and
while it will surely work with say a 16-bit RGB mode many bits will
be wasted in the process.

What are your thoughts? Would you take a patch for that now, later, never?


//Peter
Noralf Trønnes Feb. 25, 2021, 6:06 p.m. UTC | #4
Den 25.02.2021 10.58, skrev Peter Stuge:
> Hi Noralf,
> 
> Noralf Trønnes wrote:
>> The driver supports a one bit monochrome transfer format: R1. This is not
>> implemented in the gadget driver. It is added in preparation for future
>> monochrome e-ink displays.
> 
> I forgot, but I have a two-tone (black/red) e-ink display here, and I
> also have a 3-bpp RGB TFT display.
> 
> Should we add maybe R2 and R3? (or R3/R8 for number of colours?)
> 
> I'm particularly considering the 3-bpp RGB panel for GUD use now, and
> while it will surely work with say a 16-bit RGB mode many bits will
> be wasted in the process.
> 
> What are your thoughts? Would you take a patch for that now, later, never?
> 

I've been anticipating the need for more formats, but I didn't want to
add them without having a user. Otherwise I could end up adding stuff
that would never be used. If you can test, there's no problem adding
support for more formats now.

The R1 name is derived from DRM_FORMAT_R8 which is a 8 bit monochrome
(or one color channel) format.

Linux has these one byte color pixel formats currently defined:

/* color index */
#define DRM_FORMAT_C8		fourcc_code('C', '8', ' ', ' ') /* [7:0] C */

/* 8 bpp Red */
#define DRM_FORMAT_R8		fourcc_code('R', '8', ' ', ' ') /* [7:0] R */

/* 8 bpp RGB */
#define DRM_FORMAT_RGB332	fourcc_code('R', 'G', 'B', '8') /* [7:0] R:G:B
3:3:2 */
#define DRM_FORMAT_BGR233	fourcc_code('B', 'G', 'R', '8') /* [7:0] B:G:R
2:3:3 */

And these two color formats:

/* 16 bpp RG */
#define DRM_FORMAT_RG88		fourcc_code('R', 'G', '8', '8') /* [15:0] R:G
8:8 little endian */
#define DRM_FORMAT_GR88		fourcc_code('G', 'R', '8', '8') /* [15:0] G:R
8:8 little endian */

/* 32 bpp RG */
#define DRM_FORMAT_RG1616	fourcc_code('R', 'G', '3', '2') /* [31:0] R:G
16:16 little endian */
#define DRM_FORMAT_GR1616	fourcc_code('G', 'R', '3', '2') /* [31:0] G:R
16:16 little endian */


Building on that I would define a 2 bpp RG format like this in the driver:

static const struct drm_format_info gud_drm_format_rg11 = {
	.format = GUD_DRM_FORMAT_RG11,
	.num_planes = 1,
	.char_per_block = { 1, 0, 0 },
	.block_w = { 4, 0, 0 }, /* 4 pixels per block/byte */
	.block_h = { 1, 0, 0 },
	.hsub = 1,
	.vsub = 1,
};

And a 3 bpp RGB format like this:

static const struct drm_format_info gud_drm_format_rgb111 = {
	.format = GUD_DRM_FORMAT_RGB111,
	.num_planes = 1,
	.char_per_block = { 1, 0, 0 },
	.block_w = { 2, 0, 0 }, /* 2 pixels per block/byte */
	.block_h = { 1, 0, 0 },
	.hsub = 1,
	.vsub = 1,
};

The MIPI DBI standard defines 2 ways to transmit 2x 3-bpp pixels in one
byte (X=pad bit):
- Option 1: X  X  R1 G1 B1 R2 G2 B2
- Option 2: X  R1 G1 B1 X  R2 G2 B2

So maybe we should have GUD_DRM_FORMAT_RGB111_OPTION1 and
GUD_DRM_FORMAT_RGB111_OPTION2?
Or just use option 2 and let the display fix it up if needed?

What format does your 3 bpp display use?

And then something like this for the conversion function:

static size_t gud_xrgb8888_to_color(u8 *dst, const struct
drm_format_info *format,
				    u32 *src, struct drm_framebuffer *fb,
				    struct drm_rect *rect)
{
	unsigned int block_width = drm_format_info_block_width(format, 0);
	unsigned int x, y, width, height;
	u8 r, g, b, *block = dst; /* Assign to silence compiler warning */
	size_t len;

	WARN_ON_ONCE(format->char_per_block[0] != 1);

	/* Start on a byte boundary */
	rect->x1 = ALIGN_DOWN(rect->x1, block_width);
	width = drm_rect_width(rect);
	height = drm_rect_height(rect);
	len = drm_format_info_min_pitch(format, 0, width) * height;

	for (y = 0; y < height; y++) {
		for (x = 0; x < width; x++) {
			if (!(x % block_width)) {
				block = dst++;
				*block = 0;
			}

			/* r,g,b are bytes so no need to mask out anything explicitly */
			r = *src >> 16;
			g = *src >> 8;
			b = *src++;

			switch (format->format) {
			case GUD_DRM_FORMAT_RG11:
				*block <<= 2;
				*block |= ((r >> 7) << 1) | (g >> 7);
				break;
			case GUD_DRM_FORMAT_RGB111_OPTION1:
				*block <<= 3;
				*block |= ((r >> 7) << 2) | ((g >> 7) << 1) | (b >> 7);
				break;
			case GUD_DRM_FORMAT_RGB111_OPTION2:
				*block <<= 4;
				*block |= ((r >> 7) << 2) | ((g >> 7) << 1) | (b >> 7);
				break;
			default:
				WARN_ON_ONCE(1);
				return len;
			};
		}
	}

	return len;
}

Noralf.
Peter Stuge Feb. 25, 2021, 9:37 p.m. UTC | #5
Noralf Trønnes wrote:
> > I forgot, but I have a two-tone (black/red) e-ink display here, and I
> > also have a 3-bpp RGB TFT display.
> 
> I've been anticipating the need for more formats, but I didn't want to
> add them without having a user. Otherwise I could end up adding stuff
> that would never be used. If you can test, there's no problem adding
> support for more formats now.

That's great! I can't promise to to test the three-color (b/w/r) e-ink
but I'll prepare a test setup for the RGB-TFT on the weekend.


> Building on that I would define a 2 bpp RG format like this in the driver:
> 
> static const struct drm_format_info gud_drm_format_rg11 = {
> 	.format = GUD_DRM_FORMAT_RG11,
> 	.num_planes = 1,
> 	.char_per_block = { 1, 0, 0 },
> 	.block_w = { 4, 0, 0 }, /* 4 pixels per block/byte */
> 	.block_h = { 1, 0, 0 },
> 	.hsub = 1,
> 	.vsub = 1,
> };
> 
> And a 3 bpp RGB format like this:
> 
> static const struct drm_format_info gud_drm_format_rgb111 = {
> 	.format = GUD_DRM_FORMAT_RGB111,
> 	.num_planes = 1,
> 	.char_per_block = { 1, 0, 0 },
> 	.block_w = { 2, 0, 0 }, /* 2 pixels per block/byte */
> 	.block_h = { 1, 0, 0 },
> 	.hsub = 1,
> 	.vsub = 1,
> };

I can't really comment; I know next to nothing about the drm subsystem. :)


> The MIPI DBI standard defines 2 ways to transmit 2x 3-bpp pixels in one
> byte (X=pad bit):
> - Option 1: X  X  R1 G1 B1 R2 G2 B2
> - Option 2: X  R1 G1 B1 X  R2 G2 B2
> 
> So maybe we should have GUD_DRM_FORMAT_RGB111_OPTION1 and
> GUD_DRM_FORMAT_RGB111_OPTION2?
> Or just use option 2 and let the display fix it up if needed?

It would of course be lovely to be able to set up an automated DMA from
a USB endpoint to the panel in the device and not have to touch the data,
but that would require the DRM driver to support all the combinations,
which quickly becomes complicated.


> What format does your 3 bpp display use?

It supports three formats:

- R1 G1 B1 R2 G2 B2
- R1 G1 B1 X R2 G2 B2 X (your option 2)
- R1 R2 R3 (simulated monochrome, same data bit to all three subpixels)


> And then something like this for the conversion function:
> 
> static size_t gud_xrgb8888_to_color(u8 *dst, const struct

All right!


//Peter
Noralf Trønnes Feb. 26, 2021, 12:09 p.m. UTC | #6
Den 20.02.2021 18.27, skrev Noralf Trønnes:
> 
> 
> Den 19.02.2021 22.42, skrev Peter Stuge:

>>
>> More generally it's not very typical in USB to report the data size
>> separately from the data itself, if reporting size explicitly at all.
>>
>> Sizes can be part of the data structure itself (like in descriptors) but
>> on the application layer (like here) it's convenient to just decide a
>> sensible fixed maximum size and let the host try to always transfer
>> that size while accepting short transfers. Unlike read() a short
>> transfer only ever happens if and when a device intends for it,
>> so that's like an in-band handshake but "for free".
>>
>> Oh, and does/should the GUD EDID change if the panel "behind" the device
>> CPU on a hotpluggable connector changes? It wouldn't be great to require
>> GUD driver reprobe in that case. But maybe DRM requires that anyway?
>>
> 
> If gud_connector_status_req.status has changed since last poll or
> GUD_CONNECTOR_STATUS_CHANGED is set, DRM will notify userspace which
> will reprobe the connector. connector->epoch_counter++ in
> gud_connector_status_request() triggers that.
> 
>>
>> I'm sorry I didn't spot this pattern earlier, I understand that it's late
>> in the game and that changing it needs the gadget to change as well, but I
>> do really think this is a worthwhile change throughout the protocol.
>>
> 
> I see what you mean, I'll give it a try.
> 

Peter, please have a look at this diff and see if I'm on the right track
here: https://gist.github.com/notro/a43a93a3aa0cc75d930890b7b254fc0a

I want to avoid waisting a patch version cycle by being way off.

Noralf.
Noralf Trønnes Feb. 26, 2021, 12:18 p.m. UTC | #7
Den 25.02.2021 22.37, skrev Peter Stuge:
> Noralf Trønnes wrote:
>>> I forgot, but I have a two-tone (black/red) e-ink display here, and I
>>> also have a 3-bpp RGB TFT display.
>>
>> I've been anticipating the need for more formats, but I didn't want to
>> add them without having a user. Otherwise I could end up adding stuff
>> that would never be used. If you can test, there's no problem adding
>> support for more formats now.
> 
> That's great! I can't promise to to test the three-color (b/w/r) e-ink
> but I'll prepare a test setup for the RGB-TFT on the weekend.
> 

Ok, I'll add GUD_DRM_FORMAT_RGB111 in the next patch version.

Noralf.
Peter Stuge Feb. 28, 2021, 1:52 a.m. UTC | #8
Noralf Trønnes wrote:
> Peter, please have a look at this diff and see if I'm on the right track
> here: https://gist.github.com/notro/a43a93a3aa0cc75d930890b7b254fc0a

Yes that's exactly what I meant; this way the possibility for contradicting
sizes is eliminated by protocol and not just by implementation - very nice!

Some more comments, sorry if this is just because of ongoing work:

Perhaps the functions taking usb_device + ifnum could take usb_interface
instead - but I don't know if that would simplify or complicate things.
Alan mentioned this idea in similar circumstances in another thread.
I don't feel strongly, but perhaps it's cleaner.

gud_usb_control_msg() now seems almost redundant, maybe it could be removed.

In gud_usb_set() if NULL == buf then that's passed to usb_control_msg()
along with len, which likely crashes if len > 0, so it may be good to
check or enforce that, maybe with else len=0; before the gud_usb_transfer()
call.

Finally a small style note that I'd personally change a few if (ret > 0) {
blocks to have one indent level less and do each check right away, e.g. in
gud_connector_get_modes():

ret = gud_usb_get()
if (ret % EDID_LENGTH) {
	drm_err();
} else if (ret > 0) {
	edid_ctx.len = ret;
	edid = drm_do_get_edid();
}

and later on in the function by the display modes one indent level
could be saved with a goto:

if (ret <= 0)
	goto out;

but obviously no huge deal.


In general it's really helpful for device development to see error messages
when the device behaves incorrectly, the "Invalid .. size" errors are great
examples of this, but e.g. gud_get_display_descriptor() returns -EIO without
a message. Maybe there are opportunities for further helpful error messages?


Thanks a lot and kind regards

//Peter
Noralf Trønnes Feb. 28, 2021, 9:04 p.m. UTC | #9
Den 28.02.2021 02.52, skrev Peter Stuge:
> Noralf Trønnes wrote:
>> Peter, please have a look at this diff and see if I'm on the right track
>> here: https://gist.github.com/notro/a43a93a3aa0cc75d930890b7b254fc0a
> 
> Yes that's exactly what I meant; this way the possibility for contradicting
> sizes is eliminated by protocol and not just by implementation - very nice!
> 
> Some more comments, sorry if this is just because of ongoing work:
> 
> Perhaps the functions taking usb_device + ifnum could take usb_interface
> instead - but I don't know if that would simplify or complicate things.
> Alan mentioned this idea in similar circumstances in another thread.
> I don't feel strongly, but perhaps it's cleaner.
> 

I agree it's cleaner, this way I don't have to store the interface
number in gdrm.

> gud_usb_control_msg() now seems almost redundant, maybe it could be removed.
> 

There are 4 callers so I think it makes sense still.

> In gud_usb_set() if NULL == buf then that's passed to usb_control_msg()
> along with len, which likely crashes if len > 0, so it may be good to
> check or enforce that, maybe with else len=0; before the gud_usb_transfer()
> call.
> 

Ok.

> Finally a small style note that I'd personally change a few if (ret > 0) {
> blocks to have one indent level less and do each check right away, e.g. in
> gud_connector_get_modes():
> 
> ret = gud_usb_get()
> if (ret % EDID_LENGTH) {
> 	drm_err();
> } else if (ret > 0) {
> 	edid_ctx.len = ret;
> 	edid = drm_do_get_edid();
> }
> 
> and later on in the function by the display modes one indent level
> could be saved with a goto:
> 
> if (ret <= 0)
> 	goto out;
> 
> but obviously no huge deal.
> 

It makes for a better read so I'll do that.

> 
> In general it's really helpful for device development to see error messages
> when the device behaves incorrectly, the "Invalid .. size" errors are great
> examples of this, but e.g. gud_get_display_descriptor() returns -EIO without
> a message. Maybe there are opportunities for further helpful error messages?
> 

The message is printed by the caller:

	ret = gud_get_display_descriptor(intf, &desc);
	if (ret) {
		DRM_DEV_DEBUG_DRIVER(dev, "Not a display interface: ret=%d\n", ret);
		return -ENODEV;
	}

It's a debug message enabled by writing to /sys/module/drm/parameters/debug.
The reason for not making it an error message, is that I want the driver
to just ignore non-display vendor class interfaces so they can co-exist
on the device. Someone might make an open protocol gpio (vendor class)
interface driver some day, or adc, i2c, spi, rtc, or...

Thanks,
Noralf.
Peter Stuge March 1, 2021, 6:31 p.m. UTC | #10
Hi Noralf,

Peter Stuge wrote:
> I'll prepare a test setup for the RGB-TFT on the weekend.

So implementing a GUD and looking at the protocol from yet another
angle gives more new insights - surprise. :)

Here are some thoughts so far:

* GUD_REQ_SET_VERSION does still rub me wrong; it seems potentially
  quite complex to maintain compatibility in two places; both for host
  and device. I don't want to insist on removing it, but at a minimum
  it's quite unusual.
  Part of the idea in USB is that host software updates are easy if
  not fully automated but device firmware updates less so, thus
  complexity is rather placed in the host.

* It's unclear to me from reading the header files in this v6 patch set
  which GUD_REQ:s and which properties are actually mandatory in devices.
  I'm getting hints from your STM32 device and reading the driver code in
  detail, but what do you think is a good way to document what's required
  vs. optional?

* GUD_REQ_SET_BUFFER my old nemesis. :) It's great that it's optional!
  But do you see any way to turn it into a bulk message, in order to
  remove the memory barrier effect of a control transfer before bulk?

I think it would be possible to noticeably improve performance later,
by changing the host driver to submit asynchronous bulk transfers for
frame data rather than waiting for each transfer to finish; bulk
transfers will then pack optimally on the wire - but with a control
transfer in between there's no chance of achieving that.

Having only one kind of transfer in the hot path would also simplify
canceling still pending transfers (when using async later) if new data
gets flushed before the previous frame is completely transfered.

* A fair bit of the EDID isn't used or has dummy values. Have you already
  considered and dismissed some other ways of accomplishing the same?

* Sorry if I've asked before - but what's the purpose of
  GUD_REQ_SET_STATE_CHECK and GUD_REQ_SET_STATE_COMMIT? Why/when does
  drm do pipe check vs. update?

* How do you feel about passing the parameters for
  GUD_REQ_SET_CONTROLLER_ENABLE and GUD_REQ_SET_DISPLAY_ENABLE in wValue?
  It would save the transfer data stage.


Kind regards

//Peter
Noralf Trønnes March 1, 2021, 9:41 p.m. UTC | #11
Den 01.03.2021 19.31, skrev Peter Stuge:
> Hi Noralf,
> 
> Peter Stuge wrote:
>> I'll prepare a test setup for the RGB-TFT on the weekend.
> 
> So implementing a GUD and looking at the protocol from yet another
> angle gives more new insights - surprise. :)
> 

Yep, my Raspberry Pi Pico implementation discovered a bug in the
compression code. Running X on a 240x135 display produced some 2x2
buffer rectangles which was so small that they failed to compress.

> Here are some thoughts so far:
> 
> * GUD_REQ_SET_VERSION does still rub me wrong; it seems potentially
>   quite complex to maintain compatibility in two places; both for host
>   and device. I don't want to insist on removing it, but at a minimum
>   it's quite unusual.
>   Part of the idea in USB is that host software updates are easy if
>   not fully automated but device firmware updates less so, thus
>   complexity is rather placed in the host.
> 

Alright, I'll remove it :) This can also be accomplished in the device
by having a switch that turns it into a version 1 device.

> * It's unclear to me from reading the header files in this v6 patch set
>   which GUD_REQ:s and which properties are actually mandatory in devices.
>   I'm getting hints from your STM32 device and reading the driver code in
>   detail, but what do you think is a good way to document what's required
>   vs. optional?
> 

The device must respond to all requests, but can return zero on some
like the property requests. I intend to do a writeup on the wiki later
with some more details around the protocol and how DRM works for this
usecase.

My RPi Pico implementation will also serve as a guide. I've put the
protocol handling in one file for possible reuse:
- gud.h: https://gist.github.com/notro/26bbf918fa59fb89caf155d51d57a40f
- gud.c: https://gist.github.com/notro/c1b32cea591f84d3d1c94f30812c1ba6

I will publish it when I have cleaned up the rest of the code.

> * GUD_REQ_SET_BUFFER my old nemesis. :) It's great that it's optional!
>   But do you see any way to turn it into a bulk message, in order to
>   remove the memory barrier effect of a control transfer before bulk?
> 

On a 1920x1080 RGB565 full update (4MB), the control request is
negligible. I did measure it (USB 2.0), but I don't remember the
numbers, a few milliseconds.

> I think it would be possible to noticeably improve performance later,
> by changing the host driver to submit asynchronous bulk transfers for
> frame data rather than waiting for each transfer to finish; bulk
> transfers will then pack optimally on the wire - but with a control
> transfer in between there's no chance of achieving that.
> 
> Having only one kind of transfer in the hot path would also simplify
> canceling still pending transfers (when using async later) if new data
> gets flushed before the previous frame is completely transfered.
> 

The device is the bottle neck (unless it's powerful like a Pi4):

Host: compresses buffer
Host: sends ctrl transfer
Host: sends bulk transfer
Device: Decompresses buffer, much slower than a desktop CPU
Host: compresses buffer
Host: sends ctrl transfer, waits for device to respond
Device: Has decompressed, and is ready
Host: sends bulk transfer

I would have preferred to do a test on a USB 3.0 device which would put
more stress on the host, but I couldn't find an affordable one with
mainline Linux support.

As you say this can be optimized later. By using double buffering it's
possible to compress the next buffer while the previous is in transit.

lz4 compression has worked really well in my testing, always 2x or better.

I will reconsider the ctrl req if you provide me with numbers that show
it's a performance problem.

> * A fair bit of the EDID isn't used or has dummy values. Have you already
>   considered and dismissed some other ways of accomplishing the same?
> 

EDID is optional (return zero in v7), but useful if you want userspace
to give a name to the monitor in the GUI for instance.

This comment will be present in version 7:

/*
 * Display modes can be fetched as either EDID data or an array of
&gud_display_mode_req.
 *
 * If GUD_REQ_GET_CONNECTOR_MODES returns zero, EDID is used to create
display modes.
 * If both display modes and EDID are returned, EDID is just passed on
to userspace
 * in the EDID connector property.
 */

> * Sorry if I've asked before - but what's the purpose of
>   GUD_REQ_SET_STATE_CHECK and GUD_REQ_SET_STATE_COMMIT? Why/when does
>   drm do pipe check vs. update?
> 

DRM atomic mode setting has 2 stages:
- Check to see if the mode is supported, return -EINVAL if not
- Commit the mode, this is not allowed to fail.

Userspace can also do just the check phase when figuring out which
combination of options that is supported.

> * How do you feel about passing the parameters for
>   GUD_REQ_SET_CONTROLLER_ENABLE and GUD_REQ_SET_DISPLAY_ENABLE in wValue?
>   It would save the transfer data stage.
> 

That would make the request different form the others, wValue is only
used by the connector requests for the index. I prefer to use the data
stage for data. If there was any performance gain I would have looked
into it, but these requests are only used on init and when turning
on/off the display.

Noralf.
diff mbox series

Patch

diff --git a/MAINTAINERS b/MAINTAINERS
index 00112c044608..e7f71ac55f08 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5525,6 +5525,14 @@  S:	Maintained
 F:	Documentation/devicetree/bindings/display/panel/feiyang,fy07024di26a30d.yaml
 F:	drivers/gpu/drm/panel/panel-feiyang-fy07024di26a30d.c
 
+DRM DRIVER FOR GENERIC USB DISPLAY
+M:	Noralf Trønnes <noralf@tronnes.org>
+S:	Maintained
+W:	https://github.com/notro/gud/wiki
+T:	git git://anongit.freedesktop.org/drm/drm-misc
+F:	drivers/gpu/drm/gud/
+F:	include/drm/gud.h
+
 DRM DRIVER FOR GRAIN MEDIA GM12U320 PROJECTORS
 M:	Hans de Goede <hdegoede@redhat.com>
 S:	Maintained
diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
index 147d61b9674e..6d3f47933e51 100644
--- a/drivers/gpu/drm/Kconfig
+++ b/drivers/gpu/drm/Kconfig
@@ -388,6 +388,8 @@  source "drivers/gpu/drm/tidss/Kconfig"
 
 source "drivers/gpu/drm/xlnx/Kconfig"
 
+source "drivers/gpu/drm/gud/Kconfig"
+
 # Keep legacy drivers last
 
 menuconfig DRM_LEGACY
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 81569009f884..78dd8e12525d 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -124,3 +124,4 @@  obj-$(CONFIG_DRM_ASPEED_GFX) += aspeed/
 obj-$(CONFIG_DRM_MCDE) += mcde/
 obj-$(CONFIG_DRM_TIDSS) += tidss/
 obj-y			+= xlnx/
+obj-y			+= gud/
diff --git a/drivers/gpu/drm/gud/Kconfig b/drivers/gpu/drm/gud/Kconfig
new file mode 100644
index 000000000000..1c8601bf4d91
--- /dev/null
+++ b/drivers/gpu/drm/gud/Kconfig
@@ -0,0 +1,14 @@ 
+# SPDX-License-Identifier: GPL-2.0
+
+config DRM_GUD
+	tristate "GUD USB Display"
+	depends on DRM && USB
+	select LZ4_COMPRESS
+	select DRM_KMS_HELPER
+	select DRM_GEM_SHMEM_HELPER
+	select BACKLIGHT_CLASS_DEVICE
+	help
+	  This is a DRM display driver for GUD USB Displays or display
+	  adapters.
+
+	  If M is selected the module will be called gud.
diff --git a/drivers/gpu/drm/gud/Makefile b/drivers/gpu/drm/gud/Makefile
new file mode 100644
index 000000000000..68a1c622cf33
--- /dev/null
+++ b/drivers/gpu/drm/gud/Makefile
@@ -0,0 +1,4 @@ 
+# SPDX-License-Identifier: GPL-2.0
+
+gud-y				:= gud_drv.o gud_pipe.o gud_connector.o
+obj-$(CONFIG_DRM_GUD)		+= gud.o
diff --git a/drivers/gpu/drm/gud/gud_connector.c b/drivers/gpu/drm/gud/gud_connector.c
new file mode 100644
index 000000000000..64f323152976
--- /dev/null
+++ b/drivers/gpu/drm/gud/gud_connector.c
@@ -0,0 +1,738 @@ 
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright 2020 Noralf Trønnes
+ */
+
+#include <linux/backlight.h>
+#include <linux/workqueue.h>
+
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_state_helper.h>
+#include <drm/drm_connector.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_encoder.h>
+#include <drm/drm_file.h>
+#include <drm/drm_modeset_helper_vtables.h>
+#include <drm/drm_print.h>
+#include <drm/drm_probe_helper.h>
+#include <drm/drm_simple_kms_helper.h>
+#include <drm/gud.h>
+
+#include "gud_internal.h"
+
+struct gud_connector {
+	struct drm_connector connector;
+	struct drm_encoder encoder;
+	struct backlight_device *backlight;
+	struct work_struct backlight_work;
+
+	/* Supported properties */
+	u16 *properties;
+	unsigned int num_properties;
+
+	/* Initial gadget tv state if applicable, applied on state reset */
+	struct drm_tv_connector_state initial_tv_state;
+
+	/*
+	 * Initial gadget backlight brightness if applicable, applied on state reset.
+	 * The value -ENODEV is used to signal no backlight.
+	 */
+	int initial_brightness;
+
+	unsigned int num_modes;
+	size_t edid_len;
+};
+
+static inline struct gud_connector *to_gud_connector(struct drm_connector *connector)
+{
+	return container_of(connector, struct gud_connector, connector);
+}
+
+/*
+ * Use a worker to avoid taking kms locks inside the backlight lock.
+ * Other display drivers use backlight within their kms locks.
+ * This avoids inconsistent locking rules, which would upset lockdep.
+ */
+static void gud_connector_backlight_update_status_work(struct work_struct *work)
+{
+	struct gud_connector *gconn = container_of(work, struct gud_connector, backlight_work);
+	struct drm_connector *connector = &gconn->connector;
+	struct drm_connector_state *connector_state;
+	struct drm_device *drm = connector->dev;
+	struct drm_modeset_acquire_ctx ctx;
+	struct drm_atomic_state *state;
+	int idx, ret;
+
+	if (!drm_dev_enter(drm, &idx))
+		return;
+
+	state = drm_atomic_state_alloc(drm);
+	if (!state) {
+		ret = -ENOMEM;
+		goto exit;
+	}
+
+	drm_modeset_acquire_init(&ctx, 0);
+	state->acquire_ctx = &ctx;
+retry:
+	connector_state = drm_atomic_get_connector_state(state, connector);
+	if (IS_ERR(connector_state)) {
+		ret = PTR_ERR(connector_state);
+		goto out;
+	}
+
+	/* Reuse tv.brightness to avoid having to subclass */
+	connector_state->tv.brightness = gconn->backlight->props.brightness;
+
+	ret = drm_atomic_commit(state);
+out:
+	if (ret == -EDEADLK) {
+		drm_atomic_state_clear(state);
+		drm_modeset_backoff(&ctx);
+		goto retry;
+	}
+
+	drm_atomic_state_put(state);
+
+	drm_modeset_drop_locks(&ctx);
+	drm_modeset_acquire_fini(&ctx);
+exit:
+	drm_dev_exit(idx);
+
+	if (ret)
+		dev_err(drm->dev, "Failed to update backlight, err=%d\n", ret);
+}
+
+static int gud_connector_backlight_update_status(struct backlight_device *bd)
+{
+	struct drm_connector *connector = bl_get_data(bd);
+	struct gud_connector *gconn = to_gud_connector(connector);
+
+	/* The USB timeout is 5 seconds so use system_long_wq for worst case scenario */
+	queue_work(system_long_wq, &gconn->backlight_work);
+
+	return 0;
+}
+
+static const struct backlight_ops gud_connector_backlight_ops = {
+	.update_status	= gud_connector_backlight_update_status,
+};
+
+static int gud_connector_backlight_register(struct gud_connector *gconn)
+{
+	struct drm_connector *connector = &gconn->connector;
+	struct backlight_device *bd;
+	const char *name;
+	const struct backlight_properties props = {
+		.type = BACKLIGHT_RAW,
+		.scale = BACKLIGHT_SCALE_NON_LINEAR,
+		.max_brightness = 100,
+		.brightness = gconn->initial_brightness,
+	};
+
+	name = kasprintf(GFP_KERNEL, "card%d-%s-backlight",
+			 connector->dev->primary->index, connector->name);
+	if (!name)
+		return -ENOMEM;
+
+	bd = backlight_device_register(name, connector->kdev, connector,
+				       &gud_connector_backlight_ops, &props);
+	kfree(name);
+	if (IS_ERR(bd))
+		return PTR_ERR(bd);
+
+	gconn->backlight = bd;
+
+	return 0;
+}
+
+static int gud_connector_status_request(struct drm_connector *connector)
+{
+	struct gud_connector *gconn = to_gud_connector(connector);
+	struct gud_device *gdrm = to_gud_device(connector->dev);
+	struct gud_connector_status_req req;
+	u16 num_modes, edid_len;
+	int ret;
+
+	ret = gud_usb_get(gdrm, GUD_REQ_GET_CONNECTOR_STATUS,
+			  connector->index, &req, sizeof(req));
+	if (ret)
+		return ret;
+
+	switch (req.status & GUD_CONNECTOR_STATUS_CONNECTED_MASK) {
+	case GUD_CONNECTOR_STATUS_DISCONNECTED:
+		ret = connector_status_disconnected;
+		break;
+	case GUD_CONNECTOR_STATUS_CONNECTED:
+		ret = connector_status_connected;
+		break;
+	default:
+		ret = connector_status_unknown;
+		break;
+	};
+
+	num_modes = le16_to_cpu(req.num_modes);
+	edid_len = le16_to_cpu(req.edid_len);
+
+	if (edid_len % EDID_LENGTH) {
+		drm_err(connector->dev, "%s: Invalid EDID size: %u\n", connector->name, edid_len);
+		edid_len = 0;
+	}
+
+	if (req.status & GUD_CONNECTOR_STATUS_CHANGED ||
+	    gconn->num_modes != num_modes || gconn->edid_len != edid_len)
+		connector->epoch_counter += 1;
+
+	gconn->num_modes = num_modes;
+	gconn->edid_len = edid_len;
+
+	if (!num_modes && !edid_len && ret != connector_status_disconnected)
+		drm_dbg_kms(connector->dev, "%s: No modes or EDID.\n", connector->name);
+
+	return ret;
+}
+
+static int gud_connector_detect(struct drm_connector *connector,
+				struct drm_modeset_acquire_ctx *ctx, bool force)
+{
+	struct gud_device *gdrm = to_gud_device(connector->dev);
+	int idx, ret;
+
+	if (!drm_dev_enter(connector->dev, &idx))
+		return connector_status_disconnected;
+
+	if (force) {
+		ret = gud_usb_set(gdrm, GUD_REQ_SET_CONNECTOR_FORCE_DETECT,
+				  connector->index, NULL, 0);
+		if (ret) {
+			ret = connector_status_unknown;
+			goto exit;
+		}
+	}
+
+	ret = gud_connector_status_request(connector);
+	if (ret < 0)
+		ret = connector_status_unknown;
+exit:
+	drm_dev_exit(idx);
+
+	return ret;
+}
+
+struct gud_connector_get_edid_ctx {
+	struct gud_connector *gconn;
+	void *buf;
+};
+
+static int gud_connector_get_edid_block(void *data, u8 *buf, unsigned int block, size_t len)
+{
+	struct gud_connector_get_edid_ctx *ctx = data;
+	struct gud_connector *gconn = ctx->gconn;
+	size_t start = block * EDID_LENGTH;
+
+	if (start + len > gconn->edid_len)
+		return -1;
+
+	if (!block) {
+		struct gud_device *gdrm = to_gud_device(gconn->connector.dev);
+		int ret;
+
+		/* Check because drm_do_get_edid() will retry on failure */
+		if (!ctx->buf)
+			ctx->buf = kmalloc(gconn->edid_len, GFP_KERNEL);
+		if (!ctx->buf)
+			return -1;
+
+		ret = gud_usb_get(gdrm, GUD_REQ_GET_CONNECTOR_EDID, gconn->connector.index,
+				  ctx->buf, gconn->edid_len);
+		if (ret)
+			return -1;
+	}
+
+	memcpy(buf, ctx->buf + start, len);
+
+	return 0;
+}
+
+static int gud_connector_get_modes(struct drm_connector *connector)
+{
+	struct gud_connector *gconn = to_gud_connector(connector);
+	struct gud_device *gdrm = to_gud_device(connector->dev);
+	struct gud_connector_get_edid_ctx edid_ctx = {
+		.gconn = gconn,
+	};
+	struct gud_display_mode_req *reqmodes = NULL;
+	unsigned int i, num_modes = 0;
+	struct edid *edid = NULL;
+	bool edid_override;
+	int idx, ret;
+
+	if (!drm_dev_enter(connector->dev, &idx))
+		return 0;
+
+	if (connector->force) {
+		ret = gud_connector_status_request(connector);
+		if (ret < 0)
+			goto out;
+	}
+
+	edid = drm_do_get_edid(connector, gud_connector_get_edid_block, &edid_ctx);
+	edid_override = edid && !edid_ctx.buf;
+	kfree(edid_ctx.buf);
+	drm_connector_update_edid_property(connector, edid);
+
+	if (!gconn->num_modes || edid_override) {
+		num_modes = drm_add_edid_modes(connector, edid);
+		goto out;
+	}
+
+	reqmodes = kmalloc_array(gconn->num_modes, sizeof(*reqmodes), GFP_KERNEL);
+	if (!reqmodes)
+		goto out;
+
+	ret = gud_usb_get(gdrm, GUD_REQ_GET_CONNECTOR_MODES, connector->index,
+			  reqmodes, gconn->num_modes * sizeof(*reqmodes));
+	if (ret)
+		goto out;
+
+	for (i = 0; i < gconn->num_modes; i++) {
+		struct drm_display_mode *mode;
+
+		mode = drm_mode_create(connector->dev);
+		if (!mode)
+			goto out;
+
+		gud_to_display_mode(mode, &reqmodes[i]);
+		drm_mode_probed_add(connector, mode);
+		num_modes++;
+	}
+out:
+	kfree(reqmodes);
+	kfree(edid);
+	drm_dev_exit(idx);
+
+	return num_modes;
+}
+
+static int gud_connector_atomic_check(struct drm_connector *connector,
+				      struct drm_atomic_state *state)
+{
+	struct drm_connector_state *new_state;
+	struct drm_crtc_state *new_crtc_state;
+	struct drm_connector_state *old_state;
+
+	new_state = drm_atomic_get_new_connector_state(state, connector);
+	if (!new_state->crtc)
+		return 0;
+
+	old_state = drm_atomic_get_old_connector_state(state, connector);
+	new_crtc_state = drm_atomic_get_new_crtc_state(state, new_state->crtc);
+
+	if (old_state->tv.margins.left != new_state->tv.margins.left ||
+	    old_state->tv.margins.right != new_state->tv.margins.right ||
+	    old_state->tv.margins.top != new_state->tv.margins.top ||
+	    old_state->tv.margins.bottom != new_state->tv.margins.bottom ||
+	    old_state->tv.mode != new_state->tv.mode ||
+	    old_state->tv.brightness != new_state->tv.brightness ||
+	    old_state->tv.contrast != new_state->tv.contrast ||
+	    old_state->tv.flicker_reduction != new_state->tv.flicker_reduction ||
+	    old_state->tv.overscan != new_state->tv.overscan ||
+	    old_state->tv.saturation != new_state->tv.saturation ||
+	    old_state->tv.hue != new_state->tv.hue)
+		new_crtc_state->connectors_changed = true;
+
+	return 0;
+}
+
+static const struct drm_connector_helper_funcs gud_connector_helper_funcs = {
+	.detect_ctx = gud_connector_detect,
+	.get_modes = gud_connector_get_modes,
+	.atomic_check = gud_connector_atomic_check,
+};
+
+static int gud_connector_late_register(struct drm_connector *connector)
+{
+	struct gud_connector *gconn = to_gud_connector(connector);
+
+	if (gconn->initial_brightness < 0)
+		return 0;
+
+	return gud_connector_backlight_register(gconn);
+}
+
+static void gud_connector_early_unregister(struct drm_connector *connector)
+{
+	struct gud_connector *gconn = to_gud_connector(connector);
+
+	backlight_device_unregister(gconn->backlight);
+	cancel_work_sync(&gconn->backlight_work);
+}
+
+static void gud_connector_destroy(struct drm_connector *connector)
+{
+	struct gud_connector *gconn = to_gud_connector(connector);
+
+	drm_connector_cleanup(connector);
+	kfree(gconn->properties);
+	kfree(gconn);
+}
+
+static void gud_connector_reset(struct drm_connector *connector)
+{
+	struct gud_connector *gconn = to_gud_connector(connector);
+
+	drm_atomic_helper_connector_reset(connector);
+	connector->state->tv = gconn->initial_tv_state;
+	/* Set margins from command line */
+	drm_atomic_helper_connector_tv_reset(connector);
+	if (gconn->initial_brightness >= 0)
+		connector->state->tv.brightness = gconn->initial_brightness;
+}
+
+static const struct drm_connector_funcs gud_connector_funcs = {
+	.fill_modes = drm_helper_probe_single_connector_modes,
+	.late_register = gud_connector_late_register,
+	.early_unregister = gud_connector_early_unregister,
+	.destroy = gud_connector_destroy,
+	.reset = gud_connector_reset,
+	.atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+	.atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+/*
+ * The tv.mode property is shared among the connectors and its enum names are
+ * driver specific. This means that if more than one connector uses tv.mode,
+ * the enum names has to be the same.
+ */
+static int gud_connector_add_tv_mode(struct gud_device *gdrm,
+				     struct drm_connector *connector, u64 val)
+{
+	unsigned int i, num_modes;
+	const char **modes;
+	size_t buf_len;
+	char *buf;
+	int ret;
+
+	num_modes = val >> GUD_CONNECTOR_TV_MODE_NUM_SHIFT;
+
+	if (!num_modes)
+		return -EINVAL;
+
+	buf_len = num_modes * GUD_CONNECTOR_TV_MODE_NAME_LEN;
+	modes = kmalloc_array(num_modes, sizeof(*modes), GFP_KERNEL);
+	buf = kmalloc(buf_len, GFP_KERNEL);
+	if (!modes || !buf) {
+		ret = -ENOMEM;
+		goto free;
+	}
+
+	ret = gud_usb_get(gdrm, GUD_REQ_GET_CONNECTOR_TV_MODE_VALUES,
+			  connector->index, buf, buf_len);
+	if (ret)
+		goto free;
+
+	for (i = 0; i < num_modes; i++)
+		modes[i] = &buf[i * GUD_CONNECTOR_TV_MODE_NAME_LEN];
+
+	ret = drm_mode_create_tv_properties(connector->dev, num_modes, modes);
+free:
+	kfree(modes);
+	kfree(buf);
+
+	return ret;
+}
+
+static struct drm_property *
+gud_connector_property_lookup(struct drm_connector *connector, u16 prop)
+{
+	struct drm_mode_config *config = &connector->dev->mode_config;
+
+	switch (prop) {
+	case GUD_PROPERTY_TV_LEFT_MARGIN:
+		return config->tv_left_margin_property;
+	case GUD_PROPERTY_TV_RIGHT_MARGIN:
+		return config->tv_right_margin_property;
+	case GUD_PROPERTY_TV_TOP_MARGIN:
+		return config->tv_top_margin_property;
+	case GUD_PROPERTY_TV_BOTTOM_MARGIN:
+		return config->tv_bottom_margin_property;
+	case GUD_PROPERTY_TV_MODE:
+		return config->tv_mode_property;
+	case GUD_PROPERTY_TV_BRIGHTNESS:
+		return config->tv_brightness_property;
+	case GUD_PROPERTY_TV_CONTRAST:
+		return config->tv_contrast_property;
+	case GUD_PROPERTY_TV_FLICKER_REDUCTION:
+		return config->tv_flicker_reduction_property;
+	case GUD_PROPERTY_TV_OVERSCAN:
+		return config->tv_overscan_property;
+	case GUD_PROPERTY_TV_SATURATION:
+		return config->tv_saturation_property;
+	case GUD_PROPERTY_TV_HUE:
+		return config->tv_hue_property;
+	default:
+		return ERR_PTR(-EINVAL);
+	}
+}
+
+static unsigned int *gud_connector_tv_state_val(u16 prop, struct drm_tv_connector_state *state)
+{
+	switch (prop) {
+	case GUD_PROPERTY_TV_LEFT_MARGIN:
+		return &state->margins.left;
+	case GUD_PROPERTY_TV_RIGHT_MARGIN:
+		return &state->margins.right;
+	case GUD_PROPERTY_TV_TOP_MARGIN:
+		return &state->margins.top;
+	case GUD_PROPERTY_TV_BOTTOM_MARGIN:
+		return &state->margins.bottom;
+	case GUD_PROPERTY_TV_MODE:
+		return &state->mode;
+	case GUD_PROPERTY_TV_BRIGHTNESS:
+		return &state->brightness;
+	case GUD_PROPERTY_TV_CONTRAST:
+		return &state->contrast;
+	case GUD_PROPERTY_TV_FLICKER_REDUCTION:
+		return &state->flicker_reduction;
+	case GUD_PROPERTY_TV_OVERSCAN:
+		return &state->overscan;
+	case GUD_PROPERTY_TV_SATURATION:
+		return &state->saturation;
+	case GUD_PROPERTY_TV_HUE:
+		return &state->hue;
+	default:
+		return ERR_PTR(-EINVAL);
+	}
+}
+
+static int gud_connector_add_properties(struct gud_device *gdrm, struct gud_connector *gconn,
+					unsigned int num_properties)
+{
+	struct drm_device *drm = &gdrm->drm;
+	struct drm_connector *connector = &gconn->connector;
+	struct gud_property_req *properties;
+	unsigned int i;
+	int ret;
+
+	gconn->properties = kcalloc(num_properties, sizeof(*gconn->properties), GFP_KERNEL);
+	if (!gconn->properties)
+		return -ENOMEM;
+
+	properties = kcalloc(num_properties, sizeof(*properties), GFP_KERNEL);
+	if (!properties)
+		return -ENOMEM;
+
+	ret = gud_usb_get(gdrm, GUD_REQ_GET_CONNECTOR_PROPERTIES, connector->index,
+			  properties, num_properties * sizeof(*properties));
+	if (ret)
+		goto out;
+
+	for (i = 0; i < num_properties; i++) {
+		u16 prop = le16_to_cpu(properties[i].prop);
+		u64 val = le64_to_cpu(properties[i].val);
+		struct drm_property *property;
+		unsigned int *state_val;
+
+		drm_dbg(drm, "property: %u = %llu(0x%llx)\n", prop, val, val);
+
+		switch (prop) {
+		case GUD_PROPERTY_TV_LEFT_MARGIN:
+			fallthrough;
+		case GUD_PROPERTY_TV_RIGHT_MARGIN:
+			fallthrough;
+		case GUD_PROPERTY_TV_TOP_MARGIN:
+			fallthrough;
+		case GUD_PROPERTY_TV_BOTTOM_MARGIN:
+			ret = drm_mode_create_tv_margin_properties(drm);
+			if (ret)
+				goto out;
+			break;
+		case GUD_PROPERTY_TV_MODE:
+			ret = gud_connector_add_tv_mode(gdrm, connector, val);
+			if (ret)
+				goto out;
+			val = val & (BIT(GUD_CONNECTOR_TV_MODE_NUM_SHIFT) - 1);
+			break;
+		case GUD_PROPERTY_TV_BRIGHTNESS:
+			fallthrough;
+		case GUD_PROPERTY_TV_CONTRAST:
+			fallthrough;
+		case GUD_PROPERTY_TV_FLICKER_REDUCTION:
+			fallthrough;
+		case GUD_PROPERTY_TV_OVERSCAN:
+			fallthrough;
+		case GUD_PROPERTY_TV_SATURATION:
+			fallthrough;
+		case GUD_PROPERTY_TV_HUE:
+			/* This is a no-op if already added. */
+			ret = drm_mode_create_tv_properties(drm, 0, NULL);
+			if (ret)
+				goto out;
+			break;
+		case GUD_PROPERTY_BACKLIGHT_BRIGHTNESS:
+			if (val > 100) {
+				ret = -EINVAL;
+				goto out;
+			}
+			gconn->initial_brightness = val;
+			break;
+		default:
+			/* New ones might show up in future devices, skip those we don't know. */
+			drm_dbg(drm, "Unknown property: %u\n", prop);
+			continue;
+		}
+
+		gconn->properties[gconn->num_properties++] = prop;
+
+		if (prop == GUD_PROPERTY_BACKLIGHT_BRIGHTNESS)
+			continue; /* not a DRM property */
+
+		property = gud_connector_property_lookup(connector, prop);
+		if (WARN_ON(IS_ERR(property)))
+			continue;
+
+		state_val = gud_connector_tv_state_val(prop, &gconn->initial_tv_state);
+		if (WARN_ON(IS_ERR(state_val)))
+			continue;
+
+		*state_val = val;
+		drm_object_attach_property(&connector->base, property, 0);
+	}
+out:
+	kfree(properties);
+
+	return ret;
+}
+
+int gud_connector_fill_properties(struct drm_connector *connector,
+				  struct drm_connector_state *connector_state,
+				  struct gud_property_req *properties)
+{
+	struct gud_connector *gconn;
+	unsigned int i;
+
+	gconn = to_gud_connector(connector);
+
+	/* Only interested in the count? */
+	if (!connector_state)
+		return gconn->num_properties;
+
+	for (i = 0; i < gconn->num_properties; i++) {
+		u16 prop = gconn->properties[i];
+		u64 val;
+
+		if (prop == GUD_PROPERTY_BACKLIGHT_BRIGHTNESS) {
+			val = connector_state->tv.brightness;
+		} else {
+			unsigned int *state_val;
+
+			state_val = gud_connector_tv_state_val(prop, &connector_state->tv);
+			if (WARN_ON_ONCE(IS_ERR(state_val)))
+				return PTR_ERR(state_val);
+
+			val = *state_val;
+		}
+
+		properties[i].prop = cpu_to_le16(prop);
+		properties[i].val = cpu_to_le64(val);
+	}
+
+	return gconn->num_properties;
+}
+
+int gud_connector_create(struct gud_device *gdrm, unsigned int index)
+{
+	struct gud_connector_descriptor_req desc;
+	struct drm_device *drm = &gdrm->drm;
+	struct gud_connector *gconn;
+	struct drm_connector *connector;
+	struct drm_encoder *encoder;
+	int ret, connector_type;
+	u32 flags;
+
+	ret = gud_usb_get(gdrm, GUD_REQ_GET_CONNECTOR, index, &desc, sizeof(desc));
+	if (ret)
+		return ret;
+
+	gconn = kzalloc(sizeof(*gconn), GFP_KERNEL);
+	if (!gconn)
+		return -ENOMEM;
+
+	INIT_WORK(&gconn->backlight_work, gud_connector_backlight_update_status_work);
+	gconn->initial_brightness = -ENODEV;
+	flags = le32_to_cpu(desc.flags);
+	connector = &gconn->connector;
+
+	drm_dbg(drm, "Connector: index=%u type=%u flags=0x%x num_properties=%u\n",
+		index, desc.connector_type, flags, desc.num_properties);
+
+	switch (desc.connector_type) {
+	case GUD_CONNECTOR_TYPE_PANEL:
+		connector_type = DRM_MODE_CONNECTOR_USB;
+		break;
+	case GUD_CONNECTOR_TYPE_VGA:
+		connector_type = DRM_MODE_CONNECTOR_VGA;
+		break;
+	case GUD_CONNECTOR_TYPE_DVI:
+		connector_type = DRM_MODE_CONNECTOR_DVID;
+		break;
+	case GUD_CONNECTOR_TYPE_COMPOSITE:
+		connector_type = DRM_MODE_CONNECTOR_Composite;
+		break;
+	case GUD_CONNECTOR_TYPE_SVIDEO:
+		connector_type = DRM_MODE_CONNECTOR_SVIDEO;
+		break;
+	case GUD_CONNECTOR_TYPE_COMPONENT:
+		connector_type = DRM_MODE_CONNECTOR_Component;
+		break;
+	case GUD_CONNECTOR_TYPE_DISPLAYPORT:
+		connector_type = DRM_MODE_CONNECTOR_DisplayPort;
+		break;
+	case GUD_CONNECTOR_TYPE_HDMI:
+		connector_type = DRM_MODE_CONNECTOR_HDMIA;
+		break;
+	default: /* future types */
+		connector_type = DRM_MODE_CONNECTOR_USB;
+		break;
+	};
+
+	drm_connector_helper_add(connector, &gud_connector_helper_funcs);
+	ret = drm_connector_init(drm, connector, &gud_connector_funcs, connector_type);
+	if (ret) {
+		kfree(connector);
+		return ret;
+	}
+
+	if (WARN_ON(connector->index != index))
+		return -EINVAL;
+
+	if (flags & GUD_CONNECTOR_FLAGS_POLL_STATUS)
+		connector->polled = (DRM_CONNECTOR_POLL_CONNECT | DRM_CONNECTOR_POLL_DISCONNECT);
+	if (flags & GUD_CONNECTOR_FLAGS_INTERLACE)
+		connector->interlace_allowed = true;
+	if (flags & GUD_CONNECTOR_FLAGS_DOUBLESCAN)
+		connector->doublescan_allowed = true;
+
+	if (desc.num_properties) {
+		ret = gud_connector_add_properties(gdrm, gconn, desc.num_properties);
+		if (ret) {
+			dev_err(drm->dev, "Failed to add connector/%u properties\n", index);
+			return ret;
+		}
+	}
+
+	/* The first connector is attached to the existing simple pipe encoder */
+	if (!connector->index) {
+		encoder = &gdrm->pipe.encoder;
+	} else {
+		encoder = &gconn->encoder;
+
+		ret = drm_simple_encoder_init(drm, encoder, DRM_MODE_ENCODER_NONE);
+		if (ret)
+			return ret;
+
+		encoder->possible_crtcs = 1;
+	}
+
+	return drm_connector_attach_encoder(connector, encoder);
+}
diff --git a/drivers/gpu/drm/gud/gud_drv.c b/drivers/gpu/drm/gud/gud_drv.c
new file mode 100644
index 000000000000..8f0a68d65769
--- /dev/null
+++ b/drivers/gpu/drm/gud/gud_drv.c
@@ -0,0 +1,625 @@ 
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright 2020 Noralf Trønnes
+ */
+
+#include <linux/dma-buf.h>
+#include <linux/dma-mapping.h>
+#include <linux/lz4.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/string_helpers.h>
+#include <linux/usb.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_damage_helper.h>
+#include <drm/drm_debugfs.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_fb_helper.h>
+#include <drm/drm_fourcc.h>
+#include <drm/drm_gem_shmem_helper.h>
+#include <drm/drm_gem_framebuffer_helper.h>
+#include <drm/drm_managed.h>
+#include <drm/drm_print.h>
+#include <drm/drm_probe_helper.h>
+#include <drm/drm_simple_kms_helper.h>
+#include <drm/gud.h>
+
+#include "gud_internal.h"
+
+/* Only used internally */
+static const struct drm_format_info gud_drm_format_r1 = {
+	.format = GUD_DRM_FORMAT_R1,
+	.num_planes = 1,
+	.char_per_block = { 1, 0, 0 },
+	.block_w = { 8, 0, 0 },
+	.block_h = { 1, 0, 0 },
+	.hsub = 1,
+	.vsub = 1,
+};
+
+static int gud_usb_control_msg(struct usb_device *usb, u8 ifnum, bool in,
+			       u8 request, u16 value, void *buf, size_t len)
+{
+	u8 requesttype = USB_TYPE_VENDOR | USB_RECIP_INTERFACE;
+	unsigned int pipe;
+	int ret;
+
+	if (in) {
+		pipe = usb_rcvctrlpipe(usb, 0);
+		requesttype |= USB_DIR_IN;
+	} else {
+		pipe = usb_sndctrlpipe(usb, 0);
+		requesttype |= USB_DIR_OUT;
+	}
+
+	ret = usb_control_msg(usb, pipe, request, requesttype, value,
+			      ifnum, buf, len, USB_CTRL_GET_TIMEOUT);
+	if (ret < 0)
+		return ret;
+	if (ret != len)
+		return -EIO;
+
+	return 0;
+}
+
+static int gud_get_display_descriptor(struct usb_interface *interface,
+				      struct gud_display_descriptor_req *desc)
+{
+	u8 ifnum = interface->cur_altsetting->desc.bInterfaceNumber;
+	struct usb_device *usb = interface_to_usbdev(interface);
+	void *buf;
+	int ret;
+
+	buf = kmalloc(sizeof(*desc), GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	ret = gud_usb_control_msg(usb, ifnum, true, GUD_REQ_GET_DESCRIPTOR, 0, buf, sizeof(*desc));
+	memcpy(desc, buf, sizeof(*desc));
+	kfree(buf);
+	if (ret)
+		return ret;
+
+	if (desc->magic != le32_to_cpu(GUD_DISPLAY_MAGIC))
+		return -ENODATA;
+
+	DRM_DEV_DEBUG_DRIVER(&interface->dev,
+			     "version=%u flags=0x%x compression=0x%x num_formats=%u num_connectors=%u max_buffer_size=%u\n",
+			     desc->version, le32_to_cpu(desc->flags), desc->compression,
+			     desc->num_formats, desc->num_connectors,
+			     le32_to_cpu(desc->max_buffer_size));
+
+	if (!desc->version || !desc->num_formats || !desc->num_connectors ||
+	    !desc->max_width || !desc->max_height ||
+	    le32_to_cpu(desc->min_width) > le32_to_cpu(desc->max_width) ||
+	    le32_to_cpu(desc->min_height) > le32_to_cpu(desc->max_height))
+		return -EINVAL;
+
+	return 0;
+}
+
+static int gud_usb_get_status(struct usb_device *usb, u8 ifnum, u8 *status)
+{
+	u8 *buf;
+	int ret;
+
+	buf = kmalloc(sizeof(*buf), GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	ret = gud_usb_control_msg(usb, ifnum, true, GUD_REQ_GET_STATUS, 0, buf, sizeof(*buf));
+	*status = *buf;
+	kfree(buf);
+
+	return ret;
+}
+
+static int gud_status_to_errno(u8 status)
+{
+	switch (status) {
+	case GUD_STATUS_OK:
+		return 0;
+	case GUD_STATUS_BUSY:
+		return -EBUSY;
+	case GUD_STATUS_REQUEST_NOT_SUPPORTED:
+		return -EOPNOTSUPP;
+	case GUD_STATUS_PROTOCOL_ERROR:
+		return -EPROTO;
+	case GUD_STATUS_INVALID_PARAMETER:
+		return -EINVAL;
+	case GUD_STATUS_ERROR:
+		return -EREMOTEIO;
+	default:
+		return -EREMOTEIO;
+	}
+}
+
+static int gud_usb_transfer(struct gud_device *gdrm, bool in, u8 request, u16 index,
+			    void *buf, size_t len)
+{
+	struct usb_device *usb = gud_to_usb_device(gdrm);
+	void *trbuf = NULL;
+	int idx, ret;
+
+	drm_dbg(&gdrm->drm, "%s: request=0x%x index=%u len=%zu\n",
+		in ? "get" : "set", request, index, len);
+
+	if (!drm_dev_enter(&gdrm->drm, &idx))
+		return -ENODEV;
+
+	mutex_lock(&gdrm->ctrl_lock);
+
+	if (buf) {
+		if (in)
+			trbuf = kmalloc(len, GFP_KERNEL);
+		else
+			trbuf = kmemdup(buf, len, GFP_KERNEL);
+		if (!trbuf) {
+			ret = -ENOMEM;
+			goto unlock;
+		}
+	}
+
+	ret = gud_usb_control_msg(usb, gdrm->ifnum, in, request, index, trbuf, len);
+	if (ret == -EPIPE || (!ret && !in && (gdrm->flags & GUD_DISPLAY_FLAG_STATUS_ON_SET))) {
+		bool error = ret;
+		u8 status;
+
+		ret = gud_usb_get_status(usb, gdrm->ifnum, &status);
+		if (!ret) {
+			if (error && status == GUD_STATUS_OK) {
+				dev_err_once(gdrm->drm.dev,
+					     "Unexpected status OK for failed transfer\n");
+				ret = -EPIPE;
+			} else {
+				ret = gud_status_to_errno(status);
+			}
+		}
+	}
+
+	if (!ret && in && buf)
+		memcpy(buf, trbuf, len);
+
+	if (ret) {
+		drm_dbg(&gdrm->drm, "ret=%d\n", ret);
+		gdrm->stats_num_errors++;
+	}
+
+	kfree(trbuf);
+unlock:
+	mutex_unlock(&gdrm->ctrl_lock);
+	drm_dev_exit(idx);
+
+	return ret;
+}
+
+int gud_usb_get(struct gud_device *gdrm, u8 request, u16 index, void *buf, size_t len)
+{
+	return gud_usb_transfer(gdrm, true, request, index, buf, len);
+}
+
+int gud_usb_set(struct gud_device *gdrm, u8 request, u16 index, void *buf, size_t len)
+{
+	return gud_usb_transfer(gdrm, false, request, index, buf, len);
+}
+
+int gud_usb_write8(struct gud_device *gdrm, u8 request, u8 val)
+{
+	return gud_usb_set(gdrm, request, 0, &val, sizeof(val));
+}
+
+static int gud_set_version(struct usb_device *usb, u8 ifnum, u32 flags, u8 version)
+{
+	u8 *buf;
+	int ret;
+
+	buf = kmalloc(sizeof(*buf), GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	*buf = version;
+	ret = gud_usb_control_msg(usb, ifnum, false, GUD_REQ_SET_VERSION, 0, buf, sizeof(*buf));
+	kfree(buf);
+	if (ret == -EPIPE)
+		return -EPROTONOSUPPORT;
+	if (ret)
+		return ret;
+
+	if (flags & GUD_DISPLAY_FLAG_STATUS_ON_SET) {
+		u8 status;
+
+		ret = gud_usb_get_status(usb, ifnum, &status);
+		if (!ret && status != GUD_STATUS_OK)
+			ret = -EPROTONOSUPPORT;
+	}
+
+	return ret;
+}
+
+static int gud_get_properties(struct gud_device *gdrm, unsigned int num_properties)
+{
+	struct gud_property_req *properties;
+	unsigned int i;
+	int ret;
+
+	if (!num_properties)
+		return 0;
+
+	gdrm->properties = drmm_kcalloc(&gdrm->drm, num_properties, sizeof(*gdrm->properties),
+					GFP_KERNEL);
+	if (!gdrm->properties)
+		return -ENOMEM;
+
+	properties = kcalloc(num_properties, sizeof(*properties), GFP_KERNEL);
+	if (!properties)
+		return -ENOMEM;
+
+	ret = gud_usb_get(gdrm, GUD_REQ_GET_PROPERTIES, 0,
+			  properties, num_properties * sizeof(*properties));
+	if (ret)
+		goto out;
+
+	for (i = 0; i < num_properties; i++) {
+		u16 prop = le16_to_cpu(properties[i].prop);
+		u64 val = le64_to_cpu(properties[i].val);
+
+		switch (prop) {
+		case GUD_PROPERTY_ROTATION:
+			/*
+			 * DRM UAPI matches the protocol so use the value directly,
+			 * but mask out any additions on future devices.
+			 */
+			val &= GUD_ROTATION_MASK;
+			ret = drm_plane_create_rotation_property(&gdrm->pipe.plane,
+								 DRM_MODE_ROTATE_0, val);
+			break;
+		default:
+			/* New ones might show up in future devices, skip those we don't know. */
+			drm_dbg(&gdrm->drm, "Unknown property: %u\n", prop);
+			continue;
+		}
+
+		if (ret)
+			goto out;
+
+		gdrm->properties[gdrm->num_properties++] = prop;
+	}
+out:
+	kfree(properties);
+
+	return ret;
+}
+
+static int gud_stats_debugfs(struct seq_file *m, void *data)
+{
+	struct drm_info_node *node = m->private;
+	struct gud_device *gdrm = to_gud_device(node->minor->dev);
+	char buf[10];
+
+	string_get_size(gdrm->bulk_len, 1, STRING_UNITS_2, buf, sizeof(buf));
+	seq_printf(m, "Max buffer size: %s\n", buf);
+	seq_printf(m, "Number of errors:  %u\n", gdrm->stats_num_errors);
+
+	seq_puts(m, "Compression:      ");
+	if (gdrm->compression & GUD_COMPRESSION_LZ4)
+		seq_puts(m, " lz4");
+	seq_puts(m, "\n");
+
+	if (gdrm->compression) {
+		u64 remainder;
+		u64 ratio = div64_u64_rem(gdrm->stats_length, gdrm->stats_actual_length,
+					  &remainder);
+		u64 ratio_frac = div64_u64(remainder * 10, gdrm->stats_actual_length);
+
+		seq_printf(m, "Compression ratio: %llu.%llu\n", ratio, ratio_frac);
+	}
+
+	return 0;
+}
+
+static const struct drm_info_list gud_debugfs_list[] = {
+	{ "stats", gud_stats_debugfs, 0, NULL },
+};
+
+static void gud_debugfs_init(struct drm_minor *minor)
+{
+	drm_debugfs_create_files(gud_debugfs_list, ARRAY_SIZE(gud_debugfs_list),
+				 minor->debugfs_root, minor);
+}
+
+static const struct drm_simple_display_pipe_funcs gud_pipe_funcs = {
+	.check      = gud_pipe_check,
+	.update	    = gud_pipe_update,
+	.prepare_fb = drm_gem_fb_simple_display_pipe_prepare_fb,
+};
+
+static const struct drm_mode_config_funcs gud_mode_config_funcs = {
+	.fb_create = drm_gem_fb_create_with_dirty,
+	.atomic_check = drm_atomic_helper_check,
+	.atomic_commit = drm_atomic_helper_commit,
+};
+
+static const u64 gud_pipe_modifiers[] = {
+	DRM_FORMAT_MOD_LINEAR,
+	DRM_FORMAT_MOD_INVALID
+};
+
+DEFINE_DRM_GEM_FOPS(gud_fops);
+
+static const struct drm_driver gud_drm_driver = {
+	.driver_features	= DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC,
+	.fops			= &gud_fops,
+	DRM_GEM_SHMEM_DRIVER_OPS,
+	.debugfs_init		= gud_debugfs_init,
+
+	.name			= "gud",
+	.desc			= "Generic USB Display",
+	.date			= "20200422",
+	.major			= 1,
+	.minor			= 0,
+};
+
+static void gud_free_buffers_and_mutex(struct drm_device *drm, void *unused)
+{
+	struct gud_device *gdrm = to_gud_device(drm);
+
+	vfree(gdrm->compress_buf);
+	kfree(gdrm->bulk_buf);
+	mutex_destroy(&gdrm->ctrl_lock);
+	mutex_destroy(&gdrm->damage_lock);
+}
+
+static int gud_probe(struct usb_interface *interface, const struct usb_device_id *id)
+{
+	u8 ifnum = interface->cur_altsetting->desc.bInterfaceNumber;
+	struct usb_device *usb = interface_to_usbdev(interface);
+	struct device *dev = &interface->dev;
+	const struct drm_format_info *xrgb8888_emulation_format = NULL;
+	bool rgb565_supported = false, xrgb8888_supported = false;
+	struct usb_endpoint_descriptor *bulk_out;
+	struct gud_display_descriptor_req desc;
+	unsigned int num_formats = 0;
+	struct gud_device *gdrm;
+	size_t max_buffer_size = 0;
+	struct drm_device *drm;
+	u8 *formats_dev;
+	u32 *formats;
+	int ret, i;
+
+	ret = usb_find_bulk_out_endpoint(interface->cur_altsetting, &bulk_out);
+	if (ret)
+		return ret;
+
+	ret = gud_get_display_descriptor(interface, &desc);
+	if (ret) {
+		DRM_DEV_DEBUG_DRIVER(dev, "Not a display interface: ret=%d\n", ret);
+		return -ENODEV;
+	}
+
+	if (desc.version > 1) {
+		ret = gud_set_version(usb, ifnum, le32_to_cpu(desc.flags), 1);
+		if (ret) {
+			if (ret == -EPROTONOSUPPORT)
+				dev_err(dev, "Protocol version %u is not supported\n",
+					desc.version);
+			return ret;
+		}
+	}
+
+	ret = dma_coerce_mask_and_coherent(dev, DMA_BIT_MASK(64));
+	if (ret)
+		dev_warn(dev, "Failed to set DMA mask\n");
+
+	gdrm = devm_drm_dev_alloc(dev, &gud_drm_driver, struct gud_device, drm);
+	if (IS_ERR(gdrm))
+		return PTR_ERR(gdrm);
+
+	drm = &gdrm->drm;
+	drm->mode_config.funcs = &gud_mode_config_funcs;
+	ret = drmm_mode_config_init(drm);
+	if (ret)
+		return ret;
+
+	gdrm->ifnum = ifnum;
+	gdrm->flags = le32_to_cpu(desc.flags);
+	gdrm->compression = desc.compression & GUD_COMPRESSION_LZ4;
+
+	if (gdrm->flags & GUD_DISPLAY_FLAG_FULL_UPDATE && gdrm->compression)
+		return -EINVAL;
+
+	mutex_init(&gdrm->ctrl_lock);
+	mutex_init(&gdrm->damage_lock);
+	INIT_WORK(&gdrm->work, gud_flush_work);
+	gud_clear_damage(gdrm);
+
+	ret = drmm_add_action_or_reset(drm, gud_free_buffers_and_mutex, NULL);
+	if (ret)
+		return ret;
+
+	drm->mode_config.min_width = le32_to_cpu(desc.min_width);
+	drm->mode_config.max_width = le32_to_cpu(desc.max_width);
+	drm->mode_config.min_height = le32_to_cpu(desc.min_height);
+	drm->mode_config.max_height = le32_to_cpu(desc.max_height);
+
+	formats_dev = devm_kmalloc(dev, desc.num_formats, GFP_KERNEL);
+	/* Add room for emulated XRGB8888 */
+	formats = devm_kmalloc_array(dev, desc.num_formats + 1, sizeof(*formats), GFP_KERNEL);
+	if (!formats_dev || !formats)
+		return -ENOMEM;
+
+	ret = gud_usb_get(gdrm, GUD_REQ_GET_FORMATS, 0, formats_dev, desc.num_formats);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < desc.num_formats; i++) {
+		const struct drm_format_info *info;
+		size_t fmt_buf_size;
+		u32 format;
+
+		format = gud_to_fourcc(formats_dev[i]);
+		if (!format) {
+			drm_dbg(drm, "Unsupported format: 0x%02x\n", formats_dev[i]);
+			continue;
+		}
+
+		if (format == GUD_DRM_FORMAT_R1)
+			info = &gud_drm_format_r1;
+		else
+			info = drm_format_info(format);
+
+		switch (format) {
+		case GUD_DRM_FORMAT_R1:
+			xrgb8888_emulation_format = info;
+			break;
+		case DRM_FORMAT_RGB565:
+			rgb565_supported = true;
+			if (!xrgb8888_emulation_format)
+				xrgb8888_emulation_format = info;
+			break;
+		case DRM_FORMAT_XRGB8888:
+			xrgb8888_supported = true;
+			break;
+		};
+
+		fmt_buf_size = drm_format_info_min_pitch(info, 0, drm->mode_config.max_width) *
+			       drm->mode_config.max_height;
+		max_buffer_size = max(max_buffer_size, fmt_buf_size);
+
+		if (format == GUD_DRM_FORMAT_R1)
+			continue; /* Internal not for userspace */
+
+		formats[num_formats++] = format;
+	}
+
+	if (!num_formats && !xrgb8888_emulation_format) {
+		dev_err(dev, "No supported pixel formats found\n");
+		return -EINVAL;
+	}
+
+	/* Prefer speed over color depth */
+	if (rgb565_supported)
+		drm->mode_config.preferred_depth = 16;
+
+	if (!xrgb8888_supported && xrgb8888_emulation_format) {
+		gdrm->xrgb8888_emulation_format = xrgb8888_emulation_format;
+		formats[num_formats++] = DRM_FORMAT_XRGB8888;
+	}
+
+	if (desc.max_buffer_size)
+		max_buffer_size = le32_to_cpu(desc.max_buffer_size);
+retry:
+	/*
+	 * Use plain kmalloc here since devm_kmalloc() places struct devres at the beginning
+	 * of the buffer it allocates. This wastes a lot of memory when allocating big buffers.
+	 * Asking for 2M would actually allocate 4M. This would also prevent getting the biggest
+	 * possible buffer potentially leading to split transfers.
+	 */
+	gdrm->bulk_buf = kmalloc(max_buffer_size, GFP_KERNEL | __GFP_NOWARN);
+	if (!gdrm->bulk_buf) {
+		max_buffer_size = roundup_pow_of_two(max_buffer_size) / 2;
+		if (max_buffer_size < SZ_512K)
+			return -ENOMEM;
+		goto retry;
+	}
+
+	gdrm->bulk_pipe = usb_sndbulkpipe(usb, usb_endpoint_num(bulk_out));
+	gdrm->bulk_len = max_buffer_size;
+
+	if (gdrm->compression & GUD_COMPRESSION_LZ4) {
+		gdrm->lz4_comp_mem = devm_kmalloc(dev, LZ4_MEM_COMPRESS, GFP_KERNEL);
+		if (!gdrm->lz4_comp_mem)
+			return -ENOMEM;
+
+		gdrm->compress_buf = vmalloc(gdrm->bulk_len);
+		if (!gdrm->compress_buf)
+			return -ENOMEM;
+	}
+
+	ret = drm_simple_display_pipe_init(drm, &gdrm->pipe, &gud_pipe_funcs,
+					   formats, num_formats,
+					   gud_pipe_modifiers, NULL);
+	if (ret)
+		return ret;
+
+	devm_kfree(dev, formats);
+	devm_kfree(dev, formats_dev);
+
+	ret = gud_get_properties(gdrm, desc.num_properties);
+	if (ret)
+		return ret;
+
+	drm_plane_enable_fb_damage_clips(&gdrm->pipe.plane);
+
+	for (i = 0; i < desc.num_connectors; i++) {
+		ret = gud_connector_create(gdrm, i);
+		if (ret)
+			return ret;
+	}
+
+	drm_mode_config_reset(drm);
+
+	usb_set_intfdata(interface, gdrm);
+
+	ret = drm_dev_register(drm, 0);
+	if (ret)
+		return ret;
+
+	drm_kms_helper_poll_init(drm);
+
+	drm_fbdev_generic_setup(drm, 0);
+
+	return 0;
+}
+
+static void gud_disconnect(struct usb_interface *interface)
+{
+	struct gud_device *gdrm = usb_get_intfdata(interface);
+	struct drm_device *drm = &gdrm->drm;
+
+	drm_dbg(drm, "%s:\n", __func__);
+
+	drm_kms_helper_poll_fini(drm);
+	drm_dev_unplug(drm);
+	drm_atomic_helper_shutdown(drm);
+}
+
+static int gud_suspend(struct usb_interface *interface, pm_message_t message)
+{
+	struct gud_device *gdrm = usb_get_intfdata(interface);
+
+	return drm_mode_config_helper_suspend(&gdrm->drm);
+}
+
+static int gud_resume(struct usb_interface *interface)
+{
+	struct gud_device *gdrm = usb_get_intfdata(interface);
+
+	drm_mode_config_helper_resume(&gdrm->drm);
+
+	return 0;
+}
+
+static const struct usb_device_id gud_id_table[] = {
+	{ USB_DEVICE_INTERFACE_CLASS(0x1d50, 0x614d, USB_CLASS_VENDOR_SPEC) },
+	{ }
+};
+
+MODULE_DEVICE_TABLE(usb, gud_id_table);
+
+static struct usb_driver gud_usb_driver = {
+	.name		= "gud",
+	.probe		= gud_probe,
+	.disconnect	= gud_disconnect,
+	.id_table	= gud_id_table,
+	.suspend	= gud_suspend,
+	.resume		= gud_resume,
+	.reset_resume	= gud_resume,
+};
+
+module_usb_driver(gud_usb_driver);
+
+MODULE_AUTHOR("Noralf Trønnes");
+MODULE_LICENSE("Dual MIT/GPL");
diff --git a/drivers/gpu/drm/gud/gud_internal.h b/drivers/gpu/drm/gud/gud_internal.h
new file mode 100644
index 000000000000..45cb2b6d5bd3
--- /dev/null
+++ b/drivers/gpu/drm/gud/gud_internal.h
@@ -0,0 +1,149 @@ 
+/* SPDX-License-Identifier: MIT */
+
+#ifndef __LINUX_GUD_INTERNAL_H
+#define __LINUX_GUD_INTERNAL_H
+
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/usb.h>
+#include <linux/workqueue.h>
+#include <uapi/drm/drm_fourcc.h>
+
+#include <drm/drm_modes.h>
+#include <drm/drm_simple_kms_helper.h>
+
+struct gud_device {
+	struct drm_device drm;
+	struct drm_simple_display_pipe pipe;
+	struct work_struct work;
+	u8 ifnum;
+	u32 flags;
+	const struct drm_format_info *xrgb8888_emulation_format;
+
+	u16 *properties;
+	unsigned int num_properties;
+
+	unsigned int bulk_pipe;
+	void *bulk_buf;
+	size_t bulk_len;
+
+	u8 compression;
+	void *lz4_comp_mem;
+	void *compress_buf;
+
+	u64 stats_length;
+	u64 stats_actual_length;
+	unsigned int stats_num_errors;
+
+	struct mutex ctrl_lock; /* Serialize get/set and status transfers */
+
+	struct mutex damage_lock; /* Protects the following members: */
+	struct drm_framebuffer *fb;
+	struct drm_rect damage;
+	bool prev_flush_failed;
+};
+
+static inline struct gud_device *to_gud_device(struct drm_device *drm)
+{
+	return container_of(drm, struct gud_device, drm);
+}
+
+static inline struct usb_device *gud_to_usb_device(struct gud_device *gdrm)
+{
+	return interface_to_usbdev(to_usb_interface(gdrm->drm.dev));
+}
+
+int gud_usb_get(struct gud_device *gdrm, u8 request, u16 index, void *buf, size_t len);
+int gud_usb_set(struct gud_device *gdrm, u8 request, u16 index, void *buf, size_t len);
+int gud_usb_write8(struct gud_device *gdrm, u8 request, u8 val);
+
+void gud_clear_damage(struct gud_device *gdrm);
+void gud_flush_work(struct work_struct *work);
+int gud_pipe_check(struct drm_simple_display_pipe *pipe,
+		   struct drm_plane_state *new_plane_state,
+		   struct drm_crtc_state *new_crtc_state);
+void gud_pipe_update(struct drm_simple_display_pipe *pipe,
+		     struct drm_plane_state *old_state);
+int gud_connector_fill_properties(struct drm_connector *connector,
+				  struct drm_connector_state *connector_state,
+				  struct gud_property_req *properties);
+int gud_connector_create(struct gud_device *gdrm, unsigned int index);
+
+/* Driver internal fourcc for 1-bit monochrome */
+#define GUD_DRM_FORMAT_R1	0x00000122
+
+static inline u8 gud_from_fourcc(u32 fourcc)
+{
+	switch (fourcc) {
+	case GUD_DRM_FORMAT_R1:
+		return GUD_PIXEL_FORMAT_R1;
+	case DRM_FORMAT_RGB565:
+		return GUD_PIXEL_FORMAT_RGB565;
+	case DRM_FORMAT_XRGB8888:
+		return GUD_PIXEL_FORMAT_XRGB8888;
+	case DRM_FORMAT_ARGB8888:
+		return GUD_PIXEL_FORMAT_ARGB8888;
+	};
+
+	return 0;
+}
+
+static inline u32 gud_to_fourcc(u8 format)
+{
+	switch (format) {
+	case GUD_PIXEL_FORMAT_R1:
+		return GUD_DRM_FORMAT_R1;
+	case GUD_PIXEL_FORMAT_RGB565:
+		return DRM_FORMAT_RGB565;
+	case GUD_PIXEL_FORMAT_XRGB8888:
+		return DRM_FORMAT_XRGB8888;
+	case GUD_PIXEL_FORMAT_ARGB8888:
+		return DRM_FORMAT_ARGB8888;
+	};
+
+	return 0;
+}
+
+static inline void gud_from_display_mode(struct gud_display_mode_req *dst,
+					 const struct drm_display_mode *src)
+{
+	u32 flags = src->flags & GUD_DISPLAY_MODE_FLAG_USER_MASK;
+
+	if (src->type & DRM_MODE_TYPE_PREFERRED)
+		flags |= GUD_DISPLAY_MODE_FLAG_PREFERRED;
+
+	dst->clock = cpu_to_le32(src->clock);
+	dst->hdisplay = cpu_to_le16(src->hdisplay);
+	dst->hsync_start = cpu_to_le16(src->hsync_start);
+	dst->hsync_end = cpu_to_le16(src->hsync_end);
+	dst->htotal = cpu_to_le16(src->htotal);
+	dst->vdisplay = cpu_to_le16(src->vdisplay);
+	dst->vsync_start = cpu_to_le16(src->vsync_start);
+	dst->vsync_end = cpu_to_le16(src->vsync_end);
+	dst->vtotal = cpu_to_le16(src->vtotal);
+	dst->flags = cpu_to_le32(flags);
+}
+
+static inline void gud_to_display_mode(struct drm_display_mode *dst,
+				       const struct gud_display_mode_req *src)
+{
+	u32 flags = le32_to_cpu(src->flags);
+
+	memset(dst, 0, sizeof(*dst));
+	dst->clock = le32_to_cpu(src->clock);
+	dst->hdisplay = le16_to_cpu(src->hdisplay);
+	dst->hsync_start = le16_to_cpu(src->hsync_start);
+	dst->hsync_end = le16_to_cpu(src->hsync_end);
+	dst->htotal = le16_to_cpu(src->htotal);
+	dst->vdisplay = le16_to_cpu(src->vdisplay);
+	dst->vsync_start = le16_to_cpu(src->vsync_start);
+	dst->vsync_end = le16_to_cpu(src->vsync_end);
+	dst->vtotal = le16_to_cpu(src->vtotal);
+	dst->flags = flags & GUD_DISPLAY_MODE_FLAG_USER_MASK;
+	dst->type = DRM_MODE_TYPE_DRIVER;
+	if (flags & GUD_DISPLAY_MODE_FLAG_PREFERRED)
+		dst->type |= DRM_MODE_TYPE_PREFERRED;
+	drm_mode_set_name(dst);
+}
+
+#endif
diff --git a/drivers/gpu/drm/gud/gud_pipe.c b/drivers/gpu/drm/gud/gud_pipe.c
new file mode 100644
index 000000000000..478b88e69ad3
--- /dev/null
+++ b/drivers/gpu/drm/gud/gud_pipe.c
@@ -0,0 +1,475 @@ 
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright 2020 Noralf Trønnes
+ */
+
+#include <linux/dma-buf.h>
+#include <linux/lz4.h>
+#include <linux/usb.h>
+#include <linux/workqueue.h>
+
+#include <drm/drm_atomic.h>
+#include <drm/drm_connector.h>
+#include <drm/drm_damage_helper.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_format_helper.h>
+#include <drm/drm_fourcc.h>
+#include <drm/drm_framebuffer.h>
+#include <drm/drm_gem_shmem_helper.h>
+#include <drm/drm_print.h>
+#include <drm/drm_rect.h>
+#include <drm/drm_simple_kms_helper.h>
+#include <drm/gud.h>
+
+#include "gud_internal.h"
+
+static bool gud_is_big_endian(void)
+{
+#if defined(__BIG_ENDIAN)
+	return true;
+#else
+	return false;
+#endif
+}
+
+static size_t gud_xrgb8888_to_r124(u8 *dst, const struct drm_format_info *format,
+				   void *src, struct drm_framebuffer *fb,
+				   struct drm_rect *rect)
+{
+	unsigned int block_width = drm_format_info_block_width(format, 0);
+	unsigned int bits_per_pixel = 8 / block_width;
+	unsigned int x, y, width, height;
+	u8 *p, *block = dst; /* Assign to silence compiler warning */
+	size_t len;
+	void *buf;
+
+	WARN_ON_ONCE(format->char_per_block[0] != 1);
+
+	/* Start on a byte boundary */
+	rect->x1 = ALIGN_DOWN(rect->x1, block_width);
+	width = drm_rect_width(rect);
+	height = drm_rect_height(rect);
+	len = drm_format_info_min_pitch(format, 0, width) * height;
+
+	buf = kmalloc(width * height, GFP_KERNEL);
+	if (!buf)
+		return 0;
+
+	drm_fb_xrgb8888_to_gray8(buf, src, fb, rect);
+
+	p = buf;
+	for (y = 0; y < drm_rect_height(rect); y++) {
+		for (x = 0; x < drm_rect_width(rect); x++) {
+			if (!(x % block_width)) {
+				block = dst++;
+				*block = 0;
+			}
+
+			*block <<= bits_per_pixel;
+			*block |= (*p++) >> (8 - bits_per_pixel);
+		}
+	}
+
+	kfree(buf);
+
+	return len;
+}
+
+static int gud_prep_flush(struct gud_device *gdrm, struct drm_framebuffer *fb,
+			  const struct drm_format_info *format, struct drm_rect *rect,
+			  struct gud_set_buffer_req *req)
+{
+	struct dma_buf_attachment *import_attach = fb->obj[0]->import_attach;
+	u8 compression = gdrm->compression;
+	struct dma_buf_map map;
+	void *vaddr, *buf;
+	size_t pitch, len;
+	int ret = 0;
+
+	pitch = drm_format_info_min_pitch(format, 0, drm_rect_width(rect));
+	len = pitch * drm_rect_height(rect);
+	if (len > gdrm->bulk_len)
+		return -E2BIG;
+
+	ret = drm_gem_shmem_vmap(fb->obj[0], &map);
+	if (ret)
+		return ret;
+
+	vaddr = map.vaddr + fb->offsets[0];
+
+	if (import_attach) {
+		ret = dma_buf_begin_cpu_access(import_attach->dmabuf, DMA_FROM_DEVICE);
+		if (ret)
+			goto vunmap;
+	}
+retry:
+	if (compression)
+		buf = gdrm->compress_buf;
+	else
+		buf = gdrm->bulk_buf;
+
+	/*
+	 * Imported buffers are assumed to be write-combined and thus uncached
+	 * with slow reads (at least on ARM).
+	 */
+	if (format != fb->format) {
+		if (format->format == GUD_DRM_FORMAT_R1) {
+			len = gud_xrgb8888_to_r124(buf, format, vaddr, fb, rect);
+			if (!len) {
+				ret = -ENOMEM;
+				goto end_cpu_access;
+			}
+		} else if (format->format == DRM_FORMAT_RGB565) {
+			drm_fb_xrgb8888_to_rgb565(buf, vaddr, fb, rect, gud_is_big_endian());
+		}
+	} else if (gud_is_big_endian() && format->cpp[0] > 1) {
+		drm_fb_swab(buf, vaddr, fb, rect, !import_attach);
+	} else if (compression && !import_attach && pitch == fb->pitches[0]) {
+		/* can compress directly from the framebuffer */
+		buf = vaddr + rect->y1 * pitch;
+	} else {
+		drm_fb_memcpy(buf, vaddr, fb, rect);
+	}
+
+	memset(req, 0, sizeof(*req));
+	req->x = cpu_to_le32(rect->x1);
+	req->y = cpu_to_le32(rect->y1);
+	req->width = cpu_to_le32(drm_rect_width(rect));
+	req->height = cpu_to_le32(drm_rect_height(rect));
+	req->length = cpu_to_le32(len);
+
+	if (compression & GUD_COMPRESSION_LZ4) {
+		int complen;
+
+		complen = LZ4_compress_default(buf, gdrm->bulk_buf, len, len, gdrm->lz4_comp_mem);
+		if (complen < 0) {
+			compression = 0;
+			goto retry;
+		}
+
+		req->compression = GUD_COMPRESSION_LZ4;
+		req->compressed_length = cpu_to_le32(complen);
+	}
+
+end_cpu_access:
+	if (import_attach)
+		dma_buf_end_cpu_access(import_attach->dmabuf, DMA_FROM_DEVICE);
+vunmap:
+	drm_gem_shmem_vunmap(fb->obj[0], &map);
+
+	return ret;
+}
+
+static int gud_flush_rect(struct gud_device *gdrm, struct drm_framebuffer *fb,
+			  const struct drm_format_info *format, struct drm_rect *rect)
+{
+	struct usb_device *usb = gud_to_usb_device(gdrm);
+	struct gud_set_buffer_req req;
+	int ret, actual_length;
+	size_t len, trlen;
+
+	drm_dbg(&gdrm->drm, "Flushing [FB:%d] " DRM_RECT_FMT "\n", fb->base.id, DRM_RECT_ARG(rect));
+
+	ret = gud_prep_flush(gdrm, fb, format, rect, &req);
+	if (ret)
+		return ret;
+
+	len = le32_to_cpu(req.length);
+
+	if (req.compression)
+		trlen = le32_to_cpu(req.compressed_length);
+	else
+		trlen = len;
+
+	gdrm->stats_length += len;
+	/* Did it wrap around? */
+	if (gdrm->stats_length <= len && gdrm->stats_actual_length) {
+		gdrm->stats_length = len;
+		gdrm->stats_actual_length = 0;
+	}
+	gdrm->stats_actual_length += trlen;
+
+	if (!(gdrm->flags & GUD_DISPLAY_FLAG_FULL_UPDATE) || gdrm->prev_flush_failed) {
+		ret = gud_usb_set(gdrm, GUD_REQ_SET_BUFFER, 0, &req, sizeof(req));
+		if (ret)
+			return ret;
+	}
+
+	ret = usb_bulk_msg(usb, gdrm->bulk_pipe, gdrm->bulk_buf, trlen,
+			   &actual_length, msecs_to_jiffies(3000));
+	if (!ret && trlen != actual_length)
+		ret = -EIO;
+	if (ret)
+		gdrm->stats_num_errors++;
+
+	return ret;
+}
+
+void gud_clear_damage(struct gud_device *gdrm)
+{
+	gdrm->damage.x1 = INT_MAX;
+	gdrm->damage.y1 = INT_MAX;
+	gdrm->damage.x2 = 0;
+	gdrm->damage.y2 = 0;
+}
+
+static void gud_add_damage(struct gud_device *gdrm, struct drm_rect *damage)
+{
+	gdrm->damage.x1 = min(gdrm->damage.x1, damage->x1);
+	gdrm->damage.y1 = min(gdrm->damage.y1, damage->y1);
+	gdrm->damage.x2 = max(gdrm->damage.x2, damage->x2);
+	gdrm->damage.y2 = max(gdrm->damage.y2, damage->y2);
+}
+
+static void gud_retry_failed_flush(struct gud_device *gdrm, struct drm_framebuffer *fb,
+				   struct drm_rect *damage)
+{
+	/*
+	 * pipe_update waits for the worker when the display mode is going to change.
+	 * This ensures that the width and height is still the same making it safe to
+	 * add back the damage.
+	 */
+
+	mutex_lock(&gdrm->damage_lock);
+	if (!gdrm->fb) {
+		drm_framebuffer_get(fb);
+		gdrm->fb = fb;
+	}
+	gud_add_damage(gdrm, damage);
+	mutex_unlock(&gdrm->damage_lock);
+
+	/* Retry only once to avoid a possible storm in case of continues errors. */
+	if (!gdrm->prev_flush_failed)
+		queue_work(system_long_wq, &gdrm->work);
+	gdrm->prev_flush_failed = true;
+}
+
+void gud_flush_work(struct work_struct *work)
+{
+	struct gud_device *gdrm = container_of(work, struct gud_device, work);
+	const struct drm_format_info *format;
+	struct drm_framebuffer *fb;
+	struct drm_rect damage;
+	unsigned int i, lines;
+	int idx, ret = 0;
+	size_t pitch;
+
+	if (!drm_dev_enter(&gdrm->drm, &idx))
+		return;
+
+	mutex_lock(&gdrm->damage_lock);
+	fb = gdrm->fb;
+	gdrm->fb = NULL;
+	damage = gdrm->damage;
+	gud_clear_damage(gdrm);
+	mutex_unlock(&gdrm->damage_lock);
+
+	if (!fb)
+		goto out;
+
+	format = fb->format;
+	if (format->format == DRM_FORMAT_XRGB8888 && gdrm->xrgb8888_emulation_format)
+		format = gdrm->xrgb8888_emulation_format;
+
+	/* Split update if it's too big */
+	pitch = drm_format_info_min_pitch(format, 0, drm_rect_width(&damage));
+	lines = drm_rect_height(&damage);
+
+	if (gdrm->bulk_len < lines * pitch)
+		lines = gdrm->bulk_len / pitch;
+
+	for (i = 0; i < DIV_ROUND_UP(drm_rect_height(&damage), lines); i++) {
+		struct drm_rect rect = damage;
+
+		rect.y1 += i * lines;
+		rect.y2 = min_t(u32, rect.y1 + lines, damage.y2);
+
+		ret = gud_flush_rect(gdrm, fb, format, &rect);
+		if (ret) {
+			if (ret != -ENODEV && ret != -ECONNRESET &&
+			    ret != -ESHUTDOWN && ret != -EPROTO) {
+				bool prev_flush_failed = gdrm->prev_flush_failed;
+
+				gud_retry_failed_flush(gdrm, fb, &damage);
+				if (!prev_flush_failed)
+					dev_err_ratelimited(fb->dev->dev,
+							    "Failed to flush framebuffer: error=%d\n", ret);
+			}
+			break;
+		}
+
+		gdrm->prev_flush_failed = false;
+	}
+
+	drm_framebuffer_put(fb);
+out:
+	drm_dev_exit(idx);
+}
+
+static void gud_fb_queue_damage(struct gud_device *gdrm, struct drm_framebuffer *fb,
+				struct drm_rect *damage)
+{
+	struct drm_framebuffer *old_fb = NULL;
+
+	mutex_lock(&gdrm->damage_lock);
+
+	if (fb != gdrm->fb) {
+		old_fb = gdrm->fb;
+		drm_framebuffer_get(fb);
+		gdrm->fb = fb;
+	}
+
+	gud_add_damage(gdrm, damage);
+
+	mutex_unlock(&gdrm->damage_lock);
+
+	queue_work(system_long_wq, &gdrm->work);
+
+	if (old_fb)
+		drm_framebuffer_put(old_fb);
+}
+
+int gud_pipe_check(struct drm_simple_display_pipe *pipe,
+		   struct drm_plane_state *new_plane_state,
+		   struct drm_crtc_state *new_crtc_state)
+{
+	struct gud_device *gdrm = to_gud_device(pipe->crtc.dev);
+	struct drm_plane_state *old_plane_state = pipe->plane.state;
+	const struct drm_display_mode *mode = &new_crtc_state->mode;
+	struct drm_atomic_state *state = new_plane_state->state;
+	struct drm_framebuffer *old_fb = old_plane_state->fb;
+	struct drm_connector_state *connector_state = NULL;
+	struct drm_framebuffer *fb = new_plane_state->fb;
+	const struct drm_format_info *format = fb->format;
+	struct drm_connector *connector;
+	int idx, ret, num_properties;
+	struct gud_state_req *req;
+	unsigned int i;
+	size_t len;
+
+	if (WARN_ON_ONCE(!fb))
+		return -EINVAL;
+
+	if (old_plane_state->rotation != new_plane_state->rotation)
+		new_crtc_state->mode_changed = true;
+
+	if (old_fb && old_fb->format != format)
+		new_crtc_state->mode_changed = true;
+
+	if (!new_crtc_state->mode_changed && !new_crtc_state->connectors_changed)
+		return 0;
+
+	/* Only one connector is supported */
+	if (hweight32(new_crtc_state->connector_mask) != 1)
+		return -EINVAL;
+
+	if (format->format == DRM_FORMAT_XRGB8888 && gdrm->xrgb8888_emulation_format)
+		format = gdrm->xrgb8888_emulation_format;
+
+	for_each_new_connector_in_state(state, connector, connector_state, i) {
+		if (connector_state->crtc)
+			break;
+	}
+
+	if (WARN_ON_ONCE(!connector_state))
+		return -ENOENT;
+
+	num_properties = gud_connector_fill_properties(connector, NULL, NULL);
+	if (num_properties < 0)
+		return num_properties;
+
+	num_properties += gdrm->num_properties;
+
+	len = struct_size(req, properties, num_properties);
+	req = kzalloc(len, GFP_KERNEL);
+	if (!req)
+		return -ENOMEM;
+
+	gud_from_display_mode(&req->mode, mode);
+
+	req->format = gud_from_fourcc(format->format);
+	if (WARN_ON_ONCE(!req->format))
+		return -EINVAL;
+
+	req->connector = drm_connector_index(connector);
+	req->num_properties = num_properties;
+
+	num_properties = gud_connector_fill_properties(connector, connector_state,
+						       req->properties);
+
+	for (i = 0; i < gdrm->num_properties; i++) {
+		u16 prop = gdrm->properties[i];
+		u64 val;
+
+		switch (prop) {
+		case GUD_PROPERTY_ROTATION:
+			/* DRM UAPI matches the protocol so use value directly */
+			val = new_plane_state->rotation;
+			break;
+		default:
+			WARN_ON_ONCE(1);
+			ret = -EINVAL;
+			goto out;
+		}
+
+		req->properties[num_properties + i].prop = cpu_to_le16(prop);
+		req->properties[num_properties + i].val = cpu_to_le64(val);
+	}
+
+	if (drm_dev_enter(fb->dev, &idx)) {
+		ret = gud_usb_set(gdrm, GUD_REQ_SET_STATE_CHECK, 0, req, len);
+		drm_dev_exit(idx);
+	}  else {
+		ret = -ENODEV;
+	}
+out:
+	kfree(req);
+
+	return ret;
+}
+
+void gud_pipe_update(struct drm_simple_display_pipe *pipe,
+		     struct drm_plane_state *old_state)
+{
+	struct drm_device *drm = pipe->crtc.dev;
+	struct gud_device *gdrm = to_gud_device(drm);
+	struct drm_plane_state *state = pipe->plane.state;
+	struct drm_framebuffer *fb = state->fb;
+	struct drm_crtc *crtc = &pipe->crtc;
+	struct drm_rect damage;
+	int idx;
+
+	if (crtc->state->mode_changed || !crtc->state->enable) {
+		cancel_work_sync(&gdrm->work);
+		mutex_lock(&gdrm->damage_lock);
+		if (gdrm->fb) {
+			drm_framebuffer_put(gdrm->fb);
+			gdrm->fb = NULL;
+		}
+		gud_clear_damage(gdrm);
+		mutex_unlock(&gdrm->damage_lock);
+	}
+
+	if (!drm_dev_enter(drm, &idx))
+		return;
+
+	if (!old_state->fb)
+		gud_usb_write8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 1);
+
+	if (fb && (crtc->state->mode_changed || crtc->state->connectors_changed))
+		gud_usb_set(gdrm, GUD_REQ_SET_STATE_COMMIT, 0, NULL, 0);
+
+	if (crtc->state->active_changed)
+		gud_usb_write8(gdrm, GUD_REQ_SET_DISPLAY_ENABLE, crtc->state->active);
+
+	if (drm_atomic_helper_damage_merged(old_state, state, &damage)) {
+		if (gdrm->flags & GUD_DISPLAY_FLAG_FULL_UPDATE)
+			drm_rect_init(&damage, 0, 0, fb->width, fb->height);
+		gud_fb_queue_damage(gdrm, fb, &damage);
+	}
+
+	if (!crtc->state->enable)
+		gud_usb_write8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 0);
+
+	drm_dev_exit(idx);
+}
diff --git a/include/drm/gud.h b/include/drm/gud.h
new file mode 100644
index 000000000000..7200296f46b3
--- /dev/null
+++ b/include/drm/gud.h
@@ -0,0 +1,356 @@ 
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright 2020 Noralf Trønnes
+ */
+
+#ifndef __LINUX_GUD_H
+#define __LINUX_GUD_H
+
+#include <linux/types.h>
+
+/*
+ * struct gud_display_descriptor_req - Display descriptor
+ * @magic: Magic value GUD_DISPLAY_MAGIC
+ * @version: Protocol version
+ * @flags: Flags
+ *         - STATUS_ON_SET: Always do a status request after a SET request.
+ *                          This is used by the Linux gadget driver since it has
+ *                          no way to control the status stage of a control OUT
+ *                          request that has a payload.
+ *         - FULL_UPDATE:   Always send the entire framebuffer when flushing changes.
+ *                          The GUD_REQ_SET_BUFFER request will not be sent
+ *                          before each bulk transfer, it will only be sent if the
+ *                          previous bulk transfer had failed. This gives the device
+ *                          a chance to reset its state machine if needed.
+ *                          This flag can not be used in combination with compression.
+ * @compression: Supported compression types
+ *               - GUD_COMPRESSION_LZ4: LZ4 lossless compression.
+ * @max_buffer_size: Maximum buffer size the device can handle (optional).
+ *                   This is useful for devices that don't have a big enough
+ *                   buffer to decompress the entire framebuffer in one go.
+ * @min_width: Minimum pixel width the controller can handle
+ * @max_width: Maximum width
+ * @min_height: Minimum height
+ * @max_height: Maximum height
+ * @num_formats: Number of supported pixel formats
+ * @num_properties: Number of properties that are not connector properties
+ * @num_connectors: Number of connectors
+ *
+ * Devices that have only one display mode will have min_width == max_width
+ * and min_height == max_height.
+ */
+struct gud_display_descriptor_req {
+	__le32 magic;
+#define GUD_DISPLAY_MAGIC			0x1d50614d
+	__u8 version;
+	__le32 flags;
+#define GUD_DISPLAY_FLAG_STATUS_ON_SET		BIT(0)
+#define GUD_DISPLAY_FLAG_FULL_UPDATE		BIT(1)
+	__u8 compression;
+#define GUD_COMPRESSION_LZ4			BIT(0)
+	__le32 max_buffer_size;
+	__le32 min_width;
+	__le32 max_width;
+	__le32 min_height;
+	__le32 max_height;
+	__u8 num_formats;
+	__u8 num_properties;
+	__u8 num_connectors;
+} __packed;
+
+/*
+ * struct gud_property_req - Property
+ * @prop: Property
+ * @val: Value
+ */
+struct gud_property_req {
+	__le16 prop;
+	__le64 val;
+} __packed;
+
+/*
+ * struct gud_display_mode_req - Display mode
+ * @clock: Pixel clock in kHz
+ * @hdisplay: Horizontal display size
+ * @hsync_start: Horizontal sync start
+ * @hsync_end: Horizontal sync end
+ * @htotal: Horizontal total size
+ * @vdisplay: Vertical display size
+ * @vsync_start: Vertical sync start
+ * @vsync_end: Vertical sync end
+ * @vtotal: Vertical total size
+ * @flags: Bits 0-13 are the same as in the RandR protocol and also what DRM uses.
+ *         The deprecated bits are reused for internal protocol flags leaving us
+ *         free to follow DRM for the other bits in the future.
+ *         - FLAG_PREFERRED: Set on the preferred display mode.
+ */
+struct gud_display_mode_req {
+	__le32 clock;
+	__le16 hdisplay;
+	__le16 hsync_start;
+	__le16 hsync_end;
+	__le16 htotal;
+	__le16 vdisplay;
+	__le16 vsync_start;
+	__le16 vsync_end;
+	__le16 vtotal;
+	__le32 flags;
+#define GUD_DISPLAY_MODE_FLAG_PHSYNC		BIT(0)
+#define GUD_DISPLAY_MODE_FLAG_NHSYNC		BIT(1)
+#define GUD_DISPLAY_MODE_FLAG_PVSYNC		BIT(2)
+#define GUD_DISPLAY_MODE_FLAG_NVSYNC		BIT(3)
+#define GUD_DISPLAY_MODE_FLAG_INTERLACE		BIT(4)
+#define GUD_DISPLAY_MODE_FLAG_DBLSCAN		BIT(5)
+#define GUD_DISPLAY_MODE_FLAG_CSYNC		BIT(6)
+#define GUD_DISPLAY_MODE_FLAG_PCSYNC		BIT(7)
+#define GUD_DISPLAY_MODE_FLAG_NCSYNC		BIT(8)
+#define GUD_DISPLAY_MODE_FLAG_HSKEW		BIT(9)
+/* BCast and PixelMultiplex are deprecated */
+#define GUD_DISPLAY_MODE_FLAG_DBLCLK		BIT(12)
+#define GUD_DISPLAY_MODE_FLAG_CLKDIV2		BIT(13)
+#define GUD_DISPLAY_MODE_FLAG_USER_MASK		\
+		(GUD_DISPLAY_MODE_FLAG_PHSYNC | GUD_DISPLAY_MODE_FLAG_NHSYNC | \
+		GUD_DISPLAY_MODE_FLAG_PVSYNC | GUD_DISPLAY_MODE_FLAG_NVSYNC | \
+		GUD_DISPLAY_MODE_FLAG_INTERLACE | GUD_DISPLAY_MODE_FLAG_DBLSCAN | \
+		GUD_DISPLAY_MODE_FLAG_CSYNC | GUD_DISPLAY_MODE_FLAG_PCSYNC | \
+		GUD_DISPLAY_MODE_FLAG_NCSYNC | GUD_DISPLAY_MODE_FLAG_HSKEW | \
+		GUD_DISPLAY_MODE_FLAG_DBLCLK | GUD_DISPLAY_MODE_FLAG_CLKDIV2)
+/* Internal protocol flags */
+#define GUD_DISPLAY_MODE_FLAG_PREFERRED		BIT(10)
+} __packed;
+
+/*
+ * struct gud_connector_descriptor_req - Connector descriptor
+ * @connector_type: Connector type (GUD_CONNECTOR_TYPE_*).
+ *                  If the host doesn't support the type it should fall back to PANEL.
+ * @flags: Flags
+ *         - POLL_STATUS: Connector status can change (polled every 10 seconds)
+ *         - INTERLACE: Interlaced modes are supported
+ *         - DOUBLESCAN: Doublescan modes are supported
+ * @num_properties: Number of supported properties
+ */
+struct gud_connector_descriptor_req {
+	__u8 connector_type;
+#define GUD_CONNECTOR_TYPE_PANEL		0
+#define GUD_CONNECTOR_TYPE_VGA			1
+#define GUD_CONNECTOR_TYPE_COMPOSITE		2
+#define GUD_CONNECTOR_TYPE_SVIDEO		3
+#define GUD_CONNECTOR_TYPE_COMPONENT		4
+#define GUD_CONNECTOR_TYPE_DVI			5
+#define GUD_CONNECTOR_TYPE_DISPLAYPORT		6
+#define GUD_CONNECTOR_TYPE_HDMI			7
+	__le32 flags;
+#define GUD_CONNECTOR_FLAGS_POLL_STATUS		BIT(0)
+#define GUD_CONNECTOR_FLAGS_INTERLACE		BIT(1)
+#define GUD_CONNECTOR_FLAGS_DOUBLESCAN		BIT(2)
+	__u8 num_properties;
+} __packed;
+
+/*
+ * struct gud_connector_status_req - Connector status
+ * @status: Status
+ *          - DISCONNECTED: Connector is disconnected
+ *          - CONNECTED: Connector is connected
+ *          - UNKNOWN: Connection status is unknown
+ *          Flags:
+ *          - CHANGED: A change has happened since the last request
+ * @num_modes: Number of available display modes
+ * @edid_len: Length of EDID data
+ *
+ * If @num_modes is zero, EDID is used to create display modes.
+ * If both @num_modes and @edid_len are set, EDID is just passed on to userspace
+ * in the EDID connector property.
+ *
+ * Userspace will get a HOTPLUG uevent if one of the following is true:
+ * - Connection status has changed
+ * - @num_modes or @edid_len has changed
+ * - CHANGED is set
+ */
+struct gud_connector_status_req {
+	__u8 status;
+#define GUD_CONNECTOR_STATUS_DISCONNECTED	0x00
+#define GUD_CONNECTOR_STATUS_CONNECTED		0x01
+#define GUD_CONNECTOR_STATUS_UNKNOWN		0x02
+#define GUD_CONNECTOR_STATUS_CONNECTED_MASK	0x03
+#define GUD_CONNECTOR_STATUS_CHANGED		BIT(7)
+	__le16 num_modes;
+	__le16 edid_len;
+} __packed;
+
+/*
+ * struct gud_set_buffer_req - Set buffer transfer info
+ * @x: X position of rectangle
+ * @y: Y position
+ * @width: Pixel width of rectangle
+ * @height: Pixel height
+ * @length: Buffer length in bytes
+ * @compression: Transfer compression
+ * @compressed_length: Compressed buffer length
+ *
+ * This request is issued right before the bulk transfer.
+ * @x, @y, @width and @height specifies the rectangle where the buffer should be
+ * placed inside the framebuffer.
+ */
+struct gud_set_buffer_req {
+	__le32 x;
+	__le32 y;
+	__le32 width;
+	__le32 height;
+	__le32 length;
+	__u8 compression;
+	__le32 compressed_length;
+} __packed;
+
+/*
+ * struct gud_state_req - Display state
+ * @mode: Display mode
+ * @format: Pixel format GUD_PIXEL_FORMAT_*
+ * @connector: Connector index
+ * @num_properties: Number of properties in the state
+ * @properties: Array of properties
+ *
+ * The entire state is transferred each time there's a change.
+ */
+struct gud_state_req {
+	struct gud_display_mode_req mode;
+	__u8 format;
+	__u8 connector;
+	__u8 num_properties;
+	struct gud_property_req properties[];
+} __packed;
+
+/* List of supported connector properties: */
+
+/* Margins in pixels to deal with overscan, range 0-100 */
+#define GUD_PROPERTY_TV_LEFT_MARGIN			1
+#define GUD_PROPERTY_TV_RIGHT_MARGIN			2
+#define GUD_PROPERTY_TV_TOP_MARGIN			3
+#define GUD_PROPERTY_TV_BOTTOM_MARGIN			4
+/* Number of modes are placed at _SHIFT in val on retrieval */
+#define GUD_PROPERTY_TV_MODE				5
+  #define GUD_CONNECTOR_TV_MODE_NUM_SHIFT	16
+/* Brightness in percent, range 0-100 */
+#define GUD_PROPERTY_TV_BRIGHTNESS			6
+/* Contrast in percent, range 0-100 */
+#define GUD_PROPERTY_TV_CONTRAST			7
+/* Flicker reduction in percent, range 0-100 */
+#define GUD_PROPERTY_TV_FLICKER_REDUCTION		8
+/* Overscan in percent, range 0-100 */
+#define GUD_PROPERTY_TV_OVERSCAN			9
+/* Saturation in percent, range 0-100 */
+#define GUD_PROPERTY_TV_SATURATION			10
+/* Hue in percent, range 0-100 */
+#define GUD_PROPERTY_TV_HUE				11
+
+/*
+ * Backlight brightness is in the range 0-100 inclusive. The value represents the human perceptual
+ * brightness and not a linear PWM value. 0 is minimum brightness which should not turn the
+ * backlight completely off. The DPMS connector property should be used to control power which will
+ * trigger a GUD_REQ_SET_DISPLAY_ENABLE request.
+ *
+ * This does not map to a DRM property, it is used with the backlight device.
+ */
+#define GUD_PROPERTY_BACKLIGHT_BRIGHTNESS		12
+
+/* List of supported properties that are not connector propeties: */
+
+/*
+ * Plane rotation. Should return the supported bitmask on
+ * GUD_REQ_GET_PROPERTIES. GUD_ROTATION_0 is mandatory.
+ *
+ * Note: This is not display rotation so 90/270 will need scaling to make it fit (unless squared).
+ */
+#define GUD_PROPERTY_ROTATION				50
+  #define GUD_ROTATION_0			BIT(0)
+  #define GUD_ROTATION_90			BIT(1)
+  #define GUD_ROTATION_180			BIT(2)
+  #define GUD_ROTATION_270			BIT(3)
+  #define GUD_ROTATION_REFLECT_X		BIT(4)
+  #define GUD_ROTATION_REFLECT_Y		BIT(5)
+  #define GUD_ROTATION_MASK			(GUD_ROTATION_0 | GUD_ROTATION_90 | \
+						GUD_ROTATION_180 | GUD_ROTATION_270 | \
+						GUD_ROTATION_REFLECT_X | GUD_ROTATION_REFLECT_Y)
+
+/* USB Control requests: */
+
+/* Get status from the last GET/SET control request. Value is u8. */
+#define GUD_REQ_GET_STATUS				0x00
+  /* Status values: */
+  #define GUD_STATUS_OK				0x00
+  #define GUD_STATUS_BUSY			0x01
+  #define GUD_STATUS_REQUEST_NOT_SUPPORTED	0x02
+  #define GUD_STATUS_PROTOCOL_ERROR		0x03
+  #define GUD_STATUS_INVALID_PARAMETER		0x04
+  #define GUD_STATUS_ERROR			0x05
+
+/* Get display descriptor as a &gud_display_descriptor_req */
+#define GUD_REQ_GET_DESCRIPTOR				0x01
+
+/*
+ * If the host driver doesn't support the device protocol version it will send the versions it
+ * supports starting with the latest. If the device isn't backwards compatible or doesn't support
+ * the version the host suggests, it shall stall the request. The version is sent as u8.
+ */
+#define GUD_REQ_SET_VERSION				0x30
+
+/* Get supported pixel formats as a byte array of GUD_PIXEL_FORMAT_* */
+#define GUD_REQ_GET_FORMATS				0x40
+  /* R1 is a 1-bit monochrome transfer format presented to userspace as XRGB8888 */
+  #define GUD_PIXEL_FORMAT_R1			0x01
+  #define GUD_PIXEL_FORMAT_RGB565		0x40
+  #define GUD_PIXEL_FORMAT_XRGB8888		0x80
+  #define GUD_PIXEL_FORMAT_ARGB8888		0x81
+
+/*
+ * Get supported properties that are not connector propeties as a &gud_property_req array.
+ * gud_property_req.val often contains the initial value for the property.
+ */
+#define GUD_REQ_GET_PROPERTIES				0x41
+
+/* Connector requests have the connector index passed in the wValue field */
+
+/* Get connector descriptor as a &gud_connector_descriptor_req */
+#define GUD_REQ_GET_CONNECTOR				0x50
+
+/*
+ * Get properties supported by the connector as a &gud_property_req array.
+ * gud_property_req.val often contains the initial value for the property.
+ */
+#define GUD_REQ_GET_CONNECTOR_PROPERTIES		0x51
+
+/*
+ * Issued when there's a TV_MODE property present.
+ * Gets an array of the supported TV_MODE names each entry of length
+ * GUD_CONNECTOR_TV_MODE_NAME_LEN. Names must be NUL-terminated.
+ */
+#define GUD_REQ_GET_CONNECTOR_TV_MODE_VALUES		0x52
+  #define GUD_CONNECTOR_TV_MODE_NAME_LEN	16
+
+/* When userspace checks connector status, this is issued first, not used for poll requests. */
+#define GUD_REQ_SET_CONNECTOR_FORCE_DETECT		0x53
+
+/* Get connector status as &gud_connector_status_req. */
+#define GUD_REQ_GET_CONNECTOR_STATUS			0x54
+
+/* Get &gud_display_mode_req array of supported display modes */
+#define GUD_REQ_GET_CONNECTOR_MODES			0x55
+
+/* Get Extended Display Identification Data */
+#define GUD_REQ_GET_CONNECTOR_EDID			0x56
+
+/* Set buffer properties before bulk transfer as &gud_set_buffer_req */
+#define GUD_REQ_SET_BUFFER				0x60
+
+/* Check display configuration as &gud_state_req */
+#define GUD_REQ_SET_STATE_CHECK				0x61
+
+/* Apply the previous STATE_CHECK configuration */
+#define GUD_REQ_SET_STATE_COMMIT			0x62
+
+/* Enable/disable the display controller, value is u8: 0/1 */
+#define GUD_REQ_SET_CONTROLLER_ENABLE			0x63
+
+/* Enable/disable display/output (DPMS), value is u8: 0/1 */
+#define GUD_REQ_SET_DISPLAY_ENABLE			0x64
+
+#endif