Message ID | 20190627033802.1663-1-tina.zhang@intel.com (mailing list archive) |
---|---|
Headers | show |
Series | Deliver vGPU display vblank event to userspace | expand |
Hi, > Instead of delivering page flip events, we choose to post display vblank > event. Handling page flip events for both primary plane and cursor plane > may make user space quite busy, although we have the mask/unmask mechansim > for mitigation. Besides, there are some cases that guest app only uses > one framebuffer for both drawing and display. In such case, guest OS won't > do the plane page flip when the framebuffer is updated, thus the user > land won't be notified about the updated framebuffer. What happens when the guest is idle and doesn't draw anything to the framebuffer? cheers, Gerd
> -----Original Message----- > From: Gerd Hoffmann [mailto:kraxel@redhat.com] > Sent: Thursday, June 27, 2019 2:23 PM > To: Zhang, Tina <tina.zhang@intel.com> > Cc: intel-gvt-dev@lists.freedesktop.org; kvm@vger.kernel.org; linux- > kernel@vger.kernel.org; zhenyuw@linux.intel.com; Lv, Zhiyuan > <zhiyuan.lv@intel.com>; Wang, Zhi A <zhi.a.wang@intel.com>; Tian, Kevin > <kevin.tian@intel.com>; Yuan, Hang <hang.yuan@intel.com>; > alex.williamson@redhat.com > Subject: Re: [RFC PATCH v3 0/4] Deliver vGPU display vblank event to > userspace > > Hi, > > > Instead of delivering page flip events, we choose to post display > > vblank event. Handling page flip events for both primary plane and > > cursor plane may make user space quite busy, although we have the > > mask/unmask mechansim for mitigation. Besides, there are some cases > > that guest app only uses one framebuffer for both drawing and display. > > In such case, guest OS won't do the plane page flip when the > > framebuffer is updated, thus the user land won't be notified about the > updated framebuffer. > > What happens when the guest is idle and doesn't draw anything to the > framebuffer? The vblank event will be delivered to userspace as well, unless guest OS disable the pipe. Does it make sense to vfio/display? Thanks. BR, Tina > > cheers, > Gerd
> > Hi, > > > > > Instead of delivering page flip events, we choose to post display > > > vblank event. Handling page flip events for both primary plane and > > > cursor plane may make user space quite busy, although we have the > > > mask/unmask mechansim for mitigation. Besides, there are some cases > > > that guest app only uses one framebuffer for both drawing and display. > > > In such case, guest OS won't do the plane page flip when the > > > framebuffer is updated, thus the user land won't be notified about the > > updated framebuffer. > > > > What happens when the guest is idle and doesn't draw anything to the > > framebuffer? > The vblank event will be delivered to userspace as well, unless guest OS disable the pipe. > Does it make sense to vfio/display? Getting notified only in case there are actual display updates would be a nice optimization, assuming the hardware is able to do that. If the guest pageflips this is obviously trivial. Not sure this is possible in case the guest renders directly to the frontbuffer. What exactly happens when the guest OS disables the pipe? Is a vblank event delivered at least once? That would be very useful because it will be possible for userspace to stop polling altogether without missing the "guest disabled pipe" event. cheers, Gerd
On 2019.06.27 12:31:33 +0200, Gerd Hoffmann wrote: > > > Hi, > > > > > > > Instead of delivering page flip events, we choose to post display > > > > vblank event. Handling page flip events for both primary plane and > > > > cursor plane may make user space quite busy, although we have the > > > > mask/unmask mechansim for mitigation. Besides, there are some cases > > > > that guest app only uses one framebuffer for both drawing and display. > > > > In such case, guest OS won't do the plane page flip when the > > > > framebuffer is updated, thus the user land won't be notified about the > > > updated framebuffer. > > > > > > What happens when the guest is idle and doesn't draw anything to the > > > framebuffer? > > The vblank event will be delivered to userspace as well, unless guest OS disable the pipe. > > Does it make sense to vfio/display? > > Getting notified only in case there are actual display updates would be > a nice optimization, assuming the hardware is able to do that. If the > guest pageflips this is obviously trivial. Not sure this is possible in > case the guest renders directly to the frontbuffer. > > What exactly happens when the guest OS disables the pipe? Is a vblank > event delivered at least once? That would be very useful because it > will be possible for userspace to stop polling altogether without > missing the "guest disabled pipe" event. > It looks like purpose to use vblank here is to replace user space polling totally by kernel event? Which just act as display update event to replace user space timer to make it query and update planes? Although in theory vblank is not appropriate for this which doesn't align with plane update or possible front buffer rendering at all, but looks it's just a compromise e.g not sending event for every cursor position change, etc. I think we need to define semantics for this event properly, e.g user space purely depends on this event for display update, the opportunity for issuing this event is controlled by driver when it's necessary for update, etc. Definitely not named as vblank event or only issue at vblank, that need to happen for other plane change too.
On Fri, Jun 28, 2019 at 11:21:49AM +0800, Zhenyu Wang wrote: > On 2019.06.27 12:31:33 +0200, Gerd Hoffmann wrote: > > > > Hi, > > > > > > > > > Instead of delivering page flip events, we choose to post display > > > > > vblank event. Handling page flip events for both primary plane and > > > > > cursor plane may make user space quite busy, although we have the > > > > > mask/unmask mechansim for mitigation. Besides, there are some cases > > > > > that guest app only uses one framebuffer for both drawing and display. > > > > > In such case, guest OS won't do the plane page flip when the > > > > > framebuffer is updated, thus the user land won't be notified about the > > > > updated framebuffer. > > > > > > > > What happens when the guest is idle and doesn't draw anything to the > > > > framebuffer? > > > The vblank event will be delivered to userspace as well, unless guest OS disable the pipe. > > > Does it make sense to vfio/display? > > > > Getting notified only in case there are actual display updates would be > > a nice optimization, assuming the hardware is able to do that. If the > > guest pageflips this is obviously trivial. Not sure this is possible in > > case the guest renders directly to the frontbuffer. > > > > What exactly happens when the guest OS disables the pipe? Is a vblank > > event delivered at least once? That would be very useful because it > > will be possible for userspace to stop polling altogether without > > missing the "guest disabled pipe" event. > > > > It looks like purpose to use vblank here is to replace user space > polling totally by kernel event? Which just act as display update > event to replace user space timer to make it query and update > planes? I think it makes sense to design it that way, so userspace will either use the events (when supported by the driver) or a timer (fallback if not) but not both. > Although in theory vblank is not appropriate for this which > doesn't align with plane update or possible front buffer rendering at > all, but looks it's just a compromise e.g not sending event for every > cursor position change, etc. > > I think we need to define semantics for this event properly, e.g user > space purely depends on this event for display update, the opportunity > for issuing this event is controlled by driver when it's necessary for > update, etc. Definitely not named as vblank event or only issue at vblank, > that need to happen for other plane change too. I think it should be "display update notification", i.e. userspace should check for plane changes and update the display. Most events will probably come from vblank (typically plane update are actually committed at vblank time to avoid tearing, right?). That is an implementation detail though. cheers, Gerd
On 2019.06.28 07:43:46 +0200, Gerd Hoffmann wrote: > On Fri, Jun 28, 2019 at 11:21:49AM +0800, Zhenyu Wang wrote: > > On 2019.06.27 12:31:33 +0200, Gerd Hoffmann wrote: > > > > > Hi, > > > > > > > > > > > Instead of delivering page flip events, we choose to post display > > > > > > vblank event. Handling page flip events for both primary plane and > > > > > > cursor plane may make user space quite busy, although we have the > > > > > > mask/unmask mechansim for mitigation. Besides, there are some cases > > > > > > that guest app only uses one framebuffer for both drawing and display. > > > > > > In such case, guest OS won't do the plane page flip when the > > > > > > framebuffer is updated, thus the user land won't be notified about the > > > > > updated framebuffer. > > > > > > > > > > What happens when the guest is idle and doesn't draw anything to the > > > > > framebuffer? > > > > The vblank event will be delivered to userspace as well, unless guest OS disable the pipe. > > > > Does it make sense to vfio/display? > > > > > > Getting notified only in case there are actual display updates would be > > > a nice optimization, assuming the hardware is able to do that. If the > > > guest pageflips this is obviously trivial. Not sure this is possible in > > > case the guest renders directly to the frontbuffer. > > > > > > What exactly happens when the guest OS disables the pipe? Is a vblank > > > event delivered at least once? That would be very useful because it > > > will be possible for userspace to stop polling altogether without > > > missing the "guest disabled pipe" event. > > > > > > > It looks like purpose to use vblank here is to replace user space > > polling totally by kernel event? Which just act as display update > > event to replace user space timer to make it query and update > > planes? > > I think it makes sense to design it that way, so userspace will either > use the events (when supported by the driver) or a timer (fallback if > not) but not both. Agree. It's more of a userspace choice. > > > Although in theory vblank is not appropriate for this which > > doesn't align with plane update or possible front buffer rendering at > > all, but looks it's just a compromise e.g not sending event for every > > cursor position change, etc. > > > > I think we need to define semantics for this event properly, e.g user > > space purely depends on this event for display update, the opportunity > > for issuing this event is controlled by driver when it's necessary for > > update, etc. Definitely not named as vblank event or only issue at vblank, > > that need to happen for other plane change too. > > I think it should be "display update notification", i.e. userspace > should check for plane changes and update the display. > > Most events will probably come from vblank (typically plane update are > actually committed at vblank time to avoid tearing, right?). That is an > implementation detail though. > Yeah, vblank should be a good time, although driver might also do optimization e.g checking actual plane change between vblank to see if there's any real change, etc. Also that will depend on driver implementation.
> -----Original Message----- > From: intel-gvt-dev [mailto:intel-gvt-dev-bounces@lists.freedesktop.org] On > Behalf Of Gerd Hoffmann > Sent: Friday, June 28, 2019 1:44 PM > To: Zhenyu Wang <zhenyuw@linux.intel.com> > Cc: Tian, Kevin <kevin.tian@intel.com>; kvm@vger.kernel.org; linux- > kernel@vger.kernel.org; Zhang, Tina <tina.zhang@intel.com>; > alex.williamson@redhat.com; Lv, Zhiyuan <zhiyuan.lv@intel.com>; Yuan, > Hang <hang.yuan@intel.com>; intel-gvt-dev@lists.freedesktop.org; Wang, > Zhi A <zhi.a.wang@intel.com> > Subject: Re: [RFC PATCH v3 0/4] Deliver vGPU display vblank event to > userspace > > On Fri, Jun 28, 2019 at 11:21:49AM +0800, Zhenyu Wang wrote: > > On 2019.06.27 12:31:33 +0200, Gerd Hoffmann wrote: > > > > > Hi, > > > > > > > > > > > Instead of delivering page flip events, we choose to post > > > > > > display vblank event. Handling page flip events for both > > > > > > primary plane and cursor plane may make user space quite busy, > > > > > > although we have the mask/unmask mechansim for mitigation. > > > > > > Besides, there are some cases that guest app only uses one > framebuffer for both drawing and display. > > > > > > In such case, guest OS won't do the plane page flip when the > > > > > > framebuffer is updated, thus the user land won't be notified > > > > > > about the > > > > > updated framebuffer. > > > > > > > > > > What happens when the guest is idle and doesn't draw anything to > > > > > the framebuffer? > > > > The vblank event will be delivered to userspace as well, unless guest OS > disable the pipe. > > > > Does it make sense to vfio/display? > > > > > > Getting notified only in case there are actual display updates would > > > be a nice optimization, assuming the hardware is able to do that. > > > If the guest pageflips this is obviously trivial. Not sure this is > > > possible in case the guest renders directly to the frontbuffer. > > > > > > What exactly happens when the guest OS disables the pipe? Is a > > > vblank event delivered at least once? That would be very useful > > > because it will be possible for userspace to stop polling altogether > > > without missing the "guest disabled pipe" event. > > > > > > > It looks like purpose to use vblank here is to replace user space > > polling totally by kernel event? Which just act as display update > > event to replace user space timer to make it query and update planes? > > I think it makes sense to design it that way, so userspace will either use the > events (when supported by the driver) or a timer (fallback if > not) but not both. > > > Although in theory vblank is not appropriate for this which doesn't > > align with plane update or possible front buffer rendering at all, but > > looks it's just a compromise e.g not sending event for every cursor > > position change, etc. > > > > I think we need to define semantics for this event properly, e.g user > > space purely depends on this event for display update, the opportunity > > for issuing this event is controlled by driver when it's necessary for > > update, etc. Definitely not named as vblank event or only issue at > > vblank, that need to happen for other plane change too. > > I think it should be "display update notification", i.e. userspace should check > for plane changes and update the display. > > Most events will probably come from vblank (typically plane update are > actually committed at vblank time to avoid tearing, right?). That is an Yes. > implementation detail though. We have two WIP branches: one is for vblank event delivery and the other one is for page flip event delivery. Repo: - QEMU: https://github.com/intel/Igvtg-qemu.git - Kernel: https://github.com/intel/gvt-linux.git Two branches: topic/userspace_direct_flip_page_event and topic/userspace_direct_flip_vblank_event With GTK UI, the user experience is bad on branch topic/userspace_direct_flip_page_event, as most of the CPU efforts are spent on event handling in user space. However, with the DRM UI both of the two branches have good user experience, as the event handling in DRM UI is pretty simple. Thanks. BR, Tina > > cheers, > Gerd > > _______________________________________________ > intel-gvt-dev mailing list > intel-gvt-dev@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev
Hi, How about GVT-g supports both vblank and page flip events. Then QEMU UI can mask/unmask the events to decide which kind of event is expected. For DRM UI, it can decide to mask vblank event and use page flip events. We tried DRM UI with page flip events, the performance is great, even in the case of front buffer rendering. For the UI like GTK, vblank event is better. Besides, with the irq mask/unmask mechanism, userspace can dynamically choose between the UI timer and the events. Thanks. BR, Tina > -----Original Message----- > From: Zhang, Tina > Sent: Friday, June 28, 2019 4:45 PM > To: 'Gerd Hoffmann' <kraxel@redhat.com>; Zhenyu Wang > <zhenyuw@linux.intel.com> > Cc: Tian, Kevin <kevin.tian@intel.com>; kvm@vger.kernel.org; linux- > kernel@vger.kernel.org; alex.williamson@redhat.com; Lv, Zhiyuan > <zhiyuan.lv@intel.com>; Yuan, Hang <hang.yuan@intel.com>; intel-gvt- > dev@lists.freedesktop.org; Wang, Zhi A <zhi.a.wang@intel.com> > Subject: RE: [RFC PATCH v3 0/4] Deliver vGPU display vblank event to > userspace > > > > > -----Original Message----- > > From: intel-gvt-dev > > [mailto:intel-gvt-dev-bounces@lists.freedesktop.org] On Behalf Of Gerd > > Hoffmann > > Sent: Friday, June 28, 2019 1:44 PM > > To: Zhenyu Wang <zhenyuw@linux.intel.com> > > Cc: Tian, Kevin <kevin.tian@intel.com>; kvm@vger.kernel.org; linux- > > kernel@vger.kernel.org; Zhang, Tina <tina.zhang@intel.com>; > > alex.williamson@redhat.com; Lv, Zhiyuan <zhiyuan.lv@intel.com>; Yuan, > > Hang <hang.yuan@intel.com>; intel-gvt-dev@lists.freedesktop.org; Wang, > > Zhi A <zhi.a.wang@intel.com> > > Subject: Re: [RFC PATCH v3 0/4] Deliver vGPU display vblank event to > > userspace > > > > On Fri, Jun 28, 2019 at 11:21:49AM +0800, Zhenyu Wang wrote: > > > On 2019.06.27 12:31:33 +0200, Gerd Hoffmann wrote: > > > > > > Hi, > > > > > > > > > > > > > Instead of delivering page flip events, we choose to post > > > > > > > display vblank event. Handling page flip events for both > > > > > > > primary plane and cursor plane may make user space quite > > > > > > > busy, although we have the mask/unmask mechansim for > mitigation. > > > > > > > Besides, there are some cases that guest app only uses one > > framebuffer for both drawing and display. > > > > > > > In such case, guest OS won't do the plane page flip when the > > > > > > > framebuffer is updated, thus the user land won't be notified > > > > > > > about the > > > > > > updated framebuffer. > > > > > > > > > > > > What happens when the guest is idle and doesn't draw anything > > > > > > to the framebuffer? > > > > > The vblank event will be delivered to userspace as well, unless > > > > > guest OS > > disable the pipe. > > > > > Does it make sense to vfio/display? > > > > > > > > Getting notified only in case there are actual display updates > > > > would be a nice optimization, assuming the hardware is able to do that. > > > > If the guest pageflips this is obviously trivial. Not sure this > > > > is possible in case the guest renders directly to the frontbuffer. > > > > > > > > What exactly happens when the guest OS disables the pipe? Is a > > > > vblank event delivered at least once? That would be very useful > > > > because it will be possible for userspace to stop polling > > > > altogether without missing the "guest disabled pipe" event. > > > > > > > > > > It looks like purpose to use vblank here is to replace user space > > > polling totally by kernel event? Which just act as display update > > > event to replace user space timer to make it query and update planes? > > > > I think it makes sense to design it that way, so userspace will either > > use the events (when supported by the driver) or a timer (fallback if > > not) but not both. > > > > > Although in theory vblank is not appropriate for this which doesn't > > > align with plane update or possible front buffer rendering at all, > > > but looks it's just a compromise e.g not sending event for every > > > cursor position change, etc. > > > > > > I think we need to define semantics for this event properly, e.g > > > user space purely depends on this event for display update, the > > > opportunity for issuing this event is controlled by driver when it's > > > necessary for update, etc. Definitely not named as vblank event or > > > only issue at vblank, that need to happen for other plane change too. > > > > I think it should be "display update notification", i.e. userspace > > should check for plane changes and update the display. > > > > Most events will probably come from vblank (typically plane update are > > actually committed at vblank time to avoid tearing, right?). That is > > an > Yes. > > implementation detail though. > > We have two WIP branches: one is for vblank event delivery and the other > one is for page flip event delivery. > Repo: > - QEMU: https://github.com/intel/Igvtg-qemu.git > - Kernel: https://github.com/intel/gvt-linux.git > Two branches: topic/userspace_direct_flip_page_event and > topic/userspace_direct_flip_vblank_event > > With GTK UI, the user experience is bad on branch > topic/userspace_direct_flip_page_event, as most of the CPU efforts are > spent on event handling in user space. > However, with the DRM UI both of the two branches have good user > experience, as the event handling in DRM UI is pretty simple. > > Thanks. > > BR, > Tina > > > > > cheers, > > Gerd > > > > _______________________________________________ > > intel-gvt-dev mailing list > > intel-gvt-dev@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev
On 2019.06.28 12:43:47 +0000, Zhang, Tina wrote: > Hi, > > How about GVT-g supports both vblank and page flip events. Then QEMU UI can mask/unmask the events to decide which kind of event is expected. > For DRM UI, it can decide to mask vblank event and use page flip events. We tried DRM UI with page flip events, the performance is great, even in the case of front buffer rendering. For the UI like GTK, vblank event is better. > > Besides, with the irq mask/unmask mechanism, userspace can dynamically choose between the UI timer and the events. > The idea is to deliver an event to tell userspace that "guest has display update, you need to refresh your UI". How driver implementation uses either vblank or page flip to determine that is driver specific, as same routine of vfio gfx interface will be used to refresh on guest display. If userspace doesn't set irq for vfio gfx display, it would just use own timer. > > > -----Original Message----- > > From: Zhang, Tina > > Sent: Friday, June 28, 2019 4:45 PM > > To: 'Gerd Hoffmann' <kraxel@redhat.com>; Zhenyu Wang > > <zhenyuw@linux.intel.com> > > Cc: Tian, Kevin <kevin.tian@intel.com>; kvm@vger.kernel.org; linux- > > kernel@vger.kernel.org; alex.williamson@redhat.com; Lv, Zhiyuan > > <zhiyuan.lv@intel.com>; Yuan, Hang <hang.yuan@intel.com>; intel-gvt- > > dev@lists.freedesktop.org; Wang, Zhi A <zhi.a.wang@intel.com> > > Subject: RE: [RFC PATCH v3 0/4] Deliver vGPU display vblank event to > > userspace > > > > > > > > > -----Original Message----- > > > From: intel-gvt-dev > > > [mailto:intel-gvt-dev-bounces@lists.freedesktop.org] On Behalf Of Gerd > > > Hoffmann > > > Sent: Friday, June 28, 2019 1:44 PM > > > To: Zhenyu Wang <zhenyuw@linux.intel.com> > > > Cc: Tian, Kevin <kevin.tian@intel.com>; kvm@vger.kernel.org; linux- > > > kernel@vger.kernel.org; Zhang, Tina <tina.zhang@intel.com>; > > > alex.williamson@redhat.com; Lv, Zhiyuan <zhiyuan.lv@intel.com>; Yuan, > > > Hang <hang.yuan@intel.com>; intel-gvt-dev@lists.freedesktop.org; Wang, > > > Zhi A <zhi.a.wang@intel.com> > > > Subject: Re: [RFC PATCH v3 0/4] Deliver vGPU display vblank event to > > > userspace > > > > > > On Fri, Jun 28, 2019 at 11:21:49AM +0800, Zhenyu Wang wrote: > > > > On 2019.06.27 12:31:33 +0200, Gerd Hoffmann wrote: > > > > > > > Hi, > > > > > > > > > > > > > > > Instead of delivering page flip events, we choose to post > > > > > > > > display vblank event. Handling page flip events for both > > > > > > > > primary plane and cursor plane may make user space quite > > > > > > > > busy, although we have the mask/unmask mechansim for > > mitigation. > > > > > > > > Besides, there are some cases that guest app only uses one > > > framebuffer for both drawing and display. > > > > > > > > In such case, guest OS won't do the plane page flip when the > > > > > > > > framebuffer is updated, thus the user land won't be notified > > > > > > > > about the > > > > > > > updated framebuffer. > > > > > > > > > > > > > > What happens when the guest is idle and doesn't draw anything > > > > > > > to the framebuffer? > > > > > > The vblank event will be delivered to userspace as well, unless > > > > > > guest OS > > > disable the pipe. > > > > > > Does it make sense to vfio/display? > > > > > > > > > > Getting notified only in case there are actual display updates > > > > > would be a nice optimization, assuming the hardware is able to do that. > > > > > If the guest pageflips this is obviously trivial. Not sure this > > > > > is possible in case the guest renders directly to the frontbuffer. > > > > > > > > > > What exactly happens when the guest OS disables the pipe? Is a > > > > > vblank event delivered at least once? That would be very useful > > > > > because it will be possible for userspace to stop polling > > > > > altogether without missing the "guest disabled pipe" event. > > > > > > > > > > > > > It looks like purpose to use vblank here is to replace user space > > > > polling totally by kernel event? Which just act as display update > > > > event to replace user space timer to make it query and update planes? > > > > > > I think it makes sense to design it that way, so userspace will either > > > use the events (when supported by the driver) or a timer (fallback if > > > not) but not both. > > > > > > > Although in theory vblank is not appropriate for this which doesn't > > > > align with plane update or possible front buffer rendering at all, > > > > but looks it's just a compromise e.g not sending event for every > > > > cursor position change, etc. > > > > > > > > I think we need to define semantics for this event properly, e.g > > > > user space purely depends on this event for display update, the > > > > opportunity for issuing this event is controlled by driver when it's > > > > necessary for update, etc. Definitely not named as vblank event or > > > > only issue at vblank, that need to happen for other plane change too. > > > > > > I think it should be "display update notification", i.e. userspace > > > should check for plane changes and update the display. > > > > > > Most events will probably come from vblank (typically plane update are > > > actually committed at vblank time to avoid tearing, right?). That is > > > an > > Yes. > > > implementation detail though. > > > > We have two WIP branches: one is for vblank event delivery and the other > > one is for page flip event delivery. > > Repo: > > - QEMU: https://github.com/intel/Igvtg-qemu.git > > - Kernel: https://github.com/intel/gvt-linux.git > > Two branches: topic/userspace_direct_flip_page_event and > > topic/userspace_direct_flip_vblank_event > > > > With GTK UI, the user experience is bad on branch > > topic/userspace_direct_flip_page_event, as most of the CPU efforts are > > spent on event handling in user space. > > However, with the DRM UI both of the two branches have good user > > experience, as the event handling in DRM UI is pretty simple. > > > > Thanks. > > > > BR, > > Tina > > > > > > > > cheers, > > > Gerd > > > > > > _______________________________________________ > > > intel-gvt-dev mailing list > > > intel-gvt-dev@lists.freedesktop.org > > > https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev