| Message ID | 1418979967-2134-1-git-send-email-rydberg@bitmath.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Fri, Dec 19, 2014 at 4:06 AM, Henrik Rydberg <rydberg@bitmath.org> wrote: > My ISP finally gave up on the old mail address, so I am moving things > over to bitmath.org instead. Also change the status fields to better > reflect reality. > > Signed-off-by: Henrik Rydberg <rydberg@bitmath.org> > --- Dmitry, it looks like this patch never found its way to your tree... Henrik, I just used the get_maintainer to add you on CC to an input-mt patch series, and it ended up using the @euromail.se instead of your still valid one. I can resend to you the patch series if you want. Cheers, Benjamin > MAINTAINERS | 12 ++++++------ > 1 file changed, 6 insertions(+), 6 deletions(-) > > diff --git a/MAINTAINERS b/MAINTAINERS > index c721042..62b53e8 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -714,15 +714,15 @@ F: include/uapi/linux/apm_bios.h > F: drivers/char/apm-emulation.c > > APPLE BCM5974 MULTITOUCH DRIVER > -M: Henrik Rydberg <rydberg@euromail.se> > +M: Henrik Rydberg <rydberg@bitmath.org> > L: linux-input@vger.kernel.org > -S: Maintained > +S: Odd fixes > F: drivers/input/mouse/bcm5974.c > > APPLE SMC DRIVER > -M: Henrik Rydberg <rydberg@euromail.se> > +M: Henrik Rydberg <rydberg@bitmath.org> > L: lm-sensors@lm-sensors.org > -S: Maintained > +S: Odd fixes > F: drivers/hwmon/applesmc.c > > APPLETALK NETWORK LAYER > @@ -4813,10 +4813,10 @@ F: include/uapi/linux/input.h > F: include/linux/input/ > > INPUT MULTITOUCH (MT) PROTOCOL > -M: Henrik Rydberg <rydberg@euromail.se> > +M: Henrik Rydberg <rydberg@bitmath.org> > L: linux-input@vger.kernel.org > T: git git://git.kernel.org/pub/scm/linux/kernel/git/rydberg/input-mt.git > -S: Maintained > +S: Odd fixes > F: Documentation/input/multi-touch-protocol.txt > F: drivers/input/input-mt.c > K: \b(ABS|SYN)_MT_ > -- > 2.1.3 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-input" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 19, 2015 at 04:08:19PM -0500, Benjamin Tissoires wrote: > On Fri, Dec 19, 2014 at 4:06 AM, Henrik Rydberg <rydberg@bitmath.org> wrote: > > My ISP finally gave up on the old mail address, so I am moving things > > over to bitmath.org instead. Also change the status fields to better > > reflect reality. > > > > Signed-off-by: Henrik Rydberg <rydberg@bitmath.org> > > --- > > Dmitry, it looks like this patch never found its way to your tree... It went through akpm and is in mainline now. Thanks.
On Mon, Jan 19, 2015 at 7:13 PM, Dmitry Torokhov <dmitry.torokhov@gmail.com> wrote: > On Mon, Jan 19, 2015 at 04:08:19PM -0500, Benjamin Tissoires wrote: >> On Fri, Dec 19, 2014 at 4:06 AM, Henrik Rydberg <rydberg@bitmath.org> wrote: >> > My ISP finally gave up on the old mail address, so I am moving things >> > over to bitmath.org instead. Also change the status fields to better >> > reflect reality. >> > >> > Signed-off-by: Henrik Rydberg <rydberg@bitmath.org> >> > --- >> >> Dmitry, it looks like this patch never found its way to your tree... > > It went through akpm and is in mainline now. > OK, then sorry for the noise, I should have updated my tree to master before sending this :( Cheers, Benjamin -- To unsubscribe from this list: send the line "unsubscribe linux-input" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Benjamin, > Henrik, I just used the get_maintainer to add you on CC to an input-mt > patch series, and it ended up using the @euromail.se instead of your > still valid one. I can resend to you the patch series if you want. Thanks, that won't be necessary. I looked through the the patchset, and it strikes me as mostly renames without deeper explanations, which all in all puts them in the not-needed category, I am afraid. An in-depth explanation could be added as a text block somewhere without touching the current code. The only patch that does something is the last one, and what it does could easily be performed in userland, by further processing of the contacts there. I therefore see no use for any of those patches in the kernel. Is there a good reason for the first reaction to be to add special tweaks such as this one in the kernel, rather than in a dedicated userland input system? I am genuinely curious. Thanks, Henrik -- To unsubscribe from this list: send the line "unsubscribe linux-input" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Jan 20, 2015 at 11:26:20PM +0100, Henrik Rydberg wrote: > Hi Benjamin, > > > Henrik, I just used the get_maintainer to add you on CC to an input-mt > > patch series, and it ended up using the @euromail.se instead of your > > still valid one. I can resend to you the patch series if you want. > > Thanks, that won't be necessary. > > I looked through the the patchset, and it strikes me as mostly renames without > deeper explanations, which all in all puts them in the not-needed category, I am > afraid. An in-depth explanation could be added as a text block somewhere without > touching the current code. I disagree. Patches that improve readability of the code are rarely in the not-needed category. Looking at Input: mt: document input_mt_set_matrix() for example: without that comment I'd have no idea if what the function does is correct or not because I'd have no idea what it's _supposed_ to do. > The only patch that does something is the last one, and what it does could > easily be performed in userland, by further processing of the contacts there. I > therefore see no use for any of those patches in the kernel. > > Is there a good reason for the first reaction to be to add special tweaks such > as this one in the kernel, rather than in a dedicated userland input system? I > am genuinely curious. it wasn't the first reaction. it was the third, after we hacked up synaptics [1], then libinput [2], both rather unsatisfactory. Not sure what chromeos does but they'll probably would need similar code. And any other users of the evdev API of course. Anyway, yes, you're right, it can be done in userland, depending on the implementation of userland. Putting proper support for this in synaptics would require a rather large rewrite of the touch handling for example [3] So we need multiple different implementations. Of course, we could factor this out into a library, at which point we have another API to track and piece of code to maintain and slot in between the device node and the consumer of the events. And effectively we'd have then written the same code that's already in the kernel anyway (which we can't copy directly because GPL vs MIT etc.). The big ticket item though is that without this patch the message is: you can't trust protocol B to actually track the touchpoints correctly. Which goes against to what it's supposed to help with in the first place. So now we need some userspace code to verify the kernel touchpoint tracking, but since this only affects some devices you'd need some code to identify devices that need to be double checked. The kernel has that info, we don't have (all of) it in userspace. All of this effort to work around what the kernel is supposed to do in the first place. So yeah, I think there are plenty of good reasons for having this in the kernel :) Cheers, Peter [1] http://lists.x.org/archives/xorg-devel/2014-September/043866.html [2] http://lists.freedesktop.org/archives/wayland-devel/2014-September/017340.html [3] which I try to avoid given that synaptics is somewhere between hanging on life support and receiving pallative care. -- To unsubscribe from this list: send the line "unsubscribe linux-input" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Peter, > it wasn't the first reaction. it was the third, after we hacked up > synaptics [1], then libinput [2], both rather unsatisfactory. Not sure what > chromeos does but they'll probably would need similar code. And any other > users of the evdev API of course. So if this approach did not work very well in userland, why would it work any better in the kernel? Henrik -- To unsubscribe from this list: send the line "unsubscribe linux-input" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Jan 21, 2015 at 10:25 AM, Henrik Rydberg <rydberg@bitmath.org> wrote: > Hi Peter, > >> it wasn't the first reaction. it was the third, after we hacked up >> synaptics [1], then libinput [2], both rather unsatisfactory. Not sure what >> chromeos does but they'll probably would need similar code. And any other >> users of the evdev API of course. > > So if this approach did not work very well in userland, why would it work any > better in the kernel? > I would have thought that Peter's answer was clear enough: - there is a fragmentation problem: we would have to fix the bug in xorg-synaptics (which is slowly waiting for its death), libinput, ChromeOS, Qt Embedded, Kivy (I think), etc... We can do it, to some extend, but we will have to explain the following to our users: - it means that the mt protocol B can not be relied upon, because even if we state that each touch has its own slot, then it is false in this case. This was Peter last point. Also, if you compare the libinput implementation of the handling of the cursors jumps and the kernel implementation I proposed, there is a big difference in term of simplicity. In the kernel, while we are assigning the tracking IDs, we detect that there is a jump, and we "just" have to generate a new slot and close the first (done by 1 assignment of -1 to the current tracking ID). In Libinput, well, you receive a slot, there is a jump, you detect it, then you have to create a new fake kernel event to stop the current slot, create a new one, and you then have to rewind the current state of the buttons, the hysteresis, add special case handling and hopefully, you did not introduced a bug in all the complex code. So you need to write unit tests (not an argument, I concede, but this is extra work), and in the future, someone will not understand what this is all about because the kernel should guarantee that the slots are sane. If I were grumpy (and I can be, ask Peter), I would say that sure, we can add such a case in the mtdev library, but the point of having the in-kernel tracking system was to slowly get away from the head over added by mtdev. Cheers, Benjamin -- To unsubscribe from this list: send the line "unsubscribe linux-input" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Benjamin, > - there is a fragmentation problem: we would have to fix the bug in > xorg-synaptics (which is slowly waiting for its death), libinput, > ChromeOS, Qt Embedded, Kivy (I think), etc... Indeed, this is the problem I wanted to highlight. As the fragmentation problem grows (graphics, input, compositors, toolkits), the need for a common denominator grows as well. However, I do not think the kernel should be the single common denominator for all the world's problems. Rather, the purpose of the kernel is to convey hardware information and control as accurately, effectively and generically as possible. > - it means that the mt protocol B can not be relied upon, because even > if we state that each touch has its own slot, then it is false in this > case. The case we are talking about is due to information missing in the hardware. At low enough sampling frequencies, there is no way to distinguish between a moving finger and a lift-and-press action. We could flag this hardware deficiency somehow, but making shit up in order to maintain the statue that we do have enough information is just asking for trouble. I agree that this point is valid: we cannot always trust the interpretation of touchpoints for certain hardware. However, there is nothing we can do about that, except flag for it. > Also, if you compare the libinput implementation of the handling of > the cursors jumps and the kernel implementation I proposed, there is a > big difference in term of simplicity. No, this is wrong. > In the kernel, while we are assigning the tracking IDs, we detect that > there is a jump, and we "just" have to generate a new slot and close > the first (done by 1 assignment of -1 to the current tracking ID). The kernel case would have to be accompanied by parameters, under the control of some user process, where adjustments are made to accomodate different usecases such as painting, gaming, air guitar playing, flick gestures, multi-user tablets, etc, etc. That is complex and unwanted. > In Libinput, well, you receive a slot, there is a jump, you detect it, > then you have to create a new fake kernel event to stop the current > slot, create a new one, and you then have to rewind the current state > of the buttons, the hysteresis, add special case handling and > hopefully, you did not introduced a bug in all the complex code. So > you need to write unit tests (not an argument, I concede, but this is > extra work), and in the future, someone will not understand what this > is all about because the kernel should guarantee that the slots are > sane. You do not need to do any of this (except the test cases, which would be needed anyway given the context-dependent interpretation of scarse data) if you intercept the touch points as they come in from the kernel, before the contact dynamics is fully trusted. Last time I checked that was mtdev or the touch frame layer or Xinput. > If I were grumpy (and I can be, ask Peter), I would say that sure, we > can add such a case in the mtdev library, but the point of having the > in-kernel tracking system was to slowly get away from the head over > added by mtdev. No, this was not the reason. The tree main reasons were actually latency and power and code reduction. The mtdev layer still provides the functional bridge needed. However, the latency and number of cpu cycles involved in transferring the data to userland before throwing most of it away were much reduced by adding the in-kernel tracking. Collecting the diversity of solutions for older hardware was a maintainability bonus. Regarding the practical problem at hand, that double taps sometimes get misinterpreted as a "flash move", perhaps the problem really is in how we define a double tap. When I was writing a certain gesture engine, I realized I got issues with multi-finger taps. In my case the problem was not due to misinterpreted finger data, but simply because pressing four fingers simultaneously is not easy, and really not needed in order to define the gesture per se. So in order to correctly interpret gestures, I had to involve time in various ways; checking for overlaps between fingers over a short time span, looking for the maximum number of fingers within a certain timespan, etc. Not to mention palm detection; in this context, the individual touch points really lose some meaning. The double tap is no different in this regard. Nor is a flash move. But we would not want a real flash move to be interpreted as double tap, would we? My point is that the gesture context is where the ultimate decision can best be made - where there is enough information to best approximate what the imperfect hardware is trying to tell us. If the time between touch down and a fast movement to a nearby point is consistent with a tap, then maybe it is a tap. End rant. I hope it helps one way or the other. Thanks, Henrik -- To unsubscribe from this list: send the line "unsubscribe linux-input" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
changed the subject so it'll be easier to find in the archives. On Wed, Jan 21, 2015 at 08:38:46PM +0100, Henrik Rydberg wrote: > Hi Benjamin, > > > - there is a fragmentation problem: we would have to fix the bug in > > xorg-synaptics (which is slowly waiting for its death), libinput, > > ChromeOS, Qt Embedded, Kivy (I think), etc... > > Indeed, this is the problem I wanted to highlight. As the fragmentation problem > grows (graphics, input, compositors, toolkits), the need for a common > denominator grows as well. However, I do not think the kernel should be the > single common denominator for all the world's problems. Rather, the purpose of > the kernel is to convey hardware information and control as accurately, > effectively and generically as possible. > > > - it means that the mt protocol B can not be relied upon, because even > > if we state that each touch has its own slot, then it is false in this > > case. > > The case we are talking about is due to information missing in the hardware. At > low enough sampling frequencies, there is no way to distinguish between a moving > finger and a lift-and-press action. We could flag this hardware deficiency > somehow, but making shit up in order to maintain the statue that we do have > enough information is just asking for trouble. > > I agree that this point is valid: we cannot always trust the interpretation of > touchpoints for certain hardware. However, there is nothing we can do about > that, except flag for it. > > > Also, if you compare the libinput implementation of the handling of > > the cursors jumps and the kernel implementation I proposed, there is a > > big difference in term of simplicity. > > No, this is wrong. > > > In the kernel, while we are assigning the tracking IDs, we detect that > > there is a jump, and we "just" have to generate a new slot and close > > the first (done by 1 assignment of -1 to the current tracking ID). > > The kernel case would have to be accompanied by parameters, under the control of > some user process, where adjustments are made to accomodate different usecases > such as painting, gaming, air guitar playing, flick gestures, multi-user > tablets, etc, etc. That is complex and unwanted. from the testing I've done, you cannot trigger this except by lifting a finger and raising a finger. There is no per-use-case requirement, it's a hardware deficiency that the hardware simply cannot detect this specific case of finger change. Also, note that the patch series has an explicit "In order not to penalize better sensors, this parameter is not automatically enabled, but each driver has to manually set it to a reasonable value." so far we have only seen this on synaptics touchpads which has its own driver. so this wouldn't affect any device that doesn't need this anyway. so yes, we could implement a better gesture recognition like you explain below but it's also unneded on all but a few devices - which, in userspace, we cannot identify easily. > > In Libinput, well, you receive a slot, there is a jump, you detect it, > > then you have to create a new fake kernel event to stop the current > > slot, create a new one, and you then have to rewind the current state > > of the buttons, the hysteresis, add special case handling and > > hopefully, you did not introduced a bug in all the complex code. So > > you need to write unit tests (not an argument, I concede, but this is > > extra work), and in the future, someone will not understand what this > > is all about because the kernel should guarantee that the slots are > > sane. > > You do not need to do any of this (except the test cases, which would be needed > anyway given the context-dependent interpretation of scarse data) if you > intercept the touch points as they come in from the kernel, before the contact > dynamics is fully trusted. Last time I checked that was mtdev or the touch frame > layer or Xinput. mtdev is only used for protocol A devices these days so it doesn't apply on the synaptics pads. I was thinking of adding this to libevdev so at least _I_ only have to implement this once. except that for every touchpoint that I fix up in libevdev I then also have to maintain a correct tracking_id mapping since now the kernel and the userspace tracking IDs are out of sync. that leaves synaptics which for historical reasons can handle a device state but is bad at transitions. Injecting events requires a rewrite of much of the code. evdev - not as bad but similar. Since this only affects synaptics pads so far at least we don't have to worry about other drivers (in xorg at least). > > If I were grumpy (and I can be, ask Peter), I would say that sure, we > > can add such a case in the mtdev library, but the point of having the > > in-kernel tracking system was to slowly get away from the head over > > added by mtdev. > > No, this was not the reason. The tree main reasons were actually latency and > power and code reduction. The mtdev layer still provides the functional bridge > needed. However, the latency and number of cpu cycles involved in transferring > the data to userland before throwing most of it away were much reduced by adding > the in-kernel tracking. Collecting the diversity of solutions for older hardware > was a maintainability bonus. > > Regarding the practical problem at hand, that double taps sometimes get > misinterpreted as a "flash move", perhaps the problem really is in how we define > a double tap. > > When I was writing a certain gesture engine, I realized I got issues with > multi-finger taps. In my case the problem was not due to misinterpreted finger > data, but simply because pressing four fingers simultaneously is not easy, and > really not needed in order to define the gesture per se. So in order to > correctly interpret gestures, I had to involve time in various ways; checking > for overlaps between fingers over a short time span, looking for the maximum > number of fingers within a certain timespan, etc. Not to mention palm detection; > in this context, the individual touch points really lose some meaning. > > The double tap is no different in this regard. Nor is a flash move. But we would > not want a real flash move to be interpreted as double tap, would we? > > My point is that the gesture context is where the ultimate decision can best be > made - where there is enough information to best approximate what the imperfect > hardware is trying to tell us. If the time between touch down and a fast > movement to a nearby point is consistent with a tap, then maybe it is a tap. again, you cannot trigger this by moving. or at least if you do your pointer would be in the bottom right corner anyway because you have to move the finger so insanely fast that pointer control is clearly not in the picture anymore. but you can trigger it 1 out of 3 times by lifting your finger from the touchpad while trying to click a button with your thumb. right now this means that those touchpads are virtually unsuable (this is a common interaction method). and, again: this is a solution for one specific set of devices. Cheers, Peter -- To unsubscribe from this list: send the line "unsubscribe linux-input" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Peter, > from the testing I've done, you cannot trigger this except by lifting a > finger and raising a finger. There is no per-use-case requirement, it's a > hardware deficiency that the hardware simply cannot detect this specific > case of finger change. This is a good selling point, but... > again, you cannot trigger this by moving. or at least if you do your > pointer would be in the bottom right corner anyway because you have to move > the finger so insanely fast that pointer control is clearly not in the > picture anymore. this is what puts me off again. It takes me back to the old arcade days, and watching kids hammer at the controls. What is insane for one person may not be insane for another. That said, I understand and agree that there exists an upper move velocity that makes sense for any particular touchpad. So I will accept two patches on this: one to set the speed limit in the MT layer, and one which uses that limit to solve a real problem, for the touchpad in question. Let's end this thread here and leave unrelated people in peace. Thanks, Henrik -- To unsubscribe from this list: send the line "unsubscribe linux-input" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed 2015-01-21 20:38:46, Henrik Rydberg wrote: > Hi Benjamin, > > > - there is a fragmentation problem: we would have to fix the bug in > > xorg-synaptics (which is slowly waiting for its death), libinput, > > ChromeOS, Qt Embedded, Kivy (I think), etc... > > Indeed, this is the problem I wanted to highlight. As the fragmentation problem > grows (graphics, input, compositors, toolkits), the need for a common > denominator grows as well. However, I do not think the kernel should be the > single common denominator for all the world's problems. Rather, the purpose of > the kernel is to convey hardware information and control as accurately, > effectively and generically as possible. No, sorry, that's not true. Kernel should provide hardware abstraction, hiding differences between different machines. Pavel
diff --git a/MAINTAINERS b/MAINTAINERS index c721042..62b53e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -714,15 +714,15 @@ F: include/uapi/linux/apm_bios.h F: drivers/char/apm-emulation.c APPLE BCM5974 MULTITOUCH DRIVER -M: Henrik Rydberg <rydberg@euromail.se> +M: Henrik Rydberg <rydberg@bitmath.org> L: linux-input@vger.kernel.org -S: Maintained +S: Odd fixes F: drivers/input/mouse/bcm5974.c APPLE SMC DRIVER -M: Henrik Rydberg <rydberg@euromail.se> +M: Henrik Rydberg <rydberg@bitmath.org> L: lm-sensors@lm-sensors.org -S: Maintained +S: Odd fixes F: drivers/hwmon/applesmc.c APPLETALK NETWORK LAYER @@ -4813,10 +4813,10 @@ F: include/uapi/linux/input.h F: include/linux/input/ INPUT MULTITOUCH (MT) PROTOCOL -M: Henrik Rydberg <rydberg@euromail.se> +M: Henrik Rydberg <rydberg@bitmath.org> L: linux-input@vger.kernel.org T: git git://git.kernel.org/pub/scm/linux/kernel/git/rydberg/input-mt.git -S: Maintained +S: Odd fixes F: Documentation/input/multi-touch-protocol.txt F: drivers/input/input-mt.c K: \b(ABS|SYN)_MT_
My ISP finally gave up on the old mail address, so I am moving things over to bitmath.org instead. Also change the status fields to better reflect reality. Signed-off-by: Henrik Rydberg <rydberg@bitmath.org> --- MAINTAINERS | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)