diff mbox

[6/8,v3] Input: synaptics - process finger (<=3) transitions

Message ID 1313169407-4358-7-git-send-email-djkurtz@chromium.org (mailing list archive)
State Superseded
Headers show

Commit Message

Daniel Kurtz Aug. 12, 2011, 5:16 p.m. UTC
Synaptics image sensor touchpads track 5 fingers, but only report 2.
This patch attempts to deal with some idiosyncrasies of these touchpads:

 * When there are 3 or more fingers, only two are reported.
 * The touchpad tracks the 5 fingers in slot[0] through slot[4].
 * It always reports the lowest and highest valid slots in SGM and AGM
   packets, respectively.
 * The number of fingers is only reported in the SGM packet.  However,
   the number of fingers can change either before or after an AGM
   packet.
 * Thus, if an SGM reports a different number of fingers than the last
   SGM, it is impossible to tell whether the intervening AGM corresponds
   to the old number of fingers or the new number of fingers.
 * For example, when going from 2->3 fingers, it is not possible to tell
   whether tell AGM contains slot[1] (old 2nd finger) or slot[2] (new
   3rd finger).
 * When fingers are added one at at time, from 1->2->3, it is possible to
   track which slots are contained in the SGM and AGM packets:
     1 finger:  SGM = slot[0], no AGM
     2 fingers: SGM = slot[0], AGM = slot[1]
     3 fingers: SGM = slot[0], AGM = slot[2]
 * It is also possible to track which slot is contained in the SGM when 1
   of 2 fingers is removed.  This is because the touchpad sends a special
   (0,0,0) AGM packet whenever all fingers are removed except slot[0]:
     Last AGM == (0,0,0): SGM contains slot[1]
     Else: SGM contains slot[0]
 * However, once there are 3 fingers, if exactly 1 finger is removed, it
   is impossible to tell which 2 slots are contained in SGM and AGM.
   The (SGM,AGM) could be (0,1), (0,2), or (1,2). There is no way to know.
 * Similarly, if two fingers are simultaneously removed (3->1), then it
   is only possible to know if SGM still contains slot[0].
 * Since it is not possible to reliably track which slot is being
   reported, we invalidate the tracking_id every time the number of
   fingers changes until this ambiguity is resolved when:
     a) All fingers are removed.
     b) 4 or 5 fingers are touched, generates an AGM-CONTACT packet.
     c) All fingers are removed except slot[0].  In this special case, the
        ambiguity is resolved since by the (0,0,0) AGM packet.

Behavior of the driver:

When 2 or more fingers are present on the touchpad, the kernel reports
up to two MT-B slots containing the position data for two of the fingers
reported by the touchpad.  If the identity of a finger cannot be tracked
when the number-of-fingers changes, the corresponding MT-B slot will be
invalidated (track_id set to -1), and a new track_id will be assigned in
a subsequent input event report.

The driver always reports the total number of fingers using one of the
EV_KEY/BTN_TOOL_*TAP events. This could differ from the number of valid
MT-B slots for two reasons:
 a) There are more than 2 fingers on the pad.
 b) During ambiguous number-of-fingers transitions, the correct track_id
    for one or both of the slots cannot be determined, so the slots are
    invalidated.

Thus, this is a hybrid singletouch/MT-B scheme. Userspace can detect
this behavior by noting that the driver supports more EV_KEY/BTN_TOOL_*TAP
events than its maximum EV_ABS/ABS_MT_SLOT.

Signed-off-by: Daniel Kurtz <djkurtz@chromium.org>
---
 drivers/input/mouse/synaptics.c |  290 ++++++++++++++++++++++++++++++++++++--
 drivers/input/mouse/synaptics.h |    4 +
 2 files changed, 278 insertions(+), 16 deletions(-)

Comments

Henrik Rydberg Aug. 12, 2011, 9:52 p.m. UTC | #1
Hi Daniel,

> Synaptics image sensor touchpads track 5 fingers, but only report 2.
> This patch attempts to deal with some idiosyncrasies of these
> touchpads:

I appreciate the complexity of the protocol at hand, and the patch is
doing an excellent job documenting it. However, it does constitute a
fair deal of code... What about the statements below:

1. When two subsequent sgm packets report the same number of fingers
(and unless told otherwise in some other fashion?), the sgm and the
previous agm contains data for the same fingers as the previous (agm,
sgm) pair, i.e., there is no transition.

2. There is no general way to know which fingers are being reported
after a transition.

If both points above are true, how about something like this:

a) Always report slots zero and one (for two or more fingers)

b) Keep a global number, trackid

c) Parse are buffer agm

d) Parse sgm

e) If a transition occured, increase trackid by two

f) Set TRACKING_ID to trackid in slot zero and trackid+1 in slot one.

It seems to me the above will result in contiuous handling of two
unknown contacts in between transitions, and during transitions, new
contacts will be generated.

The algorithm above could possibly be improved for the 2->1
transition, but from what I gather, 1->2 and 2->3 as well as 3->2 does
not seem to be generally solvable, hence resulting in the above
anyways.

If any of points 1 and 2 are wrong, I am sure you will
enlighten/remind me. ;-)

Cheers,
Henrik
--
To unsubscribe from this list: send the line "unsubscribe linux-input" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Daniel Kurtz Aug. 15, 2011, 7:46 a.m. UTC | #2
Hi Henrik,

Thanks for the feedback.

On Sat, Aug 13, 2011 at 5:52 AM, Henrik Rydberg <rydberg@euromail.se> wrote:
>
> Hi Daniel,
>
> > Synaptics image sensor touchpads track 5 fingers, but only report 2.
> > This patch attempts to deal with some idiosyncrasies of these
> > touchpads:
>
> I appreciate the complexity of the protocol at hand, and the patch is
> doing an excellent job documenting it. However, it does constitute a
> fair deal of code... What about the statements below:
>
> 1. When two subsequent sgm packets report the same number of fingers
> (and unless told otherwise in some other fashion?), the sgm and the
> previous agm contains data for the same fingers as the previous (agm,
> sgm) pair, i.e., there is no transition.
>
> 2. There is no general way to know which fingers are being reported
> after a transition.
>
> If both points above are true, how about something like this:
>
> a) Always report slots zero and one (for two or more fingers)
>
> b) Keep a global number, trackid
>
> c) Parse are buffer agm
>
> d) Parse sgm
>
> e) If a transition occured, increase trackid by two
>
> f) Set TRACKING_ID to trackid in slot zero and trackid+1 in slot one.
>
> It seems to me the above will result in contiuous handling of two
> unknown contacts in between transitions, and during transitions, new
> contacts will be generated.
>
> The algorithm above could possibly be improved for the 2->1
> transition, but from what I gather, 1->2 and 2->3 as well as 3->2 does
> not seem to be generally solvable, hence resulting in the above
> anyways.
>
> If any of points 1 and 2 are wrong, I am sure you will
> enlighten/remind me. ;-)
>
> Cheers,
> Henrik

Actually, "in general", it is possible to know which fingers are being
reported _after_ a transition.

However:
  (1) Since the number of fingers is only reported in SGM packets, it
is not possible, _during_ a transition, to know whether the AGM was
sent for the old number of fingers, or the new number of fingers.
  (2) There is no way to tell which fingers remain when going from
3->2 fingers. Once we lose track, we can't really know what fingers we
are tracking until:
     (a) All fingers are removed
     (b) All but the 'primary' (finger slot[0]) are removed
     (c) 4 or 5 fingers are touched (this triggers an AGM_CONTACT
packet, which properly resets the driver's state)
  (3) Similarly, on 3->1 transitions, if the 'primary' (finger
slot[0]) finger is NOT the one that remains, we don't know which
finger remains.  Tracking state must be restored with (a), (b), or (c)
above.
  (4) A quick finger change, can generate a 1->1 (non-)transition with
an intervening AGM packet. In this case, the new SGM is actually
tracking slot[1], not slot[0].  If a second finger touches, this new
second finger will fill the now-empty slot[0], and, thus, be reported
in the SGM.

This results in a lot of the special handling in the switch/cases.

The resulting algorithm can accurately handle 1->2->1 transitions very
well (including drum roll, and no matter which of the two fingers is
being reported in the second '1' state).
It can deal with 1->2->3 properly in the typical "adding fingers" case.
Lastly, it also realizes when it has lost track (after 3->2 / 3->1
transition), and handles subsequent (1->2, 2->1, 2->3) transitions
carefully (by invalidating slots).

So, I think even if we started with a simpler "always use two new
slots" approach, once we added back the special casing, it would be
just as much code...

A simple example:
  On 2->3, even if we always used two new slots, we still couldn't
report them accurately, immediately, because we don't know whether the
AGM during the transition was for the "2" state, or the "3" state.
Instead, we'd need to do something like what I'm doing now: report
3:[0,-1] "three fingers down, the only one I can report is that the
one in slot[0]"

-Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-input" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/input/mouse/synaptics.c b/drivers/input/mouse/synaptics.c
index ff8c839..57104ed 100644
--- a/drivers/input/mouse/synaptics.c
+++ b/drivers/input/mouse/synaptics.c
@@ -453,6 +453,9 @@  static void synaptics_parse_agm(const unsigned char buf[],
 	default:
 		break;
 	}
+
+	/* Record that at least one AGM has been received since last SGM */
+	priv->agm_pending = true;
 }
 
 static int synaptics_parse_hw_state(const unsigned char buf[],
@@ -600,26 +603,53 @@  static void synaptics_report_slot_agm(struct input_dev *dev, int slot,
 }
 
 static void synaptics_report_mt(struct psmouse *psmouse,
-				int count,
+				struct synaptics_mt_state *mt_state,
 				const struct synaptics_hw_state *sgm)
 {
 	struct input_dev *dev = psmouse->dev;
 	struct synaptics_data *priv = psmouse->private;
 	struct synaptics_hw_state *agm = &priv->agm;
+	struct synaptics_mt_state *old = &priv->mt_state;
 
-	switch (count) {
+	switch (mt_state->count) {
 	case 0:
 		synaptics_report_slot_empty(dev, 0);
 		synaptics_report_slot_empty(dev, 1);
 		break;
 	case 1:
-		synaptics_report_slot_sgm(dev, 0, sgm);
-		synaptics_report_slot_empty(dev, 1);
+		if (mt_state->sgm == -1) {
+			synaptics_report_slot_empty(dev, 0);
+			synaptics_report_slot_empty(dev, 1);
+		} else if (mt_state->sgm == 0) {
+			synaptics_report_slot_sgm(dev, 0, sgm);
+			synaptics_report_slot_empty(dev, 1);
+		} else {
+			synaptics_report_slot_empty(dev, 0);
+			synaptics_report_slot_sgm(dev, 1, sgm);
+		}
 		break;
-	case 2:
-	case 3: /* Fall-through case */
-		synaptics_report_slot_sgm(dev, 0, sgm);
-		synaptics_report_slot_agm(dev, 1, agm);
+	default:
+		/*
+		 * If the finger slot contained in SGM is valid, and either
+		 * hasn't changed, or is new, then report SGM in MTB slot 0.
+		 * Otherwise, empty MTB slot 0.
+		 */
+		if (mt_state->sgm != -1 &&
+		    (mt_state->sgm == old->sgm || old->sgm == -1))
+			synaptics_report_slot_sgm(dev, 0, sgm);
+		else
+			synaptics_report_slot_empty(dev, 0);
+
+		/*
+		 * If the finger slot contained in AGM is valid, and either
+		 * hasn't changed, or is new, then report AGM in MTB slot 1.
+		 * Otherwise, empty MTB slot 1.
+		 */
+		if (mt_state->agm != -1 &&
+		    (mt_state->agm == old->agm || old->agm == -1))
+			synaptics_report_slot_agm(dev, 1, agm);
+		else
+			synaptics_report_slot_empty(dev, 1);
 		break;
 	}
 
@@ -627,28 +657,256 @@  static void synaptics_report_mt(struct psmouse *psmouse,
 	input_mt_report_pointer_emulation(dev, false);
 
 	/* Send the number of fingers reported by touchpad itself. */
-	input_mt_report_finger_count(dev, count);
+	input_mt_report_finger_count(dev, mt_state->count);
 
 	input_report_key(dev, BTN_LEFT, sgm->left);
 	input_sync(dev);
 }
 
+/* Handle case where mt_state->count = 0 */
+static void synaptics_image_sensor_0f(struct synaptics_data *priv,
+				      struct synaptics_mt_state *mt_state)
+{
+	synaptics_mt_state_set(mt_state, 0, -1, -1);
+	priv->mt_state_lost = false;
+}
+
+/* Handle case where mt_state->count = 1 */
+static void synaptics_image_sensor_1f(struct synaptics_data *priv,
+				      struct synaptics_mt_state *mt_state)
+{
+	struct synaptics_hw_state *agm = &priv->agm;
+	struct synaptics_mt_state *old = &priv->mt_state;
+
+	/*
+	 * If the last AGM was (0,0,0), and there is only one finger left,
+	 * then we absolutely know that SGM contains slot 0, and all other
+	 * fingers have been removed.
+	 */
+	if (priv->agm_pending && agm->z == 0) {
+		synaptics_mt_state_set(mt_state, 1, 0, -1);
+		priv->mt_state_lost = false;
+		return;
+	}
+
+	switch (old->count) {
+	case 0:
+		synaptics_mt_state_set(mt_state, 1, 0, -1);
+		break;
+	case 1:
+		/*
+		 * If mt_state_lost, then the previous transition was 3->1,
+		 * and SGM now contains either slot 0 or 1, but we don't know
+		 * which.  So, we just assume that the SGM now contains slot 1.
+		 *
+		 * If pending AGM and either:
+		 *   (a) the previous SGM slot contains slot 0, or
+		 *   (b) there was no SGM slot
+		 * then, the SGM now contains slot 1
+		 *
+		 * Case (a) happens with very rapid "drum roll" gestures, where
+		 * slot 0 finger is lifted and a new slot 1 finger touches
+		 * within one reporting interval.
+		 *
+		 * Case (b) happens if initially two or more fingers tap
+		 * briefly, and all but one lift before the end of the first
+		 * reporting interval.
+		 *
+		 * (In both these cases, slot 0 will becomes empty, so SGM
+		 * contains slot 1 with the new finger)
+		 *
+		 * Else, if there was no previous SGM, it now contains slot 0.
+		 *
+		 * Otherwise, SGM still contains the same slot.
+		 */
+		if (priv->mt_state_lost ||
+		    (priv->agm_pending && old->sgm <= 0))
+			synaptics_mt_state_set(mt_state, 1, 1, -1);
+		else if (old->sgm == -1)
+			synaptics_mt_state_set(mt_state, 1, 0, -1);
+		break;
+	case 2:
+		/*
+		 * If mt_state_lost, we don't know which finger SGM contains.
+		 *
+		 * So, report 1 finger, but with both slots empty.
+		 * We will use slot 1 on subsequent 1->1
+		 */
+		if (priv->mt_state_lost) {
+			synaptics_mt_state_set(mt_state, 1, -1, -1);
+			break;
+		}
+		/*
+		 * Since the last AGM was NOT (0,0,0), it was the finger in
+		 * slot 0 that has been removed.
+		 * So, SGM now contains previous AGM's slot, and AGM is now
+		 * empty.
+		 */
+		synaptics_mt_state_set(mt_state, 1, old->agm, -1);
+		break;
+	case 3:
+		/*
+		 * Since last AGM was not (0,0,0), we don't know which finger
+		 * is left.
+		 *
+		 * So, report 1 finger, but with both slots empty.
+		 * We will use slot 1 on subsequent 1->1
+		 */
+		synaptics_mt_state_set(mt_state, 1, -1, -1);
+		priv->mt_state_lost = true;
+		break;
+	}
+}
+
+/* Handle case where mt_state->count = 2 */
+static void synaptics_image_sensor_2f(struct synaptics_data *priv,
+				      struct synaptics_mt_state *mt_state)
+{
+	struct synaptics_mt_state *old = &priv->mt_state;
+
+	switch (old->count) {
+	case 0:
+		synaptics_mt_state_set(mt_state, 2, 0, 1);
+		break;
+	case 1:
+		/*
+		 * If previous SGM contained slot 1 or higher, SGM now contains
+		 * slot 0 (the newly touching finger) and AGM contains SGM's
+		 * previous slot.
+		 *
+		 * Otherwise, SGM still contains slot 0 and AGM now contains
+		 * slot 1.
+		 */
+		if (old->sgm >= 1)
+			synaptics_mt_state_set(mt_state, 2, 0, old->sgm);
+		else
+			synaptics_mt_state_set(mt_state, 2, 0, 1);
+		break;
+	case 2:
+		/*
+		 * If mt_state_lost, SGM now contains either finger 1 or 2, but
+		 * we don't know which.
+		 * So, we just assume that the SGM contains slot 0 and AGM 1.
+		 */
+		if (priv->mt_state_lost)
+			synaptics_mt_state_set(mt_state, 2, 0, 1);
+		/*
+		 * Otherwise, use the same mt_state, since it either hasn't
+		 * changed, or was updated by a recently received AGM-CONTACT
+		 * packet.
+		 */
+		break;
+	case 3:
+		/*
+		 * 3->2 transitions have two unsolvable problems:
+		 *  1) no indication is given which finger was removed
+		 *  2) no way to tell if agm packet was for finger 3
+		 *     before 3->2, or finger 2 after 3->2.
+		 *
+		 * So, report 2 fingers, but empty all slots.
+		 * We will guess slots [0,1] on subsequent 2->2.
+		 */
+		synaptics_mt_state_set(mt_state, 2, -1, -1);
+		priv->mt_state_lost = true;
+		break;
+	}
+}
+
+/* Handle case where mt_state->count = 3 */
+static void synaptics_image_sensor_3f(struct synaptics_data *priv,
+				      struct synaptics_mt_state *mt_state)
+{
+	struct synaptics_mt_state *old = &priv->mt_state;
+
+	switch (old->count) {
+	case 0:
+		synaptics_mt_state_set(mt_state, 3, 0, 2);
+		break;
+	case 1:
+		/*
+		 * If previous SGM contained slot 2 or higher, SGM now contains
+		 * slot 0 (one of the newly touching fingers) and AGM contains
+		 * SGM's previous slot.
+		 *
+		 * Otherwise, SGM now contains slot 0 and AGM contains slot 2.
+		 */
+		if (old->sgm >= 2)
+			synaptics_mt_state_set(mt_state, 3, 0, old->sgm);
+		else
+			synaptics_mt_state_set(mt_state, 3, 0, 2);
+		break;
+	case 2:
+		/*
+		 * After some 3->1 and all 3->2 transitions, we lose track
+		 * of which slot is reported by SGM and AGM.
+		 *
+		 * For 2->3 in this state, report 3 fingers, but empty all
+		 * slots, and we will guess (0,2) on a subsequent 0->3.
+		 *
+		 * To userspace, the resulting transition will look like:
+		 *    2:[0,1] -> 3:[-1,-1] -> 3:[0,2]
+		 */
+		if (priv->mt_state_lost) {
+			synaptics_mt_state_set(mt_state, 3, -1, -1);
+			break;
+		}
+
+		/*
+		 * If the (SGM,AGM) really previously contained slots (0, 1),
+		 * then we cannot know what slot was just reported by the AGM,
+		 * because the 2->3 transition can occur either before or after
+		 * the AGM packet. Thus, this most recent AGM could contain
+		 * either the same old slot 1 or the new slot 2.
+		 * Subsequent AGMs will be reporting slot 2.
+		 *
+		 * To userspace, the resulting transition will look like:
+		 *    2:[0,1] -> 3:[0,-1] -> 3:[0,2]
+		 */
+		synaptics_mt_state_set(mt_state, 3, 0, -1);
+		break;
+	case 3:
+		/*
+		 * If, for whatever reason, the previous agm was invalid,
+		 * Assume SGM now contains slot 0, AGM now contains slot 2.
+		 */
+		if (old->agm <= 2)
+			synaptics_mt_state_set(mt_state, 3, 0, 2);
+		/*
+		 * mt_state either hasn't changed, or was updated by a recently
+		 * received AGM-CONTACT packet.
+		 */
+		break;
+	}
+}
+
 static void synaptics_image_sensor_process(struct psmouse *psmouse,
 					   struct synaptics_hw_state *sgm)
 {
-	int count;
+	struct synaptics_data *priv = psmouse->private;
+	struct synaptics_hw_state *agm = &priv->agm;
+	struct synaptics_mt_state mt_state;
 
+	/* Initialize using current mt_state (as updated by last agm) */
+	mt_state = agm->mt_state;
+
+	/*
+	 * Update mt_state using the new finger count and current mt_state.
+	 */
 	if (sgm->z == 0)
-		count = 0;
+		synaptics_image_sensor_0f(priv, &mt_state);
 	else if (sgm->w >= 4)
-		count = 1;
+		synaptics_image_sensor_1f(priv, &mt_state);
 	else if (sgm->w == 0)
-		count = 2;
-	else
-		count = 3;
+		synaptics_image_sensor_2f(priv, &mt_state);
+	else if (sgm->w == 1)
+		synaptics_image_sensor_3f(priv, &mt_state);
 
 	/* Send resulting input events to user space */
-	synaptics_report_mt(psmouse, count, sgm);
+	synaptics_report_mt(psmouse, &mt_state, sgm);
+
+	/* Store updated mt_state */
+	priv->mt_state = agm->mt_state = mt_state;
+	priv->agm_pending = false;
 }
 
 /*
diff --git a/drivers/input/mouse/synaptics.h b/drivers/input/mouse/synaptics.h
index 20f57df..622aea8 100644
--- a/drivers/input/mouse/synaptics.h
+++ b/drivers/input/mouse/synaptics.h
@@ -161,11 +161,15 @@  struct synaptics_data {
 
 	struct serio *pt_port;			/* Pass-through serio port */
 
+	struct synaptics_mt_state mt_state;	/* Current mt finger state */
+	bool mt_state_lost;			/* mt_state may be incorrect */
+
 	/*
 	 * Last received Advanced Gesture Mode (AGM) packet. An AGM packet
 	 * contains position data for a second contact, at half resolution.
 	 */
 	struct synaptics_hw_state agm;
+	bool agm_pending;			/* new AGM packet received */
 };
 
 void synaptics_module_init(void);