ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures

We present ShadowGuides, a system for in-situ learning of multi-touch and whole-hand gestures on interactive surfaces. ShadowGuides provides on-demand assistance to the user by combining visualizations of the users current hand posture as interpreted by the system (feedback) and available postures and completion paths necessary to finish the gesture (feedforward). Our experiment compared participants learning gestures with ShadowGuides to those learning with video-based instruction. We found that participants learning with ShadowGuides remembered more gestures and expressed significantly higher preference for the help system.

Category:

Science & Technology

Tags:

• Gesture learning
• multi-finger
• displacement
• marking menus
• multi-touch
• surface computing

[ElCervantes]

Nice approach, although ideally I would imagine shadow guide as an API that pipes the recognized events and processes to given direct manipulation applications that then react in a comprehensible manner changing their visual state according to the specific gesture (similar to a translucent file-icon being displayed during drag&drop). Thus the user can learn the gestures effortlessly while already performing productive interactions. Keep up the nice work!