Software Engineering Abstractions for the Multi-Touch Revolution (2)

Multi-touch interfaces are inherently concurrent and provide a continuously stream of events,
which makes it very hard to implement advanced and collaborative gestures using traditional event handlers.
We propose a rule based language extended with spatio-temporal operators that allows a high-level approach for describing gestures.

(defrule Drag
(declare (salience 5))
?eventList[] <- (setof Cursor (same: tuioId) (tWithin: 500) (min: 2) (retract: all_but_one))
?image <- (Image)
(test (inside ?eventList[0].x ?eventList[0].y ?image.x ?image.y ?image.width ?image.height))
(not (and (Cursor (x ?x) (y ?y) {(tuioId <> ?eventList[0].tuioId)}) (test (inside ?x ?y ?image.x ?image.y ?image.width ?image.height))))
=>
(printout t "drag detected!")
(assert (Drag (events ?eventList[]) (image ?image))))




(defrule DoubleTap
(declare (salience 6))
?a1 <- (Tap (tuio_id ?aId) (objectFact ?cId))
?a2 <- (Tap (tuio_id ?aId) (objectFact ?cId) {on < (+ ?a1.time 1200)})
(test (approxEqualDistance ?a1.x ?a1.y ?a2.x ?a2.y))
=>
(printout t "DoubleTap" crlf)
(retract ?a1)
(retract ?a2)
(assert (DoubleTap (id ?a1.UID) (tuioId ?aId) (x ?a1.x) (y ?a1.y) (on ?a2.time)
(objectFact ?cId))))




(defrule CircleMerge
(declare (salience 7))
?a1 <- (Gesture (type "CircleCW"))
?a2 <- (Gesture (type "CircleCCW"))
?c1 <- (Image)
?c2 <- (Image)
(test (approxEqualTime ?a1.time ?a2.time))
(test (< ?a1.xl ?a2.xl))
(test (inside ?a1.xl ?a1.yl ?c1.x ?c1.y ?c1.w ?c1.h))
(test (inside ?a2.xl ?a2.yl ?c2.x ?c2.y ?c2.w ?c2.h))
=>
(printout t "CircleMerge (" ?a1.xl ", "?a1.yl") ("?c1.x ","?c1.y ")" crlf)
(retract ?a1)
(retract ?a2)
(?c1.OBJECT merge ?c2.OBJECT))

Category:

Science & Technology

Tags:

• Multi-touch
• Rule language
• Rete
• Gesture Recognition
• Complex Event processing