Calisto - Collaborative Media Exchange Terminal for the Internet of Services

In this video, Simon and Matthieu present their new Calisto terminal. Calisto is an innovative kiosk terminal that enables multimodal access to semantic Web services. It uses annotated media, such as pictures, videos and texts to initiate complex search processes in linked combined Web services and databases.

With our solution users synchronize a shared folder of their mobile device with the terminal. Via a special Spotlet users can access all included media. Furthermore it is also possible to throw a selected picture to the terminal. In the video Simon holds his Android Smartphone and selects a pre-annotated picture out of his shared folder. With a new interaction paradigm, called "Frisbee-gesture", a selected picture of the Brandenburg Gate is directly synchronized with the terminal and displayed on the screen. Based on the media's attached annotations a similar search (using Flickr, Google Images) is submitted over drag'n'drop and corresponding results are displayed.

Matthieu explains how a more detailed semantic search can be achieved by using linked semantic terms. These terms are listed in the side panel of the Spotlet and can be used to get more relations and media concerning the city of Berlin. In the video Simon found the German Computer pioneer Konrad Zuse who was born in Berlin, and corresponding videos.

One major feature of Calisto is that in addition to touch -- and gesture interactions, users can also interact with the terminal via natural language. To realize this, the user only needs to speak into the Smartphone (using the Android-based Calisto App) and to utter a complex speech command like "Show me Berlin on the map". After this command, semantic Web services are triggered and the results are shown on the terminal's screen, in this case a Spotlet will show a detailed map of Berlin.

The video shows also how Simon can share interesting results -- here a picture of Berlin - directly copy it to Matthieu's Smartphone. This type of interaction enables new collaborative tasks and opens new interaction possibilities with kiosk systems and multitouch terminals.

The Calisto system was developed in at the DFKI's AdvanTI-Lab in Saarbrücken, Germany

This research has been supported by the research program THESEUS, which is funded by the German Federal Ministry of Economics and Technology. The responsibility for this video lies with the DFKI.

Category:

Science & Technology

Tags:

• Semantische Services
• Calisto
• Semantic
• Semantic Service
• terminal
• Speech interactions
• objects
• Collaborative
• Media Exchange Terminal
• recommendations
• Web 2.0
• Web 3.0
• Twitter
• mobile phones
• konrad zuse
• theseus
• multimodal technology
• speech interaction
• multimodal
• annotations
• ontology
• RDF
• OWL
• Ontology-based
• dialogue platform
• spotlet
• spotlets
• community
• multitouch
• DFKI
• iPhone
• iPod
• android
• htc desire
• nexus s
• touch