The EyeHarp: An Eye Tracking Based Musical Instrument

By Zacharias Vamvakousis. A free and open source project. More info: http://theeyeharp.blogspot.com/

The main goal of the EyeHarp project is to allow people with paralysis resulting from Amyotrophic Lateral Sclerosis to play music using only their eyes. To build this, I was inspired by the EyeWriter open source initiative: "...a low-cost eye-tracking apparatus & custom software that allows graffiti writers and artists with paralysis resulting from Amyotrophic lateral sclerosis to draw using only their eyes". (http://www.eyewriter.org)

Alternatively, the instrument can be controlled using the mouse pointer (MouseHarp version). Then the free software camera mouse (http://www.cameramouse.org/) can be used to control the instrument with head movements. Any technology that can take control of the mouse pointer can be used in order to control the instrument. That way the mouseHarp could be an appropriate instrument for many cases of people with physical disabilities. The mouseHarp version is completely independent from the eyeWriter project. Combining the mouseHarp source with the source of the eyeWriter project, we get the eyeHarp! A low-cost gaze controlled musical instrument! Both versions are free and open source.

The main contribution of this project is the design of an appropriate interface to play music using any eye tracing device. In order to evaluate the interface, a low-cost eye tracking device was constructed using the instructions of the EyeWriter Project. I am grateful to the EyeWriter team. Without them I would never be able to develop the EyeHarp, as I could never afford buying a commercial 10,000 euros eye tracking device in order to have a usability evaluation while developing the project! Instead thanks to the EyeWriter initiative I spent only 50 euros for the eye tracker. The whole program is implemented in openframeworks v0.6. (http://www.openframeworks.cc/)

The EyeHarp project so far is my master in Sound And Music Computing in UPF, Barcelona. My supervisor is Rafael Ramirez.

Read the master thesis:
http://mtg.upf.edu/system/files/publications/Vamvakousis-Zacharias-Master-the...

Category:

Music

Tags:

• eye
• tracking
• harp
• eye harp
• EyeHarp
• ματοάρπα
• ματοαρπα
• ματο
• άρπα
• matoarpa
• music
• gaze
• controlled
• musical
• instrument
• mouseHarp
• headHarp
• camera mouse
• EyeWriter
• OpenFrameWorks
• Theremin
• arpeggiator
• disabled
• Quality of life
• Amyotropic
• Lateral
• Sclerosis
• paralyzed
• digital musical instrument
• electronic
• interface
• interactive
• step
• sequencer
• real
• time
• interaction
• sound
• and
• computing
• design
• HCI
• human
• computer
• EyeMusic
• software
• experimental

[mandotron]

is it possible to use the eye harp as a controller for midi devices? say if i wanted to use a piano sound instead of the sounds already there?

[TheEyeHarp]

@mandotron

Not yet, but I just started looking into it a few hours before I read your post mandotron. This is a coincidence! For my master thesis I wanted to come up with sth relatively stable as soon as possible. Now I guess there is more time to improve many things, including using it as a midi controlled. So stay tuned :)

[TheEyeHarp]

@mandotron

Did it! In combination with the LoopBe internal midi port freeware right now I control any vst virtual instrument with my eyes. The following days I will upload the executable and source code of this. So stay tuned :)

[LostButSeeking]

Good job but to be honest i found the sound of theremin a bit annoying!

Personal taste , i guess..

Anyway , I hope this makes a hit and help people with dissabilities express their music. :)

[TheEyeHarp]

@LostButSeeking

Many people say that. You can choose more sounds in the interface. Download the software and play with it with your mouse. More sounds will be included in the future and maybe it could also work as a midi controller, where the user will be able to use his own sound bank.

[stroulou]

its the best thing i have ever seen

[TheEyeHarp]

@stroulou Ποιος/ποια είσαι; Για να μιλάς με τόσο καλά λόγια, φίλος/φίλη θα είσαι!!!

[anatoli72cristina]

Η εφεύρεση συμμετέχει στον διαγωνισμό του συνεδρίου TedXAthens και εάν επιλεγεί, θα στηριχτεί η περαιτέρω ανάπτυξή της.

[anatoli72cristina]

Η εφεύρεση, που λέγεται EyeHarp, είναι ένα ηλεκτρονικό μουσικό όργανο το οποίο μπορεί να παίξει κανείς επιλέγοντας νότες, ένταση ήχου κ.λπ. με το βλέμμα ή με κινήσεις του κεφαλιού

[anatoli72cristina]

Η εφεύρεση αυτή αναπτύχθηκε από έναν έλληνα επιστήμονα, απόφοιτο του τμήματος Πληροφορικής του Πανεπιστημίου Αθηνών. Ο εφευρέτης, Ζαχαρίας Βαμβακούσης, ανέπτυξε το μουσικό αυτό όργανο που μπορεί να χρησιμοποιηθεί από άτομα με σοβαρές κινητικές δυσκολίες, στα πλαίσια της διπλωματικής του εργασίας.

[RothMobot]

Excellent work.

[thanos18]

τί άλλο να πει κανείς παρα μόνο μπράβο? 1000 μπράβο

[mandotron]

excellent work by the way. your thesis was a good read too

[antirisi]

Man, my project is open source and it is in open frameworks. For open frameworks you can study the book programming interactivity. Since the project is open source, a part of your project could be adding new features to it or improving it. What eye tracker are you going to use? Visit the web page of ITU Gaze Tracker to biuld your won eye tracker!