Reconstructions from brain activity: 3 subjects

For an explanation of the procedure please see the other video on this channel: http://www.youtube.com/watch?v=nsjDnY...
This video is organized as follows: the movie that each subject viewed while in the magnet is shown at upper left. Reconstructions for three subjects are shown in the three rows at bottom. All these reconstructions were obtained using only each subject's brain activity and a library of 18 million seconds of random YouTube video that did not include the movies used as stimuli. (In brief, the algorithm processes each of the 18 million clips through the brain model, and identifies the clips that would have produced brain activity as similar to the measured activity as possible. The clips used to fit the model, the clips used to test the model and the clips used to reconstruct the stimulus were entirely separate.) The reconstruction at far left is the Average High Posterior (AHP). The reconstruction in the second column is the Maximum a Posteriori (MAP). The other columns represent less likely reconstructions. The AHP is obtained by simply averaging over the 100 most likely movies in the reconstruction library. These reconstructions show that the process is very consistent, though the quality of the reconstructions does depend somewhat on the quality of brain activity data recorded from each subject.
For the paper (Nishimoto et al., 2011, Current Biology) go to: http://dx.doi.org/10.1016/j.cub.2011....
You can find more information about this work at our laboratory web site: http://gallantlab.org

• Category

  Science & Technology

[Hoopie Hoggle]

Now we can get sued for copyright infringement for simply having a song stuck in our heads.

[gallantlabucb]

You are correct, we did not do optimal averaging here. We simply summed across the pixels, whereas a better averaging technique would involve aligning the objects in the videos first. We didn't do that here because our main goal was to build good ENcoding models, and the DEcoding results that you see here are just a side issue.

[llVIU]

as if just seeing this shit isn't creepy enough, now you want some creepy sound reconstruction too?? search here on youtube first recording ever... that's some creepy fucking shit

[Max Loh]

I am occasionally able to remember up to 4 measures of a song from a dream. You have to wake up in "layers" like in Inception. Each time you wake up a bit, you review the music you created in your head, you play it for yourself about 3 times, then wake up further. Do anything you need, even write it down on a piece of paper many times IN THE DREAM. The last layer (real waking up) is the hardest gap to bridge but can be done.

[Will Ford]

Johnny Mnemonic - 1996

[fabianhiti]

It actually got the airplane right! :D this is incredible :3

[Helium]

LOL i see hitler

[DAT ONEGUY]

what's that one movie called that had this in it??? and the dude's brain got fried cause of too much activity

[movesliketager]

0:05 Duke Nukem

[Joey Paris]

i see bulma and goku several times

[Frédéric Marcotte Piché]

I'm about to get a Muse (Bluetooth EEG), which you can find on indiegogo, along with the development tools, and I'd like to know how this is achieved, and if you plan on making your software open-source so everybody can contribute to building better algorithms and better databases for brain image reconstruction.

I'd be highly interested in helping you achieve quality brain imaging!

[mahpokemons]

I wonder what would happen if they did this to someone on acid ._.