Visual Image Reconstruction from Human Brain

Visual Image Reconstruction from Human
Brain Activity using a Combination of
Multiscale Local Image Decoders.

Yoichi Miyawaki,1,2,6 Hajime Uchida,2,3,6 Okito Yamashita,2 Masa-aki Sato,2 Yusuke Morito,4,5 Hiroki C. Tanabe,4,5
Norihiro Sadato,4,5 and Yukiyasu Kamitani2,3,*
1National Institute of Information and Communications Technology, Kyoto, Japan
2ATR Computational Neuroscience Laboratories, Kyoto, Japan
3Nara Institute of Science and Technology, Nara, Japan
4The Graduate University for Advanced Studies, Kanagawa, Japan
5National Institute for Physiological Sciences, Aichi, Japan
6These authors contributed equally to this work
*Correspondence: kmtn@atr.jp
DOI 10.1016/j.neuron.2008.11.004



SUMMARY
Perceptual experience consists of an enormous number
of possible states. Previous fMRI studies have
predicted a perceptual state by classifying brain
activity into prespecified categories. Constraint-free
visual image reconstruction is more challenging, as
it is impractical to specify brain activity for all possible
images. In this study, we reconstructed visual images
by combining local image bases of multiple scales,
whose contrasts were independently decoded from
fMRI activity by automatically selecting relevant voxels
and exploiting their correlated patterns. Binarycontrast,
103 10-patch images (2100 possible states)
were accurately reconstructed without any image
prior on a single trial or volume basis by measuring
brain activity only for several hundred random
images. Reconstruction was also used to identify
the presented image among millions of candidates.
The results suggest that our approach provides an effective
means to read out complex perceptual states
from brain activity while discovering information
representation in multivoxel patterns.



Neuron 60, 915929, December 11, 2008 ª2008 Elsevier Inc.

Category:

Science & Technology

Tags:

• Visual
• Image
• Reconstruction
• from
• Human
• Brain
• Activity
• using
• Combination
• of
• Multiscale
• Local
• Decoders.

[Incrediblastic]

Is it possible to get the full pdf of the test results?I know that you can buy it,but if there is any other way,that would be great :)

[capaneo]

If you want the full PDF of the article PM me your Email and I will send it to you.

[Soulrazr]

This is the beginning of the end of free thought

[essemmetv]

apparently this technology may be leading us to visualize our dreams. and maybe we could also see what a patients sees when he dies?

[alexgrinkov]

it seems like they actually could filter out noise by comparing current and previous frames and removing unmatching pixels

then the picture would be almost perfect

[oguretsagressive]

@essemmetv Maybe not. The visual cortex, which they are scanning, doesn't "see" the dreams. If you remember yourself waking up during REM phase, you might have noticed that your "dream image" isn't real, it's like an overlay above the usual black background with flashing dots, which you see with your eyes closed. It was there all the time, you just stopped noticing it, distracted by a false vision. Despite of that, I'm admired by japanese scientists. They don't afraid to explore the unknown.

[anonthx]

@Incrediblastic

cell.com/neuron/abstract/S0896­-6273(08)00958-6

[kargaroc386]

@Hallo2244 apparently someone doesn't want you to ever use it, because they all worry about ethical things.

I really want to see my thoughts on a computer screen. but apparently someone else doesn't want me to be able to do that.

[Hallo2244]

This is amazing, when the technology gets further this will have so many implications! Have you ever imagined something almost unimaginable, and you lack the artistic skills or linguistic skills to explain or draw it? Soon enough that won't be a problem, and if someone has raped someone, or robbed someone, or maybe even murdered someone, we'll be able to express their face through this technology :D Technology amazes me!!

[ProgramJava]

Though I didn't fully read the article, they likely looked at fmri data, used multiple voxel (voxel being a "set" of data corresponding to a part of the brain in the fmri) and then passed it through a network which attempted to reconstruct the image, which appears uses statistics and weights to combine "decoders". To me it appears this method might be cumbersome to expand further.

Also, it would be interesting to see effects of imagining instead of seeing.

[jasonguyperson]

wow!