RESONANT VISION

[bsodfc]

This is amazing!

[jestertru]

How much of the periodicity of these signals is an artifact of the heart pumping blood (and hence a kick of metabolic activity) and from the scan rate of the data extractions itself. The modulation of the sound seems a bit too close to either rates.

Also makes me think of Jazz and wonder if some of like it is that it "maps" well to how the brain's distributed processing works.

[dlloyd1984]

Glad you asked, as this will provide a footnote to the video. You're right, the basic rhythm is that of the image acquisition itself, 1 image per second. (So, each run in the video has been speeded up considerably, an aesthetic choice.) In real time, each "take" in the video would run about 3 minutes. (more...)

[dlloyd1984]

(part 2:) The fast moving components, rendered as steel drums and talko drums, oscillate about every two seconds, so these are candidates for a physiological rhythm. In principle, ICA should isolate one or two regions that follow this rhythm; these might rightly be regarded as artifacts.

The general rise and fall (in pitch) of the other regions arguably reflects the blood oxygen level and hence the underlying neural activity. These interweaving melodic lines thus may have cognitive meaning.

[dlloyd1984]

(part 3) Is it appropriate to invoke jazz? That would be nice. A musician/composer who has heard some of the other videos on this channel says that the braintracks are in 2/4 time. Jazziness might be a property of more complex rhythms; to my ear, this video does have a little swing.

[dlloyd1984]

(part 4!) In one respect, however, the jazz sound is my addition, and not intrinsic to the data: I assigned the intensity values in the fMRI signals to a blues scale. I could have used a chromatic scale, pentatonic, or arbitrary frequencies. That was an aesthetic choice. However, the mapping of rising tones to increasing fMRI signal intensity is a genuine feature of the data. That would be audible regardless of all the other choices. (end of reply, at last....)

[TheNothingRoom]

I don't think you're hearing the brain's signals actually create anything reminiscent of jazz here. The mapping is to a jazz scale; you could also map this to other scales from other cultures and have a different but likely equally interesting result. In algorithmic composition the mapping typically indicates the genre, whereas the algorithmic material dictates the inspiration. Very interesting nonetheless.

[kzatching]

awesome!