Curating the Dark Data in the Long Tail of Science

Google Tech Talks
August 28, 2008

ABSTRACT

There is a wealth of scientific data that is almost impossible to see. This is science's dark data. Much of this data resides in the long tail of science or "small" data collection efforts. Instrumentation has made it possible to develop large collections of relatively homogeneous data, be it from space sensors or high throughput gene sequencers. The monolithic collections are easy to find and search. Dark data on the other hand may constitute the larger mass of scientific information. The collections that make up the dark data of science are much smaller but also much more numerous, being generated by thousands of scientists, on a much broader number of scientific questions, and in a complex array of formats. Unfortunately, it is also more prone to be overlooked and lost over time. Using new technology, the economics of the internet, and change in the sociology of science it is possible to make greater use of this data than was possible in the past. Data curators are the people who develop and use these technologies and procedures to make this data more useful, insuring a more efficient return on investment in the enterprise of science.

Speaker: P. Bryan Heidorn
Program Manager, National Science Foundation Division of Biological Infrastructure and Associate Professor, University of Illinois

Category:

Science & Technology

Tags:

• google
• techtalks
• techtalk
• engedu
• talk
• talks
• googletechtalks
• education

[pbheidorn]

Actually, I excluded a few dozen $1 grants... although the amount of data they generate might be far greater than the dollar value might suggest.

[ericjain]

Dealing effectively with long tail data is an important challenge, great to see that some people are giving this some thought. Bryan Heidorn does a good job of presenting the problem, but there isn't too much discussion of potential solutions.

[bjornsmedman]

@13.15: Whos the lucky bastard that got a $831 research grant? :)