Hadoop Visualization

This is a visualization of the data transfers inside a Hadoop cluster. This is all Live data from a cluster at the University of Nebraska - Lincoln. The darker red a square, the busier that square is. The streams going straight up are transfers in and out of the internet (gridftp servers).

The hotter (redder) the ground is, the more active that segment is.

Source is available publicly at: svn://t2.unl.edu/brian/HadoopViz/trunk

Category:

Science & Technology

Tags:

• HadoopViz
• new

[djw8605]

@SciPoly I added a link to the source in the description.

@dawgp0und That's actually very close to how I do it. Python for a 'collector', and c/c++ for a client visualization. For a network visualization, you can see my other visualization title "Security Visuallization"

[dawgp0und]

@SciPoly It's easy to write something similar. You can monitor the ethernet ports and their ip (it can be easily written in C++/Python with modules) to determine where the packets are going and based on that create a dynamic graph. it doesn't necessarily have to be this visually unless that is what you are going after. As for me, i like simplicity. I hope that helps.

[SciPoly]

Cool how do you generate it, can you give me a link or some more information so I can do it for our cluster too?