Google Tech Talks
November 7, 2008
ABSTRACT
To evaluate an innovation in computer systems, performance analysts measure execution time or other metrics using one or more standard workloads. The p...
Google Tech Talks November 7, 2008
ABSTRACT
To evaluate an innovation in computer systems, performance analysts measure execution time or other metrics using one or more standard workloads. The performance analyst may carefully minimize the amount of measurement instrumentation, control the environment in which measurement takes place, and repeat each measurement multiple times. Finally, the performance analyst may use statistical techniques to characterize the data.
Unfortunately, even with such a responsible approach, the collected data may be misleading due to measurement bias and observer effect. Measurement bias occurs when the experimental setup inadvertently favors a particular outcome.
Observer effect occurs if data collection alters the behavior of the system being measured. This talk demonstrates that observer effect and measurement bias are (i) large enough to mislead performance analysts; and (ii) common enough that they cannot be ignored.
While these phenomenon are well known to the natural and social sciences this talk will demonstrate that research in computer systems typically does not take adequate measures to guard against measurement bias and observer effect.
Speaker: Amer Diwan Amer Diwan is an associate professor at the University of Colorado at Boulder. Before joining the University of Colorado, he was at the University of Massachusetts at Amherst for his PhD and at Stanford University for his postdoc. His research interests include tools and techniques for understanding program performance, programmer productivity tools, program analysis, memory management, and compiler optimizations.
Like to rate videos and let people know what you think?
Automatically share your ratings, favorites, and more on Facebook, Twitter, and Google Reader with YouTube Autoshare.
Autoshare makes certain YouTube activities public on the services you choose. Select only the services you are comfortable with - like Facebook, Twitter, or Google Reader - to let your friends know what you like on YouTube. You can turn Autoshare off at any time.
Like to share videos with friends?
Automatically share your ratings, favorites, and more on Facebook, Twitter, and Google Reader with YouTube Autoshare.
Autoshare makes certain YouTube activities public on the services you choose. Select only the services you are comfortable with - like Facebook, Twitter, or Google Reader - to let your friends know what you like on YouTube. You can turn Autoshare off at any time.
One of the best talk on the subject I have ever come across in many years. I am sure if we follow the points highlighted in the talk will help us. I belive Amer will give more talks on other subjects in future. Wish him all the best.
Autoshare makes certain YouTube activities public on the services you choose. Select only the services you are comfortable with - like Facebook, Twitter, or Google Reader - to let your friends know what you like on YouTube. You can turn Autoshare off at any time.