 systematic review that our Carnegie Mellon team has been working on, deals with the extent to which the way CEOs are paid and the financial incentives they receive, to what extent does that predict the performance of their firms? And that's an important question because CEOs are paid a lot and a lot of effort and energy goes into setting their pay scale up and yet we don't really know if it makes a difference to the quality of the outcome and our review was intended to determine whether or not all the years of research that exists on this if there actually is a clear answer to that question. We had a broader question at first about performance because there's many kinds of performance that you could write a CEO contract in order to realize including corporate social responsibility, employee welfare, and we hit on focusing on financial performance, not because the others are unimportant, but because we see this as a first systematic review on the connection between financial incentives for executives and business outcomes, trying to show a proof of concept since we're the first systematic review done in management through the Campbell systematic review library that we want to provide a search of proof of concept so that other academic teams can come later and look at other outcomes in relation to CEO financial incentives or trying to set it up as a first step for a larger program and that was negotiating with the libraries what we should focus on we were thinking about our bigger picture plan to have a positive impact on practice and scholarship. That's a very important idea that systematic reviews can contribute to both the practice and the scholarship of management. Management research has been described as a really big tent. It's probably a circus tent, a three-ring, 16-ring circus with many animals and many clowns and many dancers and events and it's not necessarily clear when one looks at a body of evidence what direction it's pointing in. We have researchers in management who come from a sociology background others like myself or a psychologist still others whose training is in business and management and these different groups don't use concepts in the same way they use different kinds of research methods and so because of that diversity of approach and vocabulary and the labels that people give to measures and variables it can be really difficult even for a well skilled well focused and supported scholar to determine what the body of evidence on a management question is what it really says and that's really where the process of systematic reviews comes to be valuable. A because there's a methodology for doing systematic reviews that have been developed over the years by scholars in substantive areas but also scholars in information science and in what was once known as librarianship now is you know kind of the information search specialty that there's a capabilities that are needed and we're tapping that in ways that allow us to make the complexity of all this research much friendlier if we follow systematic review methods. We can't function as scientists without the support of librarians who serve as faculty members themselves but also as brokers between the collections in the world and the disciplines from which researchers come and working with Carnegie Mellon's librarians has really been invaluable to our initiative in terms of systematic reviews and management. The reason for that is first of all our librarians are deeply skilled in kind of the information search process that's the general process not only do they know where the the bodies are buried where the data are in both the published literature and the what we call the gray unpublished literature which is really a hard nut to crack for non-specialists they know how to access these collections but Carnegie Mellon because we have research oriented faculty they actually know the criteria that are important for framing questions for developing search strategies they know how to test the different ways in which search strings can be constructed throughout the collection and the different databases so that we get good comprehensive and reasonably complete yields in terms of relevant findings and I also think one of things that's very important is that because they're kind of multidisciplinary most of us come from our own discipline they're able to ask questions about labeling and relevance of different ways of asking a question or using concepts that an individual faculty member might have might not be able to do on his or her own. There's long been a tendency in the review process for cherry picking cherry picking means that you know in the past authors who write the synthesis or reviews of the literature pick the findings they find most interesting or even unconsciously the most consistent with their own worldviews or preconceived beliefs. The idea of having a protocol for a systematic review is you put a lot of work into getting a right and tight question and methodology and this then says having put all that work in up front and it's vetted for quality by a rigorous team of reviewers that it's a guaranteed publication for the author team so it's worth their time and effort once they get the protocol through to actually now implement it because they don't have to worry about it's being accepted if they have conformed to what they agreed to do in the protocol. So your uncertainty your risk management is up front your uncertainty is faced earlier rather than later at the publication point which is a means of ensuring low level of bias in what's reported but also it's an incentive for the team to go forward and finish the review because the outlets guaranteed if the protocol is approved. It also third benefit is that because the protocol has been pre-approved if you've got an issue that other teams might be interested in by being pre-approved other teams can't come in and do that review not until yours is published. I think one of the fascinating things for me about doing a systematic review with our team of scholars and librarians has been the learning experience prepare to learn and to ask each other questions. I think our team was very good at wondering about this wondering about that and really listening to what people with a different background had to say about an issue having now gone through the process. I'm really confident that we came to understand some critical things about the literature the very different literatures that pertain to our question of CEO contracts and firm performance that we wouldn't have been able to answer without that critical teamwork and having a heterogeneous team made a difference. And librarians rock, they really do.