 Good morning I'm very pleased to be here and pleased to follow professor Tyler with with this What I hope will be a very interesting panel My name is Nick Mitchell. I'm the independent monitor of the Denver police and Denver sheriff departments And I have a very distinguished panel of speakers here with me to discuss recent research on two subjects police legitimacy and management accountability and the relationship between perceptions of disciplinary Fairness by police officers and their adherence to the code of silence an unwillingness to report wrongdoing by fellow officers We hope that the session will be useful to Practitioners and researchers alike as well as the students who have joined us for today's symposium In terms of format, I'll make brief introductions of the panelists And then we'll have some some brief Q&A Amongst the group up here followed by Q&A with all of you for whatever time remains at the end of our session sound good Good then without further ado Dr. Robert Warden is the director of the John F. Finn Institute for Public Safety and an associate professor of criminal justice and public policy At the University of Albany in New York He holds a PhD in political science from the University of North Carolina at Chapel Hill With specializations in public administration and public policy Dr. Warden's interest revolve around questions about the accountability and responsiveness of criminal justice institutions to the public His scholarship has appeared in a number of academic journals and his research has been funded by the NIJ the BJA The New York State Division of Criminal Justice Services and a number of local governments Dr. Sanya Kutnek is a professor at the School of Criminal Justice Michigan State University She holds a doctorate in criminology University of Delaware and a doctorate in law Harvard Her research focuses on comparative and international criminology criminal justice and law She is also the author of reclaiming justice the International Tribunal for the former Yugoslavia and local courts The Fallen Blue Knights controlling police corruption and lay participation in criminal trials She is the co-author of enhancing police integrity and co-editor of contours of police integrity Which received the American Society of Criminology's International Division honorable mention her work has appeared in numerous leading academic and law journals and Robert Peacock is a doctoral candidate in the School of Criminal Justice Michigan State University And currently manages the US DOJ's ICI TAP regional program office in Kiev, Ukraine He holds an MBA in finance from Ohio University and a BA in public policy from the University of Michigan Mr. Peacock's research focuses on law enforcement personnel performance and accountability His primary interest is in police legitimacy And he served as the primary consultant or lead proposal author for more than 26 million in federal funds for more than Two dozen international training projects. His doctoral dissertation will test the validity of the process-based police legitimacy theory And without further ado, dr. Warden All right. Thank you Nick and good morning everybody First I'm going to apologize for the Glasses down on the nose look. I know it comes off as being a little pompous I don't I don't think of myself as a pompous guy, but but I am still in Denial that I'm in need of both wife vocals My and your misfortune this morning is that Sarah McLean could not join us she regrets it I regret it and you too will regret it because You're stuck with my much less engaging personality and presentational style In a blog post about this symposium earlier this week Dan Stedgeman wrote that the and I'm quoting now Push for open data is an effort well-suited to the agency's task with formal civilian oversight of law enforcement The question of what to do with this data once it is shared with the public is one that research scholars need to answer I say we I guess I'm going to tell you about a project that we conducted in and with the Cooperation of two upstate New York police departments whose openness to research Allowed us to collect data from and or based on their records and we applaud the chiefs there for their Willingness to do that now data do not always lead to the answers that you expect and That was true in this instance You may find the answers that we arrived at to be Unwelcome or at least maybe a little unduly pessimistic, but believe we believe that they make sense and And also that they're consistent with the findings of previous research when the findings of that research are considered Pretty carefully now. I'll say right now. I'm going to trip rather lightly through the details of Of our research But we'd be happy to talk with you Either in Q&A or afterward or at any other time about about the research now Our project was primarily about incorporating into management accountability Systems periodic measures of procedural justice and it was a concern driven largely by the research That Tom has done and that he just shared with all of you We wanted to do that especially through surveys of citizens who had recent contacts with the police now That seemed important to us partly because Management guru Peter Drucker is reported to have said that what gets measured gets managed Now we doubt now that this is generally true But we learned some lessons about measuring and managing procedural justice now measuring procedural justice Calls for extraordinary efforts by police agencies since it's an outcome That is not routinely captured in police records and it takes two distinct forms Officers actions in police citizen interactions and also citizens judgments about police behavior and measures of one might not tell us much about the other And we'll get to that Managing procedural justice like other efforts to introduce change in police organizations or any Organization like universities confront obstacles and we'll describe some of the forms that they take So as I said, we're gonna go pretty quickly here. I hope you don't mind Our research was conducted in those two departments in upstate New York Syracuse and Schenectady. They're both mid-sized police departments Syracuse you can see about 450 sworn Schenectady about a third that size. They both serve urban populations They resemble many other agencies across the country Schenectady is remarkable for our purposes due to it's still fairly recent history of Misconduct problems and shall we say public image? Challenges they were the subject of a DOJ pattern or practice investigation that began in 2001 and ended only in 2013 if I remember correctly and by 2009 when the city's mayor openly Speculated on disbanding the department one could say just out on the limb there that the department suffered a crisis of Legitimacy in each of those two cities. We sampled police citizen contacts twice a month from among calls for service arrests and stops and we completed on that basis a hundred interviews per city per month and Every month we summarize the findings from those surveys at each department's Comstat meeting We interviewed patrol officers and supervisors in addition to the platoon commanders at two points in time We completed the first round of interviews about six months after we began the monthly feedback and the second at the conclusion of the surveying and the Comstat based feedback and Finally, I'll have more to say about this momentarily. We drew a sample of the incidents in Schenectady for which an interview with the citizen had been completed and We obtained from the department the in-car camera recordings on those same interactions The the police services survey as we called it included a number of items all or most of them drawn from the surveys that Tom has fielded because well, we're not stupid Tom had pretty well cornered the market on how to how to measure procedural justice and that we took our cues from that And focusing on that sampled contact and together As previous research might lead you to expect those Responses to survey items formed a single scale of subjective procedural justice now This is I hope you can make this out These are some examples of the of the feedback that we delivered to the departments at their monthly Comstat meetings most months I think all of these are drawn from Schenectady as it happens and somewhat surprisingly Assessed procedural justice was a little bit higher not much But slightly higher in Schenectady than it was in Syracuse most months. We focused on department-wide Results because of the end a hundred in a month wasn't sufficient to break it down in a reliable fashion So the slides on the left were mostly what the departments got For each of a number of items you saw them listed there there But on a quarterly basis we were able to break out the findings by platoon So the platoon commanders could see particularly how their individual platoons were doing in terms of citizens subjective experience with the officers who they manage overall The levels of procedural justice Started out high. I hope you could see that maybe back here Started out high and they remained high and I probably ought to underscore that that was true Even though neither department we took Syracuse to be fairly ordinary And Schenectady to be somewhat remarkable in a maybe a somewhat unfavorable way neither department was known as a Paragon of policing virtue and yet Assessed experiences of citizens in each of those cities was was pretty favorable overall And here we take that that one index that we formed And we break it into four equal parts of the scale and Two-thirds you can see in this chart and blue and green or happy colors and red and yellow or unhappy colors almost two-thirds of the citizens who had contact with the police said that The police were quite procedurally just about 15% a little less so and about 21% Assessed procedural justice in fairly negative terms now Based on interviews with the commanders Field supervisors and patrol officers in both departments in addition to our attendance at the monthly comm stat meetings we identified Three patterns that we think formed a management continuum with respect to managing procedural justice At one end of the spectrum were those supervisors who did really nothing to draw their subordinates attention to the importance of Procedural justice they took a hands-off approach to managing procedural justice. They were in the distinct minority among supervisors They tended to state either that they never talked to subordinates about the importance of procedural justice as an outcome or They actively spoke against the department's push to stress procedurally just policing They tended to frame the administration as having made a deliberate choice to prioritize Citizens over officers and we can sum this approach up with a quote. We heard several times in interviews I tell them officer safety is the goal not customer service Now the majority of supervisors moving a little bit to the right here on this continuum Did communicate their expectation that subordinates be mindful of the quality of their interactions with citizens? They typically did so at roll call or do or during follow-up to a particular incident The people in the middle of this continuum the supervisors there Primarily differed with respect to the frequency with which they did so and the extent to which they connected that treatment to outcomes In the middle of the continuum They they reminded officers about the importance of procedural justice only only intermittently They reminded officers about the importance with simple messages like watch your tone be polite or explain yourself But on the far right of this continuum a few supervisors took the next step to connect Positive interactions with improved outcomes for officers just as Tom alluded to cooperation from citizens compliance with police Directives these supervisors not only regularly reminded officers of the importance of Procedural justice and police citizen interactions, but they connected it to important outcomes outcomes that were important for their officers Now if we think of the Initiation of procedural justice performance measurement as an intervention or a treatment Then the first post intervention context and it's connected He would have been in late December 2011 or perhaps the first half of January of 2012 Represented by the green vertical line on this graph And this is just that that index that scale of procedural justice that summarizes responses across a number of individual survey items And that index did bump up a bit in Schenectady though an increase actually preceded the Initiation of Comstat feedback and the difference in means that you see there that blue line that represents the pre and post intervention means It's really not large relative to the month-to-month fluctuation and in Syracuse where the The feedback started just about a month later The mean index actually declined a little bit Now these pre and post intervention means could be affected by other factors So we we used regression analysis to control for factors that would be expected to affect Levels of procedural justice and in those results I'm not going to show you a table at this point But it's in the paper if you want to see it in those results We found no evidence of an intervention effect on subjective procedural justice citizens judgments about how officers had performed either in the aggregate or For each in each department or on individual latunes however It's connected either head in car cameras for a number of years before we started we formed Not only a measure of citizens judgments about procedural justice But also measures of officers behavior that were independent of citizens perceptions and for that we built on decades Research using systematic social observation of police Almost all of which has been done in the past through in-person observation But we observe police citizen Interactions in what we have characterized as armchair Observation by way of in-car camera recordings video and audio Recordings applying coding protocols to capture officers actions We had trained observers code a sample of more than 400 encounters for which survey data had already been collected And we formed from those data two measures of officers actions procedurally just and procedurally unjust in each domain of procedural justice those four domains that Tom alluded to voice Quality of treatment neutrality and trustworthy motives and then we combine those domain specific scores to form overall scales Shown here as four-part ordinal variables. So we're going to call them overt procedural justice and injustice and we found there Moderate to high levels of procedural justice and low levels very low levels in our judgment of procedural injustice Now overall with each of those measures over time We could detect no systematic changes in either procedural justice or injustice before and after the Comstant feedback commenced Pre- and post-intervention means we're not different regression analysis revealed no changes that the eye is unable to see in these line charts however When we included in our models Interaction terms that allowed pre-intervention baselines and post-intervention changes to vary across the platoons We found in platoon three in Schenectady. That's the 4 p.m. The midnight shift an improvement in procedural justice And the bar chart here represents the platoons specific levels for typical incidents in Schenectady on each of those three platoons and Coincidentally we found in our interviews with Commanders, but especially field supervisors and officers testimony to the Consistency with which the platoon commander and the supervisors on the third platoon We're on the far right of that management continuum regularly reminding officers of the importance of the quality of their interactions with citizens and Connecting that that feature of their interactions with important outcomes for the police And and we think it important that we can't be absolutely sure That they were all speaking with one voice that there was consistency in the messages that officers on the third platoon Heard though there's an alternative explanation for the improvement there And that might be that they started at a relatively low level Relative at least to the other two platoons you can see that in that blue bar there There was more room for improvement on the third platoon other things being equal than there was on either platoon one Or a platoon two and and with our data we cannot separate out those two facts We don't know which one or if both of them are absolutely essential and I promise I'm probably over my time, but I'm about done So conclusions What gets measured that measure doesn't necessarily get managed in organizations like police departments I suspect the same is true in colleges and universities administrative priorities and directives are filtered by Refracted through Officers and supervisors occupational attitudes that they need to interpret what procedural justice means and how it fits with their day-to-day Work environment and I didn't talk much about this, but I'm happy to Structures like Comstat may be only rather loosely connected to day-to-day practice We certainly found that in our departments other research that Steve Mastrovsky and James Willis have done have found that in other agencies So for the people from NYPD here this morning I can tell you the Comstat and other departments often does not work the way it does in in your department and certainly didn't It's connected to your Syracuse the same time citizens Subjective experience does not always closely reflect officers behavior that too is filtered by and refracted through Citizens prior attitudes toward the police we found some evidence and other research has found evidence that People who have favorable attitudes toward the police and that they bring to their interaction with the police Tend to interpret their experience more favorably and we certainly found evidence of that kind of Disconnect in our data Citizens whose whose assessment of procedural justice was more favorable than what our observers characterized Officers actions as being and the reverse is true The people with less favorable attitudes toward the police tend to interpret their experiences in less favorable terms So partly for that reason and here's my concluding remark and partly because procedural justice is already fairly high Moving that needle of procedural justice and public trust is it's very challenging for police. Thank you very much