 We're hyped to believe that something is a terrible crisis for which something must be done." In his 1996 book, The Vision of the Anointed, the economist Thomas Sowell sketched out a pattern that many of the crusading movements of the 20th century have followed. First, they identify a great danger to society, followed by an urgent need for government action to avert impending catastrophe. A new book by Jonathan Hyte, the social psychologist and NYU professor, argues that the government must regulate social media because it's causing a teen mental health crisis. Hyte is, in many ways, a model researcher because of his rigor, transparency, and openness to descent. On this issue, however, he fits neatly into Sowell's framework. Those best equipped to get attention from the government and the media are the most articulate people Sowell observes, and they often reference opaque studies without explaining them. And Hyte is certainly articulate, his book is well written and filled with compelling insights, but he claims far too much certainty for his views based on research that is mostly junk. And he advocates restrictive government policies without doing the simple tests that might support or disprove their value. I believe we're now going to have a remote testimony from Professor Hyte. I have links to two collaborative Google documents where I've invited other experts to critique to say, what are we missing? Academic studies often make use of statistical techniques that are hard for the average person to decipher, which is a shame because most published research findings are false, as Stanford's John Ionitis argued in a 2005 paper. Ionitis wasn't just referencing the many scandals of fabricated data, conscious or unconscious bias and misrepresented findings. Even top researchers at elite institutions have been guilty of statistical malpractice. Peer review is worse than useless, better at enforcing conventional wisdom and discouraging skepticism than weeding out some standard or fraudulent work. Academics face strong pressure to publish flawed research. Few have the skill and drive to produce high quality publications at the rate required by university hiring and tenure review committees. Even the best researchers resort to doing some easy low quality studies. Bad studies tend to be the most newsworthy and the most policy relevant. Many of the paper's Hyte compiled contain coding errors and appropriate statistics and other issues. Most downloaded some data of little relevance, either cheap to generate like surveying your sophomore psychology students or data collected for a different purpose and analyze it with an off-the-shelf statistical approach. Hyte cites 476 studies in his book, which might seem to represent an overwhelming case, but two-thirds of them were published before 2010 or before the period that Hyte focuses on in the book. Only 22 of them have data on either heavy social media use or serious mental issues among adolescents and none have data on both. There are a few good studies cited in the book, for example, one co-authored by psychologist Gene Twenge that uses a large and carefully selected sample with a high response rate. It employs exploratory data analysis rather than cookbook statistical routines. Unfortunately for Hyte, the study undercuts his claim. The authors did find that heavy television watchers, video game players, and computer and phone users were less happy, but the similar graphs for these four ways of spending time suggest a specific activity didn't matter. The study actually suggests that spending an excessive amount of time in front of any one type of screen isn't healthy, not that there's anything uniquely dangerous about social media. An example of a bad study that Hyte cites in his book is one that paid $15 each to 1,787 self-selected internet respondents aged 19 to 32 to answer 15 minutes worth of questions online. Few were likely to have been teenage girls and there's no reason to expect any were depressed teenage girls who use social media. In fact, I couldn't find any studies in Hyte's compendium that spent substantive time interviewing any depressed teenage girls or heavy social media users. Another study that uses low quality data was based on surveys of 143 University of Pennsylvania students who participated for psychology undergraduate psychology students at an elite university are hardly a representative sample of the population. The authors seem to have made significant coding and random assignment errors. Hyte cites 17 studies he considers to be longitudinal that either find no effect or an effect in the opposite direction of his claim, and only four were true longitudinal studies, meaning that they analyzed the same group of people at different times to see how changes at one time, like increased social media use, were associated with future changes, like more depression. One of the studies in Hyte's list contradicted his claim, finding that depression occurs before social media use and not the other way around. I could go on. Practically all the studies either have major methodological errors, or they didn't say what Hyte claimed. I doubt many of the senators on the Judiciary Committee or members of their staff, to whom Hyte claimed an urgent need for government action, read the underlying papers. The fact that Hyte cited a lot of studies just makes the problem worse. Errors cascade. They don't cancel each other out. Even the best studies Hyte relies upon have the fatal flaw of not studying the subject. You can't establish the effect of heavy social media use on teenage girl depression unless you study heavy social media users and depressed teenage girls. None of the studies do this. Instead they study mostly adults, mostly average social media users, without serious psychological issues. When Hyte moves on to policy, a different kind of study is called for. If you want to ban phones in schools, study kids who went to phone-free schools versus a control group of kids who were allowed to use phones in school. Even this would tell you only average effects, but at least showing the changes positive on average is a step toward testing your hypothesis. To be fair, Hyte does have some solid policy proposals that don't suffer from this flaw. When he argues government should stop criminalizing free play, or internet companies should insist on more reliable age verification which parents are requesting, he doesn't need studies. When Hyte first presented his argument on sub-stack, I critiqued the studies he cited, and he responded by suggesting I have impossibly high standards for proof and that the consensus of hundreds of researchers studying related issues is strong enough to justify policy actions. My objection is that the researchers whose work Hyte relies on didn't bother to talk to teenagers who are heavy social media users and who are in treatment for depression. Based on poor quality studies, he wants the government to dictate to parents how to raise their children, overruling the judgments of those who are intimately acquainted with the individuals involved. Some have argued that it's obvious that social media is causing depression in teenage girls. It may be a contributing factor, but the purpose of social science research is not to confirm, but to challenge your knee-jerk assumptions because reality is so complicated. The proper scientific approach is to try to falsify that hypothesis. If you are going to recommend new laws to regulate social media, as Hyte has, you need to consider the likelihood that your intervention isn't going to yield the effect you want. There's a tendency among the anointed soul-observes to see little standing between intention and result, but the real world is a system of innumerable and reciprocal interactions all constrained within the confines of natural and human limitations. Often individual problems cannot be solved one by one without adding to other problems elsewhere. As any parent knows, children are complex human beings. Policies designed to control their behaviors don't often yield the results we expect. Even if we knew social media use caused depression, we wouldn't know the effect of policies that restrict social media use. We know alcohol leads to many catastrophic problems, but that doesn't tell us how to regulate youth drinking. If we stop adolescents from using social media, we don't know what they'll do instead, and it might be more harmful. There is no doubt that social media has significant effects, positive and negative, on the mental health of many young people. Dealing with this is one of the challenges modern parents must face. But soul's anointed class of academics seem happy to impose coercive solutions, on evidence it takes a one-size-fits-all approach, and that they wouldn't rely upon for their own troubled teenage daughters.