 Ladies and gentlemen, a man who needs no introduction. Hi, I'm Fufas and I'm going to talk today about some research that I did over the last few months. And there's a bit of work that has to be done to set the stage. So some of you know me, this is my third actually time talking at DEF CON. Previously I've talked about Windows networking stuff and some software security stuff. Over the last few years I've been enjoying law school. And so I thought, well how can I use the fact that I have at least mediocre hacking skills to my advantage in law school so that I can loaf through classes and get better grades rather than have to learn all this stuff. And so this is the output of some of that research. So I'm going to spend a little bit of time kind of a one slide introduction to federal civil procedure. So just so we have some terms set. I'm going to talk about some tools that I made for harvesting data. And then look at a little bit of analysis and then talk about what good it might be. So in other words, what was the point of this exercise? So I'll say up front that this is by far not a work of great genius. It is, I would consider it kind of a proof of concept. And the question is can we, without knowing anything about the law and without knowing anything about how stuff works, can we start making some interesting deductions about what the Supreme Court is going to do? So that was the question I set out to try to solve. And so unfortunately, in order to get to that solution, we're going to have to do a little bit of law background. So this is it. Civil procedure in one slide. So the life cycle of a, so by the way, we're only going to talk about civil cases. Criminal cases are interesting but we're not going to talk about them because nobody invests, stocks, make stock purchases based on criminal actions. Well, they're not supposed to. So the life cycle of a federal civil case, it starts off when someone gets mad and wants to sue someone else. So that's called pleadings. Some guy will file a complaint and the other person will have to file an answer to that complaint. So in other words, you wronged me. Hell no, I didn't. And then the battle is joined. Then we begin a phase called discovery, which is designed to keep the case out of court. And you don't have to worry about all the little words there. That's mostly to keep me from losing my place. The vertical words are the interesting ones. So discovery, you'll see, you know, depositions, interrogatories, requests for admissions, stuff like that. Basically, this is, the parties are going to exchange information so that each can size up what the other's case looks like. And if you are a defendant and you can see that the plaintiff has a really strong case, you might want to settle. And then that takes it off of the federal court's docket and you don't have to worry about it. On the other hand, if you think that the thing has no merit whatsoever, you can maybe persuade the plaintiff of that. So okay, we've done discovery. Next we're going to do some motions, trying to get evidence either included or excluded. Trying to get, if we can manage it, summary judgment, which is basically, okay, yeah, everything the plaintiff says is true, but it doesn't really amount to a hill of beans. So what? Get it out of here. Dismiss the suit. Then, assuming all that fails, we'll have to have a trial where we'll present evidence and hear arguments. And then you'll get a verdict and that will include a judgment of, you know, so you get some damages or, you know, injunctive relief or declaration or something like that. And then whoever didn't like the judgment can appeal it. So when we appeal, and you can appeal actually from any final judgment by the court. So if your case is dismissed at the motion stage, you can appeal that. In fact, a lot of the cases that the Supreme Court takes come to the Supreme Court from summary judgment. All right, so now we're at the appeal stage and we no longer say plaintiff and defendant. We say petitioner and respondent. Petitioner is the person who says, hey, the court got it wrong. Respondent is the one who says, no, the court was totally right. And so you usually would, in a federal court, you've heard your trial or, you know, whatever procedure at the district court level, you're going to appeal to the circuit court level. Then whoever doesn't like what the circuit court did, they can appeal it and try to get it into the Supreme Court. And this is called a petition for certiorari. Certiorari is the writ that the Supreme Court issues saying, yeah, we're taking the case. And it takes four justices to agree. So in other words, the Supreme Court gets, you know, I don't know, a gazillion petitions a year and they're going to select, you know, 80 or so to hear. They hear only a portion of what they do and they vote on the cases. And if four justices agree that this is one that's worth hearing, they will hear it. So now we're at the Supreme Court. So the first thing that happens is the parties file their briefs. In other words, this, your honor is based on just the law and all the precedent that we can see is why I win. After the briefs are filed, they'll be read by the justices law clerks and then by the justices, presumably. And then we'll have oral argument and then the justices will deliberate and then we'll get a result which is somebody wins or somebody loses or maybe it's totally ambiguous who won or lost or maybe they just decide not to decide. Those are all kind of conceivable outcomes. And what I'm interested in is the time between oral argument and the time when a decision gets rendered. Because that's a point at which, after which nothing new is going to be added to the case. All of the arguments have been made. All of, you know, whatever, whatever is going to be considered has been entered into the record and nothing is going to change between then and the time when a decision is rendered. So, okay, we have this window of time when the case is kind of in suspended animation and it's possible that it's even been decided. In fact, the Wisconsin Supreme Court, right after they hear oral arguments, they go and sit in the room and they vote and the case is decided right there. The world doesn't hear about that for a couple months probably because somebody has to write a well-reasoned opinion and explain why this happens. But the case is decided right then and there. So, we have this time when the case might be decided, maybe it's not, maybe it is. We don't really know what the Supreme Court does. They're pretty secretive about that. But we know, the one thing we know is there will be no more input. Whatever stuff they're going to consider, it's in there. And we just have this time when we're in limbo, not knowing what happens in the case. And that's what I wanted to see if we could do something about. So, there are a couple of resources that are available to us and the best and my favorite is a website called Oye.org. Oye is what the bailiff cries out when summoning people to come and listen to the Supreme Court oral arguments. And you can get complete information about the Supreme Court docket since we've had a Supreme Court. So, from day one all the way up you can read about all of the different cases, who were the justices, who were the advocates, a little summary of them. But what's also fascinating is you can listen to oral argument there. You can go and there's a little flash applet which I'm sure is totally secure. And then you can listen to the oral argument. It actually has a little kind of comic book style. I'll show it to you on the next slide. We can look at the pictures of the justices, whoever is speaking their picture will light up and you can see the text of what they're saying. And so you can experience as though you were there at one first street. So that's one resource. The other thing is we already have several ways of predicting case outcomes. One is the scholarly way, right? We're going to study the questions that the court is being asked to answer. We're going to research the policy issues that underlie those questions. We're going to understand the ideologies of the justices who will be voting on this and we're going to make a prediction based on that. So this is like sort of mainstream theory of predicting what the court is going to do. This is what Dalia Lithwick or whoever is on NPR these days is doing. Next you have this really funny thing called the fantasy SCOTUS project, where it's sort of like fantasy baseball. It's trying to harness the collective wisdom of legal pundits around. You can enter and sort of bet on outcomes. And it turns out that fantasy SCOTUS is not a bad predictor. They do a decent job. And then there are some people who have tried to do sort of algorithmic looks at this and make other, you know, complex systems for predicting what the outcome of a case will be. And I say the hell with all that, what can we do with absolutely no knowledge whatsoever and completely unprofessional tools? So this is what the OYA.org little let's listen to an oral argument thing looks like. So you can see that Justice Sotomayor is speaking right now. And it's kind of has a little cursor that follows along. And this is by the way the suit here is Ashcroft versus Al Kid, the suit where it was announced by the court that the federal government can detain you under the material witness statute, even if it's totally a sham. So it's not a great case as far as I'm concerned. Anyway, this is what's called the expanded view, which means you can see the pictures of the people who are speaking and you can see the text. And so I thought, well, that's pretty interesting. You can see the text somehow that text is coming to my browser. And we'll have to research how that's happening. So my plan was, let's go ahead and grab all of the docket information by docket information. I mean, what's the list of cases? And who are the Justices participating in them? And when were they heard? So we're going to grab all of that. And then we're going to grab the information about the individual cases. How did each justice vote? Did they write a decision in the case? In other words, the majority opinion of the court or a dissent or a concurrence. Did they write something that expressed an opinion? But then it turns out if you watch what that flash applet is doing through a proxy, it's really just pulls down an XML transcript of the oral argument, which is nicely annotated and well suited to parsing by Pearl. And so you can grab the oral argument transcript as well. And that's kind of fun. So I built two tools for doing those, and these are on your DEF CON DVD, I believe. I submitted them anyway. I haven't actually looked at the DVD, but they should be on there in case you want to try this at home. Get docket and case brief. And these will, they're basically just information harvesting tools. They'll pull this stuff down for you. So what we're going to do with that is parse it and sort it. So what I did was for each case, I recorded the disposition and who were the advocates and sort of the facts that I could get out of there that were easy facts. And then I also, for each justice, I made a file that contained the cases that they've heard, the cases that they voted with the majority, the cases that they voted with the minority, cases where they've written an opinion, et cetera, et cetera. And so that was all good fun. And so I just said, well, screw it, let's just get all of it. You know, let's get all of the complete record for every justice who's currently on the Supreme Court. It turns out that's about 250 megs of data, which is not that much these days. And I thought, well, let's just count the questions that they ask at oral argument. So at oral argument, what happens is you stand up and you say, you know, may it please the court and you start into your argument in about 30 seconds in, the justices start peppering you with questions. And you never really get to say what you came to say. But you can see in the transcript who's asking what questions. And I thought maybe we can spot some patterns that correlate between who's asking questions and how they're going to vote. All right. Well, there's some problems. The first is what's a question? You know, maybe a justice says something like, excuse me, you're standing on my foot or, you know, I don't know what they say. You know, something like that. And I just decided for the purpose of this analysis, anytime a justice speaks to the extent that they're recorded in the transcript, you know, a line there saying who was speaking and there's some words. That's a question. Any utterance really. The second question is what is the outcome of a case? Oftentimes there will be several questions presented to the court. They'll rule for the petitioner on some and for the respondent on others. And so it's pretty ambiguous who won. That's a, that's a tricky thing to determine. And so I decided for the purposes of this sort of proof of concept analysis, I was just going to take whatever oya.org said was the winner. I declare that's the winner too. And they're probably right most of the time anyway. The third thing is some, some data normalization issues. So we saw that screenshot with Justice Sotomayor. She sometimes appears in the XML as Sonia Sotomayor, Sonia Underbar Sotomayor. Sometimes as Justice Underbar Sotomayor. Sometimes as Justice Underbar Sonia Underbar Sotomayor. And there are other tricks like that in there. So I had to do a little bit of just fudging with the data to get it so that I was really counting correctly. And then there's something horribly wrong with the year 2008 up on oya.org. I don't know what happened. They must have had like sleepy interns or something like that. But it's, the data is horribly mangled for that. And I actually couldn't get much useful out of that year. So great, we've collected a pile of data. Can we draw any inferences from it? For some justices, no. There are three justices or four justices, sorry, for whom we really can't say anything meaningful. First, Justice Thomas has not spoken at oral argument for about six years, over five years anyway. And so this makes analysis of his questions fruitless. So he does not leak any information at all. However, if you can guess how Justice Scalia will be voting, you might have a good clue about Justice Thomas. The one that you'd really wish would leak some information is Justice Kennedy. He's generally considered the swing vote on divisive issues and he exhibits no obvious patterns. So that's very frustrating. It would be great if we could have spotted something there because we could really make some headway. But he's, as near as I can tell, a completely random questioner just popping in out of nowhere with no obvious agenda and you can't tell what he's going to do. And then we have Kagan and Sotomayor who have not been on the court long enough to establish much of a record. So, you know, they might be leaking information. We don't have enough of an actuarial tale to actually say yet. But there's some hope. In other words, they deviate radically from the average of the court, at least in a short sample range. So maybe they will be doing something interesting down the road. Okay, so we did that. And we can say that for these four, not much can be said. But for five, the remaining five, they exhibit what I would call antagonistic questioning. So in other words, if you count up the questions that they ask, whatever side they're going to vote for, that's the side they're going to ask fewer questions of. And this is generally true. And it sort of makes sense, right? If someone stands up in front of you and is espousing a position that you disagree with, you might tend to kind of get into it with them. And if somebody is just saying stuff you totally agree with, you might kind of let them off the hook and not bother them so much. So here they are in order of strength of correlation. Justice Scalia is, you know, sort of pegging the meter over at, you know, if he doesn't agree with you, he's just going to be on your case at all argument. And then all the way to the, at the other end of the spectrum, Justice Roberts is, you know, raising the needle somewhat, but isn't a strong, strongly predictive questioner. So the next question is, okay, so who could use this? You know, who might this be good for? Well, the first answer is law school nerds who are trying to write papers for their Supreme Court seminar class. But the, and pundits maybe this might be, you know, I don't think this is any substitute for actually, you know, knowing stuff, but, but when you're totally on the fence about what's going on, this might be an interesting thing to supplement with. So that's one thing to, to, you know, one community who could use it. But also litigants who are in suspense after their oral arguments take place. Some of these people own companies who have been sued, and, and they might like to insulate themselves from the upcoming ruling. And if they could know or have some hint about what it is, they could take a position that might help them a little bit. Also, can we think of any cities where you could make a bet on the outcome of an upcoming important event? So you could just, you know, you could just literally wager money on it. See what the over and under is on the upcoming case, case docket. But, and this is where I'm really headed is, is actual investors. So suppose you had prior knowledge that the Walmart class action suit would not be allowed to go forward. You know, so Walmart stock is, is maybe going to take a bump rather than a hit when that's announced. Or maybe the suit to, to strike down the California law barring the direct marketing of violent video games to, to minors. If you knew what the outcome of that was going to be, you might be able to make some fairly strategic investments because these involve large publicly traded companies. And that's sort of the inspirational thing about them. So this isn't a foolproof method by a long shot. This is really only the most superficial kind of analysis that we could be doing. And everyone that I tell about this will hit me with like a bunch of other things. Hey, have you thought about doing this or that thing? And the answer is, well, I thought about it maybe a little. I didn't have time. This is very interesting. But a couple of, of points even before, you know, thinking about, you know, doing more sophisticated analysis, even at this level, the court just tends to rule in favor of petitioners. It's about a 55-45 split there. And that kind of stands to reason too. Because why would you grant certiorari to someone who's petitioning you? Why would you agree to hear the case of someone who you didn't think had a point? You know? So if you're, if the Supreme Court's going to take a case, they probably think there's at least some merit to it. The other thing that's interesting is there are very strong groupings of outcomes. The court favors certain split patterns. So in criminal cases, you see a huge number of unanimous decisions. And you still, even in civil cases, we'll see 9-0 or 8-0 a lot of the time. And then the other sort of biggie is 5-4. You see a lot of those splits nowadays. But the biggest challenge for us as people who would like to get rich off of this technique is the fact that we don't know in advance when a decision is going to be released. So we know when oral arguments are held and we know that the decision will come sometime after that. But we can't say with any kind of precision when that's going to be. And that is frustrating because if you want to buy a stock option, the load that you pay on that option is tied directly to the duration of the option. So in other words, if you could predict it in a narrow time window, you could do a lot better on your option. You'd spend a lot less on the option itself. So talking with people who are smarter than I am about finance, the guidance that I got is that this is not the sort of thing that you even if you had really good information, even if this were a pretty ironclad method of predicting outcomes, you probably wouldn't want to do something like start a hedge fund that does only this. Because the number of cases that you're and therefore the number of trades that you'd be making is pretty small. It's maybe, you know, five or ten per term. And that's not how those funds like to work. The better plan would be to get a small cadre of investors, you know, four or five of them who have some money that they're not too risk-averse about and make some small investments based on that. And you can get a fair amount of leverage with options, you know, more than just buying the stock. In fact, that's why when you see an insider trader case, the way the SEC caught those people is they purchased large blocks of options. It's never that they bought the stock itself because they can get so much more mileage out of an option vehicle. All right, so kind of starting to wrap up here, again, this is no substitute for actually knowing stuff. I don't recommend, you know, disregarding traditional methods of insight about the Supreme Court. But like I said, if you were at a point where you were just trying to make a guess and you'd sort of exhausted your other sources of knowledge, this might be an interesting supplement to it. And it's kind of fun to be able to make, you know, somewhat sophisticated judgments with absolutely no legal training whatsoever. And that's pretty much what I came to say. So like I said, this isn't a major computer security issue or anything like that, it's a footnote. But it's kind of a fun one. And the other thing that I would like to mention is, I don't know, some people in the room will know who Fufas.net is. We are hiring, if you're a software developer with hacking interest, we would love to talk to you. So drop an email. And we'd love to see about your resume. I am going to be over, at least briefly, in the Q&A area after this. But no one is actually yanking me off the stage now. Oh, or are you? OK, so I will be over in the Q&A area shortly. Thank you very much. And have a good rest of the con.