 Hey everybody, it's Brian Wilson here. I'm the co-instructor for the 2019 MIT computational law course workshop, and I'm joined by Juan Ramirez from Inserio and today we're going to be bringing you guys a the part one of integrating interoperable apps and services with a legal platform and so what we're looking to Do some key concepts and then from there get actual experience with kind of what it looks like to actually build these apps in To end flat on so with that one I'll hand it over to you. All right, perfect. Thank you, Brian We'll welcome everybody So this will be kind of a one-on-one chat or breakout session where we go over We're going to focus on relativity and we're going to talk a little bit about e-discovery and what it is So I'm hunter mirrors. I work with Inserio. This is a company that developed applications On top of that platform right for the relativity space So we're going to start off with just a basic introduction of e-discovery work into relativity And then we'll go into building Basically a no-code application within relativity, so we don't actually need the code we'll use their tools and It's you know semi-functional application since we have some time restrictions, but this is something we can view We'll say in the next course right and this afternoon There's another course and we can extrapolate and build on what we started building and start showing you things that actually do require code So to kick that off I'm gonna I have some slides here. I'll just use them as reference and we'll be kind of going back and forth So bear with me as I switch over All right, so I'm assuming most of you know what e-discovery is if you don't e-discovery is basically the handling of electronically stored information right or ESI as the industry calls it And so this didn't exist in the past This was if you look at a black-and-white movie. This was When you'd have stacks of paper And you'd hire these associates to basically go through documentation and try to find what they call the smoking gun of a case And at some point in 2006, I believe in December Amendment to a civil procedure actually gave gave birth to e-discovery right where they actually said you can submit Electronic information as evidence and so that was basically the birth of e-discovery and all these companies that handle big data You know, it started off small but people were either scanning things in now. It's grown to Basically, you know millions of documents to hundreds of millions of documents Whereas before, you know, you'd have a stack of paper and you would review that At this point the industry is around ten billion dollars and it just keeps growing within, you know, four years We will close to double. So this is something that is You know is getting a lot of innovation just due to the the financial initiative that that it has behind it So, you know, we're focusing today on relativity and what is relativity? Well, relativity is one of like the dominant the dominant, I guess providers of e-discovery, right? They they focused at first and I'll jump slides here. They focused at first We're looking at an EDRM here of just this little blue piece of the EDRM Which is I ingest data. I review it. I analyze it and then at some point it'll get produced meaning I'll hand it over to the opposing council They're starting to expand their their coverage of all of this which kind of makes them a preferred basically platform that we want to develop on And so that that's what we've been up to since 2011. We're a relatively small team We started, you know, obviously just one with me hired the next guy and for the last couple years We've been growing and evolving as a team And the same thing with their platform, right? They started with some really janky and hard to use APIs And now it's it's very modernized and it's very technology agnostic They're based on .NET for the most part, but now that they use rest services You can basically write software on anything, right? You can do it and just client scripting if you really needed to So it's a pretty it's a pretty stable platform. It's growing. They're moving into a SAS environment So their data the data sets are growing. They're into like, you know hundreds of millions of documents or even billions at this point So it's something that that is growing And they not only do review their forward thinking in the fact that we're building applications that really have nothing to do with review But they have something to do with a specific case So we've done things for different clients where you know at a point somebody needs to visualize something in a pipeline That the you know some oil company was purchasing so they need to see how that was going all the purchases of these things We've done, you know project management type applications Which in essence have to do with the case, but they don't necessarily have to do with just review So it's it's expanding a lot and their platform is really helping and So with that so you know, I'll talk about some of the success stories I have some samples, but you know they have these innovation awards The pipeline thing is one thing that was really outside the box where it wasn't you know Analysis or searches or some AI engine that search for topics But it was something that they thought out the box and they did it But the same thing with a petition application where they won like a 20 million dollar case because we were able to identify a little bit of fraud from these petition guys Which is kind of a funny business because they had paid for signature And so we were able to identify that they started the signatures early and that you know Once we visualized what counties they were on We were able to say that they weren't you know They weren't there was basically a person in two places at once And so when it came to the actual trial that just throughout the case and it was like a 20 million dollar case So it was a really big deal So the return on investment on customized some of these things is really big for some of these law firms or some of these corporations and that's basically why I exist as a company and why Relativity is doing so well just because of their coverage and in and how they're expanding their platform similar I try to compare it to like Salesforce where they're I guess you know a CRM But you can do anything with it and there's providers out there that are already worth You know multi-billion dollar companies that are writing software on top of sales for so I mean we're not worth a billion dollars But at some point hopefully we will be But for now we'll stick to To writing like you know small case applications Yeah And I think this gets at something very important that we've tried to hit on in the course and that is that you know their technology and design are Allowing us to transition more from the law being law centered where it's just focused on regulations and Statutes and how they apply to people to being more human centered where it's focused on how people themselves interact with So it's it's kind of this Paradigm shift from you know one era where things you know Happened one way to this new more exciting era where they're designed actually for the people to make their lives a little bit easier and more intuitive Yeah, absolutely, and that's kind of a trend. We're seeing is smoothing out workflows Organizing things so you know it reduces the you know human error factor And speeds up time so a lot of firms are able to either you know bid on cases that they can win at because they're a lot more efficient now Or charge a premium because they're more accurate So we're kind of tackling things like you said in the human factor not necessarily a hundred percent where everything's done Automatically and you don't know what this thing is doing for you, but it is facilitating is like their jobs Yeah, that's awesome cool All right. Well, I will share you know, I kind of want to go through a walkthrough of what relativity is I'm sure some of you have seen it and some of you have not so I kind of just want to give a 10-minute little overview of what it is So I'm sharing a basic relativity screen, which is when you log in you see this little guy and workspaces to them is Analogous to a case and so relatively we'll have clients climb my cases and cases could be subdivided into workspaces Workspaces then have documents and documents if we look at the EDRM spectrum This is after collection, right? So we've determined what documents we need what custodians We need to kind of review documents for that that are involved in whatever, you know litigation we're in right now We've processed all this so we've kind of filtered out things. Maybe it's by date So we've done some some previous filtering so we don't shove in you know hundreds of gigs Which we could and then filter it within relativity, but the idea is to kind of we Preselect the data we want in there. So right now we're after processing since we already have some documents in here And relativity is not known for their gorgeous. Yeah, I they're known for Managing large data sets and and having powerful search tools And so you'll see this is very Excel and so this this provides an opportunity for us as well To provide very you eyes depending on what data you're working with Once you're in relativity the basic function is to basically Go through documents and see if they're relevant or not And so they go through different phases they go through a first-pass review a second-pass review And what relativity offers is additional metadata And all these things that are called fields within the relativity space are basically things that as a lit support person or a project manager for the case has added as Items of information that we're either going to add in or have been identified to collect And so we'll go through some basic ones here, which is you know, is this document hot or not, right? Is this is this responsive or not responsive? And for the most part it started off in 2006 or seven as a manual process I'm going to read this email and basically yes, this is responsive Go to the next thing. This is like a logo. This is not responsive And that's how it went Nowadays they have machine learning which you start training it and it knows What themes or context you're looking at and so you know you'll train it with a couple hundred documents And it starts learning what the content of this is so they are using some a lot of machine learning here to save time Because data sets keep growing with the average data set in 2007 I believe was in the hundreds of thousands of two hundred three hundred thousand and now the average workspace has Approximately five to ten million so to hire You know the average review rate of manual review reading a document is about I would say 40 documents an hour So if you were to try to tackle a couple million your expense of actually hiring review Either associate attorneys or you know, there's actually outsourcing of reviewers So there's there's big companies out there that will outsource the review So then you have some elasticity and in your demand whether you know you need a lot now You don't need it later. So there is some outsourcing there, but regardless there are early rates aren't low And so this is where they started implementing a lot of either early case assessment or machine learning right trying to use a lot of this Compute power to figure out how we can reduce the actual manual So they'll do first pass maybe through machine learning and then once the machine identifies things They'll do some manual, you know, I guess searching at that point or review at that point And so it reduces it quite a bit right you can you can kind of get rid of more than half of the documents that you would have to do manually To machine learning and this is all you know part of the relativity training course that they go through So it's it's not an easy onboarding with relativity. There is a lot of pieces to it And they keep expanding on it. They just keep this analysis pieces where they batted some no sequel database that they started using Elastic search as their back end for really large pieces of text And so one they're able to do concept searches And they run extremely quick compared to a sequel, which is their main back end on Like relative objects and things like that and so that's kind of the gist of relativity as a document review platform and One of the things that the relativity is really good at is letting you expand its use And what I mean by that is they have this this concept Relativity dynamic objects and they you know refer to them as rdo's and what that is basically is is allowing you to Basically create a Relational database by building entities And so when we can compare it to a database, this would be the example. I can give you here is Creating a table and if we align to a database Each field that we add to to an item could be Related to like a column, right? So what I want to go through here is a basic I don't know contact list application that we can use later on to to kind of assign tasks to people and things like that But this is kind of one-on-one for them where even even at this level with zero code people have found it really useful Where they can just keep track of things whether it's when a hard drive came in through the mail and they ingested it Or just keeping track of people involved in the case whether it be a custodian So they do a good job of giving you flexibility to to create things without actually having a higher Developers so what I've done there is created a new a new object I Can create a new field which was be would you be analogous to a column in a data table? And this was contact It comes with names so but I'll I'll be first name They have different field types and so very similar to a database which is you know Is this going to be long text which is unlimited like a blob or fixed length text we can go up to 4,000 characters And then these are the other What we saw before regarding review was it was considered a single choice because it's either you know documents either hot or not In our case, we're just going to do fixed-length text for somebody's first name and we'll just create cheese clear it out for me and Then we'll just keep creating the field that we've made last day and and it's true and there's somebody who's worked with relativity through Project that I had with the ABA Center for Innovation The I can speak a little bit to the ability to kind of customize cool things and relativity workspaces They allow you to generate You know kind of anything that you want So we had a project where we were just looking at the most common features of wrongful conviction investigations Innocence projects across the Midwest and so it it was This is something that you know the only limit I think nation and if you figure out how you can kind of get your imagination to where it works and kind of this Excel sort of database model, you know, there are a lot of Really cool opportunity there are really done a good job of highlighting and like Yeah, totally agree with you in so you worked on a case with that with them So I was a when I was a fellow with the ABA Center for Innovation We kind of did this proof-of-kind and some other of their support staff to kind of Produce our network with us and can we produce a framework for reviewing documents that allows for Distributed workers where it can be done by people who have different permissions across, you know several different locations and then can we use that information to make some data-driven insights and that was Through their academic partnership program with Janice Holman, and that's actually how One and I got connected for everybody who is here on the live stream cool cool Janice ever sees this video. Thank you to Janice for making all of these connections She's gonna appreciate the shout-out So while Brian was giving us a little background I went ahead and created a field called I think Contact type. Oh, yeah, there it is And I added some choices to that so now when I go and add or modify contacts Which we'll do right now. They're Like a single choice and relativity gives you these are all what they call tabs, which is pretty Pretty self-explanatory, but then we get into things like layouts and views right and right in here What we're supposed to see is you know a list of contacts once we start creating them They give you the flexibility if you have a thousand views or a thousand fields You don't want to see the whole thing so then you can kind of focus on what you want to look at Regarding You know what is available if you have a thousand fields in a document You don't want to see the whole thing. So it's kind of like you know in Excel where you want to filter through a couple of them You don't want to see everything. So that's what they do here And so all I have the ones I added since they're pretty much all relevant to what our example is here And so now our little grid will update Automatically has filters and things like that So if I had a list of contacts that I had created it automatically allows me to filter by the possible values that are available And so these are views when you go into layouts layouts is basically the field that they Want available to users when you're editing or adding new items? And so let's go ahead edit this layout because by default. It's only giving me the name field Which is pretty useless to me So I kind of want to go first name Last name email What's the name name in there since I might be going to do it and contact type and so we had someone here it could be like And the choices I have here, I'll just say he's an employee And so he starts popping up and now that I'm in my list If I had a thousand he'd start showing up here And name is not relevant to me since I'm gonna I'm not I'm never gonna use it I could modify it to full name at this point if I really wanted to I could create an event handler that would basically Format last name comma first name and added in the full name field automatically without me having a populated So they have different pieces of the API that that allows you to basically automate all these things Right now they're based on the dot-net framework, which is you know, we use C sharp for everything Regarding and it's a DLL that you would import this thing to tie it as soon as you to hit save It would populate this full name field that we would rename it to full name But that applies to everything you can you can automate a document to say if I do this value Then populate this other value or if this value was set make sure this other value is not empty So, you know prevent it from being saved So, I mean getting back to the contact event handlers would be a way to Basically without adding an interface create rules that people can Can use to keep some data consistency And not kind of create this database where there are duplicates or In this particular case, it's a contact so you could easily Identify it is just by sorting and things like that But you could create an event hammer that said hey, there's already a Joe Smith in here Are you sure you know it's not the same person So they have coverage of the API just about everywhere in relativity What I want to do here is that in this contact doesn't really have a ton of information So I wanted to add an address so to do that I Was going to create a new object type just called like address and then I'll show you how we would associate it Basically to the contact object and allows us to have more than one Address per because we could have added, you know address city zip to that one person But you know, he's gonna have a home address. He's gonna have an office address So this is a way that relativity kind of grows Very quickly where you start associating one thing to another And it makes it really easy without being a DVA to to really create Something useful Yeah, and I think that ability to kind of connect things that in this lecture especially where we're trying to You know figure out if we have we've got one single source of data from that data hurry it in order to Also that they themselves can start coming up with new ideas about make it You know start putting these things to get the things that I was made to do Maybe they come up with a new idea for you know an app or a service that they could that they actually don't know that they need until now And so I think that Sessions like Valuable in that way. Yeah And yeah, I think the flexibility, you know creates a little bit of complexity because people need to kind of learn But it's not that difficult And people really have created very extensive applications using just this framework, right? Just you know the the relational database type of concept So in this particular case, I'm creating probably the most interesting one that we haven't done and this is single object Which is where I am going to associate it to our contact And so if this was a database term, this is where I did a join between the two tables, right? I'm doing a basically a join between the contact and The address so we'll have to go and do the same thing we did for contact Where now we did new contact we still don't see address, so we'll go back and change the layout Actually, no, it's not here I Didn't open to association, but now we have the address and we can do the same thing here So let me go ahead and modify that let's add a new one. I'll modify the layout so you can see how I can create a new address And associate someone to it so address Same in-city state And related contact. This will be the interesting little piece here So here's related contact Did we have one? Yeah Every time you're live something happens That's all right We added Joe Smith, so let's say His brother John Let's go Oh, I'll tell you why because The descriptive field is empty and we can change that a little bit. I'll show you what I did here meaning That very first name that we didn't popular it is the what they're using as kind of a description So we actually have two items here with an empty name. We'll fix that a little bit This is address Let's say that this is my home address. Why not? And so we didn't anything for state. We made a similar choices. We didn't want people typing things out We can go ahead and take like the whole repository of states. We'll just stick to like a couple here Let's say Wisconsin So we've added a new address here. We can't tell what it is since we haven't modified the actual view yet And so this is a full-fletch address now and to actually see How this relates? I think we can go back. Oh, no, I didn't add the contact but the gist of this while I'm kind of creating these adjustments is that we've basically created like a little Like a little application that you know can can host all these contacts in there and Created basically the form to add them, right? This is like a form creator whether you know you've used like a Survey monkey or any kind of like Google forms to send out for gathering information It's basically the same thing right what information do I want to gather and collect and store? And how do I want the form to look like so we've done all that in the gist of maybe what 10 minutes? Which is you know in in DBA terms it would have it would have taken a while It would have you know required me to to talk to either in my DBA And then my DBA is gonna you know ask me about primary keys and things like that He's gonna take a while to do it and whether he's internal or not We might have to pay for that and then I have to talk to a web developer to actually give me a form You know like an HTML page that I can actually gather this data And you know we're not using any of like the Google tools or other like form data-gathering tools that are out there And so we you know we did this in five ten minutes The the UI isn't the most beautiful right is very Excel looking and very table looking But this is kind of where we'll go in the in the session That's after this and see how we can take some of this data And visualize it in a way that one it's not boring And two it makes it maybe a little easier to do some data entry because now we don't have to go to two pages Of two places to to update data And so that's kind of like the next the next session We'll we'll kind of we'll show some examples of what we've done to visualize some data that's stored within relativity as RDOs or as field data That's awesome, and I think this kind of highlights, you know the the beauty of what relativity and what you know allows you to do and that is You know you you can create a playground instead of something that is very clunky and very heavy So instead of having to focus on all of those kind of mundane steps with configuring the database So if you the database administration steps, you're just able to go in and say, hey You know I know I'm going to need this these features for this object and in order to create that have that cranked out and Is it really exciting? For somebody who's spent a lot of time looking at data because you know data doesn't just happen you have to Come or it to be Ever comp that you want on top of that in order to kind of get to the ultimate outcome that you want to Yeah, you want to be yet Right no, and I think ease of show with that Success and there are tools out there that allow you to do similar things But I think people are using this within relativity to associated whether it's case management or Tracking more documents are at it's usually associated to something to do with their case and so the tools that are out there Yes, they're great. They probably have a Better interfaces and things like that to store data because that's their core focus But in this particular case if you wanted to do something like that in a separate application You would have to either have two accounts of two different places Or find a way, you know to write now a bridge between the two apps So the beauty of this is that you can do it within the system and potentially re-utilize the data that's already stored within relatively Yeah, and and to that end, you know, we talked a little bit in the session about different types of API's that you could use to connect with the data and all of those objects and then make them into something else one of the one of the comments that we had in the pigeonhole session was Could you could you just explain very quickly about, you know What API's there are so like what it what's an API? Yeah face and how? Okay, in order to In order to get to these outcomes that we've talked about Yeah, um good point. So Basically, they have all kinds of API's They all have very similar structures, right? So if you've programmed in rest or if you program in dot-net, you know, it's some sort of object That has methods has properties things like that So the most basic one is what we just looked at which I don't think they consider an API They just they consider it just a way to build applications The most basic one might be an event handler, which we talked about Which is basically a library drop or in our case a DLL drop that associates to the object And you can do all kinds of behavior with it Yeah, you'd say whenever I mark something responsive send an email to you know the partner attorney to make sure that he's getting notified When all the very responsive documents are being identified within the case Or you can use it for data consistency, right? If I for some reason have a set date of I don't know in the future. So whenever I'm modifying a document I could do some data consistency that says, you know, this data is not right You take all kinds of actions, you know, it could be a trickle effect a lot of things that we get is If if I set this to this then do this this and that right? So they kind of a lot to how they want to see it and that's that's the event hammers You kind of could move up from there and consider agents to be the next step and agents is basically a service in windows it could be Just a window service or I'm trying to give analogy in Linux Apple like a like a demon And so what these things will do is just run on an interval and you tell it all kinds of things you could say Do you give me a report of how many people are logged in? You could say You know clean clean this up because this was temporary data So agents are very useful when it's an iterative process or you want things automatically done without human interaction On top of that would be What they call their Kepler services, which we'll go into probably not today, but tomorrow, maybe we'll touch on it today But basically Kepler is a way to reach all the RDOs all the document all the back-end data You can use rest to get to it So you can query the new contact object that we said, you know, we could say you know contact number You know ID number this this and that I want to know all their information and once you have their information you can You know post it to You can use an agent to to basically send constant emails to these guys Or it's a basic way to read and write what's in the in the repository And probably a big piece of what we do is I mean we do a mix of things, right? There's always every solution has probably a bits of the API that we have to use Custom pages is probably the biggest one, which is the ability to inject HTML into relativity So you can create something that has you know really pretty reports are really intuitive interface Something we'll see later this afternoon is you know visualizing things in Google Maps or a Google Calendar or a way to drag and drop tasks So really giving it a more modern feel and ease of use To something that you know, for example the context we're gonna later on in this afternoon session will add Like tasks and so you're gonna send a task to a contact and then instead of going to a task and updating its progress You can just drag and drop it to the different states of where it can be The other piece is that they're working on right now or that they have established is their import API Which is a way to consume mass pieces of data I mean it could be a little bit of data or it could be millions And so the import API is basically a way just to get data into relativity And they've expand that to what they call rip, which is relativity integration points And we've created rip for Google Drive to one drive To Amazon storage so it's a it's an easy way for Something automatic to go and either pull for something and ingest it or the other way around to export it as well So they basically have the whole thing covered right you want to get data in you can you want to visualize it differently you can you want something automatic to happen based on a certain piece of information that you said a document you can So they really expanded to basically anything you can do and if they don't have the coverage You can go into the wild and just go direct repository They don't recommend it, but you can you can go Wild West on it and really go straight to the data tables and that gets to one of the questions that one of the students had about you know similar programs And I and what's I think the most exciting part about this is relativity has kind of turned itself into almost like an app store for anything that's related to you know the installation of pieces You can see that optimizing legal workflows in the future So with the api provided you could you could tie it to anything right it really It really is hopefully whatever you're integrating with ways that you as well so then you know we built bridges to Billing application that's that's very specific to to law firms right where they log in their hours and that's what they're Going to use to invoice their clients And so we've tied those applications into relativity and so it does it does Lend itself to to time to basically any industry standard application And that's something we'll look at tomorrow how hot to use collaboration tools So we'll write a quick application to integrate to slack for example And slack and query objects that are in there and see you know what a document's doing or or how a task is progressing So yeah, there is definitely ability to to tie into other industry standard software And I don't know if it's me brine you look frozen But I'm frozen a brand frozen. All right. I think we're waiting for brine to rejoin In the meantime, I do see more questions Regarding sequel and its benefits. Is it faster? so relativity has And hopefully you guys can hear me Relativity has implemented multiple databases, right? They they have a really solid our relational database with sequel Um But they also know that their data sets are getting larger and larger So sequel at some point just is going to be memory intensive And so what they've done is certain pieces of relativity use elastics elastic search, which is based on the machine engine And it's incredibly fast and it's very good at Tech searching And so that's that's where they get a lot of their speed where basic functions are handled in sequel And a lot of like the mass operations and big data handling is done in no sequel database Daza asked what kind of apis there are There's both right there's dotnet framework type api And there's also rest api and usually you get both coverage If you're writing a non dotnet application, you can consume it and all using all the rest Api or I mean you can even write this the client scripting or a client code Application that has zero compilation and you can basically use notepad or any text editor to write a full functional application Since there's no server code running besides the rest services So yeah, there's there's there's both