 Hi, I'm Robin Rowe and we're here today to talk about using OpenAI GPT-2 for drafting technical documents. Give me a second here where I switch over my screen and here we go. Okay, well, first of all, thank you for the Linux Foundation for having us here today. We're going to be talking about complete a thought AI. This is where we give artificial intelligence a sample of the kind of text output that we would like and it tries to complete what we're doing. So a little background on us. Here's some past technology projects. I've worked in AR and VR cameras, traffic control, defense. We both live in Hollywood and so we also do technology behind the movies and cartoon series. So animation software and then also set-top box. So I'm the chairman of ANSI ISO 56007. This is a future world standard for innovation idea management best practices. Here are the countries that are involved. It's most of the world. And I'm going to back up for just a second. Why am I going to back up? Because he wants to talk about AI speech recognition starting in 1996. Okay, so the reason I'm going back in history here is that a lot of people find AI very confusing. It's hard to understand how it works. And in fact, nobody really understands how AI works because it has so much hidden complexity. But I founded the research lab or defense company long ago and we used hidden Markov models, which was an early version of what we have today. And it's a little bit easier to understand. So I'm going to quickly go through this or actually get real real. Okay. Well, first, it deduces hidden states and uses path probabilities. So the hidden states in this case are rainy and sunny. We know whether our people are walking, shopping or cleaning, but we don't know what the weather is. We're trying to deduce that using HMMs. And then they use predicts the next syllable. So that can predict each syllable and then each word of what is being said. So then it outputs SR text. Yeah, that's the speech recognition text. And it's precursor to the current AI that we have now. Yeah, and that is GPT dash two to complete a thought AI. Okay, so we now kind of understand how AI works. Here's the process. So all the experts are volunteers. So it's really hard to find a time to get people together and do all this stuff. Our document has to be about 50 pages in length. And so there's a process. We break it into sections and assign to writers different areas for them to take over. A blank page means writer's block, which means you don't know what to write. So you try to get help from other people. You can solo draft sections and they become LOR. That's one of the rings. My precious. Yes, we have, you know, it's it's very difficult to draft for a blank page for the group. So typically what happens is one person ends up doing the first draft. And then we get into this thing of they don't want to change my precious. You don't want to change. You never get like that, do you? I think everyone does. All right, so firstly notes. So and then we have problems of cohesion style repetition, bias skew, all kinds of stuff because we've got dozens of people writing different sections of this document and it becomes sort of a stew. So later through many drafts, years and years later, we have a standard. Yeah, five years later. Here's how we do it with GPT2. We draw a mind map first, which is free mind. Yeah, free mind is an open source program for doing mind maps, but there are a dozen different programs to do mind maps. And I'll show you a mind map in just a minute. We create table of content from map or Libre Office. Well, we do that in Libre Office. In Libre Office. The table of contents has sections and each section has topics. Write one intro paragraph per topic as text files. We have to go get GPT2, but don't get just any GPT2. Get my fork of GPT2 because I'm maintaining it. The version that is the one that OpenAI posted, they've gone into GPT3 and seem to have forgotten about it. So they're not updating? Yeah. And put our text intro files, one per topic in a folder, and then you edit example bash script to point to our data. Which I'll show you in just a minute. And then you run bash script for GPT2 to complete each topic. Yeah, then go for coffee or go for lunch depending on how long your document is. It's pretty fast, but it just sits there and chews. It doesn't really do anything exciting to show you on screen. So here's the concept of a mind map. I'm just going to go over this in about a second here. And this thing in the center that we're looking at, that is what I call a big hero six diagram if you're familiar with the movie. This has a head and two arms and two feet. And this is just a way of thinking of a table of contents as being five basic things. Behind it is the actual mind map. And I'll show you that in just a moment. So here's Hello World. This is what we gave GPT2 just to make sure that we had it working. So I took Mary head a little lamb. It's fleeces white as snow. And we get the output of she hated snow. She hated snow. She hated. How does GPT2 do know that? Charles was reportedly so terrified during the incident that he ran to his girlfriend and began weeping. Yeah, we're not going to read the whole thing to you. But you get the idea here. It's written something very bizarre, but it's written something. It's written something. And it's written in the number of lines. And we've specified in the settings how long it'll write until you tell it to stop. So we've said how much we wanted. And here it is. Okay, so we're going to exit that. And we're going to call up our script. How are we doing in time, Gabrielle? We're at a little over seven minutes. Okay, so we're halfway here. So here's our bash script. I wrote this script. It has a number of variables at the top, which are things that we set. The main thing that we care about here is the model name. These are the standard models that come with GPT2. They're humongous. And we can also create our own models. So we've got the 124 meg model here. And these are the corpus of documents that have been ground up and made into sausage that GPT2 goes through and uses to decide what to write. Then the number of samples, I'm going to have GPT2 take 12 tries at writing something. Each try has some random to it. So I get 12 different tries instead of doing one long try. And you'll see why in just a second. This top K, don't worry about that. The length is how long, you know, I'm going to do 300 characters here. The temperature is how random the thing is. The higher the temperature, the less random it is. Here's my Python script I'm going to call. This thing is mostly in Python. And then this is the extension that I wanted to write in the name of my file. Then we come down here and this is just a bash for loop that is going through and chewing on each of those files that we put in our input folder and writing a new file to the output folder that includes that concatenates what we wrote with whatever GPT2 wants to write. Okay, so what does that look like? Now we're getting to the good stuff. So here's the human input. This was written by one of our experts, probably me in this case, but maybe not. And it says this is the people section and we're going to talk about culture. So, organizational culture is a set of shared attitudes, blah, blah, blah, blah. And then we come down here. So here's what GPT2 thinks we want to write next. So sample number one. Sample number one is culture. This word has two meanings. It refers to the culture of people and the organization, e.g., culture of the company, and is a set of cultural experiences that help guide actions and decisions by employees. Keep going? Yeah. Okay. The word can also refer to the culture of persons in an organization. Cultures are a set of shared values, attitudes, and beliefs that enable individuals to participate in and contribute to the organization. Okay, so we'll stop there. You'll notice that at the end of sample one, it says cultures have different, okay, different one. Well, I told it to cut off at this many characters, so it cut off right in the middle of a sentence. Now, in the samples below, I've got to head and finish the sentence because I can anticipate where it's going, but I just wanted to show you that this is a feature that it doesn't know to cut off at the end of a sentence, although we could probably have to do that. It's just cutting off at whatever word it is, and that's fine. So here's sample two, read a sentence about. Cultural competence is the ability to recognize, understand, and influence others. Okay, and then three? Three is it is important to understand the roles that culture plays in a business, its objectives, and its business purposes. Culture is an important aspect of organizational development. The organization is what it is. This is the one I love. How does GTT2 know this? The organization is what it is. I mean, this is like really inviting now. We're getting like kind of new agey. It is, well, I was going to read the next sentence. It is what it is to be defined by its culture, by what it is to be defined by its members. It is getting a little bit new agey there. Okay, six is the culture of an organization also informs the way members respond to their circumstances in the world. This looks like it went back. Yeah, it often does this, because remember, we're doing random. So it doesn't remember that it's already done something here, and it could hit something that is essentially the same. Right. One important aspect of cultural organization is the belief in shared values and norms. Sorry, we're checking. Yep. We're coming up on our time here. Yep. There's eight. Cultural change is made by changing values and attitudes to conform to values and attitudes that are more appropriate in a new environment. The key to understanding organizational culture is that it has a deep and pervasive, pardon me, impact on people who participate in it. The cultural component of an organizational culture is very important. Yeah, that's kind of lame. That would just kind of repeat itself. Okay. So here you see that sometimes it's better than others. Let's see, I love it. That's why we're doing 12 samples, because some of these are going to be lame, and we may just delete them. Right. For example, the value of a team culture in the workplace may be defined as the organization's attitude towards teams and the individuals who are associated with them. That sounds like double talk, but okay. Yeah, it does a lot of double talk because it doesn't actually know what it's saying. It's using probability, like we talked about on the slide, to predict what the next word should be, but it doesn't actually understand anything that's saying. And number 12 is the culture can be understood, not as a behavior of individuals, but as a collection of experiences of value. That's a collective, like the boy. I'm sorry. As a collective, I was looking ahead, I'm sorry, but as a collective experience of values, beliefs, methods, and tools. Yeah. So basically, in management, people culture is a board collected. So that's very logical from GPT-2 standpoint. Yeah, that's the board collective at 12. And so that's it. We're out of time. We hope that you've enjoyed GPT-2. Go to GitHub, to Robin Rowe, GPT-2. If you want to download and play with this, it should run on just about any operating system. It is extremely tweaky to get GPT-2 to run. Just make sure that you follow the instructions there because I've walked through everything we need to do to make it actually work. Have fun with it. Bye. Bye. Thank you.