 Welcome, everyone, to AI Power Drupal, a new era for content creation and management. My name is Murray Woodman and today I'll be taking you through some of the advancements in AI and what this means for Drupal and how we can integrate AI into our Drupal sites. Taking a quick look at the agenda for today, I'll take you through some of the main developments in AI and also then go on to have a look at some integration approaches. We'll be showing you two approaches for how you can make use of these AI services. The main part of the presentation will be around augmenting content and I'll be showing you a Drupal module we've developed which allows you to plug in different services. And finally, we'll have a look at some takeaways for site owners, some practical approaches for how you might go about this on your websites. Okay, so the world is changing. I guess that's a truism, but in the world of AI over the last 10 years, that certainly is the case. AI services really have come to the forefront of the public mind. Back in 2011, we had IBM Watson winning Jeopardy, beating the best human players using natural language processing. In 2015, AlphaGo from DeepMind was able to beat the world's best Go players. Go at that point had been considered an unsolvable problem for computers and yet AlphaGo was able to beat the best players. In 2017, AlphaGo was metamorphised into AlphaZero and AlphaZero was taught to play chess and in this case, AlphaZero was able to beat Stockfish which was at the time the world's leading computer engine. The amazing thing about this was that AlphaZero was able to do this with just 24 hours of training. It did this by learning from the best chess games, but also training against itself to iteratively improve and eventually surpass 1,000 years of human experience and learning all in just 24 hours. In 2020, we've had the release of GPT-3 from OpenAI. GPT-3 is based on a large language model where much of the world's content has been scraped and crawled, ingested and a large language model has ensued. GPT-3 is able to produce textual content that is difficult to distinguish from that of a human and at the time of its release it did cause quite a bit of disquiet or concern in the community because now suddenly we have potentially a machine-generated web, not just a human one and what we'll see later in the presentation is how we can incorporate GPT-3 into Drupal. And most recently we've had Dali-2, another OpenAI release. You guys may have seen this, it's been around for a few months and you may have seen these kinds of images flooding the internet. The incredible thing here is you can give Dali a textual description and images will be generated so if you haven't given this a try, I highly recommend it. When I first saw this in action my jaw dropped on the floor and I turned to my family and said the world has changed and really it has massive consequences for content creators and artists I believe. So there are some of the examples of what's happened over the last ten years but we've also seen a lot of these services become available on the large cloud platforms out there. All of these different services are providing many different APIs which allow you to access AI services and it's not just these, there are many more as well. So once AI was for data scientists, a data scientist would get a data set and train up a model and then test and validate that and that was an iterative process that really took a lot of know-how, time, expense and expertise. But now with these new services, AI really is for a whole range of other people. It's for designers, artists and authors and people right across the enterprise as they augment the content that they have and use AI to help them do the jobs they need to do. It's also for developers as well so we have these APIs out there. This allows us to incorporate into CMSs such as Drupal and therefore it's for website owners as well. You and me, the people that want to actually achieve things. So what does all this mean for Drupal? I'm glad you asked. Firstly, let's take a composable approach. I think we've really got to embrace all these services that are out there. We shouldn't consider them a threat. We shouldn't consider that we should be trying to replicate them inside Drupal. We should take a composable approach and look at the services that are out there and use the ones that make sense to us and are suited for our own use cases. I have two main different approaches we can take for integrating AI into Drupal. The first one is training up models that are external to the site and the second one is augmenting content. So I'll be concentrating on augmenting content today but I do want to take you through the first approach first and that is of training models. So in this particular scenario we have Drupal at the center where editors are able to manage their content and use the tools that they're familiar with. And in this example here we're using a recommendation engine which is external to Drupal and this is the Reconbi recommendation engine. One of the approaches we can take here is to use something like Search API to push that index across over into Reconbi. And then Reconbi is able to use its own algorithms and smarts to look at that content, work out item similarities and deliver better results back to the users. And these will come back as recommendations to the end users in real time. The key insight here is that we're using Drupal as the source of that information. And in a similar way we can take Chatbots as another example. Rather than storing the knowledge base external to Drupal, we can have the knowledge base inside. So editors can manage articles and pages but they can also manage the knowledge base inside Drupal. We're able to use APIs to push that content across into an external system. In this example we have QuickChat AI which is an AI powered Chatbot. And QuickChat is in turn based on GPT-3. So what happens is you push your knowledge base to QuickChat. QuickChat then pushes that across to GPT-3 and trains up that large language model. And that model is now adapted to your own website. And of course, QuickChat can then serve back conversations to your end users. And what it's doing is leveraging the large language model that's in GPT-3 as well as pulling in the content from your site. And so you're kind of getting the best of both worlds. A huge pool of knowledge plus your content specific to your site. So I just wanted to call out a couple of examples there of how we can use Drupal as a source to push data out to an external system and make use of the smarts that are there. And the end result is that content is coming back in real time and improving the user experience. But the second approach, and this is the one I'm focusing on today, is that of augmenting content. And in this world, there are just huge opportunities available to us as owners of websites. For example, we have text, CMSs manage HTML and text. And there's many different ways we're able to augment this content. Firstly, we can classify it. And secondly, we can transform it. So many of these AI services will provide keyword extraction or tagging. Entity extraction, which is a little bit more structured, as well as conducting sentiment analysis on content is a positive or negative. And there are many ways we can transform the text as well. We can summarize it and make sort of descriptions. We can generate article ideas and article outlines and even write complete essays if we want to. Some of these systems can correct grammar and also do translation. So as you can see, there are a lot of opportunities here. Over in the world of images, it's a similar story. We're able to submit images to external systems, pull back labels or tags. We can also see, are they safe for work? Do they contain nudity or violence? These kinds of metadata can come back and help improve the images in your media library. And on the transformation side of things there, we can extract text out of images and make use of that. And of course, in the Dali 2 example, we can actually convert text to images. In the video world, it's similar to images. If we consider video as just a stream of images, we can take similar approaches and use label detection and entity extraction with some of these APIs. And in the world of speech, we can take speech in the form of a sound file and convert that into a transcript. And of course, you can imagine this transcript would be really good for improving search, improving SEO, and even accessibility as well. And flipping that around, we can actually take the text on a page, maybe the body field, for example. And then we can convert that into the spoken word. And this can be then delivered as a file. So each page on your site could essentially become a little mini audio book, if you will. So yeah, lots and lots of opportunities here. The key insights, it's pretty similar to the first one really. You know, the Drupal is the place where editors are maintaining their content. The smarts are an external system. But here the content is being uplifted. And this will lead to richer and better structured content. The low-hanging fruit here, or the key use cases, as I see it, it was really extracting keywords and tags, generating summaries, maybe helping writers out who have writer's block and fixing up grammar mistakes and translating content. And on the functionality side of things, I think there's some really big advances that can be made around the media library and the media browser. You could imagine content that has been tagged up is really going to help editors find that image. How many of you struggled to find that thumbnail or that image that you wanted to use in an article? Sometimes a media browser just doesn't cut it with searching for a title. Adding these tags in is really going to uplift that experience for editors as well. And of course, having that better structure is going to improve discoverability of content. It's also going to improve similarity matches that you may be doing in solar or other external systems. So yeah, there's a lot of benefits there. Okay, turning to innovation in the Drupal space, just what have we seen here? What sort of tools and approaches are out there? If you take a look at the Drupal presentations that have happened over the last few years, you'll see a lot of interest really starting to pick up around 2017, 2018. And at Drupal cons, there have been a number of presentations being given. And most of these presentations are around the concepts of keyword extractions. And summarization. When I was looking at these, the thing that really struck me is that these are sort of specific solutions for a specific problem using specific APIs. And that is a developer has seen an opportunity and scratched an edge, developed some code and come up with a nice little solution. And what, but each of these solutions has been different. And so what I thought is, well, wouldn't it be great if we could generalize this and try to build a system to make this sort of easier and more reproducible? So let's generalize the approach of getting content into Drupal. So I'd like to introduce the augmented module. This is a module we've been working at Morft over the last sort of month or so. There's nothing quite like conference led development where you've got that deadline and you got to get that code out. I'd like to say a big thanks to Elio and Naveen who've been working on these modules. It's just been released as a dev version in the last day or so. So how does the augmented module work? Well, during my research, I came across another buzzword and that is of content ops, web ops, dev ops, content ops. Here's the next one. Content ops is improving the pipeline of producing content and making that more reproducible and easier and timely. And basically just improving the way content is delivered. And basically this concept of an augmented fits into that model. So here we have an editor doing their work in Drupal and they decide that they want to augment some content. They should be able to click a button and interact with the interface. The augmented should then be able to go out to an external API, get that data back and then update the UI for the editor. And then the editor, very importantly at the end, should be able to review that content and accept it or reject it. So this is the basic flow. Two key things here. Firstly, we can have many different augmenters so we can connect to different surfaces outside. And secondly, we can update in many different ways as well. So we might want to update in the WYSIWYG or we might want to update via a button on a web form. So yeah, we've taken the generalized approach. We've developed a plugin. We've got different UI integration points for editors. And of course we have a configurable system for site builders. So let's have a look at the augmented plugin. This is basically what developers see. It's very, very simple. It just takes a single string, trying to keep it as simple as possible. And then augmented will then execute that and essentially go off to the remote service and bring back a very simple structure. And in this case, it's a keyed array of values. And then we're able to use those values to update the UI. Let's make it concrete. So we're going to have a look at a GPT-3 completion summary augmenter. So this is using the GPT-3 service from OpenAI. And this system is driven by something called a prompt. And you see the prompt on the right-hand side. Summarize for a second grader. And then we have the content that's out of the body field. And what GPT-3 then sends back is a nice little description, which is suitable for meta tag description or something you might use on a teaser or something like that. So that's sort of an example of some data we can get back from GPT-3. In a very similar way, we can slightly flip that, change that up, and we can make a keywords prompt for GPT-3. So on the right, we have extract keywords for this text. We can submit the text. And then GPT-3 will send us back an array of values. And you can see that these can be used to tag up content. And then using Google Cloud Vision API, we can take a similar approach. This time, we're using the file URL of an image. We submit that off to Google. And Google sends us back a nice little array of tags for that image, street, snapshot, town, and night. And of course, this can be used to mark up images in your media library. So editor workflow, this is where it starts to get a little bit interesting. You'll actually see this stuff in action. So for editors, we wanted to make augmenters available in a number of different situations. The first one is around action buttons on entity forms, or essentially the node edit form. When you're editing a node, editors should be able to click a button. And also in the WYSIWYG, when an editor is in the WYSIWYG, maybe they should be able to select some text and then get some augmentations back. We do have some other items here in the roadmap, mostly based around actions that can be done on entities. So for example, bulk operations, or node saves, and these kinds of things. So we'll be able to implement those soon. But yeah, let's go on and have a look at some of this stuff in action. So summarizing content. Here I have an article about Jupyter, and I want to add a summary for it. So the editor comes in, it's a very simple little article. I click the edit, and we're now in the edit screen for this particular article. And you can see the body text there. You just scroll down a little bit and we'll be able to see the button, summarize GPT-3. So this is firing up the summarize augmenter, and the content came back and was written down into that particular field. Having a look at tagging some content. So here we have another article. This particular article doesn't have any tags defined for it. So the editor can come in, edit this particular node. We'll get there shortly. And so yeah, we're going to come down to the tagging section for this particular node. We've just got an autocomplete tags thingy down here, and we're going to generate some tags from GPT-3. So this is using the keywords one. So these keywords come back as an array in the UI like this, and you're able to select the ones that look good. And you can add them in. And once the editor is happy with that, of course, they can save that node and move on with their work. Let's have a look at the WYSIWYG. Let's say I want to write an article on Nikola Tesla and his contributions to technology. I've come up with a title, but I've just got no idea what I'm going to write about. I've got writer's block. Now, I mean, this is a little bit more sort of experimental, but basically we can type this into the WYSIWYG, we can select it, and we've got this special little augmenter's button here, and we can come down and select the Create SA Outline Augmenter. And this is using GPT-3 again, and it's going to go back. And with the power of GPT-3, we've now got four things here, and you can use those as an outline. We've also got an augmenter there where you can say, write me an essay on this thing, and you can do that as well. Sorry? That's right, yeah. I have heard university students are using GPT-3 to actually answer homework problems and things like that. Okay, and let's have a look at images. So we're moving away from GPT-3 now, and we're onto Google Cloud Vision API. We have a lovely picture there of an autumn scene, and we can come in and edit this. Now, in this case, we're going to add some image tags. This sort of connection takes a little bit longer. Google's got a little bit more thinking to do on it, but we've sent that image over to Google, and it's going to use the feature, yeah, for label detection. And here we have some nice labels coming back, brown, amber, light, and leaf. So that's all pretty helpful. So that's basically, yeah, tagging up some images using the Google Cloud Vision API. Okay, that's it for editors. So that's sort of like the highlight of the presentation, but let's not forget about site builders. Let's have a look at what it looks like for them. So because this is a plug-in system, we have an area of the site where you can come in and create your own augmenters. And the cool thing here is you're not just picking an augmenter, but you're actually configuring it. And in the case of something like GPT-3, there's a lot of stuff you can configure. So in this case, we're connecting to OpenAI. We're using the DaVinci Engine, which is the most powerful one. We're defining a prompt there. You can see that prompt can be defined in a textual way. And there's a whole bunch of parameters here to fine-tune. You know, how much content we want coming back, how variable will that content be? So that's how, you know, we're able to configure the augmenters. And of course, each augmenter's going to have its own configuration there. Let's have a look at generating the buttons on the NodeEdit form. In this case, we've got a special augmenter field widget that we've made, and we can embed this into the form display. So here we've created one of these guys. And you can see we, for this body field, which is the source, we want to target the social message field. We want to use the summarize augmenter. And we want to give the button this particular name. So this is how we wire it together. It's like, for this source, let's go to this target and use this augmenter, put the button on the screen. All the magic happens. And then down in the WYSIWYG, we've also integrated that in as well. So here we have the rich text editor. And the site builder can come into that particular rich text editor, configure that one up. And we've got a nice little vertical tab there where you can pick the augmenters that you want. So for this case, we've basically put in all the augmenters, except for the image one. So this is a really nice way to uplift the experience when the editor's in the heat of the moment and writing their content. Now that's sort of the demo part of the presentation done and dusted. Time, unfortunately, is catching up with me and I have not able to go into too much more detail. But I really would be interested in talking about this stuff with you guys some more if you're interested. So tomorrow I'll put together an AI and augmenter BOF. So if anyone's interested in talking about AI or some of the things you've seen here, please feel free to come along. Now BOF stands for Birds of a Feather, and that's just where you can basically sit around and talk about things. And downstairs at the cafe and pool bar called the Charles, we will have a little get together tomorrow if you're interested. So that's a special part of the conference that's been put aside for doing BOFs. I hope to see you there, and there's lots of cool things to talk about. So in conclusion, all right, so we have seen a number of, you know, explosion really of powerful tools that are out there. You know, we have to embrace them. We have to take a composable approach, look at the services that exist and bring them into our sites. I've shown you two ways that you can integrate AI in. The first one is pushing content to an external service, and the second one is basically pulling content into augment the content that is within Drupal. Now, for website owners, what does this mean for us? Well, firstly, I would say you should be reviewing your user's needs. Take a user-centered approach and work out which content, you know, they may be interested in that they might be missing. But also look at your editor's needs. How can we improve the content production workflow for them? And once you've identified those gaps, let's try to fill them. And you can do that by then taking an audit of all the external services and the APIs that are out there. I've only shown you a couple today, but there are many, many more that you can plug in. And if it suits, you can use a module such as Augmentor to be able to integrate those in. The projects I've shown you today here are QuickChat, Recombi and Augmentor. These are all open-source modules developed at Morft, which we've released on Drupal.org. And the API is there. We've got the GPT-3 one. If you haven't tried it out, I recommend you do. Just type in GPT-3 examples. You'll get taken to a little playground. It's very, very fun. And your mind will be blown. And also Google Cloud API, well, Vision, Google Cloud Vision API is the other one we were using for the images. Okay, so that's it. I've really enjoyed putting this conference, this presentation together, and I hope you guys enjoy the rest of the conference. Thank you, and I've got time for questions. Jenny, yep. I'm curious about how accurate you find the results from the difficulty. Like if it's gone through a passage of, you know what the text on our website is like, so, you know, horrible big chunk of information. How good is it at summarizing that? That's a bit that just got me really excited. Yes, I mean, that's hard to say. I mean, I would recommend you go into the playground and paste an article in and see what you can get out, right? That would be a fun thing to do. GPT-3 has been trained on, like, basically the whole web and, you know, academic articles and newspapers and all of GitHub and all that. So it's consumed a lot of stuff. It knows a lot of things. But it stopped consuming that stuff a few years ago now. So it's sort of like a general sort of knowledge base and might not be so well suited for a specific use case if you, you know, but, you know, I would think it would probably have a reasonable, it would give it a reasonable shot of extracting, extracting, you know, knowledge out. So, you know, proofs in the pudding and I would say, yeah, give that a try. Yeah, I don't know about that. I mean, so we've got the summarized for a second grader, which is a weird one, right? And I have seen, so there's this, you know, this new job that's out there called, you know, prompt engineer, right? It's like, how can you get the best from these different models? And certainly in the world of Dali 2 and GPT-3, where you are submitting text, it's really down to the skill of the person crafting that text up. So there are some, yeah, you know, summarized for a second grader and there are, I have seen ones that say, rewrite this text. So you might be able, there is an example, like, rewrite this in the first person or rewrite it in the third person. And you could say, you know, I'm a really sort of funny crafty individual, rewrite it like I'm talking to someone at a party and then give it the text. That's how crazy it is. So who knows, maybe you could simplify it for plain language and that's just the power of, I'm sorry, a bit of a fanboy at GPT-3, but it's just amazing what you can do. So I would say give that a try and see what you come up with. So regarding creating an article, I don't know if it's all quite wrong. My understanding is that it doesn't have any understanding of what's factual and what isn't true. So if you'd like to do that, and Harvard calls the data space a bunch of plausible sounding lines, is that something that you can't down to be telling in your early years that functionality or... Well, I mean, just we're talking about GPT-3 specifically, it's consumed like all the Wikipedia and all the web, so you can ask questions and it will answer those. And some of the examples I'm seeing, it's like I have been influenced by this philosopher, this philosopher, this officer. What is the meaning of life? And it gives an amazing answer. So I would not underestimate what it can do. But if you deliver it lies, what is it going to do? So OpenAI is very particular about the use cases for it and if you start doing stuff that's a little bit off-color or weird, it will say, hey, this is not really an appropriate use. So there may be some boundaries there that you may not be able to go into. I meant more like technical specific fields like you'd have it do like a regulation and it might get a regulation wrong. Yeah. So like I'm writing an article on like about 66 of the civil aid issue and the regulation. It might say that the written body at day A has to do traveling. That's a good thing. I haven't gone into it here but one of the things you can do is you can train the GPT-3 model. So with a quick chat example we were taking the knowledge base and that gets sent over as a file and essentially sits on top of the knowledge base and when you're querying GPT-3 has then got all that information. So you could potentially feed and we haven't implemented this but you could easily implement a system that took content in Drupal, put it across into GPT-3, train that model and then you're querying against that updated model. So you could then potentially have that real information in there and then you could summarize that and you would actually be getting domain specific information coming out of that. Yeah. So the API tool for video where you run a video through and with index part throughout the video at this minute it is talking about this topic and at that minute something else. Yeah. So I mean Google Cloud Vision, I've got a video version of that. I haven't really looked into that but they will do entity extraction. So they might say a bicycle appears at one minute 20. I'm just guessing I'd have a time stamped. I haven't looked into that but maybe have a look at the Google whatever it is, you know, video one and just see what you can get out of that. But I know that they can do entity extraction from video and as well I don't want to keep talking about Google and OpenAI there's a number of other services out there where you could probably do similar things. Those examples that you gave we had a text but then you wouldn't retrieve text. Yeah. Let's see it quite a quick compared to the huge one. Yeah. Would I cash through or would I go in here? No, that was real time from GPT-3. Yeah, that wasn't fake. I think like Google has got to get the image and process it and it's probably the algorithms for decomposing that are going to be, I guess harder to do. Yeah, but I was that's right. Yeah, that was for real. That was for real. Yeah. Yeah. Yeah, yes, that's right. I don't really know. If you go back to Drupal 7 days there was just a huge explosion of all these integrations that were around a lot of tagging and descriptions. That was the first generation and somewhere between 7 and 8 those modules never came across, right? And so it's like with folksonomy and the development of tagging and all that concept that was really taking off around 2000. That was hot back then and it just seemed it never really crossed over into Drupal 8 and if you look at those old modules they don't really seem to be flourishing in this new Drupal 9 world so that's why we sort of had a look at doing this but when I reviewed the landscape there just wasn't really much that was sort of going on there. There were calls on the projects that were really trying to use that to keep problems that didn't provide a lot of tagging that would get text, language and it would try and give you tags that you could use and so the popular that is that it would give you too many tags and they were they were able to sort of correctly identify the subject that you were talking about. Right. So it started getting tags back that were not really about the thing that you talked about and it's often like regularly using the common word that's where the main problem is. Oh yes. If there's something you see these new calls and AI being able to address the context. That's really interesting I mean obviously GPT-3 is just sending back the strings and then the version of the images that were coming back they were just giving you strings as well but there are other APIs that will give you well firstly I will say on the image one you can set a minimum score so Google will give it a score like make it over 0.7 so it will sort of give you a certainty kind of value so you can improve make sure there's a threshold that it gets to and in the case of the entity extraction ones which are much more structured Google will go through and say this is a company this is a place this is a whatever so they're not just pulling out strings actually pulling out entities with an identity maybe in Wikipedia or whatever so there are yeah these tools out there are much more structured in some ways so I guess you can take your pitch from those different services I think there'll be a presentation tomorrow on that Google natural language processing API as well which is doing that entity extraction stuff.