 Hi everyone. My name is Aishan and I'm an RPA engineer at AI Singapore working on TagUI. TagUI is a command line RPA tool that's easy to use. TagUI automates processes for you and each automated workflow is called a flow. These flows are just text files with instructions, so they're easy to write, open, understand and to use. Now let's look at a few flows to watch TagUI in action. You can see here that I have three flows on my desktop. These I have saved and deployed as double clickable flows. It's really really easy to do this and we'll look at how to do it later. For now, let's watch what the flows do. The first one, update temperature, is related to our current COVID-19 situation. Our organization, NUS, requires us to lock our temperature twice daily in two separate portals. One of these is behind a login wall. So it can be a bit cumbersome to do this multiple times a day and visit two separate links. So we wrote a simple flow to update both sides or inputting of temperature just once. Let's look at how it works. So double-clicking the flow runs TagUI, which opens Chrome automatically. Of course Chrome needs to be installed. Here it asks us for our temperature in Celsius. I'll just state 36.4 and press enter. It comes to our login page in NUS. Here, I've already filled in my details and it will key in the details based on what I've filled in. When the flow goes to the login page, it will check whether I'm already logged in. If I'm already logged in, it will skip the login steps. Of course, it's clever enough to do that. Now it opens another session of Chrome so that I can go to Google Sheets. And from here, I will go and search for my correct row and update the correct column. And that's it. We've omitted both sides with a simple flow in TagUI. Let's look at the next flow, ForexGmail.com. When I double-click this, basically I'm going to go to a page on DBS website to get a list of exchange rates for that day. This is a very common situation in a lot of manufacturing companies where they record down the exchange rates for the day to use in their daily costing. So I will go to that page, copy down all of the exchange rates, save it into a CSV, and email it to an email address. Let's look at it in action. Again, it opens Chrome. Note that not all flows need to use Chrome and you can specify it not to open Chrome as well. So we have gone to this black page and then we can see in the background that the TagUI is reading each row and writing it into a CSV. It counts the number of rows in the table, loops through each row, and writes it as a row in a CSV. Now it opens Chrome. Again, visiting Gmail and it's going to send an email based on the steps that we have programmed and that's done. Now we can look at the result of the flow. There's numbers.csv and here are all exchange rates, all saved automatically. Of course, these flows can be scheduled to run daily at a certain time. It's really easy and we'll look at how to do this later. The last flow here is letter flow. What it's going to do is open a Microsoft Word document and this Microsoft Word document is basically a template with a few, you could say, fields that need to be filled in based on certain information. It will update those fields based on the information and print out the Word document. Let's look at it in action. This is going to be quite interesting because it doesn't use Chrome for once and it also uses OCR on the page. Here it's going to look for address of donor using OCR. Then it's going to move the mouse to address of donor, double click it and replace it with the correct text. Again, it uses OCR to find the donor name, the text donor name. I'm just literally typing click, double click donor name using OCR and then it's doing it for me. Of course, I'm just printing it to one note here. That's what we have for TechUI samples. We've just witnessed how powerful TechUI can be and these are just a few simple flows to demonstrate the capabilities of TechUI. Now let's look at the actual flow text files behind these flows and see how simple it is. The first one, update temp.tag. We can open this with any kind of text editor. I'm just going to be using VS Code, Visual Studio Code, because that's what I normally use anyway. This here is the entire flow. It's pretty compact and you can see that a lot of the syntax is really simple. First thing, we can look at each row. In this flow and in general in TechUI, flows are divided into steps. Each line is generally a step. The name of the step is the first word. Here the word is ask. So it's the ask step and after that we have a space and this is what we want to ask. This here, we're just going to prompt the user for text input. That's where I entered 36.4. And then now, after it prompts for text input, that input will be saved as ask underscore result. This second line is just to say we're going to create a new variable which is called user temp and we're going to assign the value of ask result just to make it, just to give it a better name, user temperature. Now we're just going to visit the URL of the NUS portal so that we can do our daily declaration. Here the step involves just pasting the entire URL. TechUI is smart enough to detect that this is a URL and I should visit it. Now here is how we write our if statement. If exists username. Here is where we're checking that there is an element that looks like username in the current web page. If there is, that means, hey, we are at the login page. We need to log in. If we don't see this page, it means we're already logged in from a previous session. Here we can see type username as what? So type username is the name of the element in the web page and this is what you want to type it as. And this username here is saved locally on my computer. It's a variable that's saved locally and this is useful because this allows us to share this flow very easily with others without having to share our username and password of course. These variables are stored in a file called techuilocal.csv. You can see it here. Techuilocal.csv. Get updated with your local variables. Here again we type the temperature. This is the name of the field element input element on the web page and you can see how simple it is really. Here the if clause, all these two steps, they are within the block of the if clause. So they will only happen if we are blocked by the login page. We have the keyboard step. The keyboard step lets us interact directly with the keyboard. Here we are holding windows key and pressing R. This lets us easily run any other applications. For example here we do windows R type chrome press enter. It's an easy way to open applications in windows although techuilocal also works on Mac and Linux. It's cross-platform. Here we can see that a lot of this automation is simply keyboard interactions using shortcuts. We fill in the URL address with control L allowing us to enter the URL in the web page. And then we after we get to the web page we control F to look for our name and then press right a number of times so that we can enter our temperature in the correct column. And that's it. Let's look at the next flow. It's going to be 4x gmail. Okay here this one is a little bit longer but no more complicated. Again here from the start we are going to visit the URL of the dbs exchange rates page. This dump step allows us to create a new file and it's going to write this into the file. It's going to write currency comma rate into a file called numbers.csv. This just creates the column headers for a csv file. As you know a csv file is a comma separated values file so the comma here denotes that currency is the first column. Rate is the second column. Next we are going to count the number of rows in this page. In this page and this is the number of rows in the foreign exchange rate table. And inside this count function we have an x path. Some of you probably be familiar with this already. This is a very common and expressive and powerful way to identify web elements on a web page. So here we have an x path expression which is the number of rows. You can see tr if any of you are familiar with html. Tr is the table rows and we are counting the table rows and we will assign that to the variable rows. So now we are going to loop through the rows and we are going to loop through this number of times based on the row count and then each time we are going to call it the variable row. Now we have the read step. This basically looks at the web page reads an element's text or attribute to a variable. So here we are going to look for this x path and we are going to get the value and store it into the variable called currency. The second value we are going to store it into the variable called rate and now we are simply going to assign these values and put them into an array, into a list and call it forex rate. Next we are going to simply write this whole forex rate as a csv row into the numbers.csv that we created earlier. Okay numbers.csv is the one we created earlier. Right just appends this text as a new row in the previous file. Then we just go into to open up another instance of Chrome. Why we open it another instance of Chrome here is because Google blocks some services when you are using an automated browser. So for example Gmail, sometimes you won't be able to access it with an automated browser. This was a recent change by Gmail and we had to adapt to it. So now we open Gmail in a separate browser. Then we click on compose.png. What was compose.png? I go back here. Compose.png is a file, it's an image of screenshot of what I want tag UI to click on. It will use image recognition on the current screen. So this is visual automation. It's going to look for anything that looks very high percentage similar to this and it's going to go to the middle of it and click it to compose a new email. We click it, we wait for a second, we type an email, we type the email subject, we click attach and then we attach our file. We attach our file and we can send it using keyboard shortcuts. Pretty simple. Let's go to the last flow. That's going to be letter flow. Here we're going to use OCR as mentioned earlier. We open the template document again and this is the path to the template document. We're going to put it onto the clipboard. Then we paste it in our run command and press enter as we see here. Next we have the wait command. We wait for a few seconds and here we have a double click. We will double click the address of donor using OCR. So how this works is tag UI will look at the current screen. It will scan the entire screen for text. Turn this entire screen image into text. It detects text that matches address of donor and it goes to the location and double clicks it. Then after that we will just replace it with 123 automation lane. It does the same for donor name and donor amount. Here we can do the rest with keyboard shortcuts. Really really really simple but you can see how powerful the OCR feature of clicking and finding locations on your screen just based on text can be how powerful it can be. Next we look at how to install tag UI for yourself so you can get started with tag UI. Here's the documentation for tag UI. It's on readthedocs.io and you just simply go to installation on the left hand side and it gives you installation instructions for windows or mac or linux. Since I'm on windows you can just click here. It gives you the direct link to the zip file. Next we want to unzip this file to C drive directly onto C drive. We have to also install our dependencies. There are two dependencies here for windows. The first one is OpenJDK and if you click here it is packaged for you. It is for using a flavor of OpenJDK called Amazon Coretto. It is completely free. The next thing we have to install is Chrome. Most probably you already have this installed so that should be fine. So then once we're done with that we just need to open our command prompt and since this is made simple for everyone we also have instructions on how to do this but most likely you know how to do this already. So we can open our command prompt and then we can add tag UI source to our path with this and then we can run our first flow. So this is our first flow and we can run it. If it starts properly without error that means you have installed tag UI properly. So here the first sample flow goes to latest movies. It waits a bit and snaps the screenshot of the image of the results. It's just a really really simple flow. If you have any problems with that you can go to having problems and then check whether these problems are applicable to yourself. If not you can also come to our GitHub page and then post an issue on our GitHub issues page and that's it for the installation. Really really simple. Again here the Amazon Coretto is an installer that you can just click through and UI windows is just a zip file and we can extract this directly onto C drive directly onto C drive. It will be just like that. Now that we have set up tag UI we have seen how easy it is to do that and now we're going to write our first flow. We go to file explorer and we navigate to the C drive tag UI flows folder. This is where we recommend you write all of your flows. You can separate them into different folders. So let me have my first flow and then I'm going to create a new file. I'm going to have a text document. I'm going to call it first flow.tag. I'll change the extension to .tag because although all our flow files are text files we require .tag extension at the end. Now I'm just going to open this with visual studio code, ds code although you can use whatever text editor you like even notepad. Okay now we have opened our flow. We need to think about what we want to do with this. So how about for a simple start we can go to a web page and click around the web page just like that. We'll look at how we can do that. So in fact why not we just come to the documentation page and this is our documentation URL. I'll copy it and I'm going to paste it here. So that means tag UI will come to this page and then and from here we want tag UI to click on main concepts just like that as a simple first flow. So how do we get tag UI to click on main concepts? As I mentioned earlier on tag UI can read web elements on a web page using the DOM using XPath but it also has more intuitive and simple systems to try to identify web elements. So here if we just do click main concepts it should work and I'll talk about why later. Concepts and then we'll do wait five so that we can see to make sure that it works and now let's run this flow. When we do click main concepts tag UI will look for the entire look through the entire DOM and look for any elements where it has a name that looks like main concepts or a class that looks like main concepts or id that looks like main concepts or even a text that looks like main concepts and it will try to click on that. So here let's run it. So how do we run this flow? Let's go to our command prompt go cmd I believe we have we have it in c drive uh tag UI slash flows slash my first flow and I'll just do tag UI first flow dot tag and now it opens Chrome as we expected it goes to the web page and it will click on main concepts. So tag UI was able to look for main concepts identify it and click it. Of course this flow is not very useful but it's just the simplest flow that we can do that demonstrates it's clicking and using the web browser. Now how do we deploy this flow as the shortcut so that we can simply double click it to run to do that we just do tag UI first flow dot tag with a dash D this is the deploy option we can also do dash deploy and we do that we generated a first flow dot cmd and we can move this first flow dot cmd to our desktop and now we can double click this to run our flow simple as that and you can see that it works all right and it's going to finish in a few seconds. Now this is probably a good time for us to look through the documentation of tag UI tag UI version 6 just released and with it we have a brand new documentation on read the docs in on top of the installation instructions we also go through the main concepts of tag UI advanced concepts and a reference of all the steps run options and other helper functions that we can use okay so let's look at what some of these look like so we go through the flows we also have the gifs showing the demonstrations of each feature and then here we have showing you how to use the deploy file the deploy option and as you can see here if you move your flow to another folder then you need to create a new shortcut file to we just do the deploy option again you also run flows from online from a url online like just here you can run the flow from the sample flow from online you can run the tag UI without the browser with headless mode as using a browser back in invisible browser you can do that in headless mode here we have the steps and it goes through how to use all the steps and the most common steps like click visit type we only went through some of the features and steps early on so here we can see more in the documentation here we can see the examples click getting started yep so again we are looking for getting started as the id name class title or even getting started in the text we also we can also provide the full x path which is expressive way and a very certain way to click on certain elements we can also click on coordinates of the screen although this is a fragile method of clicking on elements because sometimes elements can move around based on text boxes text text being longer or shorter or websites just changing their interface but this is also a possibility to click on points based on coordinates we can also click based on images like what we saw in the demonstration earlier here we have visit the webpage type this is very common we want to type something into into web input so this is the name of the input and we will type some text into it continue on we can see the different types of identifiers so these are x path this DOM click on points here we also have region identifiers identifiers just a way for us to identify what we want the step to work on so here we can do read and this is a region 300 400 to 500 550 that means we're going to look at the region between points 300 400 as x y coordinates on the screen to another point which is with 500 550 and within the rectangle bounded by these points we're going to OCR all the text there and save that text to a variable so this can be very useful to look to read documents which are scanned in so we had some companies or so they scanned in some documents and they want to match invoices or other reports make sure they match up so they also use this OCR reading okay we also have live mode which is an excellent way to test out and develop your own flows basically it is an interpreter for tech UI so you can just type in steps and then it will execute each step line by line and it will yeah it will execute all the steps line by line so you can test what works and what doesn't work if statements blah blah blah you have helper functions these are basically java script functions that we can use in our tech UI flows uh tech UI is still under active development so we're constantly trying to improve the API and make it easier and easier for users to use tech UI to write their flows some other things in uh that's possible with tech UI include using data tables so here data tables they are basically CSV files it allows us to run the flow multiple times we can have a CSV file with a thousand rows and it will ruin it will run our flow 1000 times one time for each row in the CSV file that we specify so we'll just run it with tech UI with my flow dot tech and then the name of the data table this is the data table name trade data dot CSV yeah and then it will run it with these variable values the variable username will be a test account password will be blah blah blah and so on so forth you can also run other flows within flows and a lot more so if you when you want to learn more about tech UI and you want to figure out what you can do and how to do a certain thing in tech UI you want to go to the reference this is where you can list out all we can see all the steps to interact with the mouse and keyboard these are the things you can do to interact with the web these are the things you can do so on and so forth you want to save files these are the different steps that we can do one interesting thing about tech UI also is that you can it's very powerful in that you can integrate python javascript and this visual automation library called sikuli in the tech UI flow itself so how do we do that let's look at some examples so here if you want to run some python code we can just do py space and some python statement so here we py and then this is just python code result equals 2 plus 3 and then we we can transfer this result back to our tech UI name space our tech UI common runtime by printing it to the standard std out standard out and then it when we print it to standard out in python it will be saved as py result back in the tech UI runtime so that's basically it so why not we try to look at a more involved flow a more complex flow and write it with tech UI now that we have the documentation with us so now let's use the documentation to create a second more complex flow here we have fave.com it's a website that has deals for group purchases so the deals can be quite attractive so let's say i'm really interested in some of these deals but only a sudden subset of them like i only care about the ones which are one for one because i want to go there with my girlfriend or whatever so let's get a flow that comes to this website looks for deals which are one for one and then saves them for me into a list right and it will run it every day okay so how we can do that is let's create first a new flow um we'll create our folder second flow and we'll create a new flow which i will call second.thj okay and again i'm going to open this in any uh text editor i'm just using vscode here and here first thing we'll do is the url to visit here i copy this url and i will paste it here next uh what i want to do is to write a list of the deals so the easiest way to do this is to create a csv file and if you saw from the earlier flow we write a csv file here right you know a forex exchange demonstration so you can just copy it and use the same framework the same idea so we want to dump deal and the link that sounds about right to deals.csv um we're going to count the elements uh which have the deals so it would be each of these cards one card here one card here one card here we're going to count all these so i'm going to right click and inspect so it looks like these elements are in the div class column um and a ui card segment okay so i'm going to use xpath to uh look through to identify these elements and count them so i'm going to double click to copy the class uh these should be div elements so here we're looking for all div elements which have the class equals to this and that should be good and we're going to call these cards for each card for a card from one to cards we're going to read so now we're going to look this allows us to look through um all of these cards and for each card we're going to say we want to read this card uh and so here card will look become from go from one two three four five six seven up to however many cards there are so out of all of these card elements i want to get the first one then the second one the third one the fourth one the fifth one these back takes will replace this card with either one two three four or five or whatever uh number the loop is in uh okay read from read this and we read the card but we don't just want to read the card we want to read the card and we want to go into the we want to get this right which is the name of this uh this uh this offer so which is under a class content so we're going to go a class equals to content slash h3 yeah that looks good and this will be the deal but i don't just want to grab the deal i want to grab the deal only if the deal contains a one for one so i'll do an if statement if deal contains one for one did i spell that right yeah so one for one always looks like this so i'll do a one dash four dash one so if it contains one for one then i will say i want to get the link as well so i'm going to read something to link href yeah i'm going to get this link which allows me to go to the actual deal after i've listed down all the interesting deals so this is the actual link but it's not a complete link it's a partial link which is slash Singapore so we need to add on this my fave.com at the front so i'll do a copy i'll copy from here and i'll make a note that i need to add on this my fave.com okay um so from here now we're gonna do we're gonna copy this this part here let's say deal roll equals to deal and then we're gonna add on my fave.com in front of the link and now we have our deal roll and we can write this deal roll to our deals.csv i think that looks good so again what what's happening here is we go to this web page we create this deal.csv with the column headings deal and link then we count the number of cards using xpath you can learn more about xpath on w3 schools xpath w3 schools xpath yeah this is a really good link to learn xpath and then from here we're gonna check we're gonna assign this value to the deal and if the deal contains one for one we will write the roll using the deal and the link we're gonna write the roll using the deal and the link all right so let's do this now let's run our new flow we're going to do tag ui second dot tag and check whether this works can't find forex rate clearly because we shouldn't have a forex rate it was from earlier on so this is now a deal roll so we will try again block these and we can look at the we can look here yeah it seems to be looping through the elements and okay it looks like it's finished at 16 seconds uh okay it's created our csv file and we can check here oh great so we have our deals these are the deal names they all contain this one for one which is excellent and we have our link here so we can follow these and look for the deals of our dreams right one for one choose megato i think i'll have this with my girlfriend tonight i'm just kidding okay so that's it we again we can deploy these uh this flow easily uh just using the d option which is the deploy option now we have this deploy option we can bring it to our desktop and we can also run this using a window scheduler scheduler task scheduler with task scheduler we can tell task scheduler to run this task every day at a certain time it's pretty simple i'll just go through the basics we create a basic task uh check for deals daily yes occur every one days yes start a program yes and then i'll just say please open second start and then that's it now it's going to start at 4 22 p.m every day before i go home for the day and check for any nice one for one deals that i can enjoy on that day and that's it we've learned how to use tag ui to write flows to deploy flows we've seen how easy it is to read the flows and to learn from the documentation if you're interested in helping out tag ui we're always welcoming contributors head over to our github page and you can suggest issues and make pull requests we're currently undergoing a large migration to remove the open jdk and php dependencies and everything and run everything on node j s thanks for coming and enjoy your day ahead