 Welcome to my GoLang for DevOps and Cloud Engineers course. What is Go or GoLang? From the official Go homepage, go.dev, build fast, reliable and efficient software at scale. Go is an open source programming language supported by Google. It's easy to learn and to get started with. Build in concurrency and the robust standard library. Developing an ecosystem of partners, communities and tools. Who am I? My name is Edward Viana. I am a DevOps and Cloud Specialist and Training Instructor. I started publishing on Udemy in 2015 and now have more than 250,000 students enrolled in one of my DevOps and Cloud courses. Since 2017, I have been using Go extensively as it became more popular in the DevOps and Cloud space. After years of writing Go code, I feel comfortable now to create this course and teach you all the Go tips and tricks I discovered over the years. What are the course objectives? To be able to read, understand and write Go code. To be able to write enterprise-ready applications. To be able to write applications that integrate with Rust APIs. And to be able to write applications that integrate with a cloud provider. To be able to write applications that integrate with Kubernetes. And to be able to write applications that integrate with any custom integration that has a Go SDK available. So this last one is a natural follow-up. Once you've seen enough examples, you should be able to write Go applications using any SDK available. Because you should know how to do it or how to figure out. This is the course layout. We will start by understanding Go by example. I will show you the HTTP get application. That's where we get started. It's our first application. Then we will continue with Go concepts. I will explain you the main concepts using examples. And then we're going to start integrating with SDKs, cloud providers, Kubernetes by learning how to use external packages. So there are a lot of external packages available to integrate with external providers, like cloud providers. And I have some great examples to show you how to do that. I will write all the Go programs myself in the lectures and the demos that are coming up. And lastly, a link to all course files can be found in the next lecture. It's called the source files and useful information. Save that content somewhere and make sure that you know where my GitHub repository is. Because that's where all the Go source files are located that I will show you in my lectures and demos. Before we really can get started, you need to be able to edit Go files. If you already have an editor, great. You can use that one or you can use Visual Studio Code, which I'll be using. I will show you how to download and set it up. And in the next lectures, you can just follow along. You can find Visual Studio Code by typing VS Code in any search engine and then code.visualstudio.com should be the first one that appears. And you can download a version for Windows, Mac or Linux. The download should start immediately. And then when you click on the download, it should install the installer. You should accept the agreement and then you can just install it wherever. So I'm not really changing anything. I'm just clicking next and install to have Visual Studio Code installed. Then you can launch Visual Studio Code and then you should see a Get Started page. What you can do first is you can open a new folder, a new project. I'll make a new folder, go Hello World and then select this folder. So now I am in the go Hello World folder and let's create a new file. Our first go file to make sure we have all the extensions installed. So the first error that you will see is fail to find the go binary. And you still need to install go first before you can write any go code. You can click on that link or you can go directly to the download page of go. The official web page is go def. And here you can download a version for Windows, Mac, Linux. We are Windows. We are going to download the Windows version. And then we are going to run the installer. Again, next, next and install. And then once it is installed, it's probably best to close Visual Studio Code. You might have to close it a few times in the beginning to make sure that once you install all the components, that those components are also correctly loaded. Finished. And then let's close this just to make sure that this is properly reloaded. Let's open it again. And then once you open it again, it's possible that you already see a pop-up coming to install the go extension. If not, here you can type in go and click install to install the go extension. And then it will immediately start complaining about lots of dependencies that are not installed, which you all have to install. So go tests, go modified tags, go play, deal V, go PLS. So this deal V is for debugging and go PLS is for autocomplete. So these we definitely need. And then it will all download these. And then once these are all downloaded and you see go outline, it also ask go outline. And once these are all installed, we can test our first program. So what do we need to have installed? We need to have installed go, the go extension, and then all these tools that Visual Studio code will propose. We will need another restart, but I just want to write some simple code first to test whether our setup is working. Package main, import FMT, and then fung main FMT printf hello world. I want to save this and install this one as well go imports. This will automatically import our libraries when we need them. Once this is finished, I'm just going to close it again. So this is succeeded. Close this again. Open Visual Studio code. And let's see where we are now. Still need to add a few more things. And then let's also open a terminal. And in this terminal, I'm just going to type go mod init hello world to initialize our module. And then this red line will go away after some time. And we should be able to run our first program. So let's see if we can run it or whether we need another restart. So when you run it, the output is going to be in our debug console. Run without debugging. And it says hello world. That's what you should be able to see. Start debugging. Let's try with debugging. That should also work. And this is when DLV is used. So this DLV you need for debugging. Hello world. That also works. And let's now try to go to our terminal. And you can also run in VS code in Windows. This program either using go run main.go or go run and just a dot. So on macOS what I'm doing in this course lot is go run started go. But that doesn't work here on Windows. So whenever I use go run started go go run and just a dot also works. So if you have any problem with this, make sure that. Go extensions installed. That you close Visual Studio code. You open it again. And that you install all these tools that Visual Studio proposes you to install. Go art line, go PLS and so on to install those. And then if I exit again, now you see the retina is also gone. Now it all works. So I will start with a simple program again in one of the first lectures. The goal of this lecture is just to have your ID installed to be able to run a simple program without errors from terminal or using the debug console. In this section we are going to write our first go application. We'll end up with a full working example. We're going to start from a simple hello world. And then we're going to build up until we have this full working example. I'm first going to cover command line arguments and then we're going to make API calls. API calls from a goal line program are often used. So that's why I picked this as a first example for our first application. We're going to do Jason parsing because our test server that we're going to use is going to reply Jason. I'm going to explain you about functions, custom error handling. I will then explain you the flag package. We're going to do a post request. So first we did a get request and I will do a post request. I will do some explaining about JLT authentication and how you implement that in go. Then we're going to package our code, test our code and then we will have a full working example. That will be our first go application. So quite soon in one of the first lectures we're going to start to build our HTTP client. This HTTP get client on the left is what we are going to build and it's going to make a connection to our test server. And this test server is not something we're going to focus on now. It's going to be pre built. We're just going to start it. It's going to be an API server that's going to respond with Jason JavaScript objects. We are going to invoke get requests to the test server endpoints. We will get Jason objects back that we will need to parse. Then we will also investigate how we do post requests. We will need to do a post request on the login endpoint to receive a token, a JLT token that we will then need to access protected APIs. So what I try to do with this HTTP get client is to give you a nice example of invoking APIs. As this is the foundation of what you need to know to build applications that will interact with external APIs. Later on when we will be using SDKs in Go, you will see that they will make these API calls for us. So when we are going to interact with an external system, it is often going to be an HTTP API call to connect to an external service. So learning how you make requests and how you can parse Jason objects or any other structure that an API would give us back is for me the first big step in learning how to use Golang in a cloud or cloud native environment. Let's start with our first Golang project. I opened Visual Studio Code and this is the welcome page that opens. And I'm going to create or open a folder and in that folder I'm going to create our first Golang program. So here you can start with open or you can use file open folder. That should be the same where you click we just have to open a folder and that folder is going to be our project folder. I have a folder Golang demos that is empty. I'm going to use this folder to create my demos in. So I'm going to click new folder, hello world. I'm going to call this new folder. I am now in hello world and I'm going to open this folder. So whenever I create a new file in Visual Studio Code, it will appear in this hello world folder. This is my hello world project so I can click here on create new file or I can right click new file. And the first file I'm going to call main.go and this main.go is going to have our main function. It doesn't really matter what you give as a file name for your first program. What determines how it's going to be executed is going to be the name of the first function, which is always called main if you want to execute an executable. So this is our main.go and this is going to be in executable. That means that we are going to be able to run this later on. We will also create libraries so libraries are not standalone. They also need other Golang code to be able to use those libraries and this is not going to be a library. It's going to be executable. So once we write our code, we can then execute this program and it's going to output something. When in Visual Studio Code, it shows red. That means that there is a mistake somewhere. If you hover over it, it says expected semicolon and found end of file. That means that we have to still write something to make sure that this Golang file is going to compile. Once it is compiled, we can then execute it. So let's write a little bit of code, the simplest code that we can write to execute a program, just printing something on the screen and then we can compile it and then run it. If you are going to write an executable, it always needs to be in the main package. So we first have to define what package we are going to use. We can see a package as a grouping of Go files. So this package is going to be called main package main. And you see the red color is already gone. I'm going to save this and this is actually what Golang requires. It requires that the first line is a package definition package main. If you want to execute this file in an executable, Golang is going to look for the function main. So let's try to write this function main. If you want to write a function, we type f u n c funk from the word function, a space and then the name of the function. If you are going to use the function main, then when we execute this file, this is the function that is going to be executed. We have two brackets after the main because there is no parameters that we are going to pass to this function. This function doesn't require any parameters. And then to define the start and stop of this function, we're going to use curly brackets. One curly bracket after the main function and then one to stop it. We can save this and this is actually our first Golang program that should be able to be executable, but it's not going to do anything because we haven't written anything in between. How do we execute it? In Visual Studio Code, we can open a new terminal. The new terminal will open in the directory that we specified, the hello world directory. We should have Golang installed. So if I type go version, I have go 1.18.1. If I want to compile, go in a terminal. So with commands, we can also execute it in Visual Studio Code, but I prefer to execute it always within a terminal. That is easier. Go build main.go. Now it has been built. On Windows, you will have to type dir. Here on macOS or Linux, you can type ls. Now we have a main executable, which we can execute. So on Windows, you will just type main and on Linux or Mac, you can type dot slash main and then this program will execute. It doesn't give any output because there is no output. It hasn't written anything yet. So let's try to write something in this main function so that if we build it and execute it, that we will see some output. So there is actually a shortcut when you are developing. Instead of using go build main.go and then main, you can also do go run main.go. And that will just compile it and execute it immediately. And then you don't have to enter the two commands. So that's what I will be using. If you want, you can also run it here, start debugging or run without debugging within Visual Studio Code. But I will use the terminal because I find it a bit easier. So in our main package, we have the main function. And then let's try to write our first line of Golang. If you want to print something on the screen, there is a package FMT. This FMT package is provided by Golang itself. It's not an external dependency. It's an internal dependency, so it will always be provided within the language itself. So if you type FMT, then now we can start using functions of this FMT package and it will automatically be imported by Visual Studio Code once we start doing that. So if you want to use functions from another package, you need to import it. But you typically don't have to write the import lines in Visual Studio Code because it will do that for you. So if you type FMT and then dot, we will see the functions that are available for us. And if you just want to print something on the screen, we can use printf or printlm to message something on the screen. And you see what happened here? Once I selected printlm, then it adds this import FMT. So now we can use FMT package. And in the FMT package, we can use functions. Printlm is a function, printline. And FMT is the package and the separator is a dot. So you will always see that once you want to use a function from another package, you just enter package name dot function name and then you can use this function. So it's a function. So we're going to have to add brackets. If you want to know how we can use this function, we can hover over the function name and it will explain what it does. Printlm formats using the default formats for its operands and writes to the standard output. So it writes to a standard output. Within this function, it expects any kind of variable. So we're going to pass a string. In GoLang, a string is denoted by quotes, double quotes, hello world. I'm going to save this and then it's going to print hello world. And because it's printline, it's going to add a return after that. So we don't have to do that ourselves. Go run main.go, hello world. So this is our first GoLang program that just prints something very simple on the standard output using the FMT package. So if you rather like to execute something within Visual Studio Code itself, you can go to run, run without debugging. And then it actually says that you need a GoMod file. So you need the GoMod file, which is the module file, which manages the dependencies, which we haven't created yet, but it's very easy to create. We want to use the built-in functionality of Visual Studio Code. We can do GoMod init and we can give our demo a package name, for example, hello world. And now we have GoMod here and it says if you do GoMod tidy, it will also check all the module requirements and download the necessary external modules. We don't have any external modules, so let's try to just do a run without debugging again. And then it also outputs hello world. And what has been created is just then this file, this GoMod file. This is our module hello world, Go1.18. You can still run GoMod tidy, but it's not going to do anything because we don't have any external dependencies at this time. I'll come back to this later on when we're going to have external dependencies. Now we can just ignore this GoMod file. In my demos and GitHub, you will also see that I have this GoMod file everywhere defined so that you can run always the demos in Visual Studio Code. So that is it for our simple hello world GoLang application. Let's have a look at another package to see how we can pass arguments to our executable. So if you want to pass arguments, you want to do a go run main.go, and you want to, for example, pass argument one. So if you pass this argument, it still gives hello world. But now we want to capture this argument and just output it. So to capture arguments, we need another package, the OS package. So before we print our outputs, we want to say our arguments equals OSArgs. And args hold the command line arguments starting with the program name. And it returns from this operating system package a string, a string slice, which is the same as an array. So it's not just one string, it's multiple strings. We still get an error, two errors. The first error is because we don't have the OS name declared because we also need to import it. So if you just hit save, then Visual Studio Code will import this for us. And then we have also args. Args is undeclared by the compiler. So we need to declare it first. And this needs to be of type string. So there's two ways in going to declare it. We can either say that we have this variable declared before we use it like this. var args string. And then we can use later on to assign it to args and then we can print it. So now it still gives an error because we are not using it yet. But this is the first way of declaring and then using or assigning something to this args variable, which is a slice of strings. Or in goal line, it is actually also possible to not do this. But just add a colon in front of the equal sign and then it will automatically declare this variable. So now the error that we have is it is declared but not used. So now it is declared. If it is the first time that you're using a variable, you can just use this colon sign and then depending on what the variable on the right side will output, a string slice in this case, then goal line will declare it. So now we still now we still need to output it. So what I'm going to do is I'm going to change this println print line in printf. And printf stands for format. So then I can use then they can format my string. So I say hello world. And then I'm going to add a return backslash and is return. And I'm going to say arguments. And now I can format my string. So if you want to insert something in the string, you can use a percent sign and then S for a string, or you can use V just for the value. And then with the V then goal line, the printf function in goal line will decide itself what is going to output. So this is the default, the default output, which is good in our case because we just want to output a string slice. We add another backslash and because we want another return at the end. And then we need to specify the variable that needs to be put in place here. So we can add another argument to this function. And we can say just output arcs. So arcs, the output of arcs will go right here will be replaced and then output it. Arcs contains all the command line arguments, including the starting with a program name. So the program name is going to be main. If the program executes successfully, it will be main and then the argument one. So let's have a look. Go run main.go argument one. So it says hello world, arguments, arguments. And then we have our main. So what go run does is it actually compiled it in a temporary directory. So this is a temporary directory. This is our main program name. And then we have our argument. So if you want to use this argument one in our program, how do we extract this second element in our string slice? So they can use this in our program. We typically want to do something with all the arguments, not only the first argument. So if you would add another argument, argument two, then we have the main argument one and argument two. And what we want to do is we want to make sure that we don't have the program name, but just our arguments. So then we can work with string slice functions, string slice functions. So this is string slides of three elements now, and we can slice this up in different ways. So it only outputs the two last elements of this string slice. We can say as a second argument. So this is this is our OSRX output. And then we only want to see the arguments. Then we can say I'm going to use args, which is a slice. And with square brackets, you can denote what argument you want. So if you want to only show argument, if you only want to show the second element, then you can enter one because we start counting at zero. So zero will be this argument main, and then one will be argument one, because we start counting from zero. I'm going to save this, execute it. And then you can see hello world. This is the full string slice. And if you just specify the second element with number one, then we see argument one here. And I didn't add another enter. So that's why it looks a bit strange here. I'll add another enter. So this, so this is argument one. And you see there's no square brackets anymore around it because this is not a slice anymore. This is just a string. So this, so when you, you, when you specify one element, this is now a string. But you don't really want just one element. We want to see all the elements after are executable. So you want to see argument one and argument two. We can say within goal and when working with slices, one is my starting position. And if I add a column, I can also specify the end position. The end position is going to be one, two, three. So three is going to be the end position, which is also the length of the slice. So to know the length of the slice, you can use the function, the built-in function lan. So if I say lan arcs from one to lan arcs, it will give me starting from one. So not from zero, starting from one. So it will start here. All the arguments after argument one. So argument one, argument two. And if there's an argument three, it will also add argument three. Because with the input that I have here, it will be give me the slice elements, which will be another slice actually, not just one element, but another slice. Give me another slice starting from one to the end of this slice. So one, which is this one, the length will be two. So from one to two, one, two. If we have three arguments, it will be one to three, one, two, and then the third argument. But we can see that there is a recommendation here. The recommendation here in Visual Studio Code is actually to not put this one. Because if we just remove this and we say from one to, and we leave the space empty, it will automatically go until the end. It will automatically take the length of the slice. So the easiest way to understand how to work with this is to just try it out yourself. So if I save this and do go run, then the arguments are now argument one, argument two. And if I do argument three, if I add argument three, it will show argument one, two, and three. And if I then changes in another number from, for example, one to two, and I execute argument one, two, and three, I'll only get argument one. Because it went from this zero argument one. Start from here and end at element two, but don't include element two. So we stop here and we'll only get argument one. If we change this to three, then we get argument one and two. And if we change this to the length, if you have three arguments, then the length will be four. One, two, three, four. So if you would change this to four, then we get argument one, two, and three. But you have to be careful, though, because we need to do some extra checks as well. And that I will explain in the next lecture. If we would say, give me argument one, two, five, but don't include the fifth one, I actually get a slice bounds out of range error because we only have the capacity four. And I'm asking for a range of five, which means that I'm asking for something that's not there. And then Golang will show an error. So to get used to how to work with slices in Golang, it's best to try it out yourself by changing these values a little bit and see what you get us output. Just remember that the first argument here is the starting point. The second argument here, the five is the end range, but it doesn't include the actual element. And we start counting from zero. So arcs zero is the first element, arcs one, the second element. So here we start from the second element, we go to the fifth element, but we don't include the fifth element. So play around with it a little bit, and then you will get used to how slices work. Then the last thing I want to explain is how do you run this within VS Code, within Visual Studio Code. So if you want to run arcs one, one to the end in VS Code, run, run without debugging. And then we have to go to the debug console. And then here we have hello world, the arguments, but the argument list is empty because we didn't pass any arguments. So how do we pass arguments? Go, run, and we can add a configuration. This will create the launch.json file within VS Code. And here you can then create your configurations. So here we have one configuration launch package go launch. And we should be able to add, we should be able to add arguments right here. We put a comma on the last element here. So this is JSON. And then here you have arcs. And arcs is an array in JSON. If you want to pass one argument, we say argument one. We save this back to our main. Run without debugging. And then here we have now argument one. So you can have your launch.json where you can define extra arguments. And then they will appear here in the debug console in the output when this program is executed. So when you want to pass arguments to your program, you can do that here. In the previous lecture, we just outputted our arguments. But what if you want to use a specific argument? So let's say we always want to use our first argument, which will be the second element. We refer to it as number one, element one, because we have element zero. So this is element one. This is the first argument. So first argument is going to be argument one. So what if we execute our program with one argument? It will show our first argument. But what if we execute with zero arguments? We'll get a runtime error. So what you want to do is we want to avoid getting these runtime errors. If we want to for sure use our first argument to execute our program, we need to check on this because the index is out of range on this array. So this OS args is a string, string array. Hold the command line arguments. First is our name of our program. So we want at least two elements. If you have more, that's okay. But if you have less, then we need to throw an error. So how do we check the length of our string array? We can use the LAN function. And here you will see the help of the LAN function. If you use the LAN function on an array on a pointer or on a slice, it will all be a little bit different. For example, if it's just a string, it will return the number of bytes. If it's an array or slice, it will return the number of elements in this array or slice. And this is also a good time to explain the difference between an array and a slice. So we don't really have to do anything different in goal line to work with an array or with a slice because it's all transparent for us, but there is actually a difference between an array and a slice. So when we are outputting everything of the string array here from the second element from argument one up to the end, that is a slice. So it's a dynamic view of an array. And that is the difference between a slice and an array. A slice can be of dynamic length. And that's what you should remember. Slices are always of dynamic length, whereas arrays are of static length. And you don't need to think about it much because when you pass an array or a slice, it's passed in the same way. It is just sometimes that you have to take into account, for example, with the length function, that there is actually a difference between an array and a slice. And that will always be documented in those built-in functions. In general, you probably always use slices because the length of them are dynamic. And you don't really have to think about how many elements can there be maximum in this slice. So length arcs, if it's less than two, we are going to return an error. You can say the usage of this program is hello world. If you compile it, we will call it hello world argument. New line. And then we can exit after this. If we want to exit, we can use OS exit. And then you can enter a code which is an integer between 0 and 125. If you exit a program with error code 0, it was successful. Anything else is not successful. So we will say OS exit one. It was not successful because we were missing an argument. I will save this. Go run may not go. Usage hello world and then the argument. Exit state is one because we added this line. If I type this echo dollar sign question mark, it will return the exit code of the last command. It is one. So when we are scripting and we are executing a goal line program and based on whether it was successful or not, we want to execute maybe something else. We can use this. We can use this exit code and we can say, okay, we need to stop now because this goal line program didn't execute properly if it was not returning 0. If he passed one argument, then the first argument will be argument one. We didn't get an error because we checked on this. Echo dollar sign question mark will return 0 because it was successfully executed. We can still have more arguments. Argument two will also still execute because what I did here is I'm still passing this slice to my print f. The slice is representation of the second element to the end. Going to save this and then in the next lecture we'll be able to use these lines of code because now we are sure that the first argument is passed because we are checking the length of the arguments and we will not get an error anymore when we are going to use this first element as an input for our program. Before we stop at this lecture though, I still want to show you something. We have here args equals os args but how did this args actually get populated? How is it possible that we actually got these values in there because we're just assigning these variables to our own local variable here but how come that this actual contains those arguments? So let's have a look. If you right click here and we go to definition we will actually go into the os package and have a look at the go on code in this os package. So here the args is declared in this os package and these packages they also can have an init function. So we have the package os and here we have the init function and this init function will be executed when we start using this os package and here in this init function which is written by the golang developers the golang developers will then check is our runtime windows if not, there's another init function for windows in exec windows if we are on Linux or Mac like on our platform Diamond then we're going to execute runtime args and runtime args is then declared here this is the signature of the function but the actual content is in another package it's in the runtime package. So when we are developing golang we don't really have to care that much about the runtime as long as we are not doing any system calls but here they are doing some more tricks here than what we would just write as normal golang code. So this is the proc.go we can do the same with printf so if you want to see the fmt printf function then we can right click here as well and then here we have the print.go that has a printf, sprintf, fprint and other functions you typically don't use it on the built-in packages but if you have external packages then you will see that to understand these external packages you often have to look into the code not often but it happens where you look into the code just to understand what they are actually doing in this package so now we have our arguments we are checking on the arguments we need at least one argument we can use this for our next demo we can use this argument, this first argument to supply some input to our program in this lecture I am going to show you my golang cheat sheet which is just a one-pager that shows code examples which is very handy in case you are not familiar with Go and you are still learning it and you want to know the syntax of the programming language you can print this one-pager and keep it handy for when you are writing code and you forgot how to do something then you can just have a look on this one-pager I will attach this pdf as a resource of this lecture so that you can also download it and I will keep it up to date so I will make some changes even after this lecture and as we progress with the course I will also explain more things on this cheat sheet so now I will just explain a few basics on this one-pager just to get you started this is how the current version looks like and you see there is still some white space so it is definitely not complete and for now I am going to focus on the left part of this one-pager the variables, types and conditions and maybe also the operators so that we have the first two columns covered let's start with the variables how to declare a variable in Go you can use var a is a string we can also declare a variable var a is a string and immediately give it an initial value we say here that it is a string but it is not quite necessary we can also let the compiler figure out a type so we can say the type is inferred when we say var a equals string the Go compiler will see this is a string so this a will now be a string the shortest notation is actually a column equals something and you don't even need a var keyword so I will most of the time use this notation where we say a column equals something and a will become then of type string this column is only necessary when you are declaring a new variable once you want to assign something else you can just say a equals you can also say a and b equals 1, 2 a will be equals to 1 and b will be equals to 2 and you will see this notation also a lot when we are executing functions where a function returns two variables we will say a will be the variable for the first return value and b will be the variable for the second return value in case of a function so this is the basics of declaring variables and initializing variables next is a map so map is something that you will see in our next lectures a map holds a key value pair here we declare a map so var a string string is a type a now the type is map string string the first string stands for the key and the second string stands for the value you can also make it shorter by using the make function so a colon equals make a map of string string so now you are making a map where also still the key is string and the value is string you can also initialize a map immediately with initial values here the k is a key is the value and if you would add a comma here then you can also add more key value pairs so this is a map don't worry too much about it if you are not familiar with it we will start using this in one of the lectures and then you have an array and a slice which we also will use gradually in our demos an array is fixed size and a slice is dynamic var d instead of var a string we have var d and then the type is of array with size 2 integer here we also initializing an initial value the initial value is an integer array of size 2 and the value is 1,2 so the first element is 1 second element is 2 when you declare a slice you don't have to specify the size of your array because the size is dynamic so here we have var a of string it's a string slice it can contain multiple elements but we do not need to add the size we can also use the make functions just like we were doing with a map here we make a string slice of size 5 we will also start using these make functions in our lectures here we have a make function where you make a slice of size 5 but with capacity 10 so you can give another argument to allow for more capacity so they are dynamic so they will increase if we start putting more than 5 elements in it but here we are already saying we want to have the capacity of 10 so if you already know that there is going to be 5 more elements then we can make a new slice we only want to have 5 elements in there so 5 elements will be empty strings but we will still have capacity for 5 more before we have to allocate more memory to expand our string slice we then also have a way to declare multiple variables we can just use var with run brackets and then we can line by line declare our variables a is a string, b is an integer so this is a little bit easier to declare multiple variables and then we will also be working with structs structures, struct groups multiple variables together this is how we declare it we say type my struct so this name you can choose is of type struct and within my struct I have just one string so here you can group together multiple variables so you can have d string, e string if you want it here we just have one this is how we define our type and then we can create a new variable declare a using the my struct type this is a type of struct my struct goes right here this is a type and we can then say for a is of type my struct in my struct we have the string so that means that we can also say I want a as a new variable and it's going to be of type my struct and within my struct and now we're going to use curly brackets we can say c, c comes from here is a new string so we have now a struct and c is a new string that's a variable within the struct if you wouldn't specify a string we would just take the first element in our struct so these lines are actually the same if we later on want to refer to this c within my struct a or we want to change it to a dot c which refers to the c within the struct and we can change it to a string for example in the lectures you will also see that we have struct slices or struct arrays so we can have structs within a slice or within an array or within a map to make complex data structures and then we have the types that are built in in go we have the bool which is true or false a string this is a string a byte which is an alias for unsigned int8 if you will have to work with bytes a rune which is a representation of a utf8 character it's an alias for int32 and if you would use single quotes here single quote s for example then that would be a rune so if you use a single quote s then a would be a rune float32 and float64 if you want to work with floating points so 1.32 for example would be a float then we have integers so the size of int depends on whether you are on a 32 or 64 bit system if you are on a 64 bit system then this is going to be of size 64 otherwise on 32 it would be 32 8, 16, 32 and 64 typically you would use int if you want to use an integer and then you have the same in the unsigned form which means that you can only have positive numbers but then you can obviously store more numbers and you have an unsigned int pointer which can hold a pointer address and then you have complex64 and complex128 for complex numbers those variables are going to create you can then compare using conditions so we have less than, greater than less than equals, greater than equals equals and not equals so they are very similar as in other languages logical operators or operator and an end operator and a not operator if a is greater or equal than 10 and b is greater or equal than 10 then execute something else if not a less than 5 then do something and otherwise execute this block so with if, else, if and else you can create conditions and in Go you also have a switch you can write a switch for a which means you can then check what is the value of a in case the value of a is 5 then you can do something here you can have multiple cases and you can also have a default which is optional so if you have case 5 only you can automatically go to the default so this is the switch notation and then you have operators like plus, minus multiplication and so on some simple examples are a is 5 plus 5 so that would then be 10 a plus equals 5 which is the same as a equals a plus 5 so if 5 is 10 then you would just add 5 more which is 15 and also you can have b equals a condition so is a greater or equals a 5 if yes then return a boolean which would be true so b would be a boolean type and would contain true if b was an integer then you could do b plus plus which means that you can increase b by 1 and you could also do b minus minus which is going to decrease b by 1 I will stop here and let you continue with the actual course material you can print this and keep this close to you when you are following the course and in case that you want to know how a variable is declared or what a specific time is or how to write conditions then you can have a look at this one pager in one of the next lectures I will then explain more items on this page when it is relevant to start explaining them in this demo I will show you how to make an HTTP get request so we are going to run a server I have a go lang server that we can start and then we are basically going to write a client we are going to make a connection to an HTTP server and just output what it returns we are going to output the HTTP code and the HTTP body to actually use external APIs because if you want to use an external API you will need to do HTTP requests or at least some kind of connection request and retrieve the data parse the data do something with the data and maybe make another connection request so the easiest way to show this is to make an HTTP request and show the information on it then in the next demos I will explain you how to parse the data and how to do the error handling and so on so we can use this argument as our URL that we want to reach so we can say we are going to write a program HTTP get we expect a URL and then we are going to make connection to this URL if we don't get a URL we will show this error, we will explain the usage so what do we do what is the first step is to make sure that the parameter passed is actually a URL because if it is not we should also throw an error how do we validate what it is a URL we can use some go lang packages that have this functionality in it how do you find these go lang packages you can have a look on the website you can just type in Google you will find a stack overflow thread or you will find some help documents that show you what is the easiest way to parse but the first way in go lang to find something to do something should always be is there an internal package is there a package provided by go lang that can actually satisfy my needs and for us because we just want to parse the URL it is actually included in go lang there is a URL package and that URL package we will use to parse our potential URL we are passing as an argument so how do we do that we just want to validate it so we don't really need to do anything with the output we just want to know whether it is a valid URL so there is a package URL and it has the parse request URI parse request URI parses a raw URL into a URL structure if it is not a URL it will return us an error code so this is the function signature func this is how we use it URL parse request URL we pass a string and it will return a URL a URL type that is defined within this URL package and an error if it cannot parse it so while this is handy to receive this URL type that we can then use to get a part of the information out of it to for example see whether it is HTTP or HPS we don't necessarily need this because we only want to validate whether it is a URL or not so let's start with passing our string so we are going to pass args and our second element within the array which is number one and then this will then return the URL and the error so we will say my URL and then this is the error equals this function so in my URL I will now have URL URL in this error I will have an error and I only want to check on this error I don't really need this one so if you don't need a variable you can actually replace it by an underscore so if I put an underscore here I show it to the compiler to go like that I don't need this you can just ignore this first argument I am only interested in this argument the error argument and if this error argument contains an error then I want to stop the program so what I can do is I could enter here and start a conditional just like I did here I can write if if our error is not equals to nil then it will contain an error and then I will do something but there is a shorter notation it's very handy to use a shorter notation I can say if and then my statement so first it will execute my statement and then I use a semicolon and then I can write my test my test is going to be if my error is not equals to nil which means that there is something in this error then I will output the error and exit so this error is of type error and this error is basically just a string or empty if it is empty it will be equals to nil so if it's not nil it contains an error and then I can say usage htp gets URL URL is in invalid format and the actual error is a string of the error so this will just output the error so we can save this so now at this point we are sure that args the second element of args is a valid URL and we can then use this to do our htp request how do we do an htp request well Golang also has an htp package htp and then here we have all the functions we can use including the get function so this will do an htp get we can have another look at the function signature it accepts a URL and then there is a response and again an error is being returned so we can pass the URL which is still this one we can also if you want assign a local variable that just says URL or hostname and do hostname equals args so you don't have to repeat this parameter and then it will give us a response and error so again we could wrap this into an if if you want to so we can say if and then semicolon but then we actually will have an issue with a variable scope because if you do that if response error if error is not equals to 0 then right here we are not able to use response response is actually undeclared whereas here response is declared you see the error here is different the error here is undeclared and here it is ok it's not used so if you use an if and we declare our variables within the if only within the conditional we can use the variables and outside you cannot so this can easily be changed by declaring it higher outside the conditional and now here you can see variable of response is not used but it's actually it is declared so we can use it here depending on what you like most to declare your variable here or to not use this shorter statement you can actually just split it up like this again and then your response will actually be scoped to the function and not just to the conditional and to me that's just an easier way of working so we are going to do an HTTP get of our URL it returns a response which is of type HTTP response and an error also if you see this star in front of HTTP response that means we are getting a pointer back I will have a separate lecture about pointers in general don't think about it too much just don't use pointers only when you need to use it and right now for our simple programs we don't really need to use pointers so we can just ignore it if we have an error we're going to quit our program but we're going to quit our program in a different way because it's not a user error it's a system error we actually want to see the error completely so then you can use log fatal fatal is equivalent to print followed by call os exit 1 so you could do it nicely and do an FMT printf or you can just do this log fatal which will exit our program and we can just pass our error as a parameter so you can do it either way or you can just do it like this why do I do it like this here and here different because we want to have more user friendly errors here so if the user does something wrong we just want to have very user friendly errors you might want to have user friendly errors here as well but if we are just writing this program for ourselves then log fatal error will do un-declared name log we still need to save and then log will be imported no errors here anymore response will still show an error because we haven't used it so if there is no error response will have the HTTP status code like 200 is ok 404 is not found 500 is server error and so on and it will also have the body but the body is actually special the body of the page can be bigger than our memory that we have available so our body might not contain all the data from the HTTP output so it's wrapped in a different variable and that variable is a stream once our program exits we need to make sure that we close that stream and to do that we can use another keyword that we haven't used before it's called defer defer response body close that means that once this function is finished then at the end our body which is our HTTP body it will be closed so if you have a look at this explanation here the body represents the response body the response body is streamed on demand as the body field is red it's streamed on demand so it doesn't contain all the information yet because let's just imagine that we are downloading hundreds of megabytes of content if it would all be downloaded instantly then we would get an out of memory error and that's why it is streamed and not all red at the same time so that's why it's wrapped in a stream and at the end of our program we just have to close our stream it's what you should do it's to keep it all working correctly so if you would have a different function if you wouldn't have the main function but a different function then at the end of the function it would close the body because at that point we don't need it anymore and we can close it it's not going to immediately close it it's because we use this defer keyword only here when we are at the end it will then close this body so now we can use this body this body is streamed on demand but we actually want to read all of it why do we want to read all of the body because we are actually working with very small outputs JSON outputs often on HCP calls if it's in memory you can just read it at once we know that our JSON our output will fit in memory so we can read it out once how do you read it out once? there's another function built in Golang for that it's called IO read all we'll read from a reader from an IO reader it is a stream and it will read until the end of file so we can say read all response body and then response body and then what does it output? it outputs bytes an error so we can say body error equals IO read all we just test again whether we had an error and if we have no error we can now output our body our body is an area of bytes so we still would have to transform this into a string but there is a simple string function for that let's do fmt printf htp stages code which is going to be a number so it's decimal and then we can enter a new line and our body is a string where is this htp status body? also in the response here we are using response body but if you do response and we put a dot there is other fields that we can use you have the content length and the status code status code it is an integer that's why we use %d and then we just need a body let's try to just use body this is of byte let's save this let's do a return and we ask output the body as a string so let's go to my github repository if you haven't cloned it yet make sure you clone it my github repository golang4devopscores.gith then we can go into this directory here we have here we have the test server within this test server we have the main.go so we can just do go run main.go or we can compile it or what I also typical do is I do a go run start.go which will just run all the golang programs so if you split your main.go out in multiple files then it will load all the files go run main.go this will start a server and it starts a server on port 8080 and I will just open a second screen here now I have two terminals so let's have a look at this main.go it does a listen and serve of an HP server on 8080 and let's now do the go run of our main.go which is the HP get file go run main.go make sure it's saved let's see what is the output going to be I need to of course add an argument that's the whole idea so arcs1 is going to be my URL 8080 and then there is a word endpoint that should return some JSON and here we then have HTTP status code 200 and this is the body the body is a string so the conversion between the bytes the byte array and string is actually done by our printf function because we are using this string in some cases you need to transform your byte array into a string you can also use a string so if you for sure want a string you can use a string function and this will return a string and this can be useful because sometimes it doesn't always know what to output if you have a byte array because if you say I want to output a string it will work but I'm not sure what it will do with the default value the default value is a byte I just changed it now it's actually the array of bytes so this is how it looks these are strings but they still need to be converted from bytes to strings so if you convert it to a string this should work because now we are passing as an argument a string or you can say the body as an array of bytes but printf needs to convert to a string status code 200 so if I would try some other endpoint words then I get status code 404 and then the body is found so this works we are now making an htp call to our server with this htp get this htp package we get our response our htp response which has a body which is being streamed but we read all the data from our body and then we output the body into the screen which is just a json in the next lecture we will go over some more details and then in the next demo we can parse this json into a golang variable which is a string but we can actually parse this so that's it if you get any error make sure that you don't have anything else running on port 8080 if so you just might want to change this you can also edit this main.go and change this port number if you already have something running on port 8080 so that's it so our program is getting longer now and we already started to do something useful let's continue with our htp get demo I changed the contrast a little bit of my visual studio code I hope this reads a little bit better than the previous lectures I also made the text a little bit bigger but I might have to make it a bit smaller again once we start working on bigger programs we'll see I think this is an improvement actually so let's go with this this is what it did in the previous lecture htp get of our URL and we actually just outputted the body %s we output the body but in general we are going to want to parse the output we're going to want to put it in golang variables and we haven't really talked a lot about variables we have the error type we have the body type which is byte we have these custom types here like a URL but we haven't really created any type ourselves to capture for example the json in so let's have a look at what types we need to create to capture this json I have still my test server so let's run the test server again and let's run this word endpoint so this is the json that we want to parse we have a page which is a string an input that is a string and the word is an array and this is actually written in a way that we can add some words we can have an input and we can add some words to show that it displays in the json array words we can also go to this host name in the browser we can add some words if you want or we can also just parse it here we can parse an input variable so our input can be word1 and then we have added word1 to the words and our input is word if we exit our test server then it will be cleared again so this is only, this is kept in memory when we shut down our server it's gonna be gone so we can add more words in a separate window in a browser or you can just do it here whatever you find the easiest so this is what we would like to parse so we want to create a type in golang to parse this output so we have first page which is a string input string words array of strings golang provides a type that is similar to objects in javascript that bundles together multiple different types like multiple strings or a string in an array and it's called a struct so we can define a new type of variable using the keyword type and we can call this words and this type is going to be a struct and in this struct we can then define the attributes that we would like to see in this words struct and it's going to be page input and words so I'm going to write page which is a string input which is also a string and words which is an array of strings so now we can parse this javascript object into a golang struct and we should be able to then read exactly the the page or the input like words or word1 or a single element in the array because when it outputs it's just a string when we output this right here it's just a string it's a body we haven't parsed it into the smaller parts into the the json attribute that we want to see so this is a struct but we don't really have a translation yet how a json attribute will translate to a golang attribute within this struct and in golang we can add metadata that then can be used by these packages that convert json strings to structs and structs to json strings to add this metadata we can use backticks we can say json if you're going to parse json and those json libraries will exactly look for these keywords we are looking for the json attribute page when we are going to parse this and here with input we're going to look for the input attribute and with words we're going to look for the words attribute and now that we are going to convert this when we are going to use this json library that's going to convert this string into this struct this json library will look for this metadata and when it sees page and it matches it will take the value of that attribute and put it into our field our struct field here we map a page, input and words to our struct fields so in golang we call these fields we have one mistake here I have a lowercase words here and these are uppercases and let's see what golang has to say about this struct field words has json tag but it's not exported so in golang when you use functions and when you use fields and also with the types you can export a field so either you have non-exported or exported field and this one is not exported so with the json library this will not work the json library need exported fields otherwise the json library which is json package cannot access directly these fields so we need to change our lowercase w in uppercase w so we always need uppercases here otherwise it's not going to work if you forget it then visual studio code will alert you on this and so normally it's a mistake that is easily cut so now I have page input and words and the first letter is an uppercase so that these fields are exported so that external languages can access our fields so I'm going to save this and then we only want to parse our data if our response code is 200 so let's remove this print line and let's first check our hp status code if our response status code is not equal to 200 so this is an integer so when we have an integer we need to compare an integer so if you would compare against a string it would not work you see you cannot compare mismatch types int and untyped string so if you compare the status code it's going to be without quotes 200 if it's not 200 we want to print a message and exit so I'm just going to copy paste this message and exit response status code is not 200 input output hp code is the status code and then we can print the body we can convert the body to a string or we can let printf convert it for us if the status code is is 200 we want to parse to these field words so first we need to declare another variable I will declare the variable words of type words so this is type words, this is struct with the fields and this is our variable that is declared but not used yet now I'm going to use a json package to unmartial our json string into a goal line struct unmartial parses a json encoded data and stores the result in the value pointed by v if v is nil for not a pointer it will return error so we need a pointer so when we need a pointer the external package will explain to us you need to use a pointer so then we just need to supply a reference to these words instead of just passing words so json unmartial what is our data it's our body and then any type we can pass but it needs to be a pointer so we're going to reference words and to reference we can use the ampersand sign if I would not put it again visual studio code will alert me you see it's underlined now call of unmartial parses non-pointer a second argument so that's why we reference we have a reference to this variable and then we basically pass the pointer and not just the argument unmartial returns an error error equals json unmartial I don't need to put a column here because we were using error earlier so error will be already defined we can just overwrite it so why do we have error here so why do we have a column here because body was not defined error was defined so you still need to call on here but here we don't need to call if anything is wrong we can again do a log fatal now our words should contain the correct values so I can now type FMT printf and output some of our words json parsed page string and I want I want to output all the words also as a string and now I can say in my struct I have words dot page page is going to be the field that's going to be inserted here and then I'm going to do words dot words which is an area of strings and I just actually want to return a string so let me just changes in V the default and then it will just output for us a formatted array of strings let's test this out go around main.go I'm not going to pass an input now because we just want to see the same json output what do we see parsed json parsed page is words and the words is word one you see and it's actually the FMT printf of this of this value that puts these square brackets around it to print it but let's say that we don't really like these square brackets and we just want to have a comma separated list of the words now we could actually parse this because now we have parsed the json and it's an array of strings you have string functions in go lang in a string package and you can join an array of strings together and provide a separator so let's try to do that strings.join join concatenates the elements of its first argument the array of strings to create a single string and the separator we can then define this place between the elements so we're going to join and our separator is going to be just a comma or a comma in space and then we can save this visual studio code will import strings we now have strings imported and we're going to join this array of strings or this slice of strings it always accepts slices and arrays so you don't really have to worry about a difference let's try to run it again but we only have one word so it's not going to show let's add another word word 2 so now we see if you add another input we have words is word 1, word 2 and we see it is comma separated word 1, 2 and 3 so we actually parsed our JSON this was our JSON and now we have all the values available within a struct we have 3 fields and now we can do operations on these 3 fields we can check the length the first element if it exists we can use input, we can use a page so the first step when parsing JSON is to write a struct and then you can use the unmartial function there is also a martial function to do it the other way around if you have a struct you can also convert to a JSON string and of course there is also tools that can help you convert JSON to a struct so if you write in google or any search engine JSON2 struct go along as a search string then you will find web pages where you can just copy paste JSON and it will give you this as output without the name of the variable obviously because the name I picked myself so if you don't want to write it yourself or if you have big JSON struct then the easiest way to have these structs is to just use one of these tools and paste it and then give it potentially better names if you want to give it a nice name so I want to say just one last thing that we now have used the string types the string array types and now this complex type struct is new but you can obviously use different types here different built-in types or different struct types as a field and we will have another lecture to go over more possibilities with some examples to show you other types that you can use because these are just a few types that we have been using in this lecture I want to talk a little bit about interfaces so if you go back to the demos and the codes that I showed you in the previous lectures I think I did a good job in explaining exactly what variables are returned like a URL here and how we are using these special types like a response that you can actually have a response body within this type that you can then use or that there are functions within this type that you can use to close it but that doesn't really show you what is under the hood I just told you you can use these variables like os args which are defined in a different package but there is something there is something missing here something that I didn't explain yet because we have this response body and this response body is of read closer and here we have a read all and it expects an IO reader so our types are actually different here we expect an IO reader but we are passing a read closer and if you have a look at our previous commands printf we always first provide a string so it's always a string type that we provide first and then this is a second argument of the any type so it can be an integer, it can be a string it can be an array of strings printf will figure out for us how to pass it and also you see the three dots in front of any it's a slice that means that this second parameter is optional and we can have zero or more of those any parameters or here we have exit which expects an int or here we have a string here we also have a string so you see that in most of our functions we pass a type that it expects but here in read all it expects an IO reader supplying an IO read closer so let's have a look why this is possible, why can we sometimes supply a different type to a function and that still works in GoLang and let's have a look at this streaming function because this streaming function it's actually quite important to understand how this works because it's so much used so why don't we in this lecture write our own variable that we pass to read all a streaming variable where then read all will read all the data from and then we will have an idea how this response body actually works so I'm going to remove all these lines we're going to keep the read all and we're going to create another variable over a specific type and we're going to pass that to the read all method also this is going to be a little bit more advanced lecture so don't worry if you don't immediately get it this concept of interfaces will come back multiple times to fully understand it later on so let's first have a look at this read all if I right click here and I say go to definition I will see the function signature and here we are using this read all IO reader so once we are in the IO package it is not going to show this prefix IO reader anymore so if we go here it shows IO reader but once we are in a package it is just called reader and this reader this type is defined within this OS package so if I do right click and I go to the definition I will go all the way up line 83 and here this is defined and this is a type interface so we are not asking for a specific type like a string or array of strings we are now asking for an interface and what this interface means is that what you are supplying as a parameter to this function it needs to have the function read this function signature needs to be within that variable needs to be available within that variable so if we make our own variable as within this variable if this struct has the function and we didn't do this before yet if this struct has the function read then it satisfies our requirements to use this as a parameter and then we can use it within this read all function so let's try to go back to this read all function it expects this reader which is an interface and our variable has function so within this function the only thing we can do with r is we can do r.read and if you have a look where is r used here r is used so we can only do r.read we can only execute this function read from this parameter r to then read the data so to read all the data from the parameter that we supply we can only use the read function so if we build our own variable that we're going to pass it just needs to have the read function because that's what the interface says the interface says that is defined within this package says you need to have a read function for every variable that you pass to this read all and if that is satisfied then you will not see an error and within this function we can then only access that particular function because within read all the only thing that we know is that this r has a read function so let's try to implement that and then we'll have our own reader where read all can read all the data from so our implementation is just going to be a static string because it's going to supply a string to our reader and read all can then just read that string with the implementation that we did in our previous demos that was actually the HTTP call that was happening and then the endpoint would return some information like JSON and then read all would read that information so we are not going to make any connection we're just going to supply a static string so we don't need any of this anymore we just need to have the read all you can also just create a new project if you want to keep this code and the code is also available on my GitHub repository and once we have read the body we can print it body or output body we can maybe just call it out and the parameter that we're going to supply is going to be our struct a new struct a struct that we still have to make so I'll just keep this empty for now so I will call this new struct this new struct that I'm going to create I'll call it my slow reader and why am I going to call this my slow reader because we are going to let the read all read one by one one character by one so it's going to be very slow but it's going to be nice as an example type my slow reader it's going to be a struct what do I need in my struct well I need to be able to supply a string because that's a string that read also going to read so I would say contents contents is a string how do I initiate a new variable with this type I can do either this var my slow reader instance it's an instance my slow reader is a type and this is now a new instance it's declared but not used it declares an empty instance of the struct but I don't really want to declare an empty instance I actually want to define also the contents already so there's another way to more easily declare it and that is by using this notation my slow reader instance equals my slow reader and then I'm going to use a curly bracket and then within this curly bracket I can provide values for the fields that are defined within the struct so I can say my slow reader contents hello world and I also need to supply a comma at the end because there can be other fields so this is the contents field this is my slow reader instance of the struct so now I just have this struct with contents in there and this my slow reader instance I'm going to pass to read all let me save so now my imports that I previously had are gone now I have one error my slow reader instance cannot use my slow reader instance which is a variable of type my slow reader as I O reader value to I read all so I'm trying to execute read all I need to provide a type of I O reader but why cannot not use this my slow reader does not implement I O reader so I need to implement this interface I need to be having this read method otherwise it's not going to work it says missing method reads so this assignment this passing this parameter will not work I don't have this read method let's have a look again at this interface the reader interface so this is the function that I need so every struct can contain fields but it can also contain functions so this struct my slow reader can also have a function I can say funk and then I need to define that this function is linked to my struct my slow reader so I say m space my slow reader and this m is then the variable that I can use within this function if I want to reach any of those fields within struct so m.contents for example will give me the contents of this field funk my slow reader and then the function is going to be read which is a copy paste of the interface function read will need one argument and returns an integer and an error so there will be an error here missing return let me just return something if I am at the end of the file and this is explained in the read all read all reads from r until an error or end of file and returns the data it read it so if I just say actually I didn't return any bytes because the first return argument will return how many bytes I say there's nothing to return 0 and I say I end of file so end of file is returned by read when no more input is available if I use this I save this and let's try to test this out let's go to the hello world actually actually I'm still in my github folder so let me go to the hello world you can also just close your terminal and open your terminal if you don't know anymore where you are so I'm in my hello world project now and then go run start go output empty so this is actually run and what happened is ages returns 0 and end of file but what is really happening here we can actually use debugging in golang so if I put a breakpoint and run start debugging then here I can use go inside function or just continue or skip to the next one to have a look what is really happening so I put a breaking point at read all and let's now go in this read all method and then this is the R this is the reader but this is actually my struct that is being passed my slow reader and if I continue oh I went too far it actually executes the read function so let's start over as we start go inside this function and when we hit the read function let's go inside the read function the R read function we have written ages returns end of file so then it continues is there an error yes it's not nil is the error end of file yes then return B and B is our buffer where we put everything in but our buffer is now empty because we didn't have anything in it and then we output nothing so our read function is executed but we want to read the contents and read will be executed until we say is the end of file so if you want to have a slow reading function and only read one character at a time we can use built in string slash functions in go lang just return only one by one but to return one by one the characters we need to know where we are so let's create another field POS of type integer which is just a number and using this field POS we can store the position that we are so the only thing that we need to do is we need to read the string contents one character by one character increment POS until we are at the end of the string in other words we can say only return something when POS is less than the length of the contents because once our position is more than the length of the contents then there's no characters left so let's try to write it out if POS is less than length contents we're going to do something okay we get two errors POS is not declared and contents is not declared because they are in the struct they are fields of the struct and that's why we have this M my slow reader so this M refers to our instance of our struct so if you want to access the fields we say M dot POS M dot contents there's still one problem when we define our POS it starts counting as 0 so if we say if 0 is less or equal the length of the contents so if we have one character it's going to execute and then what we're going to do is M POS plus 1 so we can do plus plus plus 1 then the next iteration is going to be 1 and 1 is still equal to 1 so it will execute again but there will be no more contents because our length is only 1 so we ideally only want to execute once if there's only one letter in our contents so if we say M POS plus 1 then for a length of 1 it will only execute once it will only iterate once if it's 2 then if it started 1 then 1 is less than 2 and then 2 is less than 2 so it will execute 2 times so I think that's what we want so an integer starts at 0 but actually it's easier for us if you just started 1 and then we can compare it like this the next step is to write data to this B variable so if we have a look here B is our buffer and B our buffer is passed to our read function and in this buffer we can actually write our contents and then this N is how many bytes have been read and this error here is either end of file or an error we can use to make a copy to make a save copy the copy function the copy built-in function copies elements from a source slice this is our source slice into a destination slice and this is destination slice as a special case it also will copy bytes from a string to a slice of bytes and that's what we need this special case because we have we want to put it in a slice of bytes and the slice of bytes is this one and the output is the output is an integer copy returns a number of elements copied so that's what we need because we need to return the number of elements copied and the error so if we say N equals copy and our source what is going to be our source we want to copy one letter by one letter from the contents so we can say contents or M contents and actually this will also work with a lower case so we can also have lower cases here because we don't really need to export this N equals copy M contents now we will copy the whole string but we want to do it letter by letter or rear whether we want to copy it from from our position which is 0 to start with and what is going to be the end position if you want only read one character by one character it's going to be the position plus 1 so if you have 0 until 1 this will reply h 1 to 2 it's going to be 0 because we start counting from 0 1 1 is E and then 2 is L but we don't include L so that should work and then we still have N declared but not used return N N0 that should be it let's try to debug this first because it's not going to work yet there is still something that is not correct but I want to show you that during the debugging my slow reader instant read has been executed we have P which is an empty byte array and then our position is going to be 0 length of contents is not showing when I hover over it but 0 is definitely smaller than the total length so we can copy into P our h look here is the first h we increase the position and we return N and it's 1 character h has been written into our buffer so I'm just going to hit continue now and now we are in the second position and now there is something interesting that happened so our M position is still 0 even though we increased it and this comes, the reason behind this is that my slow reader is being executed again and the contents the changes that we made to the fields are not saved because we are passing it as a copy and not as reference so this is one of these use cases where you actually need pointers and we are just going to say that this my slow reader here needs to be a pointer so my slow reader what we pass here is a pointer so that we can actually make changes so M is a pointer to my slow reader struct a copy of the struct and then here the implementation is now done as a pointer so we also need to pass this as a pointer so here we can say my slow reader add an M% here this is a reference and now this will be sent as a pointer whether you have to use an M% or a star will have no lecture going into the details of that but for now here this is now sent this M is now a pointer this my slow reader is now also sent as a reference because it's a reference here let's try to execute it again run star debugging so this is the first execution second execution now it's one now it's two and if you have a look here now we add the L now we added other L and now the O so we are just sending this back so this is the buffer but it doesn't send us the whole buffer that it already has it just send us a buffer that we can fill so this P is always empty to start with and then we just add the characters one by one to it so if we just execute the whole thing let's have a look so it actually outputs the hello world so this is now a working program in a real world scenario this doesn't really make any sense because you wouldn't copy it character by character you would copy as much as you can probably so that's it for this lecture we wrote our own streaming function so this is how we implemented an interface and we're able to pass our variable which is a struct with a function within our struct so we can pass this to read all and then read all will invoke this read function so this pattern of interfaces actually comes back a lot in Golang and this is just a small example of how you can do this implementation but you will see in other examples that this is going to come back let's go back to our HTTP get JSON example so this HTTP get JSON this is my starting situation now and let's try to add another endpoint that we are going to query here we assume that the endpoints will return words but what if we hit another endpoint so let's try it go run so make sure that your HTTP test server is running just with go run in this test server directory so we have another endpoint occurrence which will show the occurrence of words so if a word appears twice it will just show two times JSON cannot un-martial object into go struct field words of type string let's have a look with curl or you can just put this in your browser the words now here is not an array anymore it's an object because let's maybe add some words so that it's clear we have the words input word one or word two word two twice now we have two times word two so here now the word two appears two times and this is now an object and not an array in this javascript object so we would need to change this struct in a way that we can capture a map and an array and we can do that easily to first parse the page and only if the page is occurrence we are going to parse the words as a map and not as an array so how do we do that let's split this up type page and let's move this page right here let's call it name and then we're going to have our occurrences occurrence has no input but it has words and the word is a map with string as a key and an integer as a value this is a string and this is an integer and we'll have multiple occurrences of these words within this javascript object so we need to parse it as a map so both these endpoints have the page attribute so we can first only parse the JSON partially using our page struct check the name if the name of the page is words then we parse it with the words struct if the name is occurrence then we parse it with the occurrence struct and then let's try to iterate over this map so that you know how to do that as well so if the response code is 200 let's parse the page I'm going to copy this I'm going to call this page the type is page unmartial page also still the body then we're going to use the switch statement the switch statement is going to do the same as our if but only on one variable so we can have conditionals based on this name if the name is words then we're going to execute something if the name is occurrence then we're going to execute a different block of code and we're going to have a default and a default is going to say page not found if it's words we can execute the same code we just have to make sure that we also add case in case page name is words this needs to be executed and in case page is occurrence then we're going to unmartial a little bit different and also words page doesn't exist anymore we're just going to call it page name so if the case is occurrence then we need to unmartial the occurrence let's copy paste this and then let's replace words in occurrence and then we can have a look at our struct here we also have a word but now it's a map of key string int as value let's already try whether this works go run words so the json parsed page words and the words is 2 and 2 so we went through this page name was equals to words so we execute this code occurrence ok no error here we have a 404 not found ok that should be ok yeah so if we don't have a 200 then it's still 404 not found let's try to output this map how do we output the map then we can use a loop for a for loop for key value range because it's a map occurrence words so if we use this for statement then this will be our key and this will be our value key is a string value is an int so this is going to be our word and this is going to be the occurrence so let's try to output that string the and a new line and we get word 2 2 times let's try to add another word word 3 word 4 word 1 2 times word 1 and then 3 and 4 now let's run the occurrence word 1 2 times word 2 2 times and 3 and 4 only once so this is how we loop over a map let me now execute it a few times now you can see that unlike an array a map is not ordered so if I execute it multiple times the order of the words will actually differ so there is an important difference between a map and an array in the array the order is always the same and here the order is not guaranteed it can actually differ let's say that you want to if an element exists how would you do that there is also a way to check if an element in a map exists you can do that with an if statement so if and if you want to test for example if you have word 1 as occurrence then if you have this if statement it will return 2 variables it will return the value and whether this occurrence exists so this is a Boolean the Boolean will be true if this element exists within the map and this will be the value the integer so if our words 1 exists we can test whether it actually exists so we can test is ok true then if ok is true then I want to output the value font this is the output the value let's execute it again we found word 1 2 times and now if it changes into a word that doesn't exist for example word 5 then it doesn't output it and this ok value will actually false this is a good way to test whether an element exists or not so to recap the most important that we did in this lecture is we are parsing our JSON partially we are only checking whether there is the name attribute within this JSON object the page name attribute if the page is present then we test whether it is words so we can unmartial based on the word struct and otherwise on the occurrence struct if page name is empty if there is no JSON with the attribute page there then we will hit the default because then page will be empty string so this way of parsing JSON is actually quite useful you only parse a partial JSON the element that you know that will always be there or a specific element that you are looking for and if this element is set if this element is then words or occurrence then based on that JSON and then in our example we either have a string slice or a map and depending whether we have the map or the slice we need to process it differently otherwise we will get an error a JSON unmartial error like you have seen earlier this approach will still only work if the output is JSON so if we go and have a look here where we parse page this unmartial will still give us an error if there is no JSON so as long as your API returns JSON we are safe if it is not sending us JSON this unmartial will give us an error so you might still want to have a look here capturing this error you might want to handle what happens if the body was actually no JSON because then you will still get an error what error will you get for example if we do a go run on localhost which is the main page which is not JSON then we get this invalid character looking for the beginning of the value so it tries to parse a JSON but it didn't work so we still get an error we will have another lecture on error handling where I will cover some scenarios on how we can return different errors in this demo I am going to start from the JSON map exactly where we previously left with our words and occurrences and instead of using one main function I am going to start splitting it up so I am going to create one function to do the request and then one main function that will be responsible to parse the argument and then call the request function and if there is an error print the error so let's get started I think this we are going to keep but then here this is going to be our new function oh call it just do request and our function is taking one parameter request URL which is a string and returning two variables it's going to be a response and it's going to be an error so our response is not declared yet we will have to declare response let's first fix the parameters here so args1 doesn't exist anymore so I am going to change it into request URL because it's the argument that will be passing here and then we are going to do res from response error equals do request and then args1 we are going to pass and if there is an error then let's just output the error for now we will say there was an error and later we can still clean this up but let's just return error for now and then we have the response but what is the response so the response can actually be either words or occurrence because we are handling two API calls we have two different structs that we could potentially return so what we could do is we could say we are going to return words and occurrences but that's not very nice because if we have let's say five API calls then every time we will have to change our return variables ideally you don't really want to change your function signature every time that you add a feature or add another API call here so we are going to choose something more generic a response and you will have to declare this response and as in the earlier lecture that I showed you I told you that interfaces are used a lot so why don't we use an interface here if we use an interface as one function getResponse then our response interface here our response, our rest variable here will have this function that we can call getResponse and whether it's the words struct or the occurrence struct we can have a different getResponse function for both of them let me show you how that would work and it will become immediately clear so response we would have to declare this typeResponse is going to be an interface and I want a function that is called getResponse we don't need to supply any arguments but we are going to return a string so now if we have this response here then we can say if rest equals to nil then there was no response but if rest is not equal to nil then we can say we're going to print the response and the response is getResponse makes sense now we can call the getResponse function because it's defined in the interface and rest is of typeResponse response is an interface which has the function getResponse now we still need to return a variable that has this function implemented so if we have words what would we like to return we would just like to return words and we don't return an error so we can just say nil this is we have response an error so return variables so we say return words and nil and here the same return occurrence and then nil this default we're going to take away if there's no hit, no return because when we do a return it will stop the function so when we hit this here I will say there's no error really it just there's no response or I could say there is an error with no response it kind of depends how you want to handle it we are handling nil here as no response but you could also say oh it's actually an error there is no response and then you would have to return an error here now all the other occurrences where we have a log fatal error we should return the error instead so we cannot return any variable for response so we're just going to say nil which is going to be empty we're going to say fmdrf unmartial error so we know where it comes from and then the error and this we can copy paste everywhere everywhere we have a log fatal so these are the unmartial errors and here we can say there was an invalid output so instead of fmd printf we are using fmdrf which will create an error instead of a string and then here we have read all error return nil fmdrf read all error and the explanation we could copy this here this is a HTTP get error and we're going to say here just that the URL is not valid validation error URL is not valid and then the actual error that's a good idea so instead of exiting our program we're actually going to always return an error these ones I think I have every error and what's going to happen then if the error is not nil we're going to print the error in a later lecture we can talk about error handling let's now try to skip that a little bit so if there's an error we're just going to print it if still nothing was returned but there was no error there was no response if rs is not nil then we have a getResponse we do a getResponse but where does this getResponse come from we have a return words here and a return occurrences cannot use words as a response value in return statement word does not implement response missing method getResponse so we are saying that what we return here needs to implement the getResponse but we haven't implemented getResponse yet so let's try to do that and also if you find that the file is getting too long you can just put it out in multiple files so you could have a types.go or a words.go and occurrence.go where you put in those structs and those functions but for now I will keep it in one file so words need to have the getResponse function so I am going to say func and then going to add a function to this struct getResponse and we return string and then we need to return an actual string we have this slice here so I am just going to say fmt asprintf so we have printf which is prints on screen and asprintf we just return the string the errorf which is then returning an error I can then still format my output and I am just going to say just like we did previously join w words w is here and then here we can access our struct and I am going to go for something comma separated I am going to do the same with occurrence func o occurrence I just take a small cap of the name this is not an array so I just need to transform it a little bit let's try to get this in the same format so I am going to say out equals new string slice for key value of our occurrence words out equals append out so what is happening here we are going to append to our slice the append built-in function append an element to the end of a slice so here are some example append slice element 1 element 2 or or you can even merge a slice so if you just add one element you can just do it just add a parameter if it is another slice or multiple elements so that these can be merged together and then the result is then assigned to a new slice or to the same slice here so out equals append out and then we are going to add a string fmd as in print f I am going to say my word and then the occurrence kv or I should call it really word and occurrence and use that it is always nicer in goal line to use the actual name of variable rather than just one letter we get an error expected boolean or range expression ok so we need to add a range here otherwise we will not have the key value so this is out and then I am going to join the out slice string join and then we will have returned a string a join string add a new line here so we have the words we have the occurrence we return the words, we return the occurrence now we don't have an error anymore because we implement this interface so let's have a look whether this actually works go run words connection refused means that our server is not running let me just start the server on another screen now it is running no response maybe I should give an input word 1 word 2 word 2 ok that works occurrence that also works so now we have written a function do request like a response with a function getResponse or an error and then here in our main the only thing that we do is we check for the arguments and then we do the request based on the argument and if there is an error we are going to print it so if we just hit our main endpoint our index then we get the error unmartial error so now we can actually see where it comes from that was an unmartial error we can even specify what unmartial error it was to make it more clear what error was triggered so this is how we can use functions in go line now what I would explain next in one of the next lectures is then how we do error handling how can we handle different errors that is also going to be interesting because now we have another function but this error is pretty generic how would we be able to handle multiple errors also like I said these interfaces are pretty commonly used in go lang they are pretty powerful when you are returning different types instead of specifying multiple types in your return statement you would rather implement an interface let's talk about error handling so the start situation is HTTP get functions because we just implemented functions and like I said earlier in the previous lecture we do a request but then if there is any error we just say here is the error message but there is not much information wouldn't it be nicer that if we already did a request HTTP status code and a body to also return that in case we have it so then we can actually write our own error handling our own custom error handling with our own custom error type so what is this error actually can we figure this out so in build in there is the error defined the error is just an interface as long as we implement error string we can pass our own variables as an error we just need to make sure that we implement this function let's try to do that and let's then store some extra information our file becomes a bit bigger so I am going to write this in error.go still package main and what you use as a name doesn't really matter I use error but you can use as well not a name it doesn't really matter for goal line so let's write our own custom error type and we will call it request error because if we have a request error then we can supply some more information it's a struct and it's going to have an HTTP code which is an integer maybe a body and an error string and then we still need to implement the error function so that we will be able to pass it where the error interface is used and it was error string because it is returning a string and we will just be returning the error which is of type string now we can return our custom request error where are we going to do that? well once we did the request so if it's not 200 we already know and we return the body so you could already change it here but really where it matters for us now is starting from here what if we get an unmartial error we don't really know what was returned so if we changes into our request error and this will still work for the type error because we implement the function error and request error we expose these three fields so we can fill out those fields HTTP code is going to be the response status code the body is going to be string body and the error is still going to be the error but we can still specify it's an unmartial error but now we have to do asprintf because it's string unmartial error and then the actual text comma this should work and we can copy paste this here and we can copy paste this here we can say occurrences unmartial error so we still know where exactly this happened words unmartial error and page unmartial error save this but now we also need to check here our error well let's first run it without changing anything let's see what happens occurrence but we don't have any words input one word ok words and let's now trigger an error nothing changed because like I said the error will still be printed and the excite information is just not available because we're not using it so how do we start using it well we can test whether our error is of a specific type so is our error of a specific type we can try to change it to a different type and if that is possible then we can print some more information so how do you in golang change a variable to a different type so we can say error error is of type error but we want to be of type request error this will return two variables the request error the variable of the new type and whether it works let's see what this is let's see ok so this is a boolean if the boolean is ok if the changing of the error type to request error if that works then ok will be true and then request error will be a request error and not of type error anymore once we have type request error is we can output some more information we can say htp code was this boolean was this and we can say request error so the first argument can be request error the error itself then request error htp code and then request error body we save this let's run our program again error page and marshal error invalid character t looking for the beginning of value but the htp code was 200 and the body is the server is running let's have a look indeed error code 200 the server is running and what happened here is we tried to un-marshal this as a JSON but the first one is a t and not a curly bracket so that's why we were looking at the beginning of the value of a JSON but we didn't get a JSON so this error is a bit not very nice for end users so if you want what you can do is if json valid body or if it's not valid then we can return our request error and we can say JSON no valid JSON returned we save this try to execute it again no valid JSON returned htp code 200 the body is the server is running so now we see the htp error code the body and that there was no valid JSON so that makes it already a little bit more user friendly because if we going to hit one of these errors now we can already see what the returned body was so again these interfaces are pretty important if you want to customize your errors you are also using the error interface and to then make your own custom error you just have to make sure to implement this error function so it matches with the interface within the struct you can have whatever fields you want and you can populate whatever fields you want in this demo I am going to refactor our arguments a little bit to use the flag package the starting position is our htp get error handling and we are going to build the htp get flags so you are using osrcs to get the command line arguments to parse them ourselves which is not ideal especially when golang has a flag package for it so we can just add to the import a flag package and that should be able to handle the flags that we are going to add the arguments I am going to make this change because in the next lecture I want to add another argument so I want to add another flag that we can not only specify a URL but also a password to make our endpoint password protected so we need to be able to parse a password and still the URL so how does this package work I am going to define some variables you can define variables also with those round brackets and then you can just say request URL is a string I am also going to ask for a password which we are not going to use yet and then we can use a flag package so we are going to say flag dot string var you can use string or string var and the string var accepts a string string pointer so we need to reference this variable the name, the value the default value the usage for the help and then we can parse the flags so request URL it's going to be the first one it's called the URL default value is going to be empty and what else do we need do we need something else so I am going to save this so that the flag is imported and usage so default actually I need string var that's why I was missing one argument here so this is a default and this is usage URL to access and then this request URL will then parse to our request so we need something very important still flag dot parse and this will parse our flags we are going to do the same for password so I am going to say password line use a password to access our API we are not going to use this yet but I just already want to have it in place for our next lecture this we can then move to our main function this doesn't really belong here we are going to check the variables before we pass on to the function this works what I am going to do is use this parse URL because I will need this one later parse URL equals and then I have parse URL of type URL URL still get this error here let's change this in a printf and always exit one and then here instead of request URL I am just going to use the parse URL this is not really a difference but later on I want to use this parse URL so I will stop using this request URL remember if I have this parse URL declared like this then it is only going to be accessible within this conditional so I need to remove this equal sign and I am just going to define them here parse URL returns a URL here is a pointer don't really have to think about it too much it returns a pointer but we can use it in the same way and then our error and then parse URL is not used so let's use it let's just reply the string of the parse URL string we assemble the URL into a valid URL string so basically we parsed it using this parse request URL and then the parse URL sent to the do request and when we are going to have more functions then we will be able to send only the host name or only the scheme for example so let's have a look how this works I am going to save this so this was how I was accessing it earlier the URL is not valid the URL is not valid because I didn't pass the correct parameter now we have to parse it differently so let me give a hint here maybe let's do this URL is not valid, usage HTTP get minus H just to show the end user that minus H will give you the usage so minus H or there is even a better way to do it so let me just show you first minus H minus H then shows the usage but then now that I think of it there is also a way to print the usage flag usage print the usage I don't even think you need the printf let's see yeah this makes more sense validation error URL is not valid then the usage so that we know if we cannot parse the URL there is something wrong we didn't supply URL for example so then it shows the usage so the usage is that you have to pass a dash URL and then the URL so that makes sense no valid JSON returned but if you do rewards rewards input test cell works so this is a nicer way of parsing variables with built in flag function so in these lectures I'm always using built in functions so you don't have to depend on external libraries external packages but there is actually a very good external package available github.com spf13 cobra cobra is one of these replacements for flag that you can use it's a very nice way of parsing arguments other command line utilities already use this cobra so if you're going to parse a lot of parameters it's definitely worse to have look at this package and there are definitely also other packages around that can help you parse flags this flag package it actually does the job for very simple arguments if you just need to capture some strings but for example there is no way to make a string mandatory except to for example parse here like I do because there is no way to say this parameter is optional or this parameter is required at this time and I don't think they would actually implement this in the built in package because they tend to keep it very bare bone very stable so if you are going to work with optional and mandatory parameters have a look at these other packages so that's it for this lecture in the next lecture we will have a look at authentication and that's why we need this password for this next program that I'm going to create let me just go over a diagram to show you what the flow is going to be this is what we did right now we have the hpget client we get the words and the return is a json from the test server and then we just parse a json we can enable authentication on the test server on the right here there needs to be a password set if we start a test server with a password value as a parameter then we'll actually need authentication and to be able to do authentication you will need to provide a jdle at that's a token a json web token it's a token that is used a lot as a temporary token once you are authenticated so we first need to authenticate and then we will get the token to access the test server endpoints again so how does this authentication flow work and it's very similar to other authentication workflows that you will see in real world scenarios that's why I chose this jdle t workflow because it's pretty common to do something like that so our password is xyz and our hpget client needs to first login with this password before it will get a token so we do a post on login with password xyz so we send our password to the server and then the server will send us back a jdle t this token and this token we can then use instead of sending our password with every request we will send this token so this token if you would read it it's a json and it has a certain expiry I think we are working here with expiry of one hour so every one hour we would need to get a new token once we have a token we can then access our endpoints so for example we can get the words with the authorization header it's a bear token and then our jdle t this bear is just a keyword that is often used you can even remove this keyword it should still work because our server will remove this bear token before passing the jdle t but it's not a requirement and then the test server instead of giving back a forbidden error it will give back the json so that's how it works that's how the flow works now we just need to try and program this in our HTTP get client to first do a post request if there is a password supplied get this token and then make sure that every call to the API includes then this HTTP header and then we should get back the words our starting point is the HTTP flag program and from here we will then implement the post request so if the password is not empty password is not equals to empty then we have something in it and we should first do a post request on the login API how do we do a post request well we would have to write at first let's do do post request or do login request it's going to be a login request and we are going to get back a token and that token we will need to use to send as a header when doing the do request API call do login request and we also need to supply the password and instead of the full past URL we are going to send the schema which is HP or HPS depending what is supplied and then the host name past URL hosts and then actually the endpoint is going to be login so we are just going to supply the whole endpoint do login request and let's put this done in a separate file new file login.go package main we are going to do our login request and we have request URL login URL string and then we have what is our second parameter password is also string and we are going to reply a string which is our token and an error so this if you have two times string you can actually remove this this string because then it says login URL and password are both string and we will just put a placeholder we are going to fill this out right away and here I have a token and an error so for the error I could just copy paste this probably it's not 100% nice we could put this in another function probably but for now it will do and then let's print this token for now token and let's just exit for now for now just to see if it works token do login request now we need to see what this API will give us so if we in a second window open the test server let's try to run our test server and test this out so this is our test server and then if you just go to words we didn't get on words but our password was empty password XYZ token is empty okay so we made a request we didn't go through yet because we still because we still need to write the post request and here we also need to supply the password password XYZ and then it will start on port 8080 and then it will enforce the words so here authorization header not set because we have now a password here and we need to then make sure that we first to the post request get the token so let's get started with it let's have a look at this main node go there is actually the login request and login response that I can reuse so the login request and login response is going to be the same for me what we need to do now is to make a post request to the login endpoint so here we have the login URL so we need to do a post request on that URL and we need to pass the password using this login request struct this password needs to be wrapped in JSON so the first step would be to generate this JSON and then the second step would be to do the post request with this JSON so let's make a new variable login request login request equals the type login request and we're immediately going to fill it out the fields that are in it password so this password comes from here and this password is the field name this is the login request the type and this is the new variable name I have chosen so typically you would choose the same variable name as the type just starting with lowercase that's what I typically do then we need to marshal it so we can what we did earlier was to un-marshal a JSON into a struct now we have a struct we need to marshal it into a JSON and that will then be exported as bytes as a byte array JSON Marshall any type our type is login request and it outputs byte and error so we're going to say this is my body that I want to pass error if there is an error then I want to return it I'll just return empty string because we don't have a token un-marshal error marshal error and then the error then we need to use the body in our HTTP post we did HTTP get now we'll do HTTP post HTTP post first the URL then the content type and then the body request URL is our URL or login URL what is it? login URL I'll just call it request URL then it's the same as the other function request URL the content type content type should be application JSON it's the HTTP content type and then the body of IO reader so we can not just pass body because this is an array of bytes cannot use body variable of type byte as reader missing method read so we are missing again this method read but they are functions we can use in Golang to convert our byte array into a variable that implements this read function and there's in the bytes package a function newBuffer newBuffer from a string or newBuffer from a byte array bytes is unacclared I'm just going to save this so that bytes is imported and then we pass this body now bytes newBuffer outputs a bytesBuffer and let's just have a look at this definition bytesBuffer is a buffer and it will implement somewhere here it will implement the read function so you see these are the functions implemented for buffer truncate reset and there should be the read function as well here and this is the function that we implemented earlier you see they also using the copy function but they just try to read as much as possible from our byte that we would pass so that's it for the buffer now we have something that supports a read function post returns a response and error response and error and then from here we can actually copy paste from our do request because it's going to be the same so we have the response and error do a read all status code unmartial copy all this can then change a little bit and we are already using body here I'll just call this rust if there is an error we say an hb post error read all let's save it we don't have errors here and this is imported then we have rust if it's not we can do it we can do that response this is the response this is actually the response body I'll just call it response body and then if our response body is not valid json we can return an error if you want and then instead of page we are going to use this response our login response that's what the end point will reply unmartial the login response instead of nil we are applying an empty string and if that all goes right then we should have login response.token that we should reply so that last return reply is a token and then nil because there's no error and in any way we are just returning the error let's save this let's go if password is not equals to empty then we hit end point and now we are going to print the token let's try to test that let's run our test server let's try without password our tradition header not set let's try with a password and then it should hit this code right here password xyz token is empty token is empty so let's have a look what is happening we hit the login end point we did a post this is already correct we passed our login request application json so this is all correct but is our body correct let's just add another statement if login response token is empty let's then also return error and also return the body empty error right I don't need printf empty token replied save this and then we should go here and then it should show us what our output is empty token replied body password xyz oh this is our body this is not the response body um rest body I didn't replace this let's replace it everywhere response body response body the token was replied but our token is empty that means I'm doing something wrong and here I'm using body instead of response body so basically I am un-martialing my login request instead of un-martialing my response body it's because they have the same type it's a bit annoying if they wouldn't have the same type then it would be more easily visible but we have this extra check so that's easy then to debug save this and then again this is our token back to main so let's now remove those two lines and then we need to pass this token to our do request whenever we do a request right here we need to also supply a header an HTTP header authorization and we need to pass this token you could actually every time you do an HTTP request every time you call get supply this token and every time you have a do request because you could have more functions for do request you would have to remember that you also have to send this token or there's another method instead of always supplying it when doing HTTP get you can also pass an HTTP client to every function and make sure that when any function uses this HTTP client it automatically will send your headers that you need to your endpoint so there's a subtle difference here instead of changing this function we could actually also supply a variable that will replace this HTTP variable that then will be our client let's have a look at this HTTP get HTTP get does a get of the default client the default client and the default client is a client an HTTP client so this client we could also supply with our HTTP get method and then it would not not use this default client with our client and when we use our own HTTP client we can pass parameters so the benefit is that this code stays the same very transparent you just write HTTP get, HTTP post anything that you want but with our own client, with our own HTTP client we can then make sure that our own HTTP client has the information that we need has the headers that we want to pass in a separate part of our code and that's why we define our headers so that's much nicer to program it that way because then we have it nicely separated we have the HTTP call HTTP logic to do the get request but we have the logic that we need for the token summaries so how do we do that, we first need to pass this HTTP client so we can say client is HTTP client and then we say client.get and client.get isn't this client get function so this is a default client that is now our own HTTP client so the difference is quite subtle so if you do HTTP get we actually end up using a different function then client.get client.get is basically the same as default client.get whereas default client is the default client that is defined in this package and here we are not using the default client we are using our own HTTP client so that just leaves us defining our HTTP client we can say here client equals HTTP client and what we can do is we can use for transparency the same client here do login request so whenever we do a request we are going to pass a client so we are not going to use a default client anymore and then here we also need to say client is an HTTP client HTTP.client and instead of HTTP we use client so now we are using our own client here and here and we don't have a token to pass in our first do login request so once we have a token what we are going to do then is we are going to say client to transport we are going to define a transport variable here transport specifies the mechanism by which HP individual HP requests are made if nil which is the default the default transport is used so the default transport will be used if you don't define this but we can define our own transport method and in our transport method we can define how HP requests are made we can say then when we make an HP request we need to also supply our token as an HP header so this needs to be a round stripper we don't have it yet so let's make a new struct in transport.go still package main and let's define this HP round stripper so we are going to call this myJLOT transport myJLOT transport and then we need to define this struct type myJLOT transport struct and we are going to have a token field for our token so we can pass our token here token is token and what do we get we cannot use JLOT transport literal as HP round stripper we are not implementing HP round stripper missing method round trip so this HP round stripper is going to be another interface that expects us to implement the method roundtrip so we will have to implement roundtrip myJLOT transport function we are going to implement roundtrip but what is a signature we would have to look that up let's have a look at this transport so in the client we have transport is of type roundtripper roundtripper is an interface it is roundtrip request and response roundtrip we just have to add htp because now we are outside the package htp request and htp response we are going to give it a variable name curly brackets and then we basically have to rewrite the roundtrip method but I am not very keen on rewriting the whole function I just want to add another header to it to the request and then I want that we are using the default roundtripper rather than writing my own roundtrip so I am going to add the header but then I want to use the default roundtrip so how do I do that let's have a look here in the transport method again so we have the client transport go to definition transport specifies the mechanism by which individual htp requests are made if no default transport is used this default transport this is the one I want to use I think I can just go to the definition ok this is default transport so the default transport is a roundtripper and this is the transport that I want to use because this is a default implementation so how do I do that I am going to define another variable I want to define a transport variable and I want to have it of this htp roundtripper interface that means that I will be able to pass this default transport in my main function which is of type roundtripper and then this transport implements this roundtrip function and then I will be using the default transporter so that's quite some information there transport default transport htp default transport because it's in the package so this htp default transport is of type htp roundtripper let's have a look it's of htp roundtripper and it is using this variable transport this is transport and this transport will then implement as well the roundtrip function so let me just look for this roundtrip function here is the roundtrip function the request response and you see it's a whole logic that does the implementation to do a roundtrip so if I pass in my main.go the default transport of type roundtripper then here I can use mtransport r or let me call rec from request and then I can return this return mtransport roundtrip mtransport refers to here which is my htp default transport and this will be executed and then returned so now even though I implemented my gvld transport it is still not doing anything it's just using the default it's still using this default transport method which would be implemented anyways if our transport was nil so transport if it's nil default transport issue so the only thing that we did is we are using our own struct here my gvld transport but we are passing the default transport here and then we are saying our struct is implementing this roundtrip but actually we are just calling this default transport we are calling this roundtrip function so it executes a single transaction for us what we then still want to do though is we want to inject this header we have the request the request has a header and header has add function we are going to add the authorization and authorization is going to have bear bear token m token and what we can also do is we can say only do this if m token is not equal to is not empty then add please this bear token in the authorization header then do the normal roundtrip that you would normally do so then we set the client transport and then we pass the client using this client here but we only set the transport afterwards so here there is nothing going to happen even though if you would define the transport higher as long as we don't have a token set it will not do anything it will not deviate from the standard behavior so we have a non empty password we change our way of transporting and then we pass the client and here we don't change any code so it's now very done I find that we don't have to change any code but it will still pass our JLT let's try go run URL password it should first hit the login endpoint and then we are going to set the transport which will be my JLT transport add this header bear and then do the request do the request we will do this get request here this will pass the authorization header and then it should just reply words because we are passing the token and we hit the words endpoint with our bear token let's give it an input input word1 response word1 so you see every time we execute this we hit the login endpoint we get a token and we send this token now in reality we only need to get this token once until the token expires so when we execute it twice we don't really have to hit this endpoint this login endpoint we could just we already got and that's how it works in real world scenarios as well if we do multiple requests we only would hit the login endpoint once we get a token and as long as the token is valid then we are going to hit these other endpoints these get endpoints we could save the token somewhere in a file or in a database and then before we do this request we could check whether our JLT is valid and there are a lot of JLT functions that you can check out it's a little bit out of scope of this demo this lecture that I'm doing now so I'm not going to explain it now but you could already have a look in the main function of this server main.go here we are using go line JLT, JLT v4 and we do a parse at some point so when we we have authentication middleware in our server where we check whether the authorization token is valid and we use this JLT parse for that so there are JLT functions in external dependencies within go line that you can use to parse to create those JLT tokens and when we create our token in the login so this is this here is our login function so in our login function we actually create the JLT new with claims and we make it valid as an expiry of 1 hour so this JLT we could use for 1 hour on this server and we wouldn't have to authenticate we wouldn't have to hit this login endpoint every time what we could do here is we could check whether our token is still valid if our token is empty or not valid we could then hit this login endpoint to get a new token that's a typical implementation that's how you could do an implementation to completely make your get request transparent to basically move all the logic here and that's how you would also see in other external packages that you are going to use that's also how it is typically implemented so that's it for this lecture you see again that interfaces are broadly used in Golang and that's creating your own function with some extra logic here and then calling the default function is an approach that is often used within Golang to override some of the default behavior this lecture I'm going to move our code to a separate package so we have the package main that we have been using and we have been importing different go packages but instead of having all the code in our main package I'm going to create another package where we're going to move our code to and we're going to call this new package from our main package that means we will not have a lot of logic anymore in our main function and that's how it should be we should keep the code in the main function to a bare minimum and move everything to reusable packages so we are starting from the HTTP login codebase and now we're going to move this to a separate package so I'm still in this hello world directory which is my working directory and I have these files here arrow go login main transport and what I'm going to do is I'm going to create a directory structure I'm going to remove this binary main which was created by a go build and I'm going to create a directory which you can do here with okay there or a new folder here I'm going to create cmd cmd and then in cmd I'm going to have my binary and my main.go HTTP login package I'm going to call it and then I will have the directory where I'm going to put my package pkg I'll call it pkg api so we have our cmd login package the main.go goes in there and I'll the other files go in here the error login and transport go into the package api then we just need to make sure that we put a complete URL in the go mod so typically what you put as a module is the URL because your URL will be unique so my unique URL will be kitab.com wordfiana call lang for DevOps course HP login packaged so this is where I will push this code so I'm going to use that go 1.18 or any other go version that you are using save this and this is now what we're going to import in our main.go I'm going to close these just to make sure that I open the correct files so main.go first of all I will take these types and we're going to move them so I'm going to make a new file get.go this is now going to be package api so I need to also change this in the other files package api package api and package api and then I'm going to take my main.go move some of these things out of here into the get.go because we only want the main the main we parse the URLs and then do the request and do request can also go in our package here we go and the main.go and what we then want to do is we want to initialize this package and do it in different ways first of all you will need to import it so it will need to be import here github.com and then package api is going to be api now if I save this we are not using it anywhere so visual studio code will remove it so we will have to use it somewhere before we save where we do api. and then we can use some of these exported variables or functions so you see only the one that starts with a capital letter will be exported the variables or functions that don't start with a capital letter will not be exported are not accessible which is the same as private and public if you are familiar with Java or other languages that use the private and public keywords here whether a function or variable is exposed outside the package is whether it has a capital first letter or a lower case now there are multiple ways to use a package one of my favorite ways to do it is to use a new function and when we use this new function then we can return an interface with the correct functions now there are different ways you could also call them directly the functions but with this new keyword that you then return the interface it actually helps you later on if you want to do testing so when we want to write tests this is going to help us so that's why I'm going to use this approach api new so in api I will create an init.go or you can call it whatever you want where it's still going to be package api package api where I'm going to write this new function and this new function will then return an interface and it's going to be my api interface and I will add the word interface just so I can recognize it in my own code I like to be able to recognize interfaces by name but you are free to do whatever you want so because it starts with a it will be exposed type api interface is an interface and then I will return this and this will have my functions that I can then use in my main.go but then I need to return a variable that implements these api interfaces so I'll write my type api of type struct and here again you can choose what you're going to use lowercase or uppercase if you use uppercase then you can initialize this variable from outside the package if I use uppercase like this api or api then someone can initialize this variable directly again it depends what you want I often do it with the uppercase so if someone wants to access it directly because it wants to circumvent the new then it's possible but if you really want maybe for yourself have a package that you only can initialize using the new function you should use lowercase but I'm going to use uppercase for now because that's what I typically do some people say also that if you write a package and you don't expose them it can be a bit harder if you want to implement something customized and then you have to make a copy of the library and then make some changes to it so in general uppercase and exporting is a good idea api struct and what do I need to pass? well I'm going to need the password I'm going to need the login URL and I'm also going to need the htp client so to make it easy I'm going to make another type the options type it's going to be struct I'm going to have two options password and the login URL is also string because I need to know this login URL in the end in my package so that I know where to login then my api accepts options which is of type options and then the client is going to be the htp client I might change the latitude interface when we are going to do the testing but I will leave it for now here so I can keep the explanation for later so if you're going to do a new I'm going to ask for the options and then I'm going to return a new api this new api will have to pass options so the options this is what I need to pass in the main function and also I will have to initialize my htp client so I will say my client is of htp client and this htp client will also have transport remember we need this special, we need this myjlt transport here and then immediately what I want to do is I want to pass the transport and I also want to pass the password and login URL because we will need that so transport is going to be htp default transport what I want to do is I want to move within this transport code also this code so if password is not equal to empty then I want to do this login request so I will move this information that I need these two parameters so that this bit can also go out of the main function I don't want to have any logic whatsoever in this main function except just passing my parameters and then executing my my request so password but then let's also change it here I'm going to ask for password which is a string and login URL which also a string save this and then the password is options of password and the login URL is the options login URL so now I have all the information being passed to my transport so let's try to work on this first so that this code can go away let's go to transport and then I want to add some code here so if the token is empty and the password is not empty then I want to get this token and if I get an error I want to return an error I don't want to exit the program I just want to return an error and then it will be captured in the main function to be captured here so I'm going to say if error is not equal to nil return nil and then the error and if I have a token then I'm going to say I'm token equals token oh and then password should be mpassword because we supplied it here here we've supplied the password so when we then don't have a token but we have a password we're going to do the login request and then we're going to add the header if we have a token but we only want to execute this when we do a get request or when we do the when we do the do request here so let's not pass this client here to do a login request let's pass a new client just so that we are not sending our own that would be with weird just in case that we would do a login request and then we would do another login request because we would be in a loop done so just to make sure that we are not going to make any loops let's do another client and then the login information is now going to be from the login URL and the password is going to be also supplied when we init it to this function so then we are good here I think so let's go back to the main.go let's try to write our api new because then I can show you the whole flow we have an api instance api.new api options these are options right here so we can supply a password and a login URL password is our password login URL is going to be our login URL that we just removed but I still copy pasted it from my previous project so it is right here the parsed URL scheme parsed URL host and the login so now we have api instance and now I can do requests using this api instance api instance declared but not used and it's of api interface so let's build this interface so that we can do a call of this do request so this do request now needs to be part of this api so this api struct is what we return here we return this api here so this api needs to get the function needs to get the function do request so we can say api is now do request I will call it do cat request just to make it more clear we don't need a client anymore because the client is now in this api we have this api here, we have the hp client here that we can refer to api do request, we don't need the client but we still would need the request URL and then this then becomes a. client.cat because our client is now our hp client within this struct this is our function signature that we want to expose I'm going to my api interface that we return here my api interface is now do cat request this is not going to work because it's lowercase so it's not exposed so I'm going to expose it by making it an uppercase going to save this going to save this and this and then now I should be able to change this code do request so api interface I pass api options I get a new api interface api interface has a do request I return here my api which implements this do request right here api do request do cat request so I should be able to do the cat request the client that don't need anymore because now we initialize here the client we could still let the user supply the client if you want but we are not we are just defining it so it's easier client is the hp client and the transport is myjblt transport and that's why we need to pass this information to myjblt transport as well api instance api instance.do cat request now we don't need the client anymore because we initiated in the new function and then this still gives us an error because we moved this to the api package save this do we still have errors I don't think so so with it the new this replies the api interface the api interface has one function do cat request do cat request gets executed it uses the a client which is here this is the client that we defined with the transport the transport is here transport will be initialized token will be empty but the password will be set we do the login request with a separate client so we don't execute this roundtrip code twice with the login URL with the password and then if the token is not empty so if we retrieved here the token then we can add the authorization header and we execute the roundtrip code of the default transport we get an error somewhere here then we will return an error if we return an error then the connection will stop and it will display the error so this looks all pretty good so what do we have left in our main function so here we import a package we ask the arguments that we want we parse them we check whether it's a URL if it is a URL we parse the options password can still be empty login URL could still be unused we don't have a password protected API then we have the API instance API instance here we do a get request because this is the interface and then we check whether it's of API request error because then we can show a better error code if we have result we're going to print result so we don't have much code left in our main the idea is to have as less code as possible in our main function we can still have some parsing of the parameters and some error handling but that should really be it and even if you want we can move this to a separate package if we think this is going to be reusable and then we only have basically one line in our main.go let's start to execute it now the test server go run go password abc and then now we cannot go run go anymore because there's no go file anymore it's in the cmd directory it's in the hp login package so now we can actually also have multiple commands if you want we just have to make another directory and we can have multiple main packages if you would have multiple executables go run cmd hp login package main.go minus h it still works url localhost 8080 word abc and that works input equals xyz we have a word xyz and you see here's our first post request here's our second post request so the post request is actually working if you want to do a get request then our run trip gets executed we'll do the login request and here and here in the main get request so for the developer that didn't write this package it only sees this new function and if you wanted we could make all the other ones invisible so that it's very clear what function we can use and that was what I was saying if we export this API we will still see this API and we can define this API we can say new API equals API API and then we can pass these options and client ourselves so if you want for example to override this client ourselves we could do that but if we would make it a lower case in init.go let's make it lower case API and now it says undeclared and now it still says we are missing the do get request because we need to also make this lower case then API. we don't see the API anymore we see API interface so now we can only get the API interface the only function we see is a new function and we can still even make this login request and this login response lower case so we don't see them appearing when we are checking this API it can still make sense so we have a request error to have this one exposed because we might want to use it right here to do our own error parsing with other ones we could make them unexported so we don't see them it all depends on how you want to make this package like I said sometimes it might be useful to export your API just in case that you want to pass your own variables like the HTTP client so that's it different directory structure this is kind of what I like to use but it's all up to you in the end what kind of directory structure that you might want to use this one with CMD you will often see in other project as well and this PKG you can also often see there's no real rule you are free to use what you like so what is next I would say is we need to write some tests for our package so typically you would have minimal tests for your main function but you have much evolved tests for your packages because these are reusable and you want to make sure that these work as intended after I published these lectures about the transport a student that enrolled in this course made me aware that there's actually a mistake in this transport.go that needs correction so because we are invoking this HTTP login only once per execution so we execute it and then we only do one API call we always do the login request but what if we would do two API calls then this ifMToken equals to nothing should work because we are setting the MToken to the token if we do the login request and actually it doesn't and the reason why it doesn't and I can show it to you in a second is because I wanted to keep it simple by not using pointers so we are not using a pointer here to the struct we are not passing it as a pointer which means we can change these variables but because we are not passing it as a pointer it will not save it in the next call so let me just show you how that works in practice if I just take this do request here and I'm just going to do it twice so I'm going to do two requests on the words API then we see post on login then the words and then another post on login which shouldn't happen if we store our token and that's because we are not passing this struct in the init.go as a pointer so transport when we do a new we should pass this myjlt transport as a reference here this is still the HP round tripper and here this function is going to be a pointer so now if I change this token I change this token in my struct is going to be still available the next call because otherwise I will always have a copy of this struct so the next call is still going to have an empty token it's going to be initialized again it's going to be a new myjlt transport instead of the same one so let's try it out now and now I have the post login and two times I'm hitting the slash words endpoint because now I'm saving this token and the next call we are still using the same myjlt transport because now we are sending it as a pointer not just as a copy and I can make changes so I try to avoid using pointers not to have to explain your pointers because I'm explaining them later but here it was actually necessary to introduce pointers already because we are making a change to a variable in a struct and if we are making a change and we want to keep the change we have to use a pointer here we have to make sure that we are working with a pointer to our struct and not just a copy of our struct every time we hit this endpoint and I never really did a second api call otherwise I would have realized in the previous demo so if you want to make sure that this works as intended make sure you change it into a pointer and I will also have it updated in github we are at the HTTP login package this is our starting situation we are going to build HTTP login tests so we are going to add tests to our goal and code so we can make sure that we know that our code is correct tests are a bit difficult a little bit more difficult to write when we are using api calls you are connecting to a server so we will have to kind of intercept those connections and make sure that we populate the correct variables with information that you would get if you would make a real connection and test our functions so let's start in the get call we have the do request that makes an HTTP request and then has some logic in it that we preferably would like to test so how do you make a test in goal line we are going to make a new file get underscore test.go and in here you can already choose whether you are going to stay in the same package or whether you are going to create a new package api underscore test to make your tests as if you are outside the package which one you pick is going to influence your exported variables so if you want to use unexported variables we typically are going to use the same package get go has the do get request function so we are going to write a new function test do get request and the parameter is going to be t we are going to be able to use a t parameter it's of type testing t so there is a testing framework a testing package in goal line so we are going to import testing and this is going to help us write tests if you are going to incur an error instead of returning fmt error f or something like that we are going to do the t error f and this will throw an error if you then want to stop or go outside a function or use any of the other t functions so there is a fail or fail now there is also a clean up if you need to do a clean up there is a fatal and a fatal f so this testing is again very bare bone but it's enough to write the tests that we want if you are used to write tests in other languages then you will probably miss the assertation functions but there are other packages that you can use for assertation assertation is just comparing variables which you can do manually as well so we are going to only use the testing package how do we test it well best way to do is we are going to do we are going to call this function and then when we call this function with a specific URL we are going to test whether the response is equal to what we like so let's try that we do get request but to be able to do the get request we need the API struct so let's just apply a dummy URL it can be anything because we are not really going to call any endpoint but this is going to be API instance do get request so we need to define this API instance and this API instance is going to be of type API so this is what I was talking about we are going to use the new of the init function the new which gives us this API interface which allows us to do the get request but then we will not be able to set the clients we only allow to set the options so either we create a second new function which allows us also to set this client or we just initiate directly this struct and we pass options in the client that you want which I am going to do now options is options and client is our client but if I now use HTTP client what is going to happen is the do get request will actually make a real request to local host and that is what we want to avoid we want to create a fake client so that we can intercept these requests these get requests and return our own response so that we can still check the logic of our program without the need for a server so how do we do that? client is of type HTTP client client is of type HTTP client like here but we are only using the get function so if we have a look at the get function let me just make sure I have get and init open so the get function of the client a client get is and then we have the get function so let's create an interface type client interface which is of type interface and this client interface is going to implement the get function so now we can say the API is not of type HTTP client the client in the API is of type client interface now we still get an error here we cannot use HTTP client literal as client interface value in struct literal HTTP client does not implement client interface method get has a pointer receiver so HTTP client has this struct and let's see how if I go to this get function it's already of client interface okay let me just for one second just to show you why we are getting this error if I go to the definition you see the client is a pointer because it is a pointer we will need to make sure that we also initiate our client as a pointer because it is defined as function pointer to client and not just client so let's undo this again we have client interface but we get an error here now HTTP client does not implement the get function as a pointer receiver so let's just make a reference to this and this actually works because now it's a reference to client and it will hit the correct function in this struct so now we have implement get the client interface and now this client is of client interface and what we can do now is instead of passing an HTTP client we can pass our own client so let's make our own client type mock client is a struct and we have a function within this mock client of client interface so this one mock client has a get function we can reply something we can reply a response and the easiest is going to be that I can within my function reply any custom response so I'm going to say within the struct response output is of HTTP response and then when I hit this get function I'm going to return this response output and no error so then instead of HTTP client I can then pass this mock client so here you can also use a pointer if you want so here you can use a pointer or no pointer if you're using a pointer here then you need to reference this variable like this if you don't use a pointer then it's like this mock client and then in mock client you can pass this response output this response output will then be returned if do get request hits the get endpoint of the client response output is of type HTTP response and now we can reply a response what is in the response the status code replies this code to 200 and a body what body would we reply if we do the do get request let's take the same body that we would reply in the real world situation so that if we do here the get we then also have the same JSON that we would get and then we would check on the status code we should check we would check on whether it's valid we would un-martial it first the page and then the words so let's create our own JSON that we have exactly the same JSON as our server would return so that we can run our test and a test will then go over all this code to do this we need another struct because we need one with the page and the words together because either we send the page as a JSON or either we send the input and the words as a JSON but we actually want to send the page input and words so I'm just going to make a new type struct and instead of redefining everything I'm just going to say this first part of this struct is going to be page and the second part of this struct is going to be words I'm going to save this and then I can have here this is going to be the output of my get request words is words page the page is going to be of variable page and it's going to be words and then we're going to have words and that is of type words and then we have an input our input is abc and our words is a string a mb so we can choose we just need to make sure that we then check at the end where this was correctly then in this function get request function so now we have this in a struct but we need to be able to pass the response and response has a body and what is the body expecting an I or read closer so this is going to be bytes and this is going to be going struct so we still need to convert it to JSON let's do that first words and bytes error JSON Marshall all these words if there's an error we're going to return a Marshall error so now we have the words bytes and we can reply this but body is expecting a read closer the response body is streamed on the mant and then we have this read closer so it's not a reader it's a read closer so we need another function to be able to convert from an I or reader to a read closer and to find it you can have a look at how these libraries are written or you can write this question in a search engine to get some clues and that's how I do it typically I type these things in Google and then I find some answers on how to return a read closer so in case here to return a read closer you need to use the I or which knob closer returns a read closer with a no up close method wrapping to provide a reader so once we have an I or reader then we can wrap this in a knob closer and when the stream needs to be closed it will just close it and there's no code that's going to be executed and then we need the reader bytes new reader words bytes we're just going to have this JSON as a body that works and then we can do the do get request response error what if there is an error then we can say do get request error and then the response should be of type response if response is nil you just need to check that otherwise then we just need to stop so we're going to do fatal f so fatal f equivalent to log f followed by fail now because if it's nil we don't want to execute anything anymore we're going to say response this error is empty if it is not empty then we can check the response if response get response is and now we want to check if it's not equal to what we expect we need to throw an error what is it going to be so do request let's have a look how our code is written if we got a word then we reply the words and the words implement a get response and we just print it is a join so we can say if response is just copy this code response get response is not equal to the strings we're going to join this string we can also make a variable of that so you can either use a function or just say a comma b it kind of depends how you like it most then we're going to say if it's not the same then we're going to return error expected unexpected output expected response and then we're going to say okay the response was this not sure if this will already work but let's test let's first run the test and if this works let's run the debugging okay our test passed let's go to news let's now run debugging just to be sure so I'm going to put this break point here debug test let's go in this function do request we have request URL so request URL is not really used and then our client it should be the mock client so this is the mock client and then it hits our mock client function and then we're going to send the output so this is our output here of our reader and here are our bytes and then we're going to close it at the end so that's why we need this read closer because we also need to be able to close it we're going to read everything from the body so now we have the body page words input ABC words you see so this is what we returned and now we can run this function with our test data it's going to test where it's a page where it's a word and then it should return words and then the words has this get response and the get response should then be equal to our join string and this is how we build a simple test function for an API call now that we wrote a test for our do request we also want to write a test for a roundtrip because our roundtrip also has some logic in it the do login request is going to do a post request and we also need to test that so let's create another file transport test.go and here we will then write our test roundtrip and here it's the same what do we want to do we want to test this function so we need first to initialize this myjlt transport myjlt transport and then myjlt transport roundtrip roundtrip accepts a request so we want to initialize request as well so type hp request and what does it return a response and an error if the error is not nil fatal roundtrip error and then the explanation res we're going to test on res let's just make one simple test if res is a HTTP response so if res status code is not 200 we're going to say that we have an error status code is not 200 got and then the status code but it will be mainly here that we want to make sure that our roundtrip doesn't return an error so what happens if you already test this what would happen is we would go to this roundtrip function to this roundtrip function and then actually nothing will happen will not execute this because we don't have a password set yet and then we're going to return mtransport roundtrip so we're just going to fail because we don't have a roundtrip set so we have invalid memory address at this point because we didn't set mtransport mtransport is nil and then we are trying to invoke roundtrip on nil which is not going to work so we need to define it transport but what would we define as our transport default transport is of type roundtripper so we're just going to do the actual request so this is what we don't want so we can also write a mock roundtripper just in the same way that we wrote our mock client so type mock roundtripper to struct and then we're going to implement the same function here roundtrip request and then this default roundtrip will then be our mock function and again we're going to return a specific response roundtripper output type hspresponse and we're going to return it here at no error and then here we can say we're going to use our mock roundtripper and our roundtrip output will then be an hspresponse and let's just only reply the state's code to 100 because that's what we're checking here for let's try to run this again and our test succeeded but what did we really execute now roundtrip no token is set we don't add a header so then we just return our mock function but we really want to test this code so we will need to add a token and then when we do the login request we are passing hsp client again which we don't want so let's change this as well into an hsp client here just like we did earlier hsp client, hsp now we have a client interface so if we go here we have a client interface that we can use and then we'll be able to pass still our client but we want to pass our client in the init and not here so hsp is of hsp client interface and then we're going to pass this hsp client and then here we'll have the init slices so in the myjblt transport we'll have the hsp client which is of hsp client which is a new one it's a different one than this because we don't want to when we do the post we don't want to inject the header this needs to be a reference so this is all good this doesn't work yet because the do login request has an hsp client and this we also need to change into our interface but here we are using the posts so I just copy pastes first client is of client interface but the post doesn't work anymore why doesn't the post work anymore because our client interface only implements get so I just add post here and then we have post that works so these signatures in the interface always need to match and then we can always pass our hsp client but then in the test we'll also be able to pass our our own hsp client so that will be here next to transport we have hsp client and let's just pass our mock client we can make another mock client but let's just reuse the same one that we have for now just to keep it simple we could actually create a mock client for every file for every test file so this mock client doesn't implement the post method so let's implement the post method here we have func we wrote a get method not a post method let's use the same strategy here response output post response output and let's reply the post response output and this should be get response output and here we get response output so this we can copy paste and we're going to make it post request output post request output so what happens if you do a post we are going to reply the token the token so this will be of login response is there anything that we can copy paste login response login response and then we're going to have a token token is going to be ABC and we just need to marshal this login response bytes error json marshal login response if there is an error marshal error and then here IO closer reader login response bind so this is if you do a post request this is what's going to be replied the token is this going to work let's see okay this works and I assume that we went into our into the if no we didn't go into the if because I didn't supply a password password XYZ let's execute it again and now we get an error m token no panic assignment to entry in nil map so we have the header which is a map but it's nil that happens sometimes because we just initialize a new HP request but that's not in it let's try to define this header which is a new map so if a map is nil we cannot assign anything to the map so we need to initialize the header the header is a make HP header and HP header is a map of a string and a string so if you use the make keyword it's going to make this type this header type and then we can use this map let's try this again okay it works let's debug this let's go in it token is empty password is not empty two login requests token is ABC so that worked now we add the header and then the roundtrip is finished no error here states code 200 there's just one thing that bothers me is that we're not really checking whether we have the header how would we check for that if we are doing this other roundtrip in transport here we're doing this roundtrip we should have this header so we can test for that and we can test for that in our mock roundtripper if rack header because we had a request here that is being passed and the request if the token is not empty then we added a header with bear and then token so if the header the authorization header is not equals to bear one was it 123 ABC then we can return an error return nil an error is going to be wrong authorization header and we can output the authorization header as well save this this is going to be error F save this let's put a breaking point let's continue and our authorization is equals to bear ABC otherwise we would have gotten the error that seems to work what actually happens if we check for ABC Z run test okay wrong authorization header so this actually works so this is the nice thing about these tests that we can write our tests in a way as we are using this mock roundtripper and that we still have checks in this fake function to make sure that we are passing the correct data when we need to do the default roundtrip so this is it if we do now run package test for example then it will run the test on our full package and we now already have 60% of our statements covered you see so everything in green is already covered and so now you could actually write more tests to come close to 100 100% is not always possible and I wouldn't really aim for 100% somewhere between 80% and 100% is definitely fine in this lecture I'm going to give you a little bit more information about pointers I have used pointers quite a lot in the previous demos but I didn't really explain in detail what they mean because in go also you can go quite away without really having to understand the intricacies but then at some point it will bite you and you will spend quite some time debugging your code, figuring out why a certain variable doesn't want to update and when you hit a problem like that it most likely has to do because of pointers so I'm going to give you a few examples here to show you how pointers work let's start by declaring a simple type it can be a string, an int, a float I'm going to call it a, a is a string and then when you pass a string to a test pointer function then this will be passed as a value so it's a copy of the string what does it mean a copy of the string so a function test pointer a string when we access a here in this test pointer function it's going to be a copy of a of the variable a I say a equals another string and I output right here in my main function then what will I see string so I have string, I change it in the function test pointer to another string but because a is a copy it didn't change so when you pass a variable to a function it will always be passed as a copy and if you make a change within the function it will not change it just the copy will change will go outside the function and then you are using again the variable a that you declared in your main function which didn't change if you wanted to change you can just say I'm going to pass as a pointer my variable a and then you also have to use the m% sign here to show that you are sending the address of a not just the variable a and then here if you want to change a you also need to put a star because now it's a pointer what if I execute it now and now I get another string so what is this pointer a we are outputting here just a which is the string but what if you would put an m% here so that we are outputting the reference and if it changes as in a v then it shows us an address so basically instead of passing the string we are here passing an address and then when we change the value of that string at that address then we can actually make changes to this string in general when you are passing something to a function if you don't need to change it you can just send it as a copy so not as a pointer just like this only when it makes sense if you have variables with a lot of contents in it like a big file or something like that and you're not using the streaming the buffers that we are using then you could send it as a pointer but in general for small strings like that the best way is to avoid pointers and just send it as a copy if you don't need to change it there aren't some special cases let's say our a is a slice so this is how we can declare a slice let's say this is a string and in this string slice we have one element called string then we just need to accept a slice here and the first element is another string so we are just passing I'll change this in V so if you see the output of a slice so we have a is now a string slice we pass it without a pointer and then we're going to change it and what do we expect now it actually changed and this is because slices and maps and also channels and channels is something that we haven't explained yet are actually behaving as if you send them as a pointer so if you want to change elements within this slice that works without having to pass them as a pointer but what happens if you want to append it and that is an interesting case let's say we append another string to the string slice that has been passed and now we get back the original string so the append didn't work we could actually change the first element changed element we could change it but we cannot append and that is because if we append we assign a new slice to a which will not work in the function if you want to do things like that you will have to either return a string and then say a equals test pointer a and then return a and then it works or and or you pass it as a pointer so you can also pass it as a pointer and then we just need to change the notation again save it and then we have string and then another string so this also works but in general if you have to create another slice anyways you could as well just return it that's what I typically do so this is the same with a map so if you do make map string string and you have tests is the value so let's try that out map string string a test is new value let's output it test new value that works let's try to add another value and because maps are different than slices you can actually assign another key and that will work within the function so here you don't need to work with a map pointer so you see that depending on the type that you use the behavior of copy by value and copy by reference whereas copy by reference is just passing it as a pointer is actually different depending on the type so in general what I would recommend is to definitely write tests if you are unsure how you are passing it just to make sure that you are actually changing the value of something that you have passed whether it also really changed in the function that you are calling it and like our main function you can output it or you can write tests but you have to make sure that the value actually changed it can be a bit tricky but you quickly get used to it let's talk about differences between arrays and slices the biggest difference between arrays and slices is that arrays have a fixed length whereas slices are dynamic let's have a look how you declare an array versus a slice in an array we would declare it as var buffer and then we would specify the size 7 elements in this case and then the type byte so this array can only carry 7 elements if you try to assign something to the 8th element then you will get an error a slice on the other hand doesn't specify the length because it is dynamic so if you declare var buffer with the byte type and don't specify the length then it is a slice and then it will dynamically allocate the size that it needs still arrays are important because arrays are the building block of slices and let me show you that with a simple example here we have an array var array 1 length 7 of type integer in this array 1 we are immediately going to put an array integer of length 7 with the values 7, 3, 6, 0, 4, 9 and 10 so now we will have this array of length 7 and capacity 7 so an array always has the same length and the capacity because we specify it has 7 elements in it so it can only hold 7 elements these 7 elements are being held in memory and contain the values 7, 3, 6, 0, 4, 9, 10 so this is an array we can change these elements in this array but we cannot expand it if you want to expand it we just have to create a new array and copy the data over because that would be quite cumbersome as developer to deal with we have slices and slices like I said earlier you don't have to specify the length let's say we have var array 2 which is of type integer here you can see that we don't specify a size with our type so we have a slice in this specific case if you then have a look what is after the equal sign we are assigning to this new array 2 a part of array 1 so we are using these square brackets here to specify a range we are starting from element 1 which is the lower bound to element 3 which is the upper bound so the lower bound element 1 is 3 then the second number is 6 and that is it because the upper bound is 3 but is excluding the number of the upper bound which is not included so then we get a new slice based on our first array with just 3 and 6 so we basically took the number 3 and 6 from the first array and now this is array 2 but because this is a slice this slice is actually referring still to this array 1 we didn't create a copy this slice is a reference to our array 1 and that is why the capacity of our slice is 6 the length of our slice is 2 and the capacity is 6 this is because the underlying array has this capacity so 6 is 7-1 the original capacity of this array was 7 elements but now this is only a slice and this slice has capacity 6 so using this slice length of 2 we cannot increase the capacity to the left side where 7 is we only can increase it to the right side if we would print this slice it would still give us 3 and 6 but there is additional capacity available in this slice so a slice has 3 elements of information that Golang is internally keeping for this slice it's a length of 2 the capacity is capacity of 6 and it also refers to the element 0 what is the first element that I need to start at and this element is array 1 and the first element the first element is 3 so we start counting from 0 element 0 is 7 and element 1 is 3 so this is our element 0 of our slice we didn't go what the elements are within our slice we need to know this element 0 which is 3, length of 2 so we have 3 and 6 capacity 6 we actually have 3 capacity within this slice because we are referring to this underlying array this is an example where we refer to another array but we could as well initialize a new slice as well and then the underlying array will be empty it will have a certain length and capacity and if we add elements and there is more capacity available then the underlying array can be reused we can just increase the length until we hit the capacity if we don't have enough capacity we have to allocate a new array copy over the elements and that is exactly what the append function does so let's have a look with some code examples how this really works so here I have my example the same array that I was using in the previous lecture this is array 1 with a length of 7 and it has 7 elements in it so let's print this first to start with how do we print it can just run it and have fmt print line print the array so we just get the output of this array but what is the length and what is the capacity so let's do another print f and then we can use len array 1 and cap for capacity array 1 save this so this is the output and the length and the capacity is 7 so this is the same as in our previous lecture now let's declare array 2 which is a slice of integer and this is equal to array 1 from 1 to 3 and let's now copy this println and printf change array 1 in array 2 save this and let's have a look now what is the output array 1 capacity and length is the same and this is our slice 3 and 6 length 2 but capacity 6 so what does that even mean our capacity is 6 it means that we could if we wanted it doesn't really make a lot of sense but if you wanted we could increase our length because we have more capacity so I'll do that it doesn't really make a lot of sense to do it so how slices working go so if I say array 2 is array 2 from 0 to the length of array 1 but I want to increase it with 2 then I can do this plus 2 so the length is 2 and now length is going to be 4 if I do plus 2 going to save it go run again actually what I wanted is I want to increase from length 2 to length 4 let me just save this so you see if you increase it with too much then you already get an error so if I increase it with 2 then I see those 2 elements here because now I have a length of 4 so instead of length 2 where I only saw 3 and 6 of this slice I now also see the 0 and 4 because 0 and 4 are here and you see that is because the array is still the underlying array so what would happen if you would loop this slice so for k range array 2 array 2k which is every element in that we are going to loop plus 1 what would happen then and that's an interesting one because it will increase the elements in the slice but what's going to happen to our array 1 if you print also array 1 then you can see array 1 also changed because the underlying array of this slice is still array 1 so 7 we don't use 3 from here became 4 6 became 7 0 became 1 4 became 5 and 9 and 10 stay the same because we only have those 4 elements in our slice so you see because we are still using this array 1 as our underlying array of our slice if we change elements in our slice it actually also changes in the array this proves that a slice refers to an array and doesn't really make a copy of an array so what if we would like to initialize a slice can we determine the capacity so let's start with array 3 array 3 is another slice what is the length and capacity of our slice so it's empty it's an empty slice and the length and capacity is 0 what if we initialize 3 elements we initialize 3 elements 1, 2, 3 our length is 3 and our capacity is 3 so if we want to add another element to our slice we would then use the append function and then we would have to say array 3 is append array 3 and let's append 4 and you see that append will actually create a new slice and assign this to array 3 so 1, 2, 3 was our slice of capacity 3 of length 3 capacity 3 and now we add 4 and now you can see we have length 4 but because we did append append is also making sure that we have some extra capacity so if we have to add another element element 5 we see that our length increased but our capacity didn't increase so append if you have a look at documentation if the slice has sufficient capacity the destination will be re-slized to accommodate the new elements if it does not a new underlying array will be allocated so depending on the fact that we have enough capacity within this underlying array it will reallocate an array or not append then returns the updated slice is therefore necessary to store the result of the append that's why we have to store this result because the updated slice is what we need to store the underlying array might be different or might not be different it's only going to be different if there is no capacity so how do we create an array with our own capacity with our own length we can use make for that so array 4 can be make int and then the size so in 3 we can then again output it clear this, run this again so now we have array 4 and array 4 is 000 length 3 capacity 3 but let's say that we want capacity 9 we can actually add another argument here so the slice size specifies a length and the capacity of the slice is equal to its length a second integer argument may be provided to specify a different capacity not smaller than the length and it allocates an underlying array of this size so we can say size 3 capacity 9 size 3 and then when we do an append it doesn't have to reallocate a new array as long as we stay within this capacity 9 so this gives you a good overview of the differences between arrays and slices in general we're not going to have a lot of exposure to arrays but more to slices because as a developer you just going to use go in a way that you don't want to have to deal with the static size of an array you're just going to use slices and when it needs more capacity then the underlying array will then be updated so to fully understand this I would recommend you to try this out yourself and change the length and capacity a bit to see yourself how this works in this and in the next few lectures I want to talk about types in go I am in the types demo directory and I am going to create a few small programs to explain a bit more about types in go so let's start with the type switch it's a switch statement that we can use to determine what type a variable has so here we have our main.go with an empty main function and I am going to write a new function the function is going to be called discover type and I am going to have a variable t that I don't know the type of and if I want to accept any type in a go function I can use the any keyword and the any keyword is actually quite new it is replacement for interface so in previous go versions you could use interface and you can still use interface but now it's not encouraged anymore, you should use any so interface accepts anything but that has been replaced by the any keyword any is an alias for interface and is equivalent to interfacing always so once you accept any variable you don't know whether it's an integer or a float or a string but you can test for that depending on what type you discover you could take certain actions in your function so to do that we can use the type switch and we can use t which is a variable and then we can use the keyword type and then we are going to compare the different types in our discover type function so we can say if it is a string then we say string font and the string is then t and then now I'm going to save this and let's pass a string of our t1 is a string and then I'm going to discover type of t1 and I'm going to run this type switch main.go string font this is a string if I want to use this variable and I'm going to say t2 equals t plus and three dots this will not work invalid operation t mismatch types any and string so you cannot catenate a string to something that you don't know that is a string you can still use printf with it because printf accepts any and will then determine itself what it is so if you say string font s then printf knows that it's a string that can output something printf will try to test itself to see how it can output that variable so you can pass any variable to printf but you cannot use this variable of type any in any operation where you would need to know whether it's a string or not so you can either change the type of t so you can say t string because now we know it's a string is a string and then we can output it so this should work this is a string but you can also say value equals t type and then the value will be of the type that it recognized so if we have k string then v will be also a string so if you check here v is now a string and then we can use v v s the variable where we want to concatenate something so then this is a typical notation that you would use t type is what you want investigate what is the type of t but you also want to use the value so what we use a pointer string then we can say pointer string found and what will happen here s has the wrong argument so we cannot really output a string based just on a pointer because a pointer is a reference so let's see what happens if we pass a pointer string so that means if we pass a pointer string it's a different type and we need to have a case for the different type so now t2 is a string pointer and we are passing a reference to t1 pointer string found and then we get a reference to the pointer so what if you would use s pointer string found but we cannot really output it because it's not a string if you want to output the actual string we would have to put a star before the v to get the actual value and not the reference so I do go around now then this is a string and I get the correct output what if I have an int an integer integer is 1, 2, 3 and I didn't specify the case so pointer string found but the second time nothing was outputted because we don't have a case int we can make a catch how we can say default default is fmt print f type not found type not found just gonna add this enter type not found but what type is not found and if you want to display the type just in case that if we get an error we would like to know the type then there is a reflect package in gold that we can use so we can say time not found and then reflect type off and then the t which is of any so you see it takes any and then it returns a reflect type pointer string found it was a first discovered type and the second one type not found int so with reflect with this reflect package and the type off function we can then know what the type is so case int fmt print f we have an integer we have an integer what if we passed discovered type nil so if we use any or we use a pointer we can also pass nil nil just means that there is nothing in it it is a special type so we should also be able to figure out that type not found nil type off returns the reflection type that represents the dynamic type of i if i is nil type off returns nil so we can say my type is reflect if my type is nil then we say nil type is nil and otherwise we can say type not found my type and then we should get a nicer output type is nil if there is a function that returns this any keyword then you can use this type switch to figure out what the type is of a variable and depending on that you can take certain actions this was used quite a lot and then since go1.18 go came out with generics and generics i will explain in the next lecture it is a mechanism to accept multiple types within a function so instead of saying i accept any type and i will then start checking for any possible type you can also say i am only going to accept integers and floats for example so integers are numbers and floats can be decimal numbers so you accept two types and still within your function you can work with that type which makes it a lot easier to write code where you don't always know the variable this is one way of doing it and in the next lecture i will show you how to do it with generics i will now show you how to use generics so instead of accepting any type we are now going to accept concrete types so specifically an integer and a float for example so i am going to close this and then i am going to go to the generics main.go and i am going to write a plus 1 function so we have an integer t1 integer is 1, 2, 3 and i am going to do a plus 1 so how would i normally write this plus 1 t is an integer and i return an integer and i return t plus 1 and then i can output plus 1 is a decimal of plus 1 t1 so if i now execute this i will get 1, 2, 4 1, 2, 4 so i do plus 1 and i only accept an integer so if i have another type a float so float can be a decimal number 1, 2, 3, 0.12 this is t2 and you have passed the float then it will say cannot use a variable of type float as int value so i would have to change the value of the float into an integer if i would want to make this work and let just change this in v to let print out the side how it wants to output it 1, 2, 4 we lost the precision here because we first had to change this type to an integer which makes it then lose the decimal number and then it goes to plus 1 then it is also 1, 2, 4 instead of 1, 2, 4, plus 12 and it's not because we have a decimal here that's why it changes to a variable even if we would output a float then it will show it so let's say that i want to accept float before 1.18 i could do a plus 1 float and then i could say float 64 here float 64 here and then i would be able to do a plus 1 float that i don't need to convert this anymore and then if i do go run i get 1, 2, 4, 0.12 but with the introduction of generics go makes it easier to write just 1 plus 1 function that can serve both floats and integers we just cannot use the keyword int anymore we need to make our own type so it's going to be v and then with square brackets before the curly brackets we can then define what v is going to be we're going to accept for v v can be an integer or it can be a float 64 or it can be also an int 64 or a float 32 so then we accept the 32 and 64 bit variants so int is also an int 32 but it's not the same as int 32 so we could also add int 32 and then we have all int 64 variations so if we then now do plus 1 of this float we will get a float back so we have 1, 2, 4, 1, 2, 4.12 and let's also output a type because that can be interesting the type just like we did in the previous lecture we can do a reflect type off and we can actually do the same function again it doesn't really matter and here we do it of t2 save this and then here we see that the plus 1 here is type int and the plus 1 here is float 64 so it's not really a dynamic type that suddenly we'll be able to do whatever here whether it's an int or float is still going to discover the type and then only that type will be used and to make this more clear let me show you a few things so now we are doing plus 1 because t which is v can be an integer of float so if we have a float we can actually do a plus 1 so we could do plus 1 and that is t3 but could we do t1 which is an integer plus 1.05 no because now we get an error this 1.05 is a float we could say it's a float 64 for example and we then have an invalid operation with mismatch types int and float 64 even if it's going to be a float 32 it's not going to work we always have mismatch type int and float 32 so whereas you can do a plus 1 on a float because that's possible 123.12 plus 1 is easy to do but on an integer where we don't have the decimal precision you cannot add 0.05 to it you can add 1 you can add 1 if you want a decimal type so how would that work within our function because now we can have in our function plus 1 we can have an integer or a float so if we do plus 1.5 well that x3 doesn't work cannot convert 1.5 which is an untyped float to v so there's no way the compiler can add this together there are certain limitations still that you have to take into account that you can accept multiple types but that doesn't mean that suddenly those types are compatible you would still have to write code to handle with different types if you are using operations that are compatible with one type and not the other type also if you would do a sum so let's put that back in place sum and we have t1 and t2 which is of type v and we need to specify this again and now we return t1 plus t2 then let's do a sum of that sum of t1 and t1 and what is the type going to be sum is 246 type is int t2 and t2 sum of two floats and the type is a float but what if we would do the sum of t1 and t2 well you will get an error because if we pass an integer then the type of v is now an integer and v here is integer but then if we want to pass a float float 64 of t2 does not match the inferred type int for v so now v is an int and now it will not be able to infer that other type because our type is already int so how to solve that well you could say I'm going to pass v1 and v2 and then I will have v1 here and v2 here but what am I going to return because now I do t1 plus t2 and there is a mismatch type between v1 and v2 because how do we do t1 plus t2 when v1 and v2 are different types so this is not something that can be solved automatically so we would have to solve that for go because otherwise it cannot compile so these generics are definitely a powerful tool you just have to take into account that it's not magically going to solve having different types so you would still have to take into account the different types and when you do a sum for example this could work if you do a sum of the same types or you write code to handle the different types when you have v1 and v2 but generics solve the problem of having a lot of functions one function per type is not necessary anymore now you can have one function where you accept multiple types and then do your operations within one function let's have a look at the different types that any variable can have while doing JSON parsing and up until now is to create a custom type like type myJSON is struct and in the JSON we can have a name a string and then we say okay this is a name and then we use JSON unmartial to unmartial a JSON into this struct but let's say we don't really have an idea of the field name that we can parse then we need to solve that in a different way and using the functions that we just learned like the type switch we can actually try to figure out how our JSON is structured and we could even anticipate multiple different types of JSON so let's start with a variable a variable JSON parsed and it's of the any type and I'm going to do a JSON unmartial of a custom JSON that I don't really know what's going to be in it and I'm going to do that into I'm going to parse it into the JSON parsed so unmartial expects binds and then I'm going to parse right here so the JSON is going to be starting with the field test and the field test is going to be another object and in that JSON object I'm going to have test 2 and that test 2 is going to be an array of integers if we get an error then I'm going to exit with log fatal so what is going to be our type now of our JSON parsed let's have a look with reflect mtprintf and then reflect type of our JSON parsed go run JSON parsing may not go and reflects us that it is a map of a string and an interface so interface like this is the same as any so now we have this covered that our first element in our JSON here is actually a map of strings which makes sense because we could have another test 3 we could have another test 3 something here which is just a string or something like that we don't really know what all the fields are but now that we have this JSON parsed with any we can actually parse this manually so let's try to use a switch statement again switch of JSON parsed type and then the value not sure if you need the value let's start with a default if it's default then we say not type not found and we output a type reflect type of if it is this case map string any then map found and then we can output it map found it's a map and we have a key test and there's not a map in there so then we can continue parsing this because now FMT printf parsed it partially for us but we still have any here which is our key so we can say if our field 1 is v which is a map does the test which is this one exist within our map if this test exist then ok it will be true and then we can use field 1 to again if you want which v2 is field 1 because it's an any and add a type and we can also add a default again if the type is not found then we can say type not found so map found and we try to parse the test part which is this then we say type not found map string another interface so we have another map which is a key and a value and then we're going to have an array so we can keep on parsing this over and over again and we can do this in a recursive function where we could actually write a function where we can accept any parameter something like parse JSON element is any and then we return something and then we could actually call parse JSON from this function once we hit another any type to keep on iterating this function deeper and deeper until we have resolved everything but I'm not really a big fan of that because that's again very abstract if we know roughly what we can expect like if you know the field names we can just parse it like this we can say ok we know that there is a test field in there at some point if we find a test field and it's still of type any then we're going to parse it further because often you have some idea how the JSON looks like it just sometimes that for example test could be in some cases a map and in some cases an array and then you want to manually parse this there's also a way to override these unmartial functions if you want to write your own parser but this method of using any is also something that is often used you could also partially parse your JSON so for example if you have a struct and you know that there is a field name which is a string in our case it would be maybe test string and that is then the JSON field test but actually you don't know where this is a string you can say it's any and then you can parse just this test and then you're going to have test3 maybe that's what you know is a string so that's test3 so then you don't really have to manually parse this part but you would manually parse this part here so we could try this out so if you say this is of type myJSON myJSON what's going to happen here myJSON is going to be test so test is going to be any so now I already know the dynamic test so I don't need this code anymore so now I'm looking for test2 because this might be dynamic and test2 what is test2 going to be type not found it's an interface but it's a slice interface so now we could say if case it's a slice of something that we don't know then we could further parse it case any and then we can say FMT printf I found a slice I found a slice any let's try that out I found a slice any because now we are trying to parse this but what is this actually this is maybe an integer I found a slice integer will this work no it doesn't work actually I need to again test the type so you could then iterate over this slice for key value of range v2 because it's a slice of any we probably don't need the key so we can say v2 element what is this type printfs type of v2 element do we already know what it is yes it's a flow64 well it thinks it's a flow64 but it could as well be an integer we don't really know so we found 3 elements and reflect things this could be a float because you might also be able to have 1.123 but if we know it's an integer we could still try to convert it as an integer as well now I'm actually curious whether if I do a switch of this v2 element v2 element if this is going to give me the same what is going to be the type here v2 element value well save this ok it's also saying a float which makes sense so we could either do a switch or we could just say if reflect type of is a flow64 then well we will have to convert it to a string then we can manually convert this v2 element to a float f empty printf float is f probably it doesn't look too chaotic I'm just trying to custom parse this JSON using different ways of parsing so you can use a switch if you have multiple cases or you can also use reflect to a string to check if it's just a float and then otherwise you can say I haven't recognized it didn't recognize v2 element and this is a float so now we see 10000 but maybe we don't really want it to be a float we are sure that it's an integer so we can still convert to an integer if you want it and then we need to have a %d or you can use v if you want printf to make the decision and then we have 1, 2, 3 and then it's not a float anymore now it's an integer so we can actually convert it the float into an integer now if we had 1.05 then this would be actually a problem because now you can see we lose this precision so we would then actually need the float you could actually check whether there is precision or if you are sure that it's an integer you can always just convert it so what I typically do is if I know my input is always an integer I will just convert it from a float to an integer if I am not sure I would be able to check or I can also just use float64 and maybe figure out later whether we need this precision or not so you see if you don't really know exactly what in a JSON you can still define a struct define any and then start parsing it so this is a nice application of having to use this switch type where you can have a map or where you can have a slice and depending on what type you have you are going to then dig a bit deeper in the JSON and extract the elements that you actually need so I will commit this code into my github types demo directory so that you can also review it in this demo I am going to talk about asynchronous calls that means that some of our code will be executed asynchronous you can think of it as if code is being executed in the background let me show you an example and it will become immediately clear I am going to do FMT printf1 FMT printf2 and I am going to have a test function in between them test function and I am just going to declare the test function and here I say 3 so how will this execute go run go 1 3 2 first it will execute 1 then test function and then 3 and then 2 let's say that we want the test function to execute in the background we can use the go keyword for that the go keyword means that this test function will be executed asynchronous outside this main function so it will just continue with the printf the next printf and this test function will be executed separately this doesn't mean that our program will be executed quicker necessarily it just means that the test function will be executed separately of our main function this will not work unfortunately because we will exit our program even before we print 3 so we just have 1 2 so let me just apply a small trick here let's sleep for 2 seconds to give it enough time to execute 1 2 3 so 1 first then it immediately hits 2 and 3 so it shows 1 2 3 but even then it's not guaranteed that this 3 will be executed last it just means that this will be executed in the background and then we just continue with the next instruction and then between the function calls this function a little bit of cpu time to see where we have to execute also functions in that test function so if we have a program and we need to do something separately so we can just continue going on with our program and we want to offload something then we can use the go keyword for example if we want to do some background checking or if we for example want to have a for loop that separately needs to check something then we can write something like this every second we want to check for something while our program is running so here we are just checking something but this time sleep over here is obviously not very nice to have so in go you have a concept called channels we can create a channel let's call it c we can use the make function to create a new channel and we need to give it a type for example let's give it a boolean this channel we can pass as a parameter and then we can use it in our test function so we can say if let's declare our for loop we're going to only execute 10 times well let's make it 5 that's quicker and after this we are finished so we are going to say on this channel put true and then we exit this test function and then here after we print 2 we can say are we finished equals c and then are we finished will then be a boolean so if we then send through to this channel here we can wait until we get one boolean on the channel so once we are here are we finished we will just wait until there is something being sent to the channel and only here something will be sent to the channel after we have checked 5 times and only then we will continue with the program in the main function so let me print are we finished and then are we finished variable let me just execute it once and then I will show you how it works step by step using debugging to make it completely clear 1, 2, checking checking checking until we hit 5 and then we hit the true this line sends to the channel true so now here this variable gets assigned true and we can continue so this is blocking until we get something on the input so it means that you can also make deadlocks if you don't have anything on the channel and just wait for nothing then the program will be deadlocked so let me try to run the debugging just to see if we can make it more clear start debugging so we make our channel we pass our channel we print are we finished and now we wait and now the checking is running because we are waiting here and this other test function is just running are we finished is true and then we exit our program so what if we never put this true here what will happen then go on checking and then go realizes that we are waiting here for the are we finished we are waiting for something in the channel to appear but there is nothing so all gorotines are asleep we have a deadlock and we exit so we still need to have something right into the channel or we have a deadlock booleans are also not the only thing that we can send we can also send a string or a struct just need to make sure we pause the types correctly and I can say we are finished as a string and then we will see that we get we are finished so you can also use that as a means to communicate data between your functions once you start using the go keyboard one advice I would give you is if you can avoid using the gorotine and the channels I would avoid it because it can make your program pretty messy very quickly and if you need it then you need it obviously but if you don't need it if you can do it in another way then I would not use it because it's much more difficult to debug and you can see that you can get timing issues easily so I'm not a big user myself of this of course I have it in my programs but only once strictly necessary for example this are we finished where you are waiting for another process to finish can be very handy if you are waiting for a certain state to be true and you just wait somewhere in your program for a certain state to be true and then once it's true then you put something on the channel and it can continue but like I said it can get pretty messy very quickly so if you can avoid it by just writing normal go line code then I would do that for example where go routines are used is if you have an HTTP server and for every HTTP request that comes in you can start a go routine because every request is something independent that you want to handle and then it's a very easy way to abstract your code so there are definitely very good use cases where you can use it just use it with caution and also remember that it doesn't necessarily make your program faster because you are using go routines if you have one CPU assigned and you use 10 go routines those 10 go routines will get CPU time once at a time so it's not going to make your program quicker necessarily if you just have one CPU core when you have multiple CPUs then you have parallelism and this can make your program run faster go routines use concurrency so you can have 10 go routines running at the same time and that is not the same as parallelism where parallelism everything execute at the same time with the way that we deploy applications nowadays on the cloud you are often assigned only a single CPU and parallelism is often achieved by running multiple instances of a program rather than having parallelism within one program so it really depends how you run time looks like I would definitely suggest to read about concurrency and parallelism in go if you need to optimize for multiple CPUs before using concurrency as a way you want to achieve parallelism in this demo I am going to talk about mutual exclusion so I call it mutex demo and I am going to work a little bit with go routines and show you what can happen when you need to access variables and change them in go routines so I am first going to make a struct type my type struct and this struct has a counter which is of int and then I am going to create a go routine that is going to pass this struct so I am going to call this my type instance is of my type and I am going to run a few functions and I am immediately going to declare those functions so I am going to say function and this is going to be my function and if I want to execute it immediately then I put these round brackets straight after the function so now I have an anonymous function this would be the same as if I would define a separate function but rather than defining a separate function I am just putting it in line in my main function a separate function and this whole part will be executed asynchronously I am going to pass my type here so in the round brackets I put my type instance and then I need to say my type instance is of type my type and I am going to execute five times this go routine so I am going to have five go routines running asynchronously from each other so I am going to make a for loop and I am going to put this go routine in this for loop so I am passing this my type instance and let's try to increase the counter and let's also create a channel because if you would just print the counter here counter decimal my type instance counter I would get back zero because the go routines would run but I don't think they would get a chance of completing before our program completes as you have seen in the previous lecture we need a channel to make sure that we wait until our go routines finish finished is a new channel of type bool and I am going to say ok once I increase it I finished I finished this go routine and then I am going to iterate another five times and then check whether this true has been put on the channel so five times it will wait five times it needs to get the true put on the channel and then this will output five times true but I am not going to use this output so this output is bool I am not going to do anything with the output of this channel I am just going to have to wait five times five times something will need to have been put on the channel go run still zero and why is it zero because the counter is zero the input here will be counter D and then what is the counter the input here will be zero and the output is going to be one but what are we missing we are passing this my type instance not as a pointer so that means that even though we make changes to the counter here at the bottom we don't see those changes those changes are only visible within the function where we are using it so you can change it here or we can just pass it here as a pointer so this is one way of doing it or we can also make this immediately a pointer so then my type is a pointer here my type is a normal variable and then we can pass this pointer so it will not happen now now we have five now our counter has five and here you already immediately see that we are using go-routines so that the concurrency is actually not exactly what you would see here so it's not that every input is going to be the output of another function let me just do input output counter so our first input is zero we do plus one our first input is one we do plus one is two our input is two make three our input is three that we make four but then here for example we can already see that when it was executing our go-routines this is asynchronously so depending on how much CPU time is being given to a specific function you see that this input counter here is one so this go-routine here this input is still one because this go-routine started when the output was one so we can already see that it's quite unsafe to use this my type instance counter because you don't know exactly what the input is going to be because we launch it five times but it's not launched at the same time it is all running in the background by the time we launch our fifth go-routine we don't know how many have been already executed so let's change this a little bit and I'm going to look for let me execute ten times and I'm going to look for ten times here as well and I'm going to look for the fifth execution I'm going to say found counter equals to five and we don't need because we have this or we can remove it let's run this okay we have found our output five here so it was four and we did five but will this always be true? we don't know so let me now change this a little bit time sleep and I'm going to use the run but I'm going to use a math run so this is pseudo random number generation never use run from math to generate password because it's only pseudo-random number generation use crypto if you need cryptographic random numbers but this is good enough for our demo so between zero and five random multiply this by seconds and then because this is a number I also need to do time duration so now time duration takes an input time duration is an in64 so we are taking this int here and converting this to in64 which is now compatible with the time duration so we can multiply it by time seconds so we're going to sleep between zero and five seconds and then we're going to see if you can still find number five so what happens input zero one two three four zero five seven and then the output is all ten so this time sleep just mimics some other functions that we could do like we could do some API calls and we don't really know how long they take so in this cool funk you can see that the input is not in complete order and the output is also not going to be in complete order once we do something within this function that can take some random amount of time it was already visible earlier but only a little bit and now it's definitely visible that we don't really know we can not really rely on our input and output to exactly know what it is so if you are doing some checks whether some variable is equal to something else it might have already been changed in another go routine because this go routine runs ten times so you are not 100% sure whether you always find the number five for example even if you remove this time sleep there is a chance there is a chance that we will miss it somehow so what should we do then if you want to be 100% sure that these two statements the counter where you change a variable and then you still want to check on it then you need to make sure that this amount of code is executed at the same time and this you can achieve with mutual exclusion mutex and there is a package for that we can say mu is of type sync mutex and then we just pass this along and just before we start we can say my type instance mu lock and this lock will put a lock if the lock is already in use the calling go routine blocks until the mutex is available so this mutex might be available or might not be available but we will wait until it is available and then we will lock it and then nobody else can lock it and then we just need to unlock it we need to unlock it once we are finished and now we are 100% sure that this code between the lock and the unlock no other go routine is changing this variable so let's try that out and now it will take a lot longer to execute because we always have to wait for the lock so you can see input counter is 0 then it is 1 and now we always will find 5 and now it is nicely in order so if you only use go routines and you will make sure that no other go routine is making changes or doing something you can use this lock with this sync mutex package there is an alternative approach here if you just want to have one variable then you can also write a function so you can have also a function in this my type and then you need to make sure that you have also a pointer here you can say increase counter and then you can also do the lock right here so you can say unlock and unlock and in between you can do you can increase the counter so this is an alternative or it is M U and M U and this is just M counter if you just want to have a lock then you can use increase counter here so that's why we put the lock and the unlock between between our lines here but both approaches are possible you can have a function within your struct or or you need to have some logic within your go routine itself where you need to use lock and unlock so there is two approaches and this is a very common way of programming when you are using go routines and you are going to hit some concurrency problems then you just want to use this locking mechanism