 Can I start? Okay. Hi guys, I'm Guo Qin. I work in Gojack and the data science team. So currently, there's two team in the Singapore office, data science and there's fraud. So I've been presenting later, it's from the fraud team. Yeah. So I'm going to do some shameless promotion about Gojack for starting. So how many of you know about Gojack over here? This is a lot. Okay. That's amazing. Yeah. So we're like Uber for a motorcycle. At least it started like that. And our main base of operation is in Indonesia. But we're into a lot of stuff. We went to food and payments. So it's a great place for like anyone interested in data to work because it's like working in multiple company all at the same time. A lot of very challenging problem. Yeah. Okay. So here's my agenda. So introduction to promote Gojack. Yeah. And don't worry about office size. So we are expanding and we'll be moving towards even more spacious office. And yeah, and I'll be doing TensorFlow serving in Go. So I have to show some Python code. Not sure if that's a crime over here. So yeah, this is this is the picture of technical difficulty. Yeah. So this is a picture of a redesign app, the recent realist redesign app. So yeah, we have a lot of services. And yeah, you can, I mean, we stayed at Indonesia for a month and GoFood is pretty amazing over there. So let me see. So a really cool company profile video. The sound isn't working. Change the output. Change the output. Change to the internal speakers. Increase the volume. It's a really maximum power. I haven't shared yet, but since I think people get pumped out over this, we are always hiring engineers. So and we're pretty good on and pretty big in Go lang. So yeah, if you're interested, just approach any of us. Yeah. So I guess it's something different. Yeah. If you want to follow along, you can just go to the repo, my GitHub account. I didn't upload to the company one because you have odd issues. But yeah, you can just copy in. There's a Mac file over there so you can just make the environment directly and run the examples. So some primal. So what is TensorFlow? It's actually an open source, like general ML library. It's written in C++. Well, I'm saying a lot of language that's not Go lang, but forgive me. And Python is the biggest, one of the biggest community that use TensorFlow. So they have interface for Python code. But the underlying engine is written in C++ and it's also initiated by Google. Yeah. So yeah, what you can think about is something very SQL like. So you basically tell them what you want and describe what you want and the underlying engine built it for you. So unlike SQL, you return a table, this thing returns a graph, define a graph. So over here is a very simple linear model. You define the variable. It's usually the weights that you want, the output that you want eventually. And you put placeholders to be a promise, a promise to return a value at the end. And yeah, you have this linear model, say W multiplied by X plus B. So it's like MX plus C, linear model. For more information, just visit their website. It's actually really good. I actually took the exact number over there. So yeah, deploying TensorFlow model in Go. So the TensorFlow actually provide API to use in Go programs. But even in the official website, they say that all the training should be done, maybe on Python or some other place. Then you just execute, yeah, then you execute in Go. So you load train model and do them in Go. But why would you do that? So it actually depends, as someone mentioned, Python versus Go. Writing test is slightly easier in Go because it's a static type. You don't have to write so many. You don't have to think about so many conditions. The Go test is pretty comprehensive, the built-in one. I think I just use Go with an library for assertion and writes off the package. There is even benchmark test for it. So yeah, it's pretty easy to write test. And in general, I think you feel slightly safer in when you write Go test, Go lang. Yeah, and it depends on your company's support, right? So if your company have stronger support for Go, then maybe you want to, after training the model in Python, you want to deploy it using Go. Or maybe you're just a very hardcore fan. Yeah, and performance wise, it's actually better. Ugh, yeah. So yeah, I'll walk through an example of like training, writing a Python train model, then running it, serving it in Go. So that's actually not my secret. A lot of people share several examples, but I took a look at all the examples. They're still a little bit complex. They use the NS examples. So it's like the hollow world for ImageWrite. Yeah, so yeah, I'm gonna show the code for it. Ugh, okay, yeah. So yeah, I'm pretty sure everybody can tell. This is Python, this is Go. Yeah, so we're gonna just train a very simple straight line. So a linear line that goes through, 45 degree through the origin. So, you know, so. Sorry, increase the form? Yeah, increase the form. Sure, sure, oopsie. Maybe I'll start with the Python one. Is this okay? Yeah, sure. So yeah, so yeah, you start over here and define the placeholder. Basically like variables. Then you start off with a weight. You can just set it so that right now initialize it to be 0.3. Then you define the loss. So since it's linear, you just define like a mean square error. So yeah, then you run gradient descent. So if everybody confuses gradient descent is like a way to minimize the loss. As you can imagine yourself like trying to get to the bottom of a hill. Then you just, you know, travel at the steepest route downwards. But there are a lot of like pros over here in the data science team. So just approach them for more questions. Yeah, but is everybody okay with this? Yeah. And you initialize and you initialize the session and run it. Yeah, you run it and you know, you do it a thousand times, you train it. Then you save the model and you serve it. And so the only tricky thing is if you look at all the examples, they actually don't put the names. But over here, I named the input, I named the predict function to be, to add x, m, and c. So it's y, the thing that you're trying to predict is mx plus c, yes. Okay, so high school. Yeah, yeah. Okay, everybody okay, right? Okay. Yeah, so if you run the Python code. Yeah, so if everyone is trying to reach producers, you can ignore this CPU support. Because yeah, unless you want to optimize it for your particular CPU, then you really need to take note of it. Yeah. So you end up having this. It's basically serialized binary files to load the model. But let's look at the goal and code first. So the linear model is a binary package? No, they have binary file inside. It's not a binary package. Yeah. So that's the model we're loading. Yeah, yeah. Import TensorFlow Go, it's pretty much all you need. Then, you know, that's the model. Yeah, and that's the tag that we gave to the model. Yeah, and that's the operation. So I define input on the Python script. So here's the input. And I just make sure the name matches. So that's the only tricky thing about running it in our language. And so I also declare input. So this is the sample input that we are doing. So, you know, let's say when x is four, so y would definitely be four too, since it's a 45 degree straight line. Yeah, so. And yeah, so y is 4.004, yeah. If you run more iteration, then you're going to get closer to four. Yeah, so that's actually it. So it's actually pretty simple. Yeah, so just remember to label the operations. If not, you run into weird errors. And the errors are, the errors are not goal line errors. So it's quite difficult to debug them. Yeah, so just name everything. Make sure that when you operate them, you're calling the name and you should be able to do it. So there's like some quirks. So if you work on like more complex graph or you use estimators, it's actually not easy to load using goal. So yeah, so it depends on your project, yeah. How is a model normally run? Oh, what do you mean? It's just a goal. Oh, yeah, so yeah, I can continue on that. Yeah, there's other methods to run the model. Yeah, so you can also use the in-house they open sources thing called TensorFlow Serving. So it's actually highly performance because it's written in C++. It just, it runs a JAR PC service to serve the model. And I think a few months ago, they couldn't serve multiple model, which is very useful if you are A, B testing your stuff. But right now I think someone made a PR and you can actually serve multiple models right now. And in goal length side, you can just, you can make a GRCP call to get a prediction or you can just use their Python package to make the call. But why would you still use goal? So in our case, Alice, in our case, so someone sent a request, maybe we have to fetch some other data before we make the prediction and you want a service that can handle that, right? So you may use goal length as an in-between between the requests. Yeah, okay. TensorFlow Serving is a Google. It's not, no. No, it's a Google. Yeah, it's a Google, yeah. I thought you said goal length. Oh yeah, it's made by Google. It's open source. But can you have that service running on-premise? Yeah, yeah, yeah. It's a library. It's not a service. Yeah, it's not a service, it's a library. So you can just dockerize it and use it. Yeah, okay. So why not GRPC then? Yeah, this is actually possible. Yeah, but then, you know, if you can, it depends on your use case, right? Yeah, it depends on your use case. So if you have something simple, you could consider the goal length one. If something is really like, you know, you want more support over it, then you can use this. And you want more complex graph, you might consider this one. TensorFlow Serving, okay. And how would you deploy a Python model? Oh, okay. Outside of the goal, outside of TensorFlow Serving? How would... Yeah, how would you deploy a Python model? How do you run inferences on a Python model? Oh, that's actually, yeah, that's actually easy. So it's, for Python, it's actually really easy. When you, and you can just write the thing to run the prediction itself inside a Python code. In Python. Yeah, in Python itself. But I think someone brought out deploying Python. It's like, I guess on our experience, deploying Python versus deploying goal length, I say the biggest difference is dependency management. Python dependency management is actually pretty difficult to, yeah, to deploy. But yeah. Okay. That's it. And any questions? I just want to present a question, thank you. Oh, yeah. Yes. So it's gonna be on a training part, which is Python, and you have a prediction side, which is in code. Yeah, yeah. How do you do the test, or make sure you're using the same features? Because you mentioned that you need to make sure that the name use is correct. Yeah. But that usually integrates whole scan. Maybe made it by different people. Yeah. Make sure. Yeah, so. Integration tests. Yeah, you can do integration tests. And yeah, you can, you can have the Python, so you train the model, but you also get a Python model to come out of the prediction. Then you get the goal length model that do the inference, and you're gonna come out with a list of predictions, right? And you can generate like, you know, test cases and just run and see how fast they differ. Yeah, and that's, yeah, they, in theory, they should be similar, but in practical, you run into a lot of issues. Like sometimes decimal points rounding and all this stuff. Yeah, similar. Yeah, similar. And that's why it's difficult. Yeah, that's an excellent, excellent question. It's actually not easy to test models built in different language. Yeah, it's, we do have, we do face that problem and I think we could work a lot to make it, to generalize that process too. Yeah. Yeah, and I see a lot of this is like, if you do integration tests and when you load the model, it's kind of dynamic loading in your goal length, right? So in production, after you load the model, you need to make sure that the model you load is correct. And do you do another like, another testing after you load this dynamically in production or you just do everything in pre-production? Yeah, you should do it, you should do, you can do the testing in CI. Yeah. But it depends, right? So maybe like, you know, let's still test that for it. Yeah. But you should definitely test it. Yeah. Million things can go wrong. Yeah. Testing, not great speech about testing, I think, offloading complexity, but I think good unit test is what allows us to sleep at night, yeah. So Go check, would you run an inference service? Would you write this as an inference service written in Go? Yeah. As part of another binary, and just a function call. Architectural question, actually. Okay, so I think, I think Alice, for me, right, my next project, we'll probably try to write it in Go, at the inference plot, because it depends a lot on who is calling the service. So if the latency requirement is really high as in the latency required is really low, then, you know. You need high performance. Yeah, you need high performance. And so even if you read the binary file and serve using Go, it may not be as fast as TensorFlow serving. But, you know, TensorFlow serving means that you have another layer. So what you should do is, after you write it, you profile it and see, you know. Which is faster. Which is faster. Okay, yeah. Crew, any questions? Oh, we're hiring, just, we're saving that up. So yeah, a lot of good questions and good problems, so we can solve them together. Okay. Okay, I think that's it. Do you run long running servers in inference servers? Or is it one of course? Yeah, so. Unless for models, right? You have to, they might both still write. So you'll retrain them once in a while. So like long running, instead. Long running means the service running 24 seven. Oh, yeah, yeah, yeah. Until you create a new model. Yeah, yeah, yeah. Any issues with resource leaks? Well, thankfully, not that much, I think. No, we refresh the model, we usually do a completely different. So yeah, but between refreshing the models. Oh, they really last like a day, so. Yeah, yeah, yeah. So then you update, so okay. You can't tell whether it's a resource leak or not. Yeah, yeah, yeah. So actually, yeah, still things from my implant team, so. She would know better. Because I didn't survive to be back ends. And I wasn't very careful in resource leaks. Okay, thank you. Thank you.