 Okay, so for our next talk, we have a talk on virtual tourism and COVID times and interactive travel memories using the power of Streamlet. So this is a very exciting talk by Netesh Raghunandan. He is an engineer who works in the couch base. He's based in Germany and he loves traveling and is going to give us a talk about virtual traveling in COVID times. So hello, Netesh. Hi. Very nice to have you here. So are you always traveling or what are you based now? So I'm based in Munich, Germany. I used to travel a lot before COVID. Yeah. Yeah. Thank you. Shall I start? Yes. Okay. Hi everyone. So I'm Netesh and today I'm going to tell you about my experiences trying to design a virtual tourism because you cannot travel right these days during the COVID. So before I start, a bit about myself. So I work as a developer advocate at couch base. So I just recently started there and couch base for those of you who don't know is like a really scalable, no SQL database. But this talk is not focused on couch base. And so I really like to prototype or hack things together. And this is how this project was also born. And I really like to build things in Python. And as I mentioned earlier, I'm based in Munich and I also organize a local Python data community here. And I really love to travel. And that's one of the reasons why I came up with this idea. And you can find me on most social media platforms with my tag Netesh. Okay. So over the, so I'm originally from India and over the last five years. So I moved to Germany in 2015. And over the last five, six years, I've been traveling a lot across Europe because the borders are so close and there are no restrictions that you could easily travel. And this has been one of my major hobbies in the last few years. And I also did some traveling in 2020 just before the COVID hit. But and I had some more plans for 2020. But then I realized I couldn't do much of what I wanted to do. So the world was shutting down due to the COVID pandemic and people had to stay home and rightly so. In this scenario, I thought, what can I do to improve my disappointment at not being able to travel? And so I was looking at some options like you could do some virtual experiences on different platforms or do some events. But why not try to build something of your own? And so I took my inspiration from this Google Maps timeline. It's actually a really nice feature. So if you have your Google location history recorded or saved that you don't delete it, you can see your location history, see what you're doing at whatever points of time that you have. And it also has a lot of interesting features like gathering pictures or activities that you were doing at that time. Yeah, I mean, yeah, it's Google, but I use it all the time. So I don't really mind giving this data to Google. So what I realized is you can get all this data from this Google takeout service, which allows you to basically download the data that Google has about you. And there's also an option to get your location history. So it gives you adjacent data with all the locations that you have visited and at what points of time. There's also an open source location visualizer, history visualizer, which you can use to get a heat map of your locations. And it also runs completely in your browser, so there's no data leaking. They also have a professional version which you can use to do some more analytics on your travels. Yeah, so these were some of the inspirations that I took. Yeah, so if you look at this data, I don't think it's that clear on the screen, but essentially it's a JSON with a list of all the timestamps, latitude and longitudes. There are also some more additional features like what's the accuracy that Google believes you are at this location, also some of the activities that you are doing, whether you are traveling in a vehicle or you're on a bike or you are on foot and different things like that. For this use case, I didn't really take any of those things into consideration, so I just took the timestamps and the latitude and longitude. And there I realized that some of these data points are not in the format that you normally expect. For example, the timestamp is in milliseconds, but which you can easily convert using a pandas function. Whereas these latitude and longitudes are like quite bloated, so they are, so the values are multiplied by 10 raised to 7, so you need to divide them by 10 raised to 7 to get the right, or get the latitude and longitude values that you see in your maps. Okay, so now that I have all this data and I can do some visualizations or, yeah, now I can basically visualize this data. So I have all the clean data, how can I visualize it? Okay, so there are a lot of different ways you can do it. So you can use Jupyter notebooks, which is something which I've used a lot in the past, so you basically write your own repel, you basically write your own code snippets or visualizations, and then you can basically run it and see the results instantaneously, or you could do good old presentations like you do in those meetups where you share your travel experiences, or you can write some code or scripts that does some generation of images or does some visualizations, or you could also use things like Tableau or Power BI or Looker, which are quite limited in the customization, but you can get a lot out pretty easily. But I thought, okay, this Google Maps timeline is a web app. Why not try to build something like a web app? And, okay, before coming to the web apps, this was my workflow before. So I would create a Jupyter notebook, do write some code snippets for visualization, see what's happening with the data that I put in, like a small subset of data. It's great. You can see them instantaneously. You can tell your stories. You can customize it however you want, but there are some issues like this great talk from Joel Groves from like two years ago or so. Like Jupyter notebooks are not that great for sharing. So if you share it with someone, there's a good chance that they're not able to run it. Similarly, like there was a study, I think, from JetPress, where they explored a lot of O-Public Jupyter notebooks and found that most of them are not reproducible. So, yeah, I wanted it to be a bit more reproducible. So the other option was like go with some web apps. And I'm okay with like doing the backend stuff. I can try some decent code and get the APIs working. But my friend and skills are not very good. So when it comes to the frontend, there is a bunch of JavaScript libraries. And there are many more coming up every week or every month. And it's really hard to keep up with them. And if you're not familiar with any one of them, there's a lot of data that it's quite overwhelming to pick one. Like each framework has its pros and cons. How do I pick one? And some of them take a lot of time to get up to speed with. And this is where I came in touch with Streamlit. So Streamlit is kind of like the mix of both the rapid prototyping of Jupyter notebooks and like the web app without a lot of frontend requirements or no frontend experience required. So what Streamlit does is it allows you to take your Python scripts and convert it into an interactive web app without a lot of work. So you just add some components. So they call these things components. So you can basically insert a component which makes these scripts interactive. And it's also really easy to deploy. For example, this is one of their examples from their webpage where they give you the visualization of the inference of a model, which is used to do object detection. So you can see like, which are pedestrians or cars, for example. And you can also change all these parameters and based on that, your visualization changes instantly. The good thing about Streamlit is you don't handle any of these visualization in Streamlit. So Streamlit is just making your visualizations interactive, meaning you can do your visualizations in whichever libraries or frameworks that you're comfortable with, like Matplot, Label, Pandas or, yeah, Altair, you name it. And it supports a lot of these out of the box and you can also create your own components. So under the hood, Streamlit gives you React components, but you can also write your own components in TypeScript. But if you're using some of the existing libraries, you don't need to do it. Okay, so I realized, okay, let's try getting visualizing this Google location data on a map. But what if I can add some background information, like many times it's images that make a place memorable, that you remember a place, oh, okay, this is the image from this location and this is what you were doing. So there's also actually a pretty good API from Flickr, which is, yeah, I mean, you can publish your pictures there for free. So there are a lot of public images on Flickr and it also allows you to search for images based on GPS coordinates. Additionally, I can do the classic analytics stuff, like histograms or heat maps. Okay, so now let me just show you the app that I built to do this visualization. And also I'll walk through some of the code snippets which tells you how easy it is to get something running. Okay, so here you have like an inputs, so you can select the timeline that you're interested in and it would basically load the data in that timeline and also plot it on a map. Yeah, as you can see, there are like nearly 900K data points and so now the data is loaded and plotted on a map and you can also zoom in, zoom out. So all these map tiles are actually from OpenStreetMaps and I use MapBox because it makes things a bit easier or like a bit more performant, but you can also use like the default tiles from OpenStreetMaps, they work quite well. And yeah, so you can actually zoom and pan and do all kinds of, yeah, yeah, lookups on the locations that you're being to. And whenever something changes in this inputs, what StripDit does is it reloads the, it reruns the script based on these new parameters. So, yeah, and it also tries to center the map and since I was too zoomed in, it went into a different direction, but yeah. This is how it works. And so I can also do things like histograms, see when I was traveling, like which months or which years, yeah, so you can see that 2016 was quite high, yeah. Things like that. So this is basically just doing a count on all the different data points that I have. And you can also see that I was traveling more in the summer months, yeah. And you can also see which hours you were more active or visiting more locations. And yeah, you can see that, yeah, I was not out too late. Yeah, so that's that. And yeah, and I also want to show you like the images, how it works. So what I do is I take some random 10 locations from this and get some images based on that location. And as you can see, it's quite random because some of these images don't make sense. Like for example, this is, yeah, just random. And similarly, but for example, this one is like a local lake, for example, yeah. And you can also increase or decrease the images as you like. But yeah, I mean, this is right now just random based on the GPS coordinates that you get here. Okay. Yeah, I would also like to show you the heat map. It takes a bit of time. So I'll also just load the code so that I can show you how these things work. Yeah, it takes a few seconds for the heat map. Yeah, you can see the indicator here that it's running. Yeah, one more thing which you can do is like go to all these visualizations in full screen and yeah, also zoom out. Yeah, the heat map is now loading. Yeah. So you can see like which locations had like a higher proportion of visits compared to none. Yeah, you can see like there's nothing in the Nordics whereas in Germany where I am, there is a bit more concentration of locations. So, and if you look at the code that is generating this, so most of it, so this whole code is like around 150 lines of code and it's also open source. So all the lines that start with ST are the ones which are done using Streamlit. So there's like some page layout but the interesting thing is also when I load the script, I just load the data and here you can see that I'm just reading the CSV file with the data in the pandas data frame and there's also something called caching. This is quite important in Streamlit because the way it works is whenever one of the inputs change, the whole script is run. Meaning this function is executed every time one of the inputs change and what caching does is it allows this function results to be stored in memory so that when the next time you run the script, you don't need to compute the function unless the function changes. I would just like to highlight a few things. Like for example, this is the slider which, oops, which I used to get the selection from the user and it's just one line. You just provide the minimum and maximum values, yeah. And it gives you the selection and all this filtering is basically just a pandas filter based on this selection, yeah. And most of the outputs are written just like using streamlit.write. And for a map, all that you need to do to plot a map like this is st.map and provide it with a data frame that has a GPS, so the latitude, longitude coordinates, that's it. And the histogram, as you can see, is like a matplotlib plot and you can just plot it using streamlit.pyplot. What else do, yeah. The heatmap is generated by something called Folium which is an open source library. So it's doing all those, creating this heatmap for me and then there is like a community plugin or community component from streamlit called Folium, streamlit, which is what I used to basically show this on the results. The other interesting thing is this images, you can just provide a list of images, URLs to a streamlit.image and you can also specify how you want to lay it out, like for example here with, and yeah, you can just lay it out for you. And yeah, I mean, I can also show you like how these locations are done. So it's right now just pure, yeah, random images based on this latitude and longitude. Okay, let me go back to the presentation. So we covered the demo and how do you deploy this app once you have it? It's quite straightforward. Like there are two options. One, you can run it on your private server or private machine, in which case you can create a conflict file. So most of this conflict file tells you like which board the app should run and things like that. Also like how much, whether it's running in headless mode or not. And yeah, you can just run it using streamlit.runscript.py and if there is a streamlit conflict file in that function, in that folder, it's detected automatically. The other option is like a streamlit sharing. It's a service provided by them. It allows you to visualize or deploy your public GitHub repos for free. There is a limitation, I think, of three applications running at the same time but they also have a premium offering in case you want to do private repos. So in this case, I didn't use this public sharing because I didn't want to share it with the entire world, the data. To mention some of the observations that I have, like what streamlit is really good at is to make your Python code really interactive in like just a few lines of code. It's really simple and easy to follow. And you can also customize it however you like, basically almost all the interactive components. Deployment is also really easy and the community is also really active and the forum, if you ask some questions, you get like feedback quite instantly. If you come to some of the drawbacks with a streamlit is there's no built-in authentication, meaning there are workarounds for it, like for example, you can, there's a password field. So you could use a password field and load the script only if the password field is matching with the one stored. The other issue, especially for those people who want to do this with the corporate design guidelines, you might find that the design capabilities are offered by streamlit is limited compared to something like a plot lead. But overall, I would say it's good to have, if you want to have like a decent interface without putting in a lot of effort, especially if you're not that familiar with JavaScript and CSS. Are there alternatives? Yes, there are alternatives. Like for example, Vola tries to give you the ability to customize your Jupyter notebooks. Then, as I mentioned earlier, quite a lot of support. Then there are also other options like Bokeh and Panel. So if you're familiar with one of them, the main advantage that streamlit would offer is purely the concise code, because some of these things take a bit more verbose code to achieve the same thing. But I mean, if you are familiar with something, there's no reason to switch to streamlit. Okay, as you might be wondering, I didn't really quite reach the same level of flexibility that I wanted to. And some of the things that could be improved is like the locations. So right now, all the locations are GPS coordinates. If I can map it to a city or country, I can do a lot more of analysis, interesting analysis. But the problem with that is all these, there are reverse lookup, reverse GPS lookup options, but most of them don't support like this scale of data without paying a lot of money. An alternate it might be to reduce the data somehow by using the activity field or other fields in the dataset provided by Google. Another improvement as the images, I mean, I could connect it to my photos which would make them really cool. There's actually an option to get the data that you have stored in your Google photos. However, I'm not sure if you can actually do a query for these images using your GPS locations. So this is possible by using like a app that you create on Google Cloud, which would allow you to connect to Google photos. However, yeah, I didn't give it too much effort, yeah. So to conclude, I would say string width is great for prototyping. So it gives you a good experience, like quite similar to what you can get out of something like Jupyter notebooks. And you don't need to worry about any of these JavaScript or front end issues. It doesn't offer you everything. So it's probably like 80% of what you can achieve, but it's also a lot less of an effort to get this running. For me, I was happy to revisit some of my past travels, especially when I did this whole thing when I was in quarantine. So I couldn't do much, I couldn't even revisit, go out of my room. So this was a nice change. I got to try a different framework and also found some interesting places that I knew. I did remember visiting, so yeah, that was cool. And so I've left the references so you can find the GitHub repository which has this code. And yeah, you can also find a lot of other examples on the Streamlit examples page. And yeah, if you have any questions, I'm happy to answer them, yeah, thanks. Thank you very much for your very interesting talk. Thanks a lot for that, it was very good. So did I understand correctly that in Flickr, we can actually search based on GPS because I never knew that before? Yes, that's right. So you can actually search for all the public images on Flickr based on GPS, yeah. And in the slides, you were showing that the places that you'd visited, they would automatically bring up a series of pictures related to that location. Yes, that's correct, yeah. So that was very interesting. Is there anything else you'd like to say? I mean, not really, I mean, I would just say, like if you are looking for some interactive visualization in Python, Streamlit is a good choice. You can definitely have some fun. It's really easy to get started. So just give it a shot, I think I'm pretty sure you'll like it. Yes, and you're available if anyone has any question to ask. Yes, I'm available. That's great, thank you very much. Yeah, thanks. Thank you.