 Hi. Today I'm going to be talking about using Prometheus to centralize disparate metric data formats. My name is Matt, and I've been working at a company called Timescale since I finished my PhD. Actually, the CTO of Timescale was my PhD thesis advisor, and so I didn't fly far from the nest, so to speak. It was one of the core architects of TimescaleDB, and for the past year and a half, I've been leading Promscale, which is a project to allow users to easily ingest Prometheus data into Timescale. In my old time, I liked going skiing, going to theater shows, traveling, and taking photos, but during the past year, COVID has made most of those things impossible, so I really love to geek out on those topics with you over Twitter, Slack, or email. Today, I'm going to tell you about an interesting use case we had from one of our users, and later on in this look, I'll tell you why I think this use case really illustrates the power of Prometheus and Promscale. This use case involves a gaming company which produces many games, and the way the company is structured is that each game has a team which controls its infrastructure and observability system. But the team, we were actually working with the load testing team. This team is responsible for making sure that each of these games can scale to handle a large number of users that play these games over PCs, consoles, etc. They do this by load testing each of these games regularly via a CI pipeline and generating a report based on the results. The problem, however, that this team has is that each of these games has a different observability system, and it's a challenge to bring all of the data together in one place in order to combine with the test results and generate the report. The architecture looks something like this. The load testing team has a load generator, which generates load onto game services running in the staging environment of a games infrastructure, and there are many such games, remember. The load testing infrastructure is monitored by Prometheus pushing data into Promscale and still running in Timescale DB, and the query engine is a homegrown dashboarding system developed by the load testing team. Now, the results of the load tests are apparent in the results of the observability system inside the games. And as we mentioned before, each game might use a different observability system. The load testing team wrote data collector to collect the key observability metrics from each team's observability stack and push those metrics into Promscale. This allows all of the load testing data to be centralized in one place, including both machine metrics and game metrics. The benefits of this to the load testing team is that they're able to save metrics into their own system to analyze it over time, which makes it easy to run reports and to build visualizations using SQL and PromQL queries. I asked this user why did they use Prometheus? They mentioned that Prometheus is an industry standard with an easy to understand data layout, and it's easy to use PromQL. So, the game is a wide adoption inside the company, and the team wants to be on that kind of bleeding edge. And the team runs Kubernetes, and Prometheus is very cloud-friendly, so it seemed an extra fit. Now, everybody in this conference knows all of these reasons for using Prometheus, but I think it's good to be reminded of them so that we could advocate for Prometheus usage inside our organizations more effectively. I also asked the user why they chose Prometheus. One of the main reasons was backfill. Turns out that this team had legacy data they wanted to be in the same centralized data repository as the new data. They wanted flexible querying, they wanted both push and pull, as I showed in the architecture slide. They found it easy to use and deploy. They mentioned TimescaredDB Slack channel as giving good community support, and they needed long-term storage for Prometheus anyway, so this checked multiple boxes for them. Now, I'd like to stop and listen to why I think this is a good use case to think about. And I think the main reason is that it shows that Prometheus and its data format and PROMQL form a good basis for these hybrid systems, which combine different metrics together for analysis and reporting. And this ties in very well with PROMSCAS vision, which is to be the flexible database for Prometheus and other observability data. What do I mean by flexibility? I mean support for both push and pull for different data formats. Obviously, Prometheus remote rate, the Prometheus exposition text format, and JSON. Support for PROMQL, for dashboard inquiries and SQL for long-term analytics. Support for backfell, different data types, including not only Prometheus flows, but also strings and symbols, as well as different observability modalities. Right now, we support metrics thanks to Prometheus, but we want to add support for logs and traces as well. The flexibility allows you to put a lot of your observability data into a single database. But why would you want to do that? I think there are two main reasons. One is operational and one is analytical. On the operational side, running any kind of database or data source is hard because it is a state for service. So you have to deal with PVCs or hard disks. You have to think about backups, high availability, failover security, scaling and tuning. These are complex systems. And so why would you want to run multiple types of these systems if you could decide on just one? The other reason is that having more data inside a single database allows you to use joints to do analysis across these data types. This allows for correlations, more complex analytics, as well as predictions and ML. If you want to learn more about this, you can go to the following website to get slides and resources from this talk. I encourage you to check out TOBS, which is the observability stack for Kubernetes, which is a CLI tool that allows you to deploy a observability suite inside of Kubernetes in under five minutes. That suite includes Prometheus, Grafana, Prom Scale and other components. And of course, check us out on GitHub. It's worth mentioning that we are hosting a bit of a further discussion later on in this conference about dealing with this kind of messy world where you have multiple metric systems or legacy and cloud deployment and this plethora of complexity. And I hope to see you at that discussion. Thank you. And you can find me on Twitter at 7NY or by email at matt at timescare.com. Thanks.