 Hello everyone. Thanks for joining today's session and today's session is about tips and tricks to migrate from eager to open telemetry. So first things first, how many of you are using eager already? Okay, open telemetry. Have you started using open telemetry? Nice. Okay, then I think you will understand most of the eager parts and open telemetry parts that I'll be discussing today. So the agenda for today is prerequisites first setting the ground correct first like what are the expectations from the talk and other things and why we have to migrate and we will also go through a brief architectural overview of both eager and open telemetry and we will be talking about levels of migration. And I also have a demo on how we migrate an application. And the last one is eager and open telemetry boundaries. So today everyone is a bit worried and confused that how and where do I plug eager over open telemetry. So about me, my name is Vinit Pothlopati. I'm a product manager at timescale. I work on the observability products at timescale primarily focused on prom scale and tops. I'm also a maintainer of open telemetry operator. So if you're already using an open telemetry operator would love to hear out your experience. And if you would like to check out the products that I work prom scale and tops, they are open source. So the links are shared at the bottom. It's timescale slash prom scale and timescale slash tops. Okay. So the prerequisites. So throughout the session, I'll also be using Tom as hotel. So open telemetry synonym or a short form as hotel. So the commonly used phrase. So I'll be using that more often. And the stock is focused on traces. So open telemetry is also getting into metrics logs and other things, but we will be confined and focusing on traces. The stock isn't intended to push the migration. I'm not selling anything here. I'll just share what are the ways how you can mix and match the things and what are the upsides of migrating. And let's understand the components first before we get into the talk. So we know there are multiple components involved in tracing both in eager and open telemetry. So the instrumentation layer usually has API and SDK. So in eager, we use open tracing API and the SDK is eager client libraries. And the agent and collector is all eager. So eager also has an agent and collector and it also offers some native storage options within the collector. Like Cassandra elastic search and there is a GRPC based mechanism to plug the external databases as well. And eager also has a visualization layer, which we'll be discussing later today and how it helps you to visualize the traces even if you're using open telemetry. And in open telemetry, all it has is the instrumentation layer and the collector layer. So eager is all the way from instrumentation to storage and visualization, but open telemetry is on the instrumentation and collection stage. Okay, why you have to migrate. Now we have been hearing eager and been using eager for a while and eager has served really well for most of us and people love eager and open telemetry has been announced in 2019 cubicon and so exactly it's like four years in San Diego. And since then the project has been matured and it's expanding itself into different observability signals and adding new capabilities. So let's discuss on what are those capabilities and why does migration would make sense. So the first thing is eager client libraries have been announced and off life support earlier this year and eager community and the maintainers also are supportive and advocating towards open telemetry SDK. So if you're using eager already have to move from eager client libraries to the open telemetry SDK and will be briefly talking on how to do the instrumentation layer migration. And the second one is open telemetry is a new standard as we all know for instrumentation and collection of data. So it takes some industry best practices that were part of open tracing open census and all, and it also adds new capabilities that were not existing in eager instrumentation side. So and the collection layer is really rich it gives you options to configure different sources and destinations to ship your data. And, and the last thing is open telemetry does support auto instrumentation which means you don't have to do any code changes you can just run a sidecar and it just works. So levels of migration. So there are two levels of migration one is the instrumentation layer the other one is the collector layer. Before we get into the migrations and how it's done. Let's do a quick walkthrough to the in eager and open telemetry architecture. So the first slide is the eager architecture you can see on the left side. There is an application the eager client and the eager agent so the application is instrumented using eager that's and the spans are being pushed to eager agent and from there you can see there is eager collector and the database and the UI. So this is the complete ecosystem and the components that are involved in eager architecture and the spark jobs are something very optional if you need them you can run. Coming to the open telemetry architecture so here in the open telemetry architecture we do not see the UI layer or the storage layer as I said that's all about the instrumentation and data processing pipeline which is open telemetry collector itself. So you can see here open telemetry collector being run as an agent so. So there is so much of confusion between this agent and collector so the collect the agent is something if it is ran within the host or as a sidecar to the application then we say it as an agent but it's the same collector that's being. Running and collector access a centralized processing pipeline where directly or applications can send the spans to the collector or your agents can send the data to the collector. Okay let's start with the instrumentation layer so you can see in the instrumentation layer as I said there is this API and SDK so. So to understand what are the components involved in API you can see this image it has the tracer API context API and the meter API so the meter API is for metrics. And in the SDK you can see there is this propagator span processor and aggregator so these are the functionalities that the API and SDK offer you during the instrumentation. So as I said the migration can be done in instrumentation using two ways which is open telemetry shim and complete re instrumentation so we will be doing a demo and discussing more about the re instrumentation but there are libraries already available in the internet on how to do the open telemetry migration using the shim. There is a blog post and medium which uses the Java application as a demo to migrate from open the eager client libraries to the open telemetry using the shim. So the shim is all about it consists of set of classes that implement the open tracing while still using the open telemetry constructs so with a very minimal code change like the blog post that's been. Authored by Jurassic is published in the internet so it's already available and it's just hardly takes five minutes to migrate by just swapping out the dependencies and imports and the second one is it's all it's also helps you to do to get on to the open telemetry SDK in no time you don't have to do the complete re instrumentation. So if you have less bandwidth and if you want to use some client based sampling in the open telemetry or you want to use open telemetry SDK for some reason you can definitely start with shim. And this is the blog post that's been referencing to that's been mentioned here and the complete re instrumentation so why you have to do the complete re instrumentation so the complete re instrumentation offers a hotel as a package. So you get all the capabilities right from the scratch from the code of open telemetry the semantics and in future you can also expand your open telemetry instrumentation into metrics and locks. So this is just a starting point for you if you get started with open telemetry and it's also easily integrated with auto instrumentation application so if you if you want to do an auto instrumentation for few applications. And if you have other applications if you which you need more granular details so the auto instrumentation gives more often higher level or traces and spans with manual instrumentation you help you'll have more flexibility over what you want to capture and what you want to measure. So it easily gets integrated with auto instrumentation and the normal apps. So that's demo time let's just get started. So my demo is as a clone from open tracing tutorials so there is an open tracing tutorial in in GitHub authored by Yuri to understand how the instrumentation works so I'm just taking the same application. I'll be showing you what's the eager instrumentation like and how's the open telemetry instrumentation is like and it's a very simple app so we have a greeting client and we have a formatter and a publisher. So the greeting client basically sends a request to the formatter and the formatter basically formats the string and then greeting client publishes the string to the publisher just to do an STD out. So the demo is available in this GitHub repository so I just pushed the demo to the GitHub. Okay. I hope the demo works. Okay. So let's jump into the code before we run it and see the differences. So I have the eager application here and this is the eager client go. So no this is a client application which basically sends the request to the formatter and the publisher. So here you can see it's all the normal code that's nothing fancy it's a main. Here you have the main and then the interesting part is here in the tracing you have to see the init tracer so we all know that if you want to start with the tracing there is an init function which basically gives you the tracer and then you start using the tracer referencing it across your life cycle of the code. So here we have started instrumenting and we are saying okay start start span and then finish and here we're also using the baggage as an example how you can set the baggage to do the context propagation and pulling it. So if you do not know the feature baggage so it's basically it gives you an ability to attach metadata during the context propagation without a need to change the API. So if you have to send some metadata from your application a to all the way down after four requests you can just put it in the context and it just flows through the last application that you want to send it. So without any disturbance or changing the intermediate applications to capture it and forward it forward so it just gets attached to the context. So here we are calling the format string function and it's making an HTTP call to 8081 and we are injecting few data. So some data which is context and HTTP headers and this is HTTP headers carrier here and yeah we are doing a request and yeah then we are also calling the print hello which is a publisher service here is the publisher service. So yeah and the publisher the format or service is nothing but a server just running just waiting for the request to format the string and send it out. And if you see the publisher service the publisher service is all about just take the request and do an STD out. Okay so first we will run the eager all in one which is which basically runs the eager UI eager collector and in memory storage by default. Hope the font is visible. So the eager collector is up eager all in one and now let's let's run the eager application. So now I'm first running the publisher go run publisher dot go that's up. So so I'm sending a string as a greeting called hello to a cubic on from my client to the format and then publisher. So we can see now the format is running the publisher is running and the client is here. So first let's check the eager UI. So as I just just started the collector so now you can see. Okay there are no services or no data because we haven't yet pushed the data to eager. It's just the eager UI eager query spans itself. And now I'm just doing hello cubic on and the demo just works as this is authored by Jurassic. Like this is this is not not just a view. So it's from open tracing demo. And now let's check the traces here. So the traces are here and you can see it says hello and you can see the request has went from hello world to format or to publisher. Now let's just check the open telemetry code and check how it works. So I'll be a bit quick here. Yeah. The open this is the open telemetry instrumentation. So now you can see you have used the hotel trace baggage and all the inputs here. So I'm not using anything from eager or open telemetry and it's it's the same. You just need the tracer and then you can start the context and I feel a few areas that's even simplifies for example doing the context baggage and headers. It felt easy for me at least in go. And here we are doing an HTTP request to format or and here you have publisher. So let's now run open telemetry and check the spans. Okay. The format is up. And now let's do the same from the client but that's instrumented using hotel. Okay. So I'm just doing the same now. Hello cubic on. As it's saying. Did I change anything? No. Okay. I'll just give it another 30 seconds. Let's just check. Let's just check. Did the spans come out? Yeah. The spans aren't out from open telemetry yet. Okay. I think I got it. Yep. Now it just works. It says the comma. Yep. Now you can see the greeting client. And it's just a few seconds ago. It's been it has come from the open telemetry. And here we can go to the format or service and check the tags. So now here it says go client and it's instrumented using the library open telemetry.io. So these are the spans from open telemetry instrumented using open telemetry and they're the same. So this is what I wanted to show. It's a very small demo, but it includes all context propagation using the baggage doing an any trace and all like the basic stuff that we use during the instrumentation. And yeah, that's that's the demo I had just to show how easy it is like to run and use eager alongside open telemetry instrumentation. So now the eager collector is running and the recent feature in eager has supported ingesting the OT LP traces coming from your application. So the data is coming in the OT LP format as the exporter. And now we are pushing it to the eager. And another interesting fact that I messed to explain is that okay, if you see the tracer here you can I have written a code in a way that you can. Okay, you I have written a code in this way that now you can configure the eager endpoint or open telemetry endpoint. It just uses the any tracer if you want to use eager endpoint you want to send to the eager. It's it's this in it provider for eager. If you want to use open telemetry, you can use open telemetry. It's just logic to send traces that are being instrumented using open telemetry in both eager and open telemetry formats. So if you have some limitation on the back end that you have to send traces only in the eager format, you can use something like this and it just works. Okay, and the question comes as the context propagation like now you have 10s of applications that are already instrumented using eager and you are starting a new application from scratch for the development. The always the question is do I need to start with eager or do I need to start with open telemetry. So the recommendation is you should start with open telemetry and the context propagation just works because eager support eager supports the eager B3 and W3 C context propagation and open telemetry does support all three. But if you have different applications talking to a same application in different context propagation, the open telemetry SDK also supports enabling or using multiple context at the same time. So if application is instrumented using open telemetry, it can understand it can receive the request using the context propagation eager W3 C and B3 at a time. So you don't have to just stick with only one context propagation. So it just works and there is an article in medium written on explaining the same how you can do mix and match of context propagation between the applications. Okay, the impact. So now we have seen the migration and how you can do the migration. Now what's the impact after you migrate so obviously you have an improved tracer implementation as I said the baggage and other things just work and the switch to the open telemetry SDK. While using the open open tracing is also a possibility as using the shim just makes this easy for you. And there is also an improved performance I keep reading in the read me and in other places. So the open telemetry SDKs are more performant and access to open telemetry framework plugins. So we are there is each language has its own way of integrating into the SDKs. So it has feature rich framework plugins. And now let's understand migrating and the collector layer. So as I said we have two levels of migration one is the instrumentation layer that we have seen that's the code. And now we will see the migration into open telemetry without touching the code or without disturbing your applications. The usual architecture when you're deploying eager collector and on the left on the right you can see how you can add open telemetry collector into your existing architecture so you don't have to do any open telemetry code changes. In your applications you can just use open telemetry collector because open telemetry collector can receive the data from eager and different formats. So we'll be talking about how you can configure the collectors in the upcoming slides. But this slide is all about if you have an architecture has left eager collector your applications are sending traces from eager to the collector. And it's going to the storage storage back in all you have to do is put an open telemetry collector in between the applications and eager collector. So the question now comes is why you need an open telemetry collector when you already have an eager collector. So the open telemetry collector eases the data collection pipeline. So if you have metrics traces logs all coming into from your applications you can just run open telemetry collector and it can do it can process all this you can configure the pipelines. For example you can also use span metrics processor which is which basically generates the metrics out of your spans and the processor is available in the collector. And here in the bottom I'm just putting a storage back in as prom scale. So it just works with open telemetry and eager. And let's understand the differences between the eager and open telemetry collector. So the eager collector has more flexible sampling whereas open telemetry collector has sampling processors to but the eager collector has more in terms of for example the adaptive sampling has been released in eager one to six so it's available in collector. I'm not sure it's there in the open telemetry but yeah you have wide options for you for the sampling in eager collector and in eager collector you have entry storage options where you can use Cassandra elastic search or use the GRPC based storage plugin for the back end in open telemetry collector as I said you can use receivers exporters and process which helps you to build the data pipelines coming from different sources to the different destinations. So this is the hotel collector configuration you can see on the left it has receivers exporters and services. So in receivers you basically configure what are your sources where will you receive the data from and in the exporters you will basically configure where you want to forward the data to in services you will basically map this receivers and exporters to build a pipeline. So on the right hand side you can see my open telemetry collector is receiving data from applications which are instrumented using eager and on the middle you can see the applications being instrumented using open telemetry and they also have a Prometheus receiver which can scrape the targets to get the metrics so you can just configure the receivers and it just works and even during the exporters you can parallely stream the data to your SAS back ends or in in-house storage systems anywhere so you can also duplicate the data between systems and if you want to move away from one storage engine to another there is zero lock in all you have to do is change some configuration in the open telemetry collector and it just works. So this is the anatomy of the open telemetry collector so you can see if you see the collector on the left side it's all receivers it's otlp receivers can eager receivers so if you have any legacy applications or any other applications which are sending you some data in different formats otl collectors should definitely have the receivers for them. I think there are already like 20 plus receivers in the open telemetry collector contrary we can just use them and these are the exporters and you also have processes where you can translate the data do some kind of sampling filtering and everything and the exporters basically map this data or send the data to the back ends and the impact so if you use the open telemetry collector as I said this makes the transition much easier and it's easy to start it's all about just running the open telemetry collector and it's it gives you the ability to build the pipelines on how you want to send the data and where you want to send the data and all yep and in now you have seen in the in the second migration which is okay you can add open telemetry collector and with your eager collector so now we will also see how why you have to use otl in the eager so you can receive data from multiple sources and process the data using wide data processing tools and you can also export it to the multiple back ends and it also supports metrics traces and other things so here you can see it's the same but here we are adding the open open telemetry collector just in front of eager collector so that it just works it just sends the data to the eager collector whereas in the past we have seen we are just replacing the eager collector with open telemetry collector so it just works so if you have any storage back end that's compatible with open telemetry collector you just put the exporter saying that hey I want to send the data to Cassandra or prom scale you just configure the exporter and it just sends the data here if you are using eager in memory or badger storage you can still use the eager collector but all you have to do is configure eager exporter and it forwards the data coming from your applications to open telemetry collector to the eager collector and the impact is you can get best in both the world so you can if you have any sampling mechanisms that needs to be done you can do a sampling in open telemetry collector and you can do a sampling in eager collector too but it gets complex you just need to take some conscious decisions on how you want to do but you can leverage the in memory and badger storages and eager also offers a g rpc based remote write plug-in so it just plugs into the databases that offer that integration and querying and visualizing the traces so if you move to the open telemetry collector there is no path to how you connect to your traces how you visualize the traces and where do you store the traces so the open telemetry project is all about collecting the data instrumenting the data and collecting the data whereas with eager you can use the eager ui2 to visualize the data so if you are moving from eager to open telemetry you should make sure how does your eager ui2 integrate to for querying the traces and these are the otterland eager boundaries so on the left side you can see its otter is all about the api sdk and the collector whereas eager is the query and matured native storage back ends so in future the eager will be evolving into a platform for traces whereas open telemetry will be more like instrumentation and a collection pipeline for all the observability data and in conclusion all I want to say is start with otter in some capacity it could be the instrumentation layer using the shim or it could be a complete re-instrumentation or you can just introduce the otter collector and start using the span metric processor other features from the open telemetry collector and that's all I have thank you thanks for joining my talk before I end my talk I just want to just show you the eager ui has recently launched a monitoring tab which can give you which can visualize use this red matrix coming from your spans so now you can with open telemetry collector you can use the span metric processor it generates the matrix and you can send it to a Prometheus back end and the eager ui directly queries the data from your Prometheus and visualizes this so this is not built it's all out of the box built by eager ui all you have to do is run an open telemetry collector and configure this span metric processor so it's really cool I just wanted to show it if you're using eager you can leverage this feature yep and let's get back to questions I'll be around yeah if you have questions just you can join me