 Okay, good evening, everyone. Welcome to the last session of the day before we go on to the CNCF event party. I am Alulita Sharma, and I would like to introduce fellow members of the GC as well as maintainers on the Open Telemetry Project. Ted, would you like to go first? Sure, my name's Ted Young. I work at LightSep, but I'm an Open Telemetry Governance Committee Mediver and one of the co-founders. My name is Morgan McLean. I'm also one of the co-founders of Open Telemetry. Also on the Governance Committee, I'm a director of PM at Splunk. My name is Dan Dila. I work at Dynatrace. I am a maintainer of the Open Telemetry JS client and on the Governance Committee. Cool, and again, just to complete my introduction, Alulita Sharma, a member of the Open Telemetry Governance Committee, have been contributing to the project for multiple years now and I also lead series Observability at Apple. So, super happy to be here today. All right, so with that said, again, I wanted to kind of run through what we have accomplished on the project in the last year. We gave an update at Valencia earlier this year in the spring. And as many of you who may have attended that session or not, we in last year started working on logging, which was a big pillar for the project as well as going beyond traces and metric support. We also last year kicked off the Prometheus Open Metrics Hotel Interop Group where we actually worked through making sure that the OTLP protocol was fully compatible with Open Metrics as a metric standard as well as Prometheus, which also uses Open Metrics as the baseline and we've made good progress there. Tracing was also GAID last year in October. And again, that was functionality that we had inherited from Open Tracing and Open Censors, but then we also actually did a fundamental amount of work on tracing itself in order to not only harden the collector but also complete the implementation in the APIs and SDKs. We also started working on telemetry schemas, specifically the Open Telemetry Specification. As many of you who may be using Open Telemetry or have looked at the project, the specification again provides a requirement spec for the project to implement both on the collector agent as well as the 11 language APIs and SDKs that exist on the project for instrumentation of your applications. And then going on from there, we also ensured stability of the logs data model, which was also announced at the Valencia KubeCon conference earlier this spring. Going forward, fast forward from there, what we have actually accomplished on the project in the next six months, which I wanted to step through, very excited to see client instrumentation kick off because that's something that is fundamentally very important for end users and users and developers who are actually instrumenting with APIs and SDKs into their applications. So that SIG was started and there's been work ongoing in that area. This also includes the real user monitoring integration that was and the definition and the semantic conventions that has been done on the SIG. The end user working group was established, which actually enables end users, companies that are adopting Open Telemetry for collection to actually participate in the project, provide feedback, provide requirements and be able to work together with the maintainers and contributors to the project to implement the spec. And last but not least, big accomplishment, Metrix GA, which the project has completed and has delivered. We also are very excited in the last month to have kick started the profiling work group, which was actually one of the suggestions that started kind of at the Valencia discussions in the project meeting that we had there and then has really culminated in an OTEP that was discussed in the tag observability and then submitted as a profiling Open Telemetry enhancement proposal in the last month. So needless to say, the Open Telemetry demo, which also many of the contributors have been working on is also now available. And with that said, I also wanted to call out, there's more instrumentation down the line, better documentation work that's happening on the project, lots of activity on C++, Erlang and Swift and some of the other improvements that we will step through as we talk through some of the slides that are coming up. All right, so with that said, Daniel, over to you. Yeah, so I'm gonna give a short community update for those that were at the community meeting on Tuesday. Some of this will look familiar, but we really wanted to highlight how much of a collaborative effort Open Telemetry is and has been and continues to be even more so. For those that don't know, we are the second most active CNCF project behind Kubernetes, which you may have heard of. And this is what our contribution graph looks like for the last year. Yes, graphs that go up are always good on the slide, but in this case, it really shows that the two numbers are not only the number of active contributors, but the number of active contributing companies. So it's an extremely collaborative environment and more and more companies and individual contributors and particularly end users have really been stepping up in the last year, which has been awesome and really great to see. We have just pulled out a list here of some of the larger contributing companies. What we really wanted to highlight here though was, you know, a handful of these, almost half really, are end users and not really vendors. So the project started as sort of a very vendor heavy project, but we always had a vision for it to be built and maintained by the people who use it. And it's been really nice to see that actually begin to start happening. One other thing that I did also want to mention, we did just finish our GC elections for the last year. This year we had 513 eligible voters, 219 of which actually voted. So for anybody here that participated in the election, thank you for your participation. Without people to tell us what they want, then the project really wouldn't go anywhere. And it was nice to see that numbers were very healthy. I was re-elected as well as Alelita and Morgan re-elected this year, but also welcoming Trask Stelnacher from Microsoft to the GC. So we were very excited about that. Sorry, Ted. Yeah, so one thing I want to dig into that I think is especially of interest to the community is the new open telemetry demo project. So just to walk through really quick what this is and what it does, it is your classic kind of distributed microservice application. It's a web store that sells telescopes because that's what we like around open telemetry. And it's made up of a variety of different services in a variety of different languages all instrumented with open telemetry. So if you click to the next slide, it also has feature flagging for being able to enable and disable different kinds of problems within the demo application. So what you can do is very easily stand this up, especially using like Kubernetes operator or Docker if you're running it locally and hook it up to your backend of choice or a local Yeager instance, for example, and enable some of these problems and then go do your own investigation. So it's a great way to actually see open telemetry in action. Go to the next slide. It's also a great place to look if you're thinking about standing up open telemetry yourself. The source code is available for every service so you can go in and see how we've actually added instrumentation to that service. So if you're getting started with open telemetry and you're just looking for a nice easy, like copy paste example for how to get started, the demo project is actually a great place to find that code. And if you take this QR code, that will take you to the documentation for all of that. We also have a nice blog post that kind of like gets into more detail about this whole thing. So if you're interested in the demo project, this is kind of a good starting point. And I just wanted to call out on the demo project. Again, as you can see, there's a very interesting ontology that is built across the dependencies of the different components as well as what you instrument with. So again, go check it out. It is really interesting and would love to see more folks contribute to add your own use cases to it. That's it, Morgan. Yeah, so we're gonna talk about the roadmap. So Ali did a great introduction of all of the progress that we've made since Open Telemetry was founded and certainly in the last year on the project. So things like hotel metrics going GA and other really critical components. But we're into interesting time now, like metrics and distributed tracing were the original promise of open telemetry. Those were the two data types we'd originally called out. Logs of course were one that we added or we started working on relatively recently. But still it leaves us as a community with an opportunity here to decide what we want to work on next. And so we've taken some time here at KubeCon to meet with community members. We have about I think 75 people show up at the community meeting to discuss where we want to go from here as a project. And we'll see if this works. Oh, yep. Before we actually get into that though, I wanna describe exactly the state of things today. So distributed tracing has been generally available for some time with an Open Telemetry. And if you look at all the languages we have, obviously we've added languages over time as well, which makes it a little complex. But tracing is stable on almost all of our languages. And those that it's not stable on is in a beta state and is somewhat usable. Metrics similarly in May in KubeCon Valencia, we announced that the metrics specification was now stable. And here at KubeCon Detroit, we're able to show off the various languages in which metrics functionality is now fully baked. And that's considered GA stable. There are gonna be no breaking changes there going forward unless there's another major release of Open Telemetry years into the future. So we've achieved a lot of success here on metrics, blogs of course, being the column on the right. You see not a whole lot of blue there. But that's gonna be the next big area of focus for the project is getting logs to the same state as tracing in metrics. And of course we'll be filling in metrics as well for more languages over time. What this means in practical terms, if you're an end user or someone who's interested in adopting Open Telemetry at your organization, is that you can use Open Telemetry today to capture spans, to capture system metrics, application metrics and metadata from your services and from your infrastructure, as well as a few other sources. You can pre-process this data, typically with the Open Telemetry collector. There's a lot of flexibility and capabilities there. And then you can send it to any destination you want for final processing, storage or analytics. This is the original promise and intent of Open Telemetry and it's something that we've now generally achieved at this point, which is, we're celebrating as a community. We see certainly across the industry, like hundreds if not thousands of large organizations that are already adopted or in the midst of adopting Open Telemetry as a result, because it gives them these capabilities and allows them to send their data wherever they want in a nicely structured, correlatable format. There's a few things that we're working on now. Unfortunately this is animated one by one. So I think Alita already mentioned the end user working groups so I'm not gonna touch too much on that. But the critical items here on the roadmap are that we're working, we've expanded our web maintainer team, which is working on documentation and the Open Telemetry website, which makes it easier to adopt, easier to use. We're also gonna be focusing on improving the contributor experience going forward. In many ways though, given the graph that Daniel showed with the number of people working on the project, this is also in some sense taken care of itself because we have more maintainers and more contributors than we've had in the past, which is excellent. For the roadmap though, the very meaty items that if you're an end user, the ones you really, really care about, for those we'll start with op-amps. So this is an Open Telemetry control plane that's being specified and developed right now. Today you can configure Open Telemetry components relatively easily. The collector, for example, has a large set of YAML markup that you can modify and implement to go configure processing on the collector and configure what it receives data from and where it sends that data to. But you have to go and deploy and distribute that YAML file alongside the collector on every host that it's running on. Similarly, there's configuration for other Open Telemetry components like language agents and language libraries. With op-amp, what we'll be doing is allowing those components optionally to be controlled via APIs. So that in theory and eventually practice, you could have a central server or service somewhere that is actually controlling and configuring your Open Telemetry components live. So you can make configuration changes on the fly and you can more easily manage the configuration of your thousands or tens or hundreds of thousands of Open Telemetry components that you have running live in production or non-broad environments. It's early days for op-amp but it's very, very exciting and I know there's a lot of community enthusiasm around it. Another item of the roadmap is client instrumentation. So yes, if you looked closely in the past, you would have noticed Open Telemetry.js has had support for capturing some level of instrumentation from front end websites. This is now being formalized and extended so that we can extend people's visibility not just to their back end services but extend that all the way out to their client applications running on mobile phones, running on desktops, running on front end websites that anyone's accessing. So you can have true end-to-end observability across them. For example, you can have a distributed trace that starts from a client and actually shows you the amount of client latency or client wait time that was incurred due to local processing and also you can use it to determine the amount of internet latency that caused slowness. That's one example of what you can do with this but I think there's a lot of power and potential coming out of this. The next and probably the most sort of topical for here is profiling. We're adding profiles as a new native data type within open telemetry. This is a topic that actually came up at KubeCon Valencia, the community meeting. It hadn't really been considered previously and is now gonna be a major part of open telemetry. We have a large SIG working on defining what profiles are, how they can be captured performantly, how they can be integrated with the other data types and we're gonna implement this as a new signal type within open telemetry that'll then be implemented across the different languages that compose open telemetry. It's also kind of exciting because open telemetry, I think in the early days, there were a lot of familiar faces, the same sort of people working on everything. Profiling is one place where it's brought an influx of new users and contributors to the community which is also really exciting. And it's people who literally showed up to the community meeting in Valencia and various others who got engaged and involved which is great. We'll also be working on fully implementing logs within open telemetry. We've talked about this one a lot at previous conferences. I'm not gonna go too much detail on it. Just there's, the logging support will exist for existing log sources. So files on disk that get tailed as well as an open telemetry native logging type that'll be extremely performant. It's a binary strongly typed format for capturing logs so you don't have to parse logs multiple times with agents and backend processors. Very exciting, but of course for existing apps you would use the files on disk. Ted already mentioned the demo application so I'm gonna skip right past that. There's also some proposals around file-based configuration. This ties into op-amp. So using YAML files to configure all open telemetry components, not just the collector. Today, most of them are configured through other means. We wanna make that consistent for people who choose to have that. I'm gonna skip through these because I already talked about them but effectively client instrumentation. Again, it allows you to do things like see traces from your client apps all the way through to your backend services. Today that's somewhat segregated but in the future with this you can see those traces or other types of telemetry spanning all the way from user interactions all the way through services to database calls in the backend. I already talked about profiling as well. Gotta head ahead of my slides. Going forward, what this means. So I talked about our current status today in practical terms. In practical terms, once we have completed the items that are now on the roadmap it means that you're gonna be capturing logs and profiles in addition to the types of data you're already capturing and you're gonna be capturing those from client applications in addition to your existing services and infrastructure. And you can try all of that out inside of our wonderful pre-built demo application that Ted described. Alina, do you wanna take this? So I want to kind of go through some of the areas that have been discussed in the last few months as well as in the project discussions that we've had earlier this week at KubeCon where these are some of the proposals that have been brought up by the larger community by end users as well as other developers. And I just wanted to kind of run through some of them so that, you know, very interested in kind of getting feedback from all of you, you know, as we work through as some of these and consider them as proposals. First of all, CI-CD systems where we build and deploy our applications are becoming increasingly integrated into what is observable and capturing telemetry from CI-CD systems and defining semantic conventions for those is something that's super important as you deploy production systems and you want to ensure that what you're pushing to the client or to the server or services for that matter are, you know, fully observable and tested out, right? So again, we will, you know, this is a proposal that was made by some of the, you know, some parts of the community. So it's under consideration. Another area that, you know, is also being discussed is how do we actually enable more Kubernetes metadata to be visible and observable and increasing the visibility into the infrastructure, you know, Kubernetes-based infrastructure that most of us use for our applications and services. Another proposal is the, and again, this is a contentious one, but, you know, again, also a project in itself having a standard query format for being able to query telemetry data. Because here you are, you're going to collect petabytes and petabytes of observable, you know, telemetry data coming in from different system components, both on the client and the server side, and how do you actually query? So you might have used, you know, this kind of data, is it useful to have a standard query format? And for many of you, you might have used some query languages, SQL, how many of you use SQL? Many of you, PromQL, some of you, what about elastic query language? Right? So there's so many different query languages in the observability space at this point that it also, you know, is something that connects very much with the collection process and really making it a much richer experience. But that said, again, I think this is something under discussion that we will take to the Tag Observability workgroup to work through there and then figure out, you know, how we can work through a standardized query format proposal. Other discussions that have come up are semantic conventions. Again, establishing semantic conventions for cloud, spend, again, how many of you run on public cloud infrastructure or your own clouds? Hopefully everyone. But again, usage and spend is a big deal for most organizations and we'd like to better understand, right, how that infrastructure actually is being used by different teams across the orgs to use this infrastructure and how does that then measure into what you plan for in terms of provisioning the year and the next year. So there's a lot of intersection there and just want to, I was curious, please, you know, raise your hands if you're interested in cloud spend. Hopefully, many of you are. Emissions is another area from a sustainability standpoint that, you know, we hope to cross collaborate with some other discussions on the Tag Sustainability and Created, but also other projects in the observability space to see if there's common semantic conventions again that can be established. And last but not least, security use cases with SIM semantic conventions because that's another huge area that intersects with the kind of observable data that you want to make available for security use cases, right? So again, just wanted to call that out that these are areas that you can actually contribute in. So if you have any more experts or just working in that space, I invite you all to consider joining in into our discussions on the project as well as helping us establish those semantic conventions that can then be used by everybody, right? So again, it's an easy process to get involved. We have weekly SIG meetings and SIG calls and they're all on the Google Calendar for CNCF and would really love to see. Last but not least, I think, back to you, Morgan. We wanted to talk a bit, just to close the set about our long-term vision, right? Observability and open telemetry being a part of observability. I think people have heard pitches in things from open source groups, from vendors, from others, talking about how critical it is. But I think in practice today, most people use their observability tools as a last resort. It's the thing you run to when the alarms are going off, which when everything's gone wrong and it tells you what went wrong and hopefully how to fix it. That's how we interact with these. Certainly, observability, I think, for most people there's a perception it grew out of monitoring tools and monitoring tools are typically used in that kind of scenario. Our vision with open telemetry and our expectation is that over time observability tools, which of course are enriched and powered by the data that comes out of open telemetry, are going to be used much more frequently. I'm sure there's people here who have been in situations in their career where they're working on a sufficiently large set of services. Of course, technologies like cloud infrastructure and Kubernetes make it very easy for us to spin up all these services and make them communicate. I'm sure there's people who have been in situations where they've had to go and describe how these different services interact because you're in a sufficiently large organization where very few engineers or technical staff are able to draw the full map or a sufficiently large map of these services. This is one example of a place where I think observability tools will be used as a developer if you can rely on having a service map, like it's just one example, that's drawn, that's totally reliable from your observability tooling and indeed you can investigate that service map to see how certain interactions are processed. It means that when you're told to go extend your services because maybe you work at an e-commerce company and you've been told go implement the sales of bundled products, for example, which maybe wasn't done before your e-commerce stack, but that you now need to go touch like 10, 15 different services to do it, these tooling is going to help you during the design phase and implementation phase of doing that in a way that today you'd have to go liais of a whole bunch of people to find out how things work. This is one example, one slice, I think of how observability tools will be used going forward where they will extend from just being tooling and things that you use when all hope is lost and going on as an engineer throughout your day-to-day work cycle as you do with accessing source control and various other things or at least within the same order of magnitude. There are other scenarios like testing and others that I think also will benefit very strongly from open telemetry and observability tooling as it becomes more and more adopted across the industry and certainly within the open telemetry community this is something we're really, really excited to have a part in because it's going to make more jobs and everything way, way better and easier. So it's just a very exciting time for the community not just because of the growth that Daniel showed earlier but also I think of the potential that's coming in with open telemetry and the tooling that will be based off of it that's analyzing that data. Can you click next slide? Perfect. To formally wrap this up next steps for people if you want to use or using open telemetry there's never been a better time to adopt it like chasing is super robust it's being used across the industry metrics now is stable and really we're seeing it its adoption really accelerate across the industry which is exciting open telemetry gives you that flexibility of sending your data and processing your data anywhere you want and indeed you can send your production telemetry to any number of destinations at the same time and it also gives you that really nice data out of your applications and infrastructure in a way that you may not have had it previously. Our basic logging functionality is also fairly robust there are companies already using it in fairly high scale production like earlier this year I was at a talk by I think Lockheed Martin about how they're using it across other Kubernetes infrastructure like you can go use it, it works it's marked unstable just because it's possible the configs or things will change in the future and we're adding more functionality and of course more advanced logging scenarios like the type I described earlier profiling and client instrumentation are all coming to open telemetry soon and it means you can adopt them very very easily if you're already using open telemetry. Secondly if you want to contribute to open telemetry if there's a particular part either existing functionality or things in the roadmap that we've discussed today that you're interested in please like join us like join our channels on CNCF Slack or join the weekly SIG calls open telemetry it's not a business we have this roadmap this proposed roadmap we have up here but I and Ted and Daniel and Alita and other people can't go and tell everyone what they need to work on developers and the contributors vote with their feet and so if there's a thing here that you want to see enhanced or put more firepower behind please like come in join us you are always welcome in this community you're always welcome to help initiatives like profiling have started and taken off because tens of developers have come in all once and I want to add this and I'm going to work on the design implementation of it so if you have any interest in that please join us we'd love to have you and as I mentioned the best way to get involved is by joining either weekly SIG call or the Slack channel for that particular area in which you are interested in and with that we're done we'd love to take questions from anyone I see a couple open telemetry maintainers as well if you guys if you're just taking questions please come up here if you're a no-tell maintainer otherwise if the audience has any questions we and the maintainers are happy to answer them yes yeah you mentioned sort of query languages and things and you were looking at or exploring sort of candidates query languages have you sort of considered graphql as an option because the ability to sort of stitch schemas together and sort of map that in with other data we mentioned that one mostly because there's clearly an appetite for it like especially in the last few months people have been asking many different people have come and asking for this to be clear this project may not in fact likely won't end up being part of open telemetry we want to keep hotel very bounded on data extraction and data collection but I think as Alita mentioned the CNCF tag might pick it up in fact I mean we are actually going to start a series of discussions on exactly kind of working through the nuances of what a proposal would look like and also yes we have looked at the graphql model and several other query languages obviously graphql is very popular there are other query languages also on the client side that should be looked at but again let's go through that process and again super happy if you can join in for these discussions just to add some color onto that one of the things when you look at it you have some like very general purpose kind of querying things like SQL and then you have stuff that was like built to purpose in the observability space but it has been built to like a specific subset of the kind of grouping of signals that we're doing and there hasn't yet been as far as I can tell like an open query language that was built specifically to target the data set that we're looking at so that's kind of like one of the big questions is how can we use like pre-existing work but then adapt it to something that's specific enough to the kind of data that we're looking at that it's not like overly generalized the way SQL might be for example Hi you talked about adding cost and emissions in the roadmap right? I yesterday came across Phenops foundation in the showcase area they seem also to be in the Linux foundation hierarchy so how will the boundaries be? Can you give some more color on that? Typically how that works is that if there are collaborative projects and again there can be overlapping you know feature sets and functionality that is being looked at or discussed or worked on in different CNCF projects so typically you know that does as we work through the semantic conventions as we work through the proposal itself of what you know a baseline looks like we typically would try to go and intersect with those projects also and see if we can leverage some work that they have done so it's a collaborative effort Any other questions? Data extraction and data collection certainly you know part of the story you're trying to achieve that but data storage is under you know peace of the puzzle right? Any roadmap and any thoughts around data storage you know organizations like us if you want to implement the next question people like it next question comes is how do we implement across 300 different things that's how the scale matters We've purposely avoided doing data storage in open telemetry like to be clear there's many reasons for it one is that open telemetry has I think Daniel and Ali to show there's a lot of end users contributing but there's also a lot of vendors in the space that have been founded to just data collection it means they're very very collaborative and very cooperative I think there's a concern that if the project started getting to data storage and analytics and things like that that would perhaps change the level of cooperation there but secondly legitimately there's a number of CNCF projects that already focus on this Prometheus is a great metrics back end to processes metrics Jaeger has a really great processing distributed traces and it's fantastic probably being the most popular and so I don't know if there's a whole lot of thought in open telemetry or the whole lot of thinking that we can somehow improve on data storage and analytics in the open source where other projects like these or like skywalking or others have not already done but I think just to your point storage is very important obviously in the observability pipeline and you know a unified experience is also equally important for developers who are actually building such complex pipelines so interoperability from the collection side in terms of being able to connect with a diversity of data sinks whether that's Prometheus metrics or Jaeger for tracing or other logging databases again there are a multitude of exporters that are already available in the collector in the open telemetry sector and you can use those but if there is a data sink that is missing in the open telemetry integrations then please feel free to work with us make a proposal and typically they get added very quickly so there is a huge diversity of data sources as well as data sinks that are actually integrated into open telemetry and again if you see anything missing just please propose yeah just to slightly rephrase it we see open telemetry as a standardization project and it seems very doable to standardize describing like developing a language for describing what our systems are doing and it seems reasonable to also standardize like a processing pipeline for like egressing that data analyzing that data that really seems like a very green field area especially now when we're getting into like you have unified integrated data people bringing in all kinds of big data statistical processing stuff that wasn't available in the future so it really feels like that's not a place you want to standardize right now instead you want to be very flexible working with all the people trying to develop new stuff so that's like there's vendors involved it's like not feasible to standardize on that you want to say something while we're waiting for him I would also add that projects like Yeager and Prometheus have been working much more closely because the telemetry data between your traces metrics and logs are now so much more contextually related in a way that they never were in the past it allows those other projects to collaborate in a way that they maybe need to do more so as those projects continue to improve maybe the need for a single data storage for all three traces metrics logs profiles whatever the need for that is lessened because the existing tools are more empowered on a specification level it's all true we don't want to specify a data format we don't want to tell people how to do that on the other hand there are some efforts like establishing an OTLP data format for culinary storage and that one can be used by the collector to do a collector exporter that then writes OTLP data to disk which then can be read by other systems so it's not a specification but it is an effort from individuals like Morgan mentioned the solutions of the individuals actually define the roadmap actually define what is the features that have been developed and there have been people in the community that has written exporters to write things to disk so you can store you can read but it's not part of any specification another question over here is it difficult to only use a portion of the open telemetry or yeah the open telemetry library using the language specific prometheus libraries for metrics I've already got a mostly functioning log set up and I'm just interested in the tracing portion is that relatively easy to set up or are they super paid you need I mean that's effectively how everyone up until earlier this year was using open telemetry because only traces of this table yeah no it's super straight forward it works fine short answer yes it's completely configuration driven so you can actually set it up specifically for tracing yeah one single yeah and to add slightly more flavor there's we've also done a lot of work to separate out the API that you instrument with with the client or implementation that you would plug into that so it's also feasible to say if you do end up with instrumentation that say like open telemetry metrics instrumentation it's fairly straightforward to create a bridge from that instrumentation say into like a prometheus client that you're already running there so we really care quite a bit about that kind of flexibility all right and I think we got the signal that we're at time so thank you very much for attending everyone please get involved in the community if you're very interested in it